Introduction

My idea is to create an interactive video system where one can manipulate and trigger videos live using MIDI controllers. MIDI controllers I will be using include an electronic drum kit, a sampling pad called the SPD-SX (has 9 individual surfaces that when hit, send a MIDI signal), and possibly a two-octave keyboard. This system will be set up so one can create music and visual performances improvising or with a rehearsed set. Parameters of videos I will be manipulating include various effects and filters. Various past experiences have led me to create this project such as being a drummer, learning music production skills, and learning how to use tools in VFX software to sync visuals to music. Performances like that of Michael Schack’s, TheSushiDragon’s, AFISHAL’s, and Mr Bill’s inspired bring music visualization to live settings. Once I started using various programs like Ableton Live, Max for Live, and OBS that allow this to be possible, I was confident that I can put together my own live electronic performances and demonstrations with visual synchronization. For the future, I plan to perform live using this interactive video and music system as a physically live performance and through live streaming in more controlled environments. Most live music performances that include visuals may have video loops playing in the background that are not synced to anything, synced to pre-made music, or synced to just the beat of the song. My idea expands upon this concept by making it so one can interact with video effects while performing their instruments, while also being able to trigger videos and images. In addition, these video effects become layered if more than one is triggered at a time so performances that improvised create visually compelling and unique performances. My audience includes anyone who is interested in music and video technology and the practical ways to apply this technology in live settings. People who like electronic music will find my performances and demonstrations exciting as well because of how similar and unique it compared to existing electronic music acts.

Context

I am expanding upon current technology because my interactive video system is more customizable than what is currently done by modern live music performances using visual systems of their own. The person who has come closest to my system is AFISHAL because he can trigger videos and parts of videos in real-time with his MIDI drum pads, but cannot individually control video effects that layer on top of each other. Furthermore, my video system can be synchronized and triggered with any parameter of any audio effect and sound in Ableton Live allowing for near-complete customization of almost every aspect of the synchronized audio and video all being possible in real-time using MIDI instruments.

Technical Strategy

I will use software programs including Ableton Live 10, Max for Live, Zwobot, OBS, and various audio drivers. Ableton Live 10 is a digital audio workstation and the main engine behind this project. Inside Ableton Live 10, I will be using Max for Live devices to control videos, change various parameters, and make more customizable sound design. Open Broadcast Software (OBS) allows me to further customize my visuals if necessary, but its primary application is its feature to capture what is being displayed on my PC and project it to any display monitor, TV, green screen, and projector. OBS also allows me to live stream my live performances as it is one of the most common and powerful programs used for live streaming. It also has chroma-keying and audio routing capabilities allowing me to use a green screen in which to display my visuals; and to output audio from Ableton Live 10, electronic instruments, and microphones. I can already produce most kinds of music and play the drums and various other percussion instruments. I also have some experience creating visuals for music to so these skills will be applicable to my project. I will need to learn how to effectively put all of my skills together as this project uses many different kinds of technology and requires me to physically rehearse how to effectively use everything once all of the MIDI controllers and visuals are set up correctly. I also need to find the best possible setup configuration that requires the least amount of wires and gear.

Aesthetic Strategy

The three main aspects of my project creating my project’s style are the genres of my music I will be performing, how much my performances and demonstrations will be improvised, and what visuals I use. I will be playing genres such as neuro bass, experimental, hip hop, metal, and more. One of the purposes of my interactive video system is that it allows for one to create new visuals live with little to no planning so I will be improvising some aspects of my demonstrations and playing prepared music and visuals at different times. As of now, I am going to use free VJ loops that have many effects, which visually looks like colorful moving and morphing geometry as is common with live electronic music performance. I also plan to use some existing videos to further enhance my demonstrations.