TheWinningPitch-1200x800.jpg

Hack the Hits: TouchTone

Grand Prize Winner 2016

Overview 

For the first ever Red Bull Hack the Hits, five student teams were hand-picked to compete in a 24 hour hackathon aimed at testing the boundaries of engineering, design and music technology. The brief was simple: "Design a musical instrument that introduces a novel way to make sounds." Our team developed TouchTone, a playful and easy to use instrument. Grand Prize Winner, April 2017 

My teammates and I got excited about this hackathon from the moment we found out about it. This competition was different than the usual hackathons I had done in the past. We were only given 24 hours to deliver an idea and had to work within the constraints of a specific theme. In addition, we were the underdogs of the event being that we were the only out of state team selected to compete. We were up against four strong Bay Area teams, representing students from Stanford and UC Berkeley.

Regardless, I felt confident in our abilities to perform well given our diverse backgrounds and our shared love for music. Teaming up with two computer science students from the University of Pittsburgh, Laurence Putterman and Ritwik Gupta, we produced a stellar instrument called TouchTone out of an Arduino and Leap Motion. The panel of judges and live audience were impressed by our delightful creation.

Snapshot of our team in action as we put together the frame for our instrument. Left to Right: Ritwik Gupta, Tiffany Jiang, Laurence Putterman.

"We're all musicians but have never dabbled in the field of music technology. We thought this would be a good opportunity to make an impact in a world that has been unknown to us our whole lives." — Ritwik G.

 

Judging Criteria

The goal for Hack the Hits was to create an expressive, playable instrument that could be used in a live or studio setting.

None of my teammates had experience working with hardware prior to the competition. The three of us were used to making web apps or mobile apps for traditional hackathons. We had never made anything physical. This competition presented the opportunity for us all to do something brand new. The judging criteria was based on musicality, originality and usability of the instrument. 

  • How well could an artist incorporate the sounds from your hack into a performance or studio session?

  • How well does your hack stand out next to all the other tech out there?

  • Can anyone pick up your instrument and start playing quickly?   

Developing TouchTone

In just 24 hours, we created an instrument out of potentiometers, an Arduino and a Leap Motion. We intentionally designed TouchTone to look and play like a string instrument. Anyone who has played a real guitar or Guitar Hero would know how to pick it up to make sounds. The thinpots send their outputs to the Arduino, which relays the information over a serial connection. The custom Max for Live patch receives this data, separates the data according to the sensor, and transforms those values into MIDI.

 

Crafting the Design

We all took on the technical challenges together although once things got going, we divided up the work and I began to really hone in on the design of the instrument itself. As the product designer on the team, I got to take on the responsibility of building out the frame of the instrument. I constructed the neck and body out of sheets of cardboard. Considering our audience, we wanted TouchTone to look playful and almost childish! With this in mind, we chose to use primary colors and covered the entire instrument in red glitter tape just to give it an extra touch. We wanted people to pick this up and feel comfortable with it as if it were like picking up a toy.

A key design decision was choosing to have the body be semitransparent. People like to see the technology behind how something works. What is it that makes the sound? How is this all connected?

During the demo, many users peeked inside to check out the Arduino and figure out for themselves how we were able to build the instrument. The overall design was eye-catching and could be seen from across the room easily. You could tell people wanted to hold it in their hands and experiment. Everyone loved it! The red tape and plastic film definitely made it a huge success.

Functional and Unique

Our team put together one of the only projects that was actually functional by the end of the competition. Unfortunately, a couple other teams ran into technical difficulties that they were not able to get around. By the demo time, many of the end results did not work the way they had imagined. Our idea was also different than most because we had successfully crafted the entire instrument from scratch whereas another team, for example, hacked theirs together using parts of an existing electric guitar. 

We surprised the judges by doing more than what was asked of us for the competition. Our instrument could control the lighting and visuals in a space in addition to making sound. 

Each sensor of the potentiometers output to its own MIDI channel. Users can freely manipulate the sounds that are played aloud by choosing different instruments. Hand gestures done over the Leap Motion modulates sound effects and visual animations.

Creating Additional Features

While Laurence and Ritwik went after the technical issues in the middle of the night, I took advantage of the great mentorship we had available to us at the hackathon. There were a bunch of workshops throughout the event letting us know about different technologies and APIs that we could potentially integrate into our instrument.

I was really intrigued by a presentation given by Eric Tyler, founder of ImagineX, on how to use TouchDesigner to control the lights and visuals in a given space. He's worked with a handful of big name electronic artists, handling their stage visuals for their tours and festival shows. Since we had access to a Leap Motion, I figured why not make use of it?

Syncing Light, Sound and Graphics

With Eric's assistance, I put together a TouchDesigner project overnight that connected to the Leap Motion attached to the body of our instrument. Like a whammy bar but for visuals, the Leap Motion would detect hand gestures and respond by manipulating the animations on TV displays in the room. People really had so much fun with playing with the TouchTone, controlling both visuals and sound.

13000373_969826239733206_2303912368446538180_n.jpg
We got to present our instrument to TroyBoi, one of the guest judges! He had a blast trying out TouchTone.

We got to present our instrument to TroyBoi, one of the guest judges! He had a blast trying out TouchTone.

Final Thoughts

Out of all the hackathons I've done, this was the best one I've participated in for various reasons. It was so incredible getting to hack on something physical for once and build a working, polished product from start to finish.

I can't explain what it was like to see all these people who were so excited to get their hands on our instrument and marvel at it. It was the most rewarding hackathon experience and I learned an incredible amount about animation, prototyping, hardware hacking and more.

It's amazing what you can accomplish in 24 hours if you have the right team and a positive attitude. Our team was applauded by the judges for our multidisciplinary backgrounds. Check out some of the feature articles below:

Forbes Magazine: "Diversity Wins Big at First-Ever Red Bull Hackathon"

Carnegie Mellon HCI Institute: "Diversity and Innovation Take First Place at Red Bull Hackathon"