Video: Prototype Creates Music from Interaction with Projections

It turns out that if you connect a laptop to a webcam and a data projector mounted above a horizontal surface in a dimly-lit room, a program called parab0xx from AlgoMantra Labs could let you create music by interacting with projected images using any small, movable light source – a candle, a cellphone screen, a lighter, whatever.

It's sort of hard to grasp until you see the video:

The program's development team is currently deliberating on whether to releasecode to the open-source community, so it's possible one of these couldbe showing up to a bar, nightclub, or elsewhere soon (in fact, I swearI've seen something similar installed before... if anyoneknows what I'm referring to, please let me know).

Potential applications for this human-computer interface include"aerospace, surgery, gaming and music" among other things, according toParab0xx's creators, who describe it thusly:

The parab0xx is a software prototype that shows how you can interactwith virtual images projected on a simple living room table, using awebcam. You can play games, make music, or even edit videos. In short,
it is a kind of primitive computer that obeys the instructions of asimple LED (light-emitting diode). Alternately, you can use a luminousphone screen, a candle's flame, or even a piece of white paper as acursor...

The ten orange boxes are programmed to bounce off the edges, and moveperpetually slowly in a straight line. Apart from this, they do nothingunless somebody messes with them. The blue box follows any singularsource of light closely, and if there are multiple sources, it sits atthe place where their geometric average falls.

Every time the blue box touches any yellow box once, a 4-5 second sample of tabla
is triggered and the orange box that was hit changes itsdirection(bounces off). Normally, if the blue box is also moving, youwill end up triggering it multiple times (direction also changes thatmany times in a second). The way to trigger the sample only once is toput the blue somewhere in the path of any yellow box, take away orswitch off the light, and wait for one of the orange box to arrive.
This system allows us to create an endless number of rhythms and beatsfrom a very small, single sample.

You can even program thisarrangement to behave in predictable ways, by placing pieces of whitepaper on the black table. When the orange boxes pass over any paperperiodically, they start behaving as a source of light themselves! Youcan create some funky feedback patterns using this, and play a game ofprediction and strategy, besides creating complex tabla rhythms.