A few recent experiments...

KINECT TRACKING + SPEAKER ARRAY WAVE FIELD SYNTHESIS EXPERIMENTS:
- demonstrating KVL Kinect Tracker in combination with custom dense array beamforming software (under development at KAUST and UCSD)

link - In this demo, I've created a small tracked zone at the front of the room. When no object/person is present, a single sound beam automatedly pans around the room left-right-left-right... You'll hear that when I step into the tracked zone, the beam then snaps to my location and I am able to steer the beam by walking around. Ideally, in an acoustically treated space, the sound would be nearly inaudible unless the beam was pointing directly at you, but since this room has glass walls, you can hear quite a bit of reflection.

link - In this demo, I've set the entire room to be the tracked zone. Whoever is closest to the camera gets the beam, so basically you'll hear Paul and I stealing the beam away from each other. You can really hear the beam snap back and forth hear.

KINECT EXPERIMENTS:

link - model viewing with Kinect-based depth-sensitive face tracking.

______________________________

some older videos...

3D GAME-SPACE:

link - This video documents recent work designing interactive sound environments in 3D game space, using the Torque 3D Game Engine.

SURROUND AUDIO & VIDEO:

link - This video documents some of my recent work as resident programmer/teacher at Harvestworks. For this project, working with resident artist Hisao Ihara, I designed a performance tool for real-time manipulation of up to 300 picture/video planes in surround A/V (quad audio and video). The four audio channels have been mixed down to stereo for the above video.

LIVE VIDEO TRACKING:

link - variable-sized audio triggering interface using using brightness tracking
link - controlling a sonic blob via color/motion tracking
link - looking around in 3D space using face tracking
link - a flight simulator controlled by live video tracking
link - same as above... but with spatialized audio
link - a flight simulator controlled by the Wii Remote
link - controlling an avatar in the Torque 3D Game Engine via video tracking (more to come on this...)
______________________________