A few recent experiments...
KINECT TRACKING + SPEAKER ARRAY WAVE FIELD SYNTHESIS EXPERIMENTS:
- demonstrating KVL Kinect Tracker in combination with custom dense
array beamforming software (under development at KAUST and UCSD)
- In this demo, I've created a small tracked zone at the front of the
room. When no object/person is present, a single sound beam automatedly
pans around the room left-right-left-right... You'll hear that when I
step into the tracked zone, the beam then snaps to my location and I am
able to steer the beam by walking around. Ideally, in an acoustically
treated space, the sound would be nearly inaudible unless the beam was
pointing directly at you, but since this room has glass walls, you can
hear quite a bit of reflection.
- In this demo, I've set the entire room to be the tracked zone.
Whoever is closest to the camera gets the beam, so basically you'll
hear Paul and I stealing the beam away from each other. You can really
hear the beam snap back and forth hear.
link - model viewing with Kinect-based depth-sensitive face tracking.
some older videos...
link - This video documents recent work designing interactive sound environments in 3D game space, using the Torque 3D Game Engine.
SURROUND AUDIO & VIDEO:
- This video documents some of my recent work as resident
programmer/teacher at Harvestworks. For this project, working with
resident artist Hisao Ihara,
I designed a performance tool for real-time manipulation of up to 300
picture/video planes in surround A/V (quad audio and video). The four
audio channels have been mixed down to stereo for the above video.
LIVE VIDEO TRACKING:
link - variable-sized audio triggering interface using using brightness tracking
link - controlling a sonic blob via color/motion tracking
link - looking around in 3D space using face tracking
link - a flight simulator controlled by live video tracking
link - same as above... but with spatialized audio
link - a flight simulator controlled by the Wii Remote
link - controlling an avatar in the Torque 3D Game Engine via video tracking (more to come on this...)