During the last three years, we studied various methods for computer-based gesture analysis, with the general goal of using body movements to interact with sound and visual media. We realised a system called Gesture Follower that allows for recognition and synchronisation of gestures with digital media.
This research has largely been influenced by sustained collaborations with musicians/composers and dancers/choreographers. I will present the technology and some of these works, including recent collaborations with the composer Lorenzo Pagliei and the violinist Mari Kimura.
Recently, I focused on the usability of these tools. I will present a brand new application called Gesture Interaction Design (GID), whose main purpose is to guide the artist in the creation and design of his own gesture vocabulary.
I will also introduce an application of this technology for gesture-driven audio/video mosaicing.