Enrico Costanza, PhD student at the Media and Design Laboratory of Lausanne’s Ecole Polytechnique Fédérale, has focused his research on designing objects that can bring digital and physical worlds closer. His Audio d-touch project, developed with Simon Shelley, is a clear example of this approach. It consists of three tangible interfaces that are used for musical composition and performance. They have been suggestively named, Augmented Stave, Tangible Drum Machine and Physical Sequencer.

The latter has been used in two live performances by the Sicilian cellist and composer Giovanni Sollima. Using the sequencer Sollima is able to record audio or voice samples and organize them in variable length loops. These samples are associated with physical objects (a few simple blocks of wood) while the interface is an interactive surface bounded by a sheet of paper. Moving the blocks of wood on paper (each of which is labeled with specific signs), the musician is able to “play” his own instrument (the cello for Sollima), without a monitor: the orientation is determined by the position of the object on the sheet. A standard computer “watches” the position of objects with a webcam while the d-touch vision system (entirely coded in C++ under Linux) is able to localize them precisely. The information about the position and orientation of blocks of wood is used to control a digital audio synthesis, thus, ultimately, to give voice to a musical instrument or a virtual singer. In live performances a closer shooting of the interface is projected on the screen behind the artist, allowing the audience to share the performer’s interaction with the odd objects designed by Costanza. Another interesting element is the use of extremely cheap components, which characterize the Physical Sequencer performances to a greater extent. — Vito Campanelli, Neural. Video below: