June 14, 2005
Living Book of the Senses
Body Stories
In Living Book of the Senses--by Diane Gromala--users are able to see their physical surroundings while dynamically engaging with three-dimensional mixed realities which appear on their headsets. Users can interact with the book in dynamic ways. They can ask the book questions (via voice recognition), and can influence the book through their sensory (bio) feedback. Users wear a headset/head-tracker/color camera system that enables them to see physical reality enhanced with a virtual reality overlay. The camera inputs images/patterns and feeds them back into the ARToolkit software which then displays digital information associated with the physical markers onto the headset. The ARToolkit can calculate camera position and orientation relative to physical markers in real time for video-mediated reality.
Each reader can view AR scenes from their own visual perspective. Users can fly into the immersive world and see each other represented as avatars in the same virtual scene. Readers remaining in the AR scene have a birds'-eye view of other readers as miniature avatars in the virtual scene displayed through their headset. User-controlled dialog with the book elicits responses/answers from the book (expressed in digital data: visual, textual, auditory). As the users simultaneously interact with the book in the physical and virtual realms, the book responds to individual and multiple physical states (via biofeedback) to express resulting changes in narrative. The narrative is a cultural history of the senses. [Read Extensive Bodies, interview by Yvonne Volkart with media artist and theorist Jill Scott].
Posted by jo at June 14, 2005 09:00 AM
