September 06, 2005
TeleMurals
Blending [Distributed] Participating spaces
Telemurals is an audio-video connection where a communication space is created by breaking apart the pixels and speach of the participants at each end and reassembling them abstractly. The initial setup is straightforward. Two disjoint spaces are connected with an audio-video wall. Video and audio from each space is captured. The two images are then rendered, blended together, and projected onto the wall of their respective space. The difference between Telemurals and traditional media space connections are the image and audio transformations that evolve as people communicate through the system and the blending of the participating spaces.
Participation is required for this communication space to work. To reinforce a sense of involvement, we provide the system with some intelligence to modify its space according to certain movements and speech inflections. First, the image is rendered non-photorealstically. Second, words spoken in both spaces are captured, converted to text, and rendered on the screen in blocks left to fade away over time. The immediate feedback of seeing one’s spoken word alter the window lets them know they are adding to and affecting the shared environent. More complicated image manipulations are affected by changes in pitch and volume of the voice.
Much research has been conducted on the benefits and limitations of adding video to an audio communication channel. One of our goals here is to see how using non-photoreastic images influences conversation. Adding video to a communication channel improves the capacity for showing understanding, attention, forecasting responses, and expressing attitudes. A simple nodding of the head can express agreement or disagreement in a conversation. Gestures can convey concepts that aren’t easily expressed in words; they can express non-rational emotions, non-verbal experiences.
Yet these cues are not always properly transmitted. There may be dropped frames, audio glitches. Lack of synchronicity between image and audio can influence perceptions and trust of the speaker at the other end. Other challenges include equipment placement. For example camera placement has long been a reason for ambiguous eye gaze in audio-video links. A large camera offset gives the impression that the person you are speaking to is constantly looking elsewhere.
With Telemurals, we are creating an environment where rendered video maintains subtle cues of expression such as posture and hand motion, yet also enhances other cues. For example, changes in voice volume and pitch may be represented in the graphics of the rendered video. Audio conveys the speach inflections in conversation by altering parameters of the rendering. By adding another layer of abstraction into the video stream, we can enhance cues in a manner that is not possible in straight video streams.
Abstracting the faces is of big concern in the rendering. It builds on the intrigue of how one would appear if transplanted into this filtered space. Caricature and cartooning are such types of abstractions. There are elements of curiousity, humor, and surprise when seeing a caricature yet there is enough resemblance to make a personal connection.
The creation of an abstract environment in Telemurals has some similarities to virtual spaces in online worlds. Examples of fantasy environments for conversation can already be seen in online graphical communities. One essentially puts on a mask and enters an electronic communication space. This element of mediation removes many of the social barriers by allowing one to safely enter a stimulating world. Just as this lack of concrete identity may aid in suddenly and continually confronting new people in virtual environments, it may help in our physical-virtual hybrid. It has the added benefit of mitigating the perception of offset eye gaze and the effects of unsynchronized audio and video.
In this project, the abstraction of person, setting, and feedback provide the social catalyst for the experience. This new wall created by filtering creates an ice-breaker, a common ground for interaction, and a object for experimentation. How will one communicate in this abstracted space? How will their behavior affect their appearance and the appearance of the setting? How different is communication using photorealistic vs. non-photorealistic video? The goal here is to create a new styles of movement and speach interaction by providing a common language across the two spaces.
The first Tele-mural installation will connect two MIT graduate dormitories, Ashdown and Sidney-Pacific. The Tele-mural in Ashdown is located in the lower level in the common reacreation room. In Sidney-Pacific, the Tele-mural will be placed near the main entrance. This connection came about as the under-construction Sidney-Pacific dormitory committee was looking to put public art in its public areas and create spaces to encourage students to gather. Ashdown, the oldest graduate dormitory on campus was similarly undergoing renovations to create public spaces for social gatherings and the two dormitories were open to the idea of linking the older dormitory to the newer one. The sites within the dorms were chosen because they have high levels of traffic, are openly public , and because a large video wall aesthetically blends into the space.
Posted by jo at September 6, 2005 10:52 AM
