A Berkeley undergraduate named Charles stands in the middle of a three-meter cube. He moves slowly, watching a ghostly projection of himself move on a screen in front of him. The colors are murky, and his dark pants have completely disappeared, leaving empty space between his shirt and his shoes. Still, what’s up on the screen looks three-dimensional, like a hologram; it moves precisely as he does and is definitely him.

     
Digital Ballet Through tele-immersion, dancers
in Berkeley and at the University of Illinois
performed with each other — while remaining
2,200 miles apart. Edward Casadi

See it online

The cube, high up in the Hearst Mining Building on the Berkeley campus, looks like a boxing ring surrounded by equipment that includes 48 cameras and at least 13 computers. Its purpose is to enable long-distance collaboration by gathering full, three-dimensional information about anyone moving inside it, instantaneously creating the image, and sending it over the Internet. The team working on so-called tele-immersion (also known as three-dimensional video conferencing) comprises computer scientists, as one might expect, but also dancers and dance scholars. Charles is in the cube because he is taking CS 84, probably the only sophomore seminar in computer science to require choreography. The unlikely tele-immersion crew is led at Berkeley by electrical engineering professor Ruzena Bajcsy, is choreographed by Lisa Wymore from the theater and dance department, and includes counterparts at the University of Illinois. Though tele-immersion may seem like just another cool computer-generated effect, these researchers have grand ambitions for their project.

Some of those ambitions are practical, such as the ability to diagnose a disease from a long distance or document performances or enable simple video conferencing in three dimensions. But Wymore and Bajcsy also project that tele-immersion will help us reinvent our relationship with technology itself. What if we could use our whole human bodies, idiosyncratic and imperfect as they are, to communicate technologically, rather than just our keyboarded fingers, cell-phoned ears, and computer-screened eyes?

Bajcsy asserts that dance is a language, and she imagines using tele-immersion to analyze movement across cultures, just as written and spoken languages are analyzed. With the richness of information that tele-immersion captures, she postulates, every movement could be examined in exquisite three-dimensional detail and compared to similar movements from elsewhere. Imagine tracing the African diaspora through dance or seeing how the gypsies’ Indian roots affect their dance in different parts of the world.

But first there are some thorny technical problems to solve. The computers must separate the moving figure from its background and reassemble an entire three-dimensional human body from images of its surface, without losing more than a fraction of a second. (If they could delay by a few minutes, or even seconds, the task would be much easier.) In order for the system to reconstruct people well, they have to be wearing the right clothing. If they move too quickly, the tele-immersive image may disappear or show eight or ten arms at once. Right now, everything has to be done in super-slow motion. The system can also get confused between the front and back of the body and switch them, leaving the head facing forward over its owner’s rear end.

page 1

|

2