The new choreography
by Rebecca Allen
with Jane Nisselson

Rebecca Allen, producer of the animation, is a designer, director and researcher at the Computer Graphics Laboratory of the New York Institute of Technology. Jane Nisselson works with and writes about computer graphics systems at the Laboratory.

Throughout history we have expressed the need to create an entity that can assume human qualities and yet surpass our limitations. We expressed this need by creating artificial figures that personified an ideal: Galatea, for example, sculpted by Pygmalion and brought to life by Aphrodite. In such figures we project a perfected image of ourselves.
    In the field of computer graphics, scientists began developing methods for graphically simulating real-world objects. And, of course, before too long the computer was used to generate a realistic model of the human figure. At the New York Institute of Technology's Computer Graphics Laboratory, we have described a three-dimensional mathematical model of a female figure. Like a living person, our figure can move and bend, performing dynamic actions and gestures. She is animated with the computer.
    An example of our figure's physical capabilities can be seen in her role as Saint Catherine in the dance video production of The Catherine Wheel, directed and choreographed by Twyla Tharp. While working on the piece, Twyla visited our lab and saw my animation of the female figure. She approached me with the idea of a collaboration in which this figure would represent St. Catherine. Its premiere on PBS in the spring of 1983 marked the first time human and computer figures danced together for the public.
    As St. Catherine, our computer dancer represents an ideal who offers spiritual guidance in the face of human shortcomings. Her image appears early in the performance, materializing shortly after an animated image of the "Catherine wheel" (a bladed torture device named after the martyred saint). Through her movement, she suggests "dance" and becomes an inspiration to the Leader, played by Sara Rudner. Later St. Catherine returns and teaches the Leader a dance theme that recurs throughout the performance. They dance together, in synchrony, and end in an outstretched position as the Catherine wheel reappears. Computer Catherine is seen, a last time, ascending a staircase: a symbolic gesture to calmly remind Sara of her unrelenting pursuit of excellence.

The Notion of Motion
How is it possible to make a series of images not only move in a human way, but dance? The analysis of human motion requires the study of actual human movement as well as physical structure. I began using animation and the technique of rotoscoping as a way of accurately modeling human motion in a stylized environment. Beginning with real actions, I would simplify its essence. This differs from conventional cartoon animation techniques, which typically build fictional actions in an exaggerated form.
    The computer's capacity to make moving images from static key poses makes it an ideal tool for animation and motion analysis. Still images containing one of the many stages of an action can be transformed into a continuous movement such as walking, running or turning.

keyframe to keyframe takes a bow

    Depicting motion realistically requires a spatial context that is more sophisticated than the flat or two-dimensional space used in conventional animation. For instance, an object moving away from us appears to become smaller. A computer can calculate the correct appearance of objects as they move in space; this realistic perspective is important in accurately representing movement. Thus the computer is used as a means of analyzing human motion and as a medium in which to portray it.

Taking the First Step
For a computer-generated figure to take even one step requires an adequate animation system. The software for our system was developed over many years by a number of computer scientists at our laboratory. Using high-speed mainframe computers, they gradually fine-tuned the software into a unique system that enables us not only to move objects, but to generate complex motion for characters with articulated joints. Moving a figure or any of its parts in all directions gives us the means to create an illusion of life. The way a character moves defines emotions and personality.
    St. Catherine was modeled by taking key points from photographs of a woman's body and entering them into the computer. Connecting these points, the computer composes rigid body parts linked by joints and arranged in a hierarchy, or tree structure. The figure is viewed as a wire-frame model on a real-time state-of-the-art display. Using a joystick, I can rotate, move or scale (change the size of) the figure or any of its parts. The model is like a marionette; the animator is a high-tech puppeteer who controls and positions the figure by manipulating a joystick.
    This animation system can be illustrated by the simple example of making the figure bend forward at the waist, then return to an upright position. I begin with an image of the figure standing upright as the first "keyframe"-the first position of this movement (Fig. 1). Then I locate the waist joint and, using a joystick, rotate the upper half of the body to a bending position that becomes the second keyframe (Fig. 2). A keyframe of the figure standing upright ends the movement. This creates a three-keyframe sequence: standing, bending, then standing. After determining the length of time the movement should take, I run a program that generates all the "in-between" positions. The result is the smooth motion of a simple bow.
    Using this system to generate realistic human motion requires complicated positioning of all body parts, e.g., the head, shoulders, elbows, wrists, fingers, into keyframes. The analysis of a dance is an intricate procedure, and a dance choreographed by Twyla presented an especially difficult challenge.

computer dance

Learning to Dance
The choreography for the St. Catherine figure was based on the movements designed by Twyla for her dance company. We videotaped one of the dancers as a model of motion, and I studied individual frames of the tape to determine the key positions of particular movements. I then positioned each joint of the computer figure to correspond to the dancer's key positions. Twenty-five still frames were needed to generate each second of animation. I produced two to three keyframes for each second of motion, and from these the computer generated the rest.
    The animated figure and the human dancer had different limits and freedoms, so we adapted the choreography for St. Catherine according to her technical restrictions. Twyla would show me the moves she wanted, and we would determine if they were technically possible. Overall, the computer dancer's movements were kept close to the physical range of a human dancer, with some movements extended slightly beyond human capabilities.
    To make St. Catherine appear to dance naturally, we had to consider the physical world in which she moved. Building the illusion that the computer figure was dancing with Twyla's company demanded that she appear on the stage, in the same space as the live action. During live-action filming by a BBC crew, I recorded the position of the camera in relation to the dancers and then, with our animation system, matched my "simulated camera" to the real camera viewpoint so that Computer Catherine would appear to be in the actual space.
    While my dancer had the freedom to move in ways that may not be considered physically accurate, I used the "personality" or style of this animation system to make motion that was believable but distinctive from that of the human dancers. Part of the challenge was finding creative ways to bring the two worlds together.
    Sometimes it was the differences between the two dancers' worlds that provided a means to unite them. A human's movements are continuous, with each step leading directly into another. But the computer-drawn figure's movement is only an illusion created by a series of frames generated by the computer shown fast enough to appear as continuous motion. Each frame is an instant in time in which the computer dancer makes a movement. I could use the frames to change the timing and sequence of the dance steps. When first compositing the human and computer dancers, we discovered that Sara moved into a turn much faster than St. Catherine. By reanimating this section, I was able to reduce the number of frames in which the computer dancer made this turn. The animation system allowed me to synchronize St. Catherine's timing with Sara's so that their turns met in space.

In many ways the animator and choreographer do the same thing. Both design movements that their performers then execute. But the method of creating a dance with human dancers is very different from that used to make the computer figure dance.
    At the filming of the performance, the stage set was filled with the hustle and bustle of a live-action production. Twyla's dancers were rehearsing, the film crew was busily setting up, telephones rang and people were running out for lunch. This was very different from the atmosphere at the lab. Initially I worked with another researcher, Robert McDermott, to shape the data base for the body description, but once I began the animation my environment was isolated and internal. Typically I worked at night, when there were long stretches of uninterrupted time. Plugged into a Walkman, with the music for the dance blaring over my headphones, I would sit in a small, dark room, staring at a monitor. The figure would begin to dance as I used the joystick to "describe" to the computer which of the figure's limbs and joints should move and in what direction.
    Like the choreographer, the animator directs the dancer's movements, but the image of the dancer must also be created. Certainly St. Catherine's appearance distinguished her as being from a world other than that of the rest of Twyla's company. Her figure is composed of lines in a way that gives a random, hand-drawn quality to a computer-generated image. I wanted to break away from a mechanically rendered appearance and typical style of computer-generated graphics. Also, eliminating a lot of information found in a detailed rendering helped to focus the imagery on the figure's movements.
    Through her movement, Computer Catherine implicitly took on the form of a dancer. And as a dancer, she was able to perfect and share the essential elements of human movement that give it its expressive quality. But after all, in producing this animation, we were also bringing a knowledge of human movement and expression to the computer. Through this collaboration I was able to discover a new interpretation of the human form.

computer dance

Return to Table of Contents | Previous Article | Next Article