Cursive

A novel interaction technique for controlling expressive avatar gesture using pen gesture.

Work by Francesca Barrientos and John Canny.

[Francesca Barrientos home]

Controlling avatar gesture

An avatar is the graphical representation of a person in a virtual world and functions as the person's virtual body. In virtual  worlds, the avatar indicates presence and provides a means for nonverbal communication. With the appropriate controls, the avatar body can be posed and animated to create bodily postures and gestures.

Many virtual worlds are accessed using desktop, laptop or even palm top computers. Using these computers poses the problem of controlling a complex avatar body using only the input devices common to these computers: the keyboard, mouse and pen.

Previous solutions

Current interfaces for controlling avatar-based nonverbal communication (NVC) permit only a small range of NVC behavior. Usually the user selects a button or menu item in the user interface to select a static facial expression or posture. Such discrete controls such as buttons and menu items do not map well to the continuous movements used in hand gestures and other types of NVC. 

User controlling avatar using pen gestures.

Movement can be energetic, desultory, emphatic or have other expressive qualities. These expressive qualities can alter a gesture's intended meaning.  A wave of hello can be changed from an enthusiastic welcome to polite acknowledgement depending on the way it is performed. Part of my goal in creating Cursive was to design an interaction technique that allows a user to control the expressive qualities of an avatar's gesture.

Pen gesture to avatar gesture

Our solution is to map pen gesture to avatar gesture, using not only the identity of the pen gesture, but also its handwriting features. That is, the way the pen gesture is written controls the expressive qualities of the avatar's movement.  The avatar is provided with a library of gestures that it can perform, such as waving, bowing and shrugging. Each pen gesture is mapped to a particular avatar gesture. Then handwriting characteristics---the speed and size of writing---are mapped to expressive qualities of the avatar gesture movement..

Cursive window for accepting pen gestures.

Application

Cursive is both an  interaction technique and  an application that implements the technique. Cursive is intended to be run alongside a multi-user virtual world application. Users communicate with each other verbally using speech which would be managed by the virtual world application. As the user is speaking,  she can control the avatar's gestures using Cursive. These gesture commands are sent to all copies of the avatar that appear in the browsers of other visitors to the virtual world.

Cursive runs on the avatar driver's  machine, presenting a window for writing gestures. The architecture of Cursive is designed to work with VRML based virtual worlds that can be viewed using a standard VRML browser. The avatar is a VRML avatar with scripting nodes set up communication between the avatar and Cursive. Those who view the avatar from other machines can see the avatar's gestures without having to run special software on their own machines.

Related publications

Details of this project are described in the the following paper:

Francesca A. Barrientos and John F. Canny, "Cursive: Controlling expressive avatar gesture using pen gesture," in Collaborative Virtual Environments, September 30-October 2, 2002, Bonn, Germany, pp 113 - 119. [pdf 223KB]

in this short paper

Francesca A. Barrientos and John F. Canny, "Cursive: A novel interaction technique for controlling expressive avatar gesture," (technote) UIST '01, November 11-14, 2001, Orlando, Florida, pp.151-152. [pdf 29KB]

and my Ph.D. dissertations

Controlling Expressive Avatar Gesture.