Screen combines familiar game mechanics with virtual-reality technology to create an experience of bodily interaction with text. The language of the text together with the uncanny experience of touching words creates an experience that does not mesh easily with the usual ways of thinking about game-play or VR.
Screen is a collaboration with Andrew McClain, Shawn Greenlee, Robert Coover, Josh Carroll and Sascha Becker that was created in the Brown University immersive virtual-reality chamber (Cave), as part of a research project in spatial hypertext writing . Brown's VR chamber is similar to the University of Illinois's CAVE—a virtual environment that shows 3D images while allowing users to continue to see their own bodies and does not require users to wear encumbering equipment (such as head-mounted displays, which are essentially blindfolds with televisions inside) . Brown's Cave is an 8-ft cube, missing its top and one side, and its walls and floor are projection screens. A projector pointed at each screen alternately projects images meant for the user's left and right eyes. The user wears shutter glasses that alternately occlude the left and right eyes in synchronization with the projectors. The result is stereo VR—3D vision of computer-generated imagery—combined with the physical presence of the people and objects in the Cave.
The initial experience of Screen can be disorienting for those familiar with VR. Rather than make the walls "disappear," we project flat images onto the same plane as the walls, reinforcing their presence. The images we project are not of colorful shapes but of white text on a black background. This text at first appears in an introduction that fades in and out on the walls and then forms three traditional paragraphs, each nearly filling one of the walls. Each of these paragraphs is a character's moment of memory that gives rise to the virtual experience of touch. Each wall is read aloud after it appears. When the last has been read there is a pause, and then a word peels itself from one of the walls and flies toward the reader. If the reader does nothing, the word circles near her. Soon another word peels away, and then another, at an increasing pace, flocking around the reader. The reader can intervene in this process by batting at words with her hand. When a word is struck a sound is heard, and the word flies back toward a wall, perhaps breaking apart in the process. If a struck word is the only word off the walls, it, or part of it, will return to the space it left empty. However, if more than one word is off the walls, then a struck word may return to a different space.
Once the number of words off the walls passes a certain threshold—something that, with the increasing pace of peeling, only very active engagement can long delay—all the remaining words come free of the walls, swirl around the reader, and then collapse into the center of the Cave (Color Plate B No. 1). A final, "closing" text is then heard. In addition to creating a new form of bodily interaction with text, Screen creates three reading experiences—beginning with the familiar, stable, page-like text on the walls, followed by the word-by-word reading of peeled and struck text (where attention is focused) and then the more peripheral awareness of the arrangements of flocking words and the new (often neologistic) text being assembled on the walls. Screen was first presented in 2003 as part of the Boston Cyberarts Festival, and in 2004 it was included in "ALT+CTRL: A Festival of Independent and Alternative Games" at the Beall Center for Art and Technology, University of California, Irvine. A new version, built with G3D on Brown's Linux cluster, is in process.