Post-virtual reality: After the hype is over

This article, in slightly different form, will appear as a chapter in the paperback version of Brenda Laurel’s book Computers as Theatre, available this summer from Addison-Wesley. Laurel is a senior researcher at Interval Research in Palo Alto, CA.

A lot has happened in the evolution of interactive media in the two years since Computers as Theatre was published. By the end of 1991, movie producers, cable TV executives and theme-park entrepreneurs began to talk about “passive VR” (an oxymoron if there ever was one), and the term “virtual reality” had begun to spread like an oil slick over anything new and sexy in the world of interactive entertainment.

Meanwhile, hip young Northern Californians, initially at the forefront of VR enthusiasm, began to make jokes about “face-sucking goggles.” As the first mainstream books, movie, and TV shows picked up on the hype, the younger and more creative contributors to the VR community began quietly mutating.

In late 1992, VPL Research — pioneers in the development of enabling VR technology and wellspring of the pop-culture phenomenon — laid off most of its employees.

CHANGING THE SHINGLE TO SOMETHING LESS TAINTED

As the VR “meme” started to flame out in Northern California in 1992, many of us began scrambling to change the words on our shingles from “virtual reality” to something less tainted — telepresence, augmented reality, immersion technology. Anything to get some distance from the all-too-vivid spectacle of the hype-fueled VR-road-and-media show that rocketed VR pundits to the pinnacle of pop culture and then sent us burning back into the atmosphere, noticing too late that we were in the decaying orbit of a fad.

“Hey guys,” little voices shout from the capsule as it begins to glow, “We weren’t done yet… we were just beginning…”.

VR and video games. As a fad phenomenon, there is a striking similarity between VR and the video game business, most particularly Atari. Although it funded a long-term research lab for two years, Atari turned a deaf ear and an empty pocket to the kinds of innovation that could not be wedged directly into the installed base of hardware.

It also shunned innovation that did not fit easily into the rapidly congealing cultural milieu of its products. Atari’s most glaring error (under the Warner regime) was to grossly misunderstand the pace and magnitude of technological change, mistaking the metabolism of computers for that of packaged goods.

Finally, Atari was blind to the fact that it was developing not a consumer-products company, but a medium — one that depended as strongly on content as on technology for its appeal, and which would be ultimately driven by the dynamic between them.

Making it stupid. “At Atari,” the internal jingle went, “we’re making it stupid.” We all knew we were in the grips of an organization that had, to put it kindly, a temporally bounded agenda.

Young idealists burning with the promise of new technology found themselves in a pop-culture factory that catered to its own status quo. We got to do our work in a wonderland — a place with seemingly endless resources and all kinds of slack for creative people — if only we would couch what we were doing in the vernacular of what-was-happening-now.

‘MAKE THE TECHNOLOGY SING AND…TRANSFORM THE MOVIES’

But once you had the culture-making bug, you began to see something of where things might go and what they might become — and that’s where the problems began. They said, “license the latest movie hit and make it into a game” — and some of us thought, “make the technology sing, and maybe it will transform the movies.”

Management turned a fire hose on these strange new stirrings, but the money kept on coming. Million-dollar hackers barely out of their teens drove sports cars into the sea; marketeers consumed pounds of cocaine and ate guns for dessert. Such were the torments of the Fad Mentality.

Most of us lived. Some of us went on to the next thing. For me the next thing was what came to be called VR. An amazing amount of good work got done between 1982 and 1992, before the VR fad started to flame out. Much of it was done by the survivors of the Atari phenomenon. One good thing about living through both of them is that you learn that it’s not the end of the ride — it’s just another local minimum on the roller coaster.

ALL THAT DIED WAS THE HYPE

What went away when the fad died? Primarily, the hype. The hype positioned VR as a technology with the potential for creating a radically new entertainment medium almost instantaneously. It promised that entertainment and technology companies would emit “hi-res, hi-touch” consumer VR products in the short term. It provided the palpable icons of head-mounted displays and datagloves — encrustations on our bodies that we would be willing to put up with for the enormous rush they would enable.

Many people hypothesized that this new form of entertainment would replace both video games and TV. The more hopeful among us declared that it would transform the very nature of human communication.

Firing an entire lab. But the consumer products were not forthcoming. Movie executives that invested millions in the Great Next Thing stepped back in dismay when they did not see hi-res movies in their head-mounted displays. Mattel introduced, then shelved, the PowerGlove, firing its entire VR lab at the same time. Showroom jockeys learned that, after the hype is over, VR probably won’t sell a kitchen or an automobile. VR pundits learned, or should have learned, that the worst thing you can do is fire up public expectations when you can’t deliver product by Christmas.

The train wreck was not as bad as it might have been, because many interesting lines of thought were involved, each with its own momentum. They did not begin with VR and they will not end with it. VR was a monolithic icon for a complex network of ideas. Although the momentary nexus has passed, the work — and the vision — continue.

THE GENESIS OF VR’S TECTONIC ACTIVITIES

While it is true that the threads that make up VR all began before it had a name, the particular constellation of ideas that coalesced caused some serious tectonic activity.

The premise of involving the senses in interactive computing began with Ivan Sutherland’s forays into the world of graphics. Likewise the sense of hearing, which had found comfortable homes in signal processing and video game embellishment, suddenly became a full-fledged concern when VR kicked down the door to the senses. These lines of development, once separate, found new potency as they were integrated by those who wanted to make VR happen in the real world.

Put that there. In the interface domain, experiments such as Schmandt and Hulteen’s “Put That There” used gesture to make speech less ambiguous. Though they still gave primacy to language as the mode of interaction, “Put That There” and other experiments at the MIT Media Laboratory were nonetheless early to recognize the notion of multisensory interface, and it is perhaps to their credit that the folks at the Media Lab valiantly resisted entering into the “VR” domain at all.

The related notion of sensory immersion also has some roots in the Media Lab, with the work of Michael Naimark and Scott Fisher, among others, in the late ’70s and early ’80s. These folks noticed that something qualitatively different happens to you when your senses are surrounded compared to when you are simply gazing at (and listening to) a screen.

A drop from the hose. Naimark took to testing the idea with surround environments filled with relief projections, while Fisher economically placed it on our heads. Both were inspired by the observation that our channel of communication with present-day computers is a drop from the firehose of human bandwidth.

Both, of course, had their predecessors, but both broke new ground in bringing our attention to the nature of the effects that immersion could induce and the domain of research that concerns itself with how we orchestrate them.

THE NOTION OF THE VANISHING INTERFACE

Implicit in much of this work was the notion of the vanishing interface. Before VR, however, the vast majority of work that was done in the interface domain took for granted the notion of the computer as an explicit party to the interaction, and satisfied itself with making that interaction smooth, tractable, “natural” — nay, “intuitive.”

It is truly astounding to me that this notion has such tenacity. If it had happened in the domain of film, we would all go to see projectors instead of movies. By positing that one may treat a computer-generated world as if it were real, VR contradicts the notion that one needs a special-purpose language to interact with computers.

Blowing out the pane. In fact, it throws out the idea of the interface as a cognitive artifact, tool or anything else that should impinge upon our experience. “Direct manipulation” becomes direct sensory encounter, and the pane is blown out of the interface window.

Another tenacious idea, and one that was pioneered by Scott Fisher and others long before VR was a household word, is the concept that Fisher named “viewpoint-dependent imaging,” the idea that all imagery should be constructed from the point of view of an individual human located in the virtual environment.

YOU WILL NEVER SEE THE WORLD IN OVERVIEW

Because of this founding principle, it is hard to create an “establishing shot” in VR — David Zeltzer and his team at MIT spent the better part of a year on the problem. In fact, there is no such thing as an establishing shot in nature — no “overview,” no “camera’s eye.” Unless your spirit animal is a bird of prey, you will not see the world in overview except as an artifact of cinematic or graphical convention.

VR is utterly a first-person point-of-view medium, congruent with the human sensorium. The notion of point of view in VR is the manifestation of each person’s relationship to the world he inhabits. It is evidence that VR showed no prejudice in the throwing out of conventions — it questioned film as rigorously as interactive computing.

Getting the feel. If one is to get the feel of a place, one must walk around it, sniff it, pick things up, feel the presences of other beings with all the senses. The theatre reflected this impulse in the transition from proscenium to thrust and arena staging, and more recently in the blendings of performance, ritual and improvisation that characterized the theatrical avant-garde of the late ’60s.

Somehow in the world of computing, the proscenium, the overview, the schematic, the “graphical representation” have become the norm. But VR reminds us of what we are good at; namely, perceiving the world from a located and situated vantage point.

Involving the sensorium. Alternate views are prostheses that are helpful under some circumstances, but which do not activate all of the circuitry that evolution has so magnificently prepared for us. The premise here is that, when presented with a representation that involves the whole sensorium, a person will respond holistically — and that the whole will be more than the sum of its parts.

Here is the primary means by which feel, hunch and intuition may enter. While drama produces something like this effect by orchestrating emotion, VR produces it through sensory immediacy: the first-person point of view.

Finally, as a proponent of a dramatic notion of what human-computer interaction is all about, I could not be more pleased by developments to this point. Our experiences with VR to this point also substantiate the idea that human interactions with virtual environments are enhanced by dramatic form and structure and complex emotional textures.

As we turn toward what is new, I am encouraged by the rising importance of the social, cultural and artistic aspects of human-computer interaction.

POST-HYPE, NEW BRANCHES OF INQUIRY

New branches of inquiry, like diverging and reconverging waters, have grown out of the idea of virtual reality.

For example, VR has reinvigorated and recontextualized the study of human sensation and perception. Game designers know in their gut that good audio makes people think they are looking at higher resolution pictures, but the converse is not true. While much is known about the human visual or auditory or tactile senses, relatively little is known “scientifically” about how these senses combine.

Interacting with body and mind. The study of sensory combinatorics — how vision affects hearing, for instance, or how the two in concert affect emotion — was almost exclusively the province of the arts until VR came on the scene. I find it cause for celebration when science in the arts can do something more robust than measure the galvanic skin responses of audience members during the last scene of Hamlet. The premise that we may interact with technology through our bodies and our minds has given us occasion to reconstruct our understanding of ourselves.

Similarly, VR has heightened interest in the notions of technologically mediated and shared presence. Before VR became an object of cultural interest, the focus of work in the domain of virtual presence was on productivity, as manifest in teleoperations, teleconferencing and collaborative work.

Shifting to social. As the notion of VR recombines with technologies such as the telephone and computer networking, we are forced to shift our gaze from the economic to the social and cultural dimensions of group activity. This necessitates a broadening of our concept of what our technology is for and how it is related to a much wider domain of human intercourse.

FACILITATING THE MOVE FROM STRUCTURE TO RELATIONSHIP

As I have watched people participate in both multisensory and textual shared environments, my view of structure in dramatic interaction has changed. As an activity becomes

less “artifactual” (like painting or literature) and more ephemeral (like conversation or dancing), sensory immediacy and the prosody of experience gain primacy over structural elegance in the realtime stream of events.

In shared virtual worlds, structural elegance becomes much less about the progression of events and more concerned with facilitating the emergence of patterns and relationships. Shared presence and sensory immersion evoke constructive activities on the part of human participants to a much greater degree — and in more structurally interesting ways — than I had imagined. Perhaps because it is so disjunctive, this turn of mind encourages me that we are beginning to witness the emergence of a new paradigm.

Joining technology and culture. In a way, Medieval troubadours and jongleurs were the cultural lightning rods of their day. They performed the entire range of human archetypal characters and situations, charged with mythological energy and theatrical savoir faire, and the rawness and openness of their performances made them privy to the deep, unspoken energies that rippled through their audiences.

It was a dialogue, albeit asymmetrical, which kept certain cultural fires burning. Some say that they were also the wandering keepers of secrets, and that in their representations lay the officially suppressed arcana of archaic knowledge and wisdom. I doubt that many of them saw themselves as omniscient priests, but rather as the figures that Spoonman (a.k.a. Mark Petrakis, storyteller and hacker) calls “Killer Clowns.” They tricked you and made you laugh, even as they told you the truth.

THE BENEFITS OF BEING ON THE VR CIRCUIT

One benefit of being on “the VR circuit” was that it put me in contact with people all over the world as they pondered this new cultural icon. The aspect of VR that I find most fascinating is how it has been received and worked over by the cultures with whom I have discussed it.

From its strange childhood in military and government labs, VR emerged as a Major Concept in the pop-culture scene. It was hailed as the techno-wave of the future, with potential to transform everything from movies to medical imaging. It was also demonized as the latest in mind-control drugs and the world’s baddest war machine.

Renewing the reality debate. Philosophers adopted it as a platform for renewed debates about the nature of reality, the evolution of global culture, and the relationship of technology to the body and the physical world. Nearly everyone agreed that a head-mounted display would give you a look inside Pandora’s black box. The mythos of VR continues to be a key ingredient in the pop-culture view of how the world is changing — a many-faceted icon for the coming weird times. Why?

In the book Through the Vanishing Point, Marshall McLuhan mused about how new technologies change our consciousness:

Anything that raises the environment to high intensity, whether it be a storm in nature or violent change resulting from new technology, turns the environment into an object of attention. When it becomes an object of attention, it assumes the character of an antienvironment or an art object. [McLuhan and Parker, 1968]

Such antienvironments, McLuhan believed, “open the door of perception to people otherwise numbed in a nonperceivable situation.” Shakespeare was barking up the same tree when he said that drama “holds the mirror up to nature.” Media represent us to ourselves in a multidimensional way — beyond the content of any particular representation, the characteristics of the medium itself give us insight into the invisible cultural context. Whoever discovered water, as the saying goes, it certainly wasn’t a fish.

TRANSFORMING THE NOTION OF CONTROL

If McLuhan was right about antienvironments, the media-making impulse may be a built-in species-level survival mechanism. It this sense, VR manifests humanity’s need to encounter and transform the notion of control.

No matter how you look at VR, the control issue is center stage. The public and the press are worried about mind control — is VR addictive? Can it be used for brainwashing? Can “special interests” — from secret police to commercial advertisers — alter our beliefs and desires with hypnotic potency? Will it be used as a way to deny and circumvent the blood-and-guts realities of war? Will it replace condoms and cosmetic surgery? Will it replace the real live lovers with electronic sex?

Good reason to fear. What we fear is the loss of control — over our minds, our society, our government, our bodies and our sexuality. And with good reason.

VR functions as an antienvironment that boosts our awareness of conditions that already exist in our culture, but to which we have become, if not completely numb, at least increasingly resigned and mute. VR and its progeny may ultimately function to demonstrate that a hierarchical notion of control is a toxic philosophy in the contemporary world — not only in terms of culture and art, but also in terms of our relationships with individuals, societies and environments, and especially in terms of how we define and measure our own freedom and self-esteem.

Changing the notion of leader. There is an interesting parallel here, as researcher Rob Tow pointed out to me, between our understanding of control in a human culture and the way that scientists puzzle over the notion of control in the flocking activity of birds or schools of fish.

In the nineteenth century, a good deal of effort went into figuring out which animal was the “leader” and how that individual managed to orchestrate the behavior of the group. Some sociologists and political scientists are still more comfortable propounding theories that are based on utterly unexplained means of communication, or on “historical forces” to accommodate a command-and-control paradigm, than they are to explore control as an emergent phenomenon in group behavior.

In the late twentieth century, new metaphors of communication and control have emerged from biology and cybernetics and are making their way into sociology, politics and the general culture, just as the Von Neumann architecture is being supplanted by massively parallel computers. Networked virtual communities offer tantalizing evidence of emergent social organizations with novel topologies, lacking explicit leadership.

As a result, we are forced to abandon the notion that there is an “author” and a “user” and to replace it with an idea of collaboration, where control of the shape of the experience is shared. Form and structure emerge. As long as designers continue to see themselves as authors of one-to-many experiences, all of us will only be bottom-feeding on the fringes of fundamentally non-interactive forms.

SUBCULTURE, GIVING RISE TO NEW CULTURE

The VR subculture has begun to blend at the edges with those who are transforming music, video and computers into a context for interactivity in communal and orgiastic dimensions — what was known in the late ’80s and early ’90s as “the rave scene.”

The same tribal impulses, also predicted by McLuhan, show up in the burgeoning diversity of the Internet world and in the trends toward personalization and sensory immediacy in multimedia. These movements are driving the evolution of technology and culture with a passion that begins to tap not only the intellectual, but also the sexual and spiritual energies of a generation.

They are the early manifestations of a huge new wave of narrative vigor and joyous cultural polyphony — the double of political fragmentation and ethnic strife that currently characterize the geopolitical dimension of human culture. They flow like water around rocks on a hillside, dividing and rejoining — rushing, we sense, toward a grand convergence hidden in the canyon.

The purpose of passion. In 1991, Jean-Louis Gassée wrote:

“We humans are in love with our tools because they help us become more than we are, to overcome our limitations and extend the boundaries of what is possible to do with our brains and bodies.”

I think it’s significant that Gassée used the word “love.” It is not an “objective” word. Spearheaded by a techno-subculture, virtual reality became a meme that has had an enormously emotional impact on the collective imagination. Our relationship to the idea has been passionate, as expressed in both our hopes and our fears.

In a particular way, our passionate response to virtual reality mirrors the nature of medium itself: by inviting the body and the senses into our dance with our tools, it has extended the landscape of interaction to new topologies of pleasure, emotion and passion.

A similar transformation occurred in the Middle Ages, when theater exploded out of the textual universe of the monastery into the sensory fecundity that gave rise to Commedia dell’Arte. In the same historical period the monolithic Christian content of the drama was bathed in a wave of sensory, passionate and archetypal imagery.

It was this coming together of text, body and narrative polyphony that opened the way for Shakespeare, Grand Opera and all the vital permutations of the dramatic impulse that have come down to our day.

THE CONVERGENCE OF CULTURE AND TECHNOLOGY

The idea of virtual reality is the manifestation of a similar kind of convergence between deeper cultural impulses and the “textual” medium of computers. I think that we respond to it so passionately because we sense the magnitude of this transformation at the level of human culture and evolution.

Technology is not exclusively about rationality; content is not exclusively about information. Many of us sense the inadequacy of the paradigm that has led us to our current notions of technology, nature and consciousness. We see relationships between that paradigm and the degradation of our cultures, our habitat and our individual lives.

Terence McKenna has characterized humanity as the species that extrudes technology. What now shall we do with it? VR is a context for exploring how to make ourselves whole within this frame. It is a context in which we encounter technology with passion. The primary concern of VR is not constructing a better illusion of the world; it is learning to think about the world better. As we ponder our collective evolution, we see that passion is the prosody of intelligence.

Brenda Laurel