What is it like to be a Virtual Bat?
VR and Perception in the Eyes of the Animal
This essay talks about VR and perception. It is the second in a monthly series on topics such as VR, AI, Theatre, NFTs and Games. You can read the first essay discussing the idea of Imperfect VR here on Medium.
What is it Like to be a Bat?
Almost 50 years ago, the philosopher Thomas Nagel pondered the question: what is it like to be a bat? He then went on to provide a rather unsatisfying answer: We — in principle — cannot know. The essay argued that it is impossible to re-experience the experience of other creatures. A bat, for example, primarily perceives the world through echoes received by its highly sensitive sonar. Not only its sensory apparatus but also its whole brain is radically different from ours, and so is the inner life of a bat.
I called Nagel’s answer ‘rather unsatisfactory’ because I contend that we as humans are constantly occupied with what is going on in other people’s heads. Psychologists call this phenomenon “theory of mind” (TOM), although more so than forming theories we hypothesise, or make guesses, about the thoughts, motives and goals of others. And all too often our guesses are wrong because,in principle,it is impossible for us to know. — Despite the fact some cognitive scientists seem to suggest that the electrodes that record neural activity and the images that show blood flow inside the brain,will somehow be able to tell us the whole story, at some point in the future. Yet a map is not the territory (Alfred Korzybski). Therefore, all we have,besides our own first-hand account of what it is like to be a human, is communication. And communication does not take place between minds or brains but within social systems (more on that topic in a future essay).
Although we can not fully claim to understand either other humans or bats, what remains is a substantial difference between the levels of human-human and human-bat understanding (I leave open the question of how bats might understand humans). This gap is due to what Humberto Maturana calls “structural determinism”. While we can’t grasp or explain the mind or consciousness, despite some attempts to the opposite, the biological structures that can be observed as underlying these phenomena, still determine certain boundaries to these territories.
Take the Red-eyed Tree Frog (Agalychnis callidryas), whose eye is covered during sleep with a translucent membrane to help it detect approaching predators. Are we able to fully grasp or simulate the experience of seeing through such an eye?
Nagel’s scepticism has not discouraged artists, scientists and philosopher from imagining the sensory experiences of ‘the other’. Human echolocation and synesthesia are widely explored phenomena in art and cognitive neuroscience. Sensory implants have enabled people to substitute vision with tactile and auditory sensations. The growing cyborg movement, now represented by a formally registered society located in Berlin, is longing for new senses and sense prosthetics. Some artists are creating mixed reality experiences around phenomena like psychosis (Labyrinth Psychotica by Jennifer Kanary Nikolov(a)), empathy (The Machine to be Another by BeAnotherLab), and blindness (Notes on Blindness: Into Darkness by Arnaud Colinart). Then, there are audio and video games that play with sense modalities like Papa Sangre (Somethin’ Else), BlindSide (Aaron Rasmussen, Michael T. Astolfi) or Perception (The Deep End Games).
A Walk in the Forest
Imagine you are walking up a small path through a scenic forest, a few minutes away from where you parked your car, almost an hour from Lancaster, right in the Lake District of North West England. Some unobtrusive objects guide you along the way. After a few minutes, you arrive at a small meadow, surrounded by the trees and bushes of Grizedale forest. You see several tree stumps, inhabited by strange creatures. From the shoulders down, they look human, but their heads appear strangely different. Their faces are covered with mossy spheres that appear to be helmets, cables dangling from their backs. Tacitly you come closer. One of the helmets is empty, waiting for you. A strange attraction emanates from the device, drawing you towards it. Hesitantly you put it over your head and enter…
In her seminal book Hamlet on the Holodeck, Janet Murray characterized immersion as “…the sensation of being surrounded by a completely other reality, as different as water is from air that takes over all our attention, our whole perceptual apparatus.” Marshmallow Laser Feast created In the Eyes of the Animal, in Virtual Reality, as a vehicle for this kind of immersion that lets us deeply connect to the ‘other’. They deploy VR to let us experience different modes of perception of various animal inhabitants of the forest. While the modalities of the individual animal’s perception are based on solid research, the result is shaped by artistic interpretation. In the Eyes of the Animal, as well as their other projects, such as Observations on Being and Distortions in Spacetime, are playful virtual explorations of the world beyond our senses.
Immersed in Point Clouds
Whereas other designers such as Sólfar Studios, developers of Everest VR, are pushing towards realism, MLF has chosen a different aesthetic route. To visualize the forest, they utilize LIDAR, the technology that is used in self-driving cars to scan the environment in real-time. The results are point clouds consisting of almost a billion individual particles. This massive amount of data must be drastically reduced before it can be imported and processed within the visual interactive environment vvvv. To achieve this, the particles are run through highly optimized shader code that generates the unique aesthetics of the artwork. Finally, these visuals are sent to their custom-made helmets that contain Oculus Rift headsets and headphones. A binaural soundtrack that runs in Max/MSP can be felt through SubPac tactile bass wearables, which adds to the immersive quality of the piece.
The natural setting for In the Eyes of the Animal is the forest and it is best experienced in this environment. For practical purposes, the piece has been travelling to places such as Sundance, Playgrounds Festival, Toronto Film Festival, Yamaguchi Center and Sonar.
Virtual Storytelling
If we peek beyond the current technology-driven consumer hype, there is a huge creative space opened up by VR, that asks to be explored through artistic and scientific experiments, games and environments. Examples of these kinds of explorations include Disconnected by Thorsten Wiedemann and Sarah Lisa Vogl in which Thorsten stayed 48 hours in VR, VR Rollercoaster where the physical sensation of a rollercoaster ride is combined with virtual experiences and the creepy cooperative game Taphobos that invites its players to crawl into coffins.
The medium poses new challenges as well as opportunities for interactive storytelling. VR is unique in its physiological and cognitive affordances and its currently growing but still somewhat limited capacity to convey embodied experiences and to create virtual social spaces.
Half-Real
With its immersive qualities and three-dimensional interaction, Virtual Reality has the potential to replace the smallish, flat screens that are now our predominant interface. Games provide important pathways towards this future, as powerful experiential vehicles that allow us to access new identities, take up different roles, and rehearse unfamiliar views and perspectives. The game studies scholar Jesper Juul describes games as a hybrid form that combines real rules with fictional worlds. They constitute a medium through which we can experience possible worlds through play.
In the end, it is on us, the creators and inhabitants of these worlds, to choose, and shape the realities we wish to experience. The visions of large corporations keen on selling us their latest entertainment content, or the creative, quirky, touching, and open experiences that offer us access to a deeper meaning of what it is to be human, even through the eyes of an animal.
Note
This is an updated and extended version of the essay published in A MAZE Magazine issue 4 in August 2016, alongside an interview with Barney Steel and Ersin Han Ersin of Marshmallow Laser Feast. I mixed in some of the ideas of my presentation “How to Perceive the Virtual Image? On the Distinction Between Virtual and Real”, delivered at the Transimage conference at the University of Plymouth, UK, also in 2016.
