Virtual Reality as an experiential medium
By Assim Kalouaz
Virtual reality (VR) has found its way into an increasing number of households since its democratization in 2013, with the release of the Oculus Rift and the HTC Vive. With global lockdowns of 2020 and 2021, and the release of more affordable standalone headsets (VR headsets that require no high-end PC) such as the Oculus Quest, even more people have gotten into VR. VR has the ability to transport its users into immersive environments by fostering a sense of presence (Slater, 2009).
Presence is defined by Professor Mel Slater from University College London by a combination of:
Immersion (“a property of the valid actions that are possible within the system”) occurs through a direct effect of the user’s actions taken in the physical world, on the virtual world. For instance, crouching will give a perspective from a lower viewpoint and moving your head will change your field of view along with the perceived spatialized audio. This is possible thanks to the positional and rotational tracking of the VR headset.
Place Illusion referring to the sense of “being there” in the virtual world. This is fostered through a virtual environment that can effectively simulate the sensation of physical reality. You could think of this as the range of immersion-fostering actions you can take in the physical world that would be as effective in the virtual world (getting on the tip of your toes to reach something high, ducking to dodge an incoming projectile etc…)
And Plausibility Illusion which is the feeling that what is happening is really happening, which depends heavily on interactivity, suspension of disbelief and technological reliability (no latency, good tracking etc)
Now in virtual environment development, presence is what creatives, 3D artists, developers and storytellers strive to achieve in order to engage the user at a high level and allow them to live and experience lifelike events they could not experience otherwise. The only limitation of this medium, besides the technology itself, is the mere imagination of content creators; that is, the potential experiences that one can live in VR are endless.
Sense of Presence: Storytelling in Virtual Reality | News | Unreal Engine
While a lot of the content of current VR is closer to mainstream games some experiences are truly unique through their story or mechanics. They may be purely contemplative or interactive. These experiences can defy the user’s schemas and mental constructions of what is possible (extremely well explained here). Such transcending experiences include intimate connections with strangers by listening to the profoundly personal story they shared and sharing your own, casting magic spells using your (real) hands, or experiencing what it is like to form mental images from sounds as a blind person, to wander in space as an astronaut or a to climb mountains and reach for the highest peaks.
In order to achieve these transcending experiences, content creators have inherited pre-existing experiential design techniques to capture attention and move emotion of audiences, playing around with the sensory features of their environments. Visual realism and style or music and sound design are already defined and refined design techniques from mainstream media such as movies or theatre plays. More conceptual techniques to foster immersion exist in other forms of media such as literary fiction. VR can combine all of them to bring something new to the table with interactivity into plausible and responsive virtual settings. This grants agency to audiences and allows them to become actors of the story they unravel.
As a student, I worked on an exciting project (see below) that illustrates how users can become actors of a story by inserting diegetic interactions (interactions that make sense in their context e.g. teleportation in a virtual makes more sense if you embody an eerie sprite rather than if you were playing a soldier). Such interactions where actions you take in the real world affect the virtual world, giving you access to more perceptual information, are referred to as sensorimotor contingency, which is what fosters a high sense of presence best. On a similar note, this experience demonstrates how users can become protagonists of a virtual character’s story through engaging social interactions with the virtual agent.
La Plume et la Lanterne (Teaser)
I personally think that the best experiences are the ones that challenge us to consider what is possible and give us agency to perform or witness actions that challenge us to accommodate new mental schemas, which is one of the main contributing factors of Awe (Chirico & Gaggioli, 2018). LiminalVR is a company based in Melbourne that uses VR technologies to create emotion and mood-inducing experiences such as calm, energy and awe with which the Media and Entertainment Psychology Lab has recently started collaborating.
With VR being an incredibly rich medium in terms of the fields it connects, we want to drive collaborative research projects of different nature and size with experts from complementary fields to understand better the intricacies of experiential design in VR. The medium is still very far from its best potential and each of these collaborations will make an important contribution to the field, denting into the bubble of existing knowledge and expanding it, while raising new exciting research questions and opening doors for more exciting projects.
Chirico, A., & Gaggioli, A. (2018). Awe: “More than a feeling.” Humanistic Psychologist, 46(3), 274–280. https://doi.org/10.1037/hum0000098
Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1535), 3549–3557. https://doi.org/10.1098/rstb.2009.0138