Virtual reality (VR) is a technology that provides perceptually rich, multisensory simulations that surround users and respond to naturalistic body movements. VR has been studied by cognitive scientists in three main areas. First, VR has been used as a tool to study basic cognitive processes, for example, creating experimental stimuli using rich, realistic scenes in which people can interact naturally while still maintaining rigid experimental control. Second, VR has been studied as a medium itself; scholars have developed theories based on the medium and have researched the affordances of the technology as its own intellectual endeavor. Finally, the medium has a rich history of applications related to cognitive science, including mental health and training.

History

The emergence of VR as a medium is most associated with the work of Ivan Sutherland and Jaron Lanier (for a thorough discussion of the history of the technology, see Lanier, 2017). Although Lanier is most known as a technologist, his development of VR hardware and software was accompanied by theoretical work central to cognitive science. For example, he developed the concept of homuncular flexibility, the ability to transform one’s body by remapping biological movements of nonhuman avatars with novel limbs and other appendages [see Bodily Sensations]. However, the direct use of the medium in psychological research is most associated with the work of Jack Loomis and colleagues at the University of California, Santa Barbara in the mid 1990s (Loomis et al., 1999), which provided early theories and empirical work on VR’s use to study perception, cognition, and social interaction.

Core concepts

There are three main areas of research within cognitive science. Although there is some overlap among them, in general, scholars tend to primarily focus on one of the three.

VR as a tool to study the mind

An early paper outlined the advantages of using VR, referred to as “immersive virtual environment technology”, as a tool to study basic psychological processes (Loomis et al., 1999). The same way computers drastically changed psychological study by allowing researchers to easily present text, images, and movies in controlled sequences and measure outcomes such as reaction time, it was argued VR could similarly transform the field. Although computers give great amounts of experimental control, seeing images flashed on a screen while clicking buttons is not how people interact in the real world, and VR can provide the best of both worlds, creating realistic experiences for participants that still allow precise control by researchers. An extension of this work into social interaction (see Blascovich et al., 2002), led to the Threshold Model of Social Influence, which is a theoretical model generating research hypotheses and interpreting outcomes of studies that feature some form of social interaction [see Proxemics]. Since those papers were initially published, there have been hundreds of studies using VR as a tool to study memory, perception, decision-making, attitude change, nonverbal synchrony, and a host of other cognitive and social processes.

Studying the medium itself

VR is so drastically different from other media that many scholars have dedicated careers to creating theories and collecting empirical data about the medium itself. Early scholars explicated and studied the concept of presence (Biocca & Levy, 1995; Slater & Wilbur, 1997). Although there are many instantiations and definitions of presence, the basic concept is “the illusion of nonmediation.” When people use VR, the medium tends to disappear, and people behave in a similar manner to how they behave in the real world. There have been thousands of studies of VR as a medium, including several meta-analyses, for example, linking the immersive technological features (i.e., field of view, resolution, amount of body tracking) to the psychological outcome of presence (Cummings & Bailenson, 2015). Presence can be applied to the virtual environment itself as well as to social actors within the environment (Lee, 2004).

Applications of VR

VR has long been used in the real world in various cognitive and more general psychological applications. For example, after the 9/11 attacks in the United States, psychologists reconstructed the horrible events such as first-person views of planes hitting the towers in order to help first responders to overcome the trauma they experienced (Difede & Hoffman, 2002). Clinicians have also used the medium to implement cognitive exposure therapy, allowing therapists to safely guide patients to immersive scenes that simulate trauma (for a recent review, see Rizzo et al., 2024). Outside of clinical applications, one of the largest uses of VR has been in training scenarios, for example, leveraging the embodied aspect of the medium to improve learning transfer (Makranksy et al., 2019). In general, VR’s most compelling use cases often involve training, where learners can safely make mistakes during simulations that would be dangerous, impossible, counterproductive, or expensive (i.e., DICE) in the real world (Bailenson & Lesher, 2024).

Questions, controversies, and new developments

As VR migrates from academic laboratories into homes and classrooms around the world, researchers need to study the effect this medium will have on society. Recent technological implementations of mixed reality (sometimes described as augmented reality), where users can see real-time video of the actual world while simultaneously viewing immersive content, now cost less than a smartphone. Technology companies are hoping these new immersive headsets will become the new interface for computing and are investing billions of dollars, hoping the devices become pervasive. However, as yet there are limited studies from the research community of the cognitive and other psychological effects of VR use. More scholarship—in particular, studies with large and diverse samples—is needed (Peck et al., 2021).

Broader connections

Many VR researchers take inspiration from the vast literature on perceptual illusions. After all, VR tricks the mind into believing one has traveled to a new location or has swapped bodies into an avatar. For those looking to learn more about VR, understanding how the brain processes illusions is a great way to start.

Virtual Reality