Cognitive and Emotional Processing of Virtual Environments: The Role of Attention and Mental Models

Abstract

Major innovations in consumer electronics are at our doorsteps bringing VR technology into our living rooms. The state of presence or the illusion of non-mediation is at the core of the virtual experiences this technology provides and possesses important implications for the way we interact with virtual environments and for the things, we take away from this experience. Despite the large body of empirical research on the topic, the underlying cognitive processes have been addressed in only few publications. Thus, the goal of the dissertation project is to build upon findings from psychology and cognitive neuroscience to explain how non-mediation can emerge, when the user is presented with a virtual environment. As a conceptual framework, the processing of virtual stimuli is framed as an interaction between focused attention and fundamental cognitive capacities. I argue that non-mediation emerges effortlessly already at the perceptual level. The subsequent process of constructing a mental model of the virtual environment enables the user to interact effectively and is responsible for transfer effects to physical environments, where these models might prove useful. Non-mediation is maintained as long as the user is not distracted and new information from the virtual environments is consistent with the mental model. In the case of strong violations, however, a break in presence emerges that is conceptualized as an orienting response, through which attention is allocated to stimuli of the physical environment.

In my dissertation project, I developed a conceptual framework of presence based on the functional organization of the human cognitive system. Empirical evidence of the involved cognitive capacities at work and a reliable measure of breaks in presence are reported. The framework is also applied to the domain of natural user interfaces to explain how users adapt to input devices with varying degrees of natural mapping with the help of mental interaction models. Implications for the use of naturally mapped controllers for entertainment and for motor skill training in virtual environments are investigated. Furthermore, future research directions of the outlined research program are identified.