Immersive Space to Think
Immersive Space to Think is a design approach for sensemaking of non-quantitative datasets in Augmented or Virtual Reality (AR/VR). In our approach, users can move and orient artifacts in 3D immersive space in order to organize their thoughts. It supports offloading cognition onto the environment through various methods, such as highlighting key phrases or sentences in text documents, annotating artifacts through attached notes, or creating labels to categorize other artifacts.
Studies
Early research into the approach involved looking into how novice users interact with data artifacts in 3D space. For this study, participants worked with survey responses to Survey #32; this survey was given to enlisted members of the US Army in 1942 pertaining to racial relations and whether they should integrate the military. Participants organized their thoughts and gave an outline for an essay they’d write to answer a given prompt. Our focus was to find patterns of interaction as well as if participants could effectively use 3D space to perform a sensemaking task.
The results were published at IEEE VR ’21 and earned a best paper honorable mention. The paper and results can be seen here, and the presentation can be seen below.
Lisle, L., Davidson, K., Gitre, E. J. K., North, C. & Bowman, D.A. (2021, March). Sensemaking Strategies with the Immersive Space to Think. In 2021 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE.
We further investigated what level of mixed reality best suits IST. VR affords a private experience that cuts off the real world, thereby removing distractions for users. This could increase concentration and cognitive performance. On the other hand, AR would afford the use of real-world tools and quick context switches to real world needs, such as a quick co-worker interaction. A mixed methods comparative study was performed using a Varjo XR-3 for both the VR and pass-through AR implementations. This allowed a more direct-comparison between the two levels of mixed reality as field of view, display fidelity, and ergonomics are kept consistent. We found that a blend of the two, or augmented virtuality, was the best solution. This work was published in Frontiers for Virtual Reality and can be seen here. The abstract is below:
Analysts perform sensemaking on large complex multimedia datasets in order to extract concepts, themes, and other kinds of insights from them. Immersive analytics, in particular, puts users in virtual environments that allow them to explore data in a unique way where they can interact and move through the data. Previous research using virtual reality immersive analytics tools found users wanting to refer to real-world objects or understand the physical world around them while continuing to perform their analysis. Therefore, we designed and ran a comparative study looking at the tradeoffs between virtual and augmented reality for our immersive analytics approach: Immersive Space to Think. Through two mixed-methods studies we found that virtual reality affords users a space where users can focus more on their task, but augmented reality allows them to use various real-world tools that can increase user satisfaction. In future immersive analytics tools, we recommend a blend of the two – augmented virtuality – with pass-through portals which allow users to see various real-world tools, such as whiteboards or desks and keyboards, while still giving themselves a space to focus.
The next step was to understand how effective IST is when compared to traditional 2D displays as well as the original Space to Think concept by Andrews et al. We conducted a 3×1 within-subjects study using the Varjo XR-3 and virtual monitors for the two 2D conditions where participants performed 3 separate sensemaking tasks – one with each condition.
Analysts need to process large amounts of data in order to extract concepts, themes, and plans of action based upon their findings. Different display technologies offer varying levels of space and interaction methods that change the way users can process data using them. In a comparative study, we investigated how the use of single traditional monitor, a large, high-resolution two-dimensional monitor, and immersive three-dimensional space using the Immersive Space to Think approach impact the sensemaking process. We found that user satisfaction grows and frustration decreases as available space increases. We observed specific strategies users employ in the various conditions to assist with the processing of datasets. We also found an increased usage of spatial memory as space increased, which increases performance in artifact position recall tasks. In future systems supporting sensemaking, we recommend using display technologies that provide users with large amounts of space to organize information and analysis artifacts.
The paper can be accessed here. An online version of the presentation can be seen below.