RealityMedia

May 2020 - Present
Virtual Reality (VR) is emerging as new media of communication and expression, but the conventions for this new media have yet to come. As a graduate research assistant, I contributed to creating a web-based immersive space called 'RealityMedia' and explored how VR affords existing and new forms of expression.
Research Questions
  • Which mediums inspire and foster discursive formats in VR, and how can these be better exploited?
  • What are the new discursive techniques for VR?
  • What are the opportunities and challenges in VR?
My Contributions
Creating and designing 3D and 2D contents for the immersive spaces
‍
Tools
Maya
Blender
html/css
JavaScript
Mozilla Hubs
Click to view the webXR project

Advisors: Jay Bolter, Blair MacIntyre, Maria Engberg
Collaborators: 5+
Creating spatial narrative in VR
In considering a VR space as affording a new kind of writing space, we see that, as opposed to the linear space of a printed book, the arguments and narratives are embodied in a physical space like a museum on three distinct levels: (1) the architecture of the space itself, (2) the collection, and (3) the individual artifacts. The levels can present multiple threads. From this perspective, the problem becomes how to organize and present elements at each level in order to facilitate the “reading” of the various threads by the user.
‍
Architecture as Discursive Space
In the VR environment of RealityMedia, architecture serves as a medium for spatial discourse.
‍
Rotunda
The central rotunda embodies the whole argument of the book and suggests how the various galleries are connected in a circle of key ideas.
Gallery as Spatialized narrative
RealityMedia also consists of a series of “galleries,” many of which clearly resemble the rooms of a conventional museum. Each gallery delivers an overarching theme of all the individual artifacts and content in the space.
‍
Presence and Aura Gallery
Each of the three rooms in this gallery conveys a particular theme; discussions on what presence is, how presence manifests, and aura.
individual Artifact as narratives
Individual exhibits use typical conventions to organize and present their verbal ideas in the 2D space of a printed book: e.g. paragraphs, pagination, headings, chapters, and an index and reference list. In addition, RealityMedia also makes use of a set of expressive media forms possible only in the immersive medium of VR:
  • the meaningful arrangement of 2D content in the 3D architecture of each room
  • panoramic spheres and complete immersive experiences that are
    “embedded” in galleries but visited as independent world spaces
  • 3D visualizations of the ideas contained in the printed book in the
    form of “force graphs”
  • teleporting portals that connect stories by providing a transition
    between galleries
Design Considerations and Optimization
Designing 3D assets for webXR can be challenging because, unlike VR applications, webXR runs on a web browser and is limited by the performance of the user's device and network connection.
Content creation
For the 3D assets to be lightweight, we created low poly assets with baked textures and lighting, which reduces the amount of data that needs to be loaded and processed by the user's device, and also compression techniques to reduce the size of the assets while preserving their quality.

To manage the 2D content efficiently, we utilized HTML pages in the gallery instead of presenting information with image files. Furthermore, by using CSS and javascript, we were able to enhance the user experience by making the content interactive.
Design issues
WebXR is cross-platform - This means that the experience needs to be optimized for every possible devices. To achieve this, we constantly conducted internal tests with various devices, including the Meta Quest2 headset, to test the functionality and aesthetics of the experience and identified key design issues for VR headset users:
  • Keeping the information at the level of the user's eyesight is more critical for headset users. To address this, we adjusted the information in a more natural location that is easily readable for the user.
  • Buttons and hyperlinks are also more optimized for browser-based webXR experiences, not for VR headsets. This means that headset users need more precision when clicking, so we increased the size of the buttons and provided visual cues to indicate where the buttons are located.
  • User disorientation. To address this, we provided clear visual cues and maps, to help the user orient themselves in the virtual environment.
For design iteration, we use the trial and error method to identify issues and optimize the webXR environment for a better user experience. One approach we could take in the future is to conduct user testing, which would allow us to gather valuable feedback from users on how the webXR environment is functioning and how it can be improved.
Next Steps
The RealityMedia galleries constitute a first attempt at deploying a 3D space to communicate a discursive argument. In this prototype, we have drawn on the structure and layout of the traditional museum, because one of the roles of many art, history and science museums has been to convey ideas or tell a story in space. We knew that we could borrow the narrative techniques of the museum (e.g. wall texts, photos, videos) because most visitors to RealityMedia will already be familiar with these practices. However, a VR environment is not bound by the physical limitations of a museum.

In future versions of RealityMedia, we want to create "impossible" spaces that better reflect the shape of the argument. In particular, the use of portals allows us to define passages from one gallery to another that are not possible in the Euclidean space of our world. We also aim to investigate the roles textual information can play in VR beyond the current museum model.
JP 2023