A Step Toward The OASIS (3/5): Experience
What are the requirements for the OASIS user experience? The last two posts explored how 1) the OASIS is not a distant future, but an imminent reality and 2) “open-source reality” platforms such as YouTube in which multiple users can create and share their own virtual reality contents have a better chance in making the OASIS mainstream than the platforms that only offer exclusive contents. In this third post of the series, we will discuss what elements of user experience specific to the OASIS differentiate it from other online experiences.
The public interest in ‘metaverse’ has exploded during the past 3 months since the publication of the first post ((The first original article was released on Oct 2020 and this article was posted on Feb 2021 in Korea). Metaverse became a popular topic in multiple conferences around the world at the end of last year. At the beginning of 2021, it was a critical theme at the online CES and Venture Beat (VB) recently held a conference titled, "Into the Metaverse". Several books have been published on the topic. South Korean futurologists who have been reporting on news from the Silicon Valley and future IT trends began to focus on metaverse. While I am truly amazed by the public zeal for metaverse, I am also concerned because too many talks could be misleading.
The title of the series, “A Step Toward The OASIS”, implies that the OASIS is only one step away from becoming reality. The series aims to show how we can anticipate the future of metaverse by looking at current platforms such as ‘Fortnite’, ‘Roblox’, ‘Minecraft’, and ‘Zepeto’. On the other hand, the title also suggests that metaverse is yet-to-be completed. ‘A step’ is still missing for metaverse to enter the mainstream.
When I watched Fortnite’s Travis Scott performance on my PC, I saw a glimpse of future for metaverse. I thought, “attending such performance in VR will be possible in the near future.” I was hesitant to claim that the future of metaverse has arrived. Interestingly, the ‘near future’ became a reality sooner than I had expected when I attended the ‘Lost Horizon Festival’ by Sansar and “All Kinds of Limbo” volumetric video concert. Metaverse or OASIS experience goes beyond viewing impressive images on a 2D screen in real-time. Metaverse experience begins once I, the user/player, can feel the realistic ‘presence’ (the word that has become too prevalent) in a virtual world.
The keys to the success of the OASIS were the two new pieces of interface hardware that GSS had created, both of which were required to access the simulation: the OASIS visor and haptic gloves…. The wireless one-size-fits-all OASIS visor was slightly larger than a pair of sunglasses. It used harmless low-powered lasers to draw the stunningly real environment of the OASIS right onto its wearer’s retinas, completely immersing their entire field of vision in the online world….as did the lightweight OASIS haptic gloves, which allowed users to directly control the hands of their avatar and to interact with their simulated environment as if they were actually inside it. When you picked up objects, opened doors, or operated vehicles, the haptic gloves made you feel these non-existent objects and surfaces as if they were really right there in front of you.- READY PLAYER ONE
The OASIS Experience is a 6-Degree of Freedom Experience
Although I am hesitant to start another discussion on devices, I have to mention that what made OASIS successful was its unique ‘interface devices’ that differentiated the OASIS experience from other online experiences. Such particular experience specific to OASIS refers to the experience of 6-Degree of Freedom (6 DoF).
A 6 DoF experience expands the viewer’s visual field from a flat screen and allows the viewer to enter into the depth of the screen. Once the depth experience becomes possible, the viewer’s experience of the space transforms into a 3-dimensional one. The boundary between offline and online and that between real and mediated experience disappear. A 6 DoF experience enables the viewer to experience the online mediascape the same way we experience the real physical world.
I believe that the 6 DoF experience is as revolutionary as revolution could ever be. I predict that it will revolutionize the way we perceive experience. Humans have developed countless visual techniques to emulate ‘depth’ experience because there had been no technology to recreate depth and direction simultaneously. The development of perspective is the most representative example in history. The use of perspective in art, photography and cinema has trained humans to experience 3D spaces on 2D screens. (Yes, we had to learn to translate 2D images into 3D experiences)
However, with the new technology that can now record and recreate depth information, we no longer have to transfer 2D information into 3D. Humans are born into a 3D world and can instinctively adjust to 3D environments. Reacting to any given 3D environment does not require much training. Even when the given environment is visually different from the real world, we can easily accept the visual discrepancies and intuitively react to the space. There have been multiple research and experiments to prove that humans generally show such instinctive reaction in virtually created 3D environments. (recommended: “Experience on Demand” (2018) written by Jeremy Bailenson)
At the 2019 ‘Next Content Conference’ hosted by KOCCA (Korea Creative Content Agency), Tipatat Chennavasin, the founder of the Venture Reality Fund, declared, “Only 6 DoF experience is VR”. The audience was surprised by his remark. The debate on the importance of the 6 DoF has been ongoing in both national and international XR industry. Some have not understood its necessity in VR. The most well-known controversy is whether a 360 immersive video is VR or not. (Tipatat Chennavasin’s declaration was a comment on this issue) 360 videos cannot provide 6 DoF experience. They can immerse the viewer into the images on multiple screens, but immersive experience is not the same as the 6 DoF experience. ‘Projection mapping’ technology is becoming popular again with the emerging hype around immersive media. However, 360 immersive experience should be categorized as a screen-based media experience, not a 6 DoF experience.
The application of 6 DoF experience is crucial in designing the OASIS user experience because it reorders the priorities in production. When producing a virtual world like OASIS or metaverse, ‘immersion’ and ‘interactivity’ are the two essential axes that determine the level of virtual ‘presence’. Producers have to decide which of the two they would prioritize. The majority of people think that the realism in a virtual environment depends on how realistic (photorealistic) the space looks. However, 6 DoF does not depend on visual realism.
A 6 DoF experience is relative to each user’s movement. The environment interacts with the user’s movement. For example, let’s imagine that you are taking a walk outside. When you walk toward a tree, the distant tree comes closer and closer to you and the people once standing right next to you move farther away. Such experience is fundamentally different from looking at objects in close up or in a long shot. A 6 DoF experience is an interactive experience because the user’s surroundings change in real time as one moves around the virtual space. Interactivity takes priority over immersion. The ultimate goal for potential OASIS producers is to envision both interactivity and immersion, but if one has to choose between the two for financial reasons, interactivity should be prioritized to provide a more effective virtual experience.
Minecraft and Roblox are valuable sources of inspiration for interaction design in OASIS. Although the two games feature only about an average level of graphic, so many people are attracted to them. Why would users choose these games instead of other ones that are more visually advanced? I argue that the reason is because Minecraft and Roblox present a more mature level of interaction. Their popularity exemplifies that advanced graphics are not the prerequisites for commercial success. Of course, both games are investing a lot in improving their graphic qualities as well. Minecraft now supports ray tracing as one of its features and Roblox is developing a next generation graphic engine.
The notion of 6 DoF also applies to AR. Its application is not restricted to virtual environments where physical boundaries are completely overcome. A South Korean drama Memories of the Alhambra (2018) presented an ideal AR example. The show introduced a 6 DoF AR experience, essentially different from other smart phone-based AR experiences. In a 6 DoF AR experience, the user is able to physically approach virtual characters and not only the characters, but also the real physical surrounding of the user has to react to one’s movement. The recent experiments in connecting radar sensor to smart phones and my current interest in ‘volumetric video production technology’ attempt to create a more effective 6 DoF experience.
I put on the entertainment center’s visor and gloves, then stretched out on my mattress. The visor presented me with a three-dimensional view of the Sixer’s database, with dozens of overlapping data windows suspended in front of me. Using my gloves, I began to manipulate these windows, navigating my way through the database’s file structure.- READY PLAYER ONE
The controller used in a 6 DoF virtual environment should be simple and easy to learn. Probably the most common UI examples in VR are multi-screen windows and hand tracking devices shown in films like Minority Report and Iron Man series. The screenshot below shows the protagonist using his hands to navigate multiple virtual windows in front of him. However, such scene does not represent a 6 DoF experience. It shows an example of using a 2D screen UI within a 6 DoF virtual space.
Even in a real-life situation when you want to point at something on a screen, using a type of pointer device is easier and more accurate than using your fingers. Oculus Quest and HoloLens support hand tracking technology. However, issues arise when the tasks that could be better carried out by controllers have to be completed using hands. Hand tracking technology’s low reaction rate is another problem. We usually use our hands when we hold something or when we want to feel something with our fingertips. We are not familiar with touching (or act like touching) virtual objects in front of us. If one wishes to provide a touch experience in a 6 DoF virtual environment, hands will become useful. However, in other cases, controllers are more effective.
STAR WARS: VADER IMMORTAL and HALF-LIFE: ALYX suggest interesting applications of controllers in VR. In both cases, the users learn to apply psychokinesis instead of using their hands or controllers. The users reach for the object by stretching their arms and bring it towards them. I think recreating psychokinesis in a virtual environment is a good example of controller design for a 6 DoF experience because it is very easy to follow once you get used to it. Psychokinesis is impossible in real life, but easy to learn and practice in VR.
Anyone can enjoy supernatural experiences in a 6 DoF virtual world. A few years ago, AWE (Augmented World Expo)’s slogan was “Superpowers to the People”, which implied that granting supernatural power to OASIS users would be a critical factor that differentiates the OASIS experience from other online experiences. In such case, there has to be an appropriate context for the special power. In <Star Wars: Vader Immortal>, the user is not allowed to practice psychokinesis until s/he learns to control “the force”, a central story element of the Star Wars series. In <Half Life: Alyx>, each user must earn the “gravity gloves” to practice the power. It is important to narrativize the potential of supernatural power in a 6 DoF virtual world. OASIS is supposed to be a platform and a content.
On the floor, directly underneath my suspended haptic chair, was my Okagami Runaround omnidirectional treadmill. The treadmill was about two meters square and six centimeters thick. When it was activated, I could run at top speed in any direction and never reach the edge of the platform. If I changed the direction, the treadmill would sense it and it’s rolling surface would change direction to match me, always keeping my body near the center of its platform.- READY PLAYER ONE
I spent the majority of my time in my Shaptic Technologies HC5000 fully adjustable haptic chair. It was suspended by two jointed robotic arms anchored to my apartment’s walls and ceiling. These arms could rotate the chair on all four axes, so when I was strapped in to it, the unit could flip, spin, or shake my body to create the sensation that I was falling, flying, or sitting behind the wheel of a nuclear-powered rocket sled hurtling at Mach 2 through a canyon on the fourth moon of Altair VI.- READY PLAYER ONE
As the quotes from the novel READY PLAYER ONE suggest, ‘omnidirectional treadmill’ was its mere solution to motion sickness in VR. There are several attempts to produce such device today, but even if it gets commercialized, it will be too expensive and inefficient for its size.
The question of how to freely explore a 6 DoF virtual environment while physically staying inside a limited space emerged after HMD appeared on the commercial market. Teleporting superpower was invented as a possible answer. In order to teleport, the user simply has to point at the target space and push the trigger and let it go. It has become a common movement in VR to allow 6 DoF experience in a limited physical play space.
However, teleporting movement can easily turn into an obstacle for VR beginners because it requires training. It is a superpower that we cannot naturally act out because it is impossible in real life. Many newcomers, if not all, first trying out a 6 DoF experience in a virtual environment experience difficulty in practicing the teleporting technique.
HALF-LIFE: ALYX feature a similar issue. When a new player starts the game, the player may be mesmerized by the fact that s/he can control the surrounding virtual objects. But not many players start to move around the space. Movement is a task on a different level. The player can move by physically walking around, but once s/he takes a few steps, room scale boundaries appear, warning that the player will soon bump into the physical wall in the play area. Experienced players who have played the game on PC or those who are familiar with console games try to change the direction by using the direction key on the controller. If it is set to use the direction key on the controller, one can use it, but if it is set to teleport, then pushing the direction key is no use. One has to learn to teleport. Most players easily give up playing the game once they confront this movement issue and end up only having a negative VR experience.
Some neglect the movement interface problem by arguing that humans have easily learned to use other haptic interfaces such as smart phones and PC mouse/keyboard, which were unfamiliar and challenging at first. Yet, there are cases that provide appropriate ‘context’ to superpower movements to facilitate the users’ navigation in VR. One representative case is Tender Claw’s THE UNDER PRESENTS. Here, the player learns to acknowledge the two controllers in one’s hands and learns to pull the space towards oneself. In this case, the player moves the space, not oneself. The space contracts. THE UNDER PRESENTS player has to keep pulling in the space to arrive at a target space where s/he can obtain the mask and once that task is accomplished, one can begin the rest of the experience. The experience that follows the learning of the space movement is quite simple. In THE UNDER PRESENTS contracting and moving around the space is more than a superpower interface. Instead, it becomes a critical content that completes the world view of the experience.
If it is too challenging to contextualize and narrativize the movement element in VR, offering diverse movement options is another solution. Humans move in various ways. We can walk, run, or take different kinds of transportation. Even air travel is possible in real life. If virtual environment can utilize such diverse movement options available in real life as well as those only specific to VR (like teleport), the users will adjust better to the given virtual world.
“Unmediated” experience in OASIS
VR creates this sensation of being actually in a virtual space…with other people in a way that simply looking at flat media doesn’t provide. So VR is going to be an important part of that growing vision (metaverse).- Jesse Schell (Schell Games)
A 6 DoF experience in a virtual environment is never meant to be a solo expedition. In such extended reality space, users meet and interact with others. Virtual experience aspires to be a part of ‘extended’ reality. The use of avatars in OASIS is crucial because the OASIS experience is supposed to be social and interactive.
Single-player experience does not require avatars. Simply showing the player’s hands in VR is enough to provide a single-player experience, as in <Half Life: Alyx>. Avatars play an important role when users recognize the presence of other users and want to interact with them. Many are interested in making avatars as realistic as possible, offering as many facial expressions, actions and visual details as possible in real time.
However, the priority of interactivity in the creation of presence in VR also applies to avatar communication. Even if the avatar in front of me is only made up of geometric figures floating in space, I can feel its presence and our ‘copresence’ in the virtual setting if I can communicate and interact with it in real time. The importance of realistic appearance comes after the feeling of copresence with other avatars. We easily accept “Pengsoo” (a giant penguin character popular in South Korea), animated version of BlackPink in Zepeto, and virtual John Legend on Wave as virtual avatars if we can feel their presence in real time.
There is an ongoing debate about the influence of sound and voice in creating ‘the sense of being with others.’ There are varying opinions on whether sound or movement has a bigger impact on making the users feel the presence of others. The best option is to elevate both sound and movement qualities in VR, but there are many cases when the producer has to decide on prioritizing one over the other. The sound factor becomes a critical issue when curating an ‘immersive performance’ in a virtual setting. During certain immersive performances, the audience can only communicate through body language (which also overcomes the language barrie) so that they can fully focus on enjoying the performance itself, but I personally felt a more heightened sense of presence when I could talk with other avatars around me. One may ask, if sound becomes the central element of a virtual experience, how would it be different from traditional phone calls or video calls? There is no definite answer at this point, but I think doing a voice communication with other avatars while I exist as an avatar in a virtual setting creates a more immediate sense of copresence.
In a virtual environment with 6 DoF, your avatar is more than a character you can control. The relationship between you and your avatar changes. It becomes you. The common conception of an avatar considers it as a separate entity from its user. However, in a virtual environment where 6 DoF experience is possible, your avatar is not something you manipulate. You and your avatar both become the subject.
James Cameron’s 2009 film AVATAR presents such case. When Jake Sully, the protagonist of the film, logs into the virtual environment of the Na’vi, Jake is not playing with his avatar, but he turns into one of the Na’vi people. Jake Sully and the Na’vi characters cannot coexist in one physical setting. Jake has to transform himself into his avatar to enter into the virtual world of the Na’vi. Your avatar becomes more than a mediator between you and the virtual environment. Although you have to put on your VR device to exist as a character in a virtual environment, the boundary between your physical identity and your virtual avatar disappears.
So I…enabled the voluntary OASIS fitness lockout software on my rig…. From then on, my computer monitored my vital signs and kept track of exactly how many calories I burned during the course of each day. If I didn’t meet my daily exercise requirements, the system prevented me from logging into my OASIS account…. Once the lockout was engaged, you couldn’t disable it for two months. And the software was bound to my OASIS account, so I couldn’t just buy a new computer or go rent a booth in some public OASIS cafè.- READY PLAYER ONE
The lockout software also monitored my dietary intake. Each day I was allowed to select meals from a preset menu of healthy, low-calorie foods. The software would order the food for me online and it would be delivered to my door…. If I ordered additional food on my own, it would increase the amount of exercise I had to do each day, to offset my additional calorie intake.- READY PLAYER ONE
For these reasons, ‘connectivity’ is critical in metaverse experiences. There are several discussions on the inconvenience of devices required to log into metaverse. However, wearing an HMD is nothing compared to putting on larger assets to ski or scuba dive. Wearing a small device like HMD can be easily overlooked once people accept its necessity.
The critical inconvenience emerges when you want to move across multiple virtual environments. Most of current VR services use 2D interfaces. The user can click a virtual icon on a 2D screen and log into a virtual setting. However, in a 3D environment, the transfer of experience has to be different. One can no longer simply click a screen to move into another experience because you exist solely as yourself, an individual user. For example, in a 2D online space, you can be logged into Google, Facebook and other social network apps simultaneously because your online existence is separated from your physical body. But in a virtual environment where 6 DoF is granted, you can only be present as one single entity and therefore can’t be at multiple virtual spaces at the same time. Multitasking is a challenge in virtual environments.
Then what would be an efficient way to do multiple tasks in a virtual environment?
Let’s take a look at our physical reality on earth. Our physical reality exists within a single geography on earth. Different locations, such as house, schools, offices, shopping malls and theme parks are present on one overarching map of earth. You do not have to change the map itself to carry out any task. The same should happen in a virtual world. When you enter into a virtual environment as your avatar, you should be able to explore all kinds of virtual experiences within itself. OASIS or metaverse is not a portal or an application. It has to be a unified networked world on its own. ‘Fortnite’, ‘Roblox’, ‘Zepeto’ and ‘Minecraft’ should all be accessible with one single avatar.
MOR (Museum of Other Realities), which was the platform for last year’s Cannes-Tribeca film festival is a meaningful case that showed how convenient it is to have a single avatar to explore multiple virtual spaces. In earlier examples of virtual festivals and even in later virtual events, virtual space only served the role of an open lounge area and users had to enter and exit the area whenever they wanted to view and experience other contents. The way MOR featured its contents, such as THE BOOK OF DISTANCE, was quite different. MOR emulated offline exhibition space and curated each content in different zones. The central area of each exhibition zone was used as an experience portal where users could try out the content of their choice. Once I was inside the portal, I was not exiting the exhibition space, but entering into the virtual environment of the chosen content. When the experience was over, I could come back to the first exhibition area. MOR is a self-sufficient space where users can explore all the available contents within itself.
Recommended: Kaleidoscope’s Rene Pinell’s talk (link) on hosting the festival on MOR from Gyeonggi Content Agency’s NRP Demo Day in November 2020. (the talk begins from 5:57:32)
GSS had also licensed preexisting virtual worlds from their competitors, so content that had already been created for games like Everquest and World of Warcraft was ported over to the OASIS, and copies of Norrath and Azeroth were added to the growing catalog of OASIS planets.- READY PLAYER ONE
The OASIS in the novel READY PLAYER ONE is similarly depicted as a networked virtual world. Multiple virtual environments are connected within the OASIS. Recently, we began to realize the convenience of integrated log-in system. When the user and one’s avatar can exist as a single entity in a virtual space, such convenience is maximized. Once I am logged into a virtual world, I should be able to explore various parts of the environment without logging in and out of a particular service.
There are two ways to make such integration possible. One virtual world service can dominate all other platforms or a completely compatible system among multiple virtual worlds has to be created. In the novel <Ready Player One>, the creator of the OASIS supplies the service to everyone free of charge at first, but the conflict breaks out when global corporations attempt to monopolize and commercialize the OASIS service. The pioneers in the current market who are inspired by the novel do not withhold their ambition to monopolize the future metaverse.
On a positive note, a new discussion on creating a networked metaverse, instead of developing one dominating service has begun. “Into the Metaverse” conference centered around the topic. Tim Sweeney, the CEO of Epic Games and Fornite, emphasized the importance of “open metaverse” at the conference. Even though Mozilla Hubs gave up its business last year, the Web XR system using conventional URL to log into a virtual world continues to attract more attention.
Now that the players racing toward the metaverse have realized the significance of network in creating an appealing user experience, I wonder how the metaverse ecosystem will evolve.
Ep03. What are the requirements for the OASIS user experience?
Written by Sooyoung Choe
Original article: ixi (Feb 1, 2021)
Translator: Da Ye Kim
Cover: Mina Hyeon