Designing User Experience for Virtual Reality applications
Before jumping right into designing the visuals for the final virtual reality or augmented reality app assets, draw a rough outline of how you think your interface should appear in the virtual world. This is often called the ‘UX design phase’ or the ‘network planning phase’.
There are a number of different options for setting up the network framework for your experience, from low-fidelity options like paper prototyping, to standard 2D tools you may already be familiar with, to full-fledged apps just dedicated to building complex virtual reality or augmented reality prototypes.
Entering the world of UX in 3D can seem daunting, especially since most of our experience comes from 2D screens. Suddenly instead of being confined to a small screen, the whole world is open for you to place your interface elements. However, when focusing on the interface aspects of your design, remember that the “usable area” is often much smaller than the total area of a 360-degree environment. The user interface (UI) should often (but not always) be located somewhere within the user’s “comfort zone”.
Let’s take a look at some of the options available for creating mockups, wireframes, and prototypes. You can choose the method that best suits your workflow, but it is important to prioritize this step in the process. You can review and adjust decisions about interaction and functionality in this stage more quickly than you can in later stages. The extra time you spend up front making these decisions will save you more time in the long run.
Paper models in virtual reality design
This method is not software, but it is one used by many user experience designers, so it is included here for your consideration. Paper prototyping is a system for creating hand-drawn user interfaces for rapid design and testing. It’s a great way to quickly jot ideas down on paper and communicate those ideas among stakeholders. The system works especially well when you can upload images of your paper prototype to the device it will be running on to have a look at it there.
For example, you might display a paper prototype directly on a mobile device when designing a mobile interface, or within a web browser when designing a web interface. When designing an AR app, you can place physical objects or people within your virtual space as you might expect to see them through your device in the app, giving you a rough estimate of how the AR app will feel when placing digital holograms within the real world.
Although paper prototyping can give a general idea of what things might feel like when they’re in a virtual reality or augmented reality environment, they may not capture what an interface would feel within a 3D environment or in a real-world 3D space. Paper prototypes are usually best used for quickly simulating ideas.
Traditional User Experience Applications in Virtual Reality Design
A number of tools are available for creating 2D frames and wireframes, such as Adobe XD or Sketch. You can use these tools to quickly create wireframes and prototypes for your VR application. However, these tools alone can leave a bit to be desired for virtual and augmented reality modeling. Their purpose is to translate 2D designs into a 2D screen. The flat designs they produce may not enable you to get used to the flow of the application in a virtual environment. The image below shows using Adobe XD to create a wireframe for a standard 2D web page.
With various plugins and solutions, you can create simple experiences with these tools that can be viewed in virtual environments. The Sketch-to-VR plug-in, for example, enables you to take a standard Sketch file and display it in a virtual reality environment via WebVR. Although it is not a perfect experience, it will at least bring you one step closer to what the user might actually encounter in VR.
With the growing popularity of virtual reality and augmented reality, it wouldn’t be surprising to see these tools adapt to include the ability to more easily prototype virtual/augmented reality/mixed reality experiences in the near future.
Virtual Reality Modeling Tools
Traditional 2D methods may quickly create user interface prototypes, but there is no substitute for displaying prototypes within a 3D environment. 2D tools are designed for 2D applications. To really visualize the experience you are trying to create, you need a tool that enables you to quickly create a virtual reality environment or place objects in an augmented reality environment.
Tools appear to help create rapid virtual environmental models. Tools like StoryBoard VR, Sketchbox, and Moment are designed with virtual and augmented reality in mind, taking advantage of the strengths and working to overcome the weaknesses of both.
The image below shows a screenshot of a Sketchbox promotional video, where a user inside Sketchbox creates a mockup of the UI in virtual reality.
These storyboard and prototyping apps are specifically designed to create virtual reality, augmented reality interfaces, or storyboards while in a VR headset. Going beyond creating a loose representation of a virtual reality or augmented reality environment on a 2D screen, these tools allow you to create a virtual reality or augmented reality environment where the user will experience them in a 3D space.
You can quickly iterate over the overall look and feel of your app, place objects in 3D space, scale items in relation to each other, etc. Some apps also enable you to create a step-by-step storyboard of how an animation or series of events will be presented. This ability allows designs and animations to be quickly iterated before the time is spent fully designing and developing ideas, saving you time in the long run.
Most of these tools are for virtual reality, but some, like Moment, also include the ability to create prototypes for augmented reality experiences. Moment enables users to mark certain objects as ‘AR Objects’ which, once tagged, can only be viewed when using a virtual mobile device in a virtual reality environment.
This image shows a user prototyping AR elements within the Moments VR app. Lifting an ‘Augmented Reality Device’ into VR exposes specific AR models or assets only, allowing you to rapidly prototype AR experiences in VR.
The best way to get a feel for what the user experience looks like is to choose a tool that enables you to display your interface designs where the user will see them, whether inside a headset or on a mobile device. There are plenty of options available to do this. Try experimenting with several tools to find the ones that fit your workflow best.
Each device has a different set of strengths and weaknesses. When prototyping, consider the capability of the hardware for which you are designing. Room-wide UX design may plan for users who are navigating to reach different points of interaction in your experience. Demanding the same motion in a headset with a seating experience as Google Cardboard would be a design failure.
Whatever you choose to do, finding a way to quickly prototype virtual reality and augmented reality applications is an important step and can be the difference between an app that is frustrating to use and a really great app. Check out the virtual and augmented reality use cases.
The field of accessibility will be an emerging field of virtual / augmented reality. The onus is on content creators to ensure that virtual/augmented reality does not follow the historical path of the Web in terms of accessibility for its end users. In the early days of the web, accessibility for users with different abilities was often an afterthought. When we start defining the virtual/augmented reality space, accessibility for users with different abilities can be front and center of our minds.