Challenges with Augmented Reality

Augmented reality has always seemed to exist in the shadow of virtual reality. The idea of ​​visiting virtual worlds separate from ours has always captured the public’s imagination and took precedence over “augmenting” our current world. On the other hand, augmented reality has long held many practical applications in enterprise environments, such as industrial manufacturing. This could introduce users to technology at work, which could lead to increased consumer use of augmented reality at home.
Technology research firm Gartner expects virtual reality to reach mass adoption by 2020 to 2023, while it expects mass adoption of augmented reality a few years after that. This means that it will likely be a decade before we see widespread adoption of augmented reality, which seems like a logical conclusion.
Both virtual and augmented reality have technical issues that must be resolved. Augmented reality shares nearly all of the problems with virtual reality but has the additional problems of solving computer vision for detecting real-world objects, unique display form factors on transparent screens (if you’re not using a video camera as a stage), digital object placement, and securing digital holograms in their place in the world. Real, and much more.
Augmented Reality: Form Factors and First Impressions
Augmented reality’s biggest attempt to catch up with virtual reality and enter the spotlight of public consciousness may also prove to be one of the biggest problems it has to face. To experience virtual reality, users must purchase additional peripherals, such as headphones and costly computers to operate it. Adding augmented reality to standard mobile devices instantly puts a form of augmented reality in the hands of hundreds of millions of users.
However, that augmented reality experience is far less than ideal. Engineers at Apple and Google have done an amazing job bringing an augmented reality experience to devices that weren’t originally built for this purpose, but most consumers’ initial experience with augmented reality will be limited to what a mobile device can deliver.
As the saying goes, you never get a second chance to make a first impression. If users have a low quality augmented reality experience on their mobile devices, they can broadly associate that experience with augmented reality in general and reject the technology entirely as much as a mobile augmented reality experience can provide. They may then reject the myriad of other form factors that exist to deliver a superior AR experience.

Augmented Reality: Cost and Availability
Solving the “first impression” issue is an issue of its own. Although a number of augmented reality glasses and goggles are in development, only a select few are available for purchase in 2018, most of which are aimed at enterprise, “developer versions”, or generally not ready for public consumption.
Additionally, unlike virtual reality, for which a number of low-cost headsets are available, augmented reality headsets/glasses can easily cost thousands of dollars, leaving them out of reach of everyone except for early adopters or innovators. This cost difference could help explain why mass adoption of AR is estimated to be a few years further from virtual reality.
Augmented reality, and augmented reality headsets specifically, are largely in the early stage of adoption, the innovators stage. It can be difficult for technologies to overcome the hump of early adopters and cross the gap to reach the consumer adoption stage of the early majority.
The perceived benefit of augmented reality
The best hardware in the world means nothing if it’s not accompanied by amazing software. The public seems intrigued by the promise of augmented reality, but many are not quite sure what purpose they will be using it for. Many people understand the benefits of virtual reality, because the ability to place yourself in a completely virtual world has been explored in relative depths by the popular media. On the other hand, augmented reality has remained under the radar, making it difficult for the public to visualize how to use it.
This could be a chicken or egg scenario. Software developers do not want to build software for devices that have not reached certain consumption levels, and consumers do not want to buy devices that do not have a broad application base for them to use. Users need a compelling reason to purchase these devices.
It will probably take a few enterprising software developers to create some “essential” applications that will lead to consumer adoption. Although the path for these software developers won’t be well cut, those who create their first “killer apps” for aug will be rewarded handsomely.
There are actually a number of augmented reality apps out there that show what’s good about augmented reality.
Augmented reality and tracking
One of the biggest advantages of augmented reality is the ability to place digital objects in a realistic 3D space. Placing an object is fairly simple using a tag in the real world to indicate where the object should go, but the end goal of many augmented reality apps is tagless tracking.
Key Point
“One of the terms you will hear when discussing augmented reality is computer vision. Computer vision is a broad field of study, but in the context of augmented reality, it typically describes how a computer can make sense of the environment it sees via a digital image or video.”
Computer vision that can handle unmarked augmented reality is technologically challenging because it requires a complex understanding of 3D space in the real world. Our brains can view a scene and easily distinguish a wall, a window, and a doorway, but the computer will only see a set of pixels, with no one pixel more important than the other. Computer vision describes how a computer can take a set of pixels and understand what they mean. For example, looking at an image of a table, computer vision will allow the application not only to recognize it as a group of pixels, but also to identify it as an object in a three-dimensional space of height, width and depth.
Even in systems that can perform the required AR processing without tags, delays can occur while processing is occurring. Some AR devices are faster than others at processing the environment (with HoloLens performing remarkably well in this regard), but many AR devices suffer from some amount of latency. Move your mobile device or reposition your head fast enough, and you might see some transformation in digital holograms placed in a physical space, even on the best current generation hardware.

In the real world, though, if you notice a chair changing position or sliding on the floor when you turn your head, you’ll assume your house was haunted. These tracking issues are still common in augmented reality experiences today.
Getting the tracking right is one of the biggest challenges for augmented reality, but it will go a long way toward maintaining the illusion of the user’s digital items that are in physical space. Expect the next generation of devices to make tracking a priority and improve the current generation of tracking technology.
Augmented Reality: Field of View
Field of view (FOV) refers to the area in which digital holograms can appear. For example, FOV for mobile augmented reality is the amount of viewable space on your device’s screen. The device screen acts as a window into the world of augmented reality. Look away from this window at the digital image, and you will only see the real world, where there are no 3D images.
In some current augmented reality headsets/glasses, the digital field of view covers only a very small area inside the eyebrow or glasses, not the entire viewable area. This gives the effect of staring into the virtual world through a small window or character slot.
Similar to searching in a character slot, holograms will appear only in the area you select as a visible hologram. Any part of the hologram that falls in the area you mark the invisible hologram will be cut off at that point. As you can see, a headset with a narrow field of view has a more difficult time offering the same level of immersion as a headset with a larger field of view.
A larger FOV is clearly preferable to a smaller one. If the holograms are only shown in a small window, it’s easy to be pulled out of the experience as you see the holograms being cut into your field of view. The Meta 2 appears to have the largest field of view of the current set of headphones, claiming a 90 degree field of view, but all have a long way to go before getting close to the field of view of the human eye (about 135 degrees vertically and 200 degrees horizontally).
Improving the field of view of augmented reality experience numbers to be one of the next big leaps for augmented reality with the next generation of devices. In fact, Microsoft has already announced that it has found a way to more than double the current field of view of the next generation of HoloLens, which would be a great step toward resolving the biggest complaint most people have with HoloLens.

Augmented Reality Visuals
Like the current generation of virtual reality headsets, current augmented reality headsets struggle to meet the high-resolution requirements that consumers are accustomed to.
In addition, many current augmented reality devices suffer from weak occlusion (the effect of an object blocking another object). In augmented reality, occlusion usually refers to physical objects obscuring digital objects. You may have noticed this issue in augmented reality mobile apps like Pokémon Go: Sometimes you can create a very realistic scene with Zubat flying above the ground; Other times, Squirtle appears to be half stuck inside a wall. These visuals are due to the lack of proper occlusion in augmented reality. When occlusion is implemented correctly, digital objects can be positioned accurately and realistically in any relationship to objects in the real world – under it, partially behind it, on top of it, or whatever the simulation requires.
HoloLens and Meta 2 devices can perform a fair degree of blockage, and demo videos for the Magic Leap seem to show a very high level of blockage (although the device wasn’t shipped until mid-2018, it’s hard to predict whether production will be able to from reaching the maximum specified in his video offerings).
The image below shows a screenshot of the Magic Leap shutdown . The digital hologram of the robot is closed smoothly by the top and sides of the table. If Magic Leap is able to replicate that graphics resolution, along with this level of occlusion in a mass consumer device, it would be a huge step forward for augmented reality.
It may not seem like much, but think of it from the perspective of the least marked trend. In order to put the robot behind the table leg, the simulation software must understand 3D space and not just see an array of pixels. It needs scene rendering and the ability to calculate what should be in the foreground, what should be in the background, and where the digital hologram should fit into all of this. He needs to understand what “under the table” means and find out which parts of the table are farthest in space than the others. This is not easy to achieve.
For augmented reality to deliver a high-quality experience on a large scale to consumers, solving obstruction for dynamic environments will be an important issue to be resolved.