the ‘real world’ with elements of a digital world superimposed upon it. The term was largely introduced into mainstream awareness by the viral success of the freemium smartphone game ‘Pokémon Go’ when it exploded in the summer of 2017. Its makers, Niantic, are also co-developers of another augmented reality game set to be released later this year – Harry Potter: Wizards Unite.
But games are just one application of augmented reality and so far has represented a rather simplified version of AR. The augmented reality elements of Pokémon Go’ are pretty graphics but could never be mistaken for ‘reality’.
A recent Financial Times article recounts the author’s first experience of a different kind of augmented reality – with the technology offering a far more realistic experience:
“This was the first time I had met Abramovic, but it did not quite feel like it. Just a few minutes earlier, I was one of two-dozen people standing in the gallery watching her pace methodically around a small circular podium. Then, without any warning or dramatic special effects, her body dissolved into thin air.
This may not have been the “real” Abramovic, but her virtual twin appeared solid, tangible and in all three dimensions. I could walk around the podium until I stood eye to eye with her. I could hear her footsteps as her ghostly shadow passed me, before she reappeared again on the other side of the room”.
The journalist and the rest of the ‘two-dozen’ witnesses of what was an installation of the performance artist Marina Abramovic, were all wearing augmented reality headsets produced by the U.S. start-up ‘Magic Leap’. The company has been the recipient of over $2 billion of venture capital investment and raised that total before ever having launched a product. Silicon Valley is betting big on AR being a major technology trend.
So what’s happening now in AR, how is the technology continuing to get better, what challenges remain and what will some of the main applications be?
Superimposing animated video game characters onto a world view seen through a smartphone’s camera is ‘cool’ but it’s the tip of the iceberg when it comes to AR tech. Creating a realistic 3D, moving hologram of an artist, or anything else ‘real’ is much more difficult. The Abramovic hologram was the result of the artist spending three solid days surrounded by cameras. The combined footage ‘volumetrically’ captured her in 360 degrees. But only after 10 hours was spent on each and every minute of that footage, rendering it into a 3D image. Even the subject’s make-up and choice of clothes had to be very specific. Any reflections would have destroyed the realism of the illusion subsequently created.
It’s possible to make that level of effort for the occasional art installation but obviously that’s not going to take AR mainstream. More sophisticated software will inevitably bring costs and effort down. For now, significant investment is required but some companies already see the value and are investing. U.S. ecommerce company Wayfair, for example, is building a 3D model library that it is beginning to use to present clothes so buyers can see how particular items will look on their body type.
The best AR headsets, like Magic Leap’s and Microsoft’s HoloLense 2 are also still very expensive – over $2000. That’s obviously a traction bottleneck and limits the use of the technology to specialist professional applications for now. Cheaper, better AR headset technology will come but until it does there will also be a limit on the number of high quality AR applications being developed. Why put a lot of time and money into something with a very limited audience? But consumers or companies are not going to buy headsets until they are both cheaper and there is a more high quality content for them.
However, this kind of ‘chicken or the egg’ stage where hardware needs content to be attractive to consumers but content creators need more people owning the hardware before it’s worth the effort is not uncommon. Many new kinds of tech face and surmount this challenge and AR will too. And headsets are not necessary for every kind of application of AR. Smartphone screens or monitors are enough for a certain level AR to be leveraged by ecommerce and other industries.
Most Recent AR Developments
More sophistication integration of AI into AR applications is one development we can expect to trend over the next couple of years. Algorithms that allow computers to better ‘understand’ what they are seeing cameras will see applications better at identifying and labelling items within a user’s field of vision.
AI-based natural language processing being used for voice control within AR is another way the technology is developing. This will eliminate the need for as many icons or menus in AR worlds, improving immersion. These developments in AI will also improve gaming experiences as virtual characters get better at reacting and adapting to the behaviour of human players.
AR hardware developments are key to adoption. Until now the choice has been between big, cumbersome headsets linked to powerful computers for a more impressive experience at the cost of mobility and comfort or light headsets with technology limitations such as being able to serve high quality graphics. Later this year the first consumer-facing headsets that should be both comfortable, small and powerful will hit shelves in the form of the Oculus and Vive AR kits.
Eyeball tracking technology and wider fields of view are also aspects of AR hardware that are quickly improving and will make a big contribution to the AR experience.
Augmented Reality Applications
What are some of the concrete applications of AR technology we can expect to see in the shorter and longer term?
Communications Technology: both Facebook, through its ‘VR Spaces’ platform and Tencent’s WeChat app, which is the most popular messenger in the Chinese speaking world (and the world as a whole), are expected to introduce VR products this year.
Professional communications apps using similar technology are sure to be close behind. This will mean we should expect to soon get used to communicating with each other through AR avatars.
Retail: if there is one major bottleneck to the rise of ecommerce it is that shoppers, especially when buying clothes, still like to be able to try things on. Until now, that has been gotten around to some extent by allowing buyers to return unwanted items free of charge but obviously entails a logistics cost for retailers. VR will be able to give consumers a much clearer idea how an item of clothing will look on them and in combination with other items as well as, for example, how a new armchair would fit into their living room.
IKEA is an earlier adopter here and its IKEA Place AR app has proven to be a major hit.
Gaming: more comfortable yet powerful and affordable AR headsets are now coming to market that, along with better games developed for them, is expected to see an explosion of AR gaming over the next several years.
Education & Training: this is the area that AR applications have seen the most early traction. AR and VR have clear benefits when it comes to providing students with a realistic training environment that can give them genuine, applicable experience without exposing them and others to ‘real world risks’. Walmart has already invested in 17,000 Oculus Go VR headsets to be used in training.
Health Care: as well as being used to help train medical professions such as doctors and surgeons, VR has clear application in real life situations. A surgeon performing an operation wearing a VR headset can have all sorts of additional general and personal medical data within view, or which can be brought up on command when they want it.
Autonomous Vehicles: the next generations of electric and driverless vehicles will incorporate AR technology. Chip maker Nvidia’s DriveAR platform uses an AR dashboard display able to alert drivers or passengers to information such as roadworks and other hazards to landmarks along their route. A host of car manufacturers such as Tesla, Toyota, Audi, Mercedes-Benz and Volvo have already committed to using the technology in future models.
GPS navigation is also going to a whole new level with the help of AR. Alibaba have invested in WayRay, a start-up that projects AR data onto a car’s windshield. Direction prompts, information on lanes and right of way and hazard alerts are all incorporated.
Office & Boardroom: companies such as Spatial are in the process of developing AR tools for office and boardroom environments. AR whiteboards, pin boards and interactive design documents that can be superimposed on the real world may well become common within just a few short years.