Understanding and assessing AR headsets

A couple of weeks ago I wrote an article to help businesses and organizations understand the terminology and challenges when creating digital product configurators. It was well received and readers started reaching out to discuss other digital trends. A lot of questions were around assessing the potential of new Augmented Reality technologies, specifically headsets. This was probably driven by a huge number of announcements during CES2020.

So let’s dive into AR a little bit to understand the terminology and challenges around the topic, with the goal to better assess the potential of individual solutions.

First, the usual disclaimer: While the views in this article are mine, please be aware that I work for Microsoft around IoT and Mixed Reality things.

What are we talking about?

Humans experience the world via their senses and in principle every one of those can be digitally augmented: Screens & glasses augment the visual sense, headphones the audible sense, gloves the haptic sense and so on. While every type of augmentation is interesting, I want to focus here on the visual (and to an extend the audible) sense, specifically using Augmented Reality headsets. I will not talk about smartphone based AR (not a headset) or Virtual Reality (not AR).

Understanding device classes and modes

Currently these approaches manifest themselves as device classes, as they have to deal with different technical challenges (see below). But really these are operational modes that achieve different things in different situations and scenarios. So it’s important to note that no device class / mode is strictly better than the other, they just have different optimal use cases.

I believe that in the future these device classes will converge and one device will be able to utilize every mode, based on which provides the best user experience in a given situation. However until the technology has caught up, they will remain separated.

Mode 1: Second screens aka Heads-up-Displays aka Annotated Reality

While the visual output is separated from the physical environment, the key is some form of contextual awareness. A system used for navigation might utilize GPS and a compass to know where it is and provide the relevant information about the physical environment on the screen (“You are here and this is what’s around you”). That way, although the screen is separated from the environment, there is a contextual connection between both.

Get Directions [through Google Glass], Google

Mode 2: Unified worlds aka Digital Windows aka Mixed Reality

Such a system is also context aware, but beyond that it also has spatial awareness. For example a navigation system would use the environmental understanding to highlight the actual lane you need to take and possibly other relevant information.

AR hub, Ceres Holographics, https://www.ceresholographics.com/ar-hud.html

Mode 3: World replacement aka Augmented Virtuality

holoride, https://www.holoride.com/

Understanding the challenges for AR devices

  • Compromising on thermals makes devices warmer. While fine for laptops, you do not want to wear something as hot as your laptop directly on your head.
  • Compromising on batteries makes the devices inconvenient to use when the operating time gets shorter than the desired usage scenario.
  • As does compromising on performance when the device gets too sluggish or unresponsive to use.

Unfortunately the technology used in AR headsets is so immature that EVERYTHING has to be a compromise. There is literally no option to create an “uncompromising” device, no matter how much resources a manufacturer has, as the technology is simply not there yet.

Form factor considerations

Left: Magic Leap 1 — Right: Microsoft HoloLens 2

This way Magic Leap can use more powerful processors and bigger batteries as well as generate more heat as weight and heat are on your hip, not on your face. The drawback is that the user needs to be more mindful when handling the device (there are dangling cables involved) and the overall weight is higher (headset + external processing pack). Overall this is not a bad solution, as long as these drawbacks are acceptable in your usage scenarios.

The form factor can even be configurable, where third party manufacturers can take a device and change the shell to their requirements. An example would be the HoloLens 2 Customization Program, which allows partners to customize a HoloLens 2 device. Further abstracted are hardware platforms, where the solution might not be a device, but rather base technologies that can be used by device builders to create their own solutions quickly. An example would be the Qualcomm Snapdragon XR2 platform.

Software platform considerations

Audience considerations

Vusix smart glasses for example are clearly targeted at enterprise scenarios. Their entire website is about field service, remote assist, manufacturing or warehouse solutions. It’s a professional tool you use at work with no ambition to cater to mass market consumers or even early adopters.

Microsoft HoloLens 2 does the same thing. Microsoft calls it “The future of work”. While the branding around it is more fashionable and almost like a consumer device, the targeting is clearly enterprise and business.

Magic Leap One always had a more general purpose messaging. The product page itself is very modern and consumer oriented, the main website starts with “Magic Leap 1 for enterprise” before showing the “Magic Leap 1 at home”.

A clear target audience is important because with the hardware limitations as they are, it’s easier to develop devices around a clear use case. It’s also relevant as some use cases demand specific device considerations — for example safety certifications in certain enterprise environments like manufacturing or construction. It’s important not to evaluate every device based on their “mass market appeal”, but according to the specific use case and scenario.

Defining success

With general purpose consumer devices, every compromise is turning off yet another potential buyer group.

To even get to a general purpose & consumer grade device, it has to be:

  • 1.500 USD or less, comparing it to super high end smartphone devices.
  • Ability to produce up to 15 million units per year, similar to the iPhone X in it’s first year as a high cost, high end smartphone.
  • Around 6–8h of battery life to use it a full day, assuming it’s worn and used during commute, work and for relaxation.
  • 200 grams / 7 oz or less, making them way heavier than regular glasses (around 20 grams / 1 oz), but lighter than a bike helmet. This is already stretching it for full day use, but acceptable if the ergonomics are done right.
  • They must be inclusive, for all kinds of head shapes, hairstyles and personal accessories, as well as for users with impaired vision or hearing.
  • Rugged enough to survive a fall or two plus the usual mishandling by friends, kids, pets and tossing it into a gym bag.
  • Active developer community, including available variants of the most used smartphone apps, assuming that the goal of general purpose, consumer grade is to replace the smartphone for most use cases.

That’s kind of the point, isn’t it? The endgame is to build the next smartphone scale device and the only chance of doing so is to replace it. To make a bold prediction, I believe that a true AR consumer device has to be mode 2. Mode 3 won’t be socially acceptable for a while (as it completely replaces vision) and mode 1 is just too similar to a smartphone to warrant a large scale switch to a new form factor.

In my view, the above is the minimum it would take to even have a chance at creating a large scale consumer AR product. But feel free to disagree and substitute your own criteria and thresholds for viable AR consumer devices.

Now, to be fair there is an interesting alternative where AR headsets will start as mode 1, augmenting the smartphone. In that case the smartphone becomes an external component of the glasses — a companion device. That was actually the original idea behind Google Glass. This strategy makes perfect sense if you’re an incumbent smartphone manufacturer: Start with a companion device featuring basic functionalities (similar to a smartwatch) that over time grows into a stand-alone product. This would not cannibalize their earnings and they can build everything on their existing platform, gradually extending it. The bad news is that you’d need to be Apple, Google, Samsung or maybe Huawei to pull this one off.

Again, if your use case is not “replacing the smartphone”, but a very defined vertical, success might be selling 10.000 units plus long term service and operations contracts. These kind of devices might look strange at first as they are usually highly optimized for that specific use case. Lugging a 3kg / 6 pounds device around might seem absurd until you are a fire fighter and the thing need to be able to take a beating and still work reliably in 300 °C environments.

Is it real?

Google Glass Project — One day, Google, 2014

We learned very quickly that the actual Google Glass device is nowhere near that vision. Indeed ( would argue that neither hardware nor software of Glass did anything to advance Google towards the vision showed in the video. That said, I do believe it wasn’t Googles intent to mislead customers, developers or investors. There is a need to create such visions to make goals and ambitions of your solution tangible. However it is sometimes hard to see behind the curtain and understand what is real (existing product) and what is not (vision). If in doubt, get a hands-on demo. And make sure it’s not a prototype or engineering sample, but the actual production model.

Summary: Understanding device potential

  • What mode is it? Is it a second screen, unified world or world replacement?
  • Is it self-contained or does it rely on other, external components? Is it customizable or a platform?
  • What software platform does it use? A new, self-made one, an adapted version of something existing or mature, common platform?
  • Does it have a specific use case or is it general purpose? Likewise: Does it have a specific audience or is it fully consumer grade? Does it pass your threshold for viable consumer devices or the regulatory requirements for an enterprise device?
  • And the most important question: Does it actually exist?

Now, what are you interested in? Do you want to assess a device based on its potential to be a consumer grade, multi-million unit seller? Or how well it fits a specific enterprise use case? Analyze which manufacturer has the most forward looking vision regardless of their ability to execute? Based on that, define what answers you would prefer to see for the respective categories and then assess the device(s).

While not perfect, this should offer a robust model that allows you to assess the potential of different AR headsets and even compare devices and solutions.

Feel free to check, assess and discuss the following solutions presented at CES2020 to get a feel for it.

Living in Berlin / Germany, working at Microsoft, loving technology, society, good food, well designed games and this world in general. Views are mine, k?