The HoloLens in a nutshell

Depending on how you look at it, the HoloLens is a sort of digital glasses or a new form of portable computer, which allows 3D projections to be embedded in real space and interact with real objects. The HoloLens recognizes its environment, allowing objects to hang on walls or hide under tables, or interact with them, based on physical models. It is controlled by simple gestures, head motion, eye movement and speech recognition.

Unlike other headsets, the HoloLens is a fully independent, stand-alone computer and does not need a smartphone. The HoloLens integrates several sensors, Wi-Fi and cameras, operates for 3 hours on a single battery charge, and runs under Windows 10. Microsoft is currently offering the device in the US and Canada as a "mixed reality device" for USD 3,000. The term "mixed reality" (MR) is based on the fact that the device interacts more strongly with the environment than previous augmented reality (AR) applications,

• Virtual Reality (VR): Completely computer generated environment with the greatest possible immersion, but little relationship to the real world.
• Augmented Reality (AR): Computer-based extension of reality, often by blending or overlaying images and sound.
• Mixed Reality (MR): Mixing real worlds and environments with virtual environments.

Technology and structure

The user looks through a transparent glass visor. Optical waveguides, which look like two transparent screens, are embedded in the glass. These add light to the natural light to blend the three-dimensional worlds into the natural environment.

For spatial recognition, the HoloLens uses Kinect technology, in particular depth sensors. These continuously scan the environment. This is how the HoloLens recognizes and remembers the entire space, including walls and tables.

A Holographic Processing Unit (HPU) processes all the data from the sensors and computes the projections. The sensors comprise a gyroscope, acceleration sensor and head-tracking sensor, magnetometer, light sensor, microphones and a video camera. Everything has been integrated into a very good overall package. The high resolution and stable spatial positioning of 3D objects are extraordinary.

A game changer

The HoloLens is difficult to describe – you really need to experience this altered reality to understand what it actually means. This new kind of visualization in your own environment makes a very strong impression. Why is it a game changer?

First of all, the superior visualization. One example is the HoloAnatomy App, which shows a life-sized human body in your living room. If you walk around the body, you can understand how it works much better than by looking at a monitor.

Secondly, the way it interacts with the environment. In the RoboRaid first-person shooter game, for example, it is impressive to see how aliens knock holes in your walls and insects crawl out of them.

Thirdly, its simplicity. Children aged 6 and seniors over 70 were able to use the headset without much instruction. They could dive into the experience straight away and move around in the virtual world. This is the first device since the introduction of the iPhone with a totally new form of control that is so easy and intuitive for everyone to use.

For me, that makes the HoloLens a game changer. The device opens up totally new worlds of experience that were not possible before. The current device will certainly not reach a huge public – it is too expensive and not sufficiently available – but it shows us the future roadmap of AR and MR.

The competition is not standing idly by

The AR/MR area is in a state of rapid flux, with a huge number of announcements. Where is the HoloLens positioned in the market?

Mobile phones, because they are so widespread, are the strongest competition. With the success of Pokemon Go, the AR concept has gained breadth. Even if Pokemon Go is very trivial, it shows how quickly a large number of people can be drawn in. Tango gives you an experience similar to the HoloLens with a mobile phone. So far, only one device has been announced, the Lenovo Phab 2 Pro The experience is somewhat limited because users have to view the scene through the small screen of their phone or tablet, but if it becomes widely available to everyone as a built-in smartphone app with no visible cost, it can become huge despite this.

Magic Leap is a startup which has already been valued at over 4.5 billion and has obtained funding of over 1 billion. The company is still keeping everything under wraps, but apparently – unlike HoloLens – the light is projected directly into the user's eyes. Magic Leap technology could therefore enable a very wide field of view, and possibly smaller hardware. We have to wait and see.

There are many devices in the VR area, including HTC Vive and Samsung Gear or CardBoard. These enable much better immersion in a closed virtual world because the user is no longer aware of the real world. However, this often leads to nausea. As the user is very isolated, I see these devices being primarily used in gaming or for visualization in museums.

Where are the downsides?

The most often mentioned criticism of the HoloLens relates to its key components, the optical waveguides: The field of view is limited, with a stated value of about 30 to 40 degrees. This may affect users to a greater or lesser extent – it depends on the application how noticeable or irritating it is. The optical waveguides are also not usable in direct sunlight because they cannot overlay the sunlight strongly enough. This means that the device is primarily used indoors.

The HoloLens is very nicely finished, and considering its many functions it is very compact and well designed. However, it could be made more user friendly for the use scenarios – in particular, it is very conspicuous and irritating when a HoloLens user looks at someone else.

Application areas

Although still in the "development kit" stage, the HoloLens is a very nicely finished and mature product. Given the high price and limited availability in North America, distribution will be very concentrated – making it a business device, not a consumer device.

The following application areas have already been identified:

• Engineering firms for rapid prototyping of new designs
• Design agencies and architectural firms for visualization of their work
• Healthcare sector for presentation of body scans
• Generally in education and training for better explanation of processes, for example the inner workings of the body or of machines
• Visualizations and intuitive explanations in museums

We will have to wait and see how much these use cases actually become reality. However, the existing trial applications are impressive examples of what mixed reality can do. Who will finally win the race is not yet clear, but the HoloLens already shows us where the future is headed.

About the author

Reto Grob is Principal Software Architect at the software company Netcetera in Zurich. He is responsible for software architecture, heads up project development and supports business development with innovation topics. In addition, he is a Member of the Board at smama and Best-of-Swiss Apps and heads the Mobile Apps group in smama. His 16 years of experience have taken him into a wide variety of software stacks, with recent focus on mobile and modern web applications. Reto Grob studied Information Science at the Federal Institute of Technology in Zurich and has worked for extensive periods abroad.