In order to find your way around the world of Extended Reality, it is well worth knowing the differences between each of the above terms, although there is no clear-cut, generally accepted differentiation. Virtual Reality (VR) refers to a world beyond the one that we can physically see; it is generated by a computer. In this case there is a clear separation between the two types of reality. The user is totally surrounded by this world and is meanwhile totally cut off from the real world. Depending on the application, the user may be able to interact with the objects and events in the VR world.

When realities merge

Augmented Reality (AR) is a computer-generated extension of the real world using information and images. You can watch visible reality via a smartphone, (more rarely at present using AR glasses and headsets), while the device recognizes an object you are looking at in the real world (for example, a historic building) and displays some text about it. To do this, mobile devices need to be equipped with a good camera, movement sensors and a powerful CPU. Finally, Mixed Reality (MR) is a stronger version of AR. In this, the physical reality is further merged with the virtual (VR) Reality. The user can embed computer-generated items in real surroundings, walk around them and manipulate them interactively. VR, AR and MR are referred to jointly under the umbrella term of Extended Reality (xR). Mobile xR is the term for its implementation on smartphones and tablets. These devices are already widely available, and this allows mobile xR applications to spread rapidly. For effective Mixed Reality applications, headsets or glasses are better (such as Microsoft's Hololens, Google Glass, or Magic Leap One).

Where are we now?

1982: Damien Broderick referred to "Virtual Reality" for the first time in his science fiction novel The Judas Mandala. And 35 years on? In an edition of the Swiss weekly "Schweiz am Wochenende" in December 2017, the Silicon Valley correspondent called virtual reality a "missed revolution." Obviously the initial euphoria has evaporated. Not because the technology has basically failed to deliver, but because everything is moving a lot faster, and the VR glasses it requires are still relatively cumbersome. Augmented and Mixed Reality, which build on it, have already left pure virtual Reality behind. Nevertheless, the sector is still made fun off. People claim that it is "playing around" rather than serious research and development.

Practical applications already exist

All areas of Extended Reality are based, to put it simply, on technologies that are not so very new. 3D data and game engines. Virtual Reality is also mainly at home in computer games. This is where digital interaction between humans and computers occurs. In Augmented Reality and Mixed Reality, in addition to the human and the computer, it is the surroundings, or the real world, that join in as a third dimension. With AR, you are for the main part still moving in a two-dimensional area, because the mobile devices used to display it have traditional 2D screens.

ARKit and ARCore

Many people still remember the Peakfinder app from 2010, a unique, very expensive pioneer in the "Stone Age" of AR. Today's mobile devices are already equipped with much more highly developed basic frameworks like ARKit or ARCore from Apple or Google. They form the basis and shorten the path to totally new mobile apps, as you can use them to develop faster and cheaper applications. An example is the Ikea Place furniture catalog for the iPhone and iPad. While the camera shows your own living room, you can choose a virtual armchair from the catalog and add it to the image, so you can judge for yourself whether the size and color will fit with the place you have in mind. ARKit still has its shortcomings – for example, recognition of surfaces in the room does not always work well. Space recognition and positioning still needs to be noticeably more precise. Nevertheless, Apple has got ahead of the pack in the mobile xR world. Android has good technical AR frameworks, but these are not as widely available as on iOS, and Google will need to make a huge effort to catch up.

Many uses in education, industry and marketing

The descriptions of the following research and pilot projects are simplified, and they are representative of thousands of similar prototypes worldwide. Medical students will in future learn human anatomy from a virtual patient lying on the table in front of them, with the organs and body parts labeled on the screen. This reduces the cognitive burden of understanding the 3D representation. That allows them to focus on the functions instead of the interpretation of 2D images. This means faster learning and better quality learning. Overall, the technology opens up completely new options in education. Students do not all have to be in the same place. For example, they can see a virtual object in front of them on a table, which they can turn and open following the teacher's instructions. For the maintenance of railway carriages, technicians can, for example, walk round the carriage, while the AR application shows them the items to be inspected in their current condition and what they should look like, stores the history and even generates a paperless invoice, Or to carry out a repair, they can connect live to a colleague at headquarters who is an expert and can show them the procedure on their tablet. This improves the quality of maintenance and the cost effectiveness of the maintenance assignment. Finally, an apartment purchaser can have the interior decorator show them what the finished kitchen could look like while they are standing in the shell of the building, by pointing a tablet at the relevant location. As part of a pilot project with the Swiss railways (SBB), you can already take a virtual walk on your smartphone through the main station in Zurich. The list of the many other opportunities for tourism, marketing, media and entertainment is well beyond the scope of this article.

Headsets and glasses: not if, just when

Although xR glasses could certainly be improved in many ways, the question is not whether they will come, but when. They are particularly suitable for MR applications, while Augmented Reality, as we already mentioned, is more at home on mobile phones and tablets. For solution providers, what will make or break them will be how they generally deal with the topic of xR. From the point of view of software and content, the companies most likely to succeed in the long term are those that understand the whole gamut of Extended Reality as a continuum, and perhaps even adopt it as their core identity – unlike those that focus on a fixed point in this continuum. Especially in the business world, there is huge potential for xR, as the example applications described above demonstrate. Although specific applications are already on the market, there is still a need for improvement in both hardware and software – for example comfortable glasses, field of view, voice recognition, eye tracking, gesture control and performance. But just sitting back and waiting is not the right approach. You can already deduce from today's xR glasses how they will work in future, prepare for it, and gain some important insights.

Where is this heading?

Like any technology revolution, Extended Reality brings opportunities and risks. One thing is certain: the boundaries between what we know physically and virtual reality will disappear one day. It will no longer be "either/or"; we will transition seamlessly between the real world and the computer generated world. One day children will also play, discover and be amazed in Mixed Reality. At the Game Technology Center of the Zurich Federal Institute of Technology, a team led by Bob Sumner is researching Augmented Creativity. Figures in painting books come to life, and visits to the Art Museum become a creative game for children rather than a boring duty. How will we learn to handle all this? Will we still be able to distinguish the real world from the artificial one? How will it affect our memories: what was real and what was not? Mobile phones could become obsolete in the not-too-distant future. We will develop services that we cannot even imagine today, and use them via mobile xR. With so much uncertainty and so many new things, getting advice from experts is becoming more and more important. Because, as so often, the question will be whether all these possibilities are actually useful.