Varjo is the first one to get MR right.
I have never been a fan of the term mixed reality, and think it is misleading, confusing, and annoying. Microsoft started it by calling their Hololens AR HMD an MR HMD because they were trying to get some product differentiation and distance from AR and the damage Google had done to the term. Qualcomm tried to sidestep the politics and marketing BS by introducing the term XR, and there has been some traction for that non-descript “we’re doing something with a headset” label.
Augmented reality is one part of the metaverse, and virtual reality the other. Paul Milgram and Fumio Kishino defined them in 1994 as part of the continuum of mixed reality, the middle ground between virtual reality (completely synthetic) and AR (completely real).[i]
The idea of putting camera(s) in front of a VR HMD to give the wearer a real-world view is not new. Making the real-world view look like the real world, however, is difficult. Making the real-world AR interactive is very difficult.
It takes more than just a camera or two to make a VR HMD into an MR HMD, you have to add 6DOF, SLAM, and 3D sensing. Your vision cameras have to have very high resolution and very wide FOV. And then there are the issues of accommodation, plus all the physical aspects of where the wearer’s eyes are, how wide they are, and focus, features that are dealt with by a high-quality VR HMD. And every word, acronym, and term just mentioned adds weight, power consumption, heat, and COST. It’s easy to say MR, damn hard to do.
But if you’re a Finn and have learned to live in the dark for half the year in temperatures that freeze Xenon, you don’t wince at difficult problems, you deal with them. And that’s what Varjo did, in only three years. Well, actually it was more like five or maybe ten years, the people who started Varjo didn’t get their first job there.
The company announced the XR-1 MR HMD officially at the 10th Augmented World Expo (AWE) in May 2019 and gave demonstrations of it there.
Based on the company’s “human-vision” VR HMD system, the MR extension added true-to-life virtual content blending with reality in a full FOV. And the user can easily switch, seamlessly, from mixed reality to VR. Experience the XR-1. The VR HMD has an effective resolution of 50 megapixels per eye, over 20 times that of consumer devices. And the HMD has automatic IPD adjustment based on eye tracking.
What does that mean exactly? It’s a little difficult to explain. Imagine you are wearing a VR HMD, a VR HMD with extraordinarily high-acuity that does eye-tracking, and other optical feats to render a high-refresh, almost zero latency (<15 ms) big bright display. Imagine you are looking a full-size replication of a Volvo sedan like the one in the above photo. It’s a digital representation so you have some choices, it can remain solid and allow you to open a door or the hood and look or even get inside. Or it can be virtual and allow you to walk through it to examine various parts of its construction. Exactly what a VR experience should be, without making you sick. Behind the car is some landscape, for the demo at AWE, Varjo chose Venice. It’s nice, you can walk around the car, turn and look at Venice, look back at the car, perfect registration.
Then you push a button and Venice dissolves leaving you looking at the real world you were in, except the car is still with you.
Now the magic happens. Now you can see your hands, and feet. You can reach into the car with your hands—not paddles and manipulate the model. Push another button and the solid model of the Volvo turns into a 3D blue-lined CAD drawing. A high-resolution CAD drawing with no jaggies that’s to Varjo’s anti-aliasing techniques.
The MR subsystem uses structures light for its SLAM and it works amazingly well even in bright light as illustrated in the following photo.
The images are photorealistic mixed reality in a full field of view. The virtual objects appear as real as anything in the physical world and can themselves cast shadows or even illuminate reality. Black objects appear truly black, opaque objects indeed block real or virtual light, and semi-transparent objects refract the light from the real world behind it. And the display is HDR, so perfectly rendered image can be used for product trials and showcases.
The headset weighs about 2.6 pounds and is driven by a powerful PC with an Nvidia GeForce RTX 2080 wired via a Thunderbolt 3 cable.
Volvo is using the device to test-drive virtual car designs on the road, which is something one can’t do in a car simulation. Volvo is using it to test ideas for things like the heads-up display many years before they exist in physical form. Volvo was the first to use XR-1 prototypes since the summer of 2018 for such experiments.
Volvo has been working with Varjo for over a year, using their VR-1 HMD for design reviews and other engineering and marketing applications. As a result, Volvo has been sufficiently impressed by Varjo’s technology that it is no longer just a partner. As of May 2019, Varjo received an investment by the Volvo Cars Tech Fund, the Swedish car maker’s venture capital fund. Last year, Siemens-backed global venture firm invested $15 million in Varjo. Varjo has raised almost $50 million so far to develop the headsets and has also worked with Airbus and car companies including Audi and BMW.
The VR headset became available earlier this year and sells for $5,995. The add-on MR camera module and the headset will ship to developers in the second half of 2019, the price not yet disclosed. The company has full Unity plugin and Unreal plugin for very easy integration and is working on OpenXr support. They will be announcing list of application partners when they announce price and availability.
What do we think?
Founded in Helsinki in 2016, Varjo entered the VR enterprise and scientific market first, unlike many other HMD suppliers which started in the consumer sector. In the enterprise and scientific market sector, precision technology is more important than a low price for an initial product. Several of the consumer VR suppliers not finding the uptake in that segment that was forecasted are now looking at the enterprise and scientific market as an alternative or replacement market opportunity. Most of them are going to be sadly disappointed to find out the cost cutting they did to try and satisfy a consumer price range will keep them out of the uncompromising enterprise and scientific market.
Varjo began with the ambition of satisfying scientific and enterprise users. Most of the founders came from Microsoft and learned the demands of enterprise customers there.
The Varjo headset is similar to the VRgineers’s XTAL headset. The XTAL came out in July 2018 and Varjo announced theirs in September 2018.
Varjo means “shadow” in Finnish, and is pronounced “Var-yo.”
[i] Milgram, Paul; H. Takemura; A. Utsumi; F. Kishino (1994). “Augmented Reality: A class of displays on the reality-virtuality continuum” (pdf). Proceedings of Telemanipulator and Telepresence Technologies. pp. 2351–34.
