Canon’s Head Mounted “Mixed Reality” Augmented Reality Device is Nice but a Bit Out of Reach


The era of consumer-level augmented reality devices is slowly coming into port as we watch more manufacturers bring this technology to the public—partially through miniaturization and partially through making them at least affordable. With the Oculus Rift and Google Glass shining bright on the horizon, Canon is bringing their own AR device, or “mixed reality” system dubbed MReal, to bear. Although with a price tag of $125,000 it’s nowhere near Google Glass in likely consumer-applicability (I guess we’ll just have to wait for the price to come down.)

What should catch the eye of consumers is Canon’s technology for producing mixed reality—as a camera company, they’re in a good position to innovate the cutting edge of head mounted displays and it shows here. According to the specs of the device—revealed in a Q&A interview with Canon engineers at the forefront of mixed reality development—the MReal head mounted display uses prisms instead of mirrors to help generate a clearer and higher fidelity overlay of what the user is seeing. It also implements a multitude of sensors that use infra-red beams, gyroscopes, and magnetic sensors to determine what the viewer is looking at (aside from the cameras) so that the augmented reality display can effectively target.

The use of a prism should help greatly reduce the footprint of the head mounted display (moving it closer to something that won’t clash with basic fashion sense). It also means that the sensors and cameras used to gather data about the world can be placed closer to the wearer’s eyes. During the interview the engineers described some of the difficulties when it comes to how to place virtual objects in the world—something revealed looking at how augmented reality can be used for welding helmets to assist in welds.

Overlaying computer-generated graphics onto the world means coupling what the wearer “sees” as much as possible with what’s being displayed. If this extends beyond basic heads up display (reticles and text) but into virtual objects that means that they have to be lit similarly, cast rudimentary shadows, and etc.

“During the early stages of development, the CG images were often too bright and sharp, and didn’t fit properly within the real space,” Canon software developer Taichi Matsui said “We needed to adjust the CG rendering (including coloring and shading) to match the real space, such as the contours and colors that often blend into one another.” The team also looked into giving the objects shadows, proper lighting, and other telltales of actually existing in the environment to blend the wearer’s sense of virtuality.

Writer James Plafke from ExtremeTech got a hands on preview of this technology and has this to say about it’s capabilities:

I got to try out a few MReal demos last night, the first of which involved two car seats sitting by themselves out in the open. Above the car seats were 16 optical sensors attached to a frame. After having the HMD strapped onto my head, I sat in the passenger car seat, and a rep used an iPad to begin the demo. The full interior of a car appeared around me — the steering wheel, the dashboard, the doors, windows, and a virtual tablet attached to the dashboard. Since the HMD is a mixed reality unit, I was able to look through the windshield and see everything at the event — the people, tables, chairs, and so on. I was able to step out of the virtual car, and walk around the entire vehicle, bending down to examine the hubcaps and headlights.

Projects such as MReal represent the high end of augmented reality (and with the hundred-thousand dollar price tag will stay there for now.) A demo showing off how they can be used to sell cars is only the tip of the iceberg that this sort of high fidelity mixed/augmented reality can be used.

Augmented reality for the institution-level interest

With the example of the augmented reality welding goggles, we can see that a set such as this, potentially a few technological advances later, could be extremely useful in the medical industry. With the ability to properly overlay visual information based on other computer-aided technology could be used during brain surgery to locate a tumor, or guide a probe through the blood stream. The potential uses for extremely high fidelity CG inserted into the field of view is almost endless.

Right now, this is also exactly where the MReal technology will end up: in the hands of institutions and not individuals.

At the price tag listed, the MReal technology (while becoming groundbreaking) is highly specialized equipment that will end up being customized to particular purposes. Also noted is that it will cost almost $25,000 a year to maintain, update, and upgrade. So what we’re looking at here is essentially enterprise specialty equipment that if deployed will only appear in the most cutting-edge places, potentially where research is breaking ground for teaching or delicate work, but it will be many years before it will show up in with the boots in the trenches.

We can probably expect specialized versions of Google Glass to appear on the ground first.