Software giant Microsoft Corp. today announced that engineers working on NASA’s Orion spacecraft designed to take humanity to the moon and beyond will have access to augmented reality, including its HoloLens 2 headset, to help build future versions.

At Microsoft Ignite 2020, the company’s annual developer conference running online this year, the company also announced the availability of a private preview of a new mixed-reality service called Azure Object Anchors that allows HoloLens mixed-reality devices to recognize objects in the real world and map visuals onto them.

To make sure more people have access to its hardware, Microsoft revealed it is expanding the global availability of its HoloLens 2 headset, which is now available for purchase in Italy, Netherlands, Switzerland, Spain, Austria, Sweden, Finland, Norway, Denmark, Belgium, Portugal, Poland, Singapore, Hong Kong and Taiwan.

HoloLens 2 and the Orion spacecraft crew compartment seats

The specifications for the seats of the Orion spacecraft, part of the NASA’s Artemis program, have been carefully crafted by the American aerospace technologies company Lockheed Martin Corp. The seats need to protect people from the negative effects of acceleration, hold them in place and allow them to attend to their jobs while not getting in the way.

Now, the engineers designing and assembling the seats no longer need to use cumbersome blinders or sheaves of paper while building the seats. They can simply don mixed-reality headsets, leaving their hands free to manipulate hardware. Holographic heads-up displays overlay on their vision, highlighting needed tools and hardpoints, allowing even those with minimal training to do maintenance and installation.

“They didn’t have to refer back to a computer screen or paper drawings during that entire activity,” said Shelley Peterson, Lockheed Martin principal investigator for augmented and mixed reality. “Out on the shop floor they can put on the HoloLens 2 device, power it up, and it has all the content that they need to figure out how to do that task overlaid right there on the structure.”

The advent of augmented and mixed reality devices removes a pathway for error – the need to glance away from a task, sometimes repeatedly, in order to double-check information. This is the difference between what are called “heads down” and “heads up” devices.

This is not the first time that Lockheed Martin has put the HoloLens 2 to use. Peterson said that in a variety of highly repetitive tasks – such as those that require marking locations of hundreds of fasteners on Orion’s spacecraft disconnection release – technicians using holographic instructions finished tasks up to 90% faster and with zero errors. “The fact that we haven’t had any errors across all of these activities is phenomenal,” said Peterson.

Azure Object Anchors

During the conference Microsoft announced a private preview of an extremely powerful computer vision mixed-reality service called “Azure Object Anchors.” This service allows HoloLens devices to recognize an object in the real world and map relevant instructions or visuals onto it without requiring any expert knowledge or barcodes to line holograms up onto objects.

A real-world example of how this could be useful is a car owner who wants to repair their car could seek support from an automaker. The automaker would help the car owner repair the car at their own house using a HoloLens device by having them don a headset and look directly at the engine.

Then a technician, sitting at a computer in a remote office, would simply draw in the vision of the car owner and instruct them via audio from afar. Since the technician can see what the owner can see through the headset and add instructions and holograms, it adds a whole new level of collaboration.

The entire affair would be simple and remote and involve no cumbersome paper manuals for anyone involved.

“It’s a very hard computer vision problem,” said Microsoft Technical Fellow Alex Kipman, “but the ability to recognize an object in the real world and then graft and map holograms onto it adds a tremendous amount of value.”

According to Kipman, since the HoloLens 2 began shipping last November the company has seen demand rise. He attributes much of this increase in demand for the device and its attendant services in part because of an increased need for remote collaboration to limit the spread of COVID-19.

To make better use of mixed-reality remote assistance, Microsoft has introduced Microsoft Dynamics 365 Remote Assist, a suite of tools using HoloLens that allows two physically separate people to collaborate and solve problems in a shared mixed-reality environment.

If one frontline worker is assembling a spacecraft or airplane or is exploring a space and discovers an obstruction where the next piece of hardware needs to be installed or cannot move onwards, an engineer in another room or city can see through the worker’s HoloLens 2 exactly what the problem is and offer advice about how to adjust.

“In some industries, we’ve seen companies become willing to take the leap much faster,” Kipman said. “In a socially distanced time where travel can be risky, it’s much easier to just teleport there and see things through the eyes of the operators. Before that sounded kind of cool, like something out of a science fiction novel, but now it’s really a necessity.”

Image: Microsoft