Microsoft Corp.’s HoloLens is set to become a lot smarter.

Marc Pollefeys, the Director of Science at the company’s mixed reality group, on Sunday revealed that the next iteration of the headset will pack a co-processor for running artificial intelligence software. The chip is based on a custom design just like the camera that HoloLens uses to track hand gestures and several other components. Its purpose is to let the system harness AI for tasks that are currently handled less efficiently because of hardware limitations.

The first constraint is latency. Many operations must be performed near instantaneously to produce a smooth user experience, which rules out the possibility of relegating them to a remote data center thanks to the inevitable networking delays. That leaves local execution as the only option for AI models. But the HoloLens’ so-called Holographic Processing Unit was not built with such workloads in mind, which opens the door to performance issues.

The new chip is intended to eliminate this tradeoff. It will take over the heavy lifting involved in running a neural network to free up computational resources for other processes. Moreover, Microsoft’s Pollefeys said, the processor should enable the HoloLens to operate longer between charges when performing AI-supported tasks.

He didn’t share any specifications, but other AI-optimized systems out there should provide a general idea of what to expect. The newest of the bunch is the Movidius Neural Compute Stick that Intel Corp. unveiled last week. It’s a thumb drive equipped with a chip that has been described as capable of handling video from up to six high-definition cameras at once.

Processing media input appears to be a priority for Microsoft as well. Harry Shum, the head of its research division, took to the stage at the CVPR 2017 computer vision conference on Sunday to demonstrate how the HoloLens can use the new chip to interpret hand gestures. The fact that the processor is touted as supporting a “wide variety of [neural network] layer types” indicates it could be used to streamline other aspects the mixed-reality experience as well.

The project is the latest in a series of initiatives by the tech industry to improve the AI capabilities available on user devices. Previously, Google released a set of computer vision algorithms aimed at helping developers build more powerful object recognition apps. And rumor has it that Apple Inc. plans to join the fray as well by embedding an AI chip into future devices.

Image: Microsoft