Microsoft Corp. is upping the stakes in the race to build artificial intelligence systems with the introduction of a new initiative called Project Brainwave.

The company is embedding neural network technology into its programmable computer chips in order to accelerate deep learning, which is one of the key subsets of AI. Project Brainwave is an extension of Microsoft’s development work on field programmable gate arrays, which are microprocessors used in data centers that can be reprogrammed in real time for different computing tasks.

“We designed the system for real-time AI, which means the system processes requests as fast as it receives them, with ultra-low latency,” Doug Burger, a Microsoft distinguished engineer, wrote in a blog post Tuesday. “Real-time AI is becoming increasingly important as cloud infrastructures process live data streams, whether they be search queries, videos, sensor streams, or interactions with users.”

Microsoft said it uses Brainwave chips as accelerators in its internal servers for a variety of tasks, including calculating websites’ search engine rankings and machine learning workloads. The process essentially transforms FPGAs into hardware microservices that can speed up deep learning processes, the company said.

There are three steps involved in this process that help to achieve on-the-fly processing for AI applications. First, Microsoft said, it leverages Intel Corp.’s new Stratix chips (pictured), which allow for machine learning models to be stored directly in the FPGA’s memory, instead of the systems’ Random Access Memory. This means the models can persist inside the chips, with the RAM reserved for buffering the incoming and outgoing data.

The second step involves optimizing the design of the FPGAs so that the their full resources are used to process incoming queries. This helps increase throughput while avoiding the need to crunch queries in batches, a process that prevents real-time analysis.

Finally, Microsoft pools each of the FPGAs into a collection of microservices, using their combined processing power to tackle complex AI tasks.

Microsoft said it has been testing Brainwave internally for some time and will begin implementing it in production for services such as Bing Search and speech processing. It also aims to bring Brainwave to its Azure cloud so customers can also benefit with their own AI projects.

Microsoft isn’t alone in its efforts to accelerate machine learning in the cloud. Earlier this year Google announced its Tensor Processing Unit, which is a chip that’s dedicated to accelerating machine learning training tasks.

Photo: Intel