Besides pursuing practical applications of artificial intelligence, Intel Corp. also invests in research to advance the technology’s theoretical foundations. The latest fruit of the chipmaker’s work is an algorithm that could accelerate the development of complex AI models.

The Collaborative Evolutionary Reinforcement Learning or CERL algorithm was presented today at the ICML machine learning conference in California. It promises to speed up the training phase of AI projects, one of the most time-consuming parts of the development cycle.  

Engineers hone their models’ accuracy by having them perform a given task over and over in a simulated setting. The process typically involves the use of virtual rewards: the AI receives a point when it does something right — say, correctly identify an animal in a photo — and applies the lessons from the experience to future images. It’s a powerful approach, but Intel highlighted in the academic paper describing CERL that the method has certain limitations.

A neural network being taught through rewards-based training will often take a “safe route” to maximize the number of points it receives. The AI then becomes less likely to take risks and experiment with new approaches. This ultimately limits how much the model learns during the training, which means it may operate less effectively once it’s deployed in an application.

The issue is especially pronounced with so-called policy gradient-based machine learning methods, one of the most common ways engineers train AI models. There’s an alternative approach that doesn’t have this limitation — evolutionary reinforcement learning — but it has its own tradeoff: It requires considerably more hardware. Intel’s CERL algorithm aims to provide a solution by combining key concepts from both approaches into a hybrid workflow.

CERL launches training sessions by generating a group of neural networks with randomized settings. The algorithm picks out the models that earn the most points as they learn, discards the rest and generates a new crop of AIs based on the top-performing models. That creates a situation where each new generation of neural networks is better than its predecessor.

“The highest ranked ‘elites’ are retained and the rest discarded,” explained Somdeb Majumdar, an engineering research lead at Intel’s AI Lab. “We also perform mutation (cloning with small perturbations) … on the elites to generate high performing offspring to backfill the discarded networks.”

To accelerate the training, Intel’s researchers have equipped CERL with a mechanism they call a replay buffer. It’s a communications channel that allows the neural networks in each generation to share experiences with one another to learn faster. In parallel, a second mechanism optimizes hardware allocation so that more computational power is available for the fastest-learning models, which speeds things up even further.

The result is what Intel hails as significantly faster training times. To measure CERL’s performance, the company put the algorithm to work on Humanoid, an AI challenge that involves teaching a three-dimensional human model to walk. CERL completed the task after just 4,000 roll-outs, or interactions with the simulation, whereas a traditional evolutionary reinforcement learning system required 62.5 million rollouts.

“The CERL solution fits any large-scale decision making problem that solves a complex task,” Majumdar wrote. “It is especially suited to tasks that have multiple hierarchies of requirements like physical robotics, complex games, and autonomous driving, among others.”

Photo: Christian Rasmussen/Flickr