Researchers at Google have open-sourced a new framework that can scale up artificial intelligence model training across thousands of machines.

It’s a promising development because it should enable AI algorithm training to be performed at millions of frames per second while reducing the costs of doing so by as much as 80%, Google noted in a research paper.

That kind of reduction could help to level the playing field a bit for startups that previously haven’t been able to compete with major players such as Google in AI. Indeed, the cost of training sophisticated machine learning models in the cloud is surprisingly expensive.

One recent report by Synced found that the University of Washington racked up $25,000 in costs to train its Grover model, which is used to detect and generate fake news. Meanwhile, OpenAI paid $256 per hour to train its GPT-2 language model, while Google itself spend around $6,912 to train its BERT model for natural language processing tasks.

SEED RL is built atop of the TensorFlow 2.0 framework and works by leveraging a combination of graphics processing units and tensor processing units to centralize model inference. The inference is then performed centrally using a learner component that trains the model.

The target model’s variables and state information are kept local, and observations on them are sent to the learner at every step of the process. SEED RL also uses a network library based on the open-source universal RPC framework to minimize latency.

Google’s researchers said the learner component of SEED RL can be scaled across thousands of cores, while the number of actors that iterate between taking steps in the environment and running inference on the model to predict the next action, can scale to thousands of machines.

Overview of the architecture of SEED RL. In contrast to the IMPALA architecture, the actors only take actions in environments. Inference is executed centrally by the learner on accelerators using batches of data from multiple actors.

Google evaluated SEED RL’s efficiency by benchmarking it on the popular Arcade Learning Environment, the Google Research Football environment and several DeepMind Lab environments. The results show they managed to solve a Google Research Football task while training the model at 2.4 million frames per second using 64 Cloud Tensor Processing Unit chips. That’s around 80 times faster than previous frameworks, Google said.

“This results in a significant speed-up in wall-clock time and, because accelerators are orders of magnitude cheaper per operation than CPUs, the cost of experiments is reduced drastically,” Lasse Espeholt, a research engineer at Google Research in Amsterdam, wrote in the company’s AI blog Monday. “We believe SEED RL, and the results presented, demonstrate that reinforcement learning has once again caught up with the rest of the deep learning field in terms of taking advantage of accelerators.”

Constellation Research Inc. analyst Holger Mueller told SiliconANGLE that SEED RL looks to be another example of “reinforcement learning”, which he said is emerging as one of the most promising AI techniques to advance next generation applications.

“When you tweak software to work well with hardware, you usually see major advances and that is what Google is showing here – the combination of its SEED RL library with its TPU architecture,” Mueller said. “Not surprisingly it provides substantial performance gains over conventional solutions. This makes reinforcement learning available to the masses, although users would be locked into the Google Cloud Platform. But AI is served best in the cloud, and GCP is a very good choice for AI apps.”

Google said the code for SEED RL has been open-sourced and made available on Github, together with examples that show how to run it on Google Cloud with graphics processing units.

Photo: mohamed_hassan/Pixabay