Facebook Inc. has open-sourced another important piece of artificial intelligence software designed to help developers build “reinforcement learning models” into their software applications.

Starting today, developers can use Facebook’s Horizon software toolkit to build applications that can learn to perform specific computing tasks through trial and error.

Facebook has been using Horizon for a number of tasks for some time, including teaching its systems to predict which notifications its users are most likely to respond to. For example, Horizon helps it know that a certain user is much more likely to respond to a “like” from his mother than from dozens of others who may have also interacted on the same post. As a result, “moms’” like would be displayed more prominently in order to get that user to react.

Horizon is also used to power the personalized suggestions from Facebook’s virtual assistant M in its Messenger app.

Reinforcement learning is a subset of AI that involves using simulated environments to teach computer programs how to perform specific tasks. For example, Facebook also uses reinforcement learning to decide whether or not to stream high-quality or low-quality videos to users, depending on factors such as the strength of their cellular connection, or even their location. So it might send lower-bandwidth video if a particular user was on the subway, where the signal is weak.

Facebook isn’t the only company using reinforcement learning. One of its main rivals in AI, Google LLC, used the technology to teach its computers how to play the ancient Chinese board game “Go” without any need for human input. Google’s AI model, called AlphaGo, later beat the current “Go” world champion Lee Sedol over a series of five matches.

With reinforcement learning, computers are either rewarded or penalized based on the outcome of their actions, Facebook engineers Jason Gauci, Edoardo Conti and Kittipat Virochsiri explained in a blog post. So in the case of Facebook notifications for instance, the engineers rewarded those systems each time an alert sent to users resulted in a response.

However, when an alert failed to elicit a response from the user, the system would be penalized. Over time, the systems learned which notifications it should prioritize in order to elicit more responses in the forms of likes or comments.

Facebook’s decision to open source Horizon is something of a milestone because reinforcement learning at scale has been a tough feat to achieve for any company, Holger Mueller, principal analyst and vice president at Constellation Research Inc., told SiliconANGLE. He said the move should ensure more adoption and eyes on the platform, although challenges remain because Horizon runs on Facebook’s PyTorch machine learning framework, which is struggling to keep up with Google’s more popular TensorFlow.

“Also by offering platforms without all of the dedicated and specialized hardware, it raises questions about the implementation costs and total cost of ownership,” Mueller said. “Enterprise have to build applications fast, so they tend to choose integrated offerings. Nonetheless, this is a good move by Facebook to get more usage, value and differentiation out of the PyTorch platform.”

The Horizon code, which Facebook said is the first publicly available reinforcement learning software, is available to download via GitHub.

Image: Facebook