Engineers at the world’s top tech firms often find themselves having to build custom alternatives to existing software in order to meet the unique needs of their companies. One notable example of such a software project is Horovod, a deep learning platform created by Uber Technologies Inc. that today moved to the Linux Foundation.

The project, which made its initial debut last year, will move under the wing of the Linux Foundation’s LF Deep Learning Foundation for open-source artificial intelligence software.

Horovod is designed to simplify the training phase of AI projects. Deep learning models created from scratch need to hone their accuracy by processing sample data before they can be unleashed on live information. The process involves all kinds of challenges, not least of which is modifying the neural network to run in the designated training environment.

The task is difficult because deep learning projects usually start their life on a developer’s workstation. A training environment, in contrast, can consist of a large number of servers each potentially with multiple graphics cards. Taking advantage of all this hardware requires making extensive changes to the original AI model.

According to Uber, Horovod enables developers to accomplish the task with just a few lines of code. This not only speeds up the initial modification process but also eases debugging further down the road. That’s just as big a time saver, given the highly iterative nature of deep learning projects.

Horovod has the added benefit of being highly efficient. Uber claims the system makes it possible to train AI models roughly twice as fast as as a traditional deployment of TensorFlow, the market-leading deep learning framework created by Google LLC.

But Horovod is built to complement rather than replace TensorFlow. Uber uses the system internally to support the development of AI models for self-driving cars, fraud prevention and trip forecasting applications. Other Horovod users include Amazon.com Inc., Nvidia Corp. and Alibaba Group Holding Ltd.

Horovod is one component of a complex platform called Michelangelo that Uber has developed to serve as the foundation for its internal deep learning efforts. The company describes the software as an end-to-end system for managing every aspect of the AI development lifecycle. Given that Uber was willing to open up Horovod, it might also release the code for other Michelangelo components over time.

Photo: Uber