Uber Technologies Inc.’s autonomous driving group today open-sourced Neuropod, a technology designed to reduce the amount of coding enterprise developers have to do to build and deploy artificial intelligence models. 

The problem Neuropod seeks to address is one Uber encountered internally. Enterprises with a sizable in-house AI development operation often use several AI development frameworks, such as TensorFlow and PyTorch, across their projects. Models created with different frameworks have vastly different technical properties that can make working with them difficult.

The issue primarily boils down to application programming interfaces. Incorporating, for example, a TensorFlow computer vision model into a service requires adding support for the TensorFlow APIs to that service, as well as to many of the development tools involved in its creation. If a company then wishes to add a PyTorch model into the mix, its developers would have to perform the same work all over again. This task then has to be repeated across all of the organization’s AI applications if it requires the flexibility to use multiple AI frameworks.  

Uber created Neuropod to eliminate such duplicate work for its engineers. The tool serves as an abstraction interface between the APIs of frameworks such as TensorFlow and the applications that use them. Instead of interacting with the TensorFlow APIs, the application interacts with Neuropod. As a result, developers only need to add support for Neuropod and their workloads will automatically be compatible with multiple AI frameworks.

“Neuropod starts with the concept of a problem definition — a formal description of a “problem” for models to solve,” Vivek Panyam, a senior engineer with Uber’s autonomous driving business, explained in a blog post. “By formally defining a problem, we can treat it as an interface and abstract away the concrete implementations. Every Neuropod model implements a problem definition. As a result, any models that solve the same problem are interchangeable, even if they use different frameworks.”

The interchangeability provided by the tool has several benefits. If a software team builds an AI model for an application in one framework and then creates a newer, better model using a different framework, the new version can simply be dropped into the place of the old one. Or, if an enterprise wishes to switch its AI development workflow from, say, TensorFlow to PyTorch, that task becomes considerably easier.

A third use for Neuropod is simplifying the deployment models from to production. Neuropod bundles models into a uniform “neuropod” or package, that’s easier to work with than the APIs of AI frameworks. Uber used these packages to optimize the model serving platform with which it deploys internal models on graphics card clusters. 

“Without Neuropod, a model serving platform would need to be good at running Keras remotely, TensorFlow remotely, PyTorch remotely, TorchScript remotely, etc,” Panyam wrote. Developers would have to add separate optimizations for each. “However, by using Neuropod, the model serving service can get really good at running neuropods remotely and Neuropod can get really good at running models from multiple frameworks.”

According to Uber, its engineers so far have used Neuropod to deploy hundreds of models spanning fields such as autonomous driving and restaurant menu transcription. Because it’s open-source, other organizations could theoretically customize the tool for their own specific needs. Uber has made the code for Neuropod available on GitHub. 

Photo: Uber