Ride-sharing company Lyft Inc. today said it has open-sourced a new debugging tool for artificial intelligence data that its pricing, locations, estimate time of arrivals, mapping and self-driving developer teams have been using in-house for the last three years.

Flyte is described by Lyft as a “structured and distributed platform for concurrent, scalable and maintainable machine learning workflows.”

“With data now being a primary asset for companies, executing large-scale compute jobs is critical to the business, but problematic from an operational standpoint,” Lyft engineers Allyson Gale and Ketan Umare wrote in a blog post on Medium. “Scaling, monitoring, and managing compute clusters becomes a burden on each product team, slowing down iteration and subsequently product innovation. Moreover, these workflows often have complex data dependencies. Flyte’s mission is to increase development velocity for machine learning and data processing by abstracting this overhead.”

Gale and Umare explained that Flyte is a “multi-tenant” service, which means teams can work on separate data repositories and employ them without affecting the rest of the platform. Software code is versioned and containerized with its dependencies to ensure all executions are still reproducible. Developers can invoke different parameters, or variables internal to AI models that define its skill relating to a particular problem. Flyte can also use cached outputs from previous model executions to save time and memory.

Flyte also can link tasks together and pass data among them via a Python-based domain-specific programming language. Further, because every entity in Flyte is immutable, with each change captured as a new version, it’s possible to iterate or rollback workflows quickly and share versioned tasks across workflows.

“Flyte is built to power and accelerate machine learning and data orchestration at the scale required by modern products, companies, and applications,” the company said. “Together, Lyft and Flyte have grown to see the massive advantage a modern processing platform provides, and we hope that in open-sourcing Flyte you too can reap the benefits.”

Constellation Research Inc. analyst Holger Mueller said Flyte looks to be a useful and proven machine learning management tool that should be attractive to some companies as it spans multiple underlying compute frameworks.

“Executives will need to validate that the components Flyte has are the ones they want to build their next generation applications with, before considering it further,” Mueller said. “If there’s a fit, it’s well worth the detailed evaluation.”

Image: Lyft