UPDATED 19:34 EDT / JUNE 25 2020

AI

Databricks hands its MLflow machine learning platform to the Linux Foundation

Databricks Inc., the big-data and machine learning company that leads the commercial development of Apache Spark, today put its MLflow project into the hands of the Linux Foundation.

MLflow is a machine learning operations or MLOps platform that the company first open-sourced two years ago. The software gives developers a programmatic way to handle all of the pieces of a machine learning project, from construction, to training, fine-tuning, deployment, management and revision.

MLflow is used to track all of the datasets, model instances, model parameter and algorithms developers use in their machine learning projects. That enables them to be versioned, stored in a central repository and then repackaged so they can be used in other projects as required.

The MLflow project has proven to be a big hit in the machine learning community, with more than 2 million downloads per month. It also counts more than 200 contributors who regularly update the software.

Databricks said it made the decision to hand over stewardship of the project to the Linux Foundation in order to give it a “vendor-neutral home with an open governance model.”

The main motivation for doing so is to try to get more people and more enterprises to work on and contribute to the project, Constellation Research Inc. analyst Holger Mueller told SiliconANGLE.

“But the track record of that is mixed, not only because there are so many open source projects for developers to work on, but because other companies that compete with the gifting vendor are generally suspicious in the first phases of incubation,” Mueller said. “It’s too early to tell what this means for MLflow at this point, so the key thing to watch is if more resources join the project in the coming months.”

MLflow isn’t the only open source project for managing machine learning pipelines that’s already seen widespread adoption. One of its rival projects is kubeflow, which relies on the Kubernetes container orchestration software to manage machine learning pipelines.

Although MLflow and kubeflow are rival projects, they also differ in quite a few ways. For instance, MLflow doesn’t rely on Kubernetes as a component. Instead, it runs on local machines using Python scripts or Databricks as a hosted environment. And MLflow is far more versatile, able to work with multiple machine learning frameworks, whereas kubeflow can only be used with TensorFlow and PyTorch.

Image: Databricks

A message from John Furrier, co-founder of SiliconANGLE:

Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.

  • 15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more
  • 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network.
About SiliconANGLE Media
SiliconANGLE Media is a recognized leader in digital media innovation, uniting breakthrough technology, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — with flagship locations in Silicon Valley and the New York Stock Exchange — SiliconANGLE Media operates at the intersection of media, technology and AI.

Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.