Born from Lyft Inc.’s internal machine learning platform, Union Systems Inc. has evolved from an open-source success story into a foundational player for AI infrastructure.

At Lyft, engineers on the platform team faced the common challenge of scaling models, maintaining reliability and operationalizing machine learning at production scale. Their solution gave birth to Flight, a workflow orchestration tool that gained traction across the company before being donated to the Linux Foundation. With the unveiling of Flight 2, Union.ai is taking another bold step toward simplifying the complexity of modern AI development.

“You end up with these cycles where it takes you months to get something out,” said Chris Matteson (pictured, right), head of sales engineering at Union.ai. “But the way the data ages, if it takes you six months to go get a model out, your model’s already stale because your data’s now stale. You need to do this again. What we do is bring all of those pieces together and tie in the various disparate tools, allowing you to build that as a workflow and a pipeline that we can then automate.”

Matteson and Niels Bantilan (left), chief machine learning engineer at Union Systems Inc., spoke with theCUBE’s Rob Strechay and Paul Nashawaty at the KubeCon + CloudNativeCon NA event, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed Flight 2 as a competent platform for deploying dynamic, scalable enterprise AI apps. (* Disclosure below.)

Tackling the AI infrastructure overload

As AI infrastructure systems grow in scale and complexity, engineering teams face infrastructure sprawl. Countless point tools and bespoke workflows slow progress and inflate costs. Flight addresses this by tying together disparate tools into robust, automated workflows that handle everything from failed application programming interface calls to memory allocation, according to Matteson.

“You get so many moving parts, and it’s something that you could no longer handle with a script,” he said. “You no longer want somebody clicking through and doing it. You want this to work consistently everywhere and every time. That’s where tools like Kubernetes and now Flight are critical to help solve those challenges.”

While traditional orchestration tools struggle with static, pre-defined workflows, Flight 2 introduces complete Python-native flexibility. It enables conditional logic, loops and dynamically generated steps — essential for agentic AI systems and research workflows that evolve in real time, according to Bantilan.

“Flight 2 is our gift back to the community based on all of that feedback,” he said. “We built it in-house first, and we’re going to open source it by the end of the year, early next year. And it is a confluence of all the lessons we’ve learned on top of the agentic AI trends that we’re seeing.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of the KubeCon + CloudNativeCon NA event:

(* Disclosure: Union Systems Inc. sponsored this segment of theCUBE. Neither Union.ai nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE