When AtScale Inc. released its “Hadoop Maturity Survey” in 2015, some industry observers were surprised that business intelligence had become the number one workload on the open source platform for big data management, Hadoop (in front of data science), given the problems of scaling information technology solutions. But a customer partnership between AtScale and GoDaddy Inc., fostered by two executives who formerly worked together at Yahoo, is showing promising results in solving a thorny problem: how to do BI effectively on a massive big data platform.

“Developing data at scale, that’s the premise I’m looking for from them,” said Prashanthi Paty (pictured left), head of data and personalization at GoDaddy, while describing her work with AtScale. “We want the world.”

Paty, who was joined by Josh Klahr (pictured right), vice president of product management at AtScale, visited theCUBE, SiliconANGLE’s mobile livestreaming studio, and answered questions from hosts Lisa Martin (@Luccazara) and George Gilbert (@ggilbert41) during DataWorks Summit in San Jose, California. They discussed previous struggles to adapt the Hadoop engine for business intelligence and recent promising results brought about through their most recent collaboration. (* Disclosure below.)

Attempted to support BI at Yahoo

Klahr previously worked with Paty when both executives were part of the central data group at Yahoo. Their mission at the time was to collect data and place it on a Hadoop cluster to support BI. But they were frustrated at having to continually shrink the data to manage it in a big data platform.

“It felt like we were doing it the old way. Moving data off cluster and making it small,” Klahr said.

This frustration was the genesis for the founding of AtScale, whose mission is to provide the enterprise with a fast, secure BI platform for big data. AtScale is now a customer of GoDaddy as the two companies work to develop new tools for managing BI on Hadoop.

“They are one of the first people to come out with a product that’s solving a real problem that we’ve tried to solve for a long time,” Paty said of AtScale.

In addition to building workload specific engines on top of Hadoop and realizing insights from the platform, Paty has also been working with Spark to perform ETL (extract, transform, load) tasks. “Doing ETL with Spark, as well as using Spark SQL for queries, is looking very promising lately,” Paty concluded.

Watch the complete video interview below, and be sure to check out more of SiliconANGLE’s and theCUBE’s independent editorial coverage of DataWorks Summit. (* Disclosure: AtScale Inc. sponsored this DataWorks Summit segment on SiliconANGLE Media’s theCUBE. Neither AtScale nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE