Faster is better. That’s as true in big data as anywhere else in business. Unfortunately, data is heavy, slow and expensive to move. The most efficient play is to process data at the source, but that’s not always possible.

A more practical solution is to cut travel times and bandwidth costs by dropping the processing as near to the data source as possible, according to Yaron Haviv, founder and chief technology officer at iguazio.

“Everyone is now looking for real-time insights,” said Haviv.

He spoke to John Furrier (@furrier) and George Gilbert (@ggilbert41), co-hosts of theCUBE, SiliconANGLE’s mobile live-streaming studio, during the BigData SV 2017 conference in San Jose, CA. (*Disclosure below.)

During the interview, Haviv discussed new technologies to increase the speed of data between the cloud and the edge.

Trends, data and the cloud

If a company wants to put its data to work in the real world, it needs the insights from that data in real-time. Haviv spoke about how collecting data, storing it and then processing over a few hours simply will not work.

His company, iguazio, created a mechanism for not just pulling in data, but also running processes on it while the data is being collected. He described the process as taking in data like a stream, but the company can issue queries on it. This data can be served immediately through APIs to a dashboard.

One of the challenges the company solved to make this tech work was resolving the issue of processes colliding while updating data. “We restructured everything and didn’t take anything for granted with our architecture,” Haviv said.

He then explained what set iguazio on its current path. The company’s position offered a very unique point of view, one that let it see the cloud’s trajectory, Haviv explained. It followed the way the cloud went, while using its infrastructure skills to improve as they went.

In the face of so much data, cloud has to start distributing more, Haviv stated. People are going to push a lot more bandwidth along the edge. To handle this, a company will need something like a mini-cloud that sits somewhere between the cloud proper and the edge.

“This is a play we’re going to take,” he said.

Watch the complete video interview below, and be sure to check out more of SiliconANGLE’s and theCUBE’s coverage of BigData SV 2017. (* Disclosure: Some segments on SiliconANGLE Media’s theCUBE are sponsored. Sponsors have no editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE