Companies gearing up to make a big data play need to be careful they don’t simply spray bullets in every direction and hit everything but their target. According to some, many companies are seriously misallocating resources in their efforts to profit from data. The imbalance lies in the data pipeline worker bees versus decision makers in IT who are being underfed data intelligence, according to Eric Sammer, CTO and co-founder of Rocana Inc.

“Most organizations will devote an entire team to managing — you know, if you’re an ad-serving company or something like that — you have a data pipeline team that manages clicks and impressions and all this other kind of stuff,” Sammer said. “And then you have, like, one lone gunman who’s responsible for doing integration with 30 or 40 or 100 different systems.”

Sammer told Peter Burris (@plburris), host of theCUBE, SiliconANGLE Media’s mobile live streaming studio, that the dearth of talent higher up the chain is especially problematic at Fortune 100 companies that may have hundreds of thousands of systems.

Sammer said that this model has never really made sense, and it is increasingly untenable for companies that want to expand their digital presence. “We sort of acknowledge first class that the business end has resources and requires these resources — and then we starve IT,” he said. “And it’s a really interesting thing to think about because IT is building the systems that support the rest of the business.”

Sammer argued that IT staffers need much greater access to data intelligence and resources in order to build those systems to a higher standard to meet the digital demand.

“We really need to give operations people — DBAs, network admins, cyber liability engineers, DevOps, whatever you want to call them — the ability to go extremely deep and go back to the original sequence of events that led up to a particular incident,” he said.

What IT does not know can hurt them (and their employers)

Rocana CEO Omer Trajman also joined in on the interview, and he spoke about how keeping IT people siloed and isolated from business intelligence can hurt a company.

“IT is only responsible for a small fraction of the web experience,” he said. “A lot of it — because it’s static images — is actually run through a CDM [clinical data management], which means they don’t know the end customer’s experience.”

He explained the actual process IT teams must undergo to get information on what customers actually experience digitally. “They don’t know what was served, they don’t know the latency behind it, they don’t know the order the stuff is loaded on the page or what gets loaded on the page until a week later when the CDM provider ships them over FTP a zip file,” he said.

How data talks to humans

Sammer explained how Rocana 2.0 attempts to plumb the deep levels of the data center and deliver intelligence to IT.

“We think of the world as made up of events, whether they are log messages, log events, metric samples, business transactions — like a user started or abandoned a shopping cart,” he explained.  “All of these things are modeled as events, so we have a single way of thinking about that data.”

Sammer said this language of events allows them to understand data relationships and produce actionable insights. IT teams are then able to “take action on this mass volume of event data for things like spinning up VMs in response to increased load or inserting or updating firewall rules,” he said. “All of that really comes down to complex event processing, which underpins the Rocana reflex system,

He explained that the reflex system enables simplistic things like messaging and more complex tasks like advanced cloud orchestration; and at the very top sits a layer of advanced analytics.

Crawling, walking, running — and then sprinting

Sammer and Trajman said Rocana 2.0 can take companies from crawling to walking to running in the big data realm. Is there a next step we should look out for?

“AI is sprinting,” said Sammer. “The next generation is going to be not about automating humans out of managing the data center; it’s going to be helping them to understand this massive haystack when they’re looking for that one tiny needle.”

He explained that AI will give IT sharper, clearer intelligence on data to help them make better business decisions.

Watch the full interview below:

Photo by SiliconANGLE