Many companies are amassing big data to gain a competitive advantage with predictive analytics. What happens when the data sets get awkwardly huge and hard to manage? No problem — they just bring in special tools for processing and protecting it.

Actually, this tool glut can become a big problem, according to Christian Beedgen (pictured), co-founder and chief technology officer of Sumo Logic Inc. The key thing about processing big data, challenging in its own right, is that it creates even more data, he explained. This might include data from logs, telemetry and tracking, to name a few. Companies then face the question of what to do with it all.

“These data sets become their own big-data nightmares, potentially,” Beedgen said. “At the same time, they’re full of really useful information to maintain availability, performance, to secure your systems and so forth.”

Storing, processing and monitoring all this data could require a truckload of Apache Hadoop clusters and on-premises infrastructure. Or it could be done much more simply with infrastructure and monitoring as a service. That latter approach is Beedgen’s recommendation.

Beedgen spoke with John Furrier (@furrier) and Dave Vellante (@dvellante), co-hosts of theCUBE, SiliconANGLE Media’s mobile livestreaming studio, during the AWS re:Inforce event in Boston. They discussed the latest advancements in infrastructure and security for big data (see the full interview with transcript here). (* Disclosure below.)

One-button security still a fantasy

Sumo Logic was born in the cloud and relies on Amazon Web Services Inc. infrastructure and APIs to render ready-to-use services. APIs allow users to make light work of heavy operations, according to Beegden.

“An API is something I can automate. It’s fundamentally a very powerful abstraction that allows one guy to do the work of potentially hundreds of people running around checking network connections,” he said.

Beegden lauds AWS’ latest attempts to bake data and network security into its cloud. However, users hoping for “one-button” security will find that we are still not there yet, he added.

Also, artificial intelligence that will stand in for security pros will not likely appear anytime soon. Algorithms and analytics can help send alerts and signals. “But in the end … it ends up in somebody’s queue, and then they’ve got to burn through it,” he said.

For now, users still need build a framework for processing, monitoring and securing data Lego by Lego, according to Beegden. The point is to make sure they are the lightest, easiest to manage Legos around.

Watch the complete video interview below, and be sure to check out more of SiliconANGLE’s and theCUBE’s coverage of the AWS re:Inforce event. (* Disclosure: Sumo Logic Inc. sponsored this segment of theCUBE. Neither Sumo Logic nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE