An Atlanta-based startup is tackling the IT complexity problem with a big data approach, hoping partnerships with scores of point solution providers can yield unprecedented insight on the source of performance problems.

OpsDataStore is a data ingestion layer and analytics engine that grabs information from a wide variety of operating systems, virtualization platforms, systems management utilities and performance monitors and stores it in a Cassandra NoSQL database for analysis using the Apache Spark analytics engine and Apache Kafka messaging bus. The result is a “continuously updated topology map of the entire environment over time,” the company said. CEO Bernd Harzog called it a “common data back-end for all management software.”

The tricky part of what OpsDataStore is doing is getting the data in the first place. The company needs to convince other platform providers to map their configuration and reporting data to its integration layer, and there are potentially hundreds of sources. The current Tower of Babel of systems information is what makes management so difficult, Harzog said.

“All management data comes to you organized differently,” he said. “Everybody has different metadata and collects different information about a server. We rationalize those things so everything gets transformed into objects in our object model.”

In most cases, it will be up to the information provider to map its data to OpsDataStore’s object model. The company is launching with five partners – VMware Inc.’s vSphere, ExtraHop Networks Inc., AppDynamics Inc., Dynatrace LLC and Mekyska Management Consultants GmbH – and is in discussions with more than 15 others. However, the total ecosystem of potential data sources is much larger.

OpsDataStore is attempting to make the partnering process as simple as possible using a software development kit and published application program interfaces. The incentive for other vendors to partner with the company is “to make their data more useful,” Harzog said. “We combine their data with data from other tools and all of a sudden their customers can see things they’ve never seen before.”

The benefit of harmonizing so many data sources is in providing a single view of all of the virtual and physical resources that affect an application, along with the ability to drill down into each information source to more easily pinpoint problems. Users can view their infrastructure by application, server, virtual machine and individual resource and click through to check response times for each individual application. “No one has been able to put together those factors,” Harzog said. The company is partnering with Qlik Inc. and Tableau Software Inc. for data visualization.

The primary benefits are faster time to resolution and improved resource optimization. Harzog noted that many data centers operate at sub-20 percent utilization rates because their blanket solution to performance problems is to add hardware and bandwidth. “Owners don’t have any idea what’s impacting response time and throughput, so they throw more resources at spikes,” in activity, he said.

The software can be downloaded from the OpsDataStore website. It installs in a dedicated VMware instance and is priced by virtual machine at $100 per operating instance per year.