Cloud-based applications built with microservices are firehosing log data at DevOps teams. This is where open-source search-and-analytics engine Elasticsearch comes in for many teams. But Elasticsearch by itself doesn’t solve the problem of where to put all that data. Can a managed-service spin on Elasticsearch do the legwork for them?

The sheer amount of data is potentially great for analyzing app performance but also expensive to store and hard to get a handle on at times, according to Herain Oberoi (pictured), general manager of databases, analytics, and blockchain marketing at Amazon Web Services Inc. “[Users] either start to store that in archives or they don’t store it at all. If you store in archives, now you’ve got DevOps engineers and security experts that have to spend days to restore that data from the archive in order to search and analyze that data [with Elasticsearch],” Oberoi said.

Oberoi spoke with John Furrier, host of theCUBE, SiliconANGLE Media’s livestreaming studio, during the AWS Summit Online event. They discussed how AWS is addressing Elasticsearch’s storage weakness with smart caching. (* Disclosure below.)

Smart cache takes it in, Elasticsearch churns out new use cases

AWS has announced the general availability of UltraWarm — basically, storage-optimized Elasticsearch as a managed service. The main advantage for customers is that it introduces a distributed storage cache for frequently accessed data. It moves cold, or less frequently accessed, data to low-cost Amazon S3 storage.

The service allows DevOps teams to focus on high-level data search and analysis rather than time-consuming problems around storage, configuring and scaling. It costs about 80% less and executes queries about 50% faster than other managed ES services, according to Oberoi.

Photo: Herain Oberoi