UPDATED 14:19 EDT / MAY 10 2017

BIG DATA

Elasticsearch engine gets machine learning capabilities

Elasticsearch Global BV, which does business as Elastic, has added adding machine learning capabilities to its Elastic Stack collection of open source products for searching large databases of unstructured information. The company picked up the technology via its recent acquisition of Prelert Inc., a behavioral analytics vendor.

Elastic said the first iteration of the new machine learning features can be used to automatically identify anomalies, perform root cause analysis and reduces false positives within real-time applications. The features initially will be focused on time-series analysis, such as identifying anomalies in web traffic or abnormal application response times.

Elastic Stack combines a suite of products that were previously offered separately, including the Kibana visualization engine, Beats data shipper, Logstash log data manager and Elasticsearch for Apache Hadoop. The stack also includes proprietary extensions for security, alerts and monitoring. The entire suite is available as a free download or cloud service, with options for subscription-based support.

The company said Elasticsearch is the most popular data store that’s not a database, with 100 million downloads recorded. Google Inc. recently gave the platform a lift by adding a fully managed version to the Google Cloud Platform.

Unsupervised machine learning algorithms continually sift through piles of uncategorized data to identify patterns (pictured), which are then presented to a human analyst. Meaningless patterns are discarded and interesting ones fed back to the algorithm for refinement.

“It will automatically learn what’s normal across and period of time and use that information to detect what’s anomalous,” said Steve Kearns, product lead and product manager for Prelert. “We don’t have to understand up front what the anomalies are. The machine adapts can becomes self-maintaining.”

Elastic is attempting to make the arcane world of machine learning more accessible by making the algorithms programmable from within Kibana, an open-source visualization engine. “One of the key problems with machine learning is you have to be an expert to use it,” said Steve Dodson, who built the technology at Prelert. “We want to empower people so that if someone has the skills to create a dashboard they will be able to define and set thresholds.”

The machine learning capabilities are proprietary code that will be enhanced over time to permit multi-factor analysis and integrate with other analytics engines. The product is shipping as part of an Elastic X-Pack, which includes other open-source and proprietary products. Pricing was not announced. Additional detail about machine learning features and applications is in this blog post.

Photo: Elastic

A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU