Kyligence Inc., the developer of an analytics platform based upon the Apache Kylin open-source distributed data warehouse, today announced a new version of its self-tuning analytics acceleration platform.

The version 4.5 release features the addition of ad hoc exploration, an integrated ClickHouse open-source analytical database management system and support for real-time data sources.

Kylin is based on “data cubes,” a multidimensional representation of data that is commonly used in business intelligence. The software extracts data selectively from a database, performs computations and stores the results in distributed aggregate indexes that can be queried without requiring processing the full database.

Machine learning identifies frequently accessed datasets and creates a layer of pre-computed results that can typically satisfy about 95% of queries in under one second, the company says. Using a structure called segments, Kyligence updates changes to data without rebuilding the cube.

Kyligence Tiered Storage is a new cloud-native way to deploy, maintain and accelerate different types of queries with a single, multimodal analytics engine. It negates the need to deploy several different analytics engines to customer data pipelines, reducing both cost and maintenance overhead, the company said.

Ad hoc queries in seconds

The combination of ClickHouse and multitiered storage enables ad hoc exploration. Customers can query data that hasn’t been stored in a pre-computed cube. While results are returned more slowly than those against aggregate indexes, Kyligence said, it has reduced response times to queries that used to take 10 to 15 minutes to between 15 and 20 seconds.

“These are the kinds of questions we don’t have the bandwidth to pre-process, so while the response is a little slower we are able to answer relatively fast,” said George Demarest, senior director of marketing. “No query gets dropped and the worst case is that users get the full power of the back end.”

Kyligence provides a unified semantic layer that lets customers use their preferred visualization tools without needing to know the underlying computation and storage mechanism.

Real-time data support is added through integration with the Kafka distributed event streaming platform. Users can create queries that combine both streaming and historical data. The software processes messages continuously on a first-in, first-out basis, storing messages in the database as soon as they become outdated.

The new release also extends the semantic layer to support Microsoft Corp. Power BI data visualization system. Kyligence is also releasing a software development kit that customers can use to add their own data sources beyond built-in support for the Snowflake Inc. data cloud, relational databases and Amazon Web Services Inc.’s S3 data stores.

“It’s a bit of future-proofing that enables customers to expand the platform for their needs,” Demarest said.

Kyligence Cloud 4.5 is currently in beta test and will become generally available at the end of August on Microsoft Azure and AWS, with support for Google Cloud Platform scheduled soon after.

Image: Flickr CC