UPDATED 08:00 EDT / JANUARY 21 2021

BIG DATA

Kyligence’s Apache Kylin-based distributed data warehouse gets cloud-native features

Kyligence Inc., developer of an analytics platform based upon the Apache Kylin open-source distributed data warehouse, today released the fourth generation of the cloud version of its analytics platform.

Kyligence Cloud 4 has cloud-native features such as independent storage and compute scaling and support for object storage. The new version uses pre-computation and machine learning to create an aggregation layer that the company says can deliver subsecond query response times even against petabyte-sized databases.

The technology is rooted in the “data cube” concept that dates back to the early days of online analytical processing. It extracts data selectively from a database, performs computations in advance and stores the results in distributed aggregate indexes that are similar to the multidimensional cubes long used in business intelligence applications. Queries can then be run against the cubes without taxing the resources of the full database.

“We do the computation once and then create these cubes that you can query any number of times,” said George Demarest, senior director of marketing. “Users don’t know Kyligence is there; we just make performance go up to 10 times faster.”

Kyligence uses machine learning to identify the most frequently accessed datasets based on query history, usage patterns, data profiles and runtime metrics and creates a layer of precomputed results that can typically satisfy about 95% of queries in under one second, the company said. In addition to improving performance the technique also greatly increases the number of users who can perform analytics processing concurrently.

The technology translates the rows and columns of relational databases into keys and values. Users can perform the equivalent of tabular joins using keys and thus avoid the need for additional calculation. The new version of the cloud service stores computed results in cloud object storage.

Whereas traditional BI workloads often slowed to a crawl because of scalability limitations, “in the cloud we leverage distributed storage for much larger data cubes,” said Li Kang, head of North America operations at Kyligence. “The engine will look at input and every time you fire off a new query it analyzes it and then continuously optimizes the model design.”

The Kylin project “was initially tied to Hadoop but we have removed all those dependencies,” Demarest said. The new version is based on an Apache Spark cluster and optimized for object storage. “You can take advantage of slick cloud provisioning and install in minutes,” he said.

The semantic model provides transparent access from Microsoft Corp.’s Excel, SQL clients, popular BI tools and representational state transfer or REST application programming interfaces. Supported back-end data stores include Snowflake Computing Inc.’s Data Cloud, Amazon Web Services Inc.’s S3, Microsoft’s Azure Data Lake Storage and Azure Blob Storage.

A unified semantic layer provides a common business context so data analysts don’t need to implement business definitions such as customers and orders or technical definition such as tables and columns for each BI platform. Calculation logic is also independent of the BI toolset.

Kyligence Cloud 4 is immediately available on AWS and in the Azure Marketplace.

Image: Flickr CC

A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU