SigScalr Inc., developer of an open-source observability application designed to process large volumes of data, is emerging today with $1.76 million in pre-seed funding.

Founded by a former Salesforce Inc. observability engineer, the company’s Siglens platform is a columnar analytical database that is said to execute queries more than 50 times faster than Apache Clickhouse and over 1,000 times faster than Elasticsearch. Both are commonly used for log analytics.

The secret is what founder and Chief Executive Kunal Nawale called “micro-indexing,” in which an index is attached to each column. Dynamic compression adjusts as immutable data streams in.

“In traditional databases, when you search for a value in a column, you search the column and put an index in front of it,” he said. “That index gets really large over time. Our index uses compression on the column, and the micro-index is 1/100th the size of a conventional index.” The effect is to break columnar files into smaller segments which can be searched more quickly. Performance engineers need to uncompress only 2% of data during a search.

The tradeoff is that the technology is append-only, meaning that loglines can’t be changed retroactively. However, Nawale said that isn’t a problem in an observability context because “you never edit the log line.”

Highly scalable

In addition to the claimed speed benefits, SigLens is said to be highly scalable, supporting thousands of concurrent queries on terabytes of data with sub-second response time. It can be run on-premises to avoid cloud data egress fees or as a software-as-a-service application. Its query language is compatible with every major observability tool.

“As data volumes grow above 1 terabyte a day, you can get to paying $15,000 a month on egress fees,” Nawale said. “We let you run inside your network so you don’t incur those costs.” He expects most users to opt for the on-premises version.

Nawale said he got the idea for SigLens when he was working at Salesforce and using a popular big data platform used for collecting, searching and analyzing machine-generated data. “The vendor had no incentive to make the software more efficient,” he said. “We were spending several tens of millions of dollars on hardware.”

Most observability platforms use older, general-purpose analytical databases, he said. “We are solving this problem specifically for observability use cases.”

Siglens was released to open source just a few weeks ago. SigScalr expects to drive most of its revenue from its hosted version of the product and from advanced security features included in the enterprise edition.

Scribble Ventures Management LLC led the round with co-investments from WestWave Capital LLC and Forward Slash Capital.

Photo: Unsplash