Gone are the days when data quality used to be a back-office function. Today, data is the lifeblood of enterprises.

Since the variety and volume of data has gone through the roof based on increased usage, enterprises now rely on the freshness of data. Collibra NV equips individuals loading and consuming a company’s data with observability tools for tasks such as root cause analysis, which is needed to capture the bigger picture and ensure enhanced quality, according to Kirk Haslbeck (pictured), vice president of data quality at Collibra.

“We’ve always covered data quality, and we believe that people want to know more. They need more insights, and they want to see break records and breaking trends together so they can correlate the root cause,” Haslbeck said. “So we’re really focused on root cause analysis, business impact connecting it with lineage, catalog and metadata. And as that grows, you can actually achieve total data governance.”

Haslbeck spoke with theCUBE industry analyst Dave Vellante at  “Data Citizen’s 2022”, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed why data quality has become a burning issue in the enterprise world and how observability fits into the picture. (* Disclosure below.)

Native database pushdown

By enabling functions like compute in databases like Snowflake, BigQuery, Databricks, Delta Lake, and SQL Pushdown, this takes data intelligence and governance a notch higher, according to Haslbeck. This is made possible through native database pushdown.

“While we’ve always worked with the same databases in the past, we’re doing something called native database pushdown, where the entire compute and data activity happens in the database,” he stated. “We’re now doing all compute and data operations in databases like Snowflake. And what that means is with no install and no configuration, you could log into the Collibra Data Quality app and have all of your data quality running inside the database.”

For better decision-making, having the correct data is a game-changer, Haslbeck pointed out. Collibra helps meet this objective through data intelligence and governance.

“If you and I were going to build a new healthcare application and monitor the heartbeat of individuals, imagine if we get that wrong, what the ramifications could be?” he asked. “With the acquisition of what was a lineage company years ago and then my company OwlDQ — now Collibra Data Quality — Collibra may be the best positioned for total data governance and intelligence in the space.”

Data observability coupled with the right scale has the capability of making alerts whenever trends are being broken. This explains why root cause analysis comes into play, according to Haslbeck.

“What’s been so exciting is we have these types of observation techniques; these data monitors that can actually track past performance of every field at scale,” he said. “So we’re sort of shifting away from this world of must write a condition and then when that condition breaks, that was always known as a break record. But what about breaking trends and root cause analysis?”

Merging data observability and quality helps tackle the issue of stale data, therefore, making the timeliness and freshness of data a possibility, according to Haslbeck.

“It all points to the same idea of the thing that you’re observing may not be a data quality condition anymore,” he noted. “It may be a breakdown in the data pipeline. And with thousands of data pipelines in play for every company out there, there’s more than a couple of these happening every day.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of the Data Citizens 2022 event:

(* Disclosure: TheCUBE is a paid media partner for the Data Citizens 2022 event. Neither Collibra NV, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE