Big Data holds huge potential for enterprises looking to glean new business insights, but that potential can only be fulfilled if it’s possible to analyze said data. The problem is that a huge deal of enterprise data remains inaccessible, either because employee don’t know how to access it, or don’t know it even exists.

Enter Teradata Corp. and Australian data indexing firm Nuix Pty Ltd, which have just teamed up to build a new service that aims to shed some light on that so-called “dark data”.

According to Teradata, its Aster Analytics engine is far more capable than Nuix, but the latter company possesses a far superior data extraction and indexing engine. So the logical solution is to combine the two platforms together, giving enterprises the ability to analyze to previously impenetrable data in order to glean hitherto unknown insights.

Aster Analytics is a popular tool that leverages text and pathing analytics, and machine learning, in order to extract information from large repositories of raw data. Meanwhile, the lesser-known Nuix provides a data extraction and indexing tool that’s able to extract searchable text and metadata from a vast range of different source types, including deleted files, PDFs, massive email databases, and even data preservation tools such as NetApp Inc.’s SnapLock and EMC Corp.’s Centera.

Teradata and Nuix’s solution is this: Nuix’s software does the dirty work, digging around to discover the dark data residing in all the nooks and crannies of an organization’s IT infrastructure – much like it did with the Panama Papers – and then Aster takes over, targeting and extracting the most useful stuff and gleaning insights from it.

For now, the two companies say they’re going to create a communications compliance product for financial services companies, but later they’re hoping to create tools for a much broader range of uses, such as cybersecurity, business intelligence, litigation and information governance.

“For example, Aster’s sophisticated analytics can look for words and phrases and then apply sophisticated analytical business rules to find compliance infractions or fraud,” said Randy Lea, Americas Vice President, Business and Analytics Practice, Teradata Aster. “A human-driven investigative workflow would then leverage the results to audit the flagged documents to determine what actions to take. This example alone brings powerful new analytic techniques to investigate compliance infractions and fraud that cost companies millions of dollars per year in direct costs and fines if they go undetected.”

Image credit: ColiN00B via pixabay