The U.S. could face a shortage of up to 190,000 workers with analytical skills by 2018, according to an ominous 2011 McKinsey study that hardly ever seems to go unmentioned whenever data science is involved. But as the deadline to bridge the skills divide edges closer, it’s becoming increasingly clear that the gap is shrinking – and at accelerating rate – thanks the growing number of vendors acknowledging accessibility and simplicity as the key factors in putting the power of Hadoop at the hands of ordinary end-users.

RedPoint Global, a Massachusetts-based firm led by former Accenture partner Dale Renner, recognized that at the very outset. The company has quietly emerged at the forefront of the industry to democratize analytics, snagging $6.4 million from investors and building up a customer base numbering in the hundreds,  including global enterprises like AIG and Equifax.

In 2012, RedPoint made its first acquisition, a developer of information quality and management software called DataLink with whom it had previously partnered. After the deal, the firm’s offering was relaunched as RedPoint Data Management under the promise of making it easier for marketers to manipulate data about their audiences. Now, it’s launching for Hadoop 2.0.

Since coming into the company’s possession, the tool has evolved from something of a point solution to what is being described as full-fledged platform for ingesting, filtering and processing unstructured information. It provides a graphic interface that the company says allows users to manipulate their data and build models without having to do any coding in the notoriously complex MapReduce batch processing component of Hadoop.

The newly launched RedPoint Data Management for Hadoop does away with MapReduce altogether in favor of the YARN resource management and scheduling technology that replaced it in the 2.0 release of the analytics framework. The project brings several major benefits to the table, most notably support for a broader range of data processing schemes and interaction patterns as well as the ability to run different versions of Hadoop in the same cluster, which simplifies administration.

RedPoint pegs its offering, which was one of the first applications to be certified for use in environments running the latest version of Hortonworks’ distribution, as the first “pure YARN data quality and data integration product on the market”. But while it may be breaking new ground, the company is still facing a tough task in competing against other Hadoop ETL providers like Syncsort, which has also been making gains lately. The vendor recently debuted a new version of its Ironcluster solution that can be deployed directly on EC2 instances without using Amazon’s Elastic MapReduce provisioning service.

photo credit: Marius B via photopin cc