UPDATED 09:51 EDT / APRIL 27 2012

Big Data’s Talent Warehouses May Be In Short Supply

In “Big Data’s Big Problem: Little Talent,” an article published yesterday, the Wall Street Journal’s Ben Rooney used the success of the Splunk IPO to discuss how quickly the big data market has exploded – and how the talent pool hasn’t caught up yet.

Most of what Rooney talks about should be familiar to our readers. The act of gathering so much data is great, but a new type of professional – the big data scientist (or engineer or architect) – is required to help make sense of the chaos and turn it into more actionable business intelligence. Nothing new there.

But when Rooney describes his conversations with top-level industry executives on the problems facing the field of big data for some context, things get interesting.

“IBM started a generation of Cobol programmers. Thirty years ago we didn’t have computer-science departments; now every quality school on the planet has a CS department. Now nobody has a data-science department; in 30 years every school on the planet will have one,” EMC Corp. President and COO Pat Gelsinger told Rooney.

There’s something to that parallel: Just this past March, Frank Coleman, another EMC executive, wrote about how many data scientists may be “hiding in the shadows,” outside the traditional IT structure. That’s simply because it takes a lot of skills that, historically, haven’t been mission-critical to the IT hiring process. A data scientist needs to combine database skills and the hacker mindset with an understanding of the business and experience with complex analytical models.

Finding one or the other out there can be tricky, but as an integrated field of study, well, we’re only seeing the tip of the iceberg. President Obama’s administration has announced a $200 million Big Data Research and Development Fund that’s priming itself to help close the education gap (among other things), but in the meanwhile, it’s touch-and-go.

The Journal article quotes a few IT recruiting managers and headhunters dealing with the big data hiring problem, who say that they’re only figuring out now how to suss out the best candidates. The only consensus on what makes a good data scientist is that there is no consensus, save perhaps that familiarity with the data is as important, if not more so, than mathematical ability.

Of course, there’s a dissenting opinion: HP apparently believes that tools like its recently-acquired Autonomy suite can help companies take advantage of big data and analytics with smarter algorithms, not the support of a dedicated professional.

But, as a recent Wikibon report shows, for those who can find work as a data scientist, the rewards are out there, with salaries ranging from $60,000 to $115,000.


A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU