UPDATED 13:32 EDT / APRIL 17 2012

NEWS

14 Companies Looking for “Big Data Engineers”

Do a search on Indeed.com for “big data engineer” and you will find 26 listings. That compares to ten job listings for “big data architects.”

I became curious about this new job description when I saw a job post from Disqus.

Then I looked at the trend line.

It’s revealing to look at the companies seeking big data engineers.  Employers are looking for people who know MapReduce, Hadoop and related franeworks such as HBase, Pig and Hive. Programming languages in demand include: Java, Ruby, or C++. It really covers the gamut, which is part of the issue with using big data in a job description. Do you have MongoDB expertise? Yes, that’s applicable. Practical hands-on experience with Bayesian models and neural networks? Yes, the job may be right for you.

I found startups that focus on serving the advertising industry and financial services companies.  Consulting companies and technology giants are all using the new title.

Here they are:

  1. Amazon (6)
  2. The Climate Corporation (3)
  3. Greythorn (3)
  4. Unlisted Company (2)
  5. Jobspring Partners (2)
  6. Behance, Inc. (2)
  7. Elevate Recruiting Group (2)
  8. Krux Digital (2)
  9. Beanstock Media (1)
  10. Tribal Technologies (1)
  11. Kontagent (1)
  12. Discovery Staffing (1)
  13. Convertro (1)
  14. Disqus (1)

A few job descriptions to give a flavor of what employers are seeking:

A9.com: The Amazon subsidiary is looking for a big data engineers to:

  • Design, develop and support a map-reduce-based data aggregation pipeline for processing billions of events a day
  • Support data-mining and machine-learning algorithms using behavioral data
  • Study state of the art techniques in massively parallel frameworks and apply them to advertising problems
  • Help other engineers get the most out of the platform you own

Climate Corporation: This San Francisco-based company helps companies adjust to climate change. I like their bonus points:

Experience with Lisp and/or Clojure (functional programming languages)
Experience with large-scale machine learning techniques (examples: Google PageRank, Netflix Prize, genome sequence assembly, computational finance)
Experience with Amazon Web Services (EC2, S3, SQS, etc.)
Deep knowledge of the Hadoop ecosystem
Git version control
Frequent contributor to open source projects (show us your work on github!)

Diffbot is a visual robot for the web that can can “see” web pages like humans do. It was the inaugural startup out of Stanford’s StartX Accelerator program.

Some of the benefits:

…and if this is what you want to be doing with your day:

Building massive training data sets and mining them for product insights.
Benchmarking and optimizing performance of classifiers.
Learning and developing new algorithms for classification and training.
Building, testing and deploying code to our hybrid cloud architecture.
Working directly with the founding team.
Whatever else it takes to help Diffbot understand the web.

What do you think of the “big data engineer,” title?


A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU