UPDATED 10:55 EDT / JULY 12 2013

Netflix’s Jeff Magnusson on Big Data Visualization

Netflix is not only the world’s largest provider of streaming media, but also one of the most prominent contributors to the Hadoop ecosystem. Jeff Magnusson, Netflix’s head of Data Platform Architecture, discussed the open-source technologies his company uses to keep track of consumer trends in an interview with SiliconANGLE NewsDesk Head Writer Winston Edmondson.

Winston starts the session by asking about Netflix’s famed corporate culture. Magnusson says that the firm is constantly on the prowl for intelligent people, including analysts and other professionals who don’t necessarily possess advanced technical skills. Netflix’s engineers are tasked with developing tools that can help their less tech-savvy colleagues gain insights from data without concerning themselves with the underlying technologies.

Magnusson highlights that a lot has changed since the days of DVD deliveries: back then, most consumers didn’t interact with Netflix after ordering titles. Today, viewers generate so much data that members of his team have to use custom visualization software just to keep up.

Netflix utilizes Apache Pig for recommendations, video optimization and other important workflows. Pig syntax is highly abstracted, Magnusson explains, but raw code is often difficult to understand due to the complexity of certain scripts. He tells Winston that Netflix solved this problem with a visualization tool called Lipstick. The homegrown program transforms code into directed acyclic graphs, or DAGs, that make it easier to spot bugs in large projects. These charts also enable developers to view MapReduce jobs as they execute.

Mangusson says that Netflix’s appetite for data will continue to grow as it expands to new markets. He predicts that real-time technologies such as Storm will shape the future of the Big Data ecosystem and revolutionize user experience in the process.

Check out the video below for the full interview.

 


A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU