NEWS
NEWS
NEWS
Hortonworks’ YARN Aims to Revolutionize Hadoop Data Processing
Hortonworks is publishing a series of blog posts on its website that explain the basics and finer details of Apache Hadoop YARN. Those who are curious about YARN or want to understand its significance to Hadoop will find the blog posts beneficial. Furthermore, it should be of benefit to those who might not have even heard of YARN but who are looking for alternatives to the classic Apache Hadoop MapReduce framework.
The first post gives a background and overview of MapReduce: what it is supposed to accomplish, how people have used it, and ultimately how YARN aims to improve upon it. According to Hortonworks, Apache Hadoop MapReduce is in need of an overhaul, most notably with the JobTracker. YARN addresses issues users have faced by separating JobTracker (resource management) and job scheduling/monitoring.
According to some, Apache Hadoop is not ready for enterprise deployment because there is no method of processing data other than MapReduce. For batch data processing, MadReduce is an ideal choice, but for many, the non-batch functions, such as real-time data processing and graph processing need more than what classical MapReduce can provide.
YARN, as described by Hortonworks in post 2, is a system for managing distributed applications. The components include a ResourceManager and a NodeManager. It also includes an ApplicationMaster, a tool for coordinating resources between the ResourceManager and the NodeManager.
In post 3, Hortonworks delves into more details regarding ResourceManager, complete with diagrams breaking down the role of the component and how it interacts with others. Most of this post gets into very technical terms, but it will be helpful for those trying to understand the underlying technology behind YARN.
Hortonworks has made significant strides in the big data arena and is one of the top contributors to Apache Hadoop. It is one of several companies, including Cloudera, vying for a top seat in the Hadoop development sphere. YARN, which has been called the Next-Generation MapReduce, could help Hortonworks reach a larger segment of the big data market, including those reluctant to try Hadoop in the past.
A message from John Furrier, co-founder of SiliconANGLE:
Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.
- 15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more
- 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network.
About SiliconANGLE Media
SiliconANGLE Media is a recognized leader in digital media innovation, uniting breakthrough technology, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — with flagship locations in Silicon Valley and the New York Stock Exchange — SiliconANGLE Media operates at the intersection of media, technology and AI.
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.
