NEWS
NEWS
NEWS
LinkedIn open-sources its Groovy Gradle plugin for Hadoop
Hadoop has gotten itself another big-name contributor in the shape of business-oriented social network LinkedIn Corp., which has just made the LinkedIn Gradle Plugin for Apache Hadoop available on GitHub.
LinkedIn’s Gradle plugin was designed by the company to help it manage Hadoop jobs running on multiple frameworks, and its engineers have used it regularly while building data-driven apps.
LinkedIn engineer Alex Bain explained in a blog post that Gradle has been adopted as the company’s primary build system for the last couple of years.
“With Gradle, developers can easily extend the build system by defining their own plugins,” Bain wrote. “We developed the Hadoop plugin to help our Hadoop application developers more effectively build, test and deploy Hadoop applications. The plugin includes the Hadoop DSL, a domain-specific language for specifying jobs and workflows for Hadoop workflow managers like Azkaban and Apache Oozie.”
Gradle is written in the Groovy programming language, and its embedded DSL offers syntactic code constructs for working with jobs and workflows for Hadoop managers. This makes it possible for developers to use Groovy or Java anywhere throughout the DSL, Bain explained. The DSL simplifies the process of creating Azkaban or Oozie files, and because it’s statically complied, it can also be statically checked, he continued. This ensures developers can catch many of the common problems that crop up with workflow files at build time, as opposed to finding them later when running the actual Hadoop workflow.
LinkedIn is welcoming new contributors to the project, be it bug reports, documentation improvements, pull requests or any other ideas. Those interested can view documentation for the Gradle plugin here.
“The Hadoop Plugin and Hadoop DSL have been embraced as the standard way to develop Hadoop workflows at LinkedIn,” Bain said. “If you are writing Hadoop jobs using Gradle as your build system, you should definitely consider using the Hadoop Plugin. It will save you time and energy in developing your Hadoop workflows.”
Photo Credit: http://heatherbuckley.co.uk via Compfight cc
A message from John Furrier, co-founder of SiliconANGLE:
Support our open free content by sharing and engaging with our content and community.
Join theCUBE Alumni Trust Network
Where Technology Leaders Connect, Share Intelligence & Create Opportunities
11.4k+
CUBE Alumni Network
C-level and Technical
Domain Experts
15M+
theCUBE
Viewers
Connect with 11,413+ industry leaders from our network of tech and business leaders forming a unique trusted network effect.
SiliconANGLE Media is a recognized leader in digital media innovation serving innovative audiences and brands, bringing together cutting-edge technology, influential content, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — such as those established in Silicon Valley and the New York Stock Exchange (NYSE) — SiliconANGLE Media operates at the intersection of media, technology, and AI. .
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a powerful ecosystem of industry-leading digital media brands, with a reach of 15+ million elite tech professionals. The company’s new, proprietary theCUBE AI Video cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.
