Facebook open-sources Haxl: Big Data Haskell library

facebook-haxlYesterday, Facebook announced the open-sourcing of Haxl, a Haskell library used to greatly simplify access to remote data, such as databases and API web-based services. This release continues a trend of Facebook’s strong bent towards data science and the processing of complex datasets, and at the same time sustains a commitment to the open source community.

The Haxl project (the combination of Haskell + FXL) at Facebook provides an abstraction layer to access the various pieces of content at the ready. Haxl is as an embedded domain-specific language implemented in Haskell. In other words, Haxl is implemented in a Haskell framework that is used by application programmers to develop Haskell programs that access the various Facebook components via a uniform API.

The Haxl project is the brainchild of former Glasgow Haskell Compiler lead Simon Marlow.

Haxl provides clean coding mechanisms for writing business logic (filter rules, queries, etc.) that act on remote data. Along with the announcement, Facebook posted a blog article that outlines Haxl’s capabilities and the logic behind its implementation for developers to understand.

As an example, Facebook currently uses FXL (Haxl’s “context parser” parent) as part of a spam and malware detection system. FXL allows for the construction of context-sensitive rules that can interrelate and return answers based on a set of facts. Such as “Is this URL associated with malware?” where FXL responds with “Yes, block this” or “No, carry on.”

With Haxl, developers get the power and reliability of the Haskell programming language with a library that allows them to access the context and interrelation strengths of FXL.

Facebook’s Big Data history

As a social media company, Facebook is swimming in an ocean of user generated data. Every post, image, interaction, and like on Facebook is a data point and with 1.3 billion monthly active users (as of January 2014) that’s a lot of interrelations. The result: Facebook has a strong history with Big Data and Haxl yet another milestone.

In her article about Facebook’s Big Data story, Kristen Nicole mentions that Facebook is a prime target for Big Data solutions. Solutions such as Facebook’s specialized search strategy: the Graph Search. The Graph Search is another invention of Facebook designed to allow for filtering a group of users by specified criteria contextualized by relationships between users.

Haxl fits nicely into these other developments from Facebook in that it’s a library that abstracts away the filtering process of contextualizing interrelations in datasets.

Facebooks commitment to open source and developers

By open sourcing Haxl, Facebook is essentially letting it loose into a broader community of developers. As a result, developers will be able to work on their own repo of Haxl, see the github page for code and documentation, and implement it in their own software.

Facebook has been a strong proponent of the open source community and its benefits. Haxl is being added to growing stable of Facebook released open source projects. By the end of 2013, Facebook already had in excess of 90 repos on github.

Contributors: Kyt Dotson and Saroj Kar.

About Kyt Dotson

Kyt Dotson is a Senior Editor at SiliconAngle and works to cover beats surrounding DevOps, security, gaming, and cutting edge technology. Before joining SiliconAngle, Kyt worked as a software engineer starting at Motorola in Q&A to eventually settle at Pets911.com where he helped build a vast database for pet adoption and a lost and found system. Kyt is a published author who writes science fiction and fantasy works that incorporate ideas from modern-day technological innovation and explore the outcome of living with those technologies.