NEWS
NEWS
NEWS
Facebook’s new compression algorithm could be a game-changer for data storage
Compression is front and center at Facebook’s annual @Scale conference in San Jose. The social networking giant open-sourced a homegrown algorithm called Zstandard at the event today that is described as faster, more efficient and easier to use than many of the most popular alternatives in use today.
One of the rival compression technologies that Facebook singled out in its release announcement is Zlib, a free library originally created over 20 years ago that is employed by Git, PostreSQL and thousands of other applications. The project is based on an algorithm known as DEFLATE that in turn enjoys even broader adoption in the software world. According to the social networking giant, Zstandard sets itself apart by taking advantage of compression techniques and CPU capabilities that weren’t available two decades ago.
Facebook says that its algorithm can provide a slightly higher compression rate than Zlib while working five times faster when shrinking files and twice as fast when decompressing them. Moreover, the algorithm provides the ability to trade data density off for increased performance and vice versa depending on the use case with 22 levels of granularity. As a result, a company that wants to make the most out of its storage equipment could dial compression rates all the way up, while a firm interested in speeding up data management operations could do the opposite and focus on maiming performance. But most of the organizations that will end up adopting the algorithm can be expected to fall somewhere in between.
Zstandard made its debut alongside a new open-source called MyRocks that makes it possible to use Facebook’s speedy RocksDB storage engine with the popular MySQL database. The social networking giant claims that its internal implementation has helped reduce storage requirements buy up to 50 percent in some cases, a massive decrease that should get the technology a lot of attention in the database space.
The code for MyRocks and Zstandard is available immediately on Faceboook’s GitHub page.
Image from Facebook
A message from John Furrier, co-founder of SiliconANGLE:
Support our open free content by sharing and engaging with our content and community.
Join theCUBE Alumni Trust Network
Where Technology Leaders Connect, Share Intelligence & Create Opportunities
11.4k+
CUBE Alumni Network
C-level and Technical
Domain Experts
15M+
theCUBE
Viewers
Connect with 11,413+ industry leaders from our network of tech and business leaders forming a unique trusted network effect.
SiliconANGLE Media is a recognized leader in digital media innovation serving innovative audiences and brands, bringing together cutting-edge technology, influential content, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — such as those established in Silicon Valley and the New York Stock Exchange (NYSE) — SiliconANGLE Media operates at the intersection of media, technology, and AI. .
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a powerful ecosystem of industry-leading digital media brands, with a reach of 15+ million elite tech professionals. The company’s new, proprietary theCUBE AI Video cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.
