

Compression is front and center at Facebook’s annual @Scale conference in San Jose. The social networking giant open-sourced a homegrown algorithm called Zstandard at the event today that is described as faster, more efficient and easier to use than many of the most popular alternatives in use today.
One of the rival compression technologies that Facebook singled out in its release announcement is Zlib, a free library originally created over 20 years ago that is employed by Git, PostreSQL and thousands of other applications. The project is based on an algorithm known as DEFLATE that in turn enjoys even broader adoption in the software world. According to the social networking giant, Zstandard sets itself apart by taking advantage of compression techniques and CPU capabilities that weren’t available two decades ago.
Facebook says that its algorithm can provide a slightly higher compression rate than Zlib while working five times faster when shrinking files and twice as fast when decompressing them. Moreover, the algorithm provides the ability to trade data density off for increased performance and vice versa depending on the use case with 22 levels of granularity. As a result, a company that wants to make the most out of its storage equipment could dial compression rates all the way up, while a firm interested in speeding up data management operations could do the opposite and focus on maiming performance. But most of the organizations that will end up adopting the algorithm can be expected to fall somewhere in between.
Zstandard made its debut alongside a new open-source called MyRocks that makes it possible to use Facebook’s speedy RocksDB storage engine with the popular MySQL database. The social networking giant claims that its internal implementation has helped reduce storage requirements buy up to 50 percent in some cases, a massive decrease that should get the technology a lot of attention in the database space.
The code for MyRocks and Zstandard is available immediately on Faceboook’s GitHub page.
THANK YOU