

International Data Corp. estimates that the total volume of digital information in the world will balloon from 33 zettabytes, or trillion gigabytes, today to 175 zettabytes in 2025. This rapid growth is being felt particularly strongly by cloud providers such as Microsoft Corp., which host not just their own information but also that of countless other organizations.
To reduce the strain on its infrastructure, the company has developed a cutting-edge system for compressing data. Microsoft this morning released the specifications for the system under an open-source project dubbed Zipline.
The company touts its technology as considerably more powerful than the compression software commonly used in the industry today. Kushagra Vaid, the general manager of the Azure Hardware Infrastructure team, used the popular Zlib tool as a reference point in the blog post announcing Zipline.
Zlib is an industry-standard compression library that can be found in the Linux kernel, iOS and other foundational software platforms. Vaid wrote that Zipline provides data compression rates up to twice as high as high those offered by Zlib. Moreover, the system is described as capable of doing so while providing better throughput and lower latency than several other popular compression tools.
In practice, this means that Zipline can shrink workloads to just a fraction of their size. Microsoft claims that the system compresses the storage footprint of application data stored on Azure by as much as 92 percent. Zipline provides even greater reductions for other types of data such as machine-generated logs from connected devices.
Microsoft is open-sourcing the algorithm that the system uses to perform compression, as well as the specifications for the custom hardware on which the algorithm is designed to run. These specifications include the low-level register transfer language in which Zipline expresses data operations.
“Over time, we anticipate Project Zipline compression technology will make its way into several market segments and usage models such as network data processing, smart SSDs, archival systems, cloud appliances, general purpose microprocessor, IoT and edge devices,” Microsoft’s Vaid wrote.
Zipline is not the first component of Azure that the company has contributed to the open-source community. Previously, Microsoft Corp. released the code for an artificial intelligence engine that supports some of the cloud platform’s services. It has also shared the schematics for a homegrown chip called Cerberus that can protect a server’s firmware from tampering attempts.
Support our open free content by sharing and engaging with our content and community.
Where Technology Leaders Connect, Share Intelligence & Create Opportunities
SiliconANGLE Media is a recognized leader in digital media innovation serving innovative audiences and brands, bringing together cutting-edge technology, influential content, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — such as those established in Silicon Valley and the New York Stock Exchange (NYSE) — SiliconANGLE Media operates at the intersection of media, technology, and AI. .
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a powerful ecosystem of industry-leading digital media brands, with a reach of 15+ million elite tech professionals. The company’s new, proprietary theCUBE AI Video cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.