Microsoft open sources Project Zipline, its data compression technology for the cloud

Kareem Anderson

Similar to the HBO hit show Silicon Valley matriarch company Pied Piper, Microsoft also possess a sought after compression algorithm called Project Zipline that it has announced it is essentially making public.

As more companies make the jump to the cloud to store their tera and petabytes of historical data, access to a successful in-house compression algorithm is typically something that is kept close to the vest by cloud service providers. However, Microsoft seems confident enough in its Azure service packages that it voluntarily made its compression algorithm, hardware specification and Verilog source code available to the Open Compute Project (OCP), according to a report from TechCrunch.

Microsoft annonced at a OCP Summit 2019 keynote in Denali that as of today, developers and contributors to OCP will now have the secret sauce ingredients to Microsoft’s compression algorithm clocked at 2x higher compression ratios when compared to the industry standard Zlib-L4 64KB model.  

Leveraging data gathered about how companies utilize the cloud, Microsoft optimized its Project Zipline for use on the system level to reduce overhead, produce higher throughput rates and minimize latency greater than other compression algorithm solutions on the market.

For anyone a bit weary of Microsoft’s proclamations, the company is dogfooding its own algorithm in its Azure cloud as well as partnering with other OCP participants such as AMD, Ampere, Marvell, Broadcom, NGD Systems, Arm, and Synopsys, to name a few.