Cloudflare Boosts Cache Efficiency with Zstandard Compression
Cloudflare has prototyped a Cache Transcoding system using Zstandard (zstd) to significantly increase effective cache capacity by compressing eligible assets, trading minor CPU increases for substantial storage and bandwidth savings.
Cloudflare is exploring innovative methods to combat escalating memory costs by enhancing the efficiency of its distributed storage systems, including its Content Delivery Network (CDN). The company has prototyped a system named Cache Transcoding, which leverages Zstandard (zstd) compression within its Pingora architecture. This approach aims to dramatically expand effective cache capacity by compressing uncompressed assets like HTML, JSON, CSS, and JavaScript before they are stored on disk. The trade-off involves a slight increase in CPU usage for a significant reduction in on-disk storage and cross-data center bandwidth requirements.
During initial testing, the Cache Transcoding system reduced the size of eligible assets to approximately one-third of their original on-disk footprint. This compression is applied once when an asset enters the cache, while the storage and bandwidth savings are realized repeatedly with each subsequent access. The system decodes the assets just before serving them to clients, ensuring no alteration to the original content. This strategy is particularly effective for text-based assets, which often arrive uncompressed from origins and compress well.
Zstandard, developed by Yann Collet at Facebook and open-sourced in 2016, is a lossless compression algorithm chosen for its balance between compression ratio and speed. Unlike formats like Brotli or gzip, zstd offers comparable compression with faster encoding and decoding speeds, a critical factor for systems handling high volumes of traffic. The prototype utilizes zstd level 3, which provides substantial compression benefits without creating a CPU bottleneck during cache fills. This allows Cloudflare to maintain high performance while optimizing storage.
The Cache Transcoding system intelligently identifies which assets are suitable for compression. Media files such as images, videos, and fonts are typically already compressed and are therefore excluded to avoid unnecessary CPU expenditure. In contrast, text-based formats like HTML, JSON, CSS, and JavaScript, which constitute a significant portion of requests and a smaller portion of bytes, are prime candidates. The system focuses on compressing eligible text assets larger than 4 kibibytes (KiB) to maximize storage benefits within the allocated CPU budget.
By reducing the on-disk footprint of cached assets, Cache Transcoding increases cache density. This means each server can store more customer content, reducing the likelihood of essential data being evicted due to space constraints. Furthermore, the smaller compressed representation significantly benefits Cloudflare's Tiered Cache architecture. When assets move between data centers, less data needs to be transferred over the backbone network, leading to more efficient network utilization and reduced operational costs.
The process involves encoding eligible responses with zstd before writing them to disk. The cache metadata tracks that the stored version is compressed and retains the original content length. When a cache miss occurs, the Pingora proxy fetches the original bytes from the origin, encodes them, and stores the zstd-compressed version. On a cache hit, the compressed object is read from disk and decoded before being served. In the Tiered Cache system, compressed objects are transferred between cache tiers, remaining compressed over the wire and on disk until the final decoding step at the client-facing hop.
This innovation addresses the growing challenge of managing vast amounts of data in a cost-effective manner. By optimizing storage and bandwidth usage through intelligent compression, Cloudflare aims to enhance its service delivery capabilities and maintain its competitive edge in the global CDN market. The successful implementation of Cache Transcoding could lead to petabytes of effective storage savings across its network.