Cloudflare Slashes DNS Cache Memory Footprint by 100TB Through Aggressive Optimization

Cloudflare has announced a significant achievement in memory optimization for its high-traffic DNS services, including the popular 1.1.1.1 resolver. The company's Big Pineapple platform, which powers multiple DNS services, has undergone a series of five successive changes to its cache storage mechanisms. These optimizations have collectively slashed the memory footprint per DNS cache entry by more than half, resulting in an estimated 100 terabytes of memory being freed up across their global fleet. This substantial memory reclamation is equivalent to the RAM capacity of roughly 130 of their high-end Gen 13 servers.

Beyond the impressive memory savings, these optimizations have also yielded significant performance improvements. Cloudflare reported a 43% increase in insert throughput and a 19% reduction in lookup latency. These gains are attributed to fewer memory allocations and improved memory locality, demonstrating that Cloudflare did not sacrifice speed for efficiency.

The core of the optimization lies in how DNS cache entries are stored. Previously, Cloudflare utilized Rust's Vec<T> and String types, which carry inherent overhead. Vec<T> includes fields for pointer, length, and capacity, along with potentially over-allocated heap space, even when the data is static. Similarly, String includes a capacity field. For a cache that stores immutable DNS responses, these capacity fields are redundant and wasteful.

By switching to Box<[T]> for arrays and Box<str> for strings, Cloudflare eliminated the unnecessary capacity fields. Box<[T]> represents a fixed-size slice, meaning it cannot grow after creation, thus negating the need for capacity tracking. This change alone saved 8 bytes per field for each of the 8 Vec and String fields within a cache entry, totaling 64 bytes per entry. Across the hundreds of billions of cache entries, this resulted in over 15 terabytes of memory savings.

Further optimizations involved reducing the number of separate data structures. Instead of storing DNS answer, authority, and additional record sections in distinct lists, Cloudflare now stores them in a single list with offsets. Each offset uses a 2-byte u16 to point to the start of a section, compared to the 8-byte pointer and 8-byte length required for separate Box<[T]> structures. This consolidation saves an additional 28 bytes per entry, contributing to the overall memory reduction and also benefiting from Rust's memory alignment padding efficiencies.

Another key optimization involved how DNS record owners are handled. DNS name compression, used in the wire format, allows for efficient storage of repeated domain names. While Cloudflare previously stored the full owner name alongside each record for performance reasons, they identified that in most cases, the owner is identical to the queried domain. By inferring the owner at read time when it matches the query domain, they can eliminate storing it explicitly, saving further memory. Only when the owner differs, such as in CNAME records, is the full name stored.

These cumulative changes highlight Cloudflare's commitment to efficient infrastructure management at an immense scale. By meticulously analyzing and re-architecting their data storage mechanisms, they have not only achieved substantial memory savings but also enhanced the performance of their critical DNS services, ensuring faster and more reliable internet resolution for billions of users worldwide.