Cloudflare Halves DNS Build Delay with Targeted Zone Builder Optimizations

Cloudflare has announced substantial performance enhancements to its DNS Zone Builder, a critical component responsible for processing DNS record changes and zone settings. The company's DNS propagation pipeline, which translates customer-initiated changes into data usable by its authoritative DNS servers, has seen a dramatic increase in workload. The system now handles approximately 700 DNS record changes per second, nearly triple the rate from 2022, alongside about 100 other events like zone setting modifications. This surge in activity, coupled with the growing complexity and size of DNS zones—some now exceeding 20 million records—necessitated a fundamental overhaul of the Zone Builder's architecture.

The core challenge lay in the inefficiency of full zone rebuilds. Previously, any change, whether a single record update or a zone setting modification, could trigger a complete rebuild of the entire zone's data. This process involved reading, encoding, and comparing every record, even if only a small part of the zone had changed. For large zones, this became computationally expensive and memory-intensive, often exceeding available resources. The introduction of zone setting changes, which can affect multiple zones or critical records like CAA records at the zone apex, further complicated the process, making it difficult to determine the minimal safe rebuild scope.

To address these scaling issues, Cloudflare developed a targeted build planner and a new scheduler. The build planner is designed to identify the smallest possible set of changes that can be safely applied, avoiding unnecessary work. This involves analyzing the nature of the event—whether it's a record change or a zone setting modification—and determining its precise impact on the DNS data. The new scheduler then optimizes the processing by combining related work for the same zone, further streamlining operations.

These optimizations have yielded significant performance gains. Peak memory usage within the Zone Builder has been reduced by an impressive 85%. More critically, the time it takes for DNS changes to propagate across Cloudflare's global network has been halved. This reduction in latency means that DNS updates are reflected much faster, improving the reliability and responsiveness of services relying on Cloudflare's DNS infrastructure.

The technical implementation involved several key steps. Event producers were modified to include detailed information about changed fields and settings, along with their previous and new values. This granular data allows the system to trace the impact of each change through the DNS output generation code. However, not all events provide such specificity; some database functions only indicate a general change via a timestamp. In such cases, to maintain safety and prevent stale data, the system defaults to a more comprehensive build, underscoring the planner's commitment to avoiding uncertainty.

The Zone Builder's output to Quicksilver, Cloudflare's distributed key-value store, is divided into record keys (containing DNS answers for a specific name) and shared metadata keys (governing zone usage and special records). The targeted build planner leverages this distinction, enabling it to select one of five build types based on whether only record keys, or both record keys and metadata, need updating. This fine-grained approach ensures that only the necessary components are rebuilt.

This initiative represents a significant engineering feat, demonstrating Cloudflare's continuous effort to optimize its core infrastructure to handle massive scale and increasing complexity. By moving away from brute-force full rebuilds towards intelligent, targeted updates, the company has not only improved its own operational efficiency but also enhanced the performance and reliability of the DNS services it provides to millions of customers worldwide.