Cloudflare saves 100TB RAM optimizing pingora-ketama
Cloudflare's internal load-balancing service, Pingora Backend Router (PBR), experienced ballooning memory consumption due to storing consistent hashing rings across dozens of feature combinations. By replacing an 8-byte point struct with a compact 6-byte byte array in Rust to bypass alignment padding and deriving the exact coefficient of variation for multi-hash consistent hashing rings, Cloudflare proved they could slash the number of hashes generated per server by 90% without sacrificing load balance or suffering 32-bit collision penalties. Coupled with a careful dual-ring migration strategy that avoided origin cache stampedes, deploying these changes to the open-source pingora-ketama crate successfully reclaimed more than 100TB of RAM worldwide.
Most systems blindly cargo-cult legacy algorithm constants without testing whether those defaults hold up at modern scale or under memory constraints.
- –Inherited defaults invite bloat: Pingora inherited NGINX's baseline of 160 hashes per server, yet statistical modeling revealed that 90% of those hashes provided diminishing returns and exacerbated 32-bit hash collisions in large datacenters.
- –Struct layout nuances yield quick wins: Rust memory alignment padded a 6-byte struct (4-byte hash + 2-byte server index) into 8 bytes; packing it into a raw byte array immediately cut memory usage by 25% before algorithmic changes were introduced.
- –Deployment safety governs optimization feasibility: Modifying a hash ring shifts cache routing and risks origin downtime, requiring dual-ring concurrent routing and canary-style traffic cutovers to validate safety.
- –Exact math trumps textbook approximations: Computing exact closed-form variances instead of relying on asymptotic limits exposed significant headroom for trimming resource overhead without degrading accuracy.
DISCOVERED
1h ago
2026-09-18
PUBLISHED
2h ago
2026-09-18
RELEVANCE
AUTHOR
f311a