Cloudflare Reclaims 100 TB of RAM by Rebuilding Its Hash Ring

TL;DR

• Consistent hashing is the technique that spreads requests across a pool of servers so that adding or removing one server moves a small slice of the work; Cloudflare's open-source implementation is called pingora-ketama, and its Pingora Backend Router uses it to decide which server holds a cached file.
• Each server carried 100,000 hash points, because a 160-point baseline is multiplied by a disk-space weight of 625, and supporting several feature combinations multiplied the rings again, which left some processes holding 6 GB of hash structures.
• The predicted error margin starts near 99% with one hash per server and falls to about 8% at 160 points; each further step down costs almost ten times the hashes, and the last 90,000 points bought a 0.7% improvement.
• Thirty-two-bit hashes also collide, and Cloudflare's simulations show error rising between 10,000 and 100,000 hashes per server for a 2,048-server data centre, which is what made a 90% cut safe.
• A second change packed each hash point into six bytes, down from eight, which cut the memory used by consistent hashing by 25%.
• The migration kept both rings in memory, chose a ring per request by hash and rolled out location by location, so the cache did not invalidate; the old path came out at 100% and the network reclaimed 100 TB of RAM.
• A smaller operator can copy three habits from the work: measure the skew of the distribution, ship a routing change as a migration with both paths live, and inspect the size of the structure that holds the index.

Load balancers are the part of a platform that nobody measures until a bill or an out-of-memory kill arrives. Cloudflare measured, found that a single routing structure was holding gigabytes per process, and then did the arithmetic before touching the code. The post that describes the work is a useful template for anyone running a consistent-hashing layer of their own. Its central lesson is the value of measuring how much accuracy the system needs.

What the Pingora Backend Router does with a hash ring

Pingora Backend Router is Cloudflare's internal load-balancing service for cacheable requests. Its job is to send every request for the same URL to the same server, which is what allows one copy of a cached file per data centre and a stable way to find it again. Consistent hashing is the mechanism: server identities and request identities are hashed into a 32-bit space, and each request goes to the nearest server hash at or before its position.

The design pays off when the pool changes. Adding or removing a server moves a share of the keys and leaves the rest where they were, which is the property that keeps a cache useful during maintenance. The cost is statistical: with one hash point per server, the sizes of the ranges differ, and the share of traffic each server receives follows the range it owns. Cloudflare's engineers computed the expected value and the standard deviation for a pool of N servers, then divided one by the other to get a coefficient of variation of about 99% at 100 servers.

Where the memory went: 100,000 hashes for every server

One hash per server produces a bad spread, so every implementation gives each server several. NGINX hardcodes a baseline of 160 hash points per server, and Pingora uses the same default. At 100 servers, that baseline yields a coefficient of variation of about 8%. Cloudflare needed one more thing: servers with more disk should receive proportionally more traffic.

The ketama scaling does the weighting. The number of hashes for a server is multiplied by a weight, and Cloudflare weights by disk space. A weighting factor of 625 on top of the 160 baseline produces 100,000 hash points per server. Then the feature combinations arrive: compliance requirements and caching features mean only some servers may answer a given request, so each combination needs its own ring, and a handful of features multiplies into dozens of rings. The result, as the post puts it, was memory use of 6 GB in some cases.

The 25% saving that came from a six-byte struct

The storage change is the part a smaller team can copy without any probability work. The structure that held one hash point carried a 32-bit hash and a 32-bit server index, eight bytes in memory. The index only needs 16 bits, because the router will not coordinate more than about 65,000 servers, so an obvious edit is to declare the index as a 16-bit integer.

Rust alignment rules turn that edit into a no-op: a struct is padded to a multiple of its largest field, so four bytes of hash plus two bytes of index still occupies eight bytes. Cloudflare's answer stores both values in a six-byte array and exposes them through getters, which produces the same machine code as a packed struct without the usual objections to packing. The post measures the effect as a 25% reduction in the memory used for consistent hashing.

That saving attacks the constant factor in front of the count. Reducing the point count was the larger lever, and the curve below shows the price of each further step.

Why 90% fewer hashes kept the same spread

Accuracy has a shape, and the shape is the argument for cutting hashes. Each step down in the error margin costs almost ten times the hash points, so the second hundred thousand points do far less work than the first. With a weighting factor of 625 the calculation lands at 100,000 points per server, and Cloudflare's own summary of that number is blunt: the last 90,000 hashes added were buying a minuscule 0.7% reduction in error.

Thirty-two-bit hashes bring a second limit. As the number of points grows, collisions grow with it, following the birthday paradox, and a collision randomly drops a contribution that was meant to balance the pool. Simulating 32-bit hashes against the prediction shows the error rate starting to rise between 10,000 and 100,000 hashes per server in a data centre with 2,048 servers. That combination, flat returns and rising collisions, is what let the team remove 90% of the points without a measurable loss in distribution.

Cutting over without invalidating the cache

A better ring is still a different ring. Switching the whole network at once would send requests to servers that do not hold the file, which turns a memory optimisation into a stampede towards the origins. Cloudflare treated the change as a migration and kept both rings alive in the router, with the existing migration framework deciding per request which ring selects the backend.

That design gave two things at once: the choice stayed stable for a given request hash, and the rollback path stayed open without a redeploy. The rollout then moved in layers, from small validation locations through progressively larger groups of data centres, with the share of traffic on the new ring and the set of locations allowed to use it controlled separately. Scoping the rollout by location confined cache churn to selected data centres and kept the blast radius small. During the change the team watched backend-selection traces, ring-version counters, connection errors, process memory, startup time, cache behaviour and origin traffic.

What a smaller operator can copy from this

Cloudflare's evidence covers PBR's 32-bit hashes and the server counts it tested. Netics therefore treats the 90% cut as a configuration-specific result. Operators should rerun the distribution and collision tests with their own server counts, weights, hash widths and eligibility rules before choosing a point count. Three habits travel well. Measure the distribution before tuning it: a coefficient of variation, or simply the busiest server against the average, tells you whether your ring is skewed and whether a change helped. Treat every routing change as a migration with two code paths, because the rollback is what makes the change safe to attempt. Check the size of the structures that hold the index, since padding inside a struct is invisible in the source and visible in the resident memory.

The release half matters too. Everything described ships in the pingora-ketama crate behind an unadvertised cargo feature: the v2 ring carries the compacted storage, a faster sorting method and a configurable base number of hashes per node, while the v1 ring is untouched and both can run at once. That is the same option the network rollout used, exposed to anyone who depends on the crate. For teams that run their own load balancers, the practical follow-on is an hour with a profiler and the question of what the ring costs in memory today. It is the kind of measurement that sits behind monitoring that explains an incident, and a dashboard of averages hides it.

Sources

Source: Saving another 100TB of RAM with math (and Rust) — Cloudflare Blog, 18 September 2026, retrieved 2026-10-25 (ticket filed by Ivan on excessive memory usage from pingora-ketama in Pingora Backend Router; 6GB in some cases; more than 100TB of RAM reclaimed globally, on top of the 100TB the DNS team shed the month before; PBR routes cacheable requests by URL to keep one copy of a file per data centre; consistent hashing maps servers and tasks onto a 32-bit number line and assigns each task to the first server to its left; expected value 1/N and standard deviation for N servers, with a coefficient of variation near 99% at 100 servers and about 8% at 160 points per server; NGINX hardcodes 160 hashes per server and Pingora uses the same value as the default; ketama scales hashes by weight, with disk space as Cloudflare's weight; a weighting factor of 625 gives k = 160 x 625 = 100,000; feature combinations need a separate ring each, reaching dozens of rings; the last 90,000 hashes added were buying a minuscule 0.7% reduction in error; 32-bit collisions follow the birthday paradox and the simulated error rate rises between 10,000 and 100,000 hashes per server for data centres with 2,048 servers; the hashes generated per server were reduced by 90% without appreciable error; the original eight-byte struct carries a 32-bit hash and a 32-bit index, the index fits in 16 bits for about 65,000 servers, Rust alignment keeps the struct at eight bytes and a six-byte array with getters reduces consistent hashing memory by 25%; both rings stayed in memory with a per-request choice through the migration framework, the rollout moved from small validation locations to progressively larger groups of data centres, and the old ring was removed at 100%; the changes ship in the pingora-ketama crate behind an unadvertised cargo feature with the v2 ring carrying the compacted storage, a faster sorting method and configurable scaling of the base hashes per node).

Source: Cloudflare's engineering post of 18 September 2026, retrieved 2026-10-25 — every figure, percentage, version note and description of the migration above traces to that page, and the four real images are its own charts and a capture of its head. Netics editorial, neticslabs.com.