GitHub Rebuilds Git Storage Around Reference Compare-and-Swap

TL;DR

• GitHub is a code hosting platform where teams keep Git repositories and run their delivery pipelines from the same service.
• On 5 November 2026 GitHub published Project Rubicon, the storage design underneath its new Git architecture, and internal testing reached 5,000 concurrent agents on one repository with 7,200 pushes per second.
• The design keeps pack data in Azure Blob Storage and moves references into a replicated log service called Ref-CAS, so only the reference update needs agreement.
• Each reference update is a compare-and-swap committed by a quorum, and every commit carries a log sequence number that serves as the repository version.
• Reads begin with a version comparison, so an unchanged repository costs one check and returns no reference data at all.
• Compaction runs on separate background nodes against durable storage, which holds the pack count flat while pushes climb.
• Two earlier attempts at reference state, a single write-ahead log blob and segmented blobs, either capped writes in the mid-hundreds per second or slowed every read, and the post documents both.
• The measured ceiling comes from internal testing in a limited private preview, with a broader rollout expected early next year and six further optimizations still listed.

What a Rubicon push writes, and what it waits on

A Git object is immutable and named by the hash of its contents, which means two writers can store the same object at the same moment without agreeing on anything. What a push has to negotiate is the reference update, and that record is tiny compared with the pack carrying the content. GitHub built Rubicon on that split, and the company states it plainly in the post: "Only reference updates need agreement."

The upper half of the stack stays familiar. Requests from Git clients, the API and the web interface arrive at Spokes, which keeps routing, load balancing and the partitioning and fairness rules that spread tenants across hardware. Underneath Spokes the data plane is new: each repository maps to a small group of virtual replicas, disposable compute whose local disk is a cache that can be rebuilt at any time, and Spokes sends a repository's traffic to the same group when it can so that cache stays warm.

The write path follows four moves. A virtual replica indexes the incoming pack and uploads it to Blob Storage, then commits one transaction to Ref-CAS that lists the pack in the manifest and updates the references. A compaction node merges that pack with others later, and a read pulls the packs it needs into a local cache before serving. Correctness never depends on a request landing on a particular replica, which is why GitHub can add replicas to a busy repository without a migration.

A reference log with a quorum and a sequence number

GitHub tried object storage for reference state twice before writing its own service, and both attempts are worth reading as design notes. The first was a write-ahead log for every reference in a repository, indexed by a single blob, where each update was a conditional write against that blob. It was correct and cheap to read, and it capped throughput in the mid-hundreds per second, because only one conditional write to a blob can be in flight at a time.

The second attempt split references across several segment blobs by hash, so writes to different segments stopped waiting on each other. That raised the write ceiling and made every read check each segment, leaving read latency as slow as the slowest segment. Since reads outnumber writes by orders of magnitude, the trade went the wrong way, and the post says so with the two proposals side by side.

Ref-CAS is the third answer, and it behaves like a small consensus group dedicated to reference state. A transaction locks only the references it touches, checks that each still holds the value the writer expects, and becomes visible once a majority of the log's replicas record it. Agents mostly push to their own branches, so most updates never contend and replicate in parallel, and when two pushes touch one reference only one appends to the log.

Reads that cost one version comparison

Every Ref-CAS commit carries a log sequence number, and the repository version is the sequence number of its latest commit. A read therefore starts with a version check: the virtual replica sends the version of the reference state it already holds, and if that version is current the reply carries no references at all. An unchanged repository costs a single comparison, which matters because the read-to-write ratio on a busy repository runs heavily toward reads.

Behind each push sit CI runs, deployment systems pulling the new commit, and other agents fetching it for context. GitHub shows that amplification on one repository whose daily requests grew about 4x between March and May, with the curve continuing upward afterwards, and the post plans for at least two orders of magnitude more scale than the busiest repositories carry today.

Ref-CAS never holds or validates object data. A replica uploads a pack to Blob Storage before it commits the reference transaction, which means a reference only ever points at objects that are already durable. That ordering rule is the one worth copying first, because it makes recovery a matter of replaying pointers to data that exists.

Compaction on its own nodes, against durable storage

Every push adds a pack, and each extra pack makes lookups slightly more expensive, so the pack count cannot grow without limit. In the legacy design compaction ran on the same replicas that served traffic and had to run on every one of them. Rubicon moves that work entirely into the background, where dedicated nodes read and write durable storage directly and use Ref-CAS to tell the serving replicas what changed.

One compaction node works a shard by taking a lease in Blob Storage so that only one node touches it, reading the pack manifest to collect new packs into a segment, and sealing that segment once it reaches a limit set by pack count, size or age. The node then merges the packs, checks that the new packs contain exactly the same objects as the inputs, uploads them, adds them to the manifest, and removes the inputs in a second transaction. Garbage collection follows the same shape with separate workers for the compute-heavy part.

That verification step carries the design's weight. A compaction node that published a pack missing one object would corrupt reads for every replica, so the object comparison gates the publish, and cleanup of the old packs happens after the fact. Because each pack's hash maps to a shard, adding shards and nodes adds compaction throughput, and a compacted pack that stays under the size limit goes back into a shard by its own hash and merges again.

The measured ceiling and the open questions

GitHub's numbers come from internal testing, and the post is explicit about that: 5,000 concurrent agents on one repository, 7,200 pushes per second to that repository, and a flat latency curve as writes climb, with six further optimizations listed for later, among them fetching a single object from a large pack, grouping packs by type, and placing replicas in any region. The architecture is in limited private preview, and the post expects a broader rollout early next year.

The open question is the one every vendor benchmark leaves: how much of that capacity survives a customer's own workload. A repository where a thousand agents edit the same branch produces contention on one reference, and the design's own reasoning says most updates replicate in parallel because agents push to their own branches. Teams that run agents in parallel on a shared branch carry that contention into whichever forge they use.

For a team running its own Git forge, the transferable part is the split: immutable objects in cheap object storage, reference state in a small ordered log, and maintenance on separate workers. The Netics piece on Google Cloud's Z4M machine series covers the other end of that trade-off, local NVMe behind checkpoints, and self-hosted Git infrastructure applies the same object-store split on your own estate.

Sources

• Inside GitHub's new Git storage architecture, GitHub Blog, 2026-11-05, github.blog

Source: the GitHub Blog post "Inside GitHub's new Git storage architecture" by Karthik Raman and Brian Celenza, published 5 November 2026 and read in full on 7 November 2026. Every number and mechanism above, including the 5,000 agents, the 7,200 pushes per second, the mid-hundreds per second of the first reference-log attempt and the private-preview status, comes from that post.