Reliability fell below 90%
OpenAI published an account on Monday of why it rebuilt the storage layer under its monorepo, and the diagnosis is blunt. “Starting around the end of last year, the systems supporting our monorepo were struggling to keep up,” write Aaron Friel, Taylor Blau and Ted Nyman, members of technical staff. “We suffered slow fetches and pushes along with failed merges, with reliability at times falling below 90%.”
The cause named in the post is Codex. Agent-assisted development means more branches, more pushes, more merges, and more fetches and clones from development environments, research jobs and continuous integration. One engineer can now have several changes queued for CI at once, and each push sets several jobs fetching the same commit.
What Pando is
Pando is a new backend for Git repositories. It now powers all development in OpenAI’s monorepo, and it was deployed without interrupting engineering work or changing how the team uses Git: the Git protocol and the standard client still work.
Two design choices do the work. Pando coordinates updates to each branch independently, so pushes to unrelated branches proceed in parallel. And it reuses packfiles — Git’s format for compressing objects together — across fetches rather than building a fresh one per client, so the server can serve the same bytes to many readers with little computation.
Both choices have a stated cost. A reader looking at several branches can see them at different points in time, and a client may download objects it already holds.
The numbers
OpenAI reports that in production Pando cut median push latency by nearly five times, and that it now delivers nearly a petabyte of Git data per day. Both figures are the company’s own, from its own infrastructure, with no external measurement alongside them.
The team says it wants to send improvements from its own copy of Git upstream to the Git project, which would make the client-side work useful outside OpenAI.
Why it matters beyond one monorepo
The interesting part is not the architecture. It is that a frontier lab hit a scaling wall in source control before it hit one anywhere more exotic, because its own coding agent generates branches faster than the tooling underneath was built to absorb. Any organisation adopting agents at the same rate inherits the same bottleneck without the option of rewriting Git’s backend.
What to watch
Which of OpenAI’s Git changes land upstream, and whether other companies running coding agents at scale report the same failure mode in their own version control before they report it anywhere else.
