Before Buying More Coding Agents, Load-Test Your Software Delivery System

AI coding agents can create branches, commits, and pull requests faster than many software teams can review, test, integrate, and release them. This changes the capacity problem. The limiting factor may no longer be code generation. It may move to repository operations, continuous integration, human review, security checks, testing, deployment, or rollback. OpenAI reports that greater Codex concurrency increased source-control activity and contributed to infrastructure pressure before its Pando deployment. Independent research also indicates that review queues, CI failures, larger changes, and coordination problems can reduce the value of additional generated code. These findings do not prove that every organization needs new Git infrastructure. OpenAI operates at unusual scale, and several cited results are company-reported, observational, sponsored, or based on limited samples. An organization should therefore test its own delivery system before it buys more agent seats. A controlled 30-day comparison can measure completed delivery, branch concurrency, push and fetch latency, CI queues, reviewer time, merge conflicts, escaped defects, and rollback frequency. Leaders should expand agent use only when end-to-end delivery improves without an unacceptable increase in review burden, defects, or operational risk. The correct investment is the capacity that removes the measured constraint, not automatically more agents or a new repository platform.

OpenAI says the Pierre agreement remains subject to regulatory approval and customary closing conditions. OpenAI and Pierre remain separate and independent until closing. After closing, OpenAI intends to keep Code Storage available to customers while bringing its features and functionality into Codex and building on familiar Git workflows. The announcement makes no commitment about timing, packaging, pricing, or availability.

That distinction matters. Pando is a company-reported production deployment. Pierre is a pending acquisition and a statement of intent. Together they expose a useful operating question: if coding agents create more work in parallel, which part of the software delivery system becomes the constraint?

Source control can be one answer. It is not the default answer for every organization. Review, CI, testing, security validation, merge contention, or release discipline may bind first. The useful procurement unit is not generated code. It is completed, reviewed, tested, and safely released work.

What OpenAI's Pando account establishes

In its engineering report on Pando, OpenAI attributes a sharp rise in branches, pushes, merges, fetches, clones, and CI activity to the concurrency Codex enabled. The company says the infrastructure supporting its monorepo sometimes fell below 90 percent reliability, with slow fetches and pushes and failed merges.

OpenAI also reports that Pando preserved compatibility with the Git protocol and client, reduced median push latency by nearly fivefold, and now serves almost one petabyte of Git data each day. The reliability, latency, and data-volume figures are all unaudited and company-reported. They are not independent benchmarks and do not predict another organization's result.

The migration is more transferable than the scale claim. OpenAI says it created a Git conformance suite and deterministic simulation harnesses. It initially deployed Pando as a read-and-write-through proxy so the new and previous systems stayed synchronized. It then moved systems one at a time, migrated clients in phases through an internal Git wrapper, and gradually shifted integrations dependent on older APIs, webhooks, and downloads. The supported description is conformance testing, deterministic simulation, proxying, phased migration, and gradual traffic movement.

Pierre's Code Storage product page describes native Git access, SDKs, webhooks, and synchronization with other Git platforms for machine-oriented workloads. Those are vendor claims, not independent evidence. Axios framed the proposed acquisition as part of broader competition around GitHub, but OpenAI remains the source for Pando's performance figures and the intended Codex integration.

Independent research points downstream

A November 7, 2026 working paper, Artificial Intelligence in the Firm: Bottlenecks in Software Production, examines 445 million work events from 720,542 workers at 655 firms. The events combine GitHub coding activity, Jira issues, and Google Calendar records in a proprietary dataset supplied by engineering analytics platform Jellyfish. The authors exploit differences in when firms adopted AI assistants and agents, using an event-study design with firm-size controls and two-way fixed effects.

The paper finds that agent adoption increased intermediate code output while final software output rose less. It identifies review as a bottleneck: code waited longer for a first human reviewer, spent longer in active review, and received more requested changes and comments. The paper estimates those changes rather than observing a randomized experiment. Adoption timing, measurement coverage, firm differences, and the proprietary sample limit causal interpretation, so its estimates should not be treated as settled consensus.

The disclosure is material. The authors state that the research was funded by the Stripe Economics of AI Fellowship, Harvard Kennedy School's James M. and Cathleen D. Stone Program, and the Schmidt Sciences AI at Work Program. Jellyfish partnered on the work and supplied aggregated, anonymized proprietary data. The paper says Jellyfish reviewed the manuscript only to protect proprietary information, not for scientific content, and that the conclusions are the authors' own.

A separate study, Where Do AI Coding Agents Fail?, analyzes about 33,000 pull requests created by five agents across public GitHub projects, then qualitatively codes a sample of 600. Unmerged submissions tended to make larger changes, touch more files, fail CI more often, and encounter review or coordination problems such as duplicates, unwanted features, weak reviewer engagement, and agent misalignment.

That open-source sample is observational. Repository selection, agent mix, task mix, maintainers, and project norms can all affect whether a pull request merges. Its merge outcomes should not be generalized into an enterprise merge-rate forecast. The durable signal is narrower: change size, CI results, review dynamics, and coordination belong in any test of agent-assisted delivery.

Supplementary survey evidence points the same way but carries different limits. Black Duck's 2026 vendor-sponsored study was fielded in March by compensated third-party research firm UserEvidence. It used self-reported responses from 831 software engineering and DevOps professionals. Black Duck says the sample was vendor-neutral, while also noting that neither company's customers were excluded. Every respondent worked at an organization with at least 500 employees, 67 percent were at organizations with at least 2,000, and 78 percent worked in technology or software as a service.

Respondents named manual review at 52 percent, security testing at 51 percent, and code rework at 48 percent as common bottlenecks. These are sponsored, self-reported perceptions, not independently verified delivery outcomes. They are useful corroboration for what to measure, not proof of what every team will experience.

Why your first constraint may be elsewhere

OpenAI describes a fast-changing monorepo and data delivery approaching one petabyte a day. Most organizations are not operating at that scale. A team with modest repository traffic may never make Git storage or serving its binding constraint, even as agent use rises.

The pressure can appear elsewhere. More branches can make CI arrivals exceed runner capacity. Larger or more numerous changes can create a reviewer queue. Weak tests can let defects travel faster. Independently correct changes can conflict when they touch the same interfaces, migrations, or release train. Security validation can become a late-stage queue. A release process sized for fewer human-authored changes can become the last constraint.

This is a Stratex synthesis of OpenAI's case and the independent evidence, not a universal sequence. It also is not a reason to replace a Git provider simply because agents are in use. The response should follow the measured constraint.

A controlled 30-day load test

Choose one non-critical repository with enough routine work to observe flow. Exclude safety-critical, payment, payroll, authentication, emergency, and tightly regulated functions. Use the first 15 days as a baseline and the next 15 as a limited agent-assisted cohort. If work is seasonal or the sample is thin, extend both periods rather than force a decision.

Keep the repository, core team, work intake, definition of done, CI configuration, review policy, target environment, and release cadence comparable. Give a defined cohort a fixed number of agent seats and a bounded task set. Tag assisted work, but retain a team-level view because queues are shared. Record changes, absences, outages, and workload mix. This is a controlled operating comparison, not a randomized trial.

Review the scorecard twice a week. Daily review overweights noise; weekly review can allow a failing queue to consume most of the test. Use predeclared stop conditions for severe defects, unauthorized changes, repeated release failures, an unmanageable CI queue, or sustained reviewer overload. Set boundaries from the team's baseline, service objectives, and risk tolerance.

Open the printable 30-day scorecard

Download the accessible HTML scorecard

Define the measures before day one

Commits, lines of code, pull requests, and branch counts are activity measures. They can explain system load, but they are not delivered customer value. Normalize counts where volume matters, preserve work-item size and risk class, and do not compare raw totals when the assisted period contains materially different work.

Make a scale, pause, or investigate decision

Additional agent seats are justified only when end-to-end delivery improves without material deterioration in reviewer burden, escaped defects, or rollback frequency. The organization should define "material" from its baseline, service objectives, and risk tolerance. There is no responsible universal threshold.

Scale when completed delivery improves and quality outcomes remain inside predeclared tolerances. Pause when apparent speed is offset by review load, defects, rollbacks, or an unsustainable queue. Investigate when results are mixed, confounded, or too small to support a decision. A stop is a finding, not a failed experiment.

The remedy may be more CI capacity, better test selection, smaller changes, clearer ownership, review service objectives, fewer simultaneous branches, or a repository change. It may require no new infrastructure purchase. Load-testing the whole delivery system locates the constraint before an organization pays to add more work upstream of it.

Coding agents can make code generation cheaper. They do not automatically make integration, judgment, security validation, testing, or release cheaper. Measure the complete path, then buy the capacity that improves it.

For architecture context, see Stratex's field guide to building persistent agents with Codex.