We armed auto-merge on 108 agent-written pull requests. One merged.

Reddit r/AI_Agents News

Summary

An experiment with AI agents enabling auto-merge on 108 pull requests revealed only one merged due to CI runner saturation, showing that agent throughput doesn't translate to actual code landing without considering merge capacity.

One night, a controller session enabled auto-merge on roughly 108 pull requests across ten repositories and went quiet. By morning, exactly one had merged. The agents had drafted the changes, checked them locally, pushed them, and enabled auto-merge. By their own accounting they had completed a productive shift. What they did not understand was the machinery between a finished diff and a merged commit. Most of the pull requests were sitting red. The failures came from a shared CI runner pool that had been saturated by the burst. A single push in our busiest repository can create dozens of small check jobs, each claiming a runner and cloning the repository to perform a few seconds of work. Enough agents pushing close together created a backlog the pool could not drain. The stranger part came later. Once the runners recovered, the red pull requests remained red. Auto-merge is only a condition. It does not retry a failed check. A red pull request is inert until something reruns the failed job or creates a new commit. Our automation then spent about five hours carefully monitoring a queue that could not move. Once active triage began, four changes merged in roughly forty minutes. Their code had not changed. The stale CI failures were simply rerun. The lesson was that coding agents made authoring much cheaper, but did nothing to increase landing capacity. We were measuring work created instead of changes accepted by the shared branch. Pull requests opened looked great. Pull requests merged told the truth. This changed how I think about agent throughput. The system cannot optimize only for tasks completed, diffs produced, or pull requests opened. It needs to understand runner capacity, required-check cost, stale failures, dependency chains, merge queues, and actual merges per hour. Otherwise the agents are producing inventory faster than the landing tier can absorb it. Your agents do not ship code. Your CI and merge-control system do. Has anyone else measured how many agent-generated changes their repository can actually land per hour, rather than how many the agents can write?
Original Article

Similar Articles