独自通过，协作失败：并行LLM-Agent开发中的语义协调基准测试

# Passes Alone, Fails Together: Benchmarking Semantic Coordination in Parallel LLM-Agent Development
Source: [https://arxiv.org/html/2609.25396](https://arxiv.org/html/2609.25396)
\\correspondingauthor

DOI:[10\.1145/3842650\.3843171](https://doi.org/10.1145/3842650.3843171)ISBN:979\-8\-4007\-2968\-3/2026/10Conference:Proceedings of the 2nd Workshop on Explainable and Reliable Software Systems; October 4–9, 2026; Oakland, CA, USAProceedings of the 2nd Workshop on Explainable and Reliable Software Systems \(EXPRESS ’26\), October 4–9, 2026, Oakland, CA, USAsplashws26expressmain\-p4\-pCCS:Software and its engineering Software testing and debuggingCCS:Computing methodologies Multi\-agent systemsCCS:Software and its engineering Software reliabilityHaocheng Xia[https://orcid.org/0000-0002-8317-6217](https://orcid.org/0000-0002-8317-6217),Eugene Wu[https://orcid.org/0000-0003-4254-6688](https://orcid.org/0000-0003-4254-6688)Affiliation:Columbia University,New York,USAemail:[ewu@cs\.columbia\.edu](mailto:[email protected])andYongjoo Park[https://orcid.org/0000-0003-3786-6214](https://orcid.org/0000-0003-3786-6214)Affiliation:University of Illinois Urbana\-Champaign,Champaign,USAemail:[yongjoo@illinois\.edu](mailto:[email protected])

Received 2026\-07\-31

###### Abstract\.

Parallel coding agents can produce patches that work alone but fail when merged\. This happens when one agent changes an interface or rule that another agent still relies on\. We study these failures withstale, a benchmark for semantic coordination\. Our evaluation runs the same tests on each patch alone and on their combination, counting only failures introduced by combining the patches\. We use three tiers: synthetic tasks with controlled interface changes, pairs of merged pull requests, and constructed tasks that use real Django helpers\. Among 834 runs on 417 mined Django pairs, only one showed interference after correcting the grading procedure\. On constructed tasks using 12 Django helpers, interference occurred in 97% of runs\. A message describing the completed concurrent change recovered 82% of runs\. Reviewed pull requests may contain few unresolved parallel changes, even when agents fail on controlled tasks using real code\. The constructed failure rates do not estimate how often these problems occur in practice\.

###### Keywords:

LLM coding agents, semantic coordination, software benchmarks

††cc\-license:by## 1\.Introduction

Parallel multi\-agent systems use different coordination models, including shared state, explicit orchestration, asynchronous updates, and isolated workers whose outputs are merged afterwards\([Qian et al\., 2024](https://arxiv.org/html/2609.25396#bib.bib9);[Hong et al\., 2024](https://arxiv.org/html/2609.25396#bib.bib8)\)\. We focus on a specific scenario where workers do not receive updates from their peers while on the job\. This scenario can occur as a complete workflow or as a temporary information gap within a coordinated system\. It is efficient and harmless for independent tasks\([Pan et al\., 2025](https://arxiv.org/html/2609.25396#bib.bib13);[Ogenrwot and Businge, 2026](https://arxiv.org/html/2609.25396#bib.bib15)\), but dangerous when one patch depends on an interface or invariant that is being changed concurrently\. Each patch may then pass in isolation while their composition fails\. We call the problem of maintaining consistency across such concurrent, coupled changes*semantic coordination*\.

### Motivating Example\.

Figure[1](https://arxiv.org/html/2609.25396#acmlabel1)illustrates how independently correct patches can become inconsistent when combined\. Two workers start from a service that reads and writes prices directly in a database\. Worker A adds caching and modifies the existing update function to invalidate cached prices\. Without seeing A’s changes, Worker B adds a bulk\-update function that writes directly to the database\. Then a query returns 100 even after the database price changes to 80\. The failure arises because A introduces an invariant that B’s new write path violates: every price update must invalidate the corresponding cache\. Keeping this invariant across patches requires semantic coordination beyond conflict\-free code merging\.

\(a\) A: add caching[⬇](data:text/plain;base64,Y2FjaGUgPSB7fQoKZGVmIGdldF9wcmljZShrKToKICAgIGlmIGsgbm90IGluIGNhY2hlOgogICAgICAgIGNhY2hlW2tdID0gZGJba10KICAgIHJldHVybiBjYWNoZVtrXQoKZGVmIHVwZGF0ZV9wcmljZShrLCB2KToKICAgIGRiW2tdID0gdgogICAgKCpAXGNvbG9yYm94e2JsYWNrITEwfXtcc3RydXRcdGV4dHR0e2NhY2hlLnBvcChrLCBOb25lKX19QCop)1cache=\{\}23defget\_price\(k\):4ifknotincache:5cache\[k\]=db\[k\]6returncache\[k\]78defupdate\_price\(k,v\):9db\[k\]=v10cache\.pop\(k, None\)\(b\) B: add bulk updates[⬇](data:text/plain;base64,ZGVmIGJ1bGtfdXBkYXRlKHByaWNlcyk6CiAgICAoKkBcY29sb3Jib3h7YmxhY2shMTB9e1xzdHJ1dFx0ZXh0dHR7ZGIudXBkYXRlKHByaWNlcyl9fUAqKQ==)1defbulk\_update\(prices\):2db\.update\(prices\)New write path bypasses the cache added by A\.\(c\) Merged execution
[⬇](data:text/plain;base64,IyBJbml0aWFsbHk6IGRiID0geyJ4IjogMTAwfTsgY2FjaGUgPSB7fQpnZXRfcHJpY2UoIngiKSAgICAgICAgICAjIDEwMDsgY2FjaGVzIHByaWNlCmJ1bGtfdXBkYXRlKHsieCI6IDgwfSkgICMgZGF0YWJhc2Ugb25seQpnZXRfcHJpY2UoIngiKSAgICAgICAgICAjIDEwMDsgZXhwZWN0ZWQgODA=)12get\_price\("x"\)3bulk\_update\(\{"x":80\}\)4get\_price\("x"\)Figure 1\.Independently correct patches merge without textual conflicts but produce stale prices\.Worker A adds a price cache and invalidation to the existing update function\. Worker B adds bulk updates that bypass the cache\. After merging, a bulk update changes a database price from 100 to 80, but the cached query still returns 100\.Version control helps developers combine parallel changes, but a clean merge does not guarantee correct behavior\. For example, one patch may rename a helper that another patch still calls, add a required argument that another caller omits, or change a rule that other code relies on\. These inconsistencies may appear only when tests run on the combined code\. By then, diagnosing the failure requires understanding both patches\.

A clean merge can still fail, even when agents edit different files\. AutoGen\([Wu et al\., 2023](https://arxiv.org/html/2609.25396#bib.bib7)\)and MetaGPT\([Hong et al\., 2024](https://arxiv.org/html/2609.25396#bib.bib8)\)coordinate agents through roles and messages, as do ChatDev\([Qian et al\., 2024](https://arxiv.org/html/2609.25396#bib.bib9)\)and Agent Teams\([Anthropic, 2026](https://arxiv.org/html/2609.25396#bib.bib16)\)\. We measure failures caused by missing updates and test how many an agent can avoid when it sees the concurrent change\.

gitmergebehavioral testschangeAAalone–passchangeBBalone–passA\|BA\\parallel B\(blind\)no conflictfail=Δblind=\\Delta\_\{\\mathrm\{blind\}\}A→BA\\\!\\to\\\!B\(informed\)no conflictpassFigure 2\.A clean merge can still introduce test failures\. We count a test inΔblind\\Delta\_\{\\mathrm\{blind\}\}only if it passes with each patch alone and fails with the patches combined\. An informed agent sees the other worker’s completed edit before solving its task\. Comparing these conditions estimates how much complete information can help\.Comparison of patches alone, patches merged without coordination, and sequential informed work\. Individual patches pass; a clean blind merge introduces failures; an informed agent can adapt to the other patch\.A*blind*agent works without seeing the other worker’s changes; an*informed*agent works afterward and sees the completed edit\. We call the informed setting an*oracle*because it provides complete information that an agent working in parallel would usually lack\. This comparison estimates*semantic coordination headroom*: how much failure coordination could prevent under ideal visibility\. To measure interference between fixed patches, we count tests that pass with each patch alone but fail with both patches together\. We denote this count byΔblind\\Delta\_\{\\mathrm\{blind\}\}and also report whether it is greater than zero\. These measures describe failures detected by the tests; they do not measure defect severity or how often conflicts occur in practice\. We study three types of instances because historical pull requests may already have resolved the conflicts we want to measure\.

We make three contributions\.\(1\)We define semantic coordination headroom and a grading procedure that tests the same patches, both alone and together, using the same test set\. This separates failures caused by patch interaction from differences between agent runs or grading conditions\.\(2\)We provide a deterministic pipeline for finding related pull requests and checking that both tasks can be evaluated from a shared base commit\. It produces 447 pairs across Django, SymPy, xarray, and seaborn\. Experiments on the 417 Django pairs find almost no interference after correcting the grading procedure\.\(3\)We demonstrate the failure mechanism in controlled experiments: synthetic tasks vary how many interfaces change, and 36 constructed tasks apply these changes to 12 real Django helpers\. The controlled tasks isolate failures caused by stale information\. The code is available in the benchmark repository\.111[https://github\.com/illinoisdata/STALE\-bench](https://github.com/illinoisdata/STALE-bench)

## 2\.Benchmark Design

### Tasks and Evaluation Settings\.

In the mined tier, each instance containsNNrelated pull requests \(PRs\),\{pi\}i=1N\\\{p\_\{i\}\\\}\_\{i=1\}^\{N\}, evaluated from a shared base commitbb\. Each PR provides a task description, a human reference solution, and a hidden*gold test patch*\. The task description comes from the issue or PR, with text explaining the solution removed\. The gold tests encode the expected behavior: they fail before the fix and pass afterward \(fail\-to\-pass tests,F2Pi\\mathrm\{F2P\}\_\{i\}\)\. We define three evaluation settings:

- •solo\(pip\_\{i\}\): an agent solvespip\_\{i\}alone, producing the baseline patch\.
- •blind\(\{pi\}\\\{p\_\{i\}\\\}\): agents solve their assigned tasks independently onbb, without seeing the other agents’ changes, and the resulting code diffs are merged\.
- •informed\(pπ⁡\(1\),…,pπ⁡\(N\)p\_\{\\pi\(1\)\},\\ldots,p\_\{\\pi\(N\)\}\): agents work in orderπ\\pi, each seeing the completed edits of earlier agents\. This is the sequential oracle\.

We evaluate patches alone \(solo\) and together \(blind\) on every instance\. On the synthetic tier, we also run theinformedcondition \(§[4](https://arxiv.org/html/2609.25396#S4)\)\. The completed edit provides an ideal\-visibility baseline for evaluating coordination\. Following SWE\-bench\([Jimenez et al\., 2024](https://arxiv.org/html/2609.25396#bib.bib5)\), we grade code changes using hidden tests from the human PRs\. Before grading, we discard any test\-file edits made by the agent and apply the gold test patches\. We then run the combined test set from all tasks in*every*setting\. Using different tests for individual and merged patches can falsely suggest interference \(§[4](https://arxiv.org/html/2609.25396#S4)\)\.

### The Interference Metric\.

LetFiF\_\{i\}be the tests that fail with patchiialone, and letFblindF\_\{\\mathrm\{blind\}\}be the tests that fail after merging the patches\. Both sets come from the same combined test suite\. We define

Δblind=\|Fblind∖⋃iFi\|,\\Delta\_\{\\mathrm\{blind\}\}\\;=\\;\\big\|\\,F\_\{\\mathrm\{blind\}\}\\setminus\\textstyle\\bigcup\_\{i\}F\_\{i\}\\,\\big\|,Δblind\\Delta\_\{\\mathrm\{blind\}\}counts failures that appear only when the patches are combined\. A test that already fails with any individual patch is excluded\. We reuse the exact patches fromsoloin theblindmerge, so differences between agent runs cannot explain the result\.

We report the countΔblind\\Delta\_\{\\mathrm\{blind\}\}and an indicator of whether any interference was detected,Iblind=𝟏\[Δblind\>0\]I\_\{\\mathrm\{blind\}\}=\\mathbf\{1\}\[\\Delta\_\{\\mathrm\{blind\}\}\>0\]\. For example, three newly failing tests giveΔblind=3\\Delta\_\{\\mathrm\{blind\}\}=3andIblind=1I\_\{\\mathrm\{blind\}\}=1; they need not represent three separate defects\. Both measures depend on test coverage, and the count also depends on how tests are divided or duplicated\. We therefore compare counts only with the same test set\. A pair is*observable*, and eligible for our interference analysis, only when each agent’s patch passes its own task’s tests during grading on the combined suite\.

### Three Benchmark Tiers\.

The*synthetic*tier tests whether stale information causes interference by varying the number of changed interfaces while keeping the task and type of change fixed\. The*mined*tier uses reviewed PRs to test whether repository history contains examples of unresolved interference\. The*real\-derived*tier applies scripted changes to real Django helpers to test whether the synthetic failure mechanism also occurs in existing code\.

We also evaluate a communication condition,comm\. The agent receives a short message describing the other worker’s completed change\. We generate this message from a template and the known transformation\. This is an oracle message based on a completed change\. We do not evaluate messages generated during ongoing work\.

## 3\.Dataset Construction

We build most instances from Django, which has established tools for reproducible agent evaluation\. We use the same deterministic construction procedure for SymPy, xarray, and seaborn\.

### Mining\.

We collect merged PRs, keep those that change 1–12 runtime source files \(excluding docs/CI/formatting\), and form*candidate pairs*that edit at least one common source file, an inexpensive way to find potentially related changes\([Chockchowwat et al\., 2025](https://arxiv.org/html/2609.25396#bib.bib4)\)\. Sharing a file does not guarantee interference; we test for it in the agent study\.

### Checking the Shared Base\.

For each candidate pair, we choose the merge base of the two PR bases as the shared starting commitbb\. We then apply two filters, starting with the cheaper check\.*\(i\) Clean merge:*both code patches apply tobb, and their composition produces no textual conflict\.*\(ii\) Base validity:*for each PR, its gold tests must fail onbbwhen only the test patch is applied and pass after adding the code patch\. The gold test patch comprises the behavioral tests added or modified by the human PR\.

Checking the shared base matters because related PRs often start from different commits\. One PR’s base may already contain part of the other PR’s solution, hiding the interaction we want to test\. In one HTTP\-header pair, for example, the later PR’s base already contained a helper added by the earlier PR\. Our check rejects such a base because the relevant tests pass before we apply the candidate fix\. In a representative batch, invalid bases accounted for about one third of rejections\.

### Yield\.

From*685*mined Django PRs we formed*4,262*candidate pairs; deterministic validation across a 5\-node cluster \(320 cores\) accepted*417*as valid coupled instances\. In a representative batch,∼36%\{\\sim\}36\\%failed because the two changes*textually*conflict,∼33%\{\\sim\}33\\%failed base validity \(drift\),∼17%\{\\sim\}17\\%did not apply on a common base, and∼14%\{\\sim\}14\\%were accepted\. We limit validation cost by running tests only on pairs that pass the structural and textual filters\.

Table 1\.Benchmark summary across two model families and more than4,0004\{,\}000graded task solves\. A validated coupled pair consists of two patches that apply and merge textually on a shared, base\-valid commit; interference is measured separately by the agent study\.Synthetic:33mechanisms×\\times44degrees,88trials each \(9696blind\+\{\+\}comm cells\)\. Real\-derived:1212real symbols×\\times33mechanisms×\\times33trials==108108blind\+\{\+\}108108communication runs\.

## 4\.Preliminary Study

We evaluate more than4,0004\{,\}000graded task solves using two model families \(Table[1](https://arxiv.org/html/2609.25396#S3.T1)\)\. The synthetic experiments measure interference as more interfaces change and test how much an oracle message can prevent\. We then examine reviewed PR pairs for interference and apply the controlled changes to real Django helpers\.

### Synthetic Tier\.

Each synthetic instance addsK=4K\{=\}4helper functions to a Django checkout\. The agent writes small wrappers that call these helpers\. A scripted concurrent patch changes exactlyddhelper interfaces, whered∈\{0,…,4\}d\\in\\\{0,\\dots,4\\\}is the*staleness degree*\. It either renames helpers, adds a required argument, or changes the return type\. The expected wrapper output stays the same\. We run eight trials per setting\. A blind agent uses the old interfaces, so we expect its wrappers to fail on theddchanged helpers after merging\. An informed agent sees the new interfaces before writing its wrappers\.

Renames and required\-argument changes matched the prediction\. Blind patches failed on allddchanged helpers \(Δblind=d\\Delta\_\{\\mathrm\{blind\}\}=d, with zero variance\)\. The informed agent had no failures\. Because the wrapper tasks leave little room for alternative implementations, the result does not predict how failures scale in open\-ended development\. For return\-type changes, the informed agent still averaged0\.130\.13failures per run despite seeing the new interface\.

### Synthetic Tier with an Oracle Message\.

We next give the blind agent a message describing the scripted change, averaging about 130 tokens\. The message states the new helper names, required arguments, or return types\. With this information, the mean interference count fell from2\.502\.50to0\.040\.04, a98%98\\%reduction \(Fig\.[3](https://arxiv.org/html/2609.25396#acmlabel3)\)\. We generate the message from the scripted transformation, so the result assumes a complete and accurate description of the change\.

Figure 3\.Synthetic tier \(88trials per point, averaged over three mechanisms\)\. Blind interferenceΔblind\\Delta\_\{\\mathrm\{blind\}\}increased with staleness degreedd\(red\)\. A∼130\{\\sim\}130\-token oracle description of the finalized transformation reduced it to near zero \(green\), recovering98%98\\%\.A line plot comparing blind interference with interference after an oracle message as the number of changed helper interfaces increases\. The oracle message reduces the mean interference count from 2\.50 to 0\.04\.
### Mined PR Tier\.

For the mined PR pairs, a GPT\-5\-class coding model solves each task insoloandblindconditions on the shared base\. We run the model with mini\-swe\-agent\([Yang et al\., 2024](https://arxiv.org/html/2609.25396#bib.bib6)\), a minimal tool\-calling harness, and with OpenHands\([Wang et al\., 2025](https://arxiv.org/html/2609.25396#bib.bib14)\)\. Both use SWE\-bench\-style grading\.

Our initial grading procedure produced misleading interference counts for two reasons\. First, agents sometimes edited test files, which could prevent the gold tests from being applied consistently\. Second, we tested each individual patch only on its own task’s tests, but tested merged patches on both tasks’ tests\. A patch could therefore break another task’s test on its own, yet the failure would appear only in the merged evaluation\. Both problems falsely increasedΔblind\\Delta\_\{\\mathrm\{blind\}\}\(Fig\.[4](https://arxiv.org/html/2609.25396#acmlabel4)\)\.

soloAsoloBblindΔblind\\Delta\_\{\\mathrm\{blind\}\}naivepassnot runfail\+1\+1*\(false\)*correctedpassfailfail00*\(excluded\)*Figure 4\.Using different test sets can create false interference\. Patch B fails this test on its own, but the initial grading procedure does not run it insoloB\. Running the same tests in all conditions reveals the existing failure and excludes it fromΔblind\\Delta\_\{\\mathrm\{blind\}\}\.Two rows compare task\-specific and combined\-test grading\. Task\-specific grading misses a failure in patch B alone and counts it as interference after merging\. Combined\-test grading detects the existing failure and excludes it\.After removing agent test edits and running the combined test set in every condition, nearly all apparent interference disappeared\. Of 834 runs on 417 validated Django pairs, only one hadΔblind\>0\\Delta\_\{\\mathrm\{blind\}\}\>0, with one newly failing test\. The reviewed human patches also merged without gold\-test failures across the examined pairs\. These results suggest that merged PR history is a poor source of examples of unresolved parallel changes: development and review may have already made the patches compatible\.

GPT\-5\.5 reproduced the synthetic result on newly added helper names, reducing the likelihood that the result depends on memorized code\. On the most strongly coupled mined pairs, it showed no interference in 72 runs\. It did fail on the constructed Django tasks described next\. The difference is consistent with the mined patches having been made compatible during development and review\.

### Constructed Tasks on Real Django Helpers\.

We apply three interface transformations to 12 Django helpers to construct 36 instances \(Appendix[C](https://arxiv.org/html/2609.25396#A3)\)\. Each helper is linked to a validated PR pair that illustrates a dependency through a shared symbol\. One patch makes a scripted interface change; an agent writes the other patch, which uses the helper\.

In the blind condition, GPT\-5\.5 produced interference in 105/108 runs \(97%\), with failures on all 12 helpers despite clean textual merges\. With the oracle message, 89/108 runs \(82%\) were recovered\. Recovery reached 93% among message\-conditioned patches that merged cleanly\. These rates apply to the constructed Django tasks\.

The mining pipeline also works beyond Django\. With the same pytest\-based validation harness, it finds 11 valid pairs in SymPy, 12 in xarray, and 7 in seaborn\. Textual conflicts and incompatible base commits account for most rejections\. In practice, construction works best with PRs close in time and tests selected from the PR’s own changes\. Running entire modified test files can introduce unrelated or flaky cases\.

### Threats to Validity\.

The main measurement risk is counting an individual patch’s failure as interference\. We address this by discarding agent test edits, running the same combined test set in every condition, and requiring each patch to solve its own task in isolation\. As in SWE\-bench\([Jimenez et al\., 2024](https://arxiv.org/html/2609.25396#bib.bib5)\), we rely on the gold tests to define correct behavior\. We also check that those tests fail on the shared base before applying the human fix\.

Historical PRs have been through review, while the constructed tasks deliberately introduce breaking changes\. The latter use one scripted patch and constrain the agent to call selected helpers\. Our agent experiments are also limited to Django\. The results establish the failure mechanism and demonstrate it on 12 real helpers, but do not estimate its frequency or severity in everyday parallel development\.

The communication results assume complete descriptions of finalized changes; recovery with messages generated during parallel work remains untested\. Finally, tests can miss failures, and several tests can detect the same defect\. We therefore interpret interference counts only for a shared test set\.

## 5\.Ongoing Work and Open Problems

The experiments leave open how often interference occurs before review, how agents should communicate during parallel work, and how the results extend to more than two tasks\.

### Parallel Work Before Review\.

To measure how often interference occurs in practice, we need execution records from parallel agents before review or repair\. These records should preserve a shared starting commit, independently generated patches, and intermediate states\([Li et al\., 2024](https://arxiv.org/html/2609.25396#bib.bib2)\)\. These records would let us compare information gaps in systems with isolated workers, asynchronous updates, explicit orchestration, or shared state\.

### From Oracle Visibility to Online Communication\.

The oracle message reduces synthetic interference by98%98\\%\. In a running system, however, agents may only be able to share their plans, an unfinished patch, or an automatically generated summary\. These options differ in completeness, delay, and privacy\([Xia and Park, 2026](https://arxiv.org/html/2609.25396#bib.bib1)\)\. A comparison with affected\-symbol lists and completed edits, using the same token budget, would show how message content and timing affect recovery\.

### Scale\.

Our experiments focus on pairs of tasks\. WithN\>2N\>2tasks, a failure may require three or more changes to occur together, even if every pair works correctly\. Larger task sets also create more possible dependencies\. Extending the benchmark would test whether results for pairs predict reliability for larger groups, and whether we can estimate coordination headroom for a given set of tasks before agents start work\.

## 6\.Conclusion

Agents working from the same codebase can produce patches that pass alone but fail together, even when the merge has no textual conflicts\. Instale, we test the same patches alone and together using a common test set\. Comparing blind agents with agents that see the completed concurrent change estimates how much coordination could help\.

Synthetic tasks isolate failures caused by stale interfaces\. Mined PR pairs suggest that reviewed history rarely retains unresolved interactions\. Constructed tasks reproduce the mechanism on 12 Django helpers, with 97% interference and 82% recovery using an oracle message\. These constructed rates do not measure real\-world prevalence\. The next step is to study parallel work before review, across more repositories, with messages that agents can generate during execution\.

###### Acknowledgements\.

NSF grants \#2103794, \#2312991, \#2551201, \#2440498, and \#2312561 support this work, along with DAPLab corporate support in the form of funding and/or compute from Amazon, IntellectAI, Infosys, Tidalwave, Veris, Shopify, Microsoft, Thinking Machines, Dandy, Perplexity, and Daytona\. This work is also supported by the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support \(ACCESS\) program, which is supported by National Science Foundation grants \#2138259, \#2138286, \#2138307, \#2137603, and \#2138296\. The views and conclusions presented here are those of the authors and should not be interpreted as representing the official positions of the funding or supporting organizations\.

## Appendix ARelated Work

### Agent Benchmarks and Scaffolds\.

SWE\-bench\([Jimenez et al\., 2024](https://arxiv.org/html/2609.25396#bib.bib5)\)evaluates coding agents on real GitHub issues using reproducible tests\. SWE\-agent\([Yang et al\., 2024](https://arxiv.org/html/2609.25396#bib.bib6)\)provides the agent–computer interface used in our experiments\. These tools evaluate one task at a time\. We use the same test\-based approach to measure whether changes developed in parallel still work when combined\.

### Multi\-Agent Software Development\.

AutoGen\([Wu et al\., 2023](https://arxiv.org/html/2609.25396#bib.bib7)\), MetaGPT\([Hong et al\., 2024](https://arxiv.org/html/2609.25396#bib.bib8)\), and ChatDev\([Qian et al\., 2024](https://arxiv.org/html/2609.25396#bib.bib9)\)coordinate LLM agents through assigned roles and communication protocols\. We measure failures caused by stale information and compare them with an oracle condition in which agents see completed concurrent changes\. Evaluating messages based on plans or unfinished work remains an open question\.

### Merging and Collaboration Conflicts\.

Software\-merging research has studied both textual and semantic conflicts\([Mens, 2002](https://arxiv.org/html/2609.25396#bib.bib10);[Sousa et al\., 2018](https://arxiv.org/html/2609.25396#bib.bib11)\)\. Some tools detect conflicts by merging developer branches before developers commit their work\([Brun et al\., 2011](https://arxiv.org/html/2609.25396#bib.bib12)\); transactional approaches use isolation to avoid conflicts before commit\([Chockchowwat et al\., 2023](https://arxiv.org/html/2609.25396#bib.bib3)\)\. We study patches produced by independent coding agents and use hidden tests to detect behavioral failures after a clean textual merge\.

## Appendix BA Minimal Worked Instance

Consider a synthetic module withK=4K\{=\}4helper functions that put brackets around values\. Task A adds wrappersf1\.\.f4that call these helpers\. Task B renameswrapkktobracketkkwithout changing the returned values\. The staleness degreeddis the number of helpers B renames\. For the first helper, A writes:

defwrap1\(value\):

return"\["\+str\(value\)\+"\]"

deff1\(values\):

return","\.join\(wrap1\(v\)forvinvalues\)

With Task A alone,f1\(\[1,2\]\)returns"\[1\],\[2\]"\. Task B preserves the helper’s behavior\. The patches edit different regions, sogitmerges them without a conflict\. After the merge, however, A’s wrapper calls a name that B has removed:

\>\>\>f1\(\[1,2\]\)

NameError:name’wrap1’isnotdefined

An informed agent sees B’s rename before writing the wrapper and callsbracket1\. Incomm, the benchmark instead sends a message describing the completed edit: “wrap1was renamed tobracket1\.” The agent uses the new name and the combined code passes\.

## Appendix CA Real\-Derived Staleness Tier

We construct this tier using existing Django helpers to test the failure mechanism beyond the helpers added for the synthetic experiments\. GPT\-5\.5 writes a consumer that calls a helper, and a scripted patch changes that helper’s interface\.

### Construction\.

We select 12 helpers that can be unit\-tested from Django’s text, HTML, HTTP, and encoding modules\. Each helper is linked to a validated PR pair in which one patch modifies a symbol used by the other \(WA∩RB\>0W\_\{A\}\\cap R\_\{B\}\>0\)\. The historical pair supplies an example of the dependency; we construct a breaking version on a helper that we can test in isolation\.

For example, we linkphone2numericto PRs 18361 and 20309\. The historical dependency involves different symbols: PR 18361 changes the signature ofas\_sql, and PR 20309 adds anas\_sqlitemethod that calls it\. Those reviewed changes work together because the signature change is additive\. Our constructed task makes a breaking change to a helper while asking an agent to add a feature that calls it\.

We apply three transformations:*rename*removes the old helper name;*signature*adds a required keyword\-only argument; and*rettype*returns an object whose\.textfield contains the original result\. Applying these transformations to 12 helpers produces 36 instances\.

### Worked Example\.

Task A renamesphone2numeric\. Task B asks for a new function,phone2numeric\_clean\. It strips spaces and hyphens from its input, then uses the existing helper to convert letters to digits\. Working on the base commit without seeing A’s change, GPT\-5\.5 writes:

defphone2numeric\_clean\(phone\):

cleaned=str\(phone\)\.replace\("␣",""\)\.replace\("\-",""\)

returnphone2numeric\(cleaned\)

B passes alone, and the patches merge without a textual conflict\. The combined code fails because B still calls the old helper name:

NameError:name’phone2numeric’isnotdefined\.Didyoumean:’phone\_to\_numeric’?

Incomm, we generate a message describing Task A’s completed rename\. The agent calls the new name, and the combined code passes\.

Table 2\.Real\-derived Django tier: GPT\-5\.5,33scripted mechanisms×\\times33trials per symbol \(99runs each\)\.blinddenotes silent interference;commreceives an oracle description of the finalized scripted edit\. Across these constructed instances, blind interference occurs in105/108105/108runs \(97%97\\%\), and the oracle message recovers89/10889/108\(82%82\\%;93%93\\%among cleanly merging cases\)\.
### Results\.

Across the 12 helpers, 105 of 108 blind runs show interference \(Table[2](https://arxiv.org/html/2609.25396#A3.T2)\)\. In each counted run, the consumer passes alone, merges without textual conflicts, and fails the hidden tests after the scripted transformation\. We observe this failure on all 12 helpers\.

With the oracle message, 89/108 runs \(82%\) recover\. Among message\-conditioned patches that merge without textual conflicts, recovery reaches 93%\. All 36 return\-type runs recover, and recovery is also high for renames\. For required\-argument changes, the agent sometimes edits the helper definition itself, creating a textual conflict\.

### Scope\.

These tasks reproduce the stale\-interface failure using real Django helpers and dependency patterns found in PR history\. We script the breaking patch and give the communication condition a complete description of it\. Estimating how often unresolved dependencies occur in practice requires records of parallel agent work before review\.

## References

- Anthropic \(2026\)AnthropicOrchestrate teams of claude code sessions\.Note:Claude Code DocsDescribes Claude Code agent teams as of v2\.1\.178\. Accessed: 2026\-06\-28External Links:[Link](https://code.claude.com/docs/en/agent-teams)Cited by:[§1](https://arxiv.org/html/2609.25396#S1.SS0.SSS0.Px1.p3.1)\.
- Brunet al\.\(2011\)Y\. Brun, R\. Holmes, M\. D\. Ernst, and D\. NotkinProactive detection of collaboration conflicts\.InSIGSOFT/FSE’11 19th ACM SIGSOFT Symposium on the Foundations of Software Engineering \(FSE\-19\) and ESEC’11: 13th European Software Engineering Conference \(ESEC\-13\), Szeged, Hungary, September 5\-9, 2011,T\. Gyimóthy and A\. Zeller \(Eds\.\),pp\. 168–178\.External Links:[Link](https://doi.org/10.1145/2025113.2025139),[Document](https://dx.doi.org/10.1145/2025113.2025139)Cited by:[Appendix A](https://arxiv.org/html/2609.25396#A1.SS0.SSS0.Px3.p1.1)\.
- Chockchowwatet al\.\(2023\)S\. Chockchowwat, Z\. Li, and Y\. ParkTransactional python for durable machine learning: vision, challenges, and feasibility\.InProceedings of the Seventh Workshop on Data Management for End\-to\-End Machine Learning \(DEEM’23\),pp\. 1–5\.External Links:[Document](https://dx.doi.org/10.1145/3595360.3595855)Cited by:[Appendix A](https://arxiv.org/html/2609.25396#A1.SS0.SSS0.Px3.p1.1)\.
- Chockchowwatet al\.\(2025\)S\. Chockchowwat, S\. Thakurdesai, Z\. Li, M\. S\. Krafczyk, and Y\. ParkChipmink: efficient delta identification for massive object graphs\.Proceedings of the VLDB Endowment19\(4\),pp\. 603–616\.External Links:[Document](https://dx.doi.org/10.14778/3785297.3785303)Cited by:[§3](https://arxiv.org/html/2609.25396#S3.SS0.SSS0.Px1.p1.1)\.
- Honget al\.\(2024\)S\. Hong, M\. Zhuge, J\. Chen, X\. Zheng, Y\. Cheng, J\. Wang, C\. Zhang, Z\. Wang, S\. K\. S\. Yau, Z\. Lin, L\. Zhou, C\. Ran, L\. Xiao, C\. Wu, and J\. SchmidhuberMetaGPT: meta programming for A multi\-agent collaborative framework\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024,External Links:[Link](https://openreview.net/forum?id=VtmBAGCN7o)Cited by:[Appendix A](https://arxiv.org/html/2609.25396#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.25396#S1.SS0.SSS0.Px1.p3.1),[§1](https://arxiv.org/html/2609.25396#S1.p1.1)\.
- Jimenezet al\.\(2024\)C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. R\. NarasimhanSWE\-bench: can language models resolve real\-world github issues?\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024,External Links:[Link](https://openreview.net/forum?id=VTF8yNQM66)Cited by:[Appendix A](https://arxiv.org/html/2609.25396#A1.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.25396#S2.SS0.SSS0.Px1.p1.2),[§4](https://arxiv.org/html/2609.25396#S4.SS0.SSS0.Px5.p1.1)\.
- Liet al\.\(2024\)Z\. Li, S\. Chockchowwat, R\. Sahu, A\. Sheth, and Y\. ParkKishu: time\-traveling for computational notebooks\.Proceedings of the VLDB Endowment18\(4\),pp\. 970–985\.External Links:[Document](https://dx.doi.org/10.14778/3717755.3717759)Cited by:[§5](https://arxiv.org/html/2609.25396#S5.SS0.SSS0.Px1.p1.1)\.
- Mens \(2002\)T\. MensA state\-of\-the\-art survey on software merging\.IEEE Trans\. Software Eng\.28\(5\),pp\. 449–462\.External Links:[Link](https://doi.org/10.1109/TSE.2002.1000449),[Document](https://dx.doi.org/10.1109/TSE.2002.1000449)Cited by:[Appendix A](https://arxiv.org/html/2609.25396#A1.SS0.SSS0.Px3.p1.1)\.
- Ogenrwot and Businge \(2026\)D\. Ogenrwot and J\. BusingeAgenticFlict: a large\-scale dataset of merge conflicts in ai coding agent pull requests on github\.External Links:2604\.03551,[Document](https://dx.doi.org/10.1145/3805760.3814923),[Link](https://arxiv.org/abs/2604.03551)Cited by:[§1](https://arxiv.org/html/2609.25396#S1.p1.1)\.
- Panet al\.\(2025\)J\. Pan, X\. Wang, G\. Neubig, N\. Jaitly, H\. Ji, A\. Suhr, and Y\. ZhangTraining software engineering agents and verifiers with swe\-gym\.InForty\-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13\-19, 2025,A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, and J\. Zhu \(Eds\.\),Proceedings of Machine Learning Research, Vol\.267\.External Links:[Link](https://proceedings.mlr.press/v267/pan25g.html)Cited by:[§1](https://arxiv.org/html/2609.25396#S1.p1.1)\.
- Qianet al\.\(2024\)C\. Qian, W\. Liu, H\. Liu, N\. Chen, Y\. Dang, J\. Li, C\. Yang, W\. Chen, Y\. Su, X\. Cong, J\. Xu, D\. Li, Z\. Liu, and M\. SunChatDev: communicative agents for software development\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), ACL 2024, Bangkok, Thailand, August 11\-16, 2024,L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),pp\. 15174–15186\.External Links:[Link](https://doi.org/10.18653/v1/2024.acl-long.810),[Document](https://dx.doi.org/10.18653/V1/2024.ACL-LONG.810)Cited by:[Appendix A](https://arxiv.org/html/2609.25396#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.25396#S1.SS0.SSS0.Px1.p3.1),[§1](https://arxiv.org/html/2609.25396#S1.p1.1)\.
- Sousaet al\.\(2018\)M\. Sousa, I\. Dillig, and S\. K\. LahiriVerified three\-way program merge\.Proc\. ACM Program\. Lang\.2\(OOPSLA\),pp\. 165:1–165:29\.External Links:[Link](https://doi.org/10.1145/3276535),[Document](https://dx.doi.org/10.1145/3276535)Cited by:[Appendix A](https://arxiv.org/html/2609.25396#A1.SS0.SSS0.Px3.p1.1)\.
- Wanget al\.\(2025\)X\. Wang, B\. Li, Y\. Song, F\. F\. Xu, X\. Tang, M\. Zhuge, J\. Pan, Y\. Song, B\. Li, J\. Singh, H\. H\. Tran, F\. Li, R\. Ma, M\. Zheng, B\. Qian, Y\. Shao, N\. Muennighoff, Y\. Zhang, B\. Hui, J\. Lin,et al\.OpenHands: an open platform for AI software developers as generalist agents\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,External Links:[Link](https://openreview.net/forum?id=OJd3ayDDoF)Cited by:[§4](https://arxiv.org/html/2609.25396#S4.SS0.SSS0.Px3.p1.1)\.
- Wuet al\.\(2023\)Q\. Wu, G\. Bansal, J\. Zhang, Y\. Wu, B\. Li, E\. Zhu, L\. Jiang, X\. Zhang, S\. Zhang, J\. Liu, A\. H\. Awadallah, R\. W\. White, D\. Burger, and C\. WangAutoGen: enabling next\-gen LLM applications via multi\-agent conversation\.CoRRabs/2308\.08155\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2308.08155),[Link](https://arxiv.org/abs/2308.08155),2308\.08155Cited by:[Appendix A](https://arxiv.org/html/2609.25396#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.25396#S1.SS0.SSS0.Px1.p3.1)\.
- Xia and Park \(2026\)H\. Xia and Y\. ParkSlotGuard: stop oversharing private local context in llm agent transcripts\.InSecond Workshop on Agents in the Wild: Safety, Security, and Beyond \(AIWILD\), ICML 2026,Cited by:[§5](https://arxiv.org/html/2609.25396#S5.SS0.SSS0.Px2.p1.1)\.
- Yanget al\.\(2024\)J\. Yang, C\. E\. Jimenez, A\. Wettig, K\. Lieret, S\. Yao, K\. Narasimhan, and O\. PressSWE\-agent: agent\-computer interfaces enable automated software engineering\.InAdvances in Neural Information Processing Systems 37: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 \- 15, 2024,A\. Globersons, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. M\. Tomczak, and C\. Zhang \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2024/hash/5a7c947568c1b1328ccc5230172e1e7c-Abstract-Conference.html)Cited by:[Appendix A](https://arxiv.org/html/2609.25396#A1.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2609.25396#S4.SS0.SSS0.Px3.p1.1)\.