越多越好？金融审查任务中单智能体与多智能体 LLM 设置的比较

# The More the Better?A Comparison of Single- and Multi-Agent LLM Setups on a Financial Review Task
Source: [https://arxiv.org/html/2610.27488](https://arxiv.org/html/2610.27488)
###### Abstract

Practitioner guidance holds that a large language model reviews a document better when the work is divided among agents\. We test a narrow form of it in a controlled comparison of single\-agent and multi\-agent setups on a financial task, reviewing credit assessment memos against requirements\. One agent checks six requirements in one call\. A team of three on the same model checks the same memo, two requirements each, in the same words, with no roles or later stage\. Labels are known for every requirement on 300 synthetic memos, so misses and false alarms are measured separately\. Two primary predictions are tested on four models at a Bonferroni\-corrected level\. Under an instruction to decide each requirement once, the team catches 0\.08 to 0\.10 more of the defects that carry an explanation on the three models whose single agent misses some, at the corrected level on two\. It raises more false alarms on all four, at the corrected level on three\. Balanced accuracy, an exploratory measure, changes by 0\.02 or less, and the team uses 2\.2 to 2\.4 times the tokens\. Showing each agent the requirements it was not given reduces the added false alarms on the two Claude models, at the corrected level on one\. In exploratory comparisons, sampling the single agent three times and flagging on any draw gives higher balanced accuracy than the team on those models, and the team’s results depend on the instruction\.

Keywords\.LLM agents, multi\-agent systems, single\-agent baselines, task decomposition, LLM\-as\-a\-judge, cost\-aware evaluation

School of Engineering and Applied Science, George Washington University

Washington, DC, USA

pei\.liu@gwu\.edu

ORCIDhttps://orcid\.org/0009\-0001\-9092\-1086

## 1Introduction

Checking a document against a list of written requirements is routine work in finance, where a credit file, an audit paper or a disclosure must satisfy each one\. A common way to build such a review with a large language model is to split it among agents\. One agent checks the figures, another the calculations, a third the judgement calls, and their findings are merged\. A practitioner guide states the idea as a rule, that “for complex tasks with multiple considerations, LLMs generally perform better when each consideration is handled by a separate LLM call”\([Schluntz and Zhang 2024](https://arxiv.org/html/2610.27488#bib.bib12)\)\. The guide offers no data in support\.

Existing tests of the rule leave three things out\. Among those that divide the work without adding stages, none reports a reviewer’s miss rate together with a false\-alarm rate measured on cases known to be free of defects\. None gives the single agent as many calls as the divided reviewer, and none separates dividing the work from withholding what the other parts cover\. A team usually differs from the single agent in more than the division of work, through role descriptions, an orchestrator, a filter or a different model for one of its agents\.

A designer needs to know more than which arrangement scores higher\. A review that catches more defects and raises more false alarms is a different instrument, and whether it is a better one depends on what each error costs\.

We compare two arrangements with only the allocation of requirements to calls varied\. A single agent receives six requirements in one call\. A team of three agents on the same model receives the same six in identical words, two to a call, with no roles and no later stage\. The documents are synthetic credit memos whose labels are known by construction\. The comparison is a narrow form of the guide’s rule\. We give two considerations to a call where the rule speaks of one\. Our task is six checks of one kind on a short document, where the rule speaks of complex tasks\. Our main condition also switches off the models’ separate thinking channel and tells the reviewer to decide each requirement once\.

Two further arms separate the effect of division from the effects of what accompanies it\. One samples the single agent’s prompt three times, so that it uses as many calls as the team\. The other, added after the first results had been read, shows each team agent the requirements it was not given\.

Our contribution is to supply all three missing elements in one design\. The trade we find, more defects caught for more false alarms, reproduces earlier work\. The size of its false\-alarm side, measured on cells free of defects, is new, and so is the contrast with a team whose agents see what the others cover\. Two exploratory results are also new, a comparison of the trade with repeated draws of the single agent and its dependence on the instruction\.

## 2Related work

Teams against single agents\.[Tran and Kiela \(2026\)](https://arxiv.org/html/2610.27488#bib.bib14)give a single agent and five multi\-agent designs the same cap on thinking tokens and find the single agent ahead on average\.[Zhang et al\. \(2025\)](https://arxiv.org/html/2610.27488#bib.bib18)find that no debate framework beats chain\-of\-thought prompting in more than a fifth of their settings\.[Kim et al\. \(2025\)](https://arxiv.org/html/2610.27488#bib.bib7)relate a team’s benefit to task properties and to the single agent’s own score\. These studies score answers as right or wrong and so cannot say which of a reviewer’s two errors a design trades for the other\.

Division of a detection task\.On financial documents,[He et al\. \(2026\)](https://arxiv.org/html/2610.27488#bib.bib5)ask a model either for all error types at once or for one type in each query\. With the whole document in the prompt, asking for one type at a time raises average recall by 0\.08 to 0\.10 and cuts precision by more than half \[their Table 6\]\. The pattern reverses when the document is supplied in chunks, and they conclude from F1 that asking for all types at once is better\.[Liu et al\. \(2026\)](https://arxiv.org/html/2610.27488#bib.bib9)find that a team with no filtering stage, on one model, raises recall from 58\.7 to 75\.0 percent and lowers precision from 91\.5 to 74\.2 \[their Table 2\], and that a single agent given the team’s instruction also gains recall and loses precision, by a smaller amount\. Their filter restores the precision\. With added stages or debate a team can instead raise fewer false alarms and catch less\.[Li et al\. \(2026\)](https://arxiv.org/html/2610.27488#bib.bib8)report a team with a conductor, a judge and a reranker that passes 85 percent of defect\-free artifacts against 48 for a single detector\. With the reranker removed the team still passes 64 percent, and they test no team with all of those stages removed\.[Chen et al\. \(2025\)](https://arxiv.org/html/2610.27488#bib.bib2)find that two earlier debate protocols mostly raise precision and lower recall, and[D’Arcy et al\. \(2024\)](https://arxiv.org/html/2610.27488#bib.bib3)that a team given the single agent’s task prompt matches fewer of the human reviewers’ comments at slightly higher precision\.

Criteria in one call or several\.Two studies vary only the criteria in a judge call, as we do\.[Peng et al\. \(2026\)](https://arxiv.org/html/2610.27488#bib.bib10)find that balanced accuracy falls once a call holds four or five rubrics on long coding\-agent trajectories, and changes little on shorter reports\.[Rao and Callison\-Burch \(2026\)](https://arxiv.org/html/2610.27488#bib.bib11)find that on their two binary checklists a whole rubric in one call and one criterion at a time are equally accurate, while 3 to 14 percent of individual verdicts differ\. Neither reports the miss rate and the false\-alarm rate for each arrangement\.[Yu et al\. \(2026\)](https://arxiv.org/html/2610.27488#bib.bib17)attribute a team’s benefit to isolated contexts and find that a single agent given the team’s divided instructions in one context scores below the single agent with undivided instructions on most tasks \[their Table 3\]\. Work on judges that assess the quality of an output, as opposed to detecting defects in a document, gives mixed results on the same question\([Lee et al\. 2025](https://arxiv.org/html/2610.27488#bib.bib22);[Xiong et al\. 2025](https://arxiv.org/html/2610.27488#bib.bib24);[Sunkavalli 2026](https://arxiv.org/html/2610.27488#bib.bib23)\), and none of those tasks has defect\-free cases\.

Repeated calls\.A fair baseline lets the single agent make as many calls as the team\. Sampling a model several times and taking the most frequent answer goes back to self\-consistency decoding\([Wang et al\. 2023](https://arxiv.org/html/2610.27488#bib.bib19)\)\.[Li et al\. \(2024\)](https://arxiv.org/html/2610.27488#bib.bib21)treat each sample of one model as an agent and report that accuracy under a vote rises with the number of samples, up to 40, on reasoning and code benchmarks\.[Chen et al\. \(2024\)](https://arxiv.org/html/2610.27488#bib.bib1)show that a majority over repeated calls raises accuracy on queries where the correct answer is the single call’s most likely answer and lowers it on the others\.[Jain et al\. \(2025\)](https://arxiv.org/html/2610.27488#bib.bib6)let four of fourteen judge models reject an item\.[Teo et al\. \(2026\)](https://arxiv.org/html/2610.27488#bib.bib13)count a contradiction as detected when any of several reviews finds it, and report no false\-alarm rate for the reviewing system\.

Framing and gap\.We take our framing from[Fox et al\. \(2026\)](https://arxiv.org/html/2610.27488#bib.bib4)\. Of every remedy applied to a judge they ask whether it “slides the point along a curve or lifts the curve”, and they find that wording, voting and prompt search slide it\. What lifts it in their study is a two\-stage pipeline that extracts the facts to check and verifies each in its own call\. The pipeline adds a stage and a closed output format, so it is not a division of one task among calls of the kind we test\. Two controls in the studies above resemble the scoped team that Section 3\.3 defines, and differ from it\. Liu et al\. give a single agent the team’s instruction, and Yu et al\. give a single agent the team’s divided instructions in one context\. Both alter the single agent\. The scoped team leaves the division in place and restores information to the divided agents\. Among the studies we examined, we found none that sets a division without added stages against repeated draws, or that separates division from the loss of that information\.

## 3Experimental design

### 3\.1Setting

We study one situation\. Several agents built on the same model each receive the same complete document, and each is responsible for part of a single review whose parts draw on overlapping passages\. The agents run in parallel, do not communicate and have no tools, and the document fits comfortably in one context window\. A team can help for reasons that this design excludes, such as a task that does not fit in one context, subtasks that share no input, or agents with different models or tools\. Our result says nothing about those cases\.

### 3\.2Task and ground truth

The task is the review of a credit assessment memo against six written requirements\. Each memo is written by a program, with no language model, from one record of a public credit card dataset\([Yeh and Lien 2009](https://arxiv.org/html/2610.27488#bib.bib15);[Yeh 2009](https://arxiv.org/html/2610.27488#bib.bib16)\)\. A memo has about 1,100 words in eight sections, and we use 300 of them\.

Table 1:The six requirements and the three pairs in which the team receives them\.Each requirement is labelled on each memo\. We call one requirement on one memo a cell, so there are 1,800 cells\. A cell is clean when every statement the requirement governs is correct\. Otherwise it has one of three kinds of defect\. We writeyi​j=1y\_\{ij\}=1when memoiihas a defect under requirementjjandyi​j=0y\_\{ij\}=0when the cell is clean\. A bare defect is a statement that conflicts with another statement of the same fact, with no reason given\. A masked defect comes with its own explanation and is consistent in itself, yet contradicts other facts in the memo\. An arithmetic defect is a written\-out sum or ratio that is wrong\. Under R1, for example, one memo with a bare defect gives monthly income as NT$23,200 and then computes a ratio “against monthly income of NT$46,900”\. One with a masked defect gives monthly income as NT$37,000 and then states that the servicing test “was run on monthly income of NT$31,700, being the stated figure net of NT$5,300 in non\-recurring overtime”\. There are 1,136 clean cells, and 205 bare, 229 masked and 230 arithmetic defects\. That puts a defect in 37 percent of cells, far more than in practice\. The median memo has two defects, and 21 memos have none\.

A label was accepted when three sources agreed\. The first was the generator that set it\. The second was a separate program that recomputed it from the finished text\. The third was Claude Opus 5\.5, a larger model than any we test, which read the memo without seeing either label\. All 1,800 labels passed after one revision to the wording of two requirements, which Appendix A describes\. Every defect is an inconsistency inside the document that a program can verify\. Omissions, errors against outside facts and matters of judgement are not in the test\.

### 3\.3Arms

By “agent” we mean one call of a model with its own instructions and its own context, and by “team” several agents whose verdicts are taken together\. Each agent is a single call, so the design can equally be described as a decomposition of a judge prompt, and[Schluntz and Zhang \(2024\)](https://arxiv.org/html/2610.27488#bib.bib12)would class it as a parallel workflow\. Our team is the simplest kind, with no orchestrator or filter, and in that respect resembles the architecture that[Kim et al\. \(2025\)](https://arxiv.org/html/2610.27488#bib.bib7)call independent\. Four arms review every memo\.

LetRRbe the set of six requirements anddid\_\{i\}memoii\. A callf⁡\(di,S,C\)f\(d\_\{i\},S,C\)gives the model the memo, a setS⊆RS\\subseteq Rof requirements to judge and a setC⊆R∖SC\\subseteq R\\setminus Sof requirements shown for context only\. It returns a verdictvi​j∈\{0,1\}v\_\{ij\}\\in\\\{0,1\\\}for eachj∈Sj\\in S, where 1 is a flag\. The model, the memo, the wording of each requirement and the rest of the prompt are the same in every call\. The arms differ inSS, inCCand in the number of calls\. They differ otherwise only in the order of the requirement list, given at the end of this section, and in the seeds and, on the Qwen3\.5 models, the sampling temperature of the three draws, which Section 3\.4 describes\.

Single agentf⁡\(di,R,∅\)Teamf⁡\(di,Pm,∅\),m=1,2,3Scoped teamf⁡\(di,Pm,R∖Pm\),m=1,2,3Three drawsvi​j=1​if​∑s=13vi​j\(s\)≥t\\begin\{array\}\[\]\{@\{\}ll@\{\}\}\\textrm\{Single agent\}&f\(d\_\{i\},R,\\emptyset\)\\\\\[2\.0pt\] \\textrm\{Team\}&f\(d\_\{i\},P\_\{m\},\\emptyset\),\\quad m=1,2,3\\\\\[2\.0pt\] \\textrm\{Scoped team\}&f\(d\_\{i\},P\_\{m\},R\\setminus P\_\{m\}\),\\quad m=1,2,3\\\\\[2\.0pt\] \\textrm\{Three draws\}&v\_\{ij\}=1\\textrm\{ if \}\\sum\_\{s=1\}^\{3\}v\_\{ij\}^\{\(s\)\}\\geq t\\end\{array\}
Here\{P1,P2,P3\}\\\{P\_\{1\},P\_\{2\},P\_\{3\}\\\}is the partition ofRRinto the three pairs of Table[1](https://arxiv.org/html/2610.27488#S3.T1), and a requirement’s verdict comes from the call whoseSScontains it\. The pairs are thematic and fixed, and we test this one partition only\. The single agent is the partition with one part, and the guide’s rule of one consideration to a call is the partition with six\.

The team changes two things at once\. A team agent holds fewer requirements, and it does not see the four it was not given\. The scoped team separates the two by settingCCto those four, which appear under a heading that says other reviewers judge them\.

For the three draws,vi​j\(s\)v\_\{ij\}^\{\(s\)\}is the verdict of thess\-th sample of the single agent’s call andttis the number of flags required\. The majority rule ist=2t=2, the any\-of\-three rule ist=1t=1, and the all\-of\-three rule,t=3t=3, is used only for the line in Figure[1](https://arxiv.org/html/2610.27488#S4.F1)\. In the team a single call decides each requirement, so the majority rule is stricter for flagging than the team’s\.

The order of the requirements in the single agent’s list is rotated across memos and across draws\. The order within a team call is fixed\.

### 3\.4Models and instruction

We use four models from two families\. Qwen3\.5 at 9B and 27B parameters run locally, and Claude Haiku 5\.5 and Claude Sonnet 5\.5 through a batch interface\. Every comparison is within one model\. Answers are returned under a JSON schema that holds, for each requirement, a reasoning string, a verdict of PASS or FAIL and a quoted evidence sentence\. Separate thinking channels are switched off, and the output cap is 2,048 tokens for each requirement in every arm\. The Qwen3\.5 models are sampled at temperature 0, and at 0\.7 for the three draws, which would otherwise be near copies of one another\. The Claude models accept no temperature setting and use default sampling in every arm\. Comparisons between the three draws and the other arms are therefore confounded with temperature on the Qwen3\.5 models and not on the Claude models\.

All arms carry the same instruction on how to reason\. The reviewer is told to take each requirement once, to say in at most three sentences what it compared, and not to return to a requirement after deciding it\. We call this the decide\-once instruction\. We made it the main condition before the runs reported here, for a measurement reason\. Left to reason freely in a preliminary run, which Appendix C describes, Qwen3\.5 27B returned no verdict on about 6 percent of calls\. A failed single\-agent call forfeits six verdicts and a failed team call two, so the losses would have fallen unevenly on the arms\. The instruction departs from default practice, and it has effects of its own\. On 100 of the 300 memos, selected with a fixed seed, every arm is also run without it, and Section 4\.4 reports what changes for the single agent and the team\.

### 3\.5Scoring and analysis

A flag is a verdict of FAIL, and a false alarm is a flag on a clean cell\. For an armaawith verdictsvi​jav\_\{ij\}^\{a\}, whereaais single, team, scoped, or a rule on the three draws, we report the share of defects caught,HaH\_\{a\}, and the false\-alarm rate on clean cells,FaF\_\{a\}\.

Ha=∑i,jyi​j​vi​ja∑i,jyi​j,Fa=∑i,j\(1−yi​j\)​vi​ja∑i,j\(1−yi​j\)H\_\{a\}=\\frac\{\\sum\_\{i,j\}y\_\{ij\}\\,v\_\{ij\}^\{a\}\}\{\\sum\_\{i,j\}y\_\{ij\}\},\\qquad F\_\{a\}=\\frac\{\\sum\_\{i,j\}\(1\-y\_\{ij\}\)\\,v\_\{ij\}^\{a\}\}\{\\sum\_\{i,j\}\(1\-y\_\{ij\}\)\}
Within one class of defect, with both sums restricted to that class, the share caught is writtenHabareH\_\{a\}^\{\\mathrm\{bare\}\},HamaskedH\_\{a\}^\{\\mathrm\{masked\}\}orHaarithH\_\{a\}^\{\\mathrm\{arith\}\}\. Balanced accuracy isBa=\(Ha\+1−Fa\)/2B\_\{a\}=\(H\_\{a\}\+1\-F\_\{a\}\)/2, which weights the two errors equally whatever their frequency\. We writeΔ​H=Hteam−Hsingle\\Delta H=H\_\{\\mathrm\{team\}\}\-H\_\{\\mathrm\{single\}\}for the team’s value minus the single agent’s, andΔ​F\\Delta FandΔ​B\\Delta Blikewise\. Other contrasts are written out, as inFscoped−FteamF\_\{\\mathrm\{scoped\}\}\-F\_\{\\mathrm\{team\}\}\. An arm returns verdicts and not scores, so each arm gives a single operating point\(Fa,Ha\)\(F\_\{a\},H\_\{a\}\)\. The sums run over all memos, so rates are pooled over memos and, for all defects, over the three classes\. The tokensKaK\_\{a\}of an arm are its input and output tokens together\.

A missing verdict counts against the arm, and intervals are bootstrap intervals over memos, at 95 percent unless stated\. We call a contrast significant when its interval excludes zero\. Appendix A gives both rules in full\. Each arm was run once, so the intervals cover the sampling of memos and the noise of calls within a run, and do not cover variation between whole runs\. Repeated calls exist for the single agent’s prompt only, and Table[A8](https://arxiv.org/html/2610.27488#A5.T8)gives their spread\.

The memos, prompts, predictions and analysis code were fixed by a committed hash before each family was run\. Five predictions were set from a pilot, an earlier run of the same comparison on a separate memo set that contributes no number reported here\. In the pilot the team’s gain appeared on defects that carry an explanation and not on bare ones, which is why the masked contrast is primary\. Two were primary, thatΔ​Hmasked\>0\\Delta H^\{\\mathrm\{masked\}\}\>0and thatΔ​F\>0\\Delta F\>0\. Each model family has two models, so the two primary predictions give four tests per family\. Every primary test is reported at the Bonferroni level of 98\.75 percent, which we call the corrected level, and the secondary predictions are tested at 95 percent\. At that level the interval on the masked contrast has a half\-width of about 0\.11 on Qwen3\.5 9B, 0\.08 on Qwen3\.5 27B and 0\.07 on Haiku 5\.5, so an effect near 0\.08 could be detected on some models and not on others\. Three predictions were secondary\. They were that the team and the single agent do not differ on bare defects, that the team catches more masked defects than a majority of three draws, and that its excess of false alarms is smaller on the larger model of a family\.

The scoped team was added after the other arms had been read\. Its two predictions, fixed before it was run, were that it raises fewer false alarms than the team and that it still catches more masked defects than the single agent\. Neither is a primary prediction of the study\. The idea came from reading the Sonnet 5\.5 team’s false alarms, and the arm is compared with that same team run\. We report its contrasts at the corrected level, and Appendix B notes which also hold at 99\.375 percent, the level for eight tests per family\.

Everything else is exploratory\. That covers balanced accuracy, the signal detection analysis of Appendix D, the any\-of\-three rule and the comparison of arms with the line through the three\-draw rules, the effect of the instruction, the break\-even costs, and the rates by requirement and by memo\. Appendix B lists every prediction with its outcome\.

## 4Results

No arm on any model lost more than two calls under the decide\-once instruction\. Table[2](https://arxiv.org/html/2610.27488#S4.T2)reports the rates and Figure[1](https://arxiv.org/html/2610.27488#S4.F1)shows each arm as an operating point\. The team uses 2\.2 to 2\.4 times the single agent’s tokens, and the three draws use more\.

Table 2:Share of defects caught by class and over all defects,HH, false alarms on clean cells,FF, balanced accuracy,BB, and tokens relative to the single agent\. The majority rule flags when at least two of three draws flag, and the any rule when at least one does\. All 300 memos, decide\-once instruction\.Figure 1:Each arm as an operating point, false alarms on clean cells against the share of all defects caught, with the Claude models enlarged on the right\. A thick segment joins a model’s single agent to its team, and a star marks the scoped team\. A thin line joins three rules applied to three draws of the single agent’s prompt, flagging when all, at least two or any of the draws flag\. Bars on the single agent and the team are 95 percent intervals\. Grey lines join points of equal balanced accuracy\.### 4\.1The team flags more, and catches more below ceiling

Figure 2:The team against the single agent, on one scale\. The contrasts in masked defects caught,Δ​Hmasked\\Delta H^\{\\mathrm\{masked\}\}, and in false alarms,Δ​F\\Delta F, are primary, with intervals at the corrected level\. The contrast in balanced accuracy,Δ​B\\Delta B, is exploratory, with 95 percent intervals\. A filled marker has an interval that excludes zero, and the number beside each row is the estimate\. Table[A1](https://arxiv.org/html/2610.27488#A2.T1)gives the bounds\.On the three models whose single agent leaves masked defects uncaught, the team’s share caught is higher by 0\.08 to 0\.10, as Figure[2](https://arxiv.org/html/2610.27488#S4.F2)shows\. The prediction is confirmed at the corrected level on Qwen3\.5 27B and Haiku 5\.5 and not on Qwen3\.5 9B, where the estimate equals that on Qwen3\.5 27B and the interval is wider\. The team raises more false alarms on every model, at the corrected level on three\. The secondary prediction of no difference on bare defects was wrong\. On the three models below ceiling the team catches more of them, as Appendix B records\. In Figure[1](https://arxiv.org/html/2610.27488#S4.F1)the segment from single agent to team rises to the right on those models\.

With the two errors weighted equally the effects largely cancel\. Balanced accuracy changes by 0\.02 or less on every model, and all four intervals lie within 0\.04 of zero\. We set no equivalence margin in advance, and the change is significant on the two Claude models, a rise on Haiku 5\.5 and a fall on Sonnet 5\.5\. Sonnet 5\.5’s single agent catches every bare and masked defect, and its team catches two fewer defects and adds 15 false alarms\.

As predicted for the larger model of a family, the team’s excess of false alarms is smaller on Qwen3\.5 27B than on Qwen3\.5 9B, as Appendix B records\. In proportion to each model’s own false\-alarm rate the two excesses are about the same, and the two Claude models do not differ\.

The pooled rates conceal changes in opposite directions\. The team’s share of defects caught rises on some requirements and falls on others, as Table[A7](https://arxiv.org/html/2610.27488#A5.T7)shows, so the trade describes the review as a whole and not each part of it\. On the pooled rates, an exploratory signal detection analysis in Appendix D is consistent with a lower threshold for flagging and an unchanged ability to tell defective cells from clean ones, and does not establish that account\.

### 4\.2Scope information

We read the 15 false alarms of the Sonnet 5\.5 team\. Most fall on requirement R1\. The agent holding R1 flags the sentence that adds a housing allowance to monthly income\. That sentence falls under R3, a requirement this agent was not shown\. The single agent holds both and does not make the error\. By the rates in Table[A7](https://arxiv.org/html/2610.27488#A5.T7), R1 and R3 account for about 0\.8 of the team’s added false alarms on Sonnet 5\.5 and Qwen3\.5 27B\. On Haiku 5\.5 most of the addition falls on R4, and on Qwen3\.5 9B it is spread across five requirements\. The scoped team restores to each agent the requirements it was not shown\.

Figure 3:The scoped team, on the scale of Figure[2](https://arxiv.org/html/2610.27488#S4.F2)\. Its contrast with the single agent in masked defects caught and its contrast with the team in false alarms are the two pre\-specified contrasts, with intervals at the corrected level\. Its contrast with the team in all defects caught is exploratory, with 95 percent intervals\. Markers and numbers are as in Figure[2](https://arxiv.org/html/2610.27488#S4.F2)\. Table[A2](https://arxiv.org/html/2610.27488#A2.T2)gives the bounds\.The scoped team raises fewer false alarms than the team on Sonnet 5\.5, at the corrected level, as Figure[3](https://arxiv.org/html/2610.27488#S4.F3)shows\. The reduction on Haiku 5\.5 is the same size and is significant at 95 percent, \[−\-0\.021,−\-0\.002\]\. On Qwen3\.5 27B the reduction is not significant, and on Qwen3\.5 9B there is none\. The scoped team still catches more masked defects than the single agent on Haiku 5\.5\. That prediction is not confirmed on Qwen3\.5 9B, and on Qwen3\.5 27B the interval reaches zero\.

The reduction may not be free\. On Qwen3\.5 27B and Haiku 5\.5 the scoped team’s share of all defects caught is about 0\.02 below the team’s, mostly through arithmetic defects\. On those two models each 0\.01 taken off the false\-alarm rate comes with a fall of about 0\.015 in the share of defects caught, which is not significant\. Sonnet 5\.5 gives up nothing\.

Scope information may act by telling the agent what lies outside its requirements, or by making it more cautious because other reviewers are named\. Our prompt does not separate the two\. In Table[2](https://arxiv.org/html/2610.27488#S4.T2), Haiku 5\.5’s scoped team has the single agent’s false\-alarm rate, catches more masked defects and catches fewer arithmetic defects\. The scoped prompt adds about 160 to 250 input tokens a call to the team’s\.

### 4\.3Repeated draws

The remaining secondary prediction was that the team catches more masked defects than a majority of three draws\. It is confirmed on Haiku 5\.5 and not on Qwen3\.5 27B or Sonnet 5\.5\. On Qwen3\.5 9B the estimate is above zero, but on the Qwen3\.5 models the comparison is confounded with temperature\. Appendix B gives the estimates\. A draw of the 9B model catches fewer masked defects than its single agent at temperature 0, as Table[A8](https://arxiv.org/html/2610.27488#A5.T8)shows\.

Under the exploratory any\-of\-three rule the three\-draw agent catches more defects than the team on all four models, and its balanced accuracy is no lower than the team’s on any\. On the two Claude models it is higher by 0\.024 and 0\.011, both significant\.

A further exploratory check, unlike the signal detection analysis of Appendix D, makes no distributional assumption\. It compares each arm with the line in Figure[1](https://arxiv.org/html/2610.27488#S4.F1)that joins the three rules on three draws, taken at the arm’s own false\-alarm rate\. Haiku 5\.5’s team lies 0\.038 below the line \[−\-0\.065, 0\.003\], and its single agent 0\.052 below \[−\-0\.093,−\-0\.018\]\. On that model repeated draws therefore attain operating points that one call does not, and probably points that the team does not\. On the Qwen3\.5 models the team is 0\.002 above the line on the 9B model and 0\.032 below it on the 27B model, neither significant\. The check cannot be made on Sonnet 5\.5, which is at ceiling\.

The draws also show how much repeated calls of one prompt vary\. They disagree on 0\.39 of cells on Qwen3\.5 9B, 0\.21 on Qwen3\.5 27B, 0\.11 on Haiku 5\.5 and under 0\.01 on Sonnet 5\.5\. By the repeats in Table[A8](https://arxiv.org/html/2610.27488#A5.T8), the team’s share of masked defects caught on Haiku 5\.5 lies outside the range of four calls of the single agent’s prompt, while on Qwen3\.5 27B three draws vary as much as the team’s gain\. The single agent’s share of defects caught also differs across the three rotations of its list, by up to 0\.14 on Qwen3\.5 9B, 0\.09 on Qwen3\.5 27B and 0\.05 on Haiku 5\.5\. Each rotation covers a different 100 memos, so the order of the list and the memos are not separated\.

### 4\.4The decide\-once instruction, exploratory

On the 100 memos run with and without the instruction, removing it raises the single agent’s share of defects caught\. With calls that share a prompt and settings combined, masked defects caught rise significantly on the three models below ceiling, as Table[A3](https://arxiv.org/html/2610.27488#A3.T3)shows\. Haiku 5\.5’s false alarms do not rise with them\. For the Qwen3\.5 models these combined calls are the draws at temperature 0\.7\. In the single call at temperature 0, which Table[A3](https://arxiv.org/html/2610.27488#A3.T3)also reports, arithmetic defects caught rise on both models and masked defects caught do not rise significantly on either\.

The effect of division also depends on the instruction, as Table[3](https://arxiv.org/html/2610.27488#S4.T3)shows\. Without it the team’s excess of false alarms is larger on all four models, significantly on Qwen3\.5 27B, \+0\.088 \[0\.013, 0\.161\], and Sonnet 5\.5, \+0\.013 \[0\.003, 0\.026\]\. The team’s balanced accuracy is then significantly lower than the single agent’s on Qwen3\.5 27B and Sonnet 5\.5 and significantly higher on Haiku 5\.5\. On Qwen3\.5 9B the difference is not significant\. On Qwen3\.5 27B the team’s gain in defects caught disappears while its excess of false alarms grows\. Team calls cut off at the output cap account for part of this, as Appendix C records\. On completed verdicts alone the team still raises more false alarms and catches almost no more defects\.

Table 3:The contrastsΔ​H\\Delta Hover all defects,Δ​F\\Delta FandΔ​B\\Delta Bon the 100 paired memos, with the decide\-once instruction and without it\. 95 percent intervals\.Part of the team’s gain on the Qwen3\.5 models under the instruction may be a recovery of detection that the instruction takes from the single agent\. A direct comparison on these 100 memos, the single agent without the instruction against the team with it, does not settle the question, as Table[A4](https://arxiv.org/html/2610.27488#A3.T4)shows\.

### 4\.5What the trade costs, exploratory

Table[4](https://arxiv.org/html/2610.27488#S4.T4)expresses the trade of Section 4\.1 as the rise in the false\-alarm rate for each unit rise in the share of all defects caught\. Whether the exchange is worthwhile depends on how common defects are\. Letppbe the share of cells that have a defect,wMw\_\{M\}the cost of a missed defect andwFw\_\{F\}the cost of a false alarm\. With cost taken as linear in the two errors, armaahas an expected cost of errors per cell of

Ea=p⁡\(1−Ha\)​wM\+\(1−p\)​Fa​wF\.E\_\{a\}=p\\,\(1\-H\_\{a\}\)\\,w\_\{M\}\+\(1\-p\)\\,F\_\{a\}\\,w\_\{F\}\.
WithΔ​H\>0\\Delta H\>0over all defects, the team has the lower expected cost when

wMwF\>Δ​FΔ​H⋅1−pp\.\\frac\{w\_\{M\}\}\{w\_\{F\}\}\>\\frac\{\\Delta F\}\{\\Delta H\}\\cdot\\frac\{1\-p\}\{p\}\.
Table 4:Team against single agent under the decide\-once instruction\. The second column isΔ​F/Δ​H\\Delta F/\\Delta H, withΔ​H\\Delta Hover all defects and a 95 percent interval\. The others are the break\-even value ofwM/wFw\_\{M\}/w\_\{F\}at four values of the defect sharepp, assuming the rates carry over\.At the defect share of our memos the break\-even cost is between 0\.4 and 1\.3, and at shares of 0\.05 to 0\.01 it is roughly 10 to 60 times higher\. The ratios are imprecise\. At a share of 0\.02 the interval on Qwen3\.5 27B alone gives break\-even costs from 10 to 134\. Sonnet 5\.5 has no row, since its team catches no more than its single agent\.

Two cautions apply to Table[4](https://arxiv.org/html/2610.27488#S4.T4)\. It uses the false\-alarm rate by cell, which probably overstates the team’s cost, as Section 5 explains\. It also compares the team with the single agent only\. We set the five rows that Table[2](https://arxiv.org/html/2610.27488#S4.T2)gives for each model side by side at a defect share of 0\.02, for costs of a miss from 1 to 200 times that of a false alarm\. A majority of three draws has the lowestEaE\_\{a\}while a miss costs less than roughly 15 to 55 false alarms, depending on the model\. Above that the any\-of\-three rule has the lowest\. The team has the lowest expected cost at none of the ratios we tried, on any model\. On Qwen3\.5 9B it lies on the line joining the two three\-draw rules, so the ranking there is not robust\. Table[A10](https://arxiv.org/html/2610.27488#A5.T10)reports the share of flags that would be correct at three defect shares\.

The unit of work in practice is the file\. The share of the 300 memos with at least one false alarm is higher for the team than for the single agent on every model, by 0\.04 to 0\.19\. Table[A6](https://arxiv.org/html/2610.27488#A5.T6)reports related counts for whole memos\.

## 5Threats to validity and follow\-up studies

Four threats could change a conclusion\. For the first three we give the study that would address it, and we rank those studies by how far they could change these conclusions\.

How well the effects are measured\.On Haiku 5\.5 most of the team’s flags on clean cells quote a sentence that is defective under another requirement, and one of the single agent’s does\. We call these misfiled flags, and Table[A9](https://arxiv.org/html/2610.27488#A5.T9)counts them for every model\. The count rests on an exploratory match of text, supported by a hand check of twelve flags\. The team’s cost in false alarms is therefore probably overstated on Qwen3\.5 27B, Haiku 5\.5 and Sonnet 5\.5\. Its gain on masked defects is less secure on Qwen3\.5 27B, where three draws of one prompt vary as much as the gain, than on Haiku 5\.5, where the team lies outside the range of four calls\. A study that scores flags by what they quote, repeats its arms and samples the three draws at the temperature of the other arms would establish whether the false\-alarm cost that defines the trade is real\. We rank it second\.

The instruction\.Every primary test was run under the decide\-once instruction, and Section 4\.4 shows that on Qwen3\.5 27B the team’s balanced accuracy does not differ significantly from the single agent’s with the instruction and is lower without it\. The primary results hold for one way of telling the reviewer to reason\. Crossing the division with the instruction on the full memo set is the study we rank first, since this is the only threat for which our own data show a result changing\.

The division\.We test one pairing, two requirements to a call\. R1 and R3 fall in different pairs, and Section 4\.2 traces most of the team’s added false alarms on two models to them\. The effect sizes are specific to this pairing, and the guide’s rule as stated, one consideration to a call, is untested\. Varying the division, down to one requirement a call, would test it and ranks third\.

Generality\.The memos are short and synthetic, with internal inconsistencies in 37 percent of cells, and the strongest model is at ceiling\. Neither the rates nor the costs should be assumed to carry over to long documents with rare and varied defects, where outcomes should also be reported by file\.

## 6Guidance for agent setup

Whether a division of this kind helps depends on what the single agent misses, on the model, and on how the reviewer is told to reason\. Three implications for the design of a review system follow, ordered by the strength of the evidence behind them\. All three assume that rates measured on these memos carry over to the designer’s documents, which we have not shown\.

The first rests on the primary comparison and holds under the decide\-once instruction for the pairing we tested\. Division should be treated as a change in what is flagged and not as an improvement, and its cost in false alarms should be worked out before it is adopted\. The gain is bounded by what the single agent misses, and the false alarms are not, as the Sonnet 5\.5 rows of Table[2](https://arxiv.org/html/2610.27488#S4.T2)show\. A designer therefore needs the single agent’s miss rate and false\-alarm rate on cases known to be clean, and the defect share of the setting\. With those, the exploratory calculation behind Table[4](https://arxiv.org/html/2610.27488#S4.T4)gives the cost of a miss above which division is worthwhile relative to a single call\.

The second rests on an exploratory comparison that is free of the temperature confound on the two Claude models only\. The single agent sampled three times is the baseline against which a divided review should be judged, since it uses the same number of calls\. On those models the any\-of\-three rule gives higher balanced accuracy than the team, and a majority of three raises no more false alarms than one call, at a higher token cost than the team, as Table[2](https://arxiv.org/html/2610.27488#S4.T2)shows\. Which rule suits a setting depends, as before, on the cost of a miss\.

The third rests on 100 memos and is the least secure\. How the reviewer is told to reason should be fixed and reported before arrangements are compared, because the instruction changed the effect of division\. On Qwen3\.5 27B the team’s balanced accuracy does not differ significantly from the single agent’s with the instruction and is lower without it, as Table[3](https://arxiv.org/html/2610.27488#S4.T3)shows\. Removing a constraint on reasoning also risks calls that return no verdict\.

We recommend one practice whatever the follow\-up studies of Section 5 find\. A divided review should be judged on misses and false alarms separately, on cases known to be clean, and against the single agent given the same number of calls\.

## 7Conclusion

Under the decide\-once instruction, dividing six requirements among three agents on one model trades false alarms for defects caught, and with the two errors weighted equally the effects largely cancel, as Figure[2](https://arxiv.org/html/2610.27488#S4.F2)shows\. The trade of more defects caught for more false alarms is what[He et al\. \(2026\)](https://arxiv.org/html/2610.27488#bib.bib5)found with the whole document in the prompt and what[Liu et al\. \(2026\)](https://arxiv.org/html/2610.27488#bib.bib9)found for a team with no filter, and the small change in accuracy is what[Rao and Callison\-Burch \(2026\)](https://arxiv.org/html/2610.27488#bib.bib11)found for binary checklists\. In the terms of[Fox et al\. \(2026\)](https://arxiv.org/html/2610.27488#bib.bib4), the exploratory signal detection analysis of Appendix D is consistent with division sliding the operating point and does not establish it\. Repeated draws behave differently in our data from theirs\. On Haiku 5\.5 three draws attain operating points that one call does not, as Section 4\.3 shows\. Their task and their voting rule differ from ours, and we have not tested which of the two explains the disagreement\.

Besides the exploratory comparisons, two things are new\. The false\-alarm rate is measured on cells known to be clean, which lets the trade be sized\. The scoped team, an arm added after the first results, raises fewer false alarms than the team on the two Claude models, at the corrected level on one\.

## References

- Chenet al\.\(2024\)L\. Chen, J\. Q\. Davis, B\. Hanin, P\. Bailis, I\. Stoica, M\. Zaharia, and J\. ZouAre more LLM calls all you need? Towards the scaling properties of compound AI systems\.InAdvances in Neural Information Processing Systems 37,pp\. 45767–45790\.Note:arXiv:2403\.02419External Links:[Document](https://dx.doi.org/10.52202/079017-1455)Cited by:[§2](https://arxiv.org/html/2610.27488#S2.p4.1)\.
- Chenet al\.\(2025\)Y\. Chen, G\. Niu, J\. Cheng, B\. Han, and M\. SugiyamaWhen and why does multi\-agent debate fail and does it really underperform?\.Note:arXiv:2510\.20963Cited by:[§2](https://arxiv.org/html/2610.27488#S2.p2.1)\.
- D’Arcyet al\.\(2024\)M\. D’Arcy, T\. Hope, L\. Birnbaum, and D\. DowneyMARG: multi\-agent review generation for scientific papers\.Note:arXiv:2401\.04259Cited by:[§2](https://arxiv.org/html/2610.27488#S2.p2.1)\.
- Foxet al\.\(2026\)S\. Fox, L\. Markham, R\. Lail, and M\. KarotsierisLLM judges verify presence, not absence: omission blindness in AI clinical notes and what recovers it\.Note:arXiv:2608\.31016Cited by:[§2](https://arxiv.org/html/2610.27488#S2.p5.1),[§7](https://arxiv.org/html/2610.27488#S7.p1.1)\.
- Heet al\.\(2026\)Y\. He, Z\. Gu, Z\. Hu, Y\. Zhou, H\. Shen, J\. Liang, Z\. Dai, S\. Ma, F\. Yu, Y\. Xiao, and Z\. LiAre large language models reliable reviewers? a benchmark for error detection in financial documents\.Note:arXiv:2608\.12342Cited by:[§2](https://arxiv.org/html/2610.27488#S2.p2.1),[§7](https://arxiv.org/html/2610.27488#S7.p1.1)\.
- Jainet al\.\(2025\)S\. Jain, U\. Z\. Ahmed, S\. Sahai, and B\. LeongBeyond consensus: mitigating the agreeableness bias in LLM judge evaluations\.Note:arXiv:2510\.11822Cited by:[§2](https://arxiv.org/html/2610.27488#S2.p4.1)\.
- Kimet al\.\(2025\)Y\. Kim, K\. Gu, C\. Park, C\. Park, S\. Schmidgall, A\. A\. Heydari, Y\. Yan, Z\. Zhang, Y\. Zhuang, Y\. Liu, M\. Malhotra, P\. P\. Liang, H\. W\. Park, Y\. Yang, X\. Xu, Y\. Du, S\. Patel, T\. Althoff, D\. McDuff, and X\. LiuTowards a science of scaling agent systems\.Note:arXiv:2512\.08296Cited by:[§2](https://arxiv.org/html/2610.27488#S2.p1.1),[§3\.3](https://arxiv.org/html/2610.27488#S3.SS3.p1.1)\.
- Leeet al\.\(2025\)Y\. Lee, J\. Kim, J\. Kim, H\. Cho, J\. Kang, P\. Kang, and N\. KimCheckEval: a reliable LLM\-as\-a\-judge framework for evaluating text generation using checklists\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 15771–15798\.Cited by:[§2](https://arxiv.org/html/2610.27488#S2.p3.1)\.
- Liet al\.\(2024\)J\. Li, Q\. Zhang, Y\. Yu, Q\. Fu, and D\. YeMore agents is all you need\.Transactions on Machine Learning Research\.Note:arXiv:2402\.05120Cited by:[§2](https://arxiv.org/html/2610.27488#S2.p4.1)\.
- Liet al\.\(2026\)Z\. Li, Z\. Guo, X\. Lu, J\. Wang, J\. Song, C\. Shen, J\. Wu, and M\. SunOptArgus: a multi\-agent system to detect hallucinations in LLM\-based optimization modeling\.Note:arXiv:2605\.11738Cited by:[§2](https://arxiv.org/html/2610.27488#S2.p2.1)\.
- Liuet al\.\(2026\)W\. Liu, R\. Zhao, W\. Yuan, J\. Xu, Z\. Xu, X\. Zhang, and E\. C\. NgaiDude: a dual\-detection multi\-agent system for paper\-code discrepancy detection\.Note:arXiv:2609\.03416To appear in Proceedings of the 2026 Conference on Empirical Methods in Natural Language ProcessingCited by:[§2](https://arxiv.org/html/2610.27488#S2.p2.1),[§7](https://arxiv.org/html/2610.27488#S7.p1.1)\.
- Penget al\.\(2026\)Y\. Peng, Y\. Qi, H\. Xia, G\. He, X\. Shi, R\. Xuan, S\. Lu, Y\. Liu, Z\. Hu, Y\. Liu, and H\. PengCan LLM\-as\-a\-judge reliably verify rubrics in agentic scenarios?\.Note:arXiv:2606\.29920To appear in Proceedings of the 2026 Conference on Empirical Methods in Natural Language ProcessingCited by:[§2](https://arxiv.org/html/2610.27488#S2.p3.1)\.
- Rao and Callison\-Burch \(2026\)D\. Rao and C\. Callison\-BurchJEV vs\. LLMs as rubric judges: cheaper, faster, and wrong in the same places\.Note:arXiv:2609\.29769Cited by:[§2](https://arxiv.org/html/2610.27488#S2.p3.1),[§7](https://arxiv.org/html/2610.27488#S7.p1.1)\.
- Schluntz and Zhang \(2024\)E\. Schluntz and B\. ZhangBuilding effective agents\.Note:Anthropic Engineering blog, 19 December 2024https://www\.anthropic\.com/engineering/building\-effective\-agents\. Accessed 9 October 2026Cited by:[§1](https://arxiv.org/html/2610.27488#S1.p1.1),[§3\.3](https://arxiv.org/html/2610.27488#S3.SS3.p1.1)\.
- Stanislaw and Todorov \(1999\)H\. Stanislaw and N\. TodorovCalculation of signal detection theory measures\.Behavior Research Methods, Instruments, & Computers31\(1\),pp\. 137–149\.External Links:[Document](https://dx.doi.org/10.3758/BF03207704)Cited by:[Appendix D](https://arxiv.org/html/2610.27488#A4.p1.1)\.
- Sunkavalli \(2026\)V\. K\. SunkavalliLLM judges as raters: a pre\-registered audit of severity, halo, reliability, and version instability in LLM essay scoring on public corpora\.Note:arXiv:2608\.29517Cited by:[§2](https://arxiv.org/html/2610.27488#S2.p3.1)\.
- Teoet al\.\(2026\)R\. S\. Y\. Teo, Y\. Yamada, S\. Kotyan, Y\. Imajuku, and T\. ClanuwatBeyond imitation: a framework and benchmark for LLM\-assisted peer review\.Transactions on Machine Learning Research\.Note:arXiv:2610\.11087Cited by:[§2](https://arxiv.org/html/2610.27488#S2.p4.1)\.
- Tran and Kiela \(2026\)D\. Tran and D\. KielaSingle\-agent LLMs outperform multi\-agent systems on multi\-hop reasoning under equal thinking token budgets\.Note:arXiv:2604\.02460Cited by:[§2](https://arxiv.org/html/2610.27488#S2.p1.1)\.
- Wanget al\.\(2023\)X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. H\. Chi, S\. Narang, A\. Chowdhery, and D\. ZhouSelf\-consistency improves chain of thought reasoning in language models\.InInternational Conference on Learning Representations,Note:arXiv:2203\.11171Cited by:[§2](https://arxiv.org/html/2610.27488#S2.p4.1)\.
- Xionget al\.\(2025\)T\. Xiong, Y\. Ge, M\. Li, Z\. Zhang, P\. Kulkarni, K\. Wang, Q\. He, Z\. Zhu, C\. Liu, R\. Chen, T\. Zheng, Y\. Chen, X\. Wang, R\. Zhang, W\. Chen, and H\. HuangMulti\-Crit: benchmarking multimodal judges on pluralistic criteria\-following\.Note:arXiv:2511\.21662To appear in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2026Cited by:[§2](https://arxiv.org/html/2610.27488#S2.p3.1)\.
- Yeh and Lien \(2009\)I\. Yeh and C\. LienThe comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients\.Expert Systems with Applications36\(2\),pp\. 2473–2480\.External Links:[Document](https://dx.doi.org/10.1016/j.eswa.2007.12.020)Cited by:[§3\.2](https://arxiv.org/html/2610.27488#S3.SS2.p1.1)\.
- Yeh \(2009\)I\. YehDefault of credit card clients \[dataset\]\.Note:UCI Machine Learning RepositoryExternal Links:[Document](https://dx.doi.org/10.24432/C55S3H)Cited by:[§3\.2](https://arxiv.org/html/2610.27488#S3.SS2.p1.1)\.
- Yuet al\.\(2026\)W\. Yu, L\. Zhou, X\. Dong, S\. S\. Barath, D\. Staunton, B\. Yoon, X\. Qian, J\. Caverlee, and S\. JiWhen do multi\-agent systems help? an information bottleneck perspective\.Note:arXiv:2607\.16133Cited by:[§2](https://arxiv.org/html/2610.27488#S2.p3.1)\.
- Zhanget al\.\(2025\)H\. Zhang, Z\. Cui, J\. Chen, X\. Wang, Q\. Zhang, Z\. Wang, D\. Wu, and S\. HuStop overvaluing multi\-agent debate—we must rethink evaluation and embrace model heterogeneity\.Note:arXiv:2502\.08788Cited by:[§2](https://arxiv.org/html/2610.27488#S2.p1.1)\.

## Appendix AMemos, labels and settings

Memos\.The record supplies the credit limit, age, education, marital status and six months of repayment status, bills and payments from April to September 2005\. The dataset has no income or employment fields\. The program draws monthly income, allowances, tenure and sector experience for each record from fixed ranges tied to the credit limit and age, and writes them into sentences from templates\. Every figure a requirement tests is stated only by sentences the program controls\. A defect changes one sentence, and no other requirement governs that sentence\. An agent may still judge the sentence against a requirement of its own\. The label of each requirement is drawn independently on each memo\.

Labels\.In a first pass the third source, Claude Opus 5\.5, differed from the other two on five cells\. In four it found a figure wrong and doubted whether the sentence fell under the requirement or what the figure referred to, and in the fifth it found no error\. The wording of two requirements was made explicit, the memos were regenerated with one sentence changed, and the two affected requirements were read again on all 300 memos, with agreement on every cell\. No memo was dropped\. Opus 5\.5 belongs to the family of two of the tested models and is not one of them\.

Models and settings\.The Qwen3\.5 models ran under Ollama 0\.35\.1 with 4\-bit quantisation, Q4\_K\_M, a context of 16,384 tokens and a fixed seed, and the three draws used three seeds\. The Claude models ran through the Batch API with effort set to high\. Runs took place between 8 and 10 October 2026\. The scoped team was run one day after the other arms\. The first requirement in the single agent’s list is rotated by memo index across the three pairs, and each draw advances the rotation by one\.

Scoring rules\.A missing verdict counts as a miss on a defect and as a flag on a clean cell\. A defect counts as caught when the verdict is FAIL, whatever sentence the agent quotes\. Intervals are percentile bootstrap intervals from 5,000 resamples of memos, with arms paired by memo\.

The decide\-once instruction\.The sentences below are added to the prompt of every arm\.

> Take each requirement once\. In at most three sentences say what you compared and what it shows, then give the verdict\. Do not go back over a requirement once you have decided it\.

The scoped prompt\.A scoped call lists the requirements under two headings and ends the second list with a note\. Text in angle brackets stands for the wording of the requirements\.

> REQUIREMENTS YOU JUDGE <the two requirements of this call\> REQUIREMENTS JUDGED BY OTHER REVIEWERS <the other four requirements\> Do not give a verdict on the requirements in this second list\. Other reviewers judge them\. They are listed so that you know what they cover\.

## Appendix BPredictions and outcomes

The seven predictions follow, in three groups, each with its outcome\.

Primary, tested at the 98\.75 percent level\.

Prediction\.The team catches more masked defects than the single agent\.

Outcome\.Holds on Qwen3\.5 27B and Haiku 5\.5\. Not shown on Qwen3\.5 9B\. No room on Sonnet 5\.5\.

Prediction\.The team raises more false alarms than the single agent\.

Outcome\.Holds on Qwen3\.5 9B, Qwen3\.5 27B and Sonnet 5\.5\. Not shown at the corrected level on Haiku 5\.5\.

Secondary, tested at the 95 percent level\.

Prediction\.The team and the single agent do not differ on bare defects\.

Outcome\.Wrong\. The team catches more on three models, by 0\.132 \[0\.062, 0\.202\], 0\.029 \[0\.009, 0\.055\] and 0\.054 \[0\.025, 0\.086\]\.

Prediction\.The team catches more masked defects than a majority of three draws\.

Outcome\.Holds on Haiku 5\.5, 0\.087 \[0\.041, 0\.133\]\. Confounded with temperature on Qwen3\.5 9B, 0\.240 \[0\.153, 0\.328\]\. Not shown on Qwen3\.5 27B, 0\.013 \[−\-0\.048, 0\.076\], or Sonnet 5\.5\.

Prediction\.The team’s excess of false alarms is smaller on the larger model of a family\.

Outcome\.Holds for Qwen3\.5 as a difference, 0\.064 \[0\.030, 0\.099\]\. As a proportion of each model’s rate the excess is about the same\. Not shown for Claude, where the difference is−\-0\.002 \[−\-0\.015, 0\.011\]\.

Pre\-specified, added arm, tested at the 98\.75 percent level\.The level for eight tests per family is 99\.375 percent\.

Prediction\.The scoped team raises fewer false alarms than the team\.

Outcome\.Holds on Sonnet 5\.5, also at the eight\-test level\. At 95 percent only on Haiku 5\.5\. Not shown on the Qwen3\.5 models\.

Prediction\.The scoped team catches more masked defects than the single agent\.

Outcome\.Holds on Haiku 5\.5, also at the eight\-test level\. Lower bound 0\.000 on Qwen3\.5 27B\. Not shown on Qwen3\.5 9B\. No room on Sonnet 5\.5\.

Table A1:The contrasts of Figure[2](https://arxiv.org/html/2610.27488#S4.F2), computed from unrounded rates\.Δ​Hmasked\\Delta H^\{\\mathrm\{masked\}\}andΔ​F\\Delta Fare the two primary contrasts, with intervals at the corrected level\.Δ​B\\Delta Bis exploratory, with 95 percent intervals\.Table A2:The contrasts of Figure[3](https://arxiv.org/html/2610.27488#S4.F3)for the scoped team, computed from unrounded rates\. The first two are its pre\-specified contrasts, with intervals at the corrected level\. The last is exploratory, with 95 percent intervals\.
## Appendix CThe decide\-once instruction

Table[A3](https://arxiv.org/html/2610.27488#A3.T3)gives the change in the single agent’s rates when the instruction is removed\. Without the instruction the Qwen3\.5 27B team lost 17 of 300 calls to the output cap, its single agent 2 of 100 and its scoped team 7 of 300\. The Qwen3\.5 9B team lost 9 of 300\. No Claude call was lost\. On completed verdicts alone and without the instruction,Δ​H\\Delta His \+0\.008 andΔ​F\\Delta Fis \+0\.079 on Qwen3\.5 27B, and they are \+0\.094 and \+0\.095 on Qwen3\.5 9B\.

Table[A4](https://arxiv.org/html/2610.27488#A3.T4)bears on whether part of the team’s gain under the instruction is a recovery of detection that the instruction takes from the single agent\. It sets the single agent without the instruction against the team with it\. No difference in defects caught is significant, and the single agent without the instruction raises fewer false alarms than the team with it on Qwen3\.5 9B and more on Qwen3\.5 27B\. One hundred memos give wide intervals\.

In the preliminary run that led to the instruction, Qwen3\.5 27B kept revising its reasoning until it reached the output cap\. The alternative to the instruction was to score completed verdicts only, which would have set aside the rule, fixed in advance, that a missing verdict counts against the arm\.

Table A3:Change in the single agent’s rates when the instruction is removed, the rate without it minus the rate with it, on the 100 paired memos\. For the Claude models the single call and the three draws are pooled, and for the Qwen3\.5 models the three draws at temperature 0\.7\. The last two rows give the single call of the Qwen3\.5 models at temperature 0\. 95 percent intervals\.Table A4:The single agent without the instruction minus the team with it, one call of each arm on the 100 paired memos, for the three models below ceiling, with 95 percent intervals\.
## Appendix DSignal detection analysis

From the pooled rates we compute the two measures of equal\-variance signal detection theory\([Stanislaw and Todorov 1999](https://arxiv.org/html/2610.27488#bib.bib20)\), whose assumption of equal variance a single operating point cannot test\. Withzzthe inverse of the standard normal distribution function, discrimination isd′=z⁡\(H\)−z⁡\(F\)d^\{\\prime\}=z\(H\)\-z\(F\)and the threshold isc=−\(z\(H\)\+z\(F\)\)/2c=\-\(z\(H\)\+z\(F\)\)/2\. A lowerccmeans a reviewer more ready to flag\. The measures are not computed for Sonnet 5\.5, whose single agent flags no clean cell\. Table[A5](https://arxiv.org/html/2610.27488#A4.T5)gives the changes\. Against the single agent, the team’sd′d^\{\\prime\}does not change significantly on any of the three models and itsccfalls on all three\. The pooled rates are therefore consistent with a lower threshold and unchanged discrimination, and they do not establish that account\. The intervals ond′d^\{\\prime\}are wide\. On Haiku 5\.5 the upper bound corresponds to roughly the whole of the team’s gain in defects caught, so a real improvement of that size is not excluded\. No change ind′d^\{\\prime\}is significant within any defect class either, and the estimates rise in some classes and fall in others\. On Qwen3\.5 27B and Haiku 5\.5 the scoped team’s threshold rises back toward the single agent’s\.

Table A5:Changes ind′d^\{\\prime\}and inccunder the equal\-variance model, with 95 percent intervals, on the three models below ceiling\. By class, a common false\-alarm rate is used\. A class caught in full by either arm is not computed\.
## Appendix EFurther tables

Tables A6 to A10 give the outcomes for whole memos, the rates by requirement, the repeated calls of the single agent’s prompt, the misfiled flags and the share of flags that would be true\.

Table A6:Whole memos, decide\-once instruction\. Of the 21 memos with no defect, the number with at least one flag\. Of the 279 memos with a defect, the number with every defect caught\. The majority rule is omitted\. Twenty\-one memos with no defect are too few to estimate a rate for such files\.
Table A7:Share of defects caught,HH, and false alarms,FF, by requirement, decide\-once instruction, 300 memos\. Each entry gives single agent, team and scoped team in that order\. The numbers of defective and clean cells are 97 and 203 for R1, 130 and 170 for R2, 117 and 183 for R3, 113 and 187 for R4, 89 and 211 for R5, and 118 and 182 for R6\. The Qwen3\.5 9B model flags requirement R4 on more than half of its clean cells in every arm\. It treats a multiple quoted to one decimal place as wrong because it is rounded\.
Table A8:Repeated calls of the single agent’s prompt, decide\-once instruction, 300 memos\. For the Claude models the four repeats are the single call and the three draws, in that order\. For the Qwen3\.5 models the three draws at temperature 0\.7 are shown, with the single call at temperature 0 beside them, marked @Temp=0\. On Haiku 5\.5 the team’s masked rate and its false\-alarm rate both lie outside the range of the four repeats\. No repeat of the team or the scoped team exists\.
Table A9:Flags on clean cells and, of those, misfiled flags, whose quoted sentence is the defective sentence of a defect under another requirement in the same memo\. Decide\-once instruction, 1,136 clean cells\. The last column isΔ​F\\Delta Fwith misfiled flags set aside, with a 95 percent interval\. Sentences are matched by text after removing case and punctuation\. A looser definition that also counts the correct statement of a fact whose other statement is wrong gives the same counts on Qwen3\.5 27B and the two Claude models, and 24, 40 and 43 on Qwen3\.5 9B\.
Table A10:Share of flags that would be true at three values of the defect sharepp, from the unrounded pooled rates behind Table[2](https://arxiv.org/html/2610.27488#S4.T2)and assuming they carry over\. Each entry givesppof 0\.37, 0\.05 and 0\.02, from top to bottom\.