The Replay Gap: Static Evaluation of Model Switching in LLM Agents Scores the Wrong World

# Static Evaluation of Model Switching in LLM Agents Scores the Wrong World
Source: [https://arxiv.org/html/2608.08239](https://arxiv.org/html/2608.08239)
###### Abstract

LLM routers promise efficiency by matching each request to the cheapest adequate model, and are increasingly applied*per step*inside multi\-step agents\. Yet agentic routers are evaluated like single\-turn routers: by replaying logged trajectories and substituting another model’s recorded outputs, assuming the rest of the trajectory is unaffected\. We test this assumption with*branching rollouts*: we fork live SWE\-bench agent trajectories at controlled points, rebuild the environment, continue each fork with a different model, and compare against same\-model control forks that isolate sampling and replay noise\. Across six paired runs \(∼\\sim900 rollouts\), swaps exceed their matched control floors by\+0\.25\+0\.25to\+0\.66\+0\.66normalized edit distance \(multiplicity\-corrected CIs exclude zero\), rewriting 61–94% of post\-fork actions; 74–77% of early swaps diverge at the*first*post\-fork action, versus 6–35% of controls — leaving only 3% of replayed states valid\. Divergence decreases with fork depth in both directions\. All five outcome flips we observe occur in swap arms — upgrades rescuing unsolved instances, a downgrade losing the sole solve — and zero occur across 359 control forks\. Scoring these same swaps with a log\-stitching replay evaluator, replay mispredicts*every*success\-relevant outcome call and predicts patches with 0\.00–0\.11 similarity to reality\. Auditing the noise floor, temperature\-0 “determinism” is configuration\-dependent: FP8\-served controls diverge on over 90% of forks while AWQ\-served ones remain near\-identical; and under tight budgets the*stronger*model more often exhausts its steps without submitting\. Replay\-based benchmarks score the wrong world for agentic routing; we release our harness and all trajectories\.

## 1Introduction

Model routing has moved from research prototype to infrastructure\. Routers that select among heterogeneous LLMs per query power commercial products and open frameworks\(Ong et al\.,[2024](https://arxiv.org/html/2608.08239#bib.bib16); Chen et al\.,[2023](https://arxiv.org/html/2608.08239#bib.bib3); Hu et al\.,[2024](https://arxiv.org/html/2608.08239#bib.bib5)\), and a rapidly growing line of work pushes the routing decision*inside*multi\-step agents: selecting a model per step of a software\-engineering, tool\-use, or computer\-use trajectory\(Liu et al\.,[2026](https://arxiv.org/html/2608.08239#bib.bib13); Agarwal et al\.,[2026](https://arxiv.org/html/2608.08239#bib.bib1); Wei et al\.,[2026](https://arxiv.org/html/2608.08239#bib.bib20)\), or jointly selecting models and reasoning strategies under budgets\(Pan et al\.,[2025](https://arxiv.org/html/2608.08239#bib.bib17)\)\.

Nearly all of this work is evaluated on*logged*model outputs\. A benchmark collects each candidate model’s answer to each query once; a router is then scored by looking up the answers it would have chosen\(Hu et al\.,[2024](https://arxiv.org/html/2608.08239#bib.bib5); Huang et al\.,[2025](https://arxiv.org/html/2608.08239#bib.bib6); Lu et al\.,[2025](https://arxiv.org/html/2608.08239#bib.bib14)\)\. For single\-turn queries this is sound: the query does not depend on the router’s choice\. For agents it is not\. An agent’s action at stepkkdetermines the observation at stepk\+1k\{\+\}1; the trajectory is a closed loop through the environment\. Replay evaluation silently assumes an open loop — that if a router had substituted modelBBat stepkk, the remainder of modelAA’s logged trajectory would still have happened\.

How wrong is this assumption? That trajectories diverge*at all*is not surprising — an agent is a closed loop, and divergence is the null hypothesis\. What routing research needs, and what no one has measured, is the*magnitude and structure*of the divergence: how fast, how far above the noise floor of the serving stack itself, how it depends on swap direction and position, and whether it reaches outcomes\. Measuring this requires*branching live rollouts*— actually forking the trajectory at the decision point and rolling each candidate forward in its own environment — which is far more expensive than replay\. This paper pays that cost and reports what replay hides\.

#### Contributions\.

1. 1\.Methodology\.A branching\-rollout protocol \(and open harness\) for counterfactual evaluation of per\-step model switching: fork a live trajectory at stepkk, re\-execute the action prefix in a fresh container \(with logged replay fidelity\), seed the message history, and continue with a different model — always paired with a*same\-model control fork*that isolates sampling and environment\-replay noise \(§[3](https://arxiv.org/html/2608.08239#S3)\)\.
2. 2\.Quantifying the replay gap\.On SWE\-bench Verified\(Jimenez et al\.,[2024](https://arxiv.org/html/2608.08239#bib.bib8)\), across two swap directions, two fork positions, and three difficulty/prompt tiers \(∼\\sim900 rollouts\): swaps rewrite 61–94% of post\-fork actions, exceeding matched controls by\+0\.25\+0\.25to\+0\.66\+0\.66paired edit distance \(95% CIs exclude zero; Table[1](https://arxiv.org/html/2608.08239#S4.T1), Appendix[A](https://arxiv.org/html/2608.08239#A1)\); 74–77% of early swaps diverge at the first post\-fork action, leaving 3–8% of replayed states valid \(§[4](https://arxiv.org/html/2608.08239#S4)\)\. All five observed outcome flips occur in swap arms; zero in 359 control forks\. Scoring the same fixed switch policies with a log\-stitching replay evaluator — exactly the seed\-matched logged data such an evaluator would use — replay mispredicts every success\-relevant outcome and its predicted patches are near\-orthogonal to the patches actually produced \(§[4\.5](https://arxiv.org/html/2608.08239#S4.SS5)\)\.
3. 3\.Structure with routing implications\.Divergence decreases with fork depth in both directions; upgrades diverge immediately while late downgrades approach control behavior; and we observe the stronger model exhausting tight step budgets without submitting more often than the weak one — a possible*thoroughness tax*that budget\-aware routers may need to price \(§[4\.3](https://arxiv.org/html/2608.08239#S4.SS3)\)\.
4. 4\.An audit of “deterministic” evaluation\.At temperature 0, control forks of our AWQ\-served 14B remain near\-identical \(edit distance 0\.16–0\.23; half never diverge\), while our FP8\-served 4B’s controls diverge on 90–96% of forks \(edit distance 0\.49–0\.67\) under identical settings — so even same\-model replay is unsound on some serving stacks\. We also document how naive patch\-level metrics are inflated by empty\-vs\-empty comparisons, and report subset\-qualified numbers \(§[4\.4](https://arxiv.org/html/2608.08239#S4.SS4)\)\.

## 2Related Work

#### Routing and its benchmarks\.

Cost\-quality routing spans cascades\(Chen et al\.,[2023](https://arxiv.org/html/2608.08239#bib.bib3); Aggarwal et al\.,[2024](https://arxiv.org/html/2608.08239#bib.bib2)\), learned per\-query routers\(Ong et al\.,[2024](https://arxiv.org/html/2608.08239#bib.bib16); Ding et al\.,[2024](https://arxiv.org/html/2608.08239#bib.bib4)\), RL\-trained routers\(Zhang et al\.,[2025](https://arxiv.org/html/2608.08239#bib.bib23)\), and routing to unseen models\(Jitkrittum et al\.,[2025](https://arxiv.org/html/2608.08239#bib.bib9)\)\. Evaluation infrastructure — RouterBench\(Hu et al\.,[2024](https://arxiv.org/html/2608.08239#bib.bib5)\), RouterEval\(Huang et al\.,[2025](https://arxiv.org/html/2608.08239#bib.bib6)\), RouterArena\(Lu et al\.,[2025](https://arxiv.org/html/2608.08239#bib.bib14)\), RouteJudge\(Lai et al\.,[2026](https://arxiv.org/html/2608.08239#bib.bib11)\)— scores routers against precollected outcomes\. Our results do not question these designs for single\-turn routing; we show they do not transfer to agents\.

#### Agentic and per\-step routing\.

Session\- and step\-level routing has been deployed in serving stacks\(Liu et al\.,[2026](https://arxiv.org/html/2608.08239#bib.bib13)\)and studied for tool\-calling and computer\-use agents\(Agarwal et al\.,[2026](https://arxiv.org/html/2608.08239#bib.bib1); Wei et al\.,[2026](https://arxiv.org/html/2608.08239#bib.bib20)\); TwinRouterBench\(Yang et al\.,[2026](https://arxiv.org/html/2608.08239#bib.bib22)\)benchmarks agentic routers with static and “live dynamic” evaluation, where*dynamic*refers to changing model availability and prices — the counterfactual\-trajectory problem we study is explicitly left open\. Joint selection of models and reasoning budgets\(Pan et al\.,[2025](https://arxiv.org/html/2608.08239#bib.bib17); Ma et al\.,[2026](https://arxiv.org/html/2608.08239#bib.bib15)\)inherits the same evaluation gap once applied per step\.

#### Off\-policy evaluation\.

Reinforcement learning formalizes exactly the failure mode we measure: a*behavior*policy generates the logged trajectories, a*target*policy \(here, the router’s switch\) induces a different state distribution, and naive replay is the degenerate estimator that ignores the shift entirely\. The standard corrections carry directly: per\-decision importance sampling\(Precup et al\.,[2000](https://arxiv.org/html/2608.08239#bib.bib18)\)reweights logged returns by the target/behavior action\-probability ratio; weighted and doubly robust variants\(Jiang & Li,[2016](https://arxiv.org/html/2608.08239#bib.bib7); Thomas & Brunskill,[2016](https://arxiv.org/html/2608.08239#bib.bib19)\)trade bias for variance by combining reweighting with a learned value model\. Applying them to agentic routing raises two obstacles our data speaks to\. First, horizon: over 50\-step trajectories the product of per\-step ratios collapses the effective sample size, precisely the regime where importance sampling is known to degenerate — and our measured divergence \(first divergence at post\-fork action<1\{<\}1for early upgrades\) says the ratios are far from 1 immediately\. Second, support: an LLM policy’s action space is unbounded text, so the behavior policy assigns vanishing probability to most target actions, violating the coverage assumption these estimators need\. This makes branched ground truth valuable beyond measurement — it is the only way to*validate*whichever estimator one adopts\. We report the gap here and leave estimator design to future work\.

#### Nondeterminism in LLM serving\.

Batching\- and kernel\-induced nondeterminism at temperature 0 has been reported informally in the serving community; our same\-model control forks give it an experimental floor in the agentic setting, and show it differs sharply across quantization stacks \(§[4\.4](https://arxiv.org/html/2608.08239#S4.SS4)\)\.

## 3Method

#### Setup\.

We use the mini\-SWE\-agent scaffold\(Lieret et al\.,[2025](https://arxiv.org/html/2608.08239#bib.bib12)\)— a minimal bash\-only ReAct loop — on SWE\-bench Verified\(Jimenez et al\.,[2024](https://arxiv.org/html/2608.08239#bib.bib8)\)with the official per\-instance Docker images, a 50\-step budget, and a 28k\-token context\. Our pool contains a small modelSS\(Qwen3\-4B\-Instruct, FP8\) and a large modelLL\(Qwen3\-14B, AWQ, thinking disabled\)\(Yang et al\.,[2025](https://arxiv.org/html/2608.08239#bib.bib21)\), served by vLLM\(Kwon et al\.,[2023](https://arxiv.org/html/2608.08239#bib.bib10)\)on a single 24 GB GPU at temperature 0\. This deliberately mirrors the resource\-constrained serving regime where routing matters most\.

#### Branching protocol\.

For each instance we run a*base*trajectory with the base model to termination\. We then select fork steps at 30% and 70% of the base trajectory’s length\. For each fork: \(i\) start a fresh container; \(ii\) re\-execute the recorded actions of all pre\-fork steps, logging return\-code agreement with the recorded observations \(*replay fidelity*: across 11,702 replayed actions in 708 branches, 99\.99% of return codes match and 707/708 branches reconstruct exactly — Appendix[B](https://arxiv.org/html/2608.08239#A2), so environment reconstruction error is not a plausible source of the divergence we report\); \(iii\) seed the agent’s message history with the recorded prefix, so the branch model sees exactly what the base model saw; \(iv\) continue the rollout with the branch model to termination\. Each fork point receives two arms: aswap arm\(the other model\) and asame\-model control arm, which absorbs sampler nondeterminism, batching effects, and environment\-replay drift\. Divergence attributable to the model swap is read*relative to the control*\.

#### Runs\.

A*run pair*is one 30\-instance sweep in one swap direction: forward \(baseSS, swap up toLL\) or reverse \(baseLL, swap down toSS\)\. We execute six run pairs — \{full difficulty, easy bucket \(“<<15 min fix”\), easy with a budget\-nudged prompt\}×\\times\{forward, reverse\}, seed\-matched — yielding 717 scored branch pairs from∼\\sim900 rollouts\.

#### Metrics\.

Post\-fork action edit distance \(Levenshtein over exact command strings, normalized by the*longer*of the two post\-fork suffixes; since suffix lengths differ systematically by arm, we complement it with two length\-insensitive statistics: the first divergent action index and the fraction of branches diverging at all\);*replay validity*, the prefix\-match fraction \(share of the base’s post\-fork actions before first divergence — the states a replay evaluator would score correctly\); patch metrics — file\-set Jaccard,*patch similarity*\(character\-levelSequenceMatcherratio between the raw unified\-diff texts, in\[0,1\]\[0,1\]\), and exact identity — all subset\-qualified \(§[4\.4](https://arxiv.org/html/2608.08239#S4.SS4)\); official SWE\-bench resolution per arm; and exit\-status distributions\.

## 4Results

![Refer to caption](https://arxiv.org/html/2608.08239v1/x1.png)Figure 1:Post\-fork action divergence by direction and fork position, pooled over three run pairs per direction\. Swaps \(orange\) diverge above same\-model controls \(blue\) in every arm; divergence decreases with fork depth; upgrades diverge more than downgrades\. Per\-run\-pair values in Appendix[A](https://arxiv.org/html/2608.08239#A1)\.Table 1:Pooled post\-fork divergence \(three run pairs per direction\)\. “1st div\.” is the mean index of the first divergent action among diverged branches;*replay validity*is the fraction of post\-fork states a replay evaluator would score against the correct world\. Paired swap−\-control deltas: up@early\+0\.267\+0\.267\[0\.190,0\.3490\.190,0\.349\], up@late\+0\.254\+0\.254\[0\.189,0\.3170\.189,0\.317\], down@early\+0\.663\+0\.663\[0\.580,0\.7440\.580,0\.744\], down@late\+0\.453\+0\.453\[0\.374,0\.5340\.374,0\.534\] \(bootstrap 95% CIs; resampling is at instance level — each instance contributes exactly one control−\-swap delta per arm, so branches within an instance are never split across resamples\)\. All four deltas also exclude zero under Bonferroni correction for the four arms \(98\.75% CIs: lower bounds\+0\.172\+0\.172,\+0\.175\+0\.175,\+0\.560\+0\.560,\+0\.347\+0\.347\)\.### 4\.1The replay gap at the action level

Table[1](https://arxiv.org/html/2608.08239#S4.T1)and Figure[1](https://arxiv.org/html/2608.08239#S4.F1)give the core result\. In every direction and position, swap arms diverge above their matched controls, and the paired per\-instance deltas are significant in all four arms \(95% bootstrap CIs exclude zero; table caption\)\. Because the forward direction’s FP8\-served base is itself noisy \(§[4\.4](https://arxiv.org/html/2608.08239#S4.SS4)\), its control floor is high and the ceiling\-bounded delta \(\+0\.27\+0\.27\) understates the effect;the reverse direction, whose AWQ\-served controls are near\-deterministic, is our cleanest evidence: swapping down at the early fork adds\+0\.66\+0\.66edit distance over a0\.230\.23floor\.Early swaps diverge at the*first*post\-fork action in 73\.9% \(up\) and 76\.7% \(down\) of branches — against control action\-0 rates of 35\.2% and 5\.6% respectively, an excess of\+38\.7\+38\.7and\+71\.1\+71\.1points, so the immediate re\-decision is a swap effect, not a noise\-floor artifact\. The replay\-validity column makes the consequence concrete:a replay evaluator of an early swap scores 92–97% of post\-fork decisions against a state that never occurs\(Figure[2](https://arxiv.org/html/2608.08239#S4.F2)\)\. The pattern replicates in each of the six run pairs individually \(Appendix[A](https://arxiv.org/html/2608.08239#A1)\)\.

![Refer to caption](https://arxiv.org/html/2608.08239v1/x2.png)

![Refer to caption](https://arxiv.org/html/2608.08239v1/x3.png)

Figure 2:Left:ECDF of the first divergent post\-fork action \(pooled over arms\): 55% of all swap branches — and 74–77% of early swaps — diverge at action 0\.Right:replay validity by arm — the share of post\-fork states replay evaluation scores correctly\.
### 4\.2Outcome flips are exclusive to swap arms

Absolute resolution rates are low in our constrained regime \(base rates 0–3%; §[6](https://arxiv.org/html/2608.08239#S6)\), so we report outcome flips exactly as observed:five flip events across three instances, all in swap arms, and zero flips across 359 same\-model control branches\.At full difficulty, an early upgrade resolved an instance the small model never solved in any arm \(the late upgrade of the same instance did not\)\. In the nudged easy tier, upgrades resolveddjango\_\_django\-11163at*both*fork positions\. In the easy reverse tier, swapping down lost the one instance the large model had solved, at both fork positions, while its control retained it at both\. Thus two of the three flip instances were fork\-position invariant\. Replay evaluation is structurally unable to observe any of these events, because none of them exist on the logged trajectory\.

### 4\.3Structure: position, direction, and the thoroughness tax

Three regularities matter for routing policy\.\(i\) Position:divergence decreases with fork depth in both directions \(two positions tested\) — late handoffs inherit the base’s completed work through the workspace, making late downgrades far safer than early ones\.\(ii\) Direction:upgrades diverge immediately \(the stronger model re\-decides at once\), downgrades diverge later and less\.\(iii\) Budgets \(an observation, not yet a regularity\):in the full\-difficulty run pair, the*large*model exhausted the 50\-step budget without submitting more often than the small one \(24/30 vs 17/30 base rollouts; two\-proportionz≈2\.0z\\\!\\approx\\\!2\.0, uncorrected, single run pair\) — consistent with it exploring and verifying more per task, though “this budget starves this model on this stack” is an equally available reading\. If it generalizes, routing*up*under tight budgets can reduce completion rate — a thoroughness tax worth validating in competent regimes\. A prompt\-level budget reminder in the easy tier shifted submissions in opposite directions for the two models \(8→\\to11 and 10→\\to6 of 30; counting noise at thisnn\), suggesting budget control itself consumes the context budget\.

### 4\.4Auditing the noise floor and patch metrics

Two audit findings qualify common evaluation practice\.Temperature\-0 determinism is configuration\-dependent:control forks of the AWQ\-quantized 14B remain near\-identical to their base trajectories \(edit distance 0\.16–0\.23; half never diverge at any action\), while the FP8\-quantized 4B’s controls diverge on 90–96% of forks \(edit distance 0\.49–0\.67\) under identical decoding settings — batched FP8 serving is not a deterministic oracle, so even same\-model replay carries irreducible noise on some stacks\.Patch\-identity metrics inflate silently:when neither side submits a non\-empty patch, naive patch identity scores 1\.0; restricted to branch pairs where both sides submitted real patches, identity drops \(e\.g\., 0\.60–0\.67 for late arms and 0\.00 for early arms in the forward runs, on smallnn\)\. We report all patch\-level numbers subset\-qualified and recommend the practice\.

### 4\.5Replay\-stitch predictions vs\. branched ground truth

The preceding sections show replay’s*inputs*are invalid; here we test whether its*predictions*fail, using no new rollouts\. Because our forward and reverse tiers are seed\-matched, for every instance we hold both models’ full logged standalone trajectories*and*the live branched result of each fixed switch policy \(up/down×\\timesearly/late\)\. A logged\-outcome evaluator in the style of single\-turn router benchmarks predicts a switch’s result from the target model’s logged run on the same instance; we compare those predictions to the branched truth \(Table[2](https://arxiv.org/html/2608.08239#S4.T2)\)\. We do not claim any specific published benchmark implements precisely this stitching rule; we test the*assumption they share*— that a model’s logged performance on an instance transfers to the switched\-to position — which any log\-based evaluator of mid\-trajectory switching must make in some form\. Our result bounds what that assumption can deliver in this regime\.

Table 2:A log\-stitching replay evaluator vs\. branched ground truth\. “Outcome agree” is inflated by the majority class \(nearly all rollouts fail\); the decisive calls are the success\-relevant ones\. “Patch sim\.” is the mean similarity between the replay\-predicted patch and the patch the switch actually produced, over pairs where either is non\-empty\.nndiffers slightly from Table[1](https://arxiv.org/html/2608.08239#S4.T1)because a branch is scoreable here only if its instance’s seed\-matched standalone run exists \(one easy\-tier instance has none, removing one branch per down arm\)\.Raw outcome agreement is 97–100% — and vacuous, since predicting universal failure is almost always right in this regime\. The decisive calls tell the real story:replay predicted failure for all three switching successes that actually occurred, and both successes it did predict never materialized — 0\-for\-5 on every prediction where success was at stake\.These five decisive calls are*not*the five flip events of §4\.2: the two downgrade losses are cases replay happens to call correctly \(both models’ logged runs fail, and so does the switch\), while the two false successes are prediction errors invisible at the flip level; the three missed rescues are common to both counts\. At the patch level, replay’s predicted patch has similarity 0\.00–0\.11 to the patch the switch actually produces\. For calibration, even a model predicting*itself*from logs \(its base patch vs\. its own same\-model control branch\) reaches only 0\.31 \(FP8 4B\) to 0\.53 \(AWQ 14B\) — so any log\-based evaluator is capped by serving\-stack nondeterminism before the cross\-model error is even added\. \(These ceilings are computed over the same either\-patch\-non\-empty subset as the rest of the column; the never\-diverged empty\-vs\-empty control pairs that make the AWQ controls near\-identical in §[4\.4](https://arxiv.org/html/2608.08239#S4.SS4)are excluded, which is why 0\.53 coexists with that determinism\.\) For a null reference: the trivial always\-predict\-failure evaluator scores 2/5 on these same calls \(it is right wherever replay hallucinated a success\) — so the logged information made the stitch evaluator*strictly worse than a constant predictor*\. The success count is small \(five decisive events\) and we claim no more than the sign of the result; but the sign is uniform, and it is the direction replay’s structural blindness predicts\.

## 5Implications

For evaluation:agentic router benchmarks need live or branched evaluation; static replay scores 92–97% of post\-fork decisions against invalid states for early swaps \(61–97% across all swap arms\), cannot observe outcome flips at all, and — tested directly on our own policies — mispredicted every success\-relevant outcome \(§[4\.5](https://arxiv.org/html/2608.08239#S4.SS5)\)\. Our harness and branched dataset provide a starting point, and off\-policy estimators over logged trajectories \(importance sampling, doubly robust\) are the natural cheap approximation to validate against it\.For routers\(stated as hypotheses our regime motivates but cannot confirm — §[6](https://arxiv.org/html/2608.08239#S6)\): fork position and direction look like first\-class features — hand off downward late, once the work is done; escalate hard instances early, before the trajectory ossifies; and price a model’s step appetite, not only its per\-step quality\. Validating these in competent regimes is the immediate next step\.

## 6Limitations

This is a deliberately controlled pilot: one scaffold, one benchmark family, one model family \(two quantized members\),n=30n\{=\}30instances per run pair at temperature 0, and a 24 GB serving budget \(28k context\) that keeps absolute resolution rates low \(0–3%\)\. Action\-level divergence and the one\-sided flip evidence do not require competent agents to be valid — replayed states are off\-distribution regardless of task success — but we show the*inputs*to replay evaluation are invalid rather than demonstrating end\-to\-end router mis\-ranking; measuring mis\-ranking in competent regimes, with unquantized pools and a third fork position, is the immediate next step alongside OPE\-corrected estimators\. Quantization differences \(FP8 vs AWQ\) entangle model capability with serving stack — our up/down comparisons are therefore between*deployment configurations*, not pure model scales\. Prefix replay assumes command\-level environment determinism, which we verify rather than assume \(99\.99% return\-code agreement over 11,702 replayed actions; Appendix[B](https://arxiv.org/html/2608.08239#A2)\), and the branch model inherits the base model’s in\-context style through the replayed prefix, which is part of the phenomenon rather than an artifact, but merits study\.

## Reproducibility

The branching harness, all run configurations, and every analysis script behind the numbers in this paper are released at[https://github\.com/AshrithaG/replay\-gap](https://github.com/AshrithaG/replay-gap), and the full∼\\sim900\-rollout branched\-trajectory dataset \(base and fork trajectories, per\-step actions and observations, fork metadata, replay\-fidelity logs, token counts, patches, and SWE\-bench outcomes\) at[https://huggingface\.co/datasets/ashritha0907/replay\-gap\-trajectories](https://huggingface.co/datasets/ashritha0907/replay-gap-trajectories)\.

## References

- Agarwal et al\. \(2026\)Sharad Agarwal, Pooria Namyar, Alec Wolman, Rahul Ambavat, Ankur Gupta, and Qizheng Zhang\.Switchcraft: Ai model router for agentic tool calling\.*arXiv preprint arXiv:2605\.07112*, 2026\.
- Aggarwal et al\. \(2024\)Pranjal Aggarwal, Aman Madaan, et al\.Automix: Automatically mixing language models\.*arXiv preprint arXiv:2310\.12963*, 2024\.
- Chen et al\. \(2023\)Lingjiao Chen, Matei Zaharia, and James Zou\.Frugalgpt: How to use large language models while reducing cost and improving performance\.*arXiv preprint arXiv:2305\.05176*, 2023\.
- Ding et al\. \(2024\)Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Rühle, Laks V\.S\. Lakshmanan, and Ahmed Hassan Awadallah\.Hybrid llm: Cost\-efficient and quality\-aware query routing\.In*International Conference on Learning Representations \(ICLR\)*, 2024\.
- Hu et al\. \(2024\)Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, and Shriyash Kaustubh Upadhyay\.Routerbench: A benchmark for multi\-llm routing systems\.*arXiv preprint arXiv:2403\.12031*, 2024\.
- Huang et al\. \(2025\)Zhongzhan Huang, Guoming Ye, Guowen Liang, Yuanyuan Zeng, Yingjie Chen, Yang Zhong, Xiao Yang, Wei Zheng, and Liang Lin\.Routereval: A comprehensive benchmark for routing llms to explore model\-level scaling up in llms\.*arXiv preprint arXiv:2503\.10657*, 2025\.
- Jiang & Li \(2016\)Nan Jiang and Lihong Li\.Doubly robust off\-policy value evaluation for reinforcement learning\.In*International Conference on Machine Learning \(ICML\)*, 2016\.
- Jimenez et al\. \(2024\)Carlos E\. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan\.Swe\-bench: Can language models resolve real\-world github issues?In*International Conference on Learning Representations \(ICLR\)*, 2024\.
- Jitkrittum et al\. \(2025\)Wittawat Jitkrittum, Harikrishna Narasimhan, Ankit Singh Rawat, Jeevesh Juneja, Congchao Wang, Chen\-Yu Lee, Pradeep Shenoy, Rina Panigrahy, Aditya Krishna Menon, and Sanjiv Kumar\.Universal model routing for efficient llm inference\.*arXiv preprint arXiv:2502\.08773*, 2025\.
- Kwon et al\. \(2023\)Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E\. Gonzalez, Hao Zhang, and Ion Stoica\.Efficient memory management for large language model serving with pagedattention\.In*Proceedings of the 29th Symposium on Operating Systems Principles \(SOSP\)*, 2023\.
- Lai et al\. \(2026\)Guannan Lai, Haoran Hu, and Han\-Jia Ye\.Routejudge: An open platform for reproducible and preference\-aware llm routing\.*arXiv preprint arXiv:2606\.18774*, 2026\.
- Lieret et al\. \(2025\)Kilian Lieret, Carlos E\. Jimenez, et al\.mini\-swe\-agent: The 100\-line ai agent that solves github issues\.[https://github\.com/SWE\-agent/mini\-swe\-agent](https://github.com/SWE-agent/mini-swe-agent), 2025\.
- Liu et al\. \(2026\)Xunzhuo Liu, Bowei He, Huamin Chen, Haichen Zhang, Andy Luo, and the vLLM Semantic Router Team\.Session\-aware agentic routing: Continuity\-aware model selection for long\-horizon llm agents\.[https://vllm\.ai/blog/2026\-06\-02\-session\-aware\-agentic\-routing](https://vllm.ai/blog/2026-06-02-session-aware-agentic-routing), 2026\.
- Lu et al\. \(2025\)Yifan Lu, Rixin Liu, Jiayi Yuan, Xingqi Cui, Shenrun Zhang, Hongyi Liu, and Jiarong Xing\.Routerarena: An open platform for comprehensive comparison of llm routers\.*arXiv preprint arXiv:2510\.00202*, 2025\.
- Ma et al\. \(2026\)Siyuan Ma, Bo Gao, Xiaojun Jia, Simeng Qin, Tianlin Li, Ke Ma, Xiaoshuang Jia, Wenqi Ren, and Yang Liu\.Odar: Principled adaptive routing for llm reasoning via active inference\.*arXiv preprint arXiv:2602\.23681*, 2026\.
- Ong et al\. \(2024\)Isaac Ong, Amjad Almahairi, Vincent Wu, Wei\-Lin Chiang, Tianhao Wu, Joseph E\. Gonzalez, M\. Waleed Kadous, and Ion Stoica\.Routellm: Learning to route llms with preference data\.*arXiv preprint arXiv:2406\.18665*, 2024\.
- Pan et al\. \(2025\)Zhihong Pan, Kai Zhang, Yuze Zhao, and Yupeng Han\.Route to reason: Adaptive routing for llm and reasoning strategy selection\.*arXiv preprint arXiv:2505\.19435*, 2025\.
- Precup et al\. \(2000\)Doina Precup, Richard S\. Sutton, and Satinder Singh\.Eligibility traces for off\-policy policy evaluation\.In*International Conference on Machine Learning \(ICML\)*, 2000\.
- Thomas & Brunskill \(2016\)Philip S\. Thomas and Emma Brunskill\.Data\-efficient off\-policy policy evaluation for reinforcement learning\.In*International Conference on Machine Learning \(ICML\)*, 2016\.
- Wei et al\. \(2026\)Jinbiao Wei, Kangqi Ni, Yilun Zhao, Guo Gan, and Arman Cohan\.Step\-level optimization for efficient computer\-use agents\.*arXiv preprint arXiv:2604\.27151*, 2026\.
- Yang et al\. \(2025\)An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al\.Qwen3 technical report\.*arXiv preprint arXiv:2505\.09388*, 2025\.
- Yang et al\. \(2026\)Pei Yang, Wanyi Chen, Tongyun Yang, Pengbin Feng, Jiarong Xing, Wentao Guo, Yuhang Yao, Yuhang Han, Hanchen Li, Xu Wang, Zeyu Wang, Jie Xiao, Anjie Yang, Liang Tian, Lynn Ai, Eric Yang, and Tianyu Shi\.Twinrouterbench: Fast static and live dynamic evaluation for realistic agentic llm routing\.*arXiv preprint arXiv:2605\.18859*, 2026\.
- Zhang et al\. \(2025\)Haozhen Zhang, Tao Feng, and Jiaxuan You\.Router\-r1: Teaching llms multi\-round routing and aggregation via reinforcement learning\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2025\.

## Appendix APer\-run\-pair divergence

Mean post\-fork normalized action edit distance per run pair\. The swap\-above\-control ordering holds in all 24 cells\.

Table 3:Replication across tiers: the swap arm exceeds its matched control in every cell of every run pair\.
## Appendix BPrefix\-replay fidelity

Every branch re\-executes the base trajectory’s pre\-fork actions in a fresh container; we compare each replayed action’s return code against the one recorded in the base trajectory\. Table[4](https://arxiv.org/html/2608.08239#A2.T4)reports the agreement\. Environment reconstruction is near\-exact — one mismatched action in 11,702, and 707 of 708 branches replayed perfectly — so the post\-fork divergence reported in §[4](https://arxiv.org/html/2608.08239#S4)cannot be attributed to replay error in the prefix\.

Table 4:Prefix\-replay fidelity\. “Branches exact” is the fraction of branches in which every replayed action’s return code matched the recording\.