Lessons from Luck: Do LLM-Written Lessons Carry Chance into an Agent's Next Decision?

# Do LLM-Written Lessons Carry Chanceinto an Agent’s Next Decision?
Source: [https://arxiv.org/html/2610.33029](https://arxiv.org/html/2610.33029)
## Lessons from Luck: Do LLM\-Written Lessons Carry Chance into an Agent’s Next Decision?

###### Abstract

Self\-improving agents write lessons from the outcomes of their runs; when outcomes are noisy, a lesson can carry chance into later decisions\. We measure one link of this chain in a registered study\. In 256 synthetic worlds, each a recurring task with two interchangeable tools of recorded reliability, a scripted previous run used the more reliable tool \(a good decision\) or the less reliable one \(a bad decision\), with an assigned outcome: success or a transient failure\. A model wrote a lesson from the run with ReasoningBank’s memory\-writing instruction; a fresh session of the same model, holding the authoritative log and the previous result, chose the next run’s first tool with one memory item: that lesson, the lesson from the other outcome, an irrelevant item, or an unreasoned directive\. The protocol was frozen, deposited and timestamped before any confirmatory call; 11 LLM endpoints gave 90,084 answers\. After a chance failure of the better tool, the panel’s lessons carried the chance \(registered reading*luck\-carrying*; interaction\+0\.112\+0\.112, 95% interval \[\+0\.097\+0\.097,\+0\.126\+0\.126\]\): readers named the worse tool in 11\.6% of next runs with the lesson written from that failure, against 0\.4% with an irrelevant item and 0\.4% with the lesson written from a success\. The effect was\+0\.042\+0\.042where the log stated that failures are independent and\+0\.181\+0\.181where it did not \(descriptive\)\. After a lucky success of the worse tool, the interaction was*bounded below 0\.10*\(\+0\.002\+0\.002\), a bound, not evidence of no harm\. In exploratory readings, 6 endpoints carried the chance \(up to\+0\.352\+0\.352\); 5 were bounded below 0\.10\. In secondary comparisons, a three\-sentence decision\-quality note in the writing instruction lowered the worse\-tool rate after a chance failure to 7\.5%, and one instruction for both outcomes with a no\-change option to 3\.1%; neither met the registered harm\-reduction rule at every such endpoint\.

## 1Introduction

Agents built on large language models \(LLMs\) increasingly learn from their own runs by writing lessons into memory and reading them before later runs\. Reflexion keeps verbal self\-reflections after failed trials\[[Shinn et al\., 2023](https://arxiv.org/html/2610.33029#bib.bib21)\]; ExpeL and Agent Workflow Memory extract insights or reusable workflows from past trajectories\[[Zhao et al\., 2023](https://arxiv.org/html/2610.33029#bib.bib34),[Wang et al\., 2024](https://arxiv.org/html/2610.33029#bib.bib28)\]; Dynamic Cheatsheet maintains an evolving memory of strategies at test time\[[Suzgun et al\., 2025](https://arxiv.org/html/2610.33029#bib.bib22)\]; and ReasoningBank distils reasoning strategies from both successful and failed trajectories, sending each trajectory to a success or a failure instruction according to its outcome\[[Ouyang et al\., 2025](https://arxiv.org/html/2610.33029#bib.bib17)\]\.

Outcomes, however, are noisy\. A sound decision can fail by chance and an unsound one can succeed by chance\. Judging a decision by its outcome rather than by the information available when it was made is the outcome bias of decision research\[[Baron and Hershey, 1988](https://arxiv.org/html/2610.33029#bib.bib3)\], and knowing an outcome makes it seem more predictable than it was\[[Fischhoff, 1975](https://arxiv.org/html/2610.33029#bib.bib8)\]\. A memory\-writing step that turns one outcome into a durable rule could therefore carry chance forward: a lesson written after a chance failure of a sound decision may tell later runs to abandon it\. No adversary is needed for this error\. Recent audits of agent memory and self\-improvement give reason to look: memory gains vary across repeated runs and task orders\[[Ye et al\., 2026](https://arxiv.org/html/2610.33029#bib.bib32)\], self\-assigned rewards in memory banks inflate\[[Asadolahi et al\., 2026](https://arxiv.org/html/2610.33029#bib.bib2)\], and harness proposers fix failures that never happened when, among other conditions, their instructions presuppose failures\[[Wang et al\., 2026d](https://arxiv.org/html/2610.33029#bib.bib27)\]\. Whether a chance outcome, with decision quality held fixed, is written into a lesson that moves a later decision was not, as far as we found, measured \(Section[2](https://arxiv.org/html/2610.33029#S2)\)\.

We measure one link of this chain\. With the decision, the information available when it was made and the task held fixed, does a lesson written from a chance outcome change the tool a fresh session of the same model would choose next, beyond what the raw outcome alone does? We ask it in a setting favourable to the reader: it holds the authoritative run log and is told the previous run’s result, so it could check the lesson against the evidence\. In 256 synthetic worlds, each a recurring task with two interchangeable tools of recorded reliability, a scripted previous run used the more reliable tool \(a good decision\) or the less reliable one \(a bad decision\), and its outcome was assigned\. A model wrote a lesson from that run with ReasoningBank’s outcome\-routed writing instruction or one of two variants; a fresh session of the same model then named the tool it would call first in the next run, with one memory item: that lesson, the lesson written from the other outcome, an irrelevant item, or an unreasoned directive \(Section[3](https://arxiv.org/html/2610.33029#S3)\)\. The primary contrasts are differences\-in\-differences between readers that hold the same information and differ only in the memory item \(Section[4](https://arxiv.org/html/2610.33029#S4)\)\. The protocol, code and stimuli were frozen, deposited on Zenodo and timestamped in a Bitcoin block before any confirmatory call; the run was launched once on 11 LLM endpoints and analysed once, on the locked records, by the registered analysis\.

The findings \(Section[5](https://arxiv.org/html/2610.33029#S5)\):

- •After a chance failure of the better tool, the panel’s lessons carried the chance \(registered reading*luck\-carrying*;\+0\.112\+0\.112\[\+0\.097\+0\.097,\+0\.126\+0\.126\]\): with the lesson written from that failure in memory, readers named the worse tool in 11\.6% of next runs, against 0\.4% with an irrelevant item and 0\.4% with the lesson written from a success, every reader seeing the same log\. The lesson moved readers in combination with the reported failure: read after a reported success of the better tool, the same lesson led to the worse tool in 0\.2% of next runs\.
- •Most of this effect arose where the record left the failure open to interpretation \(a registered descriptive split\): the interaction was\+0\.181\+0\.181\[\+0\.160\+0\.160,\+0\.203\+0\.203\] in the worlds whose record did not state that failures are independent, and\+0\.042\+0\.042\[\+0\.031\+0\.031,\+0\.053\+0\.053\] in the worlds whose record stated it\. Only under the stated model is a worse\-tool choice a clear error; where independence is not stated, a recent failure can be read as a sign of a current fault, and the choice is a deviation from the log\-based benchmark\. In both, the lesson added no observation beyond the reader’s record\.
- •After a lucky success of the worse tool, the outcome\-by\-memory interaction was*bounded below 0\.10*\(\+0\.002\+0\.002\[−0\.001\-0\.001,\+0\.005\+0\.005\]\); this bounds the interaction and does not show that no harm occurred\.
- •In exploratory endpoint readings, Gemini 3\.8 Flash, Grok 4\.7, Qwen 3\.8 Max, DeepSeek V4 Pro, Mistral Large 4 and Kimi K3 carried the chance, and Claude Opus 5\.5, Claude Sonnet 5\.5, Claude Haiku 5\.5, GPT\-6\.1 Sol and GPT\-6 Luna were bounded below 0\.10; a bound is not a zero: at Claude Haiku 5\.5 and GPT\-6 Luna the estimates were small but positive by theirttintervals\.
- •In secondary comparisons, a three\-sentence decision\-quality note in the writing instruction lowered the panel’s worse\-tool rate after a chance failure to 7\.5%, and one instruction for both outcomes with a no\-change option to 3\.1%; under either, the interaction stayed above zero by itsttinterval, and neither met the registered harm\-reduction rule at every endpoint\.
- •At two endpoints, low response to the unreasoned directive coexisted with luck\-carrying lessons \(exploratory\)\.

The study isolates one stored lesson, read once, in synthetic worlds with a constrained choice\. It does not execute self\-improvement, retrieval or accumulated memory, does not estimate how often real agents meet such situations, tests the remedies only with outcomes that did not signal a change in a tool’s reliability, and claims no mechanism \(Section[7](https://arxiv.org/html/2610.33029#S7)\)\. Appendix[C](https://arxiv.org/html/2610.33029#A3)prints example lessons and the answers they led to\.

## 2Related work

#### Self\-written experience memory\.

Agents that store lessons from past runs and read them later include Reflexion’s verbal reflections on task feedback\[[Shinn et al\., 2023](https://arxiv.org/html/2610.33029#bib.bib21)\], ExpeL’s insights extracted from successful and failed trajectories\[[Zhao et al\., 2023](https://arxiv.org/html/2610.33029#bib.bib34)\], Agent Workflow Memory’s reusable workflows\[[Wang et al\., 2024](https://arxiv.org/html/2610.33029#bib.bib28)\], Dynamic Cheatsheet’s evolving test\-time memory\[[Suzgun et al\., 2025](https://arxiv.org/html/2610.33029#bib.bib22)\], Agentic Context Engineering’s evolving contexts\[[Zhang et al\., 2025](https://arxiv.org/html/2610.33029#bib.bib33)\], and ReasoningBank, whose memory\-writing instructions this study adapts\[[Ouyang et al\., 2025](https://arxiv.org/html/2610.33029#bib.bib17)\]\. Earlier designs reflect over a stream of stored experiences\[[Park et al\., 2023](https://arxiv.org/html/2610.33029#bib.bib18)\]or keep a library of learned skills\[[Wang et al\., 2023](https://arxiv.org/html/2610.33029#bib.bib23)\]\.

#### When self\-written memory misleads\.

Several recent studies document how such loops fail\. Repeated runs and task\-order shuffles of two memory\-based self\-improving methods show high variance and order dependence, and in their full text evaluator errors produce spurious memories\[[Ye et al\., 2026](https://arxiv.org/html/2610.33029#bib.bib32)\]\. Self\-assigned rewards in label\-free memory banks inflate wrong episodes, which agents then reuse\[[Asadolahi et al\., 2026](https://arxiv.org/html/2610.33029#bib.bib2)\]\. Reflections can repeat themselves and reinforce false beliefs\[[Dixit et al\., 2026](https://arxiv.org/html/2610.33029#bib.bib6)\]; an agent that writes its own conclusions into a store it later reads can be captured by the errors it wrote\[[Chen et al\., 2026](https://arxiv.org/html/2610.33029#bib.bib4)\]; agents follow retrieved experiences closely, so errors in memory propagate\[[Xiong et al\., 2025](https://arxiv.org/html/2610.33029#bib.bib31)\]; and harness proposers add guardrails against failures that never happened when, among other conditions, their instruction presupposes failures\[[Wang et al\., 2026d](https://arxiv.org/html/2610.33029#bib.bib27)\]\. In harness evolution, a revision can be promoted on a lucky rollout\[[Wei et al\., 2026](https://arxiv.org/html/2610.33029#bib.bib30)\]; Zhu et al\. argue that single\-agent experience construction lacks a reference distribution to separate stochastic failures from grounding errors\[[Zhu et al\., 2026](https://arxiv.org/html/2610.33029#bib.bib35)\]; and verifier signals attached to memory records have been proposed to govern which trajectories, including accidental successes, are stored and reused\[[Wang et al\., 2026b](https://arxiv.org/html/2610.33029#bib.bib25)\]\. These studies observe or argue that errors, noise and luck can enter self\-written memory\.

#### Outcome, hindsight and attribution\.

People judge decisions by their outcomes\[[Baron and Hershey, 1988](https://arxiv.org/html/2610.33029#bib.bib3)\], and knowing an outcome makes it seem predictable\[[Fischhoff, 1975](https://arxiv.org/html/2610.33029#bib.bib8)\]\. In a replication of a study of hindsight effects on judgements of legal responsibility, LLMs showed effects much larger than people’s\[[Almeida et al\., 2024](https://arxiv.org/html/2610.33029#bib.bib1)\]; LLMs also show hindsight\-sensitive shifts when shown complete clinical timelines\[[Matsuura et al\., 2026](https://arxiv.org/html/2610.33029#bib.bib13)\], asymmetries between attributing failures as actor and as observer\[[Li et al\., 2026](https://arxiv.org/html/2610.33029#bib.bib12)\], and overweighting of advice that contradicts their own earlier answer\[[Kumaran et al\., 2025](https://arxiv.org/html/2610.33029#bib.bib11)\]\. In repeated choice, they weight positive and negative outcomes asymmetrically\[[Schubert et al\., 2024](https://arxiv.org/html/2610.33029#bib.bib19),[Wang et al\., 2026a](https://arxiv.org/html/2610.33029#bib.bib24)\]and react strongly to the most recent outcomes\[[Horowitz and Plonsky, 2025](https://arxiv.org/html/2610.33029#bib.bib10)\]\. These are judgements or choices within one context\. Here a model writes a persistent lesson from a run whose outcome was assigned by chance, and a later session of the same model, which holds the evidence, reads it\.

#### Memory poisoning\.

Adversaries can insert records or instructions into an agent’s memory\[[Chen et al\., 2024](https://arxiv.org/html/2610.33029#bib.bib5),[Dong et al\., 2025](https://arxiv.org/html/2610.33029#bib.bib7)\], or craft locally correct experiences that reflective agents consolidate into over\-generalised rules\[[Wang et al\., 2026c](https://arxiv.org/html/2610.33029#bib.bib26)\]\. The planted item of this study, an unreasoned directive to use the worse tool, is a positive control of that kind; the lessons studied are written by a model without an adversary\.

#### Position\.

Prior work thus identifies misleading experience, including accidentally successful trajectories: evaluator errors and run\-to\-run variance produce spurious memories\[[Ye et al\., 2026](https://arxiv.org/html/2610.33029#bib.bib32)\], locally correct experiences crafted by an adversary are consolidated into over\-generalised rules\[[Wang et al\., 2026c](https://arxiv.org/html/2610.33029#bib.bib26)\], and verifier signals have been proposed to govern which trajectories, including accidental successes, are stored\[[Wang et al\., 2026b](https://arxiv.org/html/2610.33029#bib.bib25)\]\. The contribution here is therefore not the observation that self\-written memory can mislead but a decomposition of one way it does: an outcome assigned at random to a scripted decision of known quality, a lesson written from it by a model, and readers of the same model that hold the same evidence and differ only in the memory item\. A targeted search made before the study was designed \(October 2026; arXiv records and full texts where available; not systematic\) did not identify this design\. This study belongs to a line of work on how agents treat inherited state before acting\[[Nakayashiki, 2026a](https://arxiv.org/html/2610.33029#bib.bib14),[Nakayashiki, 2026b](https://arxiv.org/html/2610.33029#bib.bib15),[Nakayashiki, 2026c](https://arxiv.org/html/2610.33029#bib.bib16)\]\.

## 3Design

#### At a glance\.

Each world supplies four scripted previous runs: a good or a bad decision, crossed with a success or a failure\. For each endpoint \(a model served by one provider\), three writing instructions turn each run into a lesson, twelve lessons per world\. The same endpoint then answers as a reader in the four previous\-run conditions with each of eight memory arms: 32 reader requests per world and endpoint\. The reader names the tool it would call first; nothing is executed\. The primary contrasts \(Section[4](https://arxiv.org/html/2610.33029#S4)\) are named P2 for the good decision and P3m for its mirror, the bad decision\.

### 3\.1Worlds

Each of the 256 worlds is a recurring operational task \(exporting data, sending invoices, rebuilding a search index and the like\) with two interchangeable tools that differ only in their recorded reliability\. There are 16 domains with 16 worlds each; within a domain, the better tool’s listing position \(first or second\) and the cue \(below\) are balanced\. The run log shows the last 180, 240, 300 or 365 runs of each tool, with success rates of 0\.80–0\.93 for the better tool and 0\.50–0\.70 for the worse one \(a gap of at least 0\.15\); the two tools cost the same, and a failed run has to be re\-run by hand\. Tool names are coined, checked against a word list and against the domain’s texts, and the two names of a world start with different letters\. The generator accepts a world only if the better tool’s posterior mean success rate after one more failure exceeds the worse tool’s after one more success by at least 0\.10, so the better tool stays better after any single further outcome\.

The*cue*is a world\-level factor\. In explicit worlds the record adds the sentence “There have been no outages: failures of either tool are independent one\-off events, and the next run has always started normally\.” In implicit worlds the record shows only the counts and does not state whether failures are independent\. Half of the worlds \(128\) are explicit\.

### 3\.2Scripted runs and lesson writers

A previous run is a scripted trajectory in ReasoningBank’s extraction format\[[Ouyang et al\., 2025](https://arxiv.org/html/2610.33029#bib.bib17)\]: the task, then three reasoning\-and\-action steps in which an agent reads the run log, states the record and its choice, calls the tool and reports the returned text and that the run ended\. In a good decision the agent calls the better tool \(“the more reliable tool”\); in a bad decision it calls the worse tool and gives no reason\. The outcome is a success or a transient, non\-diagnostic error drawn from the domain’s pool, with “this run allows no retry”\. Nothing else differs between a world’s four runs\.

Three instructions write a lesson from a run \(Appendix[A](https://arxiv.org/html/2610.33029#A1)\):

- •RB\(deployed\): ReasoningBank’s success instruction after a success and its failure instruction after a failure, as released in the authors’ code \(repository, file and commit in Appendix[A](https://arxiv.org/html/2610.33029#A1)\), verbatim except two declared replacements \(“an expert in web navigation” became “an expert in running operational tasks”, and the rule against naming entities from the task allows naming the tools\)\. The released instructions differ in three places from those printed in the ReasoningBank paper; among them, the released failure instruction asks for “concrete, actionable recovery procedures” \(Appendix[A](https://arxiv.org/html/2610.33029#A1)\)\. The results concern the released instructions\.
- •RBD\(deployed plus a remedy\): RB with one added bullet of three sentences: “Before writing, judge whether the agent’s decision was sound given only what was known before the outcome\. A single outcome is not by itself evidence that the decision was right or wrong\. Recommend a different action only if the decision was unsound given that information\.”
- •N: one instruction used after both outcomes \(RB’s success instruction with its four outcome\-specific phrases made neutral\) with the added sentence “If nothing about how the task should be done needs to change, say so\.” The writer still reads the run, including its outcome; the registration calls this instruction “outcome\-blind” because the instruction does not depend on the outcome\. N differs from RB in two ways \(no routing by outcome and an explicit no\-change option\), so N against RB is a compound comparison\.

The stored memory is the writer’s whole response, trimmed, inserted as one block, as the registration specifies; a response can hold up to three formatted memory items\. ReasoningBank distils an experience into memory items with a title, a description and content and reuses all items of a retrieved experience\[[Ouyang et al\., 2025](https://arxiv.org/html/2610.33029#bib.bib17)\]; the results concern its extraction instructions within this representation\. No sampling parameter is sent \(ReasoningBank’s released code extracts at temperature 1\.0; here every endpoint samples at its provider’s default\)\. Each endpoint writes the lessons that its own readers read, so differences between endpoints cannot be assigned separately to writing or to reading\. Each cell has one generation: one lesson per instruction, decision and outcome, and one answer per cell and arm\.

### 3\.3Reader and memory arms

The reader is a fresh session of the same model, instructed as an operations agent that reads saved memory items before each run\. It sees the task; both tools with one\-line descriptions \(“both operational today”\); the run log \(“authoritative; stable for months”\), with the cue sentence in explicit worlds; the equal costs and the hand re\-run; the previous run \(“toolwas used and it succeeded”, or “failed” with the returned text, followed by “This is a new scheduled run of the task\. The previous run is finished and will not be repeated\.”\); one memory item; and the question “Which tool will you call first for this run?”\. It answers in JSON withfirst\_tool, restricted to the two tool names by structured output, and a reason \(Appendix[B](https://arxiv.org/html/2610.33029#A2)\)\. The memory item is one of eight*arms*:

- •RB, RBD, N\(matched\): the lesson that instruction wrote from this decision’s run with the*same*outcome as the previous\-run line;
- •RBx, RBDx, Nx\(swapped\): the lesson that instruction wrote from this decision’s run with the*other*outcome \(same world, same endpoint\), shown after the previous\-run line’s outcome;
- •NEUTRAL: a true, irrelevant operational fact of the domain, in the same format;
- •PLANTED: “Useworse toolfirst … Callworse toolfirst on every run of this task\.”, with no reason\.

### 3\.4Panel and serving

The panel is 11 endpoints served through OpenRouter, each pinned to one provider without fallbacks \(Appendix[D](https://arxiv.org/html/2610.33029#A4)\): Claude Opus 5\.5, Claude Sonnet 5\.5, Claude Haiku 5\.5, GPT\-6\.1 Sol, GPT\-6 Luna, Gemini 3\.8 Flash, Grok 4\.7, Qwen 3\.8 Max, DeepSeek V4 Pro, Mistral Large 4 and Kimi K3\. They were chosen by operational gates of a development pilot \(completion, refusals, truncation, contract integrity, cost and projected wall\-clock time\), never by an effect; Kimi K3, dropped by the cost gate alone, was restored by the author, who holds the budget\. Requests carry no sampling or reasoning parameters; each writer and reader may produce up to 16,000 output tokens; every response must report the registered model and provider and a billed cost not above the upstream cost, or the run stops\. All calls ran on 9 October 2026 \(UTC\)\.

### 3\.5Registration and execution

#### Development pilot\.

Five development runs on worlds from other seeds tested the instrument and set operational parameters \(registration, §2\), and they shaped the design\. After the first attempt, in which the strongest writer repeated the run’s statement that failures are independent as its reason, that statement became the world\-level cue\. After the descriptive results of the second attempt had been read, the memory\-only reader it used was dropped \(an audit found that it measured listing position rather than the lesson’s direction\), the whole writer response became the stored memory, and the mirror contrast P3m and the planted control were added\. Later revisions \(the swapped arms, the qualifiers, the betting tests and the reading words\) followed two reviews of the protocol by language\-model reviewers rather than further outcome analyses\. No pilot data enter the analysis\.

#### Freeze, deposit and timestamp\.

The package of 74 files \(the registration, code, worlds, prompts, analysis code, the pilot’s records and the design reviews\) was frozen with a SHA\-256 manifest \(0fc1f680ef7d241692e7726dc9dd5a7df45ea45905e6f13bb295255e2fe864be\), deposited publicly on Zenodo \(doi:10\.5281/zenodo\.23253562; verified as public without a token at 2026\-10\-09 04:03:33 UTC\) and its manifest timestamped with OpenTimestamps; the stamp was attested in Bitcoin block 970581 \(2026\-10\-09 04:23:19 UTC\)\. The launcher refused to call any model unless it re\-verified the manifest, the Bitcoin attestation and the public deposit’s files\.

#### Run and analysis\.

A smoke run on 2 worlds from another seed passed and was locked\. The confirmatory run was launched once and ended without a stop, an exclusion or a contract violation: 33,792 writer requests and 90,106 reader requests, for 543\.41 USD \(writers 245\.20, readers 298\.21\)\. The run was locked with a completion manifest over 123,907 files, itself timestamped \(attested in Bitcoin block 970623, 2026\-10\-09 11:33:13 UTC\), and the registered analysis read it once, through that lock\.

#### Deviations and corrections\.

The registration asks that an independent re\-implementation of the analysis be compared with it before any report\. The report’s generated tables were first produced before that comparison; the two results reviews found this, it is recorded as a deviation, and every report file was regenerated after the comparison, which found no disagreement\. The registration’s introduction calls the remedy “a one\-sentence remedy”; its text, given in the registration’s stimuli section and above, is one bullet of three sentences\.

## 4Estimands and analysis

#### Outcome\.

Y=1Y=1if the reader’s first tool is the worse tool and00if it is the better one\. An answer counts if it is valid andfirst\_toolnames one of the two tools; a registered rule recovers an unparsed answer only when thefirst\_toolvalues in its raw text name exactly one of the two tools \(it recovered 0\)\.Y⁡\[o,a\]Y\[o,a\]denotes the answer in the cell whose previous\-run line reports outcomeoo\(fffailure,sssuccess\) with memory armaa, for the decision named\.

#### Primary contrasts

\(instruction RB; per endpoint and as the panel mean\):

P2 \(good decision\)=\(Y⁡\[f,RB\]−Y⁡\[s,RB\]\)−\(Y⁡\[f,NEUTRAL\]−Y⁡\[s,NEUTRAL\]\),\\displaystyle=\\bigl\(Y\[f,\\mathrm\{RB\}\]\-Y\[s,\\mathrm\{RB\}\]\\bigr\)\-\\bigl\(Y\[f,\\mathrm\{NEUTRAL\}\]\-Y\[s,\\mathrm\{NEUTRAL\}\]\\bigr\),P3m \(bad decision\)=\(Y⁡\[s,RB\]−Y⁡\[f,RB\]\)−\(Y⁡\[s,NEUTRAL\]−Y⁡\[f,NEUTRAL\]\)\.\\displaystyle=\\bigl\(Y\[s,\\mathrm\{RB\}\]\-Y\[f,\\mathrm\{RB\}\]\\bigr\)\-\\bigl\(Y\[s,\\mathrm\{NEUTRAL\}\]\-Y\[f,\\mathrm\{NEUTRAL\}\]\\bigr\)\.P2 is the good decision’s outcome\-by\-memory interaction: positive when the lesson arm’s choice depends on the outcome more than the neutral arm’s does, toward the worse tool after a chance failure of the better one\. P3m is the mirror for the bad decision, toward the worse tool after its lucky success\. Within each previous\-run outcome, the two readers compared hold the same log and the same previous\-run line and differ only in the memory item; the interaction then compares these differences across the two outcomes\.*Qualifiers*locate a detected primary in its adverse cell \(the chance outcome\):A2=Y⁡\[f,RB\]−Y⁡\[f,NEUTRAL\]\\mathrm\{A2\}=Y\[f,\\mathrm\{RB\}\]\-Y\[f,\\mathrm\{NEUTRAL\}\]asks whether the lesson raised the worse\-tool rate above the irrelevant item, andL2=Y⁡\[f,RB\]−Y⁡\[f,RBx\]\\mathrm\{L2\}=Y\[f,\\mathrm\{RB\}\]\-Y\[f,\\mathrm\{RBx\}\]whether the lesson written from the failure raised it above the lesson written from the success; A3 and L3 are the mirrors in the bad decision’s success cell\.

#### Secondary quantities\.

The same contrasts under RBD and N; the lesson\-source effect in the benign cell,L2b=Y⁡\[s,RBx\]−Y⁡\[s,RB\]\\mathrm\{L2b\}=Y\[s,\\mathrm\{RBx\}\]\-Y\[s,\\mathrm\{RB\}\]; remedy contrasts \(RBD minus RB and N minus RB, paired by world\), where a remedy is credited with reducing harm only if both its P and its A differences have 95% intervals below 0; the planted increment Sus, the mean over the four cells ofY⁡\[PLANTED\]−Y⁡\[NEUTRAL\]Y\[\\mathrm\{PLANTED\}\]\-Y\[\\mathrm\{NEUTRAL\}\], which gates nothing; P2 within explicit and within implicit worlds \(descriptive\); and the absolute worse\-tool rates of every cell and arm\.

#### Normative basis\.

In explicit worlds, under the stationary model that the worlds state, the better tool is the optimal choice after either outcome \(the generator’s margin guarantees it\)\. In implicit worlds the log\-based choice is the benchmark, and departures from it are reported as deviations rather than errors\. Independently of any model, a matched lesson is written from information that the reader already holds, so P2, P3m, A2 and A3 have normative value 0 for a reader that knows this, whenever the log and the previous\-run line are the evidence\. The reader itself was told only that its memory items were saved from earlier runs of the task, so it could take a lesson to carry experience beyond the record; the benchmark is an ideal reader that knows the item adds no observation, not a claim about what this reader could infer\. The swapped lesson describes an outcome that the reader was not told, so no normative value is claimed for L: it is the total effect of the inserted lesson’s source outcome \(its advice, its repetition of the outcome, its style and its apparent history together\), with the reader’s explicit text held fixed\.

#### Inference\.

Per contrast and endpoint, the per\-world value is computed on complete worlds and the estimate is their mean\. The registered 95% interval is the mean plus or minus the Student\-ttquantile times the standard error, except that a sample whose values all equalvvon the contrast’s support\[a,b\]\[a,b\]gets\[v−\(v−a\)​q,v\+\(b−v\)​q\]\[v\-\(v\-a\)q,\\ v\+\(b\-v\)q\]withq=1−0\.0251/nq=1\-0\.025^\{1/n\}; this is why an estimate of exactly zero can carry a non\-zero interval\. The supports are\[−2,2\]\[\-2,2\]for P and for the remedy differences of P and L, and\[−1,1\]\[\-1,1\]for A, L, T, Sus and the remedy differences of A\. Beside thetttest, a betting test that is valid in finite samples for world values that are independent and bounded but not identically distributed, following[Waudby\-Smith and Ramdas \[2024\]](https://arxiv.org/html/2610.33029#bib.bib29)and[Shafer \[2021\]](https://arxiv.org/html/2610.33029#bib.bib20): forH0H\_\{0\}: the mean of the worlds’ expected values is at mostmm, with the values rescaled to\[0,1\]\[0,1\]andm′=\(m−a\)/\(b−a\)m^\{\\prime\}=\(m\-a\)/\(b\-a\), the terminal wealthK=∏w\(1\+λ⁡\(yw−m′\)\)K=\\prod\_\{w\}\\bigl\(1\+\\lambda\(y\_\{w\}\-m^\{\\prime\}\)\\bigr\)of one fixed betλ=0\.75/m′\\lambda=0\.75/m^\{\\prime\}has𝔼⁡\[K\]≤1\\mathbb\{E\}\[K\]\\leq 1underH0H\_\{0\}by the inequality of arithmetic and geometric means, andp=min⁡\(1,1/K\)p=\\min\(1,1/K\)by Markov’s inequality; a mean of at leastmmis tested with the values and the null negated\. A primary contrast is*detected*if its estimate is positive, at least 200 worlds are complete, and both the Holm\-adjusted\[[Holm, 1979](https://arxiv.org/html/2610.33029#bib.bib9)\]two\-sidedttpp\-values and the Holm\-adjusted one\-sided bettingpp\-values \(family of the two primaries\) are below 0\.05\. A quantity is*robustly positive*if itsttlower limit is above 0 and its bettingppfor a mean at most 0 is at most 0\.025, and*robustly below 0\.10*if itsttupper limit is below 0\.10 and its bettingppfor a mean at least 0\.10 is at most 0\.025\. Missing\-data bounds set every missing answer, over all 256 worlds, in the direction that lowers \(or raises\) the contrast\.

#### Panel averages\.

The panel’s contrasts average, within each world, the endpoints complete for that world \(on worlds with at least 6 complete endpoints\), and then average over worlds; the panel’s cell rates \(Table[2](https://arxiv.org/html/2610.33029#S5.T2)\) average the endpoints’ rates; the panel’s missing\-data bounds weight all endpoints equally\. The three need not coincide exactly\.

#### Readings\.

Each primary contrast gets one registered word\.*Luck\-carrying*: detected, and in the adverse cell both qualifiers \(A and L\) are robustly positive; it means that after a chance outcome the lesson written from it moved the reader toward the worse tool more than an irrelevant item and more than the lesson written from the other outcome, and that the lesson arm’s outcome dependence exceeded the neutral arm’s; no mechanism \(attribution, reasoning about chance\) is claimed\.*Outcome\-dependent*: detected, but a qualifier is not robustly positive\.*Bounded below 0\.10*: robustly below 0\.10; it bounds the outcome\-by\-memory interaction only, not absolute error rates or memory harm that does not depend on the outcome\.*Unresolved*otherwise, and*descriptive only*below 200 complete worlds\. A word carries the suffix “complete cases; not robust to missing answers” if the relevant missing\-data bound fails its check, and every reading is reported with a flag for an adverse cell at ceiling \(the neutral arm’s worse\-tool rate above 0\.90\)\. The panel’s two readings \(P2 and P3m\) are the confirmatory headline\. The endpoints’ readings are exploratory: they are not adjusted across endpoints \(with eleven independent null endpoints, the registered rule gives about a 6% chance that at least one shows a false detection\), and a panel reading does not imply that every endpoint shares it\. Secondary quantities are reported with intervals, without multiplicity adjustment\.

#### Operating characteristics

\(registered, simulated before the run\)\. With no effect, a false detection of either primary occurred in at most 0\.6% of replications in every null scenario simulated, including heterogeneous world means and shared per\-world states \(thettrule alone reached 10\.6% in the heterogeneous null, which is why the betting test is required\)\. Per endpoint, a true P2 of 0\.10 was detected in 96\.8% of replications at a background discordance of 0\.02 between cells and in 56\.3% at 0\.10\.

## 5Results

The writer stage produced 33,789 lessons\. 3 were missing, all from a good decision’s failure run \(Claude Opus 5\.5 \(RB\), Gemini 3\.8 Flash \(RBD\) and Qwen 3\.8 Max \(RBD\)\): 1 refusal and 2 empty responses; so 6 reader cells were never scheduled\. The reader stage returned 90,084 valid answers and 22 unparsed ones, none recoverable\. For P2, 11 endpoints were complete in 252 worlds and 10 in 4; for P3m, 11 in 254 and 10 in 2\. No adverse cell was at ceiling in any of the 24 panel and endpoint checks, and every reading below holds at its missing\-data bounds\.

### 5\.1The confirmatory readings

Table 1:The panel’s primary contrasts, their qualifiers and the planted increment \(instruction RB\): estimate with its registered 95% interval\.ptp\_\{t\}: two\-sidedtttest;pbetp\_\{\\mathrm\{bet\}\}: one\-sided betting test of a mean at most 0; both Holm\-adjusted over the two primaries, unadjusted for the other rows\. “Check” is the registered criterion the row feeds \(rob\.: robustly; LL: lower limit\); P3m’s bound below 0\.10 is its own registered check \(thettupper limit and a betting test of a mean at least 0\.10\)\. “Lower” and “upper” are the missing\-data bounds, which weight all endpoints equally; this is why the lower bound of P2 can exceed its estimate\.#### Good decision:*luck\-carrying*\.

After a chance failure of the better tool, the panel’s P2 was\+0\.112\+0\.112\[\+0\.097\+0\.097,\+0\.126\+0\.126\] over 256 worlds, detected by both families \(Holm\-adjustedp=p=\{\}1\.8×10−361\.8\\times 10^\{\-36\}for thetttest and7\.0×10−57\.0\\times 10^\{\-5\}for the betting test; Table[1](https://arxiv.org/html/2610.33029#S5.T1)\)\. In the adverse cell the lesson raised worse\-tool choices above the irrelevant item \(A2\+0\.112\+0\.112\[\+0\.097\+0\.097,\+0\.127\+0\.127\]\) and above the lesson written from a success \(L2\+0\.112\+0\.112\[\+0\.098\+0\.098,\+0\.126\+0\.126\]\), both robustly, and the three quantities hold at their lower missing\-data bounds\. In rates \(Table[2](https://arxiv.org/html/2610.33029#S5.T2)\): with the failure\-written lesson in memory, readers named the worse tool in 11\.6% of next runs, against 0\.4% with the irrelevant item and 0\.4% with the success\-written lesson; every reader saw the same log, in which the better tool remained better\.

#### Bad decision:*bounded below 0\.10*\.

After a lucky success of the worse tool, the panel’s P3m was\+0\.002\+0\.002\[−0\.001\-0\.001,\+0\.005\+0\.005\]: not detected \(Holm\-adjustedtt\-testp=p=\{\}0\.2520\.252\) and robustly below 0\.10, also at its upper missing\-data bound\. The worse\-tool rates in that cell were 0\.4% with the lesson and 0\.3% with the irrelevant item\. The reading bounds the outcome\-by\-memory interaction; it does not bound the absolute rate of worse\-tool choices or memory effects that do not depend on the outcome\. In the same cell the lesson\-source effect L3 was small but above zero by itsttinterval \(\+0\.004\+0\.004\[\+0\.002\+0\.002,\+0\.006\+0\.006\]\), not robustly positive\.

#### Planted increment\.

The unreasoned directive moved the panel’s readers by Sus\+0\.137\+0\.137\[\+0\.130\+0\.130,\+0\.143\+0\.143\] \(*increment above 0\.10*\)\. This was carried by 4 endpoints \(Gemini 3\.8 Flash, Qwen 3\.8 Max, DeepSeek V4 Pro and Mistral Large 4\); the other 7 stayed at or below\+0\.004\+0\.004\.

Table 2:Worse\-tool rates by decision, previous\-run outcome and memory arm \(panel mean of the endpoints’ rates\)\. The adverse cells are good/failure and bad/success\. RBx, RBDx and Nx are the lessons written from the other outcome\.

### 5\.2Where it happens \(exploratory\)

−0\.1\-0\.1000\.10\.10\.20\.20\.30\.30\.40\.4Claude Opus 5\.5Claude Sonnet 5\.5Claude Haiku 5\.5GPT\-6\.1 SolGPT\-6 LunaGemini 3\.8 FlashGrok 4\.7Qwen 3\.8 MaxDeepSeek V4 ProMistral Large 4Kimi K3contrast \(difference\-in\-differences of worse\-tool rates\)P2 \(good decision\)P3m \(bad decision\)Figure 1:Per\-endpoint P2 \(filled\) and P3m \(open\) with their registered 95% intervals \(exploratory; not adjusted across endpoints\)\. The dashed line is the registered 0\.10 margin\.Table 3:Per\-endpoint readings \(exploratory\): estimates with their registered 95% intervals; “bounded” is*bounded below 0\.10*\. Complete worlds: 253–256 for P2 and 255–256 for P3m\. Sus is the planted increment; Appendix[E](https://arxiv.org/html/2610.33029#A5)gives the qualifiers and Sus with intervals\.The panel average hides a split \(Figure[1](https://arxiv.org/html/2610.33029#S5.F1), Table[3](https://arxiv.org/html/2610.33029#S5.T3)\)\. P2 carried the chance at 6 endpoints \(Gemini 3\.8 Flash, Grok 4\.7, Qwen 3\.8 Max, DeepSeek V4 Pro, Mistral Large 4 and Kimi K3\) and was bounded below 0\.10 at 5 \(Claude Opus 5\.5, Claude Sonnet 5\.5, Claude Haiku 5\.5, GPT\-6\.1 Sol and GPT\-6 Luna\)\. The largest effects were Qwen 3\.8 Max \(\+0\.352\+0\.352\) and DeepSeek V4 Pro \(\+0\.277\+0\.277\)\. Claude Opus 5\.5, Claude Sonnet 5\.5 and GPT\-6\.1 Sol never named the worse tool in the four cells that P2 uses\. At two bounded endpoints, Claude Haiku 5\.5 and GPT\-6 Luna, the P2ttinterval lay above zero and A2 and L2 were robustly positive \(Claude Haiku 5\.5:\+0\.047\+0\.047\[\+0\.019\+0\.019,\+0\.075\+0\.075\]; GPT\-6 Luna:\+0\.031\+0\.031\[\+0\.010\+0\.010,\+0\.053\+0\.053\]\), but the finite\-sample betting test did not confirm a detection \(Holm\-adjustedpbet=p\_\{\\mathrm\{bet\}\}=\{\}0\.0510\.051and0\.1570\.157\), so their registered word is*bounded below 0\.10*; the small positive estimates stand as reported \(see also Section[5\.7](https://arxiv.org/html/2610.33029#S5.SS7)\)\.

The mirror P3m was bounded below 0\.10 at all 11 endpoints\. DeepSeek V4 Pro came closest: P3m\+0\.031\+0\.031\[\+0\.010\+0\.010,\+0\.053\+0\.053\], with A3 and L3 robustly positive; its Holm\-adjustedtt\-testppwas below 0\.05 \(0\.00450\.0045\) but its bettingppwas not \(0\.0780\.078\), so it was not detected\. After a lucky success of the worse tool, its reader named the worse tool in 8 of 256 worlds with the lesson and in 0 with the irrelevant item\.

### 5\.3What carried the chance \(secondary and exploratory\)

With the failure\-written lesson, the worse\-tool rate was 11\.6% when the previous\-run line reported the failure and 0\.2% when the same lesson was read after a success of the better tool \(L2b\+0\.001\+0\.001\[−0\.001\-0\.001,\+0\.003\+0\.003\]\)\. The effect thus came from the lesson in combination with the reported failure\. The study has no reader that saw the lesson without a previous\-run line and no memory item that restated the failure without a lesson, so it does not separate the lesson’s advice from a second report of the failure, and it does not test whether the previous\-run line is necessary\.

Across endpoints, the share of lessons that named the worse tool did not determine the registered P2 reading \(a registered lesson descriptive, for the RB lessons written from the good decision’s failure run\): among the endpoints that carried the chance the shares were Gemini 3\.8 Flash 26%, Grok 4\.7 0%, Qwen 3\.8 Max 39%, DeepSeek V4 Pro 38%, Mistral Large 4 38% and Kimi K3 29%; among the bounded ones, Claude Opus 5\.5 67%, Claude Sonnet 5\.5 5%, Claude Haiku 5\.5 10%, GPT\-6\.1 Sol 2% and GPT\-6 Luna 0%\. These are endpoint aggregates, not a manipulation of naming\. An illustrative failure lesson written by Claude Opus 5\.5 in one world reads: “One failure is expected variance \(roughly 1 in 8 runs\) and does not show the selection was wrong\.” Appendix[C](https://arxiv.org/html/2610.33029#A3)prints that lesson in full with its readers’ answers, and three lessons that moved their readers, chosen by fixed rules: at Qwen 3\.8 Max, at Grok 4\.7 \(a lesson that does not name the worse tool\) and an RBD lesson at Mistral Large 4\. In each of the three, the lesson recommends switching to the other tool after an error within a run, and the reader’s stated reason applies this to the next run’s first choice; how often lessons take this form was not measured\.

### 5\.4Remedies \(secondary\)

Table 4:Worse\-tool rates after a chance failure of the better tool under four memory arms, for the endpoints that carried the chance \(exploratory\)\.Table 5:Remedy differences against RB for the endpoints that carried the chance \(exploratory\), with registered 95% intervals\. “Credited”: the registered rule credits the remedy with reducing harm \(both the P and the A difference have intervals below 0\)\.On the panel, both remedies met the registered harm\-reduction rule\. With the decision\-quality note \(RBD\), the differences against RB were−0\.039\-0\.039\[−0\.053\-0\.053,−0\.026\-0\.026\] for P2 and−0\.041\-0\.041\[−0\.054\-0\.054,−0\.027\-0\.027\] for A2, and the worse\-tool rate after a chance failure was 7\.5%; with the single instruction for both outcomes \(N\),−0\.085\-0\.085\[−0\.097\-0\.097,−0\.073\-0\.073\] and−0\.086\-0\.086\[−0\.098\-0\.098,−0\.074\-0\.074\], and a rate of 3\.1%\. N’s estimated reduction was larger, but no registered test compares the two remedies, and N differs from RB in two ways\. Under either remedy the interaction stayed above zero by itsttinterval \(P2\+0\.072\+0\.072\[\+0\.061\+0\.061,\+0\.082\+0\.082\] under RBD and\+0\.027\+0\.027\[\+0\.020\+0\.020,\+0\.034\+0\.034\] under N; bettingppfor a mean at most 0:0\.00130\.0013and0\.0800\.080\); no reading is registered for these secondary quantities\. By endpoint \(Tables[4](https://arxiv.org/html/2610.33029#S5.T4)and[5](https://arxiv.org/html/2610.33029#S5.T5)\), RBD met the rule for 3 of the 6 endpoints that carried the chance \(Gemini 3\.8 Flash, Grok 4\.7 and DeepSeek V4 Pro\) and N for 5 \(all but Mistral Large 4\)\. For Mistral Large 4, the worse\-tool rate after a chance failure was 16\.9% with RB, 18\.4% with RBD and 15\.2% with N\. The bad decision’s remedy flags compare against a P3m that was already bounded on the panel\.

### 5\.5The independence statement \(secondary, descriptive\)

In worlds whose log stated that failures are independent one\-off events, the panel’s P2 was\+0\.042\+0\.042\[\+0\.031\+0\.031,\+0\.053\+0\.053\]; in worlds whose log did not state it,\+0\.181\+0\.181\[\+0\.160\+0\.160,\+0\.203\+0\.203\]\. P2 was smaller in the explicit worlds and above zero in both; with equal numbers of worlds in the two halves, most of the panel’s estimate came from the implicit worlds\. The split is registered as descriptive; it compares different worlds and is not a test of the statement’s effect\.

### 5\.6The planted directive and failure\-written lessons \(exploratory\)

Kimi K3 and Grok 4\.7 named the worse tool after the planted directive in 0 and 4 of their 1,024 answers to it \(Sus0\.0000\.000and\+0\.004\+0\.004\), yet their P2 was luck\-carrying \(\+0\.125\+0\.125and\+0\.105\+0\.105\)\. At these two endpoints a near\-absent response to this single unreasoned directive coexisted with lessons that carried the chance\. The study does not identify a mechanism and does not test any defence\.

### 5\.7Checks

Answers are counted by theirfirst\_toolfield, as registered\. A few answers named one tool infirst\_toolwhile their reason argued for the other\. One of them, in Claude Haiku 5\.5’s good\-decision success cell, sets that endpoint’s only negative per\-world P2 value; read by its reason, the betting test would have detected Claude Haiku 5\.5’s P2\. The registered word stands, and the dependence is disclosed\. Among the lessons that ended normally and were shown to readers as registered were a one\-line non\-lesson \(a remark by Qwen 3\.8 Max about its knowledge cut\-off\), a leaked reasoning tag and a few misread counts\.

An independent re\-implementation of the analysis was written by a separate agent instructed to work from the registration and the record\-format notes alone and not to open the analysis code \(whether it complied was not verified\)\. It recomputed the primaries, qualifiers, missing\-data bounds, planted increment, secondary contrasts and remedy contrasts, per endpoint and for the panel: 3,480 numbers agreed within1\.5×10−151\.5\\times 10^\{\-15\}\(floating\-point rounding\), with 0 disagreements among the 36 readings compared\. It did not recompute the bootstrap, the cue splits, the Wilson intervals, the regret or the lesson descriptives, and did not verify the lock, manifest or timestamp records; it resolved 15 ambiguities of the registration, none of which changes a reading\. Two reviews of the results, one by OpenAI’s Codex and one by a separate Claude instance, recomputed the registered summaries from the locked records and found them right; they also identified wording and disclosure defects in the internal report from which this paper is written\.

## 6Discussion

#### A lesson can outweigh the record it summarises\.

Every reader held the authoritative log, in which the better tool remained better, and was told the previous run’s result\. When the log and the previous\-run line are the evidence, a matched lesson adds no observation the reader lacks, so for a reader that knows where the lesson came from the matched contrasts have normative value zero \(Section[4](https://arxiv.org/html/2610.33029#S4); this reader was told only that its memory came from earlier runs\); under the explicit worlds’ stated model, the better tool is the optimal choice after either outcome\. With an irrelevant item or the success\-written lesson in memory, the worse\-tool rate after a chance failure stayed near zero \(0\.4% and 0\.4%\); with the failure\-written lesson it rose to 11\.6% on the panel and to 39\.1% at Qwen 3\.8 Max\. The registered reading was*luck\-carrying*at 6 of the 11 endpoints, and at 2 more \(Claude Haiku 5\.5 and GPT\-6 Luna\) the adverse\-cell effects A2 and L2 were robustly positive without a detected P2\. No reader was given an observation about the tools beyond the log and the previous\-run line, and within a previous\-run outcome the readers differed only in the memory item\. The change came from the lesson together with the reported failure: read after a reported success, the same lesson left the rate at 0\.2%\.

#### Most of the effect arose where the record left room for doubt\.

The interaction was\+0\.181\+0\.181in the worlds whose record did not state that failures are independent and\+0\.042\+0\.042in those whose record did \(descriptive\)\. Where independence is not stated, a recent failure can be read as a sign of a current fault, so a worse\-tool choice there is a deviation from the log\-based benchmark rather than a clear error; the lesson still added no observation beyond the record\. Where it is stated, the smaller effect that remained is an error under the stated model\. A reader of the panel average should therefore not take all of it as an increase in clear errors\.

#### The two kinds of luck gave different readings\.

After a chance failure of a good decision, the panel’s lessons carried the chance; after a lucky success of a bad decision, the panel’s outcome\-by\-memory interaction was bounded below 0\.10\. The bound does not establish that lessons never entrench a worse choice, nor that memory does no harm independent of the outcome; at one endpoint \(DeepSeek V4 Pro\) the mirror effect was positive by itstttest\. One explanation of the difference, which this study did not test, is the outcome routing itself: ReasoningBank’s failure instruction asks the writer to “reflect and think why the trajectory failed”, to propose “strategies to prevent the failure” and, in its released form, to prefer “concrete, actionable recovery procedures” \(Appendix[A](https://arxiv.org/html/2610.33029#A1)\), which presupposes that something about the run was at fault and invites a change of action, while after a success the log itself argues against the worse tool\. The remedies are consistent with this explanation without settling it: a note asking the writer to judge the decision rather than the outcome, and a single instruction for both outcomes, both lowered the effect on the panel\. Another difference could contribute: the scripted good decision states its reason \(“the more reliable tool”\) and the bad decision gives none \(Section[3\.2](https://arxiv.org/html/2610.33029#S3.SS2)\), so the two decisions’ runs differ in more than quality\.

#### Two routes in the examples\.

The example lessons that moved their readers \(Appendix[C](https://arxiv.org/html/2610.33029#A3)\) recommend a recovery procedure, switching to the other tool after an error within a run, and their readers applied it to the next run’s first choice\. They point to two routes that the design does not separate: a writer that turns a chance failure into advice against the tool, and a reader that applies a conditional recovery step outside its condition\. Both carry the chance into the next decision; their shares were not measured, and a lesson that explicitly limits the step to the run in which the error occurred could help distinguish them\.

#### The same pipeline behaved differently across models\.

The endpoint readings are exploratory and the panel is a sample of serving configurations, but the spread was wide: from no worse\-tool choice at all in the cells that P2 uses \(Claude Opus 5\.5, Claude Sonnet 5\.5 and GPT\-6\.1 Sol\) to\+0\.352\+0\.352at Qwen 3\.8 Max\. How a memory\-writing pipeline behaves with one model need not carry over to another\.

#### Implications for memory design\.

Four follow within the study’s scope\. \(i\) Writing instructions that route by outcome deserve a test with chance outcomes before deployment; the instrument here \(matched and swapped lessons, an irrelevant item, a planted directive, an information\-matched reader\) is such a test, and its frozen package is public\. \(ii\) Asking the writer to judge the decision against the information available before the outcome, or using one instruction for both outcomes with an explicit option to change nothing, lowered the effect on the panel, but neither met the harm\-reduction rule at every endpoint that carried the chance, and both were tested only with outcomes that did not signal a change in a tool’s reliability; whether they also suppress learning from outcomes that do is open \(Section[7](https://arxiv.org/html/2610.33029#S7)\)\. \(iii\) Stating in the record that failures are independent was associated with a smaller effect, descriptively\. \(iv\) The planted directive and the lessons are different kinds of memory items: at two endpoints, a near\-absent response to the unreasoned directive coexisted with lessons that carried the chance, so a test with planted directives alone would not have shown the effect there\.

## 7Limitations

- •One link of the chain\.One stored lesson, read once, after one scripted previous run\. The study does not execute self\-improvement, retrieval or accumulated memory, and the reader names a tool without running it; lessons that accumulate, are retrieved among many, or are consolidated may behave differently, in either direction\.
- •Synthetic worlds and a constrained choice\.Two\-tool operational worlds in 16 surface templates, with a first\-tool choice restricted by structured output\. The study does not estimate how often real agents meet such worlds\.
- •A favourable reader\.The reader held the authoritative record\. Whether a reader without it would be moved more was not tested\.
- •Lesson and reported failure together\.The effect was measured with the lesson read alongside a previous\-run line that reported the failure; after a reported success the same lesson barely moved readers\. No reader saw the lesson without the previous\-run line, and no memory item restated the failure without a lesson, so the study cannot separate the lesson’s advice from a second report of the failure\. A memory item that records the same failure as a fact, in the same format and without interpretation or advice, is the most direct next control\.
- •Remedies tested only with non\-diagnostic outcomes\.No outcome in the study signalled a change in a tool’s underlying reliability: failures were transient, and the better tool stayed better after either outcome\. Whether the remedies also suppress learning from outcomes that do signal such a change \(a broken or changed tool\) was not tested; N in particular offers the writer an explicit option to change nothing\.
- •One deployed instruction and one representation\.RB is ReasoningBank’s released instruction adapted by two declared replacements, one of which lets the writer name the tools; the released instruction differs from the printed one in three places \(Appendix[A](https://arxiv.org/html/2610.33029#A1)\)\. Neither the printed instruction nor the unmodified naming rule was tested\. RBD and N are two variants; the whole writer response is stored as one block\. Other memory\-writing designs and representations were not tested, and N against RB is a compound comparison\.
- •Writer and reader together\.Each endpoint wrote the lessons its readers read, so endpoint differences cannot be assigned to writing or to reading separately; each cell had one generation, and the study has not been replicated\.
- •Endpoints as served\.11 endpoints at their providers’ default sampling on one day, each through one pinned provider; served models change\. The panel is a sample of serving configurations, not of task families, and the endpoint readings are exploratory\.
- •Counting byfirst\_tool\.A few answers’ reasons disagree with theirfirst\_tool; one of them decides whether the betting test detects Claude Haiku 5\.5’s P2 \(Section[5\.7](https://arxiv.org/html/2610.33029#S5.SS7)\)\.
- •No mechanism\.The readings say what moved, not why\. The explanations in Section[6](https://arxiv.org/html/2610.33029#S6)are untested; the scripted good and bad decisions differ in whether a reason is stated \(Section[3\.2](https://arxiv.org/html/2610.33029#S3.SS2)\); and the design does not separate a writer’s advice from a reader’s misapplication of it\.
- •A pilot\-informed design\.The cue, the reader, the stored\-memory representation and two quantities were changed after inspecting development runs \(Section[3\.5](https://arxiv.org/html/2610.33029#S3.SS5)\); the confirmatory worlds and run were untouched by them\.
- •Reviews by language models\.The protocol, results and manuscript reviews and the re\-implementation were done by language\-model agents \(Section[8](https://arxiv.org/html/2610.33029#S8)\); no external human reviewer has assessed the study\.

## 8Statements

#### Data and code availability\.

The frozen package of the study \(the registration, the runner and serving code, the world generator and the confirmatory worlds, every prompt, the registered analysis code, the development pilot’s records and the design reviews; 74 files with their SHA\-256 manifest and its OpenTimestamps proof\) is public on Zenodo \(doi:10\.5281/zenodo\.23253562\)\. The registered analysis’s output, its generated supplement \(cell rates with Wilson intervals, the qualifiers’ missing\-data bounds, secondary intervals, missing and unparsed answers with the providers’ finish reasons, regret and lesson descriptives\) and the re\-implementation’s comparison accompany this paper as ancillary files \(Appendix[H](https://arxiv.org/html/2610.33029#A8)\)\. The locked records of the confirmatory run are public on Zenodo \(doi:10\.5281/zenodo\.23272809\): every writer and reader request and record, the worlds, the completion manifest with its timestamp proof, a flat machine\-readable extract of all records and a script that checks the records against the manifest; the two call ledgers are included with one field, a local file path, replaced by the file’s name\. Run on these records, the registered analysis code reproduces the registered output\. Appendix[C](https://arxiv.org/html/2610.33029#A3)prints four lessons and answers from them\.

#### Use of language models\.

The author used language\-model assistants for most of the work: Anthropic’s Claude \(Claude Opus 5\.5, principally through Claude Code\) for the literature search, the selection among candidate questions under a documented procedure, the design, the implementation and execution of the runner, the analysis and audit tooling, the independent re\-implementation and the drafting of this manuscript; and OpenAI’s Codex \(GPT\-6\.1 Sol\) for critique of the design, the results and this manuscript, alongside separate Claude instances that audited them\. The author also obtained a critique of a draft from OpenAI’s ChatGPT; its points were checked against the records before any change was made\. The author set the research programme, approved the study’s execution and its spending, reviewed the outcome and is responsible for the manuscript\. No language model is an author\. The models studied are experimental subjects, not tools of the analysis: their choices were scored by the registeredfirst\_toolrule, without a language\-model judge\.

#### Ethics\.

The study involved no human participants and no personal data\. The models were accessed through OpenRouter under the providers’ terms, on synthetic and benign operational tasks\. The single writer refusal was returned by a provider’s content filter on one such task\.

## References

- Almeida et al\. \[2024\]Guilherme F\. C\. F\. Almeida, José Luiz Nunes, Neele Engelmann, Alex Wiegmann, and Marcelo de Araújo\.Exploring the psychology of LLMs’ moral and legal reasoning\.*Artificial Intelligence*, 333:104145, 2024\.doi:10\.1016/j\.artint\.2024\.104145\.arXiv:2308\.01264\.
- Asadolahi et al\. \[2026\]Mohammad Asadolahi, Amir Amini, Samira Talebi, Amirfarhad Farhadi, and Azadeh Zamanifar\.Memory reward inflation in self\-improving LLM agents, 2026\.URL[https://arxiv\.org/abs/2608\.00017](https://arxiv.org/abs/2608.00017)\.arXiv:2608\.00017\.
- Baron and Hershey \[1988\]Jonathan Baron and John C\. Hershey\.Outcome bias in decision evaluation\.*Journal of Personality and Social Psychology*, 54\(4\):569–579, 1988\.doi:10\.1037/0022\-3514\.54\.4\.569\.
- Chen et al\. \[2026\]Wenhui Chen, Jianlin Chen, Ziyao Lin, and Chi Man Vong\.Self\-cleaning and captured anyway: One measured primitive for error in a store an agent writes to itself, and what a falling score actually measures, 2026\.URL[https://arxiv\.org/abs/2609\.25052](https://arxiv.org/abs/2609.25052)\.arXiv:2609\.25052\.
- Chen et al\. \[2024\]Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li\.AgentPoison: Red\-teaming LLM agents via poisoning memory or knowledge bases, 2024\.URL[https://arxiv\.org/abs/2407\.12784](https://arxiv.org/abs/2407.12784)\.arXiv:2407\.12784\.
- Dixit et al\. \[2026\]Prakhar Dixit, Sadia Kamal, and Tim Oates\.Honest lying: Understanding memory confabulation in reflexive agents, 2026\.URL[https://arxiv\.org/abs/2605\.29463](https://arxiv.org/abs/2605.29463)\.arXiv:2605\.29463\.
- Dong et al\. \[2025\]Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, and Zhen Xiang\.Memory injection attacks on LLM agents via query\-only interaction, 2025\.URL[https://arxiv\.org/abs/2503\.03704](https://arxiv.org/abs/2503.03704)\.arXiv:2503\.03704\.
- Fischhoff \[1975\]Baruch Fischhoff\.Hindsight is not equal to foresight: The effect of outcome knowledge on judgment under uncertainty\.*Journal of Experimental Psychology: Human Perception and Performance*, 1\(3\):288–299, 1975\.doi:10\.1037/0096\-1523\.1\.3\.288\.
- Holm \[1979\]Sture Holm\.A simple sequentially rejective multiple test procedure\.*Scandinavian Journal of Statistics*, 6\(2\):65–70, 1979\.URL[https://www\.jstor\.org/stable/4615733](https://www.jstor.org/stable/4615733)\.
- Horowitz and Plonsky \[2025\]Idan Horowitz and Ori Plonsky\.LLM agents display human biases but exhibit distinct learning patterns, 2025\.URL[https://arxiv\.org/abs/2503\.10248](https://arxiv.org/abs/2503.10248)\.arXiv:2503\.10248\.
- Kumaran et al\. \[2025\]Dharshan Kumaran, Stephen M Fleming, Larisa Markeeva, Joe Heyward, Andrea Banino, Mrinal Mathur, Razvan Pascanu, Simon Osindero, Benedetto de Martino, Petar Velickovic, et al\.How overconfidence in initial choices and underconfidence under criticism modulate change of mind in large language models, 2025\.URL[https://arxiv\.org/abs/2507\.03120](https://arxiv.org/abs/2507.03120)\.arXiv:2507\.03120\.
- Li et al\. \[2026\]Bobo Li, Rui Wu, Zibo Ji, Meishan Zhang, Hao Fei, Min Zhang, Mong\-Li Lee, and Wynne Hsu\.Taming actor\-observer asymmetry in agents via dialectical alignment, 2026\.URL[https://arxiv\.org/abs/2604\.19548](https://arxiv.org/abs/2604.19548)\.arXiv:2604\.19548\.
- Matsuura et al\. \[2026\]Misaki Matsuura, Sayantan Kumar, Ojas Kadam, and Jeremy C\. Weiss\.Hindsight bias in clinical temporal reasoning: How future data exposure affects large language model judgment, 2026\.URL[https://arxiv\.org/abs/2609\.13454](https://arxiv.org/abs/2609.13454)\.arXiv:2609\.13454\.
- Nakayashiki \[2026a\]Kazuki Nakayashiki\.Checking before acting: A critical review of state revalidation in long\-horizon LLM agents, 2026a\.URL[https://doi\.org/10\.5281/zenodo\.23221641](https://doi.org/10.5281/zenodo.23221641)\.
- Nakayashiki \[2026b\]Kazuki Nakayashiki\.Plan pointers and record\-directive form in budgeted verification of inherited agent memory, 2026b\.URL[https://arxiv\.org/abs/2609\.03450](https://arxiv.org/abs/2609.03450)\.arXiv:2609\.03450\.
- Nakayashiki \[2026c\]Kazuki Nakayashiki\.When stale constraints go unchecked: Budgeted verification failures in inherited agent memory, 2026c\.URL[https://arxiv\.org/abs/2608\.25553](https://arxiv.org/abs/2608.25553)\.arXiv:2608\.25553\.
- Ouyang et al\. \[2025\]Siru Ouyang, Jun Yan, I\-Hung Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T\. Le, Samira Daruki, Xiangru Tang, et al\.ReasoningBank: Scaling agent self\-evolving with reasoning memory, 2025\.URL[https://arxiv\.org/abs/2509\.25140](https://arxiv.org/abs/2509.25140)\.arXiv:2509\.25140\.
- Park et al\. \[2023\]Joon Sung Park, Joseph C\. O’Brien, Carrie J\. Cai, Meredith Ringel Morris, Percy Liang, and Michael S\. Bernstein\.Generative agents: Interactive simulacra of human behavior, 2023\.URL[https://arxiv\.org/abs/2304\.03442](https://arxiv.org/abs/2304.03442)\.arXiv:2304\.03442\.
- Schubert et al\. \[2024\]Johannes A\. Schubert, Akshay K\. Jagadish, Marcel Binz, and Eric Schulz\.In\-context learning agents are asymmetric belief updaters, 2024\.URL[https://arxiv\.org/abs/2402\.03969](https://arxiv.org/abs/2402.03969)\.arXiv:2402\.03969\.
- Shafer \[2021\]Glenn Shafer\.Testing by betting: A strategy for statistical and scientific communication\.*Journal of the Royal Statistical Society Series A: Statistics in Society*, 184\(2\):407–431, 2021\.doi:10\.1111/rssa\.12647\.
- Shinn et al\. \[2023\]Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao\.Reflexion: Language agents with verbal reinforcement learning, 2023\.URL[https://arxiv\.org/abs/2303\.11366](https://arxiv.org/abs/2303.11366)\.arXiv:2303\.11366\.
- Suzgun et al\. \[2025\]Mirac Suzgun, Mert Yuksekgonul, Federico Bianchi, Dan Jurafsky, and James Zou\.Dynamic Cheatsheet: Test\-time learning with adaptive memory, 2025\.URL[https://arxiv\.org/abs/2504\.07952](https://arxiv.org/abs/2504.07952)\.arXiv:2504\.07952\.
- Wang et al\. \[2023\]Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar\.Voyager: An open\-ended embodied agent with large language models, 2023\.URL[https://arxiv\.org/abs/2305\.16291](https://arxiv.org/abs/2305.16291)\.arXiv:2305\.16291\.
- Wang et al\. \[2026a\]Haomiaomiao Wang, Tomás E Ward, and Lili Zhang\.Comparative reversal learning reveals rigid adaptation in LLMs under non\-stationary uncertainty, 2026a\.URL[https://arxiv\.org/abs/2604\.04182](https://arxiv.org/abs/2604.04182)\.arXiv:2604\.04182\.
- Wang et al\. \[2026b\]Haoyu Wang, Guangyuan Dong, He Liang, Zijing Zhang, Jiachen Luo, Chuang Liu, Chao Xue, and Hao Tang\.MemGuard: Persisting verifier signals for LLM\-agent memory governance, 2026b\.URL[https://arxiv\.org/abs/2608\.21867](https://arxiv.org/abs/2608.21867)\.arXiv:2608\.21867\.
- Wang et al\. \[2026c\]Kaixiang Wang, Jiong Lou, Zhaojiacheng Zhou, and Jie Li\.OEP: Poisoning self\-evolving LLM agents via locally correct but non\-transferable experiences, 2026c\.URL[https://arxiv\.org/abs/2605\.18930](https://arxiv.org/abs/2605.18930)\.arXiv:2605\.18930\.
- Wang et al\. \[2026d\]Su Wang, Pin Qian, Yifan Lin, Jingzhou Xu, Yihang Chen, Xiaochong Jiang, Lifei Liu, and Haoran Yu\.Phantom guardrails: When self\-improving agent harnesses fix failures that never happened, 2026d\.URL[https://arxiv\.org/abs/2607\.13083](https://arxiv.org/abs/2607.13083)\.arXiv:2607\.13083\.
- Wang et al\. \[2024\]Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig\.Agent Workflow Memory, 2024\.URL[https://arxiv\.org/abs/2409\.07429](https://arxiv.org/abs/2409.07429)\.arXiv:2409\.07429\.
- Waudby\-Smith and Ramdas \[2024\]Ian Waudby\-Smith and Aaditya Ramdas\.Estimating means of bounded random variables by betting\.*Journal of the Royal Statistical Society Series B: Statistical Methodology*, 86\(1\):1–27, 2024\.doi:10\.1093/jrsssb/qkad009\.arXiv:2010\.09686\.
- Wei et al\. \[2026\]Zhijie Wei, Ferris Tan, and Jinghui Wang\.Scale and selection: What makes automatic harness evolution work for visual\-interface robot agents, 2026\.URL[https://arxiv\.org/abs/2609\.39304](https://arxiv.org/abs/2609.39304)\.arXiv:2609\.39304\.
- Xiong et al\. \[2025\]Zidi Xiong, Yuping Lin, Wenya Xie, Pengfei He, Zirui Liu, Jiliang Tang, Himabindu Lakkaraju, and Zhen Xiang\.How memory management impacts LLM agents: An empirical study of experience\-following behavior, 2025\.URL[https://arxiv\.org/abs/2505\.16067](https://arxiv.org/abs/2505.16067)\.arXiv:2505\.16067\.
- Ye et al\. \[2026\]Qinyuan Ye, Yu Li, Yada Pruksachatkun, Jiaxin Zhang, and Chien\-Sheng Wu\.On the fragility of self\-improving agents: Variance, task order, and underspecification, 2026\.URL[https://arxiv\.org/abs/2608\.18066](https://arxiv.org/abs/2608.18066)\.arXiv:2608\.18066\.
- Zhang et al\. \[2025\]Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong, Vamsidhar Kamanuru, Jay Rainton, Chen Wu, Mengmeng Ji, Hanchen Li, et al\.Agentic context engineering: Evolving contexts for self\-improving language models, 2025\.URL[https://arxiv\.org/abs/2510\.04618](https://arxiv.org/abs/2510.04618)\.arXiv:2510\.04618\.
- Zhao et al\. \[2023\]Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong\-Jin Liu, and Gao Huang\.ExpeL: LLM agents are experiential learners, 2023\.URL[https://arxiv\.org/abs/2308\.10144](https://arxiv.org/abs/2308.10144)\.arXiv:2308\.10144\.
- Zhu et al\. \[2026\]Shiding Zhu, Yudi Qi, Yajie Wang, Jiaze Li, Chao Song, Yaorui Shi, Yibo Miao, Hanqi Gao, and Kai Zhang\.Escaping the self\-confirmation trap: An Execute\-Distill\-Verify paradigm for agentic experience learning, 2026\.URL[https://arxiv\.org/abs/2606\.24428](https://arxiv.org/abs/2606.24428)\.arXiv:2606\.24428\.

## Appendix AWriter instructions

RB is ReasoningBank’s pair of instructions\[[Ouyang et al\., 2025](https://arxiv.org/html/2610.33029#bib.bib17)\]\(Apache\-2\.0\), taken from the authors’ released code: the string constantsSUCCESSFUL\_SIandFAILED\_SIofWebArena/prompts/memory\_instruction\.pyin the repository[https://github\.com/google\-research/reasoning\-bank](https://github.com/google-research/reasoning-bank)at commited80611788292ea739f1effd31f16c53823b8a0d\(file SHA\-256073d354bc84a24ba8a2c697133b5d832fe89ca67a53705fb0af419528f7b11ee\), fetched on 2026\-10\-08 and read without executing the file\. The text, its provenance record and the licence are in the deposited package\.

The released instructions differ from those printed in the ReasoningBank paper \(arXiv version 2, Appendix A\.1\) in three places, the same in both instructions except where noted:

- •the printed note “Do not mention specific websites, queries, or string contents, but rather focus on the generalizable insights\.” reads in the code “Prefer concrete, actionable procedures over abstract principles\. Do not embed specific product names, queries, or literal string contents from the task\.”, with “recovery procedures” in the failure instruction;
- •the printed description field, “one sentence summary of the memory item”, reads “one sentence summary describing when or when NOT to use the memory item”;
- •the printed content field, “1\-3 sentences describing the insights learned to successfully accomplishing the task”, ends “… to successfully accomplishing similar tasks in the future” in the success instruction and “… to avoid such failures and successfully accomplishing similar tasks in the future” in the failure instruction\.

This study sent the released text with two declared replacements: “an expert in web navigation” became “an expert in running operational tasks”, and “Do not embed specific product names, queries, or literal string contents from the task\.” became “Do not embed specific queries or literal string contents from the task, but you may name the tools\.” Neither the printed instructions nor the released text without these replacements was run\. The failure instruction, as sent:

Youareanexpertinrunningoperationaltasks\.Youwillbegivenauserquery,thecorrespondingtrajectorythatrepresents\*\*howanagentattemptedtoresolvethetaskbutfailed\*\*\.

\#\#Guidelines

Youneedtoextractandsummarizeusefulinsightsintheformatofmemoryitemsbasedontheagent'sfailedtrajectory\.

Thegoalofsummarizedmemoryitemsistobehelpfulandgeneralizableforfuturesimilartasks\.

\#\#Importantnotes

\-Youmustfirstreflectandthinkwhythetrajectoryfailed,andthensummarizewhatlessonsyouhavelearnedorstrategiestopreventthefailureinthefuture\.

\-Youcanextract\*atmost3\*memoryitemsfromthetrajectory\.

\-Youmustnotrepeatsimilaroroverlappingitems\.

\-Preferconcrete,actionablerecoveryproceduresoverabstractprinciples\.Donotembedspecificqueriesorliteralstringcontentsfromthetask,butyoumaynamethetools\.

\#\#OutputFormat

YouroutputmuststrictlyfollowtheMarkdownformatshownbelow:

\`\`\`

\#MemoryItemi

\#\#Title<thetitleofthememoryitem\>

\#\#Description<onesentencesummarydescribingwhenorwhenNOTtousethememoryitem\>

\#\#Content<1\-3sentencesdescribingtheinsightslearnedtoavoidsuchfailuresandsuccessfullyaccomplishingsimilartasksinthefuture\>

\`\`\`

RBD adds, as the first bullet under “Important notes” of both RB instructions: “Before writing, judge whether the agent’s decision was sound given only what was known before the outcome\. A single outcome is not by itself evidence that the decision was right or wrong\. Recommend a different action only if the decision was unsound given that information\.” N is RB’s success instruction with its four outcome\-specific phrases made neutral and the sentence “If nothing about how the task should be done needs to change, say so\.” added; it is used after both outcomes\. Every rendered instruction’s SHA\-256 is asserted by the package’s tests\.

## Appendix BReader prompt

The reader’s system message:

Youareanoperationsagentthatrunsthesametasksregularly\.Beforeeachrunyoureadthememoryitemsthatweresavedfromearlierrunsofthetask\.

An example user message \(a sample world from the package’s rendering samples, not a confirmatory world; good decision, failure, the lesson written from the failure\):

Task:Exportthenightlysalesrecordstothereportingbucket\.

Toolsavailableforthistask\(bothoperationaltoday\):

\-export\_via\_ellery:exportsthenight'ssalesrecordstothereportingbucketthroughtheElleryroute

\-export\_via\_zanric:exportsthenight'ssalesrecordstothereportingbucketthroughtheZanricroute

Runlog\(authoritative;stableformonths;last240runsofeachtool\):export\_via\_ellerysucceededin202of240;export\_via\_zanricsucceededin145of240\.Therehavebeennooutages:failuresofeithertoolareindependentone\-offevents,andthenextrunhasalwaysstartednormally\.Bothtoolscostthesametocall,andafailedrunhastobere\-runbyhand\.

Previousrunofthistask:export\_via\_ellerywasusedanditfailed\("ERROR503:upstreamtemporarilyunavailable"\)\.Thisisanewscheduledrunofthetask\.Thepreviousrunisfinishedandwillnotberepeated\.

Memoryitemssavedfromearlierrunsofthistask:

<thewriter'swholeresponsetothegood\-decisionFAILUREepisode,e\.g\.\#MemoryItem1\.\.\.\>

Whichtoolwillyoucallfirstforthisrun?ReplyinJSONwiththetool'snameandaone\-sentencereason\.

## Appendix CExample lessons and answers

Four examples from the locked records of the confirmatory run, all from the good decision’s failure run \(a chance failure of the better tool\), chosen by rules fixed before the records were searched and applied in world\-identifier order\. Each of A–C is the first world in which the reader with the matched lesson named the worse tool while the reader with the irrelevant item named the better tool: A at the endpoint with the largest P2 \(Qwen 3\.8 Max, RB\); B at Grok 4\.7, among RB lessons that do not name the worse tool by the registered descriptive’s rule \(its full name or its coined part\); C at the one endpoint that carried the chance and for which neither remedy was credited \(Mistral Large 4, RBD\)\. D is the Claude Opus 5\.5 lesson quoted in Section[5\.3](https://arxiv.org/html/2610.33029#S5.SS3); both of its readers, with the lesson and with the irrelevant item, named the better tool\. Each lesson is printed as its reader received it \(its SHA\-256 equals the memory digest recorded with the reader’s answer\), with non\-ASCII punctuation rendered in ASCII; an answer is itsfirst\_toolandreasonfields\. The examples illustrate single cases; they do not show how often lessons take these forms\. In A–C the lesson recommends switching to the other tool after an error within a run, and the reader applied this to the next run’s first choice\.

### A: Qwen 3\.8 Max, world w20261009\_0004

Worldw20261009\_0004\(backup;cue:explicit\)\.Bettertoolbackup\_via\_ellery,worsetoolbackup\_via\_aldous\.

Endpointqwen/qwen3\.8\-max\-0902\.Gooddecision,chancefailure\.Lesson\(RB,writtenfromthefailure\):

\#MemoryItem1

\#\#TitleFallbacktotheotherbackuptoolwhenthefirstbackuptoolfails

\#\#DescriptionUsewhenabackupoperationusingbackup\_via\_elleryorbackup\_via\_aldouserrorsandthetaskstillallowsanotheraction\.

\#\#ContentAfailurefromonebackuptoolshouldbetreatedastool\-specific,notasfinaltaskfailure\.Immediatelytrythealternatebackuptoolinthesamerunbeforecallingfinish,especiallywhenread\_run\_logindicatesfailuresareindependentandcostsareequal\.

\#MemoryItem2

\#\#TitleDonotfinishuntilbackupsuccessisconfirmedorrecoveryoptionsareexhausted

\#\#DescriptionUsebeforecallingfinishafteranytoolerrorinaone\-shotoperationalrun\.

\#\#ContentVerifythatthebackupcallreturnedasuccessfulcompletionratherthananerror\.Ifitfailed,pursueavailablerecoverystepssuchasanalternatetool,andonlyfinishunsuccessfullywhennofurtherpermittedactionremains\.

Readerwiththislesson:first\_tool=backup\_via\_aldous\(theworsetool\)\.

Reason:backup\_via\_elleryjustfailedandthestoredguidancesaystofallbacktothealternatebackuptoolimmediately,withbothtoolscostingthesameandfailuresindependent\.

Readerwiththeirrelevantitem:first\_tool=backup\_via\_ellery\(thebettertool\)\.

Reason:Thepreviousfailuredoesnotaffectthisindependentrun,andbackup\_via\_elleryhasamuchhigherrecentsuccessrate\(309/365vs\.196/365\)\.

### B: Grok 4\.7, world w20261009\_0018

Worldw20261009\_0018\(ci;cue:explicit\)\.Bettertoolbuild\_via\_verran,worsetoolbuild\_via\_kestle\.

Endpointx\-ai/grok\-4\.7\.Gooddecision,chancefailure\.Lesson\(RB,writtenfromthefailure;itdoesnotnametheworsetool\):

\#MemoryItem1

\#\#TitleFailoverafterschedulereviction

\#\#DescriptionUsewhenabuildorjobtoolfailsbecauseitsnodewasdrainedortheschedulerevictedthejob,andanequivalentalternatetoolexists\.

\#\#ContentTreatnode\-drainandscheduler\-evictionerrorsaspath\-specificinfrastructurefailures,notasproofthetaskcannotrun\.Immediatelycalladifferentequivalentbuilderorexecutorinsteadoffinishing;donotretryonlythesamedrainedpathiftheerrorsaysthatinvocationcannotberepeated\.

\#MemoryItem2

\#\#TitleDonotpickatoolfromlong\-runaveragesalone

\#\#DescriptionUsewhenseveraltoolscanperformthesamejobandarunlogshowsstable,independentone\-offfailureswithnocurrentoutagesignal\.

\#\#ContentHistoricalsuccessrateisaweaktie\-break,notahealthcheck\.Beforecommitting,inspectcurrentschedulerornodestatus;ifthatisunavailable,bereadytoswitchtoolsonthefirstinfrastructureerrorratherthantreatingthehistoricallybettertoolastheonlyattempt\.

\#MemoryItem3

\#\#TitleSeparatemanualre\-runnotesfromsessionconstraints

\#\#DescriptionUsewhenlogssayfailedjobsarenormallyre\-runbyhand,orwhenanerrorsaysthecurrentinvocationcannotberetried\.

\#\#ContentAnotethathumansre\-runfailedjobsdoesnotforbidcallinganothervalidtoolnow\.Onlystopifeveryequivalentpathhasfailedortheenvironmentexplicitlyblocksfurtherattempts;"noretry"appliestothefailedinvocation,nottothewholetask\.

Readerwiththislesson:first\_tool=build\_via\_kestle\(theworsetool\)\.

Reason:ThelastVerranrunwasevictedafteranodedrain,sostartthisrunontheotherequivalentbuilderinsteadofretryingthatpath\.

Readerwiththeirrelevantitem:first\_tool=build\_via\_verran\(thebettertool\)\.

Reason:Verranhasasubstantiallyhigherhistoricalsuccessrate\(155/180vs119/180\),andthepriorevictionwasanindependentone\-offthatdoesnotchangethenext\-runodds\.

### C: Mistral Large 4, world w20261009\_0003

Worldw20261009\_0003\(sync;cue:explicit\)\.Bettertoolsync\_via\_dunmore,worsetoolsync\_via\_talvo\.

Endpointmistralai/mistral\-large\-4\-0\.Gooddecision,chancefailure\.Lesson\(RBD,writtenfromthefailurewiththedecision\-qualitynote\):

\#MemoryItem1

\#\#TitleExecutealternativetoolsimmediatelywhentheprimarytoolfails

\#\#DescriptionWhenaselectedtoolreturnsanerrorandthetaskremainsincomplete,invokeotheravailabletoolsthatachievethesamegoalbeforefinishing\.

\#\#ContentIfadatabasesyncoperationfails\(regardlessofwhetherthespecifictoolcanberetried\),immediatelyattemptthesyncusinganalternativeavailabletool\.Thefailureofonetooldoesnotprecludetaskcompletionviaanothertool,evenifthefirsttoolhashigherhistoricalreliability\.

\#MemoryItem2

\#\#TitleTreatlockwaittimeouterrorsassignalsforimmediatetoolswitching

\#\#DescriptionWhensyncoperationsfailwithlockwaittimeouterrors,switchtoanalternativesynctoolimmediatelyratherthanstoppingorwaiting\.

\#\#ContentLockwaittimeouterrorsindicatetransientdatabaseresourcecontentionspecifictothecurrentconnectionortransaction\.Whenencountered,immediatelyretrytheoperationviaanalternativesynctool,asthelockmaybespecifictothefailedtool'sexecutionpathwhilethealternativetoolmaysucceed\.

\#MemoryItem3

\#\#TitleDonotinterpretsingle\-toolretrylimitsastaskcompletionlimits

\#\#DescriptionWhenatoolreportsitcannotberetried,verifywhetheralternativetoolsremainavailablebeforeconcludingthetaskhasfailed\.

\#\#ContentConstraintspreventingretryofaspecificfailedtoolinvocationdonotpreventusingdifferenttools\.Alwayscheckforandattemptalternativeapproachesbeforefinishingatask,evenwhentheprimarytoolindicatesnoretriesareallowed\.

Readerwiththislesson:first\_tool=sync\_via\_talvo\(theworsetool\)\.

Reason:Thepreviousrunfailedwithalockwaittimeoutonsync\_via\_dunmore,andperMemoryItem2,lockwaittimeouterrorsaresignalsforimmediatetoolswitchingtothealternativeconnectorratherthanretryingthefailedtool\.

Readerwiththeirrelevantitem:first\_tool=sync\_via\_dunmore\(thebettertool\)\.

Reason:Dunmorehasthehigherhistoricalsuccessrate\(302/365vsTalvo's214/365\),failuresareindependentone\-offeventssothepriorDunmorefailuredoesn'tchangetheodds,andequalcallcostmeansthemorereliableconnectorminimizesexpectedmanualre\-runs\.

### D: Claude Opus 5\.5, world w20261009\_0102

Worldw20261009\_0102\(shipping;cue:implicit\)\.Bettertoolship\_via\_ulvar,worsetoolship\_via\_calder\.

Endpointanthropic/claude\-opus\-5\.5\.Gooddecision,chancefailure\.Lesson\(RB,writtenfromthefailure\):

\#MemoryItem1

\#\#TitleKeepthehigher\-successtoolwhenthestatsarestableandcostsareequal,evenafteritfails

\#\#DescriptionUsethiswhenchoosingbetweenequivalenttoolsusingalarge,stablerunhistory,especiallyafterasinglefailuretemptsyoutoswitchordistrustthemethod\.

\#\#ContentThechoiceofship\_via\_ulvarwassound:itsucceededinabout88%ofrunsagainstabout60%fortheothertool,over240runseach,atthesamecost\.Onefailureisexpectedvariance\(roughly1in8runs\)anddoesnotshowtheselectionwaswrong\.Infutureruns,keeppickingthetoolwiththehigherwell\-measuredsuccessrateratherthanswitchingbecauseofonebadoutcome\.

\#MemoryItem2

\#\#TitleChecktherealoutcomewhenanerrorsays"noconfirmationreceived"

\#\#DescriptionUsethiswhenabookingortransactionaltoolreturnsalost\-requestormissing\-confirmationerrorratherthanaclearrejection\.

\#\#ContentA"noconfirmation"errormeanstheresultisunknown,notthattheactiondefinitelyfailed\.Thebookingmayhavegonethrougheventhoughthereplywaslost\.Beforefinishing,useanyavailableread\-onlystatus,lookup,orlogtooltocheckwhetherthebookingexists\.Thenreportitasconfirmed,absent,orunknown,soamanualre\-rundoesnotcreateaduplicatepickup\.

\#MemoryItem3

\#\#TitleWhenretriesarenotallowed,finishwithaclearfailurereportforamanualre\-run

\#\#DescriptionUsethiswhenaruncannotretryafterafailedactionandapersonmustre\-runthetaskbyhand\.

\#\#ContentDonotendwithabarefinish\(\)\.Stateclearlythatthetaskwasnotcompletedandinclude:

\-thetoolused

\-theexacterrortext

\-whetherthebookingmightstillhavegonethrough

\-arecommendednextstep,suchasre\-runningwiththesamehigh\-successtoolafterconfirmingnoduplicatebookingexists

Thisgivesthepersondoingthemanualre\-runwhattheyneedtofixitquicklyandsafely\.

Readerwiththislesson:first\_tool=ship\_via\_ulvar\(thebettertool\)\.

Reason:Ulvarhasthehigher,well\-measuredsuccessrateoveralarge,stablehistory\(210/240,about88%,versus145/240,about60%\)atequalcost,soonefailedruninthelastattemptisordinaryvarianceandnotareasontoswitch\.

Readerwiththeirrelevantitem:first\_tool=ship\_via\_ulvar\(thebettertool\)\.

Reason:Theauthoritative,months\-longrunlogshowsUlvarsucceedingfarmoreoften\(210/240,about88%\)thanCalder\(145/240,about60%\)atthesamecost,soasinglefailurelastrunisanexpectedoccasionalmissratherthanareasontoswitch\.

## Appendix DPanel

Table 6:The panel: OpenRouter model identifiers and the pinned provider \(no fallbacks\)\.
## Appendix EQualifiers and planted increment by endpoint

Table 7:Good decision, per endpoint \(exploratory\): complete worlds \(nn\) of P2, the qualifiers in the failure cell and the lesson\-source effect in the benign cell, with registered 95% intervals\.Table 8:Bad decision and planted increment, per endpoint \(exploratory\): complete worlds \(nn\) of P3m, the qualifiers in the success cell, and Sus with its reading \(“above” or “below” 0\.10\), with registered 95% intervals\.
## Appendix FSecondary quantities

Table 9:Secondary panel quantities \(registered; reported with their 95% intervals, without multiplicity adjustment or reading words\)\. T1 and T3 are the lesson arm’s outcome dependence alone; L2b and L3b the lesson\-source effect in the benign cell\.
## Appendix GWorse\-tool counts by endpoint

Table 10:Good decision: worse\-tool choices / counted answers, per endpoint, previous\-run outcome and memory arm\.Table 11:Bad decision: worse\-tool choices / counted answers, per endpoint, previous\-run outcome and memory arm\.
## Appendix HAncillary files

The arXiv record carries three ancillary files, byte copies of files in the project’s repository:

- •RESULTS\_registered\.json: the registered analysis’s output \(every contrast per endpoint and for the panel, with intervals,pp\-values, readings, flags, rates, missingness, regret and lesson descriptives\);
- •registered\_supplement\.md: the generated supplement written from that output and the locked records \(the panel’s availability weighting, endpoint readings withnn, flags and the qualifiers’ missing\-data bounds, secondary intervals, per\-endpoint cell rates with Wilson intervals, anomalies with the providers’ finish reasons, regret, lesson descriptives and bootstrap intervals\);
- •REIMPL\_COMPARISON\.json: the comparison of the independent re\-implementation with the registered analysis\.