Local Edits, Global Ripples: Replay-Informed Policy Adaptation for Workflow Synthesis

# Local Edits, Global Ripples: Replay-Informed Policy Adaptation for Workflow Synthesis
Source: [https://arxiv.org/html/2609.12127](https://arxiv.org/html/2609.12127)
Manqing Mao, Hong Wang, Samson Koelle, Jie Yuan, Zhuoer Wang, James Feng,Affiliation:Yanjun Lin, Daniel Edmiston, Nikki Lijing Kuang, Zhecheng Sheng, Wei NiuAffiliation:Amazon, Inc\.Affiliation:\{manqingm, niuwei\}@amazon\.com

###### Abstract

Prompt\-policy editing offers a practical way to improve agents that synthesize executable workflows without updating the underlying model\. However, persistent prompt editing hastwo coupled properties\. First,*edit locality does not imply effect locality*: an edit confined to one predefined policy segment can ripple through downstream execution, altering behavior far beyond the edited segment\. Second,*edit effects are composition\-sensitive*: edits that work in isolation can interfere after composition, causing one or both to lose their benefit or become harmful\. Persistent adaptation must therefore supporttwo distinct decisions: identifying*where*the policy should change from execution feedback, and determining*whether*the resulting edit remains safe to persist after composition\. To address these challenges, we introduceRIPPLE\(Replay\-InformedPersistentPolicyLocalization andEditing\), which addresses these two properties by separating where an edit is made from whether it remains safe after composition\. It diagnoses failed trajectories, maps each actionable failure to a predefined policy segment, and restricts the correction to that part of the policy\. RIPPLE then evaluates candidates against the same iteration\-start policy to compare their isolated gains, before replaying promising edits after previously accepted updates to expose downstream effects and interactions\. Only edits that remain safe under composition are retained\. We evaluate RIPPLE onFlow\-HO, a synthetic held\-out benchmark for executable workflow synthesis\. RIPPLE improves validation success by up to23\.1%23\.1\\%and yields positive gains on two additional frozen language\-model backbones, while maintaining strong edit efficiency and low execution cost\. Targeted interaction analysis further demonstrates both properties:a segment\-local tool\-use edit changes downstream resource resolution and validation, while an edit beneficial in isolation becomes harmful after composition\.

## 1Introduction

LLM agents are increasingly tasked with producing executable artifacts, such as workflow JSON, infrastructure configurations, and service\-integrated automations\. Unlike free\-form responses, these artifacts must satisfy an execution contract: they must conform to a schema, implement the requested behavior, resolve environment\-specific resources, and pass external validation\. A workflow can therefore appear semantically plausible but remain undeployable because one field is invalid, one transition is broken, one identifier is stale, or one repair is incomplete\.

For frozen\-model agents, persistent improvement can target the agent’s*prompt policy*: the instructions that govern requirement interpretation, planning, tool use, editing, validation, and repair\. Such edits can be reviewed or rolled back without changing weights\. However, prompt editing has two coupled properties:First, edit locality does not imply effect locality:a local instruction change can ripple through execution, redirecting tool calls, altering resource resolution and workflow structure, and changing validation or repair behavior\([Cohen et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib37);[Sclar et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib38)\)\.Second, edits can interact under composition:edits that are beneficial in isolation may interfere once combined, weakening or even reversing one another’s gains\. Prompt adaptation is therefore a sequential policy\-composition problem, not one\-shot selection: isolated gains are insufficient evidence for persistence\.

Existing methods provide useful mechanisms for proposing and selecting prompt updates\. Prompt optimizers search over candidate instructions\([Zhou et al\., 2023](https://arxiv.org/html/2609.12127#bib.bib1);[Yang et al\., 2024a](https://arxiv.org/html/2609.12127#bib.bib2);[Guo et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib3);[Agrawal et al\., 2025](https://arxiv.org/html/2609.12127#bib.bib4);[Yuksekgonul et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib11)\), while recent prompt\- and skill\-evolution systems use held\-out evidence to decide which updates should persist\([Yang et al\., 2026](https://arxiv.org/html/2609.12127#bib.bib6);[Ma et al\., 2026](https://arxiv.org/html/2609.12127#bib.bib7);[Moll et al\., 2026](https://arxiv.org/html/2609.12127#bib.bib8);[Shawn, 2026](https://arxiv.org/html/2609.12127#bib.bib9)\)\. However, these mechanisms do not jointly resolve two challenges for executable agents\.First, execution evidence must be localized into an edit\.A low reward or validation failure identifies an unsuccessful execution, but not the responsible behavior or instruction\. Failures in clarification, tool use, schema construction, edit locality, and artifact completeness implicate different prompt segments\.Second, an edit must be evaluated in the policy state where it will persist\.An edit that improves the base policy can become redundant or harmful after previous updates are composed\.*The relevant criterion is therefore not whether an edit works in isolation, but whether it remains useful and safe after composition with the evolving shared policy\.*

We introduceRIPPLE\(Replay\-InformedPersistentPolicyLocalization andEditing\) to address this question\. The name reflects both properties above: a local policy edit canripple through the agent’s execution flow, whilethe direction of that ripple can change as the policy evolves\. RIPPLE decomposes adaptation into three decisions\. For*what failed*, it assigns trajectory\-grounded labels with a deterministic taxonomy\. For*where to edit*, it maps each actionable diagnosis to a predefined prompt segment and instantiates a bounded patch from a versioned library, constraining the locus of the policy change\. For*whether to persist*, it ranks candidates by local gain and promotes them sequentially, testing whether each local change remains safe after composition\. After each acceptance, RIPPLE evaluates the next candidate on top of all previously accepted edits, so downstream interactions are evaluated in the policy state where the candidate would actually persist\. The model, tools, and execution environment remain fixed, while accepted patches form auditable, rollbackable checkpoints\.

We evaluate RIPPLE onFlow\-HO, a synthetic held\-out suite for generating executable workflows represented as JSON\. RIPPLE consistently improves validation success and composite reward across multiple frozen language models while maintaining strong edit efficiency and low execution cost\. Ablations show that replay design and patch composition materially shape the resulting policy, while targeted interaction analysis reveals that local gains can reverse after composition\. Together, these results support both aspects of the problem: local edits can have non\-local effects, and their value depends on the evolving policy context\. In summary, our contributions are:

1. 1\.Failure\-localized policy editing\.We formulate frozen\-agent adaptation over a segmented prompt policy, mapping trajectory\-grounded diagnoses from a predefined failure taxonomy to target policy segments and bounded corrective edits\.
2. 2\.Composition\-aware persistence\.We introduce a replay\-gated promotion procedure that separates local utility from compositional safety by evaluating each candidate against the advancing policy formed by previously accepted edits under separate reward and correctness constraints\.
3. 3\.Flow\-HO evaluation and interaction analysis\.We construct Flow\-HO and evaluate RIPPLE across effectiveness, replay design, composition safety, and promotion\-rule trade\-offs, showing substantial held\-out gains and exposing failure modes that arise only through patch interaction\.

## 2Persistent Prompt\-Policy Adaptation

### 2\.1Problem Setup

##### Frozen executable\-agent policy\.

We study closed\-schema executable workflow synthesis with a frozen, tool\-using LLM agent\. Each taskxxconsists of a natural\-language requestqqand an initial workflowy0y\_\{0\}, wherey0=∅y\_\{0\}=\\varnothingfor generation tasks:

x=\(q,y0\),τ∼pθ\(τ∣x,π\),θfixed\.\\ x=\(q,y\_\{0\}\),\\qquad\\tau\\sim p\_\{\\theta\}\(\\tau\\mid x,\\pi\),\\qquad\\theta\\text\{ fixed\}\.\\\(1\)The prompt policyπ\\pigoverns the agent’s multi\-turn interaction with tools and an external validator\. Onlyπ\\piis adapted; the model parameters, tool interface, and execution environment remain fixed\.

A completed trajectoryτ\\taureceives a bounded composite reward:

R⁡\(τ,x\)=ρ⁡\(S⁡\(τ\),C⁡\(τ\),E⁡\(τ\),K⁡\(τ\)\),\\ R\(\\tau;x\)=\\rho\\\!\\left\(S\(\\tau\),C\(\\tau\),E\(\\tau\),K\(\\tau\)\\right\),\\\(2\)whereSSdenotes validation success,CCis workflow correctness,EEis edit efficiency, andKKis normalized execution cost\. The exact task\-specific definitions are given in Appendix[B\.3\.4](https://arxiv.org/html/2609.12127#A2.SS3.SSS4)\. Because workflow correctness is also constrained separately during promotion, we retain bothRRandCCas policy\-level metrics\. ForX∈\{R,C\}X\\in\\\{R,C\\\}, define the expected metric on taskxxas

vX\(π;x\)=𝔼τ∼pθ\(⋅∣x,π\)\[X\(τ;x\)\]\.\\ v\_\{X\}\(\\pi;x\)=\\mathbb\{E\}\_\{\\tau\\sim p\_\{\\theta\}\(\\cdot\\mid x,\\pi\)\}\\left\[X\(\\tau;x\)\\right\]\.\\\(3\)Given a task pool𝒟\\mathcal\{D\}and an aggregation rulegg, the pool\-level score of policyπ\\pion metricXXis

VXg​\(π,𝒟\)=g⁡\(\(vX​\(π,x\)\)x∈𝒟\)\.\\ V\_\{X\}^\{g\}\(\\pi;\\mathcal\{D\}\)=g\\\!\\left\(\\big\(v\_\{X\}\(\\pi;x\)\\big\)\_\{x\\in\\mathcal\{D\}\}\\right\)\.\\\(4\)Section[3\.3](https://arxiv.org/html/2609.12127#S3.SS3)instantiatesggas either a uniform mean or a macro\-average over workflow families\.

##### Ordered segment\-level updates\.

The prompt policy is partitioned into a fixed set of behavioral segments𝒮\\mathcal\{S\}\. Letℱ\\mathcal\{F\}denote a failure taxonomy andℒ\\mathcal\{L\}a versioned library of bounded patches, where eacha∈ℒa\\in\\mathcal\{L\}is typed by a failure familyf⁡\(a\)∈ℱf\(a\)\\in\\mathcal\{F\}and a target segments⁡\(a\)∈𝒮s\(a\)\\in\\mathcal\{S\}\.

An adaptation is an ordered sequence of accepted patches:

P=\(a1,…,at\),aj∈ℒ,πP=π0⊕P\.\\ P=\(a\_\{1\},\\ldots,a\_\{t\}\),\\qquad a\_\{j\}\\in\\mathcal\{L\},\\qquad\\pi\_\{P\}=\\pi\_\{0\}\\oplus P\.\\\(5\)The operator⊕\\oplusapplies patches to their designated segments in acceptance order\. Order matters because an edit may reinforce, override, or conflict with instructions already present in the policy\.

For a candidate patchaa, an accepted prefixPP, metricXX, and task pool𝒟\\mathcal\{D\}, we define the prefix\-conditioned marginal effect as:

ΔXg​\(a∣P;𝒟\)=VXg​\(π0⊕P⊕a,𝒟\)−VXg​\(π0⊕P,𝒟\)\.\\Delta\_\{X\}^\{g\}\(a\\mid P;\\mathcal\{D\}\)=V\_\{X\}^\{g\}\(\\pi\_\{0\}\\oplus P\\oplus a;\\mathcal\{D\}\)\-V\_\{X\}^\{g\}\(\\pi\_\{0\}\\oplus P;\\mathcal\{D\}\)\.\(6\)Thus, the value of a persistent edit depends on both the patchaaand the policy prefixPP\.

##### Persistent\-editing objective\.

Given a training pool𝒟train\\mathcal\{D\}\_\{\\mathrm\{train\}\}, a replay pool𝒟replay\\mathcal\{D\}\_\{\\mathrm\{replay\}\}, and a patch budgetHH, the ideal adaptation objective is:

P⋆∈arg⁡maxP=\(a1,…,at\),t≤H\\displaystyle P^\{\\star\}\\in\\arg\\max\_\{P=\(a\_\{1\},\\ldots,a\_\{t\}\),\\,t\\leq H\}VR​\(π0⊕P,𝒟train\)\\displaystyle V\_\{R\}\(\\pi\_\{0\}\\oplus P;\\mathcal\{D\}\_\{\\mathrm\{train\}\}\)\(7\)s\.t\.\\displaystyle\\text\{s\.t\.\}ΔRg​\(aj∣P<j;𝒟replay\)≥εR,\\displaystyle\\Delta\_\{R\}^\{g\}\(a\_\{j\}\\mid P^\{<j\};\\mathcal\{D\}\_\{\\mathrm\{replay\}\}\)\\geq\\varepsilon\_\{R\},ΔCg\(aj∣P<j;𝒟replay\)≥εC,j=1,…,t,\\displaystyle\\Delta\_\{C\}^\{g\}\(a\_\{j\}\\mid P^\{<j\};\\mathcal\{D\}\_\{\\mathrm\{replay\}\}\)\\geq\\varepsilon\_\{C\},\\qquad j=1,\\ldots,t,whereP<j=\(a1,…,aj−1\)P^\{<j\}=\(a\_\{1\},\\ldots,a\_\{j\-1\}\)\. The training objective favors patch sequences that improve performance on the training pool\. Before a patch is committed, it must also satisfy reward and correctness thresholds on the replay pool when evaluated on top of all previously accepted patches\. Small negative replay deltas is allowed to avoid rejecting potentially useful edits due to rollout variation, but these thresholds are empirical safeguards rather than guarantees on unseen requests\.

Solving Eqn\.[7](https://arxiv.org/html/2609.12127#S2.E7)requires to evaluate all ordered subsets of patches up to sizeHH\. Exhaustive search is prohibitively expensive since each evaluation requires new tool\-calling rollouts\. Section[3](https://arxiv.org/html/2609.12127#S3)therefore approximates this objective with a localized candidate set and a greedy sequential evaluation\.

### 2\.2Two Decisions in Persistent Policy Editing

Eqn\.[7](https://arxiv.org/html/2609.12127#S2.E7)defines the objective and constraints for a complete patch sequence, but not how to choose the next patch\. At each step, two decisions remain: which edit should be proposed from a failed execution, and whether that edit should persist after earlier edits have been accepted\.

##### Decision 1: Which edit should be proposed?

A low reward or validation failure shows that an execution was unsuccessful, but it does not identify the responsible behavior or the policy segment that should change\. The same scalar outcome may arise from different failures and therefore require different edits\. Candidate proposal must instead be grounded in execution evidence:

z⁡\(τ\)⟶f⟶s⟶a∈ℒf,s,\\ z\(\\tau\)\\longrightarrow f\\longrightarrow s\\longrightarrow a\\in\\mathcal\{L\}\_\{f,s\},\\\(8\)wherez⁡\(τ\)z\(\\tau\)denotes trajectory evidence,ffa diagnosed failure family,ssthe target policy segment, andaaa bounded patch\. This decision reduces open\-ended prompt rewriting to a localized, typed update\.

##### Decision 2: Should the edit persist?

An edit that improves the initial policy may become redundant or harmful once combined with previously accepted edits\. The relevant quantity is therefore its marginal effect on the current policy,ΔXg​\(a∣P;𝒟\)\\Delta\_\{X\}^\{g\}\(a\\mid P;\\mathcal\{D\}\), rather than its isolated effect onπ0\\pi\_\{0\}\. Composite reward creates a separate risk: gains in validation, edit locality, or execution efficiency can offset a decline in workflow correctness\. RIPPLE therefore evaluates reward and correctness separately before committing an edit\.

These two decisions define RIPPLE’s structure: proposing an edit is decomposed into identifying*what*failed and*where*to edit, while replay determines*whether*the resulting patch should persist\.

## 3RIPPLE: Localized and Replay\-Gated Policy Editing

RIPPLE approximates the constrained patch\-sequence objective in Eqn\.[7](https://arxiv.org/html/2609.12127#S2.E7)through greedy sequential policy improvement over the patch library\. Its design separates two scopes of adaptation: localization constrains where a policy change is introduced, while advancing replay evaluates whether the resulting behavioral effects remain safe after composition\. At iterationkk, training rollouts are used to diagnose failures, construct localized candidate patches, and rank them by isolated gain\. Replay rollouts then evaluate each candidate on top of the accepted prefix before commitment\. The final evaluation set remains outside the adaptation loop\. Figure[1](https://arxiv.org/html/2609.12127#S3.F1)summarizes one adaptation iteration\. Sections[3\.1](https://arxiv.org/html/2609.12127#S3.SS1)–[3\.2](https://arxiv.org/html/2609.12127#S3.SS2)describe candidate construction, Section[3\.3](https://arxiv.org/html/2609.12127#S3.SS3)describes replay\-gated promotion, and Section[3\.4](https://arxiv.org/html/2609.12127#S3.SS4)describes the outer loop\. The full multi\-iteration procedure is given in Appendix[B\.3\.3](https://arxiv.org/html/2609.12127#A2.SS3.SSS3)\.

![Refer to caption](https://arxiv.org/html/2609.12127v1/figures/framework.png)Figure 1:Overall framework of RIPPLE\.At iterationkk, RIPPLE \(i\) selects informative training rollouts and diagnoses failures with a predefined taxonomy \(WHAT failed\); \(ii\) localizes each diagnosis to a policy segment and instantiates bounded patches from a versioned libraryℒ\\mathcal\{L\}\(WHERE to edit\); and \(iii\) ranks candidates by training gain and sequentially evaluates them through a two\-signal replay gate against the advancing policy \(WHETHER to persist\)\. Accepted patches form the next policyπk\+1\\pi\_\{k\+1\}\. Flow\-HO is reserved for final evaluation\.### 3\.1What Failed: Rollout Generation and Behavioral Diagnosis

##### Rollout generation\.

Ford∈𝒟traind\\in\\mathcal\{D\}\_\{\\mathrm\{train\}\}, RIPPLE samplesN=3N=3trajectories under the current policy:

τd,n\(k\)∼pθ\(⋅∣d,πk\),n=1,…,N\.\\ \\tau\_\{d,n\}^\{\(k\)\}\\sim p\_\{\\theta\}\(\\cdot\\mid d,\\pi\_\{k\}\),\\qquad n=1,\\ldots,N\.\\\(9\)Each rollout records the interaction trace, generated workflow, validator feedback, and execution cost\.

##### Verifiable reward\.

Each rollout receives a scalar programmatic reward that is a bounded weighted combination of four signals\. The weighting and task\-specific variants are given in Appendix[B\.3\.4](https://arxiv.org/html/2609.12127#A2.SS3.SSS4):

R⁡\(τ\)=f⁡\(S⁡\(τ\),C⁡\(τ\),E⁡\(τ\),K⁡\(τ\)\),\\ R\(\\tau\)=f\\big\(S\(\\tau\),C\(\\tau\),E\(\\tau\),K\(\\tau\)\\big\),\\\(10\)whereS∈\{0,1\}S\\in\\\{0,1\\\}is service validation success \(whether the generated workflow is accepted by the service validator\),C∈\[0,1\]C\\in\[0,1\]is workflow correctness \(whether the workflow implements the intended structure and requested modification\),E∈\[0,1\]E\\in\[0,1\]is edit efficiency \(whether the modification remains within the declared scope\), andK∈\[0,1\]K\\in\[0,1\]is normalized execution cost\. For each datapointddand policyπ\\pi, RIPPLE estimatesvX​\(π,d\)v\_\{X\}\(\\pi;d\)fromNNindependent rollouts:

X¯π​\(d\)=1N​∑n=1NX⁡\(τd,nπ\),X∈\{R,C\}\.\\bar\{X\}\_\{\\pi\}\(d\)=\\frac\{1\}\{N\}\\sum\_\{n=1\}^\{N\}X\\left\(\\tau\_\{d,n\}^\{\\pi\}\\right\),\\qquad X\\in\\\{R,C\\\}\.\(11\)whereτd,nπ\\tau\_\{d,n\}^\{\\pi\}is thenn\-th rollout of policyπ\\pion datapointdd\. The resulting per\-datapoint means are then combined into a pool\-level score by the aggregation rulegg\.

##### Informative\-trajectory selection\.

RIPPLE prioritizes for diagnosis the training datapoints with the highest rollout\-level reward variance under the current policy\. For eachd∈𝒟traind\\in\\mathcal\{D\}\_\{\\mathrm\{train\}\}, it computes:

vk​\(d\)=Varn=1,…,N⁡\[R⁡\(τd,n\(k\)\)\]\.\\ v\_\{k\}\(d\)=\\operatorname\{Var\}\_\{n=1,\\ldots,N\}\\left\[R\\\!\\left\(\\tau\_\{d,n\}^\{\(k\)\}\\right\)\\right\]\.\\\(12\)It then selects theBdiagB\_\{\\mathrm\{diag\}\}datapoints with the highest reward variance and diagnoses the lowest\-reward rollout from each selected datapoint\. High variance indicates inconsistent behavior under the current policy, while the lowest\-reward rollout provides a concrete failure trace for diagnosis\.

τd−=arg⁡minn∈\{1,…,N\}⁡R⁡\(τd,n\(k\)\)\.\\ \\tau\_\{d\}^\{\-\}=\\arg\\min\_\{n\\in\\\{1,\\ldots,N\\\}\}R\\\!\\left\(\\tau\_\{d,n\}^\{\(k\)\}\\right\)\.\\\(13\)

##### Predicate\-based behavioral diagnosis\.

For each selected trajectoryτ\\tau, RIPPLE applies predefined rules to the execution log, checking for signals such as missing clarification, failed tool use, validator errors, ineffective repair, out\-of\-scope changes, and incomplete output\. The rules return the actionable failure families and their supporting evidence, without an LLM call:

Diagnose⁡\(τ\)=\(ℱ⁡\(τ\),z⁡\(τ\)\),ℱ⁡\(τ\)⊆ℱ,\\ \\operatorname\{Diagnose\}\(\\tau\)=\\bigl\(\\mathcal\{F\}\(\\tau\),z\(\\tau\)\\bigr\),\\qquad\\mathcal\{F\}\(\\tau\)\\subseteq\\mathcal\{F\},\\\(14\)wherez⁡\(τ\)z\(\\tau\)records the triggered predicates and supporting trace evidence, andℱ⁡\(τ\)\\mathcal\{F\}\(\\tau\)contains the actionable failure families supported by that evidence\. Multiple families may be activated because an upstream error can produce several downstream symptoms, or because distinct failures co\-occur in the same trajectory\. For attribution, fixed precedence rules designate one activated family as primary, preferring direct and causally earlier evidence; the remaining actionable labels are retained and may seed additional candidate patches\. Full predicates and precedence rules are given in Appendix[B\.3\.1](https://arxiv.org/html/2609.12127#A2.SS3.SSS1)\.

RIPPLE uses six failure families:F1 Clarification\(missing required information\),F2 Tool use\(missing, incorrect, or misordered tool calls\),F3 Schema\(schema or service\-constraint violations\),F4 Repair behavior\(failure to act on validator feedback\),F5 Edit locality\(out\-of\-scope modifications\), andF6 Completeness\(partial or unresolved artifacts\)\. The primary label captures the earliest causal failure, while secondary labels retain additional actionable symptoms\. When multiple labels are plausible, RIPPLE prefers the earliest causal failure in the execution trace over the final visible symptom\. Full disambiguation rules and the evidence schema are provided in Appendix[B\.3\.1](https://arxiv.org/html/2609.12127#A2.SS3.SSS1)\.

### 3\.2Where to Edit: Segment\-Typed Patch Construction

##### Policy segmentation\.

Before adaptation begins, the base prompt is partitioned into seven behavioral segments:REQ\_UNDERSTANDING,CLARIFY,PLAN,TOOL\_USE,EDIT,VALIDATE, andFINAL\_OUTPUT\. Each segment governs a distinct stage of agent execution\. Holding boundaries fixed keeps localization consistent across iterations and gives every patch a stable and reviewable target\. RIPPLE adapts the policy only by appending bounded instructions to designated segments\.

##### Failure\-localized patch construction\.

For each selected trajectoryτ\\tauand each diagnosed failure familyf∈ℱ⁡\(τ\)f\\in\\mathcal\{F\}\(\\tau\), RIPPLE deterministically selects a target segmentssand retrieves a patchaa:

s=Localize⁡\(f,z⁡\(τ\)\),a=Retrieve⁡\(ℒ,f,s\)\.\\ s=\\operatorname\{Localize\}\\\!\\left\(f,z\(\\tau\)\\right\),\\qquad a=\\operatorname\{Retrieve\}\\\!\\left\(\\mathcal\{L\},f,s\\right\)\.\\\(15\)The patch libraryℒ\\mathcal\{L\}contains predefined entries indexed by failure family and target segment\. Given a diagnosed failurefflocalized to segmentss, RIPPLE first retrieves the entry matching the\(f,s\)\(f,s\)pair\. If no exact match exists, it falls back to a generic entry targeting the same segment\. The retrieved instruction is appended verbatim toss; no LLM is invoked\. Each library entry stores a patch ID, failure family, target segment, and instruction, making the procedure deterministic and reproducible\.

##### Candidate aggregation and coverage ordering\.

Let𝒫⁡\(τ\)\\mathcal\{P\}\(\\tau\)denote the set of patches nominated by a selected trajectoryτ\\tau\. The support count of patchaaat iterationkkis

ck\(a\)=∑τ∈𝒯k\[a∈𝒫\(τ\)\],\\ c\_\{k\}\(a\)=\\sum\_\{\\tau\\in\\mathcal\{T\}\_\{k\}\}\\mathbf\{1\}\\\!\\left\[a\\in\\mathcal\{P\}\(\\tau\)\\right\],\\\(16\)where𝒯k\\mathcal\{T\}\_\{k\}denotes the trajectories selected for diagnosis\. The countck​\(a\)c\_\{k\}\(a\)measures how many independent failure traces nominate patchaa, so a larger count indicates stronger support for that update\. RIPPLE merges nominations by patch ID, removes previously accepted patches, and ranks candidates within each failure family byck​\(a\)c\_\{k\}\(a\)\. It then applies a coverage\-first round robin, allowing each represented family to contribute one candidate before any family contributes a second\. The resulting ordered candidate set proceeds to training\-side evaluation\.

### 3\.3Whether to Persist: Policy Improvement and Replay\-Constrained Promotion

RIPPLE separates candidate ranking from commitment\. Training\-side pre\-qualification evaluates every patch from the same iteration\-start policyπk\\pi\_\{k\}, making their one\-step gains directly comparable\. Replay\-side promotion then evaluates candidates sequentially on top of previously accepted edits, testing each patch in the policy state where it would actually persist\. This separation preserves fair ranking while testing whether a local edit’s downstream effects remain safe after composition\.

##### Training\-side policy improvement\.

Each candidate patcha∈𝒜ka\\in\\mathcal\{A\}\_\{k\}defines a candidate policyπka=πk⊕a\\pi\_\{k\}^\{a\}=\\pi\_\{k\}\\oplus a\. To compare candidates on a common basis, RIPPLE evaluates every patch against the same iteration\-start policyπk\\pi\_\{k\}, isolating its one\-step effect from interactions with other candidates\. The resulting one\-step training gain is:

Δtrain​\(a\)=V^R​\(πka,𝒟train\)−V^R​\(πk,𝒟train\),\\ \\Delta\_\{\\mathrm\{train\}\}\(a\)=\\widehat\{V\}\_\{R\}\(\\pi\_\{k\}^\{a\};\\mathcal\{D\}\_\{\\mathrm\{train\}\}\)\-\\widehat\{V\}\_\{R\}\(\\pi\_\{k\};\\mathcal\{D\}\_\{\\mathrm\{train\}\}\),\\\(17\)whereV^R​\(π,𝒟\)=\|𝒟\|−1​∑d∈𝒟R¯π​\(d\)\\widehat\{V\}\_\{R\}\(\\pi;\\mathcal\{D\}\)=\|\\mathcal\{D\}\|^\{\-1\}\\sum\_\{d\\in\\mathcal\{D\}\}\\bar\{R\}\_\{\\pi\}\(d\)is the empirical mean reward over the task pool\. Only positive\-gain candidates advance:𝒜k\+=\{a∈𝒜k:Δtrain​\(a\)\>0\}\\mathcal\{A\}\_\{k\}^\{\+\}=\\\{a\\in\\mathcal\{A\}\_\{k\}:\\Delta\_\{\\mathrm\{train\}\}\(a\)\>0\\\}, and RIPPLE sorts them by decreasingΔtrain​\(a\)\\Delta\_\{\\mathrm\{train\}\}\(a\)\. This stage identifies promising edits and determines their evaluation order; replay\-side promotion decides whether they remain safe after composition\.

##### Replay\-side policy promotion\.

After sorting𝒜k\+\\mathcal\{A\}\_\{k\}^\{\+\}by decreasingΔtrain\\Delta\_\{\\mathrm\{train\}\}, letaja\_\{j\}denote thejj\-th candidate patch in this order\. LetPk<jP\_\{k\}^\{<j\}denote the ordered sequence of candidates accepted among the firstj−1j\-1positions\. The policy used to evaluateaja\_\{j\}and its candidate extension areπk<j=πk⊕Pk<j\\pi\_\{k\}^\{<j\}=\\pi\_\{k\}\\oplus P\_\{k\}^\{<j\}andπ~k,j=πk<j⊕aj\\tilde\{\\pi\}\_\{k,j\}=\\pi\_\{k\}^\{<j\}\\oplus a\_\{j\}\. RIPPLE first averages theN=3N=3rollouts for each replay datapoint\. It then computes a single replay score from these per\-datapoint means using one of two weighting schemes:

- •Uniform aggregationguniformg\_\{\\mathrm\{uniform\}\}: all replay datapoints receive equal weight;
- •GT\-family\-balanced aggregationgbalancedg\_\{\\mathrm\{balanced\}\}\(*coverage\-weighted*in labels\): datapoints are averaged within each ground\-truth workflow family, then equally across represented families \(Appendix[C\.2](https://arxiv.org/html/2609.12127#A3.SS2)\)\.

Candidateaja\_\{j\}is evaluated marginally against the latest accepted policy:

ΔRg​\(aj∣Pk<j\)=g⁡\[R¯π~k,j\]−g⁡\[R¯πk<j\],ΔCg​\(aj∣Pk<j\)=g⁡\[C¯π~k,j\]−g⁡\[C¯πk<j\]\.\\ \\Delta\_\{R\}^\{g\}\(a\_\{j\}\\mid P\_\{k\}^\{<j\}\)=g\\\!\\left\[\\bar\{R\}\_\{\\tilde\{\\pi\}\_\{k,j\}\}\\right\]\-g\\\!\\left\[\\bar\{R\}\_\{\\pi\_\{k\}^\{<j\}\}\\right\],\\qquad\\Delta\_\{C\}^\{g\}\(a\_\{j\}\\mid P\_\{k\}^\{<j\}\)=g\\\!\\left\[\\bar\{C\}\_\{\\tilde\{\\pi\}\_\{k,j\}\}\\right\]\-g\\\!\\left\[\\bar\{C\}\_\{\\pi\_\{k\}^\{<j\}\}\\right\]\.\\\(18\)Candidateaja\_\{j\}is accepted only when both replay deltas satisfy their tolerances:

Accept⁡\(aj∣Pk<j\)⇔ΔRg​\(aj∣Pk<j\)≥εR∧ΔCg​\(aj∣Pk<j\)≥εC\.\\ \\operatorname\{Accept\}\(a\_\{j\}\\mid P\_\{k\}^\{<j\}\)\\iff\\Delta\_\{R\}^\{g\}\(a\_\{j\}\\mid P\_\{k\}^\{<j\}\)\\geq\\varepsilon\_\{R\}\\ \\land\\ \\Delta\_\{C\}^\{g\}\(a\_\{j\}\\mid P\_\{k\}^\{<j\}\)\\geq\\varepsilon\_\{C\}\.\\\(19\)We useεR=−0\.05\\varepsilon\_\{R\}=\-0\.05andεC=−0\.10\\varepsilon\_\{C\}=\-0\.10\. Ifaja\_\{j\}is accepted,π~k,j\\tilde\{\\pi\}\_\{k,j\}becomes the policy used to evaluate the next candidate; if rejected, the current policy remains unchanged\. Intuitively, each accepted patch changes the policy context in which later patches are judged\. Replay therefore evaluates each candidate against the current accepted policy, making downstream effects and interactions with earlier edits observable before persistence\. The gate constrains each patch’s replay regression rather than cumulative change fromπk\\pi\_\{k\}; because the tolerances are negative, small regressions are allowed and may accumulate across patches\. Checking correctness separately guards against patches that pass the reward threshold despite a loss in workflow correctness\. Appendix[C\.3](https://arxiv.org/html/2609.12127#A3.SS3)analyzes when this constraint binds, while Appendix[C\.1](https://arxiv.org/html/2609.12127#A3.SS1)discusses the AND rule and tolerance choices\.

### 3\.4Policy Iteration and Replay Maintenance

The patches accepted at iterationkkform the ordered sequencePk=\(ak,1,…,ak,mk\)P\_\{k\}=\(a\_\{k,1\},\\ldots,a\_\{k,m\_\{k\}\}\), and the next iteration starts fromπk\+1=πk⊕Pk\\pi\_\{k\+1\}=\\pi\_\{k\}\\oplus P\_\{k\}\. If no patch is accepted,Pk=∅P\_\{k\}=\\varnothingandπk\+1=πk\\pi\_\{k\+1\}=\\pi\_\{k\}\. Thus, each outer iteration carries forward only edits that pass replay\-side promotion\.

RIPPLE supports two replay modes: \(i\)Static replayalways uses𝒟replay=𝒟core\\mathcal\{D\}\_\{\\mathrm\{replay\}\}=\\mathcal\{D\}\_\{\\mathrm\{core\}\}; and \(ii\)Adaptive replayuses𝒟replay\(k\)=𝒟core∪𝒟hard\(k\)∪𝒟recent\(k\)\\mathcal\{D\}\_\{\\mathrm\{replay\}\}^\{\(k\)\}=\\mathcal\{D\}\_\{\\mathrm\{core\}\}\\cup\\mathcal\{D\}\_\{\\mathrm\{hard\}\}^\{\(k\)\}\\cup\\mathcal\{D\}\_\{\\mathrm\{recent\}\}^\{\(k\)\}\. After scoring the iteration\-kktraining rollouts and before promotion, RIPPLE refreshes the𝒟hard\(k\)\\mathcal\{D\}\_\{\\mathrm\{hard\}\}^\{\(k\)\}with persistently low\-performing examples and the𝒟recent\(k\)\\mathcal\{D\}\_\{\\mathrm\{recent\}\}^\{\(k\)\}with examples whose success status changed\. The fixed core provides a stable reference, the refreshed subsets track current failure modes; the resulting replay set gates that iteration’s candidates\.

An iteration with no accepted patch increments a patience counter; any acceptance resets it\. RIPPLE stops after two consecutive no\-update iterations\. Full algorithm is shown in Appendix[B\.3\.3](https://arxiv.org/html/2609.12127#A2.SS3.SSS3)\.

Figure 2:Cross\-backbone reward improvement on Flow\-HO modification\.\(a\)Per\-family pairedΔ​R=RRIPPLE−Rπ0\\Delta R=R\_\{\\mathrm\{RIPPLE\}\}\-R\_\{\\pi\_\{0\}\}: every backbone improves on at least three of four families, and the two regressions \(red\-outlined\) are backbone\-specific—no family regresses across all three models\.\(b\)Reward\-component deltas \(percentage points;KKnegated so up is better\): the gain is carried by service\-validation success \(Δ​S≥\+6\.8\\Delta S\\geq\+6\.8pp on every backbone;\+23\.1\+23\.1pp on Haiku\), while workflow correctness \(Δ​C\\Delta C\) is flat across backbones \(\|Δ​C\|≤0\.9\|\\Delta C\|\\leq 0\.9pp\)\. Haiku uses RIPPLE\-U; Gemma and Ministral use transfer runs withN=3N=3per task\.

## 4Experiments

In this section, we examine whether RIPPLE improves held\-out workflow quality across frozen policy backbones and whether replay supports safe edit persistence under iterative composition\. We first describe the experimental setup and then organize the evaluation for 4 research questions \(RQs\):

RQ1\. Cross\-backbone effectiveness\.Does RIPPLE improve held\-out workflow quality across frozen policy backbones with different capability levels?

RQ2\. Replay design\.How do aggregation, replay maintenance, gating, and iteration depth jointly affect patch acceptance, policy evolution, and the quality of the selected policy?

RQ3\. Composition safety\.Can the advancing replay baseline detect and reject edits that are beneficial in isolation but harmful after composition with previously accepted edits?

RQ4\. Promotion rules\.How do stricter acceptors affect policy composition and quality?

### 4\.1Experimental Setup

##### Flow\-HO\.

Flow\-HO is a synthetic held\-out benchmark of executable workflow tasks in a JSON\-based flow language\. Its primary split contains 39 workflow\-modification tasks, where the agent must modify a corrupted workflow according to a request and pass external validation\. The four ground\-truth families are disjoint from training and replay and are never used for adaptation or model selection\. Appendix[B\.1](https://arxiv.org/html/2609.12127#A2.SS1)provides details and the secondary workflow\-generation setting\.

##### Systems\.

Claude Haiku 4\.5 is the primary frozen policy backbone for RQ1–RQ4; Gemma 3\-12B\-it and Ministral 3\-14B provide two lower\-capability transfer settings \(Appendix B\.5\.1\)\. Transfer backbones evaluate only baseπ0\\pi\_\{0\}and RIPPLE\. No variant selection is performed on transfer backbones\. On Haiku, we compareπ0\\pi\_\{0\}with five RIPPLE variants:RIPPLE\-U\(uniform, adaptive replay\),RIPPLE\-S\(family\-balanced, static replay\),Coverage\-Adaptive\(family\-balanced, adaptive replay\),Train\-Only\(no replay promotion\), andMulti\-Iteration\. RQ1 refers to RIPPLE\-U simply asRIPPLE\. RQ4 additionally compares controlled SkillOpt\-style and SkillGen\-style acceptors with candidate trajectories held fixed\. Appendix[B\.2](https://arxiv.org/html/2609.12127#A2.SS2)provides adaptation and baseline details\.

##### Protocol\.

We useN=3N=3rollouts per datapoint and set the diagnosis budget toBdiag=15B\_\{\\mathrm\{diag\}\}=15\. We report service\-validation success, workflow correctness, and composite reward\. Haiku deltas are paired againstπ0\\pi\_\{0\}over the same 39 tasks, with10,00010\{,\}000family\-cluster bootstrap replicates; Gemma and Ministral use paired point estimates\. Edit efficiency, execution cost, model settings are reported in Appendices[B\.4\.1](https://arxiv.org/html/2609.12127#A2.SS4.SSS1)and[B\.3\.5](https://arxiv.org/html/2609.12127#A2.SS3.SSS5)\.

### 4\.2RQ1: Held\-Out Effectiveness across Frozen Backbones

Table[1](https://arxiv.org/html/2609.12127#S4.T1)reports the primary cross\-backbone comparison; the full Haiku design study appears in §[4\.3](https://arxiv.org/html/2609.12127#S4.SS3):

RIPPLE primarily improves deployability on the principal backbone\.On Haiku 4\.5, RIPPLE raises service\-validation success from54\.7%54\.7\\%to77\.8%77\.8\\%\(\+23\.1%\+23\.1\\%\) and composite reward from0\.5250\.525to0\.6440\.644; both paired family\-cluster intervals exclude zero\. Correctness rises by only0\.8%0\.8\\%and remains unresolved\. The main effect is a higher rate of workflows that survive service validation rather than a broad shift in structural similarity\.

The gain remains efficient and spans the benchmark families\.RIPPLE also attains the strongest edit efficiency and lowest execution cost among the harmonized systems\. Its mean reward exceeds the base policy in all four Flow\-HO families\. Appendix[B\.4\.1](https://arxiv.org/html/2609.12127#A2.SS4.SSS1)provides the family\-level results\.

Table 1:Flow\-HO modification results across different frozen backbones\. RIPPLE improves validation success and reward in all three settings\.†\\daggermarks a paired family\-cluster 95% interval excluding zero \(Appendix B\.3\.5\);∗marks the Ministral protocol \(Appendix B\.5\.5\)\. Best means are bold\.
##### Positive untuned gains transfer to both weaker backbones\.

RIPPLE raises service\-validation success by6\.86\.8pp on both Gemma and Ministral and improves reward on both, while correctness is unchanged on Gemma and slightly lower on Ministral\. Accepted patches are largely backbone\-specific, consistent with diagnosis adapting to distinct failure profiles \(Appendix[B\.5\.4](https://arxiv.org/html/2609.12127#A2.SS5.SSS4)\)\. Crucially, both runs reuse Haiku’s proposer, patch library, replay design, and gate tolerances without backbone\-specific tuning\.These positive gains therefore demonstrate untuned transfer, and the residual Haiku–transfer gap should not be interpreted as a method ceiling\.Ministral only bypasses the compute\-saving positive\-gain pre\-filter; replay gating is unchanged \(Appendix[B\.5\.5](https://arxiv.org/html/2609.12127#A2.SS5.SSS5)\)\. We treat these point estimates as transfer evidence rather than a precise cross\-backbone ranking\.

##### Gains are broad across families and driven by service validation\.

Figure[2](https://arxiv.org/html/2609.12127#S3.F2)\(a\) shows positive reward gains on all four families for Haiku and three of four for each transfer backbone, with regressions occurring on different families\. Panel \(b\) shows that service\-validation success drives most of the gain \(Δ​S=\+23\.1,\+6\.8,\+6\.8\\Delta S=\+23\.1,\+6\.8,\+6\.8pp\), while workflow correctness remains essentially flat \(\|Δ​C\|≤0\.9\|\\Delta C\|\\leq 0\.9pp\)\. Full results are in Appendices[B\.5\.3](https://arxiv.org/html/2609.12127#A2.SS5.SSS3)and[B\.5\.2](https://arxiv.org/html/2609.12127#A2.SS5.SSS2)\.

Table 2:Haiku design study on Flow\-HO modification\. Arrows show changes from RIPPLE\-U; RQ2 contrasts each ablation with Coverage\-Adaptive\.Boldandunderlinemark the top two raw means\.

### 4\.3RQ2: Replay Design and Iteration Depth

##### Replay design governs which policy persists\.

Table[2](https://arxiv.org/html/2609.12127#S4.T2)ablates aggregation, replay maintenance, gating, and iteration depth around Coverage\-Adaptive\. Uniform aggregation and static replay both outperform Coverage\-Adaptive in success and reward, with the matched static\-versus\-adaptive success interval excluding zero, showing that aggregation and replay maintenance change checkpoint selection rather than merely its evaluation\. Train\-Only reaches similar aggregate performance but accepts a broader patch set, including edits rejected by replay; thus, similar held\-out means can mask materially different persistent policies, and replay gating controls which local gains are allowed to compose into the shared policy\. More adaptation is likewise not inherently better: Multi\-Iteration adds two patches yet underperforms its one\-iteration parent, while checkpoints with the same patch count can differ substantially in reward\.These results show that persistent performance is determined by which edits survive, their order, and their interactions, not simply by patch count or iteration depth\.Appendices[B\.4\.2](https://arxiv.org/html/2609.12127#A2.SS4.SSS2)and[B\.4\.3](https://arxiv.org/html/2609.12127#A2.SS4.SSS3)provide the patch decisions and convergence traces\.

### 4\.4RQ3: Detecting Destructive Edit Interactions

##### Composition can reverse a locally beneficial edit\.

To answer RQ3, we isolate F2a, which restricts resource lookup to newly introduced references \(Table\)\. Against the shared iteration\-start policyπ0\\pi\_\{0\}, F2a passes training\-side pre\-qualification withΔtrain=\+0\.0465\\Delta\_\{\\mathrm\{train\}\}=\+0\.0465\. After four earlier patches are composed, however, the same edit becomes strongly destructive: replay reward and correctness fall by0\.22570\.2257and0\.12180\.1218, respectively, while service\-validation success drops from35\.4%35\.4\\%to0%0\\%\. The reversal is interaction\-induced: an earlier patch requires concrete resource resolution, whereas F2a exempts identifiers already present in the corrupted input, allowing stale values to survive into publication\. Advancing\-baseline replay exposes this failure and rejects the patch before persistence\.Thus, edit quality is policy\-context dependent: an edit that is beneficial againstπ0\\pi\_\{0\}can become harmful after composition\.Appendix[E](https://arxiv.org/html/2609.12127#A5)provides the full trace\-level analysis\.

### 4\.5RQ4: Promotion Rules and Checkpoint Trade\-offs

##### Promotion rules select distinct policy\-composition trade\-offs\.

To answer RQ4, we hold the proposer, candidate order, and replay evidence fixed and vary only the acceptor \(Table\)\. The controlled swap uses the Coverage\-Adaptive trajectory, while RIPPLE\-U remains the strongest checkpoint and the headline reference in Table[1](https://arxiv.org/html/2609.12127#S4.T1)\. RIPPLE’s tolerant two\-signal gate accepts four patches, whereas the SkillOpt\-style\([Yang et al\., 2026](https://arxiv.org/html/2609.12127#bib.bib6)\)and SkillGen\-style\([Ma et al\., 2026](https://arxiv.org/html/2609.12127#bib.bib7)\)rules both retain only F3a, converging on the same one\-patch checkpoint that we callSkill\-Compact\(π0⊕F3a\\pi\_\{0\}\\oplus\\text\{F3a\}\); this isolates promotion stringency from proposal quality\. Skill\-Compact improves correctness by5\.45\.4pp and reward by0\.0260\.026over the four\-patchε\\varepsilon\-AND checkpoint \(paired95%95\\%intervals\[\+1\.7,\+9\.4\]\[\+1\.7,\+9\.4\]and\[\+0\.003,\+0\.051\]\[\+0\.003,\+0\.051\]\), with only a1\.7%1\.7\\%difference in success\. Yet these gains are specific to the matched acceptor probe: RIPPLE\-U remains the best overall configuration in the main evaluation, while both checkpoints win on subsets of requests, indicating that acceptance stringency changes not only checkpoint size but also where gains accrue\. A post hoc oracle further reveals selector headroom\.Thus, promotion is a substantive policy\-construction decision: stricter acceptors can produce more compact checkpoints and improve certain metrics, but they reshape rather than uniformly improve the resulting policy and do not supersede RIPPLE\-U as the strongest overall configuration\.Appendix[B\.4\.2](https://arxiv.org/html/2609.12127#A2.SS4.SSS2)provides request\-level and selector analyses\.

## 5Conclusion

We introducedRIPPLE, a replay\-informed framework for persistent prompt\-policy adaptation in executable workflow synthesis\. RIPPLE turns execution failures into bounded, segment\-localized edits and promotes each candidate only after replay against the evolving accepted policy\. On Flow\-HO, RIPPLE substantially improves service\-validation success on Claude Haiku 4\.5 and transfers consistently across two additional frozen backbones, while preserving correctness and achieving strong edit efficiency at low execution cost\. Mechanistic analyses reveal that local gains can reverse after composition and that promotion rules select distinct checkpoints even under a fixed proposer and replay evidence\. Together, these findings establish persistent prompt adaptation as a sequential policy\-composition problem: the value of an edit depends not only on its local utility, but on how it interacts with the policy in which it persists\.In this sense, a local policy edit can create global ripples through downstream execution, making compositional safety central to persistent adaptation\.

## References

- L\. A\. Agrawal, S\. Tan, D\. Soylu, N\. Ziems, R\. Khare, K\. Opsahl\-Ong, A\. Singhvi, H\. Shandilya, M\. J\. Ryan, M\. Jiang, C\. Potts, K\. Sen, A\. G\. Dimakis, I\. Stoica, D\. Klein, M\. Zaharia, and O\. KhattabGEPA: reflective prompt evolution can outperform reinforcement learning\.arXiv preprint arXiv:2507\.19457\.Cited by:[§A\.1](https://arxiv.org/html/2609.12127#A1.SS1.SSS0.Px1.p1.1),[Appendix F](https://arxiv.org/html/2609.12127#A6.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.12127#S1.p3.1)\.
- Chenet al\.\(2024\)X\. Chen, M\. Lin, N\. Schärli, and D\. ZhouTeaching large language models to self\-debug\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§A\.4](https://arxiv.org/html/2609.12127#A1.SS4.p1.1)\.
- Chenget al\.\(2024\)C\. Cheng, A\. Nie, and A\. SwaminathanTrace is the next AutoDiff: generative optimization with rich feedback, execution traces, and LLMs\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§A\.1](https://arxiv.org/html/2609.12127#A1.SS1.SSS0.Px2.p1.1)\.
- Cohenet al\.\(2024\)R\. Cohen, E\. Biran, O\. Yoran, A\. Globerson, and M\. GevaEvaluating the ripple effects of knowledge editing in language models\.Transactions of the Association for Computational Linguistics \(TACL\)12\.Cited by:[§A\.1](https://arxiv.org/html/2609.12127#A1.SS1.SSS0.Px1.p1.1),[Appendix A](https://arxiv.org/html/2609.12127#A1.p2.1),[§1](https://arxiv.org/html/2609.12127#S1.p2.1)\.
- DeepSeek\-AIet al\.\(2025\)DeepSeek\-AI, D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang,et al\.DeepSeek\-R1: incentivizing reasoning capability in LLMs via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[§A\.2](https://arxiv.org/html/2609.12127#A1.SS2.SSS0.Px1.p1.1)\.
- Fernandoet al\.\(2023\)C\. Fernando, D\. Banarse, H\. Michalewski, S\. Osindero, and T\. RocktäschelPromptbreeder: self\-referential self\-improvement via prompt evolution\.arXiv preprint arXiv:2309\.16797\.Cited by:[§A\.1](https://arxiv.org/html/2609.12127#A1.SS1.SSS0.Px2.p1.1)\.
- Fuet al\.\(2024\)Y\. Fu, D\. Kim, J\. Kim, S\. Sohn, L\. Logeswaran, K\. Bae, and H\. LeeAutoGuide: automated generation and selection of context\-aware guidelines for large language model agents\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§A\.3](https://arxiv.org/html/2609.12127#A1.SS3.p1.1)\.
- Gouet al\.\(2024\)Z\. Gou, Z\. Shao, Y\. Gong, Y\. Shen, Y\. Yang, N\. Duan, and W\. ChenCRITIC: large language models can self\-correct with tool\-interactive critiquing\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§A\.2](https://arxiv.org/html/2609.12127#A1.SS2.p1.1)\.
- Guanet al\.\(2025\)X\. Guan, L\. L\. Zhang, Y\. Liu, N\. Shang, Y\. Sun, Y\. Zhu, F\. Yang, and M\. YangrStar\-Math: small LLMs can master math reasoning with self\-evolved deep thinking\.arXiv preprint arXiv:2501\.04519\.Cited by:[§A\.2](https://arxiv.org/html/2609.12127#A1.SS2.SSS0.Px1.p1.1)\.
- Guoet al\.\(2024\)Q\. Guo, R\. Wang, J\. Guo, B\. Li, K\. Song, X\. Tan, G\. Liu, J\. Bian, and Y\. YangEvoPrompt: connecting LLMs with evolutionary algorithms yields powerful prompt optimizers\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§A\.1](https://arxiv.org/html/2609.12127#A1.SS1.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.12127#S1.p3.1)\.
- Hosseiniet al\.\(2024\)A\. Hosseini, X\. Yuan, N\. Malkin, A\. Courville, A\. Sordoni, and R\. AgarwalV\-STaR: training verifiers for self\-taught reasoners\.arXiv preprint arXiv:2402\.06457\.Cited by:[§A\.2](https://arxiv.org/html/2609.12127#A1.SS2.SSS0.Px1.p1.1)\.
- Huet al\.\(2024\)S\. Hu, C\. Lu, and J\. CluneAutomated design of agentic systems\.arXiv preprint arXiv:2408\.08435\.Cited by:[§A\.3](https://arxiv.org/html/2609.12127#A1.SS3.p1.1)\.
- Huanget al\.\(2024\)J\. Huang, X\. Chen, S\. Mishra, H\. S\. Zheng, A\. W\. Yu, X\. Song, and D\. ZhouLarge language models cannot self\-correct reasoning yet\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§A\.2](https://arxiv.org/html/2609.12127#A1.SS2.p1.1)\.
- Jimenezet al\.\(2024\)C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. NarasimhanSWE\-bench: can language models resolve real\-world GitHub issues?\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§A\.4](https://arxiv.org/html/2609.12127#A1.SS4.p1.1)\.
- Khattabet al\.\(2024\)O\. Khattab, A\. Singhvi, P\. Maheshwari, Z\. Zhang, K\. Santhanam, S\. Vardhamanan, S\. Haq, A\. Sharma, T\. T\. Joshi, H\. Moazam, H\. Miller, M\. Zaharia, and C\. PottsDSPy: compiling declarative language model calls into self\-improving pipelines\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§A\.1](https://arxiv.org/html/2609.12127#A1.SS1.SSS0.Px1.p1.1)\.
- Kumaret al\.\(2020\)A\. Kumar, A\. Zhou, G\. Tucker, and S\. LevineConservative Q\-learning for offline reinforcement learning\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 1179–1191\.Cited by:[§A\.2](https://arxiv.org/html/2609.12127#A1.SS2.p2.1)\.
- Lambertet al\.\(2024\)N\. Lambert, J\. Morrison, V\. Pyatkin, S\. Huang, H\. Ivison, F\. Brahman, L\. J\. V\. Miranda,et al\.Tulu 3: pushing frontiers in open language model post\-training\.arXiv preprint arXiv:2411\.15124\.Cited by:[§A\.2](https://arxiv.org/html/2609.12127#A1.SS2.SSS0.Px1.p1.1)\.
- Maet al\.\(2026\)Y\. Ma, Y\. Huang, H\. Bao, H\. Zhuang, S\. Shukla, M\. Galley, X\. Zhang, and S\. FeuerriegelSkillGen: verified inference\-time agent skill synthesis\.arXiv preprint arXiv:2605\.10999\.Cited by:[2nd item](https://arxiv.org/html/2609.12127#A1.I1.i2.p1.1),[§B\.2](https://arxiv.org/html/2609.12127#A2.SS2.p2.1),[Appendix F](https://arxiv.org/html/2609.12127#A6.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.12127#S1.p3.1),[§4\.5](https://arxiv.org/html/2609.12127#S4.SS5.SSS0.Px1.p1.1)\.
- Mollet al\.\(2026\)J\. Moll, J\. Corbeil, J\. Pan, M\. Hadamitzky, D\. Rueckert, L\. Adams, and K\. BressemGRASP: gated regression\-aware skill proposer for self\-improving LLM agents\.arXiv preprint arXiv:2605\.29668\.Cited by:[1st item](https://arxiv.org/html/2609.12127#A1.I1.i1.p1.1),[Appendix F](https://arxiv.org/html/2609.12127#A6.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.12127#S1.p3.1)\.
- Opsahl\-Onget al\.\(2024\)K\. Opsahl\-Ong, M\. J\. Ryan, J\. Purtell, D\. Broman, C\. Potts, M\. Zaharia, and O\. KhattabOptimizing instructions and demonstrations for multi\-stage language model programs\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 9340–9366\.Cited by:[§A\.1](https://arxiv.org/html/2609.12127#A1.SS1.SSS0.Px1.p1.1),[Appendix F](https://arxiv.org/html/2609.12127#A6.SS0.SSS0.Px2.p1.1)\.
- Pryzantet al\.\(2023\)R\. Pryzant, D\. Iter, J\. Li, Y\. T\. Lee, C\. Zhu, and M\. ZengAutomatic prompt optimization with “gradient descent” and beam search\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 7957–7968\.Cited by:[§A\.1](https://arxiv.org/html/2609.12127#A1.SS1.SSS0.Px1.p1.1)\.
- Schnabel and Neville \(2024\)T\. Schnabel and J\. NevilleSAMMO: a general\-purpose framework for prompt optimization\.arXiv preprint arXiv:2404\.02319\.Cited by:[§A\.1](https://arxiv.org/html/2609.12127#A1.SS1.SSS0.Px1.p1.1)\.
- Sclaret al\.\(2024\)M\. Sclar, Y\. Choi, Y\. Tsvetkov, and A\. SuhrQuantifying language models’ sensitivity to spurious features in prompt design or: how i learned to start worrying about prompt formatting\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§A\.1](https://arxiv.org/html/2609.12127#A1.SS1.SSS0.Px1.p1.1),[Appendix A](https://arxiv.org/html/2609.12127#A1.p2.1),[§1](https://arxiv.org/html/2609.12127#S1.p2.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§A\.1](https://arxiv.org/html/2609.12127#A1.SS1.SSS0.Px4.p1.1)\.
- Shawn \(2026\)Z\. ShawnPACE: anytime\-valid acceptance tests for self\-evolving agents\.arXiv preprint arXiv:2606\.08106\.Cited by:[4th item](https://arxiv.org/html/2609.12127#A1.I1.i4.p1.1),[Appendix F](https://arxiv.org/html/2609.12127#A6.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.12127#S1.p3.1)\.
- Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, E\. Berman, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§A\.3](https://arxiv.org/html/2609.12127#A1.SS3.p1.1)\.
- Wanget al\.\(2023\)G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. AnandkumarVoyager: an open\-ended embodied agent with large language models\.arXiv preprint arXiv:2305\.16291\.Cited by:[§A\.3](https://arxiv.org/html/2609.12127#A1.SS3.p1.1)\.
- Wanget al\.\(2024a\)X\. Wang, Y\. Chen, L\. Yuan, Y\. Zhang, Y\. Li, H\. Peng, and H\. JiExecutable code actions elicit better LLM agents\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§A\.4](https://arxiv.org/html/2609.12127#A1.SS4.p1.1)\.
- Wanget al\.\(2024b\)X\. Wang, C\. Li, Z\. Wang, F\. Bai, H\. Luo, J\. Zhang, N\. Jojic, E\. P\. Xing, and Z\. HuPromptAgent: strategic planning with language models enables expert\-level prompt optimization\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§A\.1](https://arxiv.org/html/2609.12127#A1.SS1.SSS0.Px2.p1.1)\.
- Wanget al\.\(2024c\)Z\. Z\. Wang, J\. Mao, D\. Fried, and G\. NeubigAgent workflow memory\.arXiv preprint arXiv:2409\.07429\.Cited by:[§A\.3](https://arxiv.org/html/2609.12127#A1.SS3.p1.1)\.
- Xianget al\.\(2025\)J\. Xiang, J\. Zhang, Z\. Yu, X\. Liang, F\. Teng, J\. Tu, F\. Ren, X\. Tang, S\. Hong, C\. Wu, and Y\. LuoSelf\-supervised prompt optimization\.arXiv preprint arXiv:2502\.06855\.Cited by:[§A\.1](https://arxiv.org/html/2609.12127#A1.SS1.SSS0.Px2.p1.1)\.
- Yanget al\.\(2024a\)C\. Yang, X\. Wang, Y\. Lu, H\. Liu, Q\. V\. Le, D\. Zhou, and X\. ChenLarge language models as optimizers\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§A\.1](https://arxiv.org/html/2609.12127#A1.SS1.SSS0.Px1.p1.1),[Appendix F](https://arxiv.org/html/2609.12127#A6.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.12127#S1.p3.1)\.
- Yanget al\.\(2024b\)J\. Yang, C\. E\. Jimenez, A\. Wettig, K\. Lieret, S\. Yao, K\. Narasimhan, and O\. PressSWE\-agent: agent\-computer interfaces enable automated software engineering\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§A\.4](https://arxiv.org/html/2609.12127#A1.SS4.p1.1)\.
- Yanget al\.\(2026\)Y\. Yang, Z\. Gong, W\. Huang, Q\. Yang, Z\. Zhou, Z\. Huang, Y\. Li, X\. Gao, Q\. Dai, B\. Liu, K\. Qiu, Y\. Yang, D\. Chen, X\. Yang, and C\. LuoSkillOpt: executive strategy for self\-evolving agent skills\.arXiv preprint arXiv:2605\.23904\.Cited by:[3rd item](https://arxiv.org/html/2609.12127#A1.I1.i3.p1.1),[§B\.2](https://arxiv.org/html/2609.12127#A2.SS2.p2.1),[Appendix F](https://arxiv.org/html/2609.12127#A6.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.12127#S1.p3.1),[§4\.5](https://arxiv.org/html/2609.12127#S4.SS5.SSS0.Px1.p1.1)\.
- Yaoet al\.\(2024\)S\. Yao, N\. Shinn, P\. Razavi, and K\. Narasimhanτ\\tau\-Bench: a benchmark for tool\-agent\-user interaction in real\-world domains\.arXiv preprint arXiv:2406\.12045\.Cited by:[§A\.4](https://arxiv.org/html/2609.12127#A1.SS4.p1.1)\.
- Yuksekgonulet al\.\(2024\)M\. Yuksekgonul, F\. Bianchi, J\. Boen, S\. Liu, Z\. Huang, C\. Guestrin, and J\. ZouTextGrad: automatic “differentiation” via text\.arXiv preprint arXiv:2406\.07496\.Cited by:[§A\.1](https://arxiv.org/html/2609.12127#A1.SS1.SSS0.Px1.p1.1),[Appendix F](https://arxiv.org/html/2609.12127#A6.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.12127#S1.p3.1)\.
- Zhanget al\.\(2025a\)J\. Zhang, J\. Xiang, Z\. Yu, F\. Teng, X\. Chen, J\. Chen, M\. Zhuge, X\. Cheng, S\. Hong, J\. Wang, B\. Zheng, B\. Liu, Y\. Luo, and C\. WuAFlow: automating agentic workflow generation\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§A\.4](https://arxiv.org/html/2609.12127#A1.SS4.p1.1)\.
- Zhanget al\.\(2025b\)L\. Zhang, A\. Hosseini, H\. Bansal, M\. Kazemi, A\. Kumar, and R\. AgarwalGenerative verifiers: reward modeling as next\-token prediction\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§A\.2](https://arxiv.org/html/2609.12127#A1.SS2.SSS0.Px1.p1.1)\.
- Zhaoet al\.\(2024\)A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. HuangExpeL: LLM agents are experiential learners\.InProceedings of the AAAI Conference on Artificial Intelligence \(AAAI\),Cited by:[§A\.3](https://arxiv.org/html/2609.12127#A1.SS3.p1.1)\.
- Zhenget al\.\(2023\)L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. StoicaJudging LLM\-as\-a\-judge with MT\-Bench and chatbot arena\.InAdvances in Neural Information Processing Systems,Vol\.36,pp\. 46595–46623\.Cited by:[§A\.2](https://arxiv.org/html/2609.12127#A1.SS2.p2.1)\.
- Zhouet al\.\(2024\)W\. Zhou, Y\. Ou, S\. Ding, L\. Li, J\. Wu, T\. Wang, J\. Chen, S\. Wang, X\. Xu, N\. Zhang, H\. Chen, and Y\. E\. JiangSymbolic learning enables self\-evolving agents\.arXiv preprint arXiv:2406\.18532\.Cited by:[§A\.1](https://arxiv.org/html/2609.12127#A1.SS1.SSS0.Px2.p1.1)\.
- Zhouet al\.\(2023\)Y\. Zhou, A\. I\. Muresanu, Z\. Han, K\. Paster, S\. Pitis, H\. Chan, and J\. BaLarge language models are human\-level prompt engineers\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§A\.1](https://arxiv.org/html/2609.12127#A1.SS1.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.12127#S1.p3.1)\.

## Appendix Contents

## Appendix ARelated Work

RIPPLE lies at the intersection of automatic prompt optimization, verification\-driven improvement, and persistent skill or prompt evolution\. Related methods share a generate–evaluate–promote pattern; RIPPLE differs in how it localizes execution evidence, represents edits, and evaluates them after composition with an evolving policy\.

We organize this discussion around the two properties from Section[1](https://arxiv.org/html/2609.12127#S1)\. First,edit locality does not imply effect locality: a local textual change can ripple through downstream behavior, as documented in the knowledge\-editing\([Cohen et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib37)\)and prompt\-sensitivity\([Sclar et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib38)\)literatures\. Second,the value of an edit is policy\-context dependent: whether an improvement survives depends on the accepted prefix already in force\. The subsections below thread these two properties through prompt optimization \(A\.1\), verification\-driven improvement \(A\.2\), persistent skill evolution \(A\.3\), and executable workflow synthesis \(A\.4\), highlighting where each family addresses one property, both, or neither\.

### A\.1Automatic Prompt and Policy Optimization

##### Search and textual optimization\.

APE\([Zhou et al\., 2023](https://arxiv.org/html/2609.12127#bib.bib1)\)and OPRO\([Yang et al\., 2024a](https://arxiv.org/html/2609.12127#bib.bib2)\)optimize candidate instructions from task scores, while ProTeGi/APO\([Pryzant et al\., 2023](https://arxiv.org/html/2609.12127#bib.bib10)\)and TextGrad\([Yuksekgonul et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib11)\)turn natural\-language feedback into targeted textual updates\. EvoPrompt\([Guo et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib3)\)and GEPA\([Agrawal et al\., 2025](https://arxiv.org/html/2609.12127#bib.bib4)\)use evolutionary refinement, whereas DSPy/MIPRO\([Khattab et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib12);[Opsahl\-Ong et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib13)\)and SAMMO\([Schnabel and Neville, 2024](https://arxiv.org/html/2609.12127#bib.bib39)\)optimize instructions, demonstrations, or structured metaprompt fragments within LLM programs\. Collectively, these methods show that black\-box textual policies can improve without gradient access to the underlying model\. Because they score each candidate against a task metric on the base policy alone, however, they do not test whether an edit remains safe after other edits have been composed\. The observation that small textual changes can produce non\-local behavior shifts\([Cohen et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib37);[Sclar et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib38)\)is precisely what motivates evaluating a candidate in the advancing policy state, not only againstπ0\\pi\_\{0\}\.

##### Search, reflection, and evolutionary proposers\.

A parallel line refines prompts through explicit search or self\-referential rewriting: PromptAgent\([Wang et al\., 2024b](https://arxiv.org/html/2609.12127#bib.bib20)\)runs Monte Carlo tree search over error\-reflection actions to reach expert\-level prompts, Promptbreeder\([Fernando et al\., 2023](https://arxiv.org/html/2609.12127#bib.bib21)\)co\-evolves task and mutation prompts under LLM\-graded fitness, and SPO\([Xiang et al\., 2025](https://arxiv.org/html/2609.12127#bib.bib18)\)removes the need for reference labels by comparing candidate outputs with a pairwise LLM judge\. Agent Symbolic Learning\([Zhou et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib40)\)extends the textual\-gradient view to whole\-agent parameters \(prompts, tools, and pipeline structure\), and Trace / OptoPrime\([Cheng et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib19)\)generalizes the pattern by treating execution traces and rich textual feedback as “gradients” that an LLM optimizer applies to heterogeneous parameters, including prompts and code\. Both optimize the current parameter jointly against a task loss; neither retains a versioned accepted prefix against which a new candidate must remain safe, so gains that depend on the current parameter state are not distinguished from gains that would compose across accepted edits\. RIPPLE shares the search\-and\-reflect intuition but replaces LLM\-driven proposal with deterministic retrieval from a fixed patch library indexed by predicate diagnosis; unbounded proposal, in our setting, expands the search space faster than the replay gate can safely filter it\.

##### Relation to RIPPLE\.

RIPPLE follows the same broad propose\-and\-select template but constrains proposal before evaluation: diagnosed execution evidence selects a predefined policy segment and a bounded library patch\. It also decouples ranking from persistence\. Shared\-parent training gains rank candidates, whereas replay tests each candidate after previously accepted edits have been composed\. A patch can therefore be useful in isolation yet unsafe in the policy state in which it would persist\.

##### Model\-weight adaptation\.

Weight\-updating methods such as GRPO\([Shao et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib5)\)optimize model parameters from rollout outcomes\. RIPPLE instead keeps the model endpoint fixed and adapts only the prompt policy, so accepted changes remain inspectable, versionable, and reversible\.

### A\.2Verification\-Driven and Conservative Improvement

External verification is often necessary for reliable improvement\. Intrinsic self\-correction can reduce accuracy without oracle feedback\([Huang et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib14)\), whereas verify\-then\-revise systems such as CRITIC\([Gou et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib15)\)use tools or external checks to repair individual outputs\. RIPPLE extends the same verify\-before\-commit principle from a single response to a persistent policy edit\.

The replay gate also parallels conservative policy improvement in offline RL\([Kumar et al\., 2020](https://arxiv.org/html/2609.12127#bib.bib16)\), but applies an empirical constraint: RIPPLE withholds a training\-improving candidate when replay reward or workflow correctness exceeds the allowed regression under the advancing accepted prefix\. Automated correctness signals make repeated promotion practical, and prior LLM\-as\-a\-judge work suggests that model\-based evaluation can correlate with human preferences\([Zheng et al\., 2023](https://arxiv.org/html/2609.12127#bib.bib17)\)\. RIPPLE nevertheless combines automated correctness with programmatic service validation rather than relying on an LLM judge alone\.

##### Verifiable rewards and iterative acceptance\.

A related family scales verification\-driven improvement to full model training\. Reinforcement Learning with Verifiable Rewards\([Lambert et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib23)\)and DeepSeek\-R1\([DeepSeek\-AI et al\., 2025](https://arxiv.org/html/2609.12127#bib.bib22)\)update model weights against deterministic checkers \(math answers, unit tests\) rather than preference models; rStar\-Math\([Guan et al\., 2025](https://arxiv.org/html/2609.12127#bib.bib24)\)advances a base policy in rounds gated by a learned process\-preference model over trajectories; and learned verifiers such as V\-STaR\([Hosseini et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib25)\)and Generative Verifiers\([Zhang et al\., 2025b](https://arxiv.org/html/2609.12127#bib.bib26)\)train models to rerank generations at inference\. RIPPLE inherits the same verifiable\-signal premise but stays at the symbolic\-policy layer: no weight updates, no learned verifier, no best\-of\-NNreranking of outputs\. Verification enters only at promotion time, as a two\-signal replay constraint that a candidate patch must pass against an advancing accepted prefix before it enters the deployed policy\.

### A\.3Persistent Skill and Prompt Evolution

The idea of persisting textual artifacts across tasks has an older lineage\. Voyager\([Wang et al\., 2023](https://arxiv.org/html/2609.12127#bib.bib27)\)maintains a lifelong library of executable skills paired with an automatic curriculum and self\-verification loop; Reflexion\([Shinn et al\., 2023](https://arxiv.org/html/2609.12127#bib.bib28)\)treats per\-episode natural\-language reflections as verbal reinforcement over a fixed policy; ExpeL\([Zhao et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib31)\)accumulates free\-form insights from training tasks and retrieves them at inference; ADAS\([Hu et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib29)\)lets an LLM meta\-agent program new agents into an ever\-growing archive; Agent Workflow Memory\([Wang et al\., 2024c](https://arxiv.org/html/2609.12127#bib.bib30)\)induces reusable workflows from browser trajectories and calls them back on later web tasks; and AutoGuide\([Fu et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib42)\)distills state\-conditioned natural\-language guidelines from offline experience and retrieves the applicable ones at test time\. Both AWM’s workflow\-level artifacts and ADAS’s agent\-graph edits produce persistent behavior changes whose downstream effects are non\-local, and whose value can shift once earlier artifacts are already in place\. AutoGuide is the closest external analog to RIPPLE’s persistent guideline layer, but it selects guidelines by state at inference rather than gating their entry into the deployed policy against an advancing accepted prefix\. Together, these systems establish that acquired text can carry non\-trivial behavior across tasks, but none couple acquisition to a replay\-based falsifiability rule against the current accepted prefix—any generated skill, reflection, guideline, or workflow that improves the demonstration is retained\.

The closest methods to RIPPLE also use explicit promotion rules for persistent textual updates:

- •GRASP\([Moll et al\., 2026](https://arxiv.org/html/2609.12127#bib.bib8)\)induces an open\-vocabulary failure taxonomy, proposes ADD/MODIFY/REMOVE skill edits, and promotes candidates when fixes exceed regressions under a hard regression budget\.
- •SkillGen\([Ma et al\., 2026](https://arxiv.org/html/2609.12127#bib.bib7)\)compares paired rollouts with and without a skill and promotes it when repair gains exceed a minimum paired\-count threshold\.
- •SkillOpt\([Yang et al\., 2026](https://arxiv.org/html/2609.12127#bib.bib6)\)optimizes a monolithic skill file and accepts a candidate only when it strictly improves a disjoint selection score; ties are rejected\.
- •PACE\([Shawn, 2026](https://arxiv.org/html/2609.12127#bib.bib9)\)uses paired outcomes and an anytime\-valid e\-process for sequential acceptance on a reused development pool, together with a fresh audit pool\.

Replay\-gated promotion is therefore not unique to RIPPLE\. The distinction lies in its deployment target and the structure surrounding the gate\. First, RIPPLE targets schema\-validated executable workflows, for which service validity, structural correctness, edit locality, resource resolution, and execution cost can diverge\. Second, diagnosis uses a fixed six\-family workflow taxonomy rather than a run\-specific open vocabulary\. Third, every update is typed by both failure family and prompt segment and retrieved from a versioned library\. Finally, promotion constrains reward and workflow correctness separately while evaluating candidates against an advancing accepted prefix\. These choices trade open\-ended coverage for deterministic, auditable policy changes suited to persistent deployment\.

### A\.4Executable Workflow Synthesis and Verifiable\-Reward Agents

RIPPLE evaluates on schema\-validated executable workflows, and its verifier stack \(service validity, structural correctness, resource resolution, execution\) mirrors a growing class of code\-agent benchmarks\. SWE\-bench\([Jimenez et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib32)\)grades LLM patches against hidden unit tests on real GitHub issues, andτ\\tau\-Bench\([Yao et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib33)\)exercises multi\-turn tool\-using dialogue against a simulated user and environment with passkexecution scoring; both establish the “deterministic checker as reward” regime that Flow\-HO adapts to workflow synthesis\. On the agent side, SWE\-agent\([Yang et al\., 2024b](https://arxiv.org/html/2609.12127#bib.bib34)\)wraps an LLM in an edit–run–observe loop over a live repository, CodeAct\([Wang et al\., 2024a](https://arxiv.org/html/2609.12127#bib.bib35)\)unifies actions as executable code so that interpreter feedback drives per\-step revision, and Self\-Debug\([Chen et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib36)\)shows that executable feedback alone—compiler and interpreter output, with no human critique—suffices for intra\-episode repair\. Closer to persistent workflow construction, AFlow\([Zhang et al\., 2025a](https://arxiv.org/html/2609.12127#bib.bib41)\)searches over code\-represented workflow graphs with Monte Carlo tree search to synthesize whole pipelines under a task metric, but the search yields a single deployed workflow rather than a versioned sequence of typed prompt\-segment edits\. In all of these systems, executable feedback is consumed only within the current task or a one\-shot search: the prompt policy is fixed at deployment, so any signal extracted from one instance is discarded when the next arrives\. RIPPLE occupies a complementary slot in this landscape—the same class of executable signals is retained across tasks by admitting only those prompt\-segment edits that pass the two\-signal replay gate against an advancing baseline, converting intra\-episode feedback into cross\-task, versioned policy state\.

## Appendix BImplementation Details and Additional Experiments

### B\.1Benchmarks

##### Flow\-HO\.

Flow\-HO is a synthetic held\-out suite of executable workflow tasks expressed in a JSON\-based flow language\. It covers workflow generation and modification\. Every example was created for this study from publicly documented flow constructs; no customer workflow or interaction data is included\.

The primary study evaluatesworkflow modification: the agent receives a natural\-language request and a corrupted workflowy0y\_\{0\}, then must return a validated workflow that implements the requested change\.Workflow generationis secondary: the agent receives only the request and constructs a schema\-valid workflow from scratch\.

#### B\.1\.1Workflow Modification

The modification benchmark contains 12 ground\-truth workflow families partitioned into disjoint training, replay\-core, and Flow\-HO pools \(Table[3](https://arxiv.org/html/2609.12127#A2.T3)\)\. A family contains all corruption variants derived from one ground\-truth workflow, and no family crosses pool boundaries\.

Table 3:Modification\-pool composition\. Ground\-truth workflow families are disjoint across training, replay core, and Flow\-HO\.##### Held\-out families\.

The four Flow\-HO modification families are denoted A–D\. Table[4](https://arxiv.org/html/2609.12127#A2.T4)summarizes their task counts and topology\.Blockscounts workflow actions,typescounts distinct action types, andtransitionscounts base, conditional, and errorNextActionedges\. The families span distinct structural regimes rather than minor variants of one template, supporting the family\-level analysis in §[4\.2](https://arxiv.org/html/2609.12127#S4.SS2)\.

Table 4:Structural properties of the four Flow\-HO modification families\. B/T/E denotes the numbers of blocks, distinct action types, and transitions; transitions include base, conditional, and error edges\.
##### Corruption procedure\.

Each corrupted input is generated deterministically while preserving a resolvable graph and a valid start action\. The operator names denote the requested repair:add\-blockremoves one action and redirects incoming edges;reroutechanges one outgoing or error transition;modify\-configoverwrites one parameter or metadata field; andreplace\-logicremoves a self\-contained subgraph and reconnects the surviving path\. Metadata for deleted actions are removed, and every remaining edge resolves to an existing action\.

Across the 82 modification datapoints \(35 training, 8 replay core, and 39 Flow\-HO\), the corruption mix is 25add\-block, 19modify\-config, 19reroute, and 19replace\-logicexamples\. Pool\-specific counts are stored with the benchmark metadata\.

#### B\.1\.2Workflow Generation

The secondary generation benchmark contains 49 synthetic workflows with 8–25 actions\. Each datapoint stores a natural\-language request, a ground\-truth workflow, a coarse complexity label, a scenario category, and metadata for missing\-information slots and refusal conditions\. Generation uses the same agent and validation infrastructure, but correctness is computed with the identifier\-invariant metric in Appendix[B\.3\.4](https://arxiv.org/html/2609.12127#A2.SS3.SSS4)\. Edit efficiency is fixed at one because no starting workflow is modified\.

### B\.2Baselines and Controlled Acceptors

Baseπ0\\pi\_\{0\}is a human\-authored segmented prompt policy created once with assistance from a stronger reference model and then frozen\. Every RIPPLE configuration starts from the sameπ0\\pi\_\{0\}and changes it only through accepted library patches\.

RQ4 changes only the acceptor, holding RIPPLE’s proposer, candidate order, and recorded replay outcomes fixed\. TheSkillOpt\-stylerule\([Yang et al\., 2026](https://arxiv.org/html/2609.12127#bib.bib6)\)accepts only when replay reward strictly improves,ΔRg\>0\\Delta\_\{R\}^\{g\}\>0; ties are rejected\. TheSkillGen\-stylerule\([Ma et al\., 2026](https://arxiv.org/html/2609.12127#bib.bib7)\)accepts when

round⁡\(m​N​ΔSg\)≥max⁡\{2,⌈0\.05​m​N⌉\},\\operatorname\{round\}\(mN\\,\\Delta\_\{S\}^\{g\}\)\\geq\\max\\\{2,\\lceil 0\.05mN\\rceil\\\},wherem=\|𝒟replay\|m=\|\\mathcal\{D\}\_\{\\mathrm\{replay\}\}\|andNNis the rollout count per replay datapoint\. These controlled acceptor swaps are not end\-to\-end reproductions of the original systems\. Both select the same one\-patch checkpoint,π0⊕F3a\\pi\_\{0\}\\oplus\\mathrm\{F3a\}, which we callSkill\-Compact\. Its metrics come from a separate matched acceptor\-probe cache and are analyzed in RQ4 rather than ranked with the harmonized systems in Table[1](https://arxiv.org/html/2609.12127#S4.T1)\.

### B\.3Framework Details

This subsection specifies the deterministic components summarized in Section[3](https://arxiv.org/html/2609.12127#S3): diagnosis and candidate construction \(Appendix[B\.3\.1](https://arxiv.org/html/2609.12127#A2.SS3.SSS1)\), replay\-gated promotion and replay\-pool maintenance \(Appendix[B\.3\.2](https://arxiv.org/html/2609.12127#A2.SS3.SSS2)\), the multi\-iteration procedure \(Appendix[B\.3\.3](https://arxiv.org/html/2609.12127#A2.SS3.SSS3)\), the reward function \(Appendix[B\.3\.4](https://arxiv.org/html/2609.12127#A2.SS3.SSS4)\), and rollout and uncertainty settings \(Appendix[B\.3\.5](https://arxiv.org/html/2609.12127#A2.SS3.SSS5)\)\.

Table 5:Predicate\-based failure taxonomy for behavioral credit assignment\. Direct evidence precedes consequence fallbacks; F6 refinements can override the default target segment\.#### B\.3\.1Candidate Construction: Diagnosis, Library, and Aggregation

##### Taxonomy and library provenance\.

We constructed the taxonomy*top\-down*from the major actionable failure loci in executable workflow synthesis, rather than inducing it from Flow\-HO outcomes\. Intuitively, a failed trajectory can arise because the agent did not obtain the needed information \(F1\), interact with tools or resources correctly \(F2\), produce a schema\-valid artifact \(F3\), recover from validation feedback \(F4\), preserve the intended scope of an edit \(F5\), or complete the requested workflow \(F6\)\. These categories correspond to distinct intervention points in the agent policy, making the taxonomy useful for localizing*where*corrective credit should be assigned rather than merely describing observed errors\.

The six super\-families F1–F6 therefore predate the Flow\-HO evaluation used in this paper and were fixed before the final held\-out split was constructed\. Their tie\-break orderings reflect task\-aware causal priority rather than empirical prevalence in training rollouts; for example, clarification and tool\-use failures take precedence when editing an existing flow\. The F6 subrefinements F6a–F6h were introduced iteratively during method development: F6a–F6d were derived from an initial reward\-component profile of low\-rewardπ0\\pi\_\{0\}training rollouts; F6e and F6f targeted missing\-block and edge failures observed in subsequent adaptation runs; F6g and F6h were added after an iteration in which no candidate was accepted, targeting the parameter\-key and edge failure modes identified in that trace\. Every library entry was frozen before its own replay\-gate evaluation, and no accepted patch was rewritten after its replay outcome\. The 23 entries were authored by the paper authors with reference to the public flow\-schema documentation, and retrieval invokes no LLM\.

##### Predicate\-based diagnosis\.

For each selected low\-reward trajectoryτ\\tau, RIPPLE returns an actionable failure\-family setℱ⁡\(τ\)\\mathcal\{F\}\(\\tau\)and an evidence recordz⁡\(τ\)z\(\\tau\)\. Direct predicates identify observed causal behavior; consequence predicates capture downstream symptoms and are considered only when direct evidence is unavailable\. Table[5](https://arxiv.org/html/2609.12127#A2.T5)summarizes the six families and their default target segments\.

When multiple predicates fire, direct evidence takes precedence\. Remaining ties follow a task\-specific order: F1, F2, F3, F4, F5, F6 for modification, and F3, F6, F2, F1, F4, F5 for generation\. A hallucinated action type overrides either order with F3, F6, F1, F2, F4, F5\. The first family is marked primary; other supported families remain actionable and may nominate additional patches\.

##### Completeness refinement\.

F6 is refined using the reward\-component trace: no output maps to F6a/PLAN; missing blocks to F6e/PLAN; missing requested capabilities to F6b/REQ\_UNDERSTANDING; unresolved values to F6c/TOOL\_USE; incorrect edges to F6f/EDIT; and residual under\-specification to F6d/EDIT\. F6g is a second preregistered requirement\-coverage patch targetingREQ\_UNDERSTANDING\. Every refinement retrieves a fixed library entry; none triggers free\-form patch generation\.

##### Segment\-typed patch library\.

The base prompt is parsed into seven fixed segments—REQ\_UNDERSTANDING,CLARIFY,PLAN,TOOL\_USE,EDIT,VALIDATE, andFINAL\_OUTPUT—covering request interpretation, clarification, planning, resource resolution, workflow editing, validation and repair, and response completion\. The default family\-to\-segment map isF→CLARIFYF1\\\!\\to\\\!\\texttt\{CLARIFY\},F→TOOL\_USEF2\\\!\\to\\\!\\texttt\{TOOL\\\_USE\},F→EDITF3\\\!\\to\\\!\\texttt\{EDIT\},F→VALIDATEF4\\\!\\to\\\!\\texttt\{VALIDATE\},F→EDITF5\\\!\\to\\\!\\texttt\{EDIT\}, andF→EDITF6\\\!\\to\\\!\\texttt\{EDIT\}; the F6 refinements above override this default\.

The versioned libraryℒ\\mathcal\{L\}contains 23 preregistered entries of the form\(id,f,s,instruction\)\(\\text\{id\},f,s,\\text\{instruction\}\)\. Given familyffand localized segmentss,Retrievefirst selects an entry matching both and otherwise falls back to an entry forss\. The stored instruction is appended verbatim to the target segment\. Retrieval invokes no LLM, so each accepted edit is deterministic, reviewable, and independently reversible\.

##### Aggregation\.

Repeated nominations are merged by patch ID, and previously accepted IDs are removed\. The support countck​\(a\)c\_\{k\}\(a\)records how many selected trajectories nominate patchaa\. Candidates are sorted by support within each failure family, then interleaved by a coverage\-first round robin: every represented family contributes one candidate before any family contributes a second\. The leading candidates form𝒜k\\mathcal\{A\}\_\{k\}and proceed to promotion\.

#### B\.3\.2Promotion: Two\-Signal Replay Gate and Replay Pool

##### Empirical scores and replay deltas\.

For policyπ\\pi, metricX∈\{R,C\}X\\in\\\{R,C\\\}, and pool𝒟\\mathcal\{D\}, define

V^Xg​\(π,𝒟\)=g⁡\(\(X¯π​\(d\)\)d∈𝒟\),X¯π​\(d\)=1N​∑n=1NX⁡\(τd,nπ\)\.\\widehat\{V\}\_\{X\}^\{g\}\(\\pi;\\mathcal\{D\}\)=g\\\!\\left\(\(\\bar\{X\}\_\{\\pi\}\(d\)\)\_\{d\\in\\mathcal\{D\}\}\\right\),\\qquad\\bar\{X\}\_\{\\pi\}\(d\)=\\frac\{1\}\{N\}\\sum\_\{n=1\}^\{N\}X\(\\tau\_\{d,n\}^\{\\pi\}\)\.Uniform aggregation weights datapoints equally\. Ground\-truth\-family\-balanced aggregation first averages within each workflow family and then weights represented families equally\. The implementation label*coverage\-weighted*refers to this macro\-average, not to coverage over diagnosed failure families \(Appendix[C\.2](https://arxiv.org/html/2609.12127#A3.SS2)\)\.

For thejj\-th ranked candidateaja\_\{j\}, the empirical replay delta corresponding to Equation[6](https://arxiv.org/html/2609.12127#S2.E6)is

Δ^Xg​\(aj∣Pk<j;𝒟replay\(k\)\)=V^Xg​\(πk⊕Pk<j⊕aj,𝒟replay\(k\)\)−V^Xg​\(πk⊕Pk<j,𝒟replay\(k\)\)\.\\widehat\{\\Delta\}\_\{X\}^\{g\}\(a\_\{j\}\\mid P\_\{k\}^\{<j\};\\mathcal\{D\}\_\{\\mathrm\{replay\}\}^\{\(k\)\}\)=\\widehat\{V\}\_\{X\}^\{g\}\(\\pi\_\{k\}\\oplus P\_\{k\}^\{<j\}\\oplus a\_\{j\};\\mathcal\{D\}\_\{\\mathrm\{replay\}\}^\{\(k\)\}\)\-\\widehat\{V\}\_\{X\}^\{g\}\(\\pi\_\{k\}\\oplus P\_\{k\}^\{<j\};\\mathcal\{D\}\_\{\\mathrm\{replay\}\}^\{\(k\)\}\)\.A candidate is promoted only when both deltas satisfy\(εR,εC\)=\(−0\.05,−0\.10\)\(\\varepsilon\_\{R\},\\varepsilon\_\{C\}\)=\(\-0\.05,\-0\.10\)\. Acceptance advances the prefix; rejection leaves it unchanged\. Train\-Only skips replay promotion after training\-side pre\-qualification\.

##### Replay\-pool composition\.

The promotion pool is assembled once per outer iteration, after scoring the parent\-policy training rollouts and before evaluating candidates\. Adaptive replay uses

𝒟replay\(k\)=𝒟core∪𝒟hard\(k\)∪𝒟recent\(k\),\\mathcal\{D\}\_\{\\mathrm\{replay\}\}^\{\(k\)\}=\\mathcal\{D\}\_\{\\mathrm\{core\}\}\\cup\\mathcal\{D\}\_\{\\mathrm\{hard\}\}^\{\(k\)\}\\cup\\mathcal\{D\}\_\{\\mathrm\{recent\}\}^\{\(k\)\},with duplicates removed\. The fixed core contains eight datapoints spanning three ground\-truth workflow families\. The hard slot contains up to four non\-core training datapoints with parent\-policy mean reward below0\.40\.4and at least two failed rollouts, ordered from lowest reward upward\. The recent slot contains up to four non\-core datapoints whose task\-success status changed betweenπk−1\\pi\_\{k\-1\}andπk\\pi\_\{k\}; any unfilled positions are backfilled from the low\-reward pool\. At iteration 1, the recent slot is initialized by the same backfill rule\.

Static replay uses only𝒟core\\mathcal\{D\}\_\{\\mathrm\{core\}\}and omits both adaptive slots\. RIPPLE\-U and Coverage\-Adaptive use adaptive replay but different aggregators; RIPPLE\-S combines static replay with balanced aggregation\. Because adaptive replay may include training datapoints, it is a mixed stability pool rather than a fully disjoint validation set\. Flow\-HO remains outside the adaptation loop\.

#### B\.3\.3Iteration Procedure

Algorithm[1](https://arxiv.org/html/2609.12127#alg1)specifies one outer iteration, including replay construction, shared\-parent candidate ranking, and advancing\-prefix promotion\. Algorithm[2](https://arxiv.org/html/2609.12127#alg2)composes accepted sequences across iterations, checkpoints after every iteration, and stops after two consecutive iterations with no accepted patch\.

Algorithm 1One RIPPLE adaptation iteration1:Current policy

πk\\pi\_\{k\}; training pool

𝒟train\\mathcal\{D\}\_\{\\mathrm\{train\}\}; replay core

𝒟core\\mathcal\{D\}\_\{\\mathrm\{core\}\}; replay mode

m∈\{Static,Adaptive\}m\\in\\\{\\textsc\{Static\},\\textsc\{Adaptive\}\\\}and history

Hk−1H\_\{k\-1\}; patch library

ℒ\\mathcal\{L\}; aggregator

gg; rollout count

NN; diagnosis budget

BdiagB\_\{\\mathrm\{diag\}\}; tolerances

\(εR,εC\)\(\\varepsilon\_\{R\},\\varepsilon\_\{C\}\); previously accepted patch IDs

A<kA\_\{<k\}\.

2:Next policy

πk\+1\\pi\_\{k\+1\}; accepted sequence

PkP\_\{k\}; updated history

HkH\_\{k\}; iteration record

ℛk\\mathcal\{R\}\_\{k\}\.

3:Phase 1 — WHAT failed: rollout generation and diagnosis

4:for all

d∈𝒟traind\\in\\mathcal\{D\}\_\{\\mathrm\{train\}\}do

5:Sample

τd,n\(k\)∼pθ\(⋅∣d,πk\)\\tau\_\{d,n\}^\{\(k\)\}\\sim p\_\{\\theta\}\(\\cdot\\mid d,\\pi\_\{k\}\)for

n=1,…,Nn=1,\\ldots,N
6:

X¯πk​\(d\)←N−1​∑n=1NX⁡\(τd,n\(k\)\)\\bar\{X\}\_\{\\pi\_\{k\}\}\(d\)\\leftarrow N^\{\-1\}\\sum\_\{n=1\}^\{N\}X\(\\tau\_\{d,n\}^\{\(k\)\}\)for

X∈\{R,C\}X\\in\\\{R,C\\\}
7:

vk​\(d\)←Varn⁡\[R⁡\(τd,n\(k\)\)\]v\_\{k\}\(d\)\\leftarrow\\operatorname\{Var\}\_\{n\}\[R\(\\tau\_\{d,n\}^\{\(k\)\}\)\];

τd−←arg⁡minn⁡R⁡\(τd,n\(k\)\)\\tau\_\{d\}^\{\-\}\\leftarrow\\arg\\min\_\{n\}R\(\\tau\_\{d,n\}^\{\(k\)\}\)
8:endfor

9:

Hk←UpdateReplayHistory​\(Hk−1,\{τd,n\(k\)\}d,n\)H\_\{k\}\\leftarrow\\textsc\{UpdateReplayHistory\}\(H\_\{k\-1\},\\\{\\tau\_\{d,n\}^\{\(k\)\}\\\}\_\{d,n\}\)
10:if

m=Staticm=\\textsc\{Static\}then

11:

𝒟replay\(k\)←𝒟core\\mathcal\{D\}\_\{\\mathrm\{replay\}\}^\{\(k\)\}\\leftarrow\\mathcal\{D\}\_\{\\mathrm\{core\}\}
12:else

13:

\(𝒟hard\(k\),𝒟recent\(k\)\)←RefreshReplay​\(Hk\)\(\\mathcal\{D\}\_\{\\mathrm\{hard\}\}^\{\(k\)\},\\mathcal\{D\}\_\{\\mathrm\{recent\}\}^\{\(k\)\}\)\\leftarrow\\textsc\{RefreshReplay\}\(H\_\{k\}\)
14:

𝒟replay\(k\)←𝒟core∪𝒟hard\(k\)∪𝒟recent\(k\)\\mathcal\{D\}\_\{\\mathrm\{replay\}\}^\{\(k\)\}\\leftarrow\\mathcal\{D\}\_\{\\mathrm\{core\}\}\\cup\\mathcal\{D\}\_\{\\mathrm\{hard\}\}^\{\(k\)\}\\cup\\mathcal\{D\}\_\{\\mathrm\{recent\}\}^\{\(k\)\}
15:endif

16:

𝒯k←\{τd−:d∈TopBdiag⁡\(vk\)\}\\mathcal\{T\}\_\{k\}\\leftarrow\\\{\\tau\_\{d\}^\{\-\}:d\\in\\operatorname\{Top\}\_\{B\_\{\\mathrm\{diag\}\}\}\(v\_\{k\}\)\\\}
17:for all

τ∈𝒯k\\tau\\in\\mathcal\{T\}\_\{k\}do

18:

\(ℱ⁡\(τ\),z⁡\(τ\)\)←Diagnose​\(τ\)\(\\mathcal\{F\}\(\\tau\),z\(\\tau\)\)\\leftarrow\\textsc\{Diagnose\}\(\\tau\)
19:endfor

20:Phase 2 — WHERE to edit: segment\-typed patch construction

21:for all

τ∈𝒯k\\tau\\in\\mathcal\{T\}\_\{k\}do

22:

𝒫⁡\(τ\)←∅\\mathcal\{P\}\(\\tau\)\\leftarrow\\varnothing
23:for all

f∈ℱ⁡\(τ\)f\\in\\mathcal\{F\}\(\\tau\)do

24:

s←Localize​\(f,z​\(τ\)\)s\\leftarrow\\textsc\{Localize\}\(f,z\(\\tau\)\);

a←Retrieve​\(ℒ,f,s\)a\\leftarrow\\textsc\{Retrieve\}\(\\mathcal\{L\},f,s\)
25:

𝒫⁡\(τ\)←𝒫⁡\(τ\)∪\{a\}\\mathcal\{P\}\(\\tau\)\\leftarrow\\mathcal\{P\}\(\\tau\)\\cup\\\{a\\\}
26:endfor

27:endfor

28:

ck\(a\)←∑τ∈𝒯k𝟏\[a∈𝒫\(τ\)\]c\_\{k\}\(a\)\\leftarrow\\sum\_\{\\tau\\in\\mathcal\{T\}\_\{k\}\}\\mathbf\{1\}\[a\\in\\mathcal\{P\}\(\\tau\)\]for each nominated patch

aa
29:

𝒜k←CoverageFirstOrder​\(\{a∈⋃τ𝒫⁡\(τ\):id⁡\(a\)∉A<k\},ck\)\\mathcal\{A\}\_\{k\}\\leftarrow\\textsc\{CoverageFirstOrder\}\(\\\{a\\in\\bigcup\_\{\\tau\}\\mathcal\{P\}\(\\tau\):\\operatorname\{id\}\(a\)\\notin A\_\{<k\}\\\},c\_\{k\}\)
30:Phase 3 — WHETHER to persist: shared\-parent ranking and replay promotion

31:for all

a∈𝒜ka\\in\\mathcal\{A\}\_\{k\}do

32:

πka←πk⊕a\\pi\_\{k\}^\{a\}\\leftarrow\\pi\_\{k\}\\oplus a
33:

Δtrain​\(a\)←V^R​\(πka,𝒟train\)−V^R​\(πk,𝒟train\)\\Delta\_\{\\mathrm\{train\}\}\(a\)\\leftarrow\\widehat\{V\}\_\{R\}\(\\pi\_\{k\}^\{a\};\\mathcal\{D\}\_\{\\mathrm\{train\}\}\)\-\\widehat\{V\}\_\{R\}\(\\pi\_\{k\};\\mathcal\{D\}\_\{\\mathrm\{train\}\}\)
34:endfor

35:

𝒜k\+←\{a∈𝒜k:Δtrain​\(a\)\>0\}\\mathcal\{A\}\_\{k\}^\{\+\}\\leftarrow\\\{a\\in\\mathcal\{A\}\_\{k\}:\\Delta\_\{\\mathrm\{train\}\}\(a\)\>0\\\}, ordered by decreasing

Δtrain\\Delta\_\{\\mathrm\{train\}\}
36:

Pk←\(\)P\_\{k\}\\leftarrow\(\);

ℛk←∅\\mathcal\{R\}\_\{k\}\\leftarrow\\varnothing
37:for

j=1,…,\|𝒜k\+\|j=1,\\ldots,\|\\mathcal\{A\}\_\{k\}^\{\+\}\|do

38:

aj←𝒜k\+​\[j\]a\_\{j\}\\leftarrow\\mathcal\{A\}\_\{k\}^\{\+\}\[j\];

Pk<j←PkP\_\{k\}^\{<j\}\\leftarrow P\_\{k\}
39:

πk<j←πk⊕Pk<j\\pi\_\{k\}^\{<j\}\\leftarrow\\pi\_\{k\}\\oplus P\_\{k\}^\{<j\};

π~k,j←πk<j⊕aj\\widetilde\{\\pi\}\_\{k,j\}\\leftarrow\\pi\_\{k\}^\{<j\}\\oplus a\_\{j\}
40:for

X∈\{R,C\}X\\in\\\{R,C\\\}do

41:

ΔXg​\(aj∣Pk<j;𝒟replay\(k\)\)←V^Xg​\(π~k,j,𝒟replay\(k\)\)−V^Xg​\(πk<j,𝒟replay\(k\)\)\\Delta\_\{X\}^\{g\}\(a\_\{j\}\\mid P\_\{k\}^\{<j\};\\mathcal\{D\}\_\{\\mathrm\{replay\}\}^\{\(k\)\}\)\\leftarrow\\widehat\{V\}\_\{X\}^\{g\}\(\\widetilde\{\\pi\}\_\{k,j\};\\mathcal\{D\}\_\{\\mathrm\{replay\}\}^\{\(k\)\}\)\-\\widehat\{V\}\_\{X\}^\{g\}\(\\pi\_\{k\}^\{<j\};\\mathcal\{D\}\_\{\\mathrm\{replay\}\}^\{\(k\)\}\)
42:endfor

43:if

ΔRg​\(aj∣Pk<j\)≥εR\\Delta\_\{R\}^\{g\}\(a\_\{j\}\\mid P\_\{k\}^\{<j\}\)\\geq\\varepsilon\_\{R\}and

ΔCg​\(aj∣Pk<j\)≥εC\\Delta\_\{C\}^\{g\}\(a\_\{j\}\\mid P\_\{k\}^\{<j\}\)\\geq\\varepsilon\_\{C\}then

44:

Pk←Pk∥\(aj\)P\_\{k\}\\leftarrow P\_\{k\}\\mathbin\{\\\|\}\(a\_\{j\}\)⊳\\trianglerightadvance the replay baseline

45:Log

\(aj,ΔRg,ΔCg,Accept\)\(a\_\{j\},\\Delta\_\{R\}^\{g\},\\Delta\_\{C\}^\{g\},\\textsc\{Accept\}\)in

ℛk\\mathcal\{R\}\_\{k\}
46:else

47:Log

\(aj,ΔRg,ΔCg,Reject\)\(a\_\{j\},\\Delta\_\{R\}^\{g\},\\Delta\_\{C\}^\{g\},\\textsc\{Reject\}\)in

ℛk\\mathcal\{R\}\_\{k\}
48:endif

49:endfor

50:

πk\+1←πk⊕Pk\\pi\_\{k\+1\}\\leftarrow\\pi\_\{k\}\\oplus P\_\{k\}
51:return

\(πk\+1,Pk,Hk,ℛk\)\(\\pi\_\{k\+1\},P\_\{k\},H\_\{k\},\\mathcal\{R\}\_\{k\}\)

Algorithm 2Multi\-iteration RIPPLE with checkpointing and patience stopping1:Base policy

π0\\pi\_\{0\}; training pool

𝒟train\\mathcal\{D\}\_\{\\mathrm\{train\}\}; replay core

𝒟core\\mathcal\{D\}\_\{\\mathrm\{core\}\}; replay mode

mm; patch library

ℒ\\mathcal\{L\}; aggregator

gg; rollout count

NN; diagnosis budget

BdiagB\_\{\\mathrm\{diag\}\}; tolerances

\(εR,εC\)\(\\varepsilon\_\{R\},\\varepsilon\_\{C\}\); maximum iterations

KmaxK\_\{\\max\}\.

2:Final policy

πfinal\\pi\_\{\\mathrm\{final\}\}and ordered patch sequence

PP\.

3:Initialize persistent state

4:

P←\(\)P\\leftarrow\(\);

A<0←∅A\_\{<0\}\\leftarrow\\varnothing;

H−1←∅H\_\{\-1\}\\leftarrow\\varnothing;

u←0u\\leftarrow 0⊳\\trianglerightconsecutive no\-update iterations

5:Greedy outer\-loop policy improvement

6:for

k=0,…,Kmax−1k=0,\\ldots,K\_\{\\max\}\-1do

7:

\(πk\+1,Pk,Hk,ℛk\)←RIPPLE\-Iter​\(πk,𝒟train,𝒟core,m,Hk−1,ℒCLOSE,\(\\pi\_\{k\+1\},P\_\{k\},H\_\{k\},\\mathcal\{R\}\_\{k\}\)\\leftarrow\\textsc\{RIPPLE\-Iter\}\\big\(\\pi\_\{k\},\\mathcal\{D\}\_\{\\mathrm\{train\}\},\\mathcal\{D\}\_\{\\mathrm\{core\}\},m,H\_\{k\-1\},\\mathcal\{L\},
8:

OPENg,N,Bdiag,εR,εC,A<k\)g,N,B\_\{\\mathrm\{diag\}\},\\varepsilon\_\{R\},\\varepsilon\_\{C\},A\_\{<k\}\\big\)⊳\\trianglerightAlgorithm[1](https://arxiv.org/html/2609.12127#alg1)

9:

P←P∥PkP\\leftarrow P\\mathbin\{\\\|\}P\_\{k\};

A<k\+1←A<k∪\{id⁡\(a\):a∈Pk\}A\_\{<k\+1\}\\leftarrow A\_\{<k\}\\cup\\\{\\operatorname\{id\}\(a\):a\\in P\_\{k\}\\\}
10:PersistCheckpoint\(k\+1,πk\+1,P,ℛk\)\(k\+1,\\pi\_\{k\+1\},P,\\mathcal\{R\}\_\{k\}\)

11:if

Pk=∅P\_\{k\}=\\varnothingthen

12:

u←u\+1u\\leftarrow u\+1
13:else

14:

u←0u\\leftarrow 0
15:endif

16:if

u≥2u\\geq 2then

17:break

18:endif

19:endfor

20:

πfinal←π0⊕P\\pi\_\{\\mathrm\{final\}\}\\leftarrow\\pi\_\{0\}\\oplus P
21:return

\(πfinal,P\)\(\\pi\_\{\\mathrm\{final\}\},P\)

#### B\.3\.4Reward Function

##### Workflow modification\.

The modification reward is

Rmod​\(τ\)=0\.3​S​\(τ\)\+0\.3​C​\(τ\)\+0\.2​S​\(τ\)​C​\(τ\)\+0\.1​E​\(τ\)−0\.1​K​\(τ\)\.R\_\{\\mathrm\{mod\}\}\(\\tau\)=0\.3S\(\\tau\)\+0\.3C\(\\tau\)\+0\.2S\(\\tau\)C\(\\tau\)\+0\.1E\(\\tau\)\-0\.1K\(\\tau\)\.The interaction term rewards workflows that are both correct and service\-validated; edit efficiency and execution cost provide lighter shaping\.

##### Component definitions\.

S⁡\(τ\)∈\{0,1\}S\(\\tau\)\\in\\\{0,1\\\}equals one only when the agent produces a candidate workflow with at leastmax⁡\(⌊ncor/2⌋,3\)\\max\(\\lfloor n\_\{\\mathrm\{cor\}\}/2\\rfloor,3\)actions, wherencorn\_\{\\mathrm\{cor\}\}is the action count of the corrupted input, and a post\-hoc create\-and\-validate operation succeeds\. The temporary workflow is deleted immediately after evaluation\.

C⁡\(τ\)∈\[0,1\]C\(\\tau\)\\in\[0,1\]is a directed field\-level comparison with the ground truth, keyed by action identifier and covering action type, ground\-truth parameter keys, and all base, conditional, and error transitions\. Missing actions mismatch all corresponding fields; extra actions receive a fixed three\-field penalty; empty\-workflow edge cases are handled explicitly\.

E⁡\(τ\)=1−UCR⁡\(τ\)E\(\\tau\)=1\-\\operatorname\{UCR\}\(\\tau\), where the unnecessary\-change ratio counts generated actions absent from both the corrupted and ground\-truth workflows, edits to actions that were already correct, and deletions of ground\-truth actions\. A trajectory with no edits receivesE=1E=1; success and correctness separately penalize failure to implement the request\.

Execution cost combines interaction and token components:

I⁡\(τ\)\\displaystyle I\(\\tau\)=α​min⁡\(turns⁡\(τ\)Tmax,1\)\+\(1−α\)​min⁡\(tools⁡\(τ\)Umax,1\),\\displaystyle=\\alpha\\min\\\!\\left\(\\frac\{\\operatorname\{turns\}\(\\tau\)\}\{T\_\{\\max\}\},1\\right\)\+\(1\-\\alpha\)\\min\\\!\\left\(\\frac\{\\operatorname\{tools\}\(\\tau\)\}\{U\_\{\\max\}\},1\\right\),K⁡\(τ\)\\displaystyle K\(\\tau\)=β​min⁡\(tokens⁡\(τ\)Cmax,1\)\+\(1−β\)​I​\(τ\),\\displaystyle=\\beta\\min\\\!\\left\(\\frac\{\\operatorname\{tokens\}\(\\tau\)\}\{C\_\{\\max\}\},1\\right\)\+\(1\-\\beta\)I\(\\tau\),withTmax=20T\_\{\\max\}=20,Umax=50U\_\{\\max\}=50,Cmax=100,000C\_\{\\max\}=100\{,\}000,α=0\.4\\alpha=0\.4, andβ=0\.7\\beta=0\.7\. Rollouts are operationally capped at 15 turns; the larger normalization constants leave headroom below saturation\.

##### Workflow generation\.

Generation uses

Rgen​\(τ\)=Cgen​\(τ\)−0\.1​K​\(τ\),R\_\{\\mathrm\{gen\}\}\(\\tau\)=C\_\{\\mathrm\{gen\}\}\(\\tau\)\-0\.1K\(\\tau\),whereCgenC\_\{\\mathrm\{gen\}\}is identifier\-invariant and combines square\-root multiset\-F1 scores over action types, edges, parameter keys, and normalized parameter values\. Dynamically generated identifiers are replaced with placeholders before comparison\. Service validation is reported separately but does not enterRgenR\_\{\\mathrm\{gen\}\};EEis fixed at one for bookkeeping\.

#### B\.3\.5Rollout Configuration and Uncertainty

##### Backbones and runtime\.

Each experiment fixes both the hosted model endpoint and the tool\-enabled runtime\. Haiku 4\.5 is the principal backbone; Gemma 3\-12B\-it and Ministral 3\-14B are used only for cross\-backbone transfer\. All backbones share the same base policyπ0\\pi\_\{0\}and patch libraryℒ\\mathcal\{L\}\.

##### User simulator\.

When the agent asks a clarification question during a rollout, a deterministic user simulator answers on behalf of the requester\. The simulator is a Python program that consults each task’s ground\-truth metadata for the missing\-information slots and refusal conditions defined in the benchmark, and returns the corresponding answer verbatim; it does not paraphrase, does not volunteer information beyond what was asked, and makes no LLM calls\. This lets the agent exercise theCLARIFYsegment during training and diagnosis while keeping user responses reproducible across rollouts and backbones\. The diagnosis module that maps trajectories to failure families is also a deterministic Python program\.

##### Tools and environment\.

The agent can read, write, edit, and search files; validate workflows through the same create\-and\-validate path used to defineSS; and resolve environment resources such as routing targets, functions, bots, prompts, schedules, reusable workflows, and integrations\. Rollouts execute in a dedicated evaluation environment, and temporary validation resources are deleted after use\.

##### Rollout execution and caching\.

A rollout is capped at 15 agent turns, with per\-turn and total timeouts\. It terminates after a completed validation path or after two consecutive turns without tool calls, and one retry is permitted for a latency\-only stall\. We retain each endpoint’s default sampling parameters\. Rollouts are cached by\(example\_id,policy\_hash,run\_idx\)\(\\text\{example\\\_id\},\\text\{policy\\\_hash\},\\text\{run\\\_idx\}\), so any policy edit invalidates the corresponding cached trajectory\.

##### Environment confounds\.

Tool incompleteness, ambiguous resource descriptions, and systematic infrastructure failures are not modeled separately\. They can therefore appear as F2 symptoms even when no prompt patch can correct the underlying environment issue\.

##### Family\-cluster bootstrap for Haiku deltas\.

Haiku deltas are paired by Flow\-HO datapoint\. ForX∈\{S,C,R\}X\\in\\\{S,C,R\\\}, defineΔ​X​\(d\)=X¯system​\(d\)−X¯ref​\(d\)\\Delta X\(d\)=\\bar\{X\}\_\{\\mathrm\{system\}\}\(d\)\-\\bar\{X\}\_\{\\mathrm\{ref\}\}\(d\), where each bar averages theN=3N=3rollouts\. The 39 paired differences are grouped into four ground\-truth workflow families\. Each of10,00010\{,\}000bootstrap replicates resamples four family labels with replacement, includes every paired datapoint from each sampled family, and computes the pooled datapoint mean\. Repeated labels therefore repeat the entire cluster\. The reported interval is the 2\.5th–97\.5th percentile range\.

Resampling complete clusters preserves within\-family dependence; because family sizes differ, the pooled statistic remains datapoint\-weighted\. The principal Haiku deltas and their family\-cluster95%95\\%intervals are summarized in Table[6](https://arxiv.org/html/2609.12127#A2.T6)\.

Table 6:Family\-cluster bootstrap95%95\\%intervals for the principal Haiku deltas relative toπ0\\pi\_\{0\}\(10,00010\{,\}000replicates over four ground\-truth workflow\-family clusters\)\.As shown in Table[6](https://arxiv.org/html/2609.12127#A2.T6), both the service\-validation success gain \(\+23\.1\+23\.1pp\) and the composite reward gain \(\+0\.119\+0\.119\) have intervals that exclude zero, whereas the workflow\-correctness gain remains unresolved\. With only four clusters, these intervals should be interpreted as descriptive family\-cluster uncertainty for the current benchmark, not as a population\-level guarantee or an independent confirmation result\.

### B\.4Haiku Principal\-Study Supplementary Results

This section provides the supplementary analyses for the Haiku RQ1–RQ4 study in §[4](https://arxiv.org/html/2609.12127#S4)\. Cross\-backbone results for Gemma and Ministral appear in Appendix[B\.5](https://arxiv.org/html/2609.12127#A2.SS5)\.

Table 7:Flow\-HO modification reward components\.S,C,E,S,C,E,andKKare percentages; lowerKKis better\.RRuses the\[0,1\]\[0,1\]components in Appendix[B\.3\.4](https://arxiv.org/html/2609.12127#A2.SS3.SSS4)\.‡Skill\-Compact uses a separate acceptor\-probe cache and is descriptive\.#### B\.4\.1Reward\-component breakdown and family\-level consistency

Table[7](https://arxiv.org/html/2609.12127#A2.T7)decomposes the Flow\-HO modification reward across the harmonized Haiku systems\. RIPPLE\-U attains the highest validation success, edit efficiency, and composite reward, together with the lowest execution cost; RIPPLE\-S attains the highest workflow correctness\. Skill\-Compact is included only as a descriptive checkpoint from the separate acceptor\-probe cache\.

Figure[3](https://arxiv.org/html/2609.12127#A2.F3)plots per\-family mean reward with1\.96×SEM1\.96\\times\\mathrm\{SEM\}intervals, and Table[8](https://arxiv.org/html/2609.12127#A2.T8)gives the underlying numeric means and rollout counts\. Their source caches and rollout counts differ: the base usesN=3N=3from the acceptor\-probe cache, whereas RIPPLE\-U and RIPPLE\-S useN=10N=10from the design\-study cache\. The comparison is therefore descriptive and supports family\-level directionality, not paired inference\.

Figure 3:Mean reward by Flow\-HO modification family in the Haiku study\. Error bars show1\.96×SEM1\.96\\times\\mathrm\{SEM\}over task\-level means; the comparison is descriptive because caches and rollout counts differ\.Table 8:Per\-family reward means with1\.96×SEM1\.96\\times\\mathrm\{SEM\}intervals\. Base usesN=3N=3from the acceptor\-probe cache; RIPPLE\-U and RIPPLE\-S useN=10N=10from the design\-study cache, so the intervals are descriptive rather than harmonized and paired\.Figure 4:Accepted\-patch count versus Flow\-HO modification reward\. Error bars show±1\\pm 1SEM across 39 held\-out tasks; points with equal counts are offset only for visibility\. Skill\-Compact is descriptive because it uses the acceptor\-probe cache\.Figure 5:Coverage\-Adaptive convergence\. \(A\) Replay score after each accepted patch; shading marks iteration\-2 acceptances\. \(B\) Flow\-HO scores at selected checkpoints\. The dashed RIPPLE\-U line follows a different four\-patch sequence\.
#### B\.4\.2Convergence and checkpoint complementarity

Figure[4](https://arxiv.org/html/2609.12127#A2.F4)places the harmonized Haiku systems on the patch\-count–reward plane\. Equal\-size patch sets occupy different quality regimes, and the six\-patch Multi\-Iteration policy trails the principal variants\. Figure[5](https://arxiv.org/html/2609.12127#A2.F5)traces Coverage\-Adaptive through iterations 1 and 2; the dashed line shows the distinct four\-patch RIPPLE\-U sequence\. Table[9](https://arxiv.org/html/2609.12127#A2.T9)records the replay score after each acceptance in the Coverage\-Adaptive and RIPPLE\-U trajectories, including the two iteration\-2 additions \(F6e, F6g\); Table[10](https://arxiv.org/html/2609.12127#A2.T10)reports the paired Flow\-HO reward, correctness, and validation\-success values at the same checkpoints\. Together they show that iteration\-2 acceptances raise the replay score but do not carry over to Flow\-HO\.

Table 9:Replay scores along the Coverage\-Adaptive accepted sequence \(Figure[5](https://arxiv.org/html/2609.12127#A2.F5)A\)\. Each row is the advancing baseline after the listed patch is accepted\.Table 10:Flow\-HO modification scores at selected Coverage\-Adaptive checkpoints \(Figure[5](https://arxiv.org/html/2609.12127#A2.F5)B;n=39n=39,N=3N=3\)\. RIPPLE\-U is a reference with a different four\-patch sequence\.Figure[6](https://arxiv.org/html/2609.12127#A2.F6)shows request\-level complementarity between Skill\-Compact and Coverage\-Adaptive\. Under strict sign counting, Skill\-Compact is higher on67%67\\%of tasks\. With the predefined\|Δ​R\|≤0\.02\|\\Delta R\|\\leq 0\.02tie band, the counts are 15 compact wins, 9 broader\-checkpoint wins, and 15 ties\. A post hoc oracle that chooses the better checkpoint per request raises mean reward from0\.6250\.625to0\.6650\.665; this is an upper bound, not a deployable routing rule\. A request\-only selector usingqqandy0y\_\{0\}recovers none of this headroom under leave\-one\-family\-out evaluation\. A diagnostic selector recovers61%61\\%but observes the ground\-truth workflow and is therefore nondeployable\.

Figure 6:ECDF of task\-level reward differences, Skill\-Compact minus Coverage\-Adaptive \(n=39n=39\)\. Positive values favor the compact checkpoint; the sign change shows that neither checkpoint dominates all requests\.Table 11:Patch outcomes by configuration\.A: accepted;×\\times: replay rejection;−Δ\-\\Delta: training pre\-filter rejection; blank: not proposed\. Train\-Only bypasses replay promotion\.
#### B\.4\.3Cross\-configuration patch profile

Table[11](https://arxiv.org/html/2609.12127#A2.T11)records each proposed patch and its terminal status across Haiku configurations\. The three one\-iteration gated variants each accept four patches, but the sets differ: F3a is accepted by Coverage\-Adaptive and RIPPLE\-U but rejected under static replay; F2a is rejected by Coverage\-Adaptive but accepted by RIPPLE\-U and RIPPLE\-S; and F5a is accepted only by RIPPLE\-S among the gated one\-iteration variants\. Train\-Only accepts six patches because replay cannot remove positive\-training\-gain candidates\. Multi\-Iteration retains the four Coverage\-Adaptive patches and adds F6e and F6g in iteration 2\. Similar patch counts therefore do not imply similar policies or held\-out performance\.

### B\.5Cross\-Backbone Transfer Supplementary Results

This section provides the supplementary tables for the Gemma and Ministral transfer runs summarized in §[4\.2](https://arxiv.org/html/2609.12127#S4.SS2)\. Both use the sameN=3N=3rollout protocol and patch library as the Haiku study\. Ministral additionally bypasses the training\-side positive\-gain pre\-filter \(Appendix[B\.5\.5](https://arxiv.org/html/2609.12127#A2.SS5.SSS5)\)\.

#### B\.5\.1Backbone tier bracket

Table[12](https://arxiv.org/html/2609.12127#A2.T12)reports the base\-π0\\pi\_\{0\}pilot on six frozen backbones, evaluated on the same 39 Flow\-HO modification tasks withN=3N=3\. For descriptive model selection, we define peer \(S≥40%S\\geq 40\\%\), upper\-headroom \(30%≤S<40%30\\%\\leq S<40\\%\), lower\-headroom \(20%≤S<30%20\\%\\leq S<30\\%\), and collapse \(S<20%S<20\\%\) bands\. The study selects one backbone from each non\-collapse band\. Collapse\-band backbones are excluded because the base policy provides too little reliable execution capability for the present adaptation study\.

Table 12:Base\-π0\\pi\_\{0\}pilot on six frozen backbones \(39 Flow\-HO tasks,N=3N=3\)\. Descriptive success bands are peer \(S≥40%S\\geq 40\\%\), upper\-headroom \(30%≤S<40%30\\%\\leq S<40\\%\), lower\-headroom \(20%≤S<30%20\\%\\leq S<30\\%\), and collapse \(S<20%S<20\\%\); one backbone is selected from each non\-collapse band\.
#### B\.5\.2Cross\-backbone reward\-component breakdown

Table[13](https://arxiv.org/html/2609.12127#A2.T13)decomposes Flow\-HO modification reward for Gemma and Ministral\. On Gemma, validation success and reward increase while correctness, edit efficiency, and execution cost remain nearly unchanged\. On Ministral, validation success and reward also increase and edit efficiency improves slightly; execution cost rises, primarily through the token component, as the accepted patches lengthen the segmented prompt\.

Table 13:Cross\-backbone reward components on Flow\-HO modification \(n=39n=39,N=3N=3\)\.S,C,E,S,C,E,andKKare percentages; lowerKKis better\.RRfollows Appendix[B\.3\.4](https://arxiv.org/html/2609.12127#A2.SS3.SSS4), andboldmarks the better mean within each backbone\.
#### B\.5\.3Cross\-backbone family\-level consistency

Table[14](https://arxiv.org/html/2609.12127#A2.T14)reports means for the same four Flow\-HO modification families used in the Haiku analysis\. Validation success improves in three of four families on each backbone, and the regressing family differs: family C for Gemma and family B for Ministral\. Family D shows the largest reward gain on both backbones, consistent with recovering headroom where the base policy is weakest\. Because each transfer run uses one frozen protocol, these values support descriptive family\-level directionality rather than paired inference\.

Table 14:Per\-family Flow\-HO modification means for Gemma 12B and Ministral 14B \(N=3N=3\)\. Validation success improves in three of four families on each backbone; the regressing family differs across backbones\. Family D has the largest reward gain for both\.
#### B\.5\.4Accepted patches and per\-patch replay evidence

Table[15](https://arxiv.org/html/2609.12127#A2.T15)lists accepted patches in promotion order\. The transfer policies are largely backbone\-specific: Gemma accepts onePLANpatch and twoEDITpatches, whereas Ministral accepts oneREQ\_UNDERSTANDINGpatch and twoEDITpatches\. The shared concentration onEDITreflects recurring schema and parameter failures; the remaining segment differs with the diagnosed backbone\-specific failure profile\.

BackboneConfigurationAccepted patchesSegment sequenceHaiku 4\.5RIPPLE\-UF2a, F3a, F5b, F6bTOOL, EDIT, EDIT, REQHaiku 4\.5RIPPLE\-SF2a, F5a, F6c, F6dTOOL, EDIT, TOOL, EDITHaiku 4\.5Coverage\-AdaptiveF3a, F6c, F6b, F6dEDIT, TOOL, REQ, EDITGemma 3\-12BRIPPLEF6a, F3a, F3bPLAN, EDIT, EDITMinistral 14B∗RIPPLEF6g, F3d, F3eREQ, EDIT, EDITTable 15:Accepted patches in promotion order\. REQ and TOOL abbreviateREQ\_UNDERSTANDINGandTOOL\_USE\.∗Ministral bypasses the training pre\-filter; see Appendix[B\.5\.5](https://arxiv.org/html/2609.12127#A2.SS5.SSS5)\.Table[16](https://arxiv.org/html/2609.12127#A2.T16)reports the training and replay deltas behind these decisions\. On Gemma, the replay gate rejects F5a in iteration 2 despite a positiveΔtrain\\Delta\_\{\\mathrm\{train\}\}, paralleling the replay\-side discrimination in the Haiku F2a case study \(§[4\.4](https://arxiv.org/html/2609.12127#S4.SS4)\)\. On Ministral, the three highest\-ranked candidates in the diagnosis\-matched pool are evaluated after bypassing the training pre\-filter; all three satisfy the unchanged two\-signal replay gate\.

BackboneIter\.PatchSegmentΔtrain\\Delta\_\{\\mathrm\{train\}\}ΔRg\\Delta\_\{R\}^\{g\}ΔCg\\Delta\_\{C\}^\{g\}ΔSg\\Delta\_\{S\}^\{g\}OutcomeGemma 12B1F6aPLAN\+0\.066\+0\.066\+0\.013\+0\.013\+0\.029\+0\.029\+0\.000\+0\.000acceptedGemma 12B1F5aEDIT−0\.016\-0\.016–––train pre\-filterGemma 12B1F3aEDIT−0\.079\-0\.079–––train pre\-filterGemma 12B2F3aEDIT\+0\.032\+0\.032\+0\.038\+0\.038\+0\.004\+0\.004\+0\.077\+0\.077acceptedGemma 12B2F3bEDIT\+0\.007\+0\.007\+0\.082\+0\.082\+0\.033\+0\.033\+0\.154\+0\.154acceptedGemma 12B2F5aEDIT\+0\.006\+0\.006−0\.074\-0\.074–−0\.154\-0\.154replay rejectMinistral 14B∗1F6gREQ\_UNDERSTANDING−0\.016\-0\.016\+0\.033\+0\.033\+0\.057\+0\.057\+0\.026\+0\.026acceptedMinistral 14B∗1F3dEDIT−0\.035\-0\.035\+0\.007\+0\.007−0\.031\-0\.031\+0\.026\+0\.026acceptedMinistral 14B∗1F3eEDIT−0\.067\-0\.067\+0\.014\+0\.014\+0\.047\+0\.047\+0\.000\+0\.000accepted

Table 16:Per\-patch training and replay deltas for Gemma and Ministral\. Training gain is measured againstπk\\pi\_\{k\}on the diagnosis pool; replay deltas use the current accepted prefix\. Dashes denote uncomputed or unrecorded quantities\. Gemma F5a is rejected by replay despite positive training gain\.∗Ministral bypasses the training pre\-filter \(Appendix[B\.5\.5](https://arxiv.org/html/2609.12127#A2.SS5.SSS5)\); the replay gate is unchanged\.
#### B\.5\.5Ministral protocol: bypassing the training pre\-filter

RIPPLE’s persistence decision is the replay\-side AND rule,ΔRg≥−0\.05\\Delta\_\{R\}^\{g\}\\geq\-0\.05andΔCg≥−0\.10\\Delta\_\{C\}^\{g\}\\geq\-0\.10\. The training conditionΔtrain\>0\\Delta\_\{\\mathrm\{train\}\}\>0is a compute\-saving pre\-filter that avoids replay evaluation for candidates with a negative training estimate\. Every accepted Haiku and Gemma patch already satisfies this condition, so the pre\-filter does not change their accepted sets\.

For Ministral, all six candidates examined across two candidate pools have training deltas in\[−0\.10,0\]\[\-0\.10,0\]\. With the pre\-filter enabled, no candidate reaches replay and the policy remainsπ0\\pi\_\{0\}\. In the reported run, we bypass the pre\-filter for the final diagnosis\-matched top three and apply the unchanged replay gate\. F6g, F3d, and F3e are accepted sequentially with the deltas in Table[17](https://arxiv.org/html/2609.12127#A2.T17)\. The three candidates from the earlier pool were not reevaluated under the bypassed protocol, so the trace does not establish how replay would have classified them\.

PatchSegment𝚫train\\bm\{\\Delta\}\_\{\\textbf\{train\}\}𝚫Rg\\bm\{\\Delta\}\_\{R\}^\{g\}𝚫Cg\\bm\{\\Delta\}\_\{C\}^\{g\}𝚫Sg\\bm\{\\Delta\}\_\{S\}^\{g\}OutcomeReported diagnosis\-matched pool; pre\-filter bypassedF6gREQ−0\.016\-0\.016\+0\.033\+0\.033\+0\.057\+0\.057\+0\.026\+0\.026acceptedF3dEDIT−0\.035\-0\.035\+0\.007\+0\.007−0\.031\-0\.031\+0\.026\+0\.026acceptedF3eEDIT−0\.067\-0\.067\+0\.014\+0\.014\+0\.047\+0\.047\+0\.000\+0\.000acceptedEarlier pool; pre\-filter enabledF5aEDIT−0\.037\-0\.037not replay\-evaluatedscreenedF2aTOOL−0\.048\-0\.048not replay\-evaluatedscreenedF3aEDIT−0\.097\-0\.097not replay\-evaluatedscreenedTable 17:Ministral candidate outcomes\. The reported run bypasses the positive\-training\-gain pre\-filter for the final three candidates and retains the advancing replay gate\. Earlier candidates were screened under the default pre\-filter and were not replay\-classified under the bypassed protocol\. REQ and TOOL abbreviateREQ\_UNDERSTANDINGandTOOL\_USE\.

## Appendix CDiscussion

### C\.1Why the replay gate uses two signals

RIPPLE promotes a candidate only when replay reward and workflow correctness both remain within their tolerances\. Composite reward alone is insufficient because gains in validation success, edit efficiency, or execution cost can compensate for a correctness loss\. Conversely, a structurally closer workflow can still fail service validation\. Separate constraints keep both dimensions visible rather than permitting unrestricted compensation within one scalar\.

The gate uses an AND rule because each signal defines an independent admissibility condition; an OR rule would allow an arbitrarily large regression in one metric whenever the other passed\. The thresholds are mildly negative rather than strict\-improvement tests because replay effects are estimated from few rollouts on a small pool\. The gate is therefore a bounded empirical safeguard, not a guarantee of monotone improvement\.

##### Marginal rather than cumulative control\.

Each decision is relative to the latest accepted prefix, not to the iteration\-start policy\. Small admissible regressions can therefore accumulate\. In the Coverage\-Adaptive trace, F3a raises replay reward from0\.4310\.431to0\.5030\.503, after which the next three accepted patches reduce it to0\.4960\.496,0\.4560\.456, and0\.4430\.443\. Every marginal change remains aboveεR=−0\.05\\varepsilon\_\{R\}=\-0\.05, and the final checkpoint remains above the iteration\-start policy, but it falls below the one\-patch checkpoint\. A cumulative budget or an additional constraint againstπk\\pi\_\{k\}would provide stronger control when sequence\-level monotonicity is required\.

### C\.2What “coverage\-weighted” aggregation means

The implementation label*coverage\-weighted*denotesgbalancedg\_\{\\mathrm\{balanced\}\}: datapoints are averaged within each ground\-truth workflow family, and represented families are then weighted equally\. It does not refer to coverage over failure families or accepted patch types\.

The two notions of family are distinct\. Aground\-truth workflow familygroups structurally related tasks and defines replay aggregation and cluster resampling\. Afailure family\(F1–F6\) is inferred from trajectory evidence and indexes candidate patches\. Candidate quotas and coverage\-first ordering operate on failure families;gbalancedg\_\{\\mathrm\{balanced\}\}operates on workflow families\.

### C\.3When the correctness constraint becomes binding

Across 20 logged candidate decisions in the four principal modification runs, seven candidates are rejected: two by reward alone and five by both reward and correctness\. No candidate fails only the correctness condition\. For this candidate set and these thresholds, a reward\-only replay gate would therefore reproduce the same decisions\. This differs from Train\-Only, which removes replay promotion entirely\.

In the present trace, the correctness constraint is a latent safeguard rather than an additional source of rejections\. Its structural motivation remains: composite reward can tolerate a correctness decline when other components improve\. Demonstrating incremental decision value, however, requires candidate trajectories on which the reward and correctness conditions disagree\.

## Appendix DSegmented Prompt Policy: Base Text and Accepted Deltas

For auditability, this section shows the segment\-level changes in the four\-patch Coverage\-Adaptive checkpoint, whose accepted sequence is F3a, F6c, F6b, and F6d\. The originalπ0\\pi\_\{0\}text is preserved within each segment, and every patch appends one bounded instruction to its designated location\. RIPPLE\-U and RIPPLE\-S select different four\-patch sequences and are not shown here\.

Only the modified segments—REQ\_UNDERSTANDING,TOOL\_USE, andEDIT—are included; the remaining four segments are identical toπ0\\pi\_\{0\}\. Service\-specific names are replaced with functional aliases for presentation without changing the policy logic\.

### D\.1REQ\_UNDERSTANDING \(modified by F6b\)

F6b is appended to the unchanged request\-understanding segment\.

REQ\_UNDERSTANDING: Base Policy and RIPPLE DeltaBase policyπ0\\pi\_\{0\}\[REQ\_UNDERSTANDING\_START\]When the user provides a flow JSON and a modification request:1\. Read the existing flow JSON carefully to understand its current structure2\. Identify the type of modification requested \(add block, reroute, modifyconfiguration, or replace logic\)3\. Determine which blocks in the flow are likely affected by the request4\. Note the flow’s overall topology: entry point, main paths, error paths,and terminal blocks\[REQ\_UNDERSTANDING\_END\]Accepted patch F6b \(appended\)TRIGGER: when the request names multiple capabilities or features\.Enumerate every capability the request asks for and ensure each one mapsto at least one concrete element in your output with its required settingspopulated\. Treat an unaddressed capability as incomplete work, not anoptional extra\.

Effect\.F6b turns multi\-capability requests into an explicit coverage checklist, reducing partial outputs\.

### D\.2TOOL\_USE \(modified by F6c\)

F6c adds a resolve\-before\-fill requirement to the existing tool\-use contract\.

TOOL\_USE: Base Policy and RIPPLE DeltaBase policyπ0\\pi\_\{0\}\[TOOL\_USE\_START\]Use the available tools to resolve resource identifiers when needed:\- get\_routing\_targets: retrieve identifiers for routing blocks\- get\_functions: retrieve function identifiers for function\-call blocks\- get\_bots / get\_bot\_aliases: retrieve conversational\-bot configuration\- get\_prompts: retrieve prompt identifiers for message blocks\- get\_schedules: retrieve schedule identifiers for schedule\-check blocks\- get\_workflows: retrieve workflow identifiers for workflow\-transfer blocks\- validate\_workflow: validate the final output through the external validator\[TOOL\_USE\_END\]Accepted patch F6c \(appended\)TRIGGER: when an element references an external resource \(routing target, function,bot, prompt, schedule\)\. Resolve the real identifier or value for thatresource before filling the field; do not leave required values blank orsubstitute a placeholder\. If a value cannot be resolved, surface thatrather than emitting an empty field\.

Effect\.F6c requires concrete resource resolution before populating a referenced field and makes unresolved values explicit\.

### D\.3EDIT \(modified by F3a and F6d\)

TheEDITsegment receives F3a and F6d in acceptance order\. Because its base text is unchanged, only the appended instructions are shown\.

EDIT: Accepted RIPPLE DeltasF3a: schema\-complete action insertionBefore adding an Action of a type that has not appeared earlier in the flow,look up the required Parameters and Errors entries in the workflow\-language knowledge base\.Include all required fields\. Do not use placeholders for required fields\.F6d: complete element detailsTRIGGER: when an element has been added but its details may be incomplete\.For each element, complete its outgoing connections \(default, conditional,and error branches\), its metadata, and every required sub\-field for itstype\. A structurally present but under\-specified element is unfinished\.

Effect\.F3a requires a schema lookup before inserting an unseen action type; F6d requires the resulting action, branches, metadata, and required subfields to be complete before editing terminates\.

### D\.4Summary of policy delta

All four updates are additive and segment\-scoped: no base instruction, segment boundary, or surrounding knowledge scaffold is overwritten\. The checkpoint can therefore be reviewed, attributed, and rolled back patch by patch\.

## Appendix ECase Study: A Locally Helpful Patch with a Destructive Interaction

This case follows F2a on the Coverage\-Adaptive trajectory\. The patch instructs the agent to use resource\-lookup tools only for newly introduced references and to preserve existing identifiers that appear valid\. The rule is locally plausible because it can avoid redundant lookups, but its effect depends on the instructions already active in the policy\.

##### Shared\-parent training estimate\.

F2a is evaluated against the iteration\-start policyπk=π0\\pi\_\{k\}=\\pi\_\{0\}and receivesΔtrain=\+0\.0465\\Delta\_\{\\mathrm\{train\}\}=\+0\.0465, passing the positive\-gain pre\-filter\. This estimate supports local utility under the shared parent; it does not test the patch after composition\.

##### Advancing\-prefix replay estimate\.

By promotion time, the accepted prefix is\(F3a,F6c,F6b,F6d\)\(\\mathrm\{F3a\},\\mathrm\{F6c\},\\mathrm\{F6b\},\\mathrm\{F6d\}\)\. Appending F2a yields the marginal replay effects in Table[18](https://arxiv.org/html/2609.12127#A5.T18)\.

Table 18:Marginal replay effect of appending F2a to the four\-patch Coverage\-Adaptive prefix\. Both promotion conditions \(ΔRg≥−0\.05\\Delta\_\{R\}^\{g\}\\geq\-0\.05andΔCg≥−0\.10\\Delta\_\{C\}^\{g\}\\geq\-0\.10\) fail, and service\-validation success falls to zero\.RIPPLE therefore rejects F2a before it enters the persistent checkpoint\.

##### Interaction mechanism\.

F6c requires every referenced resource to be resolved to a concrete value\. F2a narrows that instruction by exempting identifiers already present in the corrupted input\. Such identifiers can be syntactically plausible yet stale in the evaluation environment; preserving them suppresses the revalidation that F6c would otherwise trigger, leading to publication failures\.

##### What the case establishes\.

The logged trace shows that F2a’s estimated effect depends on policy context: it is positive under shared\-parent training evaluation and destructive after the accepted prefix is active\. An evaluation anchored toπ0\\pi\_\{0\}would answer a different question and would not directly test this composition\. The trace does not establish F2a’s isolated replay effect or the population frequency of the interaction; it demonstrates why commit\-time evaluation should use the policy state in which the patch would persist\.

## Appendix FFuture Work

##### Stronger confirmation and gate analysis\.

The clearest extensions are a request\-disjoint confirmation pool, more ground\-truth families, and repeated runs across independent environments\. Harmonized reruns of Skill\-Compact and the transfer backbones would permit paired uncertainty estimates\. Threshold sweeps, leave\-one\-patch\-out attribution, and cumulative replay budgets could separate proposal quality, marginal safety, and sequence\-level regression; anytime\-valid rules such as PACE\([Shawn, 2026](https://arxiv.org/html/2609.12127#bib.bib9)\)could support sequential reuse of a development pool\.

##### Broader tasks and full\-system baselines\.

Flow\-HO can be complemented by public execution\-graded environments such as ALFWorld,τ\\tau\-Bench, and SWE\-bench Verified\. Full comparisons with GRASP\([Moll et al\., 2026](https://arxiv.org/html/2609.12127#bib.bib8)\), SkillGen\([Ma et al\., 2026](https://arxiv.org/html/2609.12127#bib.bib7)\), and SkillOpt\([Yang et al\., 2026](https://arxiv.org/html/2609.12127#bib.bib6)\)would vary both proposer and acceptor beyond RQ4’s controlled acceptor swap\. Comparisons with GEPA\([Agrawal et al\., 2025](https://arxiv.org/html/2609.12127#bib.bib4)\), MIPROv2\([Opsahl\-Ong et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib13)\), OPRO\([Yang et al\., 2024a](https://arxiv.org/html/2609.12127#bib.bib2)\), and TextGrad\([Yuksekgonul et al\., 2024](https://arxiv.org/html/2609.12127#bib.bib11)\)would quantify the value of diagnosis, segmentation, and replay\-constrained persistence\.

##### Initialization and model migration\.

The study starts from one human\-authoredπ0\\pi\_\{0\}\. Sweeps over minimal, generated, and expert initial policies could test dependence on initialization and convergence to similar fixed points\. A deployment study could carry an adapted checkpoint across backbone upgrades and identify patches that transfer, become redundant, or require reversal\.

##### Backbone\-specific tuning\.

Our transfer runs reuse Haiku’s proposer, patch library, replay composition, and gate tolerances without backbone\-specific adaptation\. The resulting Gemma and Ministral gains therefore reflect transfer under a shared configuration rather than tuned per\-backbone performance, and the residual Haiku\-to\-transfer gap should not be interpreted as a method ceiling\. Backbone\-specific patch libraries, replay composition, and re\-calibrated\(εR,εC\)\(\\varepsilon\_\{R\},\\varepsilon\_\{C\}\)tolerances may recover additional headroom; we leave this tuning budget unmeasured\.