基于动作特定信用分配的执行后学习

# Learning after Execution with Action-Specific Credit
Source: [https://arxiv.org/html/2610.14573](https://arxiv.org/html/2610.14573)
Yufeng TianJiahao TangAffiliation:The Chinese University of Hong Kong, Shenzhen, ChinaAffiliation:Central South University, Changsha, ChinaMinghao WuAffiliation:The Chinese University of Hong Kong, Shenzhen, China

###### Abstract

Execution is often used as terminal feedback: it scores a generated program, but the resulting state is not used for another learning action\. Learning after execution instead uses the execution result as an intermediate state that supports an additional learning interaction for the same policy\. This creates a credit\-assignment problem because the two actions share the same final objective but play different roles in the trajectory\. Accordingly, action\-specific credit assigns the initial action credit from the final trajectory outcome and the corrective action credit for improvement over the intermediate state\. Image\-to\-TikZ serves as an executable visual testbed for this setting\. On 945 held\-out examples, learning after execution strengthens the shared policy under single\-call inference and also produces a post\-execution corrective capability\. Single\-call render success reaches 92\.70%, compared with 88\.25% for continued single\-stage GRPO after the same number of additional training iterations\. This corrective capability is not reproduced by continued single\-stage training and depends on the intermediate state\. Alternative credit definitions also induce different group\-relative preferences over the same trajectories\. Together, these results suggest that executable tasks contain useful learning structure beyond a terminal reward\. Code is available at[![[Uncaptioned image]](https://arxiv.org/html/2610.14573v1/logos/github.png)GitHub](https://github.com/tianyufengcn/Learning-after-Execution.git)\.

## 1Introduction

Executing a generated program exposes the result of the model’s prediction\. This result can be scored to train the policy, but it also provides a state on which the policy can act again\([Zheng et al\., 2024](https://arxiv.org/html/2610.14573#bib.bib1);[Li et al\., 2025](https://arxiv.org/html/2610.14573#bib.bib3);[Bi et al\., 2024](https://arxiv.org/html/2610.14573#bib.bib2)\)\. In a single\-stage interaction, execution marks the end of the trajectory\. The executed outcome provides a training signal, but the state produced by execution is not used for another learning action\. Rather than ending the interaction at this point, we ask whether the execution state can support an additional learning interaction, and whether learning from this interaction strengthens the shared policy itself\.

This setting introduces a credit\-assignment problem because the two actions share the same final objective but serve different roles in the trajectory\([Qu et al\., 2026](https://arxiv.org/html/2610.14573#bib.bib4);[Du et al\., 2026](https://arxiv.org/html/2610.14573#bib.bib5)\)\. The initial action generates the intermediate output and determines the state from which the corrective action begins, while the corrective action inherits that state and seeks to improve it\. The final trajectory outcome is relevant to both actions, but the corrective action must also be evaluated relative to the state it receives\. Training should therefore account for both where the trajectory ends and what changes after execution\.

We study learning after execution by using the execution state of the policy’s own prediction to create an additional learning interaction from the same training target\. For each target, the shared policy first generates and executes an initial program, then revises or repairs it from the resulting state\. We assign action\-specific credit to the two actions\. The initial action receives credit from the final trajectory outcome, while the corrective action receives credit for improvement over the intermediate result\. Both actions jointly update the same policy\.

![Refer to caption](https://arxiv.org/html/2610.14573v1/fig01.png)Figure 1:Overview of learning after execution with action\-specific credit\. Execution first provides a score for the initial action; learning after execution additionally uses the resulting render or failure state for a corrective action by the same policy\. The initial action receives final\-outcome credit, while the corrective action receives improvement\-relative credit\. The right panel illustrates the Image\-to\-TikZ instantiation with valid and failed executions\.We use Image\-to\-TikZ as an executable visual testbed for this problem\([Belouadi et al\., 2024b](https://arxiv.org/html/2610.14573#bib.bib10);[Saito et al\., 2025](https://arxiv.org/html/2610.14573#bib.bib11)\)\. Image\-to\-TikZ reconstructs a target image as an executable TikZ program and belongs to the broader family of image\-to\-code\([Zhou et al\., 2026](https://arxiv.org/html/2610.14573#bib.bib12);[Yang et al\., 2025](https://arxiv.org/html/2610.14573#bib.bib6);[Wu et al\., 2025](https://arxiv.org/html/2610.14573#bib.bib7);[Si et al\., 2025](https://arxiv.org/html/2610.14573#bib.bib8);[Roberts et al\., 2024](https://arxiv.org/html/2610.14573#bib.bib9)\)\. A generated program can be executed to reveal either a rendered image or an execution failure\. A valid render makes the visual difference from the target observable, while a failed execution identifies a program that still requires repair\. The result before and after correction can therefore be evaluated under the same execution and visual criteria\. This makes Image\-to\-TikZ suitable for studying both how a policy learns from an execution state and what changes when it acts on that state\.

Using Image\-to\-TikZ, we study learning after execution through a series of controlled comparisons\. We separate improvement of the shared policy from the effect of invoking a corrective action at inference time, and distinguish learned post\-execution correction from behavior that may arise from continued single\-stage training alone\. We further examine how alternative credit definitions shape the group\-relative learning signals assigned to the two actions\. Together, these analyses characterize whether learning after execution strengthens the shared policy, produces a distinct corrective capability, and how its learning is shaped by action\-specific credit\. Our main contributions are:

- •We introduce learning after execution as a training setting in which execution is used not only to score a prediction, but also to produce an intermediate state that provides additional learning experience for the same policy\.
- •We formulate action\-specific credit for this setting, preserving a shared final\-quality objective while matching credit to the different roles of initial generation and correction\.
- •We show that learning from post\-execution interaction strengthens the shared policy even under single\-call inference, while also producing a corrective capability that is not reproduced by continued single\-stage training alone\.

## 2Preliminaries: Code Supervision and Execution Feedback

Before introducing learning after execution, we define the program generation, execution score, and single\-stage training setting used throughout the paper\. In our Image\-to\-TikZ instantiation, a target imageI∗I^\{\\ast\}is mapped to an executable TikZ programCC\. ExecutingCCeither produces a rendered image or results in execution failure\. These outcomes define the execution score used by Direct\-GRPO and by our method in Section[3\.1](https://arxiv.org/html/2610.14573#S3.SS1)\.

### 2\.1Supervised Image\-to\-TikZ

Given paired training data𝒟=\{\(Ii∗,Ci∗\)\}i=1N\\mathcal\{D\}=\\\{\(I\_\{i\}^\{\\ast\},C\_\{i\}^\{\\ast\}\)\\\}\_\{i=1\}^\{N\}, whereIi∗I\_\{i\}^\{\\ast\}is a target image andCi∗C\_\{i\}^\{\\ast\}is its reference TikZ program, supervised fine\-tuning trains the model to generateCCconditioned onIi∗I\_\{i\}^\{\\ast\}\. The supervision is defined on the reference program rather than on the executed outcome of a program sampled from the policy\. Different TikZ programs can produce similar rendered figures, while small code changes can alter the visual result or break execution\. The rendered consequence of the model’s own prediction therefore does not directly participate in the supervised training objective\.

### 2\.2Direct Reinforcement Learning with Execution Feedback

Execution exposes the consequence of a generated program\. For each target image, we sample a group ofKKprogramsCi∼πθold\(⋅∣I∗\)C\_\{i\}\\sim\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(\\cdot\\mid I^\{\\ast\}\), fori=1,…,Ki=1,\\ldots,K, and evaluate their executed outputs\. We define the execution score

S⁡\(C,I∗\)=\{RSim⁡\(ℰ⁡\(C\),I∗\),if​C​produces a valid render,0,otherwise\.S\(C;I^\{\\ast\}\)=\\begin\{cases\}\\operatorname\{RSim\}\\\!\\left\(\\mathcal\{E\}\(C\),I^\{\\ast\}\\right\),&\\text\{if \}C\\text\{ produces a valid render\},\\\\\[2\.0pt\] 0,&\\text\{otherwise\}\.\\end\{cases\}\(1\)
ℰ\\mathcal\{E\}denotes theLaTeXexecution and rasterization pipeline\. Following TikZilla, we use RSim as the visual similarity reward, which compares patch\-level representations from the DeTikZify\-V2 image encoder using Earth Mover’s Distance\([Kusner et al\., 2015](https://arxiv.org/html/2610.14573#bib.bib14);[Belouadi et al\., 2024b](https://arxiv.org/html/2610.14573#bib.bib10);[Greisinger and Eger, 2026](https://arxiv.org/html/2610.14573#bib.bib31)\)\. The score combines executability and visual fidelity: failed executions receive zero, while valid renders are scored by their similarity to the target\.

For Direct\-GRPO, each sampled program receivesSi=S⁡\(Ci,I∗\)S\_\{i\}=S\(C\_\{i\};I^\{\\ast\}\)\. Within each target\-image group, we mean\-center these scores to obtain group\-relative advantages and optimize them with the clipped GRPO objective\([Shao et al\., 2024](https://arxiv.org/html/2610.14573#bib.bib13)\)\. Implementation details are given in Appendix[B](https://arxiv.org/html/2610.14573#A2)\. We refer to this single\-stage setting as Direct\-GRPO\.

Direct\-GRPO learns from the executed outcomes of programs sampled from the current policy\. Its optimization signal reflects both executability and rendered visual fidelity\. Execution, however, still marks the end of the interaction: the result is used to score the sampled program, but the policy does not observe the rendered output or failure state as input for another action\. The next section uses this execution result as an intermediate state on which the same policy can act again\.

## 3Learning after Execution with Action\-Specific Credit

Direct\-GRPO uses execution as terminal feedback\. We instead use the execution result as an intermediate state on which the same policy can act again\. The policy first generates and executes a program, then revises or repairs it based on the observed execution result\. The initial and corrective actions are jointly trained with action\-specific credit\.

### 3\.1Execution\-Conditioned Correction

LetI⋆I^\{\\star\}denote the target image\. At training iterationkk, the same policyπθk\\pi\_\{\\theta\_\{k\}\}is called twice within each trajectory\. The first call directly generates an initial TikZ program,

C0,i∼πθk\(⋅∣I⋆\),C\_\{0,i\}\\sim\\pi\_\{\\theta\_\{k\}\}\\left\(\\cdot\\mid I^\{\\star\}\\right\),\(2\)
whereiiindexes one sampled trajectory\. We executeC0,iC\_\{0,i\}using the sameLaTeXrendering pipeline introduced in Section[2\.2](https://arxiv.org/html/2610.14573#S2.SS2)and compute

St,i=S⁡\(Ct,i,I⋆\),t∈\{0,1\},S\_\{t,i\}=S\(C\_\{t,i\};I^\{\\star\}\),\\qquad t\\in\\\{0,1\\\},\(3\)
using the execution score defined in Section[2\.2](https://arxiv.org/html/2610.14573#S2.SS2)\. Thus,S0,iS\_\{0,i\}evaluates the intermediate program produced by the first call\. The score is used for credit computation; it is not itself the observation provided to the second call\.

The execution result determines the context for the corrective action\. IfC0,iC\_\{0,i\}produces a valid rendered imageI0,iI\_\{0,i\}, the model receives the target image, the current render, and the current TikZ program\. It can therefore compare the target with what its own program actually produced\. If no valid render is produced, the model instead receives the target image, the current TikZ attempt, and a compact execution statuse0,ie\_\{0,i\}\. The corrective action is sampled as

C1,i∼\{πθk\(⋅∣I⋆,I0,i,C0,i\),valid Stage 0 render,πθk\(⋅∣I⋆,C0,i,e0,i\),otherwise\.C\_\{1,i\}\\sim\\begin\{cases\}\\pi\_\{\\theta\_\{k\}\}\\left\(\\cdot\\mid I^\{\\star\},I\_\{0,i\},C\_\{0,i\}\\right\),&\\text\{valid Stage~0 render\},\\\\\[3\.0pt\] \\pi\_\{\\theta\_\{k\}\}\\left\(\\cdot\\mid I^\{\\star\},C\_\{0,i\},e\_\{0,i\}\\right\),&\\text\{otherwise\}\.\\end\{cases\}\(4\)
The first branch performs visual revision\. The second branch repairs, completes, or replaces an unsuccessful program\. We then executeC1,iC\_\{1,i\}and obtain the final scoreS1,iS\_\{1,i\}using Eq\.[3](https://arxiv.org/html/2610.14573#S3.E3)\.

Stage 0 and Stage 1 are two calls to the same policyπθk\\pi\_\{\\theta\_\{k\}\}under different contexts\. The first call observes only the target image\. The second call observes the execution state produced by the first action\. No parameter update occurs between the two calls\. Both actions are sampled before the shared policy is updated\.

### 3\.2Action\-Specific Credit from Incremental Rewards

Both actions serve the same final\-quality objective,J⁡\(θ\)=𝔼τ∼πθ​\[S1\]J\(\\theta\)=\\mathbb\{E\}\_\{\\tau\\sim\\pi\_\{\\theta\}\}\[S\_\{1\}\]\. With two actions and no discounting, we decompose the execution score into incremental rewards and use the corresponding returns\-to\-go as action\-specific credit:

r0,i\\displaystyle r\_\{0,i\}=S0,i,\\displaystyle=S\_\{0,i\},r1,i\\displaystyle\\qquad r\_\{1,i\}=S1,i−S0,i,\\displaystyle=S\_\{1,i\}\-S\_\{0,i\},\(5\)G0,i\\displaystyle G\_\{0,i\}=r0,i\+r1,i=S1,i,\\displaystyle=r\_\{0,i\}\+r\_\{1,i\}=S\_\{1,i\},G1,i\\displaystyle\\qquad G\_\{1,i\}=r1,i=S1,i−S0,i\.\\displaystyle=r\_\{1,i\}=S\_\{1,i\}\-S\_\{0,i\}\.
The incremental rewards telescope to the final score,r0,i\+r1,i=S1,ir\_\{0,i\}\+r\_\{1,i\}=S\_\{1,i\}\. The initial action therefore receives the return of the complete trajectory, while the corrective action receives the improvement relative to the intermediate state it inherits\.

For the initial action, assigning onlyS1,i−S0,iS\_\{1,i\}\-S\_\{0,i\}would omit its immediate rewardS0,iS\_\{0,i\}and, for a fixed final score, favor a lower intermediate score\. We therefore use the complete returnG0,i=S1,iG\_\{0,i\}=S\_\{1,i\}and evaluate the initial action by the final quality reached by the trajectory\.

For the corrective action,G1,i=S1,i−S0,iG\_\{1,i\}=S\_\{1,i\}\-S\_\{0,i\}directly measures the score change from the state it receives\. Its raw credit is positive when the execution score increases and negative when the score decreases\. Because unsuccessful executions receive zero, the same definition also covers execution repair and failure: ifS0,i=0S\_\{0,i\}=0, thenG1,i=S1,i≥0G\_\{1,i\}=S\_\{1,i\}\\geq 0; ifS1,i=0S\_\{1,i\}=0, thenG1,i=−S0,i≤0G\_\{1,i\}=\-S\_\{0,i\}\\leq 0\. Visual revision and execution repair are therefore handled by the same credit definition\.

### 3\.3Group\-Relative Optimization

For each target image, we sampleKKcomplete trajectories, withK=8K=8in all experiments using the corrective action\. Each trajectoryτi=\(C0,i,S0,i,C1,i,S1,i\)\\tau\_\{i\}=\(C\_\{0,i\},S\_\{0,i\},C\_\{1,i\},S\_\{1,i\}\)contains an independently sampled initial action followed by a corrective action conditioned on its own execution state\. Thus, a group containsKKindependently sampled trajectories rather than one shared Stage 0 program followed by multiple Stage 1 candidates\.

The action\-specific returns in Eq\.[5](https://arxiv.org/html/2610.14573#S3.E5)are centered separately within the same target\-image group\. With the implementation scaleλ=10\\lambda=10,

At,i=λ⁡\(Gt,i−1K​∑j=1KGt,j\),t∈\{0,1\}\.A\_\{t,i\}=\\lambda\\left\(G\_\{t,i\}\-\\frac\{1\}\{K\}\\sum\_\{j=1\}^\{K\}G\_\{t,j\}\\right\),\\qquad t\\in\\\{0,1\\\}\.\(6\)
We subtract only the group mean and do not normalize by the group standard deviation\.A0,iA\_\{0,i\}is assigned to the completion tokens ofC0,iC\_\{0,i\}, whileA1,iA\_\{1,i\}is assigned to those ofC1,iC\_\{1,i\}\.

The distinction between raw credit and group\-relative advantage is important\. A positiveG1,iG\_\{1,i\}means that the corrective action increases the execution score, but it does not necessarily implyA1,i\>0A\_\{1,i\}\>0; the latter also depends on the mean improvement within the sampled group\.

For the corrective action,S0,iS\_\{0,i\}is determined beforeC1,iC\_\{1,i\}is sampled and acts as a state\-dependent baseline\. Appendix[E](https://arxiv.org/html/2610.14573#A5)further compares improvement\-relative and absolute final\-score credit\.

The group\-relative objective is applied separately to the two action sets,

ℒ=0\.5​ℒGRPO​\(C0,A0\)\+0\.5​ℒGRPO​\(C1,A1\)\.\\mathcal\{L\}=0\.5\\mathcal\{L\}\_\{\\mathrm\{GRPO\}\}\(C\_\{0\},A\_\{0\}\)\+0\.5\\mathcal\{L\}\_\{\\mathrm\{GRPO\}\}\(C\_\{1\},A\_\{1\}\)\.\(7\)
Both terms optimize the same policy parameters\. All Stage 0 and Stage 1 actions in a training iteration are sampled from the same policyπθk\\pi\_\{\\theta\_\{k\}\}before any parameter update\. The corresponding losses for the two stages are optimized sequentially over action microbatches using the shared policy\.

## 4Experiments

The experiments examine how using execution during training affects both initial generation and subsequent correction\. We first evaluate overall performance under direct and post\-execution inference\. We then study the corrective action, the role of action\-specific credit, and how correction varies across executed intermediate states\.

### 4\.1Experimental Setup

##### Models and Training Data\.

Qwen3\-VL\-8B\-Instruct\([Bai et al\., 2025](https://arxiv.org/html/2610.14573#bib.bib15)\)is used as the base model throughout the experiments\. All supervised and reinforcement learning data are drawn from DaTikZ\-V4\([Greisinger and Eger, 2026](https://arxiv.org/html/2610.14573#bib.bib31)\)\. Direct\-SFT is trained on approximately 4K Image\-to\-TikZ pairs\. Direct\-GRPO is initialized from the final SFT checkpoint and trained on 1K samples for 125 training iterations\. Our method is initialized from the final Direct\-GRPO checkpoint and trained on the same 1K samples for 125 training iterations\.

##### Evaluation Data\.

A temporally held\-out evaluation set is constructed using the same source collection procedure as DaTikZ\-V4, restricted to data available from 2026 onward\. Samples are collected from arXiv, GitHub, and TeX StackExchange\. The 1,095 examples are divided into two non\-overlapping subsets\.Dev\-150contains 150 examples for checkpoint selection, ablations, and behavioral analysis\.Final\-945contains the 945 examples and is reserved for final evaluation\.

##### Evaluation Metrics\.

Render rate measures whether the generated TikZ program executes successfully\. Visual fidelity is evaluated with RSim\([Greisinger and Eger, 2026](https://arxiv.org/html/2610.14573#bib.bib31)\), DINO\([Caron et al\., 2021](https://arxiv.org/html/2610.14573#bib.bib16)\), DreamSim\([Fu et al\., 2023](https://arxiv.org/html/2610.14573#bib.bib17)\), and CLIPImg\([Radford et al\., 2021](https://arxiv.org/html/2610.14573#bib.bib18)\)\. RSim is also used as the reinforcement learning reward\. Unless stated otherwise, visual metrics use the*fixed\-zero*convention, where failed executions receive a score of zero\.

##### Inference Protocol\.

All models are evaluated with temperature1\.01\.0, top\-pp0\.90\.9, repetition penalty1\.01\.0, and at most 8192 generated tokens\. Generation stops at the first complete`\\end\{document\}`\. Stage 0 denotes the initial output produced from the target image alone, while Stage 1 denotes the raw output produced after observing the Stage 0 execution state\. We additionally evaluate an execution\-aware selection rule over the two outputs already produced\. If one stage fails to render, the valid output is retained\. If both render, the output with the higher RSim score is selected, with ties assigned to Stage 0\. No additional model generation is required by the selection rule\.

### 4\.2Overall Results

We first compare the main models on Final\-945\. Our initializer is the final Direct\-GRPO checkpoint, evaluated with the same Stage 0/Stage 1 interaction before further training\. Because decoding is stochastic, its Stage 0 result and the separately sampled Direct\-GRPO baseline can differ slightly\. As an additional control, we continue Direct\-GRPO from the same checkpoint on the same training set for another 100 training iterations\.

Table 1:Main results on Final\-945\. Visual metrics use the fixed\-zero convention\.*Direct*denotes single\-call generation\.*Stage 0*is the initial single\-call output of our policy, while*Stage 1*is produced after execution\-conditioned correction\.*Selected*chooses between the two outputs using execution success and RSim\. Direct\-GRPO \(\+100\) continues single\-stage GRPO for 100 additional training iterations from the same Direct\-GRPO checkpoint\. Best results are shown in bold and second\-best results are underlined\.MethodOutputRenderSuccess↑\\uparrowRSim↑\\uparrowDINO↑\\uparrowDreamSim↑\\uparrowCLIPImg↑\\uparrowBase modelDirect71\.64%0\.40330\.60250\.55450\.6435SFTDirect64\.66%0\.35540\.54270\.49600\.5781Direct\-GRPODirect81\.90%0\.47090\.70460\.64680\.7410Direct\-GRPO \(\+100\)Direct88\.25%0\.51350\.76290\.70190\.7996Ours \(initializer\)Stage 083\.07%0\.47660\.71290\.65440\.7489Ours \(initializer\)Stage 183\.17%0\.47810\.71420\.65560\.7501OursStage 092\.70%0\.53820\.80140\.73300\.8372OursStage 192\.80%0\.54460\.80460\.74010\.8409OursSelected94\.18%0\.56490\.82210\.75930\.8583Table[1](https://arxiv.org/html/2610.14573#S4.T1)shows that SFT does not improve performance on this held\-out set, whereas Direct\-GRPO reaches a render rate of81\.90%81\.90\\%, together with consistent improvements across all four visual metrics\. Optimizing executed outcomes provides a substantially stronger signal than reference code supervision alone\.

Continuing Direct\-GRPO for another 100 training iterations provides further gains, raising render success to88\.25%88\.25\\%\. Starting from the same Direct\-GRPO checkpoint and after the same number of additional training iterations, our Stage 0 output reaches92\.70%92\.70\\%and is higher under all four visual metrics\. Continued single\-stage training therefore does not reproduce the gain obtained with our training\. Importantly, this gain is already present in Stage 0, which uses the same single\-call generation setting as the direct baselines\.

Simply adding a second execution\-conditioned call is insufficient\. At initialization, Stage 0 and Stage 1 differ little\. After training, raw Stage 1 reaches92\.80%92\.80\\%render success and achieves slightly higher visual similarity than Stage 0 across all four metrics\. Execution\-aware selection further raises render success to94\.18%94\.18\\%and improves all four visual metrics\.

These results separate two effects\. Training strengthens the shared policy itself, as shown by the improved Stage 0 output, while the second action provides an additional opportunity to act on the execution state\. We examine the latter behavior next\.

### 4\.3Learning Dynamics of Generation and Correction

Since Stage 0 already improves substantially, we isolate the corrective action itself\. For visual correction, we consider only examples where both stages render successfully, termed the*paired success*subset\. For a visual similarity metricMM, we define

Δ​M=M⁡\(I1,I⋆\)−M⁡\(I0,I⋆\),\\Delta M=M\(I\_\{1\},I^\{\\star\}\)\-M\(I\_\{0\},I^\{\\star\}\),\(8\)
whereI0I\_\{0\}andI1I\_\{1\}are the rendered outputs before and after correction, andI⋆I^\{\\star\}is the target image\. This comparison excludes gains from execution recovery and isolates the visual change produced by Stage 1 on already executable outputs\. For the correction\-outcome composition, we use a small neutral region:Δ​RSim\>0\.005\\Delta\\mathrm\{RSim\}\>0\.005is counted as improvement,\|Δ​RSim\|≤0\.005\|\\Delta\\mathrm\{RSim\}\|\\leq 0\.005as neutral, andΔ​RSim<−0\.005\\Delta\\mathrm\{RSim\}<\-0\.005as degradation\.

![Refer to caption](https://arxiv.org/html/2610.14573v1/fig02.png)Figure 2:Learning dynamics of our method on Dev\-150\. The figure tracks Stage 0 and Stage 1 render rates, paired visual changesΔ​M\\Delta M, the distribution of correction outcomes, and execution transitions across training checkpoints\.Figure[2](https://arxiv.org/html/2610.14573#S4.F2)shows that the initializer does not yet act as a reliable visual corrector\. On paired\-success examples, its average Stage 1 change is negative under all four visual metrics\. Strong direct generation therefore does not automatically provide the ability to revise an executed result\. Further single\-stage optimization shows the same pattern: when Continued Direct\-GRPO is evaluated with the same execution\-conditioned second call, its paired visual changes remain close to zero\.

This behavior changes during training\. At step 75, the average Stage 1 change is positive under RSim, DINO, DreamSim, and CLIPImg\. RSim is used for training\. The concurrent gains under the other three metrics show that the visual improvement is not confined to the reward metric\. Correction quality is not monotonic across training, even though render rate improves much more consistently\. This shows that initial generation quality and the effect of the corrective action can evolve differently\. We return to this difference in Section[4\.5](https://arxiv.org/html/2610.14573#S4.SS5)\.

The correction\-outcome distribution also changes substantially\. The fraction of paired\-success examples improved by Stage 1 rises from12\.2%12\.2\\%for the initializer to42\.5%42\.5\\%at step 75 and46\.8%46\.8\\%at step 125\. Most successful Stage 0 executions remain executable after Stage 1, while failed\-to\-successful recovery remains relatively rare on Dev\-150\. Together, these results show that training changes both the initial action and the behavior of the subsequent corrective action\. Checkpoint 100 is selected on Dev\-150 for subsequent behavioral analyses and final evaluation on Final\-945, while training dynamics are examined across the full checkpoint trajectory\.

### 4\.4Credit Assignment in Post\-Execution Learning

Table 2:Factorized ablation of action\-specific credit assignment\. Rows vary Stage 0 credit and columns vary Stage 1 credit\.Stage 1 creditStage 0 creditλ⁡\(S1−S0\)\\lambda\(S\_\{1\}\-S\_\{0\}\)λ​S1\\lambda S\_\{1\}λ​S1\\lambda S\_\{1\}A \(Ours\)Cλ​S0\\lambda S\_\{0\}BDWe factorize the action\-specific credit design into a2×22\\times 2ablation\. Stage 0 credit is either the final trajectory scoreS1S\_\{1\}or the local intermediate scoreS0S\_\{0\}, while Stage 1 credit is either the improvementS1−S0S\_\{1\}\-S\_\{0\}or the absolute final scoreS1S\_\{1\}\. This yields the four variants in Table[2](https://arxiv.org/html/2610.14573#S4.T2), with A corresponding to our method\.

##### Downstream versus local credit for the initial action\.

The A–B comparison isolates Stage 0 credit\. At step 50, B already achieves a slightly higher Stage 0 render rate than A, yet its subsequent Stage 1 correction is nearly neutral\. A produces a substantially larger correction at the same checkpoint and remains positive at step 75, whereas B becomes slightly negative\. Thus, optimizing the intermediate output itself does not produce the same subsequent correction behavior\. Using downstream creditS1S\_\{1\}instead favors initial actions according to the quality reached after the corrective action\.

##### Improvement\-relative versus absolute credit for the corrective action\.

The A–C comparison isolates Stage 1 credit\. We recompute absoluteS1S\_\{1\}\-based Stage 1 advantages on the 8,000 trajectories collected from the completed A run\. The difference is substantial at both trajectory and group levels: replacingS1−S0S\_\{1\}\-S\_\{0\}withS1S\_\{1\}flips the sign of the group\-relative advantage for37\.2%37\.2\\%of trajectories and changes the highest\-advantage trajectory in74\.5%74\.5\\%of target\-image groups\. The two credit definitions therefore induce substantially different group\-relative preferences over corrective actions\.

![Refer to caption](https://arxiv.org/html/2610.14573v1/fig03.png)Figure 3:Effect of action\-specific credit assignment\. The figure compares Stage 0 quality and correction gain under alternative credit definitions, measures how the resulting group\-relative preferences differ, and tracks the training signal of all four ablation variants\.
##### Optimization behavior of the credit variants\.

Variants C and D share absolute Stage 1 credit and enter the same failure regime from step 26 despite using different Stage 0 credits\. Their Stage 1 generations become substantially longer and normal document termination deteriorates, causing most outputs to fail execution\. For D, the average Stage 1 completion approaches 7K tokens and86%86\\%of completions reach the generation limit\.

Once both stage scores collapse to zero within a group, reward variation disappears and the group\-relative advantages, policy loss, and gradient norm become zero\. The zero learning signal therefore follows the generation failure rather than preceding it\. In contrast, A and B remain executable and retain nonzero training signals on the same groups\. C and D differ in Stage 0 credit but share the same absolute Stage 1 credit, making this the common factor associated with their early failure\. Together, the fixed\-trajectory preference analysis and observed optimization behavior show thatS1S\_\{1\}andS1−S0S\_\{1\}\-S\_\{0\}are not interchangeable credit signals for the corrective action\.

### 4\.5Correction Outcomes Across Intermediate States

The previous analysis shows that the effect of the corrective action is not constant across training\. We therefore examine how Stage 1 outcomes vary with the quality of the executed Stage 0 state\. Using the Dev\-selected checkpoint at step 100, Figure[4](https://arxiv.org/html/2610.14573#S4.F4)shows a clear pattern across intermediate\-state quality\. Stage 1 provides the largest average gain when the intermediate render is weak\. For Stage 0 RSim below0\.40\.4, it improves over Stage 0 in76%76\\%of paired examples\. As Stage 0 quality increases, both the average correction gain and the probability of improvement generally decrease\. In the strongest group, the average change becomes negative and Stage 1 improves over Stage 0 in only23%23\\%of cases\. Additional qualitative examples covering successful revision, execution repair, over\-revision, preservation, and unresolved cases are provided in Appendix[A](https://arxiv.org/html/2610.14573#A1)\.

![Refer to caption](https://arxiv.org/html/2610.14573v1/fig04.png)Figure 4:Correction behavior as a function of Stage 0 quality\. We report the mean Stage 1 change in RSim and the frequency with which Stage 1 achieves higher RSim than Stage 0 across Stage 0 quality ranges, together with qualitative examples of beneficial and harmful corrections\.This pattern helps explain the non\-monotonic correction curves in Figure[2](https://arxiv.org/html/2610.14573#S4.F2)\. Weak intermediate renders leave larger discrepancies for Stage 1 to address\. As Stage 0 becomes stronger, the remaining errors become smaller and further modification carries a greater risk of degrading an already good reconstruction\. Thus, a stronger Stage 0 policy can reduce the average marginal value of correction even when Stage 1 remains capable of improving weak states\.

##### Execution\-Aware Selection\.

The asymmetric effect of correction motivates selecting between the two outputs already produced by the policy\. We use the execution\-aware rule defined in Section[4\.1](https://arxiv.org/html/2610.14573#S4.SS1), which requires no additional model generation\. On Dev\-150, checkpoint 100 gives the strongest selected result\. Selection raises render success to97\.3%97\.3\\%, compared with96\.7%96\.7\\%for Stage 0 and94\.7%94\.7\\%for raw Stage 1, while RSim\-FZ reaches0\.56460\.5646\. DINO, DreamSim, and CLIPImg also improve although they are not used by the selector\.

The checkpoint and selection rule are then frozen before evaluation on Final\-945\. As shown in Table[1](https://arxiv.org/html/2610.14573#S4.T1), selection raises render success from92\.80%92\.80\\%for raw Stage 1 to94\.18%94\.18\\%and improves all four fixed\-zero visual metrics\. Stage 0 can therefore be used directly as the single\-call output, while the post\-execution action and selection provide an optional additional refinement mode\.

## 5Related Work

##### Rendering\- and execution\-aware reinforcement learning\.

RLRF\([Rodríguez et al\., 2025b](https://arxiv.org/html/2610.14573#bib.bib30)\)optimizes vector graphics with rewards from rendered SVG outputs, while TikZilla\([Greisinger and Eger, 2026](https://arxiv.org/html/2610.14573#bib.bib31)\)applies GRPO to Text\-to\-TikZ using rendered visual similarity\. In code synthesis, RLEF\([Gehring et al\., 2025](https://arxiv.org/html/2610.14573#bib.bib32)\)and related work\([Le et al\., 2022](https://arxiv.org/html/2610.14573#bib.bib23);[Liu et al\., 2023](https://arxiv.org/html/2610.14573#bib.bib24)\)use execution feedback to improve generated programs\. These methods establish execution as an effective learning signal\. We focus on the additional learning and credit\-assignment problem that arises when the execution result becomes an intermediate state for another action\.

##### Iterative correction and multi\-turn policy learning\.

Iterative improvement has been explored through search, self\-correction, and multi\-turn reinforcement learning\([Madaan et al\., 2023](https://arxiv.org/html/2610.14573#bib.bib25);[Shinn et al\., 2023](https://arxiv.org/html/2610.14573#bib.bib26);[Gou et al\., 2024](https://arxiv.org/html/2610.14573#bib.bib27);[Yao et al\., 2023](https://arxiv.org/html/2610.14573#bib.bib28);[Chen et al\., 2024](https://arxiv.org/html/2610.14573#bib.bib29)\)\. DeTikZify refines programs with MCTS, while Sketch2Diagram selects among independently generated candidates\. Recent agentic RL methods study finer\-grained credit across interaction steps, including GiGPO\([Feng et al\., 2025](https://arxiv.org/html/2610.14573#bib.bib33)\)and turn\-level policy optimization\([Ding et al\., 2026](https://arxiv.org/html/2610.14573#bib.bib34)\)\. Our setting instead uses the execution state produced by the policy’s own initial program as the context for a subsequent action, and assigns credit to the initial and corrective actions according to their different roles in the trajectory\.

##### Graphics program synthesis\.

Recent work increasingly treats scientific figure reconstruction as graphics program synthesis rather than raster image generation\([Belouadi et al\., 2025](https://arxiv.org/html/2610.14573#bib.bib20);[Rodríguez et al\., 2025a](https://arxiv.org/html/2610.14573#bib.bib21);[Belouadi et al\., 2024a](https://arxiv.org/html/2610.14573#bib.bib19);[Wu et al\., 2023](https://arxiv.org/html/2610.14573#bib.bib22)\)\. DeTikZify\([Belouadi et al\., 2024b](https://arxiv.org/html/2610.14573#bib.bib10)\)generates TikZ programs from figures and sketches with inference\-time refinement, while Sketch2Diagram\([Saito et al\., 2025](https://arxiv.org/html/2610.14573#bib.bib11)\)studies sketch\-to\-TikZ generation with data augmentation and multi\-candidate inference\. Omni\-I2C\([Zhou et al\., 2026](https://arxiv.org/html/2610.14573#bib.bib12)\)further highlights the difficulty of high\-fidelity image\-to\-code generation across diverse domains\. We use Image\-to\-TikZ as an executable visual testbed for studying learning after execution\.

## 6Conclusion and Limitations

The study shows that execution can provide more than a terminal learning signal\. It can expose an intermediate state from which additional learning becomes possible\. Using the same training targets, shared policy, and execution score, our method turns this state into a second learning interaction and assigns credit according to the distinct roles of initial generation and correction\. This simple change strengthens the policy even under single\-call inference, while also producing an additional post\-execution corrective capability\. More broadly, our results suggest that executable tasks may contain useful learning structure beyond the scalar reward obtained at the end of generation, and that carefully assigning credit can expose this structure without requiring a more complex reward design\. At the same time, our experiments provide an initial demonstration of this principle rather than a complete characterization of its scope\. Our study is limited to one 8B backbone, one executable visual program domain, and a relatively small training set\. Correction is not always beneficial for strong intermediate states, motivating learned stopping or routing, while the instability of absolute corrective credit remains specific to our current RSim\-based setting\.

## References

- \[1\]T\. Zheng, G\. Zhang, T\. Shen, X\. Liu, B\. Y\. Lin, J\. Fu, W\. Chen, and X\. Yue\(2024\)OpenCodeInterpreter: integrating code generation with execution and refinement\.InFindings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11\-16, 2024,L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Findings of ACL, Vol\.ACL 2024,pp\. 12834–12859\.External Links:[Link](https://doi.org/10.18653/v1/2024.findings-acl.762),[Document](https://dx.doi.org/10.18653/V1/2024.FINDINGS-ACL.762)Cited by:[§1](https://arxiv.org/html/2610.14573#S1.p1.1)\.
- \[2\]Q\. Li, X\. Dai, X\. Li, W\. Zhang, Y\. Wang, R\. Tang, and Y\. Yu\(2025\)CodePRM: execution feedback\-enhanced process reward model for code generation\.InFindings of the Association for Computational Linguistics, ACL 2025, Vienna, Austria, July 27 \- August 1, 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Findings of ACL, Vol\.ACL 2025,pp\. 8169–8182\.External Links:[Link](https://doi.org/10.18653/v1/2025.findings-acl.428),[Document](https://dx.doi.org/10.18653/V1/2025.FINDINGS-ACL.428)Cited by:[§1](https://arxiv.org/html/2610.14573#S1.p1.1)\.
- \[3\]Z\. Bi, Y\. Wan, Z\. Wang, H\. Zhang, B\. Guan, F\. Lu, Z\. Zhang, Y\. Sui, H\. Jin, and X\. Shi\(2024\)Iterative refinement of project\-level code context for precise code generation with compiler feedback\.InFindings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11\-16, 2024,L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Findings of ACL, Vol\.ACL 2024,pp\. 2336–2353\.External Links:[Link](https://doi.org/10.18653/v1/2024.findings-acl.138),[Document](https://dx.doi.org/10.18653/V1/2024.FINDINGS-ACL.138)Cited by:[§1](https://arxiv.org/html/2610.14573#S1.p1.1)\.
- \[4\]C\. Qu, S\. Dai, H\. Cai, J\. Xu, S\. Wang, and D\. Yin\(2026\)MatchTIR: fine\-grained supervision for tool\-integrated reasoning via bipartite matching\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), ACL 2026, San Diego, California, United States, July 2\-7, 2026,M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),pp\. 11953–11968\.External Links:[Link](https://doi.org/10.18653/v1/2026.acl-long.549),[Document](https://dx.doi.org/10.18653/V1/2026.ACL-LONG.549)Cited by:[§1](https://arxiv.org/html/2610.14573#S1.p2.1)\.
- \[5\]L\. Du, G\. Xue, X\. Liang, Q\. Huang, Y\. Ding, X\. Shi, Z\. Yijun, J\. Qi, W\. Zhu, J\. Li, and M\. Zhang\(2026\)Escaping the echo trap: on credit assignment failure in multi\-turn LLM self\-reflection\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), ACL 2026, San Diego, California, United States, July 2\-7, 2026,M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),pp\. 35393–35405\.External Links:[Link](https://doi.org/10.18653/v1/2026.acl-long.1636),[Document](https://dx.doi.org/10.18653/V1/2026.ACL-LONG.1636)Cited by:[§1](https://arxiv.org/html/2610.14573#S1.p2.1)\.
- \[6\]J\. Belouadi, S\. P\. Ponzetto, and S\. Eger\(2024\)DeTikZify: synthesizing graphics programs for scientific figures and sketches with tikz\.InAdvances in Neural Information Processing Systems 37: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 \- 15, 2024,A\. Globersons, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. M\. Tomczak, and C\. Zhang \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2024/hash/9a8d52eb05eb7b13f54b3d9eada667b7-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.14573#S1.p4.1),[§2\.2](https://arxiv.org/html/2610.14573#S2.SS2.p3.1),[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px3.p1.1)\.
- \[7\]I\. Saito, H\. Yoshida, and K\. Sakaguchi\(2025\)Sketch2Diagram: generating vector diagrams from hand\-drawn sketches\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,External Links:[Link](https://openreview.net/forum?id=KvaDHPhhir)Cited by:[§1](https://arxiv.org/html/2610.14573#S1.p4.1),[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px3.p1.1)\.
- \[8\]J\. Zhou, C\. Zhang, X\. Feng, Q\. Zhang, H\. Qiu, L\. He, D\. Ye, X\. Gao, and J\. Zhang\(2026\)Omni\-i2c: A holistic benchmark for high\-fidelity image\-to\-code generation\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), ACL 2026, San Diego, California, United States, July 2\-7, 2026,M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),pp\. 37761–37798\.External Links:[Link](https://doi.org/10.18653/v1/2026.acl-long.1753),[Document](https://dx.doi.org/10.18653/V1/2026.ACL-LONG.1753)Cited by:[§1](https://arxiv.org/html/2610.14573#S1.p4.1),[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px3.p1.1)\.
- \[9\]C\. Yang, C\. Shi, Y\. Liu, B\. Shui, J\. Wang, M\. Jing, L\. Xu, X\. Zhu, S\. Li, Y\. Zhang, G\. Liu, X\. Nie, D\. Cai, and Y\. Yang\(2025\)ChartMimic: evaluating lmm’s cross\-modal reasoning capability via chart\-to\-code generation\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,External Links:[Link](https://openreview.net/forum?id=sGpCzsfd1K)Cited by:[§1](https://arxiv.org/html/2610.14573#S1.p4.1)\.
- \[10\]C\. Wu, Z\. Liang, Y\. Ge, Q\. Guo, Z\. Lu, J\. Wang, Y\. Shan, and P\. Luo\(2025\)Plot2Code: A comprehensive benchmark for evaluating multi\-modal large language models in code generation from scientific plots\.InFindings of the Association for Computational Linguistics: NAACL 2025, Albuquerque, New Mexico, USA, April 29 \- May 4, 2025,L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),Findings of ACL, Vol\.NAACL 2025,pp\. 3006–3028\.External Links:[Link](https://doi.org/10.18653/v1/2025.findings-naacl.164),[Document](https://dx.doi.org/10.18653/V1/2025.FINDINGS-NAACL.164)Cited by:[§1](https://arxiv.org/html/2610.14573#S1.p4.1)\.
- \[11\]C\. Si, Y\. Zhang, R\. Li, Z\. Yang, R\. Liu, and D\. Yang\(2025\)Design2Code: benchmarking multimodal code generation for automated front\-end engineering\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2025 \- Volume 1: Long Papers, Albuquerque, New Mexico, USA, April 29 \- May 4, 2025,L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),pp\. 3956–3974\.External Links:[Link](https://doi.org/10.18653/v1/2025.naacl-long.199),[Document](https://dx.doi.org/10.18653/V1/2025.NAACL-LONG.199)Cited by:[§1](https://arxiv.org/html/2610.14573#S1.p4.1)\.
- \[12\]J\. S\. Roberts, T\. Lee, C\. H\. Wong, M\. Yasunaga, Y\. Mai, and P\. Liang\(2024\)Image2Struct: benchmarking structure extraction for vision\-language models\.InAdvances in Neural Information Processing Systems 37: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 \- 15, 2024,A\. Globersons, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. M\. Tomczak, and C\. Zhang \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2024/hash/d0718553fd6b227a353c6432cf893285-Abstract-Datasets/_and/_Benchmarks/_Track.html)Cited by:[§1](https://arxiv.org/html/2610.14573#S1.p4.1)\.
- \[13\]M\. J\. Kusner, Y\. Sun, N\. I\. Kolkin, and K\. Q\. Weinberger\(2015\)From word embeddings to document distances\.InProceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6\-11 July 2015,F\. R\. Bach and D\. M\. Blei \(Eds\.\),JMLR Workshop and Conference Proceedings, Vol\.37,pp\. 957–966\.External Links:[Link](http://proceedings.mlr.press/v37/kusnerb15.html)Cited by:[§2\.2](https://arxiv.org/html/2610.14573#S2.SS2.p3.1)\.
- \[14\]C\. Greisinger and S\. Eger\(2026\)TikZilla: scaling text\-to\-tikz with high\-quality data and reinforcement learning\.CoRRabs/2603\.03072\.External Links:[Link](https://doi.org/10.48550/arXiv.2603.03072),[Document](https://dx.doi.org/10.48550/ARXIV.2603.03072),2603\.03072Cited by:[§C\.1](https://arxiv.org/html/2610.14573#A3.SS1.p1.1),[§C\.3](https://arxiv.org/html/2610.14573#A3.SS3.p1.1),[§2\.2](https://arxiv.org/html/2610.14573#S2.SS2.p3.1),[§4\.1](https://arxiv.org/html/2610.14573#S4.SS1.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2610.14573#S4.SS1.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px1.p1.1)\.
- \[15\]Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo\(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.CoRRabs/2402\.03300\.External Links:[Link](https://doi.org/10.48550/arXiv.2402.03300),[Document](https://dx.doi.org/10.48550/ARXIV.2402.03300),2402\.03300Cited by:[§2\.2](https://arxiv.org/html/2610.14573#S2.SS2.p4.1)\.
- \[16\]S\. Bai, Y\. Cai, R\. Chen, K\. Chen, X\. Chen, Z\. Cheng, L\. Deng, W\. Ding, C\. Gao, C\. Ge, W\. Ge, Z\. Guo, Q\. Huang, J\. Huang, F\. Huang, B\. Hui, S\. Jiang, Z\. Li, M\. Li, M\. Li, K\. Li, Z\. Lin, J\. Lin, X\. Liu, J\. Liu, C\. Liu, Y\. Liu, D\. Liu, S\. Liu, D\. Lu, R\. Luo, C\. Lv, R\. Men, L\. Meng, X\. Ren, X\. Ren, S\. Song, Y\. Sun, J\. Tang, J\. Tu, J\. Wan, P\. Wang, P\. Wang, Q\. Wang, Y\. Wang, T\. Xie, Y\. Xu, H\. Xu, J\. Xu, Z\. Yang, M\. Yang, J\. Yang, A\. Yang, B\. Yu, F\. Zhang, H\. Zhang, X\. Zhang, B\. Zheng, H\. Zhong, J\. Zhou, F\. Zhou, J\. Zhou, Y\. Zhu, and K\. Zhu\(2025\)Qwen3\-vl technical report\.CoRRabs/2511\.21631\.External Links:[Link](https://doi.org/10.48550/arXiv.2511.21631),[Document](https://dx.doi.org/10.48550/ARXIV.2511.21631),2511\.21631Cited by:[§4\.1](https://arxiv.org/html/2610.14573#S4.SS1.SSS0.Px1.p1.1)\.
- \[17\]M\. Caron, H\. Touvron, I\. Misra, H\. Jégou, J\. Mairal, P\. Bojanowski, and A\. Joulin\(2021\)Emerging properties in self\-supervised vision transformers\.In2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10\-17, 2021,pp\. 9630–9640\.External Links:[Link](https://doi.org/10.1109/ICCV48922.2021.00951),[Document](https://dx.doi.org/10.1109/ICCV48922.2021.00951)Cited by:[§C\.3](https://arxiv.org/html/2610.14573#A3.SS3.p1.1),[§4\.1](https://arxiv.org/html/2610.14573#S4.SS1.SSS0.Px3.p1.1)\.
- \[18\]S\. Fu, N\. Tamir, S\. Sundaram, L\. Chai, R\. Zhang, T\. Dekel, and P\. Isola\(2023\)DreamSim: learning new dimensions of human visual similarity using synthetic data\.InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2023/hash/9f09f316a3eaf59d9ced5ffaefe97e0f-Abstract-Conference.html)Cited by:[§C\.3](https://arxiv.org/html/2610.14573#A3.SS3.p1.1),[§4\.1](https://arxiv.org/html/2610.14573#S4.SS1.SSS0.Px3.p1.1)\.
- \[19\]A\. Radford, J\. W\. Kim, C\. Hallacy, A\. Ramesh, G\. Goh, S\. Agarwal, G\. Sastry, A\. Askell, P\. Mishkin, J\. Clark, G\. Krueger, and I\. Sutskever\(2021\)Learning transferable visual models from natural language supervision\.InProceedings of the 38th International Conference on Machine Learning, ICML 2021, 18\-24 July 2021, Virtual Event,M\. Meila and T\. Zhang \(Eds\.\),Proceedings of Machine Learning Research, Vol\.139,pp\. 8748–8763\.External Links:[Link](http://proceedings.mlr.press/v139/radford21a.html)Cited by:[§C\.3](https://arxiv.org/html/2610.14573#A3.SS3.p1.1),[§4\.1](https://arxiv.org/html/2610.14573#S4.SS1.SSS0.Px3.p1.1)\.
- \[20\]J\. A\. Rodríguez, H\. Zhang, A\. Puri, R\. Pramanik, A\. Feizi, P\. Wichmann, A\. K\. Mondal, M\. R\. Samsami, R\. Awal, P\. Taslakian, S\. Gella, S\. R\. Mudumba, D\. Vázquez, C\. Pal, and M\. Pedersoli\(2025\)Rendering\-aware reinforcement learning for vector graphics generation\.InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diego, CA, USA, December 2\-7, 2025 / Mexico City, Mexico, November 30 \- December 5, 2025,D\. Belgrave, C\. Zhang, L\. N\. Montoya, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, N\. Chen, I\. V\. M\. Ruíz, and A\. Loaiza\-Bonilla \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2025/hash/57126328c3b40cf618a34f1c5df24d8a-Abstract-Conference.html)Cited by:[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px1.p1.1)\.
- \[21\]J\. Gehring, K\. Zheng, J\. Copet, V\. Mella, T\. Cohen, and G\. Synnaeve\(2025\)RLEF: grounding code llms in execution feedback with reinforcement learning\.InForty\-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13\-19, 2025,A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, and J\. Zhu \(Eds\.\),Proceedings of Machine Learning Research, Vol\.267\.External Links:[Link](https://proceedings.mlr.press/v267/gehring25a.html)Cited by:[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px1.p1.1)\.
- \[22\]H\. Le, Y\. Wang, A\. D\. Gotmare, S\. Savarese, and S\. C\. Hoi\(2022\)CodeRL: mastering code generation through pretrained models and deep reinforcement learning\.InAdvances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 \- December 9, 2022,S\. Koyejo, S\. Mohamed, A\. Agarwal, D\. Belgrave, K\. Cho, and A\. Oh \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2022/hash/8636419dea1aa9fbd25fc4248e702da4-Abstract-Conference.html)Cited by:[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px1.p1.1)\.
- \[23\]J\. Liu, Y\. Zhu, K\. Xiao, Q\. Fu, X\. Han, W\. Yang, and D\. Ye\(2023\)RLTF: reinforcement learning from unit test feedback\.Trans\. Mach\. Learn\. Res\.2023\.External Links:[Link](https://openreview.net/forum?id=hjYmsV6nXZ)Cited by:[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px1.p1.1)\.
- \[24\]A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang, S\. Gupta, B\. P\. Majumder, K\. Hermann, S\. Welleck, A\. Yazdanbakhsh, and P\. Clark\(2023\)Self\-refine: iterative refinement with self\-feedback\.InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2023/hash/91edff07232fb1b55a505a9e9f6c0ff3-Abstract-Conference.html)Cited by:[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px2.p1.1)\.
- \[25\]N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. Yao\(2023\)Reflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2023/hash/1b44b878bb782e6954cd888628510e90-Abstract-Conference.html)Cited by:[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px2.p1.1)\.
- \[26\]Z\. Gou, Z\. Shao, Y\. Gong, Y\. Shen, Y\. Yang, N\. Duan, and W\. Chen\(2024\)CRITIC: large language models can self\-correct with tool\-interactive critiquing\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024,External Links:[Link](https://openreview.net/forum?id=Sx038qxjek)Cited by:[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px2.p1.1)\.
- \[27\]S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. R\. Narasimhan, and Y\. Cao\(2023\)ReAct: synergizing reasoning and acting in language models\.InThe Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1\-5, 2023,External Links:[Link](https://openreview.net/forum?id=WE\_vluYUL-X)Cited by:[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px2.p1.1)\.
- \[28\]X\. Chen, M\. Lin, N\. Schärli, and D\. Zhou\(2024\)Teaching large language models to self\-debug\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024,External Links:[Link](https://openreview.net/forum?id=KuPixIqPiq)Cited by:[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px2.p1.1)\.
- \[29\]L\. Feng, Z\. Xue, T\. Liu, and B\. An\(2025\)Group\-in\-group policy optimization for LLM agent training\.InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diego, CA, USA, December 2\-7, 2025 / Mexico City, Mexico, November 30 \- December 5, 2025,D\. Belgrave, C\. Zhang, L\. N\. Montoya, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, N\. Chen, I\. V\. M\. Ruíz, and A\. Loaiza\-Bonilla \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2025/hash/420c9f777c0b4f78d515e53cf74d58b2-Abstract-Conference.html)Cited by:[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px2.p1.1)\.
- \[30\]Y\. Ding, H\. Le, S\. Han, K\. Ruan, Z\. Jin, V\. Kumar, Z\. Wang, and A\. Deoras\(2026\)Empowering multi\-turn tool\-integrated agentic reasoning with group turn policy optimization\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), ACL 2026, San Diego, California, United States, July 2\-7, 2026,M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),pp\. 42409–42423\.External Links:[Link](https://doi.org/10.18653/v1/2026.acl-long.1962),[Document](https://dx.doi.org/10.18653/V1/2026.ACL-LONG.1962)Cited by:[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px2.p1.1)\.
- \[31\]J\. Belouadi, E\. Ilg, M\. Keuper, H\. Tanaka, M\. Utiyama, R\. Dabre, S\. Eger, and S\. P\. Ponzetto\(2025\)Tikzero: zero\-shot text\-guided graphics program synthesis\.InIEEE/CVF International Conference on Computer Vision, ICCV 2025, Honolulu, HI, USA, October 19\-25, 2025,pp\. 17793–17806\.External Links:[Link](https://doi.org/10.1109/ICCV51701.2025.01653),[Document](https://dx.doi.org/10.1109/ICCV51701.2025.01653)Cited by:[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px3.p1.1)\.
- \[32\]J\. A\. Rodríguez, A\. Puri, S\. Agarwal, I\. H\. Laradji, S\. Rajeswar, D\. Vázquez, C\. Pal, and M\. Pedersoli\(2025\)StarVector: generating scalable vector graphics code from images and text\.InThirty\-Ninth AAAI Conference on Artificial Intelligence, Thirty\-Seventh Conference on Innovative Applications of Artificial Intelligence, Fifteenth Symposium on Educational Advances in Artificial Intelligence, AAAI 2025, Philadelphia, PA, USA, February 25 \- March 4, 2025,T\. Walsh, J\. Shah, and Z\. Kolter \(Eds\.\),pp\. 29691–29693\.External Links:[Link](https://doi.org/10.1609/aaai.v39i28.35369),[Document](https://dx.doi.org/10.1609/AAAI.V39I28.35369)Cited by:[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px3.p1.1)\.
- \[33\]J\. Belouadi, A\. Lauscher, and S\. Eger\(2024\)AutomaTikZ: text\-guided synthesis of scientific vector graphics with tikz\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024,External Links:[Link](https://openreview.net/forum?id=v3K5TVP8kZ)Cited by:[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px3.p1.1)\.
- \[34\]R\. Wu, W\. Su, K\. Ma, and J\. Liao\(2023\)IconShop: text\-guided vector icon synthesis with autoregressive transformers\.ACM Trans\. Graph\.42\(6\),pp\. 230:1–230:14\.External Links:[Link](https://doi.org/10.1145/3618364),[Document](https://dx.doi.org/10.1145/3618364)Cited by:[§5](https://arxiv.org/html/2610.14573#S5.SS0.SSS0.Px3.p1.1)\.

Appendix

Appendix Overview

§[A](https://arxiv.org/html/2610.14573#A1)Additional Qualitative Results and Failure Cases\.[A](https://arxiv.org/html/2610.14573#A1)

§[A\.1](https://arxiv.org/html/2610.14573#A1.SS1)Qualitative Progression across Learning Settings\.[A\.1](https://arxiv.org/html/2610.14573#A1.SS1)

§[A\.2](https://arxiv.org/html/2610.14573#A1.SS2)Successful Visual Revision\.[A\.2](https://arxiv.org/html/2610.14573#A1.SS2)

§[A\.3](https://arxiv.org/html/2610.14573#A1.SS3)Execution Repair\.[A\.3](https://arxiv.org/html/2610.14573#A1.SS3)

§[A\.4](https://arxiv.org/html/2610.14573#A1.SS4)Over\-Revision of Strong Intermediate States\.[A\.4](https://arxiv.org/html/2610.14573#A1.SS4)

§[A\.5](https://arxiv.org/html/2610.14573#A1.SS5)Preservation and Unresolved Cases\.[A\.5](https://arxiv.org/html/2610.14573#A1.SS5)

§[B](https://arxiv.org/html/2610.14573#A2)Implementation Details\.[B](https://arxiv.org/html/2610.14573#A2)

§[B\.1](https://arxiv.org/html/2610.14573#A2.SS1)Full Optimization Algorithm\.[B\.1](https://arxiv.org/html/2610.14573#A2.SS1)

§[B\.2](https://arxiv.org/html/2610.14573#A2.SS2)Prompt and Routing Contract\.[B\.2](https://arxiv.org/html/2610.14573#A2.SS2)

§[B\.3](https://arxiv.org/html/2610.14573#A2.SS3)Execution Routing and State Construction\.[B\.3](https://arxiv.org/html/2610.14573#A2.SS3)

§[B\.4](https://arxiv.org/html/2610.14573#A2.SS4)Training Configuration\.[B\.4](https://arxiv.org/html/2610.14573#A2.SS4)

§[B\.5](https://arxiv.org/html/2610.14573#A2.SS5)Execution and Reward Environment\.[B\.5](https://arxiv.org/html/2610.14573#A2.SS5)

§[C](https://arxiv.org/html/2610.14573#A3)Data and Evaluation Protocol\.[C](https://arxiv.org/html/2610.14573#A3)

§[C\.1](https://arxiv.org/html/2610.14573#A3.SS1)Training Data\.[C\.1](https://arxiv.org/html/2610.14573#A3.SS1)

§[C\.2](https://arxiv.org/html/2610.14573#A3.SS2)Held\-Out Evaluation Data\.[C\.2](https://arxiv.org/html/2610.14573#A3.SS2)

§[C\.3](https://arxiv.org/html/2610.14573#A3.SS3)Evaluation Metrics\.[C\.3](https://arxiv.org/html/2610.14573#A3.SS3)

§[C\.4](https://arxiv.org/html/2610.14573#A3.SS4)Inference and Rendering Protocol\.[C\.4](https://arxiv.org/html/2610.14573#A3.SS4)

§[C\.5](https://arxiv.org/html/2610.14573#A3.SS5)Checkpoint Selection\.[C\.5](https://arxiv.org/html/2610.14573#A3.SS5)

§[D](https://arxiv.org/html/2610.14573#A4)Extended Learning Dynamics and Controls\.[D](https://arxiv.org/html/2610.14573#A4)

§[D\.1](https://arxiv.org/html/2610.14573#A4.SS1)Complete Learning Dynamics\.[D\.1](https://arxiv.org/html/2610.14573#A4.SS1)

§[D\.2](https://arxiv.org/html/2610.14573#A4.SS2)Paired\-Success Correction Dynamics\.[D\.2](https://arxiv.org/html/2610.14573#A4.SS2)

§[D\.3](https://arxiv.org/html/2610.14573#A4.SS3)Continued Direct\-GRPO Controls\.[D\.3](https://arxiv.org/html/2610.14573#A4.SS3)

§[D\.4](https://arxiv.org/html/2610.14573#A4.SS4)Paired Bootstrap on Final\-945\.[D\.4](https://arxiv.org/html/2610.14573#A4.SS4)

§[E](https://arxiv.org/html/2610.14573#A5)Credit Assignment Analysis\.[E](https://arxiv.org/html/2610.14573#A5)

§[E\.1](https://arxiv.org/html/2610.14573#A5.SS1)Factorized Credit Variants\.[E\.1](https://arxiv.org/html/2610.14573#A5.SS1)

§[E\.2](https://arxiv.org/html/2610.14573#A5.SS2)Downstream versus Local Stage 0 Credit\.[E\.2](https://arxiv.org/html/2610.14573#A5.SS2)

§[E\.3](https://arxiv.org/html/2610.14573#A5.SS3)Direct Audit of Credit\-Induced Preferences\.[E\.3](https://arxiv.org/html/2610.14573#A5.SS3)

§[E\.4](https://arxiv.org/html/2610.14573#A5.SS4)Optimization Behavior of the Credit Variants\.[E\.4](https://arxiv.org/html/2610.14573#A5.SS4)

§[E\.5](https://arxiv.org/html/2610.14573#A5.SS5)Summary\.[E\.5](https://arxiv.org/html/2610.14573#A5.SS5)

§[F](https://arxiv.org/html/2610.14573#A6)State\-Dependent Correction and Post\-Execution Behavior\.[F](https://arxiv.org/html/2610.14573#A6)

§[F\.1](https://arxiv.org/html/2610.14573#A6.SS1)Full State\-Dependent Correction Analysis\.[F\.1](https://arxiv.org/html/2610.14573#A6.SS1)

§[F\.2](https://arxiv.org/html/2610.14573#A6.SS2)State Dependence across Checkpoints\.[F\.2](https://arxiv.org/html/2610.14573#A6.SS2)

§[F\.3](https://arxiv.org/html/2610.14573#A6.SS3)Revision and Repair Routing during Training\.[F\.3](https://arxiv.org/html/2610.14573#A6.SS3)

§[F\.4](https://arxiv.org/html/2610.14573#A6.SS4)Execution Transitions and Repair\.[F\.4](https://arxiv.org/html/2610.14573#A6.SS4)

§[F\.5](https://arxiv.org/html/2610.14573#A6.SS5)Execution\-Aware Selection\.[F\.5](https://arxiv.org/html/2610.14573#A6.SS5)

§[F\.6](https://arxiv.org/html/2610.14573#A6.SS6)Summary\.[F\.6](https://arxiv.org/html/2610.14573#A6.SS6)

## Appendix AAdditional Qualitative Results and Failure Cases

![Refer to caption](https://arxiv.org/html/2610.14573v1/app_fig13.png)Figure 5:Additional qualitative examples of our method from Final\-945\.![Refer to caption](https://arxiv.org/html/2610.14573v1/app_fig08.png)Figure 6:Qualitative progression across the main learning settings on Final\-945\. Base model, Direct\-SFT, and Direct\-GRPO show single\-call predictions\. Ours Stage 0 is also a single\-call prediction from the trained policy, while Stage 1 is produced after observing the Stage 0 execution state\. The examples include visible corrective changes and cases where an already strong Stage 0 result is largely preserved\.This section provides additional qualitative examples from Final\-945\. All examples from our method use the checkpoint selected on Dev\-150 at step 100 and the same inference protocol as the quantitative evaluation\. The examples complement the quantitative analyses by illustrating changes across learning settings, successful visual revision, execution repair, over\-revision, preservation, and unresolved cases\.

### A\.1Qualitative Progression across Learning Settings

Figure[6](https://arxiv.org/html/2610.14573#A1.F6)compares outputs across the main learning settings studied in the paper\. For each target image, we show predictions from the base model, Direct\-SFT, Direct\-GRPO, and Stage 0 and Stage 1 of our trained policy\. The examples complement the aggregate results in Table[1](https://arxiv.org/html/2610.14573#S4.T1)\. Execution\-feedback training can substantially change the direct prediction relative to supervised code learning, while our Stage 0 output shows that the shared policy itself improves further\. Stage 1 then provides an additional opportunity to act on discrepancies revealed by execution\.

Stage 1 does not necessarily introduce a visible change\. When Stage 0 already closely matches the target, the Stage 1 render can remain effectively unchanged\. These cases are consistent with the state\-dependent analysis in Appendix[F\.1](https://arxiv.org/html/2610.14573#A6.SS1), where the marginal value of correction decreases as the intermediate state becomes stronger\.

### A\.2Successful Visual Revision

Figure[7](https://arxiv.org/html/2610.14573#A1.F7)shows cases in which Stage 0 already produces a valid program but its rendered result still differs visibly from the target\. Stage 1 receives the target image, the Stage 0 render, and the corresponding TikZ program, and revises the program from this executed state\.

![Refer to caption](https://arxiv.org/html/2610.14573v1/app_fig09.png)Figure 7:Successful visual revisions from Final\-945 at checkpoint 100\. Stage 0 already renders successfully, and Stage 1 revises the executed state to better match the target\. The examples illustrate both large structural corrections and more localized changes to geometry, visual elements, or layout\.The examples show that execution\-conditioned correction is not restricted to small cosmetic edits\. In some cases, Stage 1 substantially reorganizes an incorrect visual configuration, for example by changing geometric structure or restoring a missing layout component\. In others, the correction is more localized, such as modifying the shape, placement, or appearance of a visual element\.

The second action is conditioned on the specific program and render produced by Stage 0 rather than independently resampling from the target image\. The examples therefore illustrate how the policy can use the observed execution result to modify its own previous program\. The cross\-metric improvements reported in Section[4\.3](https://arxiv.org/html/2610.14573#S4.SS3)and Appendix[F\.1](https://arxiv.org/html/2610.14573#A6.SS1)show that these changes are not confined to RSim\.

### A\.3Execution Repair

The corrective action is also used for execution repair\. When Stage 0 fails, no rendered image is available\. Stage 1 instead receives the target image, the unsuccessful TikZ program, and a compact execution status\. Figure[8](https://arxiv.org/html/2610.14573#A1.F8)shows examples in which this route turns an invalid Stage 0 attempt into a valid program\.

![Refer to caption](https://arxiv.org/html/2610.14573v1/app_fig10.png)Figure 8:Successful execution repair from Final\-945\. Stage 0 does not produce a valid render and therefore receives zero execution score\. Stage 1 conditions on the failed program and its execution status and produces a valid TikZ program\. The illustrated repairs include cases in which Stage 0 has already produced a complete document that nevertheless fails compilation\.The selected examples include complete Stage 0 generations that fail during compilation, after which Stage 1 modifies the existing program into an executable one\. The corrective action therefore operates on an unsuccessful program state rather than simply appending missing tokens\. Repair and visual revision receive different observations but use the same improvement\-relative credit\. For an invalid Stage 0 execution,S0=0S\_\{0\}=0, so a successful repair with positive visual similarity receives positive corrective credit\. Repair trajectories are less frequent than visual\-revision trajectories, as quantified in Appendix[F\.3](https://arxiv.org/html/2610.14573#A6.SS3), while their execution outcomes are analyzed in Appendix[F\.4](https://arxiv.org/html/2610.14573#A6.SS4)\.

### A\.4Over\-Revision of Strong Intermediate States

Correction is not always beneficial\. Figure[9](https://arxiv.org/html/2610.14573#A1.F9)shows examples in which Stage 0 already provides a strong reconstruction and Stage 1 modifies the program in a way that reduces visual fidelity\.

![Refer to caption](https://arxiv.org/html/2610.14573v1/app_fig11.png)Figure 9:Examples of over\-revision at checkpoint 100\. Stage 0 already provides a useful reconstruction, while the Stage 1 modification reduces RSim or introduces visible artifacts\. The execution\-aware selection rule retains Stage 0 in these cases\.Some Stage 1 outputs introduce visible changes that are unsupported by the target, such as additional graphical elements, distorted annotations, or large rendering artifacts\. These examples provide a qualitative counterpart to the negative average correction observed for the strongest Stage 0 bucket in Appendix[F\.1](https://arxiv.org/html/2610.14573#A6.SS1)\. Together with the preservation cases in Figure[6](https://arxiv.org/html/2610.14573#A1.F6), they illustrate the asymmetric value of the corrective action: substantial modification can help when the executed state contains clear discrepancies, while an already strong state may be better left largely unchanged\.

When correction\-enabled inference is used, execution\-aware selection can retain Stage 0 when the subsequent correction is worse\. The rule does not generate a new candidate; it only chooses between the two outputs already produced\.

### A\.5Preservation and Unresolved Cases

Stage 1 does not always substantially change the state produced by Stage 0\. Figure[10](https://arxiv.org/html/2610.14573#A1.F10)shows both useful preservation and cases where important errors remain unresolved\. When Stage 0 already provides a strong reconstruction, preserving the existing result can avoid unnecessary modification\. In other cases, however, Stage 0 still contains visible errors and Stage 1 changes little\.

![Refer to caption](https://arxiv.org/html/2610.14573v1/app_fig12.png)Figure 10:Examples of preservation and unresolved cases on Final\-945\. Stage 1 may preserve an already strong Stage 0 result, or leave an imperfect reconstruction largely unchanged\.Some unsuccessful Stage 0 programs also remain non\-executable after the repair action\. Together with the successful revisions, execution repairs, and over\-revisions shown above, these examples illustrate the range of behaviors of the corrective action: it can improve, degrade, or largely preserve the execution state it receives\.

## Appendix BImplementation Details

This section provides implementation details omitted from the main text\. We first give the complete optimization procedure, followed by the prompt templates and execution\-dependent routing used for the two actions\. We then summarize the training configuration and execution environment\.

### B\.1Full Optimization Algorithm

Algorithm[1](https://arxiv.org/html/2610.14573#alg1)summarizes one training iteration of our method\. For each target image, we sampleK=8K=8independent complete trajectories\. Each trajectory contains an initial action followed by a corrective action conditioned on the execution state produced by that initial action\. Both actions are sampled from the same pre\-update policy, and no parameter update occurs during rollout collection\.

Algorithm 1One training iteration of our method1:Policy

πθk\\pi\_\{\\theta\_\{k\}\}, target\-image batch

ℬ\\mathcal\{B\}, group size

K=8K=8, credit scale

λ=10\\lambda=10
2:foreach target image

I⋆∈ℬI^\{\\star\}\\in\\mathcal\{B\}do

3:for

i=1,…,Ki=1,\\ldots,Kdo

4:Sample initial program

C0,i∼πθk\(⋅∣I⋆\)C\_\{0,i\}\\sim\\pi\_\{\\theta\_\{k\}\}\(\\cdot\\mid I^\{\\star\}\)
5:Execute

C0,iC\_\{0,i\}and obtain Stage 0 score

S0,i=\{RSim⁡\(I0,i,I⋆\),if execution succeeds,0,otherwiseS\_\{0,i\}=\\begin\{cases\}\\mathrm\{RSim\}\(I\_\{0,i\},I^\{\\star\}\),&\\text\{if execution succeeds\},\\\\ 0,&\\text\{otherwise\}\\end\{cases\}
6:if

C0,iC\_\{0,i\}produces a valid render

I0,iI\_\{0,i\}then

7:Construct a visual\-revision state

\(I⋆,I0,i,C0,i\)\(I^\{\\star\},I\_\{0,i\},C\_\{0,i\}\)
8:Sample

C1,i∼πθk\(⋅∣I⋆,I0,i,C0,i\)C\_\{1,i\}\\sim\\pi\_\{\\theta\_\{k\}\}\(\\cdot\\mid I^\{\\star\},I\_\{0,i\},C\_\{0,i\}\)
9:else

10:Construct a repair state

\(I⋆,C0,i,e0,i\)\(I^\{\\star\},C\_\{0,i\},e\_\{0,i\}\)
11:Sample

C1,i∼πθk\(⋅∣I⋆,C0,i,e0,i\)C\_\{1,i\}\\sim\\pi\_\{\\theta\_\{k\}\}\(\\cdot\\mid I^\{\\star\},C\_\{0,i\},e\_\{0,i\}\)
12:endif

13:Execute

C1,iC\_\{1,i\}and obtain

S1,i=\{RSim⁡\(I1,i,I⋆\),if execution succeeds,0,otherwiseS\_\{1,i\}=\\begin\{cases\}\\mathrm\{RSim\}\(I\_\{1,i\},I^\{\\star\}\),&\\text\{if execution succeeds\},\\\\ 0,&\\text\{otherwise\}\\end\{cases\}
14:Compute action\-specific returns

G0,i=S1,i,G1,i=S1,i−S0,iG\_\{0,i\}=S\_\{1,i\},\\qquad G\_\{1,i\}=S\_\{1,i\}\-S\_\{0,i\}
15:endfor

16:Mean\-center and scale the two return sets independently

A0,i=λ⁡\(G0,i−1K​∑j=1KG0,j\),A1,i=λ⁡\(G1,i−1K​∑j=1KG1,j\)A\_\{0,i\}=\\lambda\\left\(G\_\{0,i\}\-\\frac\{1\}\{K\}\\sum\_\{j=1\}^\{K\}G\_\{0,j\}\\right\),\\qquad A\_\{1,i\}=\\lambda\\left\(G\_\{1,i\}\-\\frac\{1\}\{K\}\\sum\_\{j=1\}^\{K\}G\_\{1,j\}\\right\)
17:endfor

18:Optimize the Stage 0 loss over action microbatches with weight

0\.50\.5
19:Optimize the Stage 1 loss over action microbatches with weight

0\.50\.5

Equation[7](https://arxiv.org/html/2610.14573#S3.E7)in the main text summarizes the relative weighting of the two action losses\. In the implementation, the two losses are optimized sequentially over action microbatches rather than accumulated into one optimizer update\. All Stage 0 and Stage 1 trajectories in an iteration are generated before this optimization begins\.

Both Direct\-GRPO and our method use the same clipped token\-level loss form and Dr\.GRPO\-style fixed\-length normalization\. For our method, letℳa,b\\mathcal\{M\}\_\{a,b\}denote action microbatchbbof stagea∈\{0,1\}a\\in\\\{0,1\\\}, and letθa,b\\theta\_\{a,b\}denote the current policy parameters when that microbatch is processed\. For each completion token, the implementation first computes a detached reference log probability from the current policy,

qa,i,tref=sg⁡\[log⁡πθa,b​\(ca,i,t∣xa,i,ca,i,<t\)\],q^\{\\mathrm\{ref\}\}\_\{a,i,t\}=\\operatorname\{sg\}\\\!\\left\[\\log\\pi\_\{\\theta\_\{a,b\}\}\(c\_\{a,i,t\}\\mid x\_\{a,i\},c\_\{a,i,<t\}\)\\right\],\(9\)wheresg⁡\[⋅\]\\operatorname\{sg\}\[\\cdot\]denotes stop\-gradient andxa,ix\_\{a,i\}is the action\-specific context\. A differentiable forward pass of the same current policy is then used to form

ra,i,t​\(θ\)=exp⁡\(log⁡πθ​\(ca,i,t∣xa,i,ca,i,<t\)−qa,i,tref\),r\_\{a,i,t\}\(\\theta\)=\\exp\\left\(\\log\\pi\_\{\\theta\}\(c\_\{a,i,t\}\\mid x\_\{a,i\},c\_\{a,i,<t\}\)\-q^\{\\mathrm\{ref\}\}\_\{a,i,t\}\\right\),\(10\)with the differentiable forward evaluated at the current parametersθ=θa,b\\theta=\\theta\_\{a,b\}before the corresponding update\.

The clipped token\-level surrogate is

ℓa,i,t​\(θ\)=−min⁡\[ra,i,t​\(θ\)​Aa,i,clip⁡\(ra,i,t​\(θ\),1−ϵlow,1\+ϵhigh\)​Aa,i\]\.\\ell\_\{a,i,t\}\(\\theta\)=\-\\min\\left\[r\_\{a,i,t\}\(\\theta\)A\_\{a,i\},\\;\\operatorname\{clip\}\\left\(r\_\{a,i,t\}\(\\theta\),1\-\\epsilon\_\{\\mathrm\{low\}\},1\+\\epsilon\_\{\\mathrm\{high\}\}\\right\)A\_\{a,i\}\\right\]\.\(11\)
For an action microbatchℳa,b\\mathcal\{M\}\_\{a,b\}, the optimized loss is

ℒa,b=waNa​Tmax​∑i∈ℳa,b∑t=1Tmaxma,i,t​ℓa,i,t​\(θ\),w0=w1=0\.5,\\mathcal\{L\}\_\{a,b\}=\\frac\{w\_\{a\}\}\{N\_\{a\}T\_\{\\max\}\}\\sum\_\{i\\in\\mathcal\{M\}\_\{a,b\}\}\\sum\_\{t=1\}^\{T\_\{\\max\}\}m\_\{a,i,t\}\\,\\ell\_\{a,i,t\}\(\\theta\),\\qquad w\_\{0\}=w\_\{1\}=0\.5,\(12\)whereNaN\_\{a\}is the number of local actions for stageaain the training iteration,TmaxT\_\{\\max\}is the maximum completion length, andma,i,tm\_\{a,i,t\}masks padding and non\-completion positions\. The denominator remains fixed at the stage level when the actions are split into microbatches\. Completions that reach the generation limit are masked from the policy loss\.

We useϵlow=0\.20\\epsilon\_\{\\mathrm\{low\}\}=0\.20andϵhigh=0\.28\\epsilon\_\{\\mathrm\{high\}\}=0\.28\. No group\-standard\-deviation normalization is applied, and no KL term is added \(β=0\\beta=0\)\. Only completion tokens contribute to the policy loss\. Because the detached reference log probabilities are recomputed from the current policy immediately before each differentiable forward pass, they are not a frozen copy of the policy that generated the rollout\. Consistent with this implementation, the recorded clipping ratio is zero throughout the formal 8B run, so the clipping branch is inactive; the advantage\-weighted token loss and its gradients remain active\.

For the formal 8B run, training uses four ranks with two target images per rank, yielding eight target\-image groups per training iteration\. Each group contains eight complete trajectories, and each rank receives 16 actions per stage for optimization\. With an action microbatch size of two, each stage is processed in eight action microbatches\. Under the executed ZeRO\-2 configuration, each such microbatch reaches an AdamW update boundary, giving eight updates for Stage 0 followed by eight updates for Stage 1, or 16 AdamW updates per training iteration\. Checkpoint labels such as 25, 50, 75, 100, and 125 therefore refer to training iterations rather than individual optimizer updates\.

### B\.2Prompt and Routing Contract

Three prompt templates are used: direct generation, visual revision, and execution repair\. The direct\-generation prompt is shared with Direct\-SFT and Direct\-GRPO and is used for Stage 0 of our method\. Stage 1 uses either the revision or repair prompt according to the Stage 0 execution result\. The templates below reproduce the prompts used in the formal experiments; bracketed placeholders denote dynamically inserted state information\.

#### B\.2\.1Direct Generation Prompt

The direct prompt receives only the target image\.

Figure 11:Direct generation prompt\.
#### B\.2\.2Visual Revision Prompt

When Stage 0 renders successfully, Stage 1 receives the target image, the current Stage 0 render, and the current TikZ source\. The source is inserted at<CURRENT TIKZ PROGRAM\>\.

Figure 12:Visual revision prompt\.
#### B\.2\.3Execution Repair Prompt

When Stage 0 does not produce a valid render, Stage 1 receives the target image, the unsuccessful TikZ attempt, and a compact execution status instead of a current render\.

Figure 13:Execution repair prompt\.

### B\.3Execution Routing and State Construction

Routing is determined solely by the execution result of Stage 0\. A successful render routes the trajectory to visual revision; otherwise it is routed to repair or completion\. The routing decision does not use RSim or any learned threshold\.

For the revision route, the Stage 1 multimodal context is ordered as

target image→current rendered image→current TikZ source\.\\text\{target image\}\\;\\rightarrow\\;\\text\{current rendered image\}\\;\\rightarrow\\;\\text\{current TikZ source\}\.The rendered image and TikZ source are produced by the same Stage 0 action, so the second call observes the visual consequence of the exact program it is asked to modify\.

For the repair route, the context contains

target image→compact execution status→current TikZ attempt\.\\text\{target image\}\\;\\rightarrow\\;\\text\{compact execution status\}\\;\\rightarrow\\;\\text\{current TikZ attempt\}\.The status is intentionally compact rather than a full compiler log\. Depending on the failure, it can include the tagshit\_max\_tokens,document\_incomplete, and the leading category of the renderer/compiler error;render\_failedis used as a fallback\.

A generated program is considered complete at the first`\\end\{document\}`\. Generation terminates independently for each sequence once this string is produced, subject to a maximum of 8192 completion tokens for either action\.

### B\.4Training Configuration

The training subsets and initialization sequence are summarized in Appendix[C\.1](https://arxiv.org/html/2610.14573#A3.SS1)\. Here we report the optimization and runtime configuration used for Direct\-SFT, Direct\-GRPO, and our method\.

Table 3:Optimization and runtime configuration for Direct\-SFT, Direct\-GRPO, and our method\. The final column reports the configuration used by our method in the main experiments\.Direct\-SFTDirect\-GRPOOursTraining duration3 epochs1 epoch / 125 iterations1 epoch / 125 iterationsLearning rate1×10−41\\times 10^\{\-4\}5×10−65\\times 10^\{\-6\}5×10−65\\times 10^\{\-6\}Weight decay–10−210^\{\-2\}10−210^\{\-2\}LoRAr=256,α=512r=256,\\ \\alpha=512continuedcontinuedGroup sizeKK–88GRPO loss–Dr\.GRPODr\.GRPOβ\\beta–00Clip range–\[0\.80,1\.28\]\[0\.80,1\.28\]\[0\.80,1\.28\]\[0\.80,1\.28\]Reward normalization–nonenoneCredit scaleλ\\lambda––10Action loss weights––0\.5/0\.50\.5/0\.5Sampling temperature–1\.01\.0Top\-pp–0\.90\.9Maximum completion2,0488,1928,192 / actionImage resolution448×448448\\times 448448×448448\\times 448448×448448\\times 448PrecisionBF16BF16BF16Gradient checkpointing✓✓✓Distributed trainingDDPZeRO\-2ZeRO\-2A100\-80GB GPUs244Direct\-SFT uses a global batch size of 32 \(two GPUs, eight examples per GPU, and two gradient\-accumulation steps\)\. Direct\-GRPO uses two target prompts per device with gradient accumulation of eight andK=8K=8sampled completions per target\. Across four devices, this corresponds to eight target\-image groups per training iteration and one AdamW update per iteration\.

Our method also uses two target images per rank on four ranks, yielding eight target\-image groups per training iteration; each group contains eight independent complete trajectories\. The policy loss is evaluated in action microbatches of two\. Each rank therefore processes 16 actions per stage as eight action microbatches, which gives 16 AdamW updates per training iteration under the executed ZeRO\-2 configuration, as detailed in Appendix[B\.1](https://arxiv.org/html/2610.14573#A2.SS1)\. The main run uses random seed 20260823\.

For our method, rollout generation uses temperature1\.01\.0, top\-pp0\.90\.9, repetition penalty1\.01\.0, and a maximum of 8192 tokens for each action\. Stage 0 and Stage 1 generation use microbatch sizes of six and three, respectively\. Training uses AdamW with weight decay10−210^\{\-2\}, a constant learning\-rate schedule, no warmup, and gradient clipping at norm1\.01\.0\.

### B\.5Execution and Reward Environment

Direct\-GRPO and our method use the RSim\-based execution score defined in Eq\.[1](https://arxiv.org/html/2610.14573#S2.E1)during reinforcement learning\. For our method, execution uses TeX Live 2026 and attemptspdflatex,lualatex, andxelatex, with shell escape disabled\. Compilation has a 30\-second timeout, and rasterization uses 150 DPI before conversion to the448×448448\\times 448model input resolution\. Failed compilation or rendering receivesSt=0S\_\{t\}=0\. The same execution score is used for both actions; their learning signals differ through the action\-specific returns defined in Section[3\.2](https://arxiv.org/html/2610.14573#S3.SS2)\.

## Appendix CData and Evaluation Protocol

This section provides additional details on the training subsets, held\-out evaluation data, evaluation metrics, inference protocol, and checkpoint selection used throughout the experiments\.

### C\.1Training Data

All supervised and reinforcement\-learning data are drawn from DaTikZ\-V4\[[14](https://arxiv.org/html/2610.14573#bib.bib31)\]\. Direct\-SFT is trained on 3,851 eligible image–TikZ pairs obtained from an approximately 4K candidate subset after preprocessing\. Direct\-GRPO uses 1,000 examples sampled from this supervised\-training pool\. Our method is initialized from the final Direct\-GRPO policy and reuses the same 1,000 target images\. The corrective action is generated from execution states produced by the policy’s own predictions, so no additional training examples or paired program annotations are introduced\.

Table 4:Training data used by the main learning settings\. The Direct\-GRPO subset is contained in the supervised\-training pool, and the same 1K subset is reused by our method\.MethodTraining ExamplesInitializationTraining SignalDirect\-SFT3,851BaseReference TikZ codeDirect\-GRPO1,000Direct\-SFTExecuted outcomeSSOurs1,000Direct\-GRPOS1S\_\{1\}/S1−S0S\_\{1\}\-S\_\{0\}For the continued Direct\-GRPO controls in Section[4\.2](https://arxiv.org/html/2610.14573#S4.SS2), we continue Direct\-GRPO from the same final Direct\-GRPO checkpoint on the same 1K target images while retaining the single\-stage Direct\-GRPO objective\. We evaluate controls after 100 additional training iterations, matching the training\-iteration budget and target\-image exposure at the Dev\-selected checkpoint of our method, and after 125 additional training iterations as an endpoint comparison\. The controls do not match the number of low\-level optimizer updates performed by our implementation; the corresponding update schedule is described in Appendix[B](https://arxiv.org/html/2610.14573#A2)\.

### C\.2Held\-Out Evaluation Data

We construct the evaluation set independently from the training subsets using the same broad source families as DaTikZ\-V4: arXiv, GitHub, and TeX StackExchange\. Evaluation examples are restricted to source records from 2026 onward; the examples used in this work span January to mid July 2026\. Each retained item contains an executable reference TikZ program and its corresponding rendered target image, and all examples undergo manual review during dataset construction\.

The paper uses 1,095 held\-out examples\. We fix 150 examples asDev\-150for checkpoint selection, ablations, and behavioral analysis, and reserve the remaining 945 examples asFinal\-945for final evaluation\. Final\-945 is not used for checkpoint selection or method development\.

Table 5:Composition of the held\-out evaluation set\. All samples are drawn from source records dated in 2026\.SplitarXivGitHubTeX\.SETotalDev\-150824325150Final\-945458309178945All5403522031,095The two splits contain distinct evaluation examples and serve different purposes throughout the paper: all checkpoint\-dependent analysis is performed on Dev\-150, while Final\-945 is evaluated only after the checkpoint and inference procedure have been fixed\.

### C\.3Evaluation Metrics

We evaluate both executability and visual fidelity\.*Render success*indicates whether a generated program can be successfully compiled and rasterized\. For a valid renderII, visual fidelity to the target imageI⋆I^\{\\star\}is measured using four complementary image\-similarity metrics: RSim\[[14](https://arxiv.org/html/2610.14573#bib.bib31)\], DINO\[[17](https://arxiv.org/html/2610.14573#bib.bib16)\], DreamSim\[[18](https://arxiv.org/html/2610.14573#bib.bib17)\], and CLIPImg\[[19](https://arxiv.org/html/2610.14573#bib.bib18)\]\.

RSim uses the DeTikZify\-V2 vision encoder and serves as the only visual optimization reward in Direct\-GRPO and our method\. DINO, DreamSim, and CLIPImg are used only for evaluation and do not participate in training or execution\-aware selection\. We report DreamSim similarity as one minus the perceptual distance returned by the official DreamSim implementation\.

Table 6:Evaluation metrics and their roles\. All main visual results use the fixed\-zero convention\.MetricRole in TrainingMain Evaluation ConventionRender successExecution signalSuccess / failureRSimRL reward \+ evaluationFailure→0\\rightarrow 0DINOEvaluation onlyFailure→0\\rightarrow 0DreamSimEvaluation onlyFailure→0\\rightarrow 0CLIPImgEvaluation onlyFailure→0\\rightarrow 0Under the*fixed\-zero*convention, unsuccessful executions receive zero for every visual metric and successful executions retain their measured similarity\. This convention is used for the main aggregate results because it reflects both executability and visual quality\. When analyzing the visual effect of the corrective action independently of execution recovery, we instead restrict to the*paired\-success*subset in which both Stage 0 and Stage 1 render successfully, as defined in Section[4\.3](https://arxiv.org/html/2610.14573#S4.SS3)\.

### C\.4Inference and Rendering Protocol

All paper\-active evaluations use stochastic decoding with temperature1\.01\.0, top\-pp0\.90\.9, repetition penalty1\.01\.0, a maximum of 8192 newly generated tokens, and early termination at the first complete`\\end\{document\}`\.

Direct models produce a single output from the target image\. For our method, Stage 0 is generated from the target image alone and is reported as the single\-call inference output\. In correction\-enabled evaluation, the same policy is called again after Stage 0 execution to produce Stage 1 according to the routing procedure in Appendix[B\.3](https://arxiv.org/html/2610.14573#A2.SS3)\.

The final*Selected*output is chosen only from the two programs already produced:

- •if only Stage 0 renders, Stage 0 is retained;
- •if only Stage 1 renders, Stage 1 is retained;
- •if both render, the program with higher RSim is retained;
- •ties are assigned to Stage 0;
- •if neither renders, the selected output is unsuccessful\.

No additional generation is performed by the selection procedure\.

Generation parameters and evaluation metrics are unified across the paper\-active evaluations\. The Direct Final\-945 evaluation uses a 120\-second compilation timeout, whereas the correction\-enabled renderer uses a 30\-second timeout; all successful Direct predictions in Final\-945 execute within 30 seconds\. Both pipelines normalize rendered images to448×448448\\times 448before visual evaluation\.

### C\.5Checkpoint Selection

Checkpoints of our method are evaluated on Dev\-150 at the initializer and after 25, 50, 75, 100, and 125 training iterations\. For consistency with the training logs and figures, we retain the checkpoint labels Step 25 through Step 125 throughout the paper\. For each checkpoint, we evaluate Stage 0, raw Stage 1, and the execution\-aware selected output using the same inference and metric protocol\.

Table[7](https://arxiv.org/html/2610.14573#A3.T7)reports the quantities used to compare the checkpoints\. Among the evaluated checkpoints, checkpoint 100 achieves the highest selected render rate as well as the highest selected RSim, DINO, DreamSim, and CLIPImg scores\. We therefore fix checkpoint 100 for subsequent single\-policy behavioral analyses and for the final evaluation on Final\-945\. Training\-dynamics analyses instead report the full checkpoint trajectory\.

Table 7:Checkpoint evaluation for our method on Dev\-150\. Visual metrics are computed on the execution\-aware selected output using the fixed\-zero convention\. Checkpoint 100 gives the highest selected render rate and all four selected visual metrics\.CheckpointStage 0 RenderStage 1 RenderSelected RenderRSimDINODreamSimCLIPImgInitializer82\.00%82\.67%82\.67%0\.47360\.70880\.64970\.7468Step 2587\.33%88\.00%89\.33%0\.51610\.76670\.70760\.8116Step 5089\.33%88\.00%90\.67%0\.52230\.78050\.71600\.8196Step 7591\.33%90\.00%92\.00%0\.54540\.79750\.73860\.8416Step 10096\.67%94\.67%97\.33%0\.56460\.83260\.76770\.8774Step 12595\.33%92\.67%95\.33%0\.55490\.81170\.75380\.8666The selected checkpoint is fixed before Final\-945 evaluation\. Final\-945 is therefore used only to report final performance rather than to choose between training checkpoints or analysis settings\.

## Appendix DExtended Learning Dynamics and Controls

This section provides the complete numerical results behind the learning dynamics reported in the main text, compares our method with continued single\-stage Direct\-GRPO under matched training\-iteration budgets, and quantifies sample\-level uncertainty for selected Final\-945 comparisons\. All Dev\-150 checkpoint results follow the evaluation protocol described in Appendix[C](https://arxiv.org/html/2610.14573#A3)\.

### D\.1Complete Learning Dynamics

We evaluate the initializer and checkpoints at steps 25, 50, 75, 100, and 125 on Dev\-150\. Table[8](https://arxiv.org/html/2610.14573#A4.T8)reports the execution behavior of both actions together with the four possible execution transitions\. We denote successful and failed executions byS\\mathrm\{S\}andF\\mathrm\{F\}, respectively\. For example,FS\\mathrm\{FS\}denotes a Stage 0 failure followed by a successful Stage 1 repair\.

Table 8:Complete execution dynamics of our method on Dev\-150\. Each row contains 150 examples\.SS\\mathrm\{SS\},SF\\mathrm\{SF\},FS\\mathrm\{FS\}, andFF\\mathrm\{FF\}denote the execution outcomes of Stage 0 and Stage 1, respectively\.CheckpointStage 0 RenderStage 1 RenderSSSFFSFFInitializer82\.00%82\.67%1230126Step 2587\.33%88\.00%1292316Step 5089\.33%88\.00%1304214Step 7591\.33%90\.00%1343112Step 10096\.67%94\.67%141414Step 12595\.33%92\.67%139407The execution rates improve substantially over training\. Stage 0 rises from82\.00%82\.00\\%for the initializer to96\.67%96\.67\\%at step 100, while Stage 1 rises from82\.67%82\.67\\%to94\.67%94\.67\\%\. Most of this change comes from the reduction ofFF\\mathrm\{FF\}trajectories\. As Stage 0 improves, failed initial executions also become less frequent, so the repair route is exercised less often on later Dev\-150 checkpoints\. Appendix[F](https://arxiv.org/html/2610.14573#A6)analyzes routing and repair in more detail\.

### D\.2Paired\-Success Correction Dynamics

To isolate visual correction from execution recovery, we use the paired\-success subset defined in Section[4\.3](https://arxiv.org/html/2610.14573#S4.SS3)\. For each visual metricMM, the reported quantity is

Δ​M=M⁡\(I1,I⋆\)−M⁡\(I0,I⋆\)\.\\Delta M=M\(I\_\{1\},I^\{\\star\}\)\-M\(I\_\{0\},I^\{\\star\}\)\.Positive values indicate that Stage 1 improves the already executable Stage 0 state\.

Table[9](https://arxiv.org/html/2610.14573#A4.T9)reports the complete paired\-success visual dynamics\. The correction\-outcome categories follow the definition used in the main text:Δ​RSim\>0\.005\\Delta\\mathrm\{RSim\}\>0\.005is an improvement,\|Δ​RSim\|≤0\.005\|\\Delta\\mathrm\{RSim\}\|\\leq 0\.005is neutral, andΔ​RSim<−0\.005\\Delta\\mathrm\{RSim\}<\-0\.005is a degradation\.

Table 9:Visual correction dynamics on the paired\-success subset of Dev\-150\. The fourΔ\\Deltacolumns report the mean Stage 1 minus Stage 0 change\. The last three columns show the composition of correction outcomes under the±0\.005\\pm 0\.005RSim neutral region used in the main text\.CheckpointPairednnΔ\\DeltaRSimΔ\\DeltaDINOΔ\\DeltaDreamSimΔ\\DeltaCLIPImgImproveNeutralDegradeInitializer123−0\.0031\-0\.0031−0\.0027\-0\.0027−0\.0039\-0\.0039−0\.0017\-0\.001712\.20%65\.04%22\.76%Step 25129\+0\.0013\+0\.0013−0\.0014\-0\.0014\+0\.0043\+0\.0043\+0\.0042\+0\.004228\.68%48\.06%23\.26%Step 50130\+0\.0098\+0\.0098\+0\.0053\+0\.0053\+0\.0077\+0\.0077\+0\.0016\+0\.001643\.85%26\.92%29\.23%Step 75134\+0\.0121\+0\.0121\+0\.0058\+0\.0058\+0\.0112\+0\.0112\+0\.0063\+0\.006342\.54%30\.60%26\.87%Step 100141−0\.0012\-0\.0012−0\.0072\-0\.0072−0\.0047\-0\.0047−0\.0000\-0\.000043\.26%19\.86%36\.88%Step 125139\+0\.0065\+0\.0065\+0\.0057\+0\.0057\+0\.0025\+0\.0025−0\.0013\-0\.001346\.76%19\.42%33\.81%The complete trajectory highlights the distinction between initial generation quality and corrective behavior\. The initializer has negative average Stage 1 changes under all four visual metrics\. By steps 50–75, correction becomes more effective, and all four mean changes are positive at step 75\. The trend is not monotonic: at step 100, Stage 0 reaches its highest render rate, while the average paired\-success change becomes slightly negative\. At step 125, RSim, DINO, and DreamSim again show positive average changes\.

Checkpoint selection is therefore based on the complete inference result rather than the paired\-success correction gain alone\. Step 100 provides the strongest selected result on Dev\-150 \(Appendix[C\.5](https://arxiv.org/html/2610.14573#A3.SS5)\), while Appendix[F\.1](https://arxiv.org/html/2610.14573#A6.SS1)shows that its corrective behavior varies with the quality of the intermediate state\.

### D\.3Continued Direct\-GRPO Controls

We compare our method with continued single\-stage Direct\-GRPO from the same Direct\-GRPO initializer and on the same 1K training images\. The\+100\+100comparison matches the additional training\-iteration budget and target\-image exposure at the Dev\-selected checkpoint used for final evaluation\. The\+125\+125comparison uses the corresponding training endpoint\. The controls match initialization, training data, training\-iteration count, and target\-image exposure, but not the number of low\-level optimizer updates; our method additionally samples the execution\-conditioned corrective action during training\.

Table[10](https://arxiv.org/html/2610.14573#A4.T10)compares single\-call outputs on Final\-945\. For our method, this is the Stage 0 output generated directly from the target image without invoking correction at inference time\.

Table 10:Continued Direct\-GRPO single\-call results on Final\-945\. Both methods start from the same Direct\-GRPO checkpoint and use the same 1K training images\. The\+100\+100row corresponds to the Dev\-selected checkpoint of our method;\+125\+125reports the training endpoint\.Additional IterationsMethodRenderRSimDINODreamSimCLIPImg\+100\+100Continued Direct\-GRPO88\.25%0\.51350\.76290\.70190\.7996\+100\+100Ours \(Stage 0\)92\.70%0\.53820\.80140\.73300\.8372\+125\+125Continued Direct\-GRPO87\.20%0\.51190\.75570\.69840\.7952\+125\+125Ours \(Stage 0\)93\.54%0\.53560\.80110\.73460\.8431Continued Direct\-GRPO improves over the original Direct\-GRPO checkpoint, but our Stage 0 output is higher under render success and all four visual metrics at both corresponding training endpoints\. The gain from training is therefore already present in the initial action and does not require the second call at inference time\.

We also evaluate the continued Direct\-GRPO policies with the same execution\-conditioned second call\. Table[11](https://arxiv.org/html/2610.14573#A4.T11)reports paired\-success visual changes on Final\-945\. At both corresponding endpoints, the continued single\-stage policies produce changes close to zero, whereas our corrective action yields larger positive mean changes across the reported metrics\.

Table 11:Continued Direct\-GRPO paired\-success correction on Final\-945\.Δ​M\\Delta Mdenotes Stage 1 minus Stage 0 on examples where both outputs render successfully\.AdditionalIterationsMethodPairednnΔ\\DeltaRSimΔ\\DeltaDINOΔ\\DeltaDreamSimΔ\\DeltaCLIPImg\+100\+100Continued Direct\-GRPO828\+0\.0002\+0\.0002\+0\.0019\+0\.0019\+0\.0020\+0\.0020−0\.0007\-0\.0007\+100\+100Ours863\+0\.0073\+0\.0073\+0\.0034\+0\.0034\+0\.0074\+0\.0074\+0\.0031\+0\.0031\+125\+125Continued Direct\-GRPO815\+0\.0017\+0\.0017\+0\.0013\+0\.0013\+0\.0009\+0\.0009\+0\.0008\+0\.0008\+125\+125Ours869\+0\.0079\+0\.0079\+0\.0046\+0\.0046\+0\.0068\+0\.0068\+0\.0035\+0\.0035For the\+125\+125continued Direct\-GRPO correction analysis, paired statistics use the Stage 0 records from the correction\-enabled evaluation run\. The canonical single\-call Direct result is reported separately in Table[10](https://arxiv.org/html/2610.14573#A4.T10)\.

### D\.4Paired Bootstrap on Final\-945

We quantify uncertainty over the fixed Final\-945 evaluation set using a paired bootstrap\. For each comparison, we resample the same 945 example indices with replacement for both methods and compute the difference in their mean performance\. We use 10,000 paired resamples with a fixed random seed of 20260914 and report percentile95%95\\%intervals\. Positive differences indicate better performance for the second method\.

Table[12](https://arxiv.org/html/2610.14573#A4.T12)reports three fixed comparisons: from supervised learning to direct execution feedback, from Direct\-GRPO to the Stage 1 output of our trained policy, and from raw Stage 1 to execution\-aware selection\.

Table 12:Paired bootstrap uncertainty on the fixed Final\-945 evaluation set \(N=945N=945, 10,000 paired resamples\)\. Each entry reports the observed difference with its95%95\\%bootstrap interval\. Positive values favor the second method\. Render differences are in percentage points; visual metrics use fixed\-zero scores\. Intervals quantify uncertainty over the fixed evaluation samples rather than variation across training seeds\.MetricSFT→\\rightarrowDirect\-GRPODirect\-GRPO→\\rightarrowOurs Stage 1Stage 1→\\rightarrowSelectedΔ\\DeltaRender Success\+17\.25\+17\.25pp\[\+13\.86,\+20\.63\]\[\+13\.86,\+20\.63\]\+10\.90\+10\.90pp\[\+8\.25,\+13\.54\]\[\+8\.25,\+13\.54\]\+1\.38\+1\.38pp\[\+0\.74,\+2\.12\]\[\+0\.74,\+2\.12\]Δ\\DeltaRSim\+0\.1154\+0\.1154\[\+0\.0962,\+0\.1345\]\[\+0\.0962,\+0\.1345\]\+0\.0738\+0\.0738\[\+0\.0580,\+0\.0892\]\[\+0\.0580,\+0\.0892\]\+0\.0203\+0\.0203\[\+0\.0162,\+0\.0249\]\[\+0\.0162,\+0\.0249\]Δ\\DeltaDINO\+0\.1619\+0\.1619\[\+0\.1334,\+0\.1905\]\[\+0\.1334,\+0\.1905\]\+0\.1000\+0\.1000\[\+0\.0765,\+0\.1238\]\[\+0\.0765,\+0\.1238\]\+0\.0175\+0\.0175\[\+0\.0113,\+0\.0243\]\[\+0\.0113,\+0\.0243\]Δ\\DeltaDreamSim\+0\.1508\+0\.1508\[\+0\.1244,\+0\.1773\]\[\+0\.1244,\+0\.1773\]\+0\.0932\+0\.0932\[\+0\.0721,\+0\.1149\]\[\+0\.0721,\+0\.1149\]\+0\.0192\+0\.0192\[\+0\.0137,\+0\.0254\]\[\+0\.0137,\+0\.0254\]Δ\\DeltaCLIPImg\+0\.1629\+0\.1629\[\+0\.1333,\+0\.1939\]\[\+0\.1333,\+0\.1939\]\+0\.0999\+0\.0999\[\+0\.0755,\+0\.1245\]\[\+0\.0755,\+0\.1245\]\+0\.0174\+0\.0174\[\+0\.0111,\+0\.0246\]\[\+0\.0111,\+0\.0246\]The intervals exclude zero for all five metrics in each comparison\. Direct execution feedback provides a large improvement over supervised code learning, the trained policy provides a further positive shift over the preceding Direct\-GRPO checkpoint, and execution\-aware selection adds a smaller but consistent improvement over raw Stage 1\.

These intervals quantify uncertainty over the fixed Final\-945 sample set\. They do not measure variation across independent training runs\.

## Appendix ECredit Assignment Analysis

This section expands the credit\-assignment analysis in Section[4\.4](https://arxiv.org/html/2610.14573#S4.SS4)\. We first report the controlled comparison between downstream and local credit for the initial action\. We then compare the group\-relative preferences induced by alternative credit definitions on the same sampled trajectories, followed by the optimization behavior of the four factorized variants\.

### E\.1Factorized Credit Variants

The ablation varies the credit assigned to the initial and corrective actions\. Stage 0 either receives the final trajectory scoreS1S\_\{1\}or its local intermediate scoreS0S\_\{0\}\. Stage 1 either receives its improvement over the intermediate state,S1−S0S\_\{1\}\-S\_\{0\}, or the absolute final scoreS1S\_\{1\}\. All variants use the same group centering and scaleλ=10\\lambda=10defined in Section[3\.3](https://arxiv.org/html/2610.14573#S3.SS3)\.

Table 13:Factorized credit\-assignment variants\. Variant A is our method\. The table reports the raw credit before group centering and scaling\.VariantStage 0 creditStage 1 creditA \(Ours\)S1S\_\{1\}S1−S0S\_\{1\}\-S\_\{0\}BS0S\_\{0\}S1−S0S\_\{1\}\-S\_\{0\}CS1S\_\{1\}S1S\_\{1\}DS0S\_\{0\}S1S\_\{1\}A versus B isolates the Stage 0 credit while keeping improvement\-relative Stage 1 credit fixed\. A versus C isolates the Stage 1 credit while keeping downstream Stage 0 credit fixed\. Variants C and D both use absolute Stage 1 credit with different Stage 0 definitions\.

### E\.2Downstream versus Local Stage 0 Credit

Table[14](https://arxiv.org/html/2610.14573#A5.T14)gives the complete Dev\-150 comparison between variants A and B at the checkpoints used in the main analysis\. The paired visual changes are computed only on examples for which both stages render successfully\.

Table 14:Comparison of downstream Stage 0 credit \(A\) and local Stage 0 credit \(B\) on Dev\-150\.Δ​M\\Delta Mdenotes the mean Stage 1 minus Stage 0 change on the paired\-success subset\. A and B use the same improvement\-relative Stage 1 credit and differ only in whether Stage 0 receivesS1S\_\{1\}orS0S\_\{0\}\.CheckpointVariantStage 0 RenderStage 1 RenderΔ\\DeltaRSimΔ\\DeltaDINOΔ\\DeltaDreamSimΔ\\DeltaCLIPImgStep 25A87\.33%88\.00%\+0\.0013\+0\.0013−0\.0014\-0\.0014\+0\.0043\+0\.0043\+0\.0042\+0\.0042B84\.67%86\.00%\+0\.0012\+0\.0012−0\.0015\-0\.0015\+0\.0005\+0\.0005−0\.0007\-0\.0007Step 50A89\.33%88\.00%\+0\.0098\+0\.0098\+0\.0053\+0\.0053\+0\.0077\+0\.0077\+0\.0016\+0\.0016B90\.00%89\.33%\+0\.0002\+0\.0002\+0\.0023\+0\.0023\+0\.0038\+0\.0038\+0\.0001\+0\.0001Step 75A91\.33%90\.00%\+0\.0121\+0\.0121\+0\.0058\+0\.0058\+0\.0112\+0\.0112\+0\.0063\+0\.0063B88\.00%88\.67%−0\.0008\-0\.0008−0\.0023\-0\.0023−0\.0037\-0\.0037−0\.0014\-0\.0014The comparison separates Stage 0 quality from the behavior of the subsequent corrective action\. At step 50, B has a slightly higher Stage 0 render rate than A \(90\.00%90\.00\\%versus89\.33%89\.33\\%\), and its Stage 0 RSim\-FZ is also higher \(0\.50950\.5095versus0\.49600\.4960\)\. Its paired Stage 1 correction, however, is nearly neutral:Δ​RSim=\+0\.0002\\Delta\\mathrm\{RSim\}=\+0\.0002, compared with\+0\.0098\+0\.0098for A\. By step 75, A retains a positive correction gain of\+0\.0121\+0\.0121, whereas B becomes slightly negative at−0\.0008\-0\.0008\.

Thus, a stronger intermediate output at one checkpoint does not imply a stronger subsequent correction\. UsingS1S\_\{1\}rather thanS0S\_\{0\}for Stage 0 changes which initial actions are favored according to the quality reached after the corrective action\.

### E\.3Direct Audit of Credit\-Induced Preferences

The trained ablations compare the behavior produced by different credit definitions\. We additionally inspect the credit signals while holding the sampled trajectories fixed\. For this analysis, we use the completed A training run, containing 1,000 target\-image groups and eight independently sampled complete trajectories per group, for a total of 8,000 trajectories\.

For Stage 0, we compare the downstream advantage

A0​\(S1\)=S1−S1¯A\_\{0\}\(S\_\{1\}\)=S\_\{1\}\-\\overline\{S\_\{1\}\}with the local advantage

A0​\(S0\)=S0−S0¯\.A\_\{0\}\(S\_\{0\}\)=S\_\{0\}\-\\overline\{S\_\{0\}\}\.For Stage 1, we compare the improvement\-relative advantage

A1​\(S1−S0\)=\(S1−S0\)−\(S1−S0\)¯A\_\{1\}\(S\_\{1\}\-S\_\{0\}\)=\(S\_\{1\}\-S\_\{0\}\)\-\\overline\{\(S\_\{1\}\-S\_\{0\}\)\}with the absolute\-score advantage

A1​\(S1\)=S1−S1¯\.A\_\{1\}\(S\_\{1\}\)=S\_\{1\}\-\\overline\{S\_\{1\}\}\.The common factorλ=10\\lambda=10is omitted because it does not affect correlation, sign, or ranking\.

Table 15:Comparison of group\-relative advantages induced by alternative credit definitions on the same 8,000 trajectories from 1,000 target\-image groups\. Sign flips are computed over all trajectories, with exact\-zero advantages counted as neither positive nor negative\. Top\-1 change denotes the fraction of groups whose highest\-advantage trajectory changes under the alternative definition\.Advantage ComparisonPearsonSpearmanSign FlipGroup Top\-1 ChangePairwise OrderingChangeA0​\(S1\)A\_\{0\}\(S\_\{1\}\)vs\.A0​\(S0\)A\_\{0\}\(S\_\{0\}\)0\.74080\.708321\.4%46\.1%–A1​\(S1−S0\)A\_\{1\}\(S\_\{1\}\-S\_\{0\}\)vs\.A1​\(S1\)A\_\{1\}\(S\_\{1\}\)0\.38480\.314637\.2%74\.5%41\.76%For Stage 0, the two definitions remain moderately correlated but produce different group\-relative preferences\. The advantage sign changes for1,715/8,000=21\.4%1\{,\}715/8\{,\}000=21\.4\\%of trajectories, and the highest\-advantage trajectory changes in461/1,000=46\.1%461/1\{,\}000=46\.1\\%of target\-image groups\.

The distinction is larger for Stage 1\. Improvement\-relative and absolute credit have a Pearson correlation of0\.38480\.3848and a Spearman correlation of0\.31460\.3146\. The advantage sign changes for2,974/8,000=37\.2%2\{,\}974/8\{,\}000=37\.2\\%of trajectories, and the highest\-advantage corrective trajectory changes in745/1,000=74\.5%745/1\{,\}000=74\.5\\%of groups\.

We additionally compare every unordered pair of trajectories within each group\. For a pair\(i,j\)\(i,j\), we compare the sign of

A1,i​\(S1−S0\)−A1,j​\(S1−S0\)A\_\{1,i\}\(S\_\{1\}\-S\_\{0\}\)\-A\_\{1,j\}\(S\_\{1\}\-S\_\{0\}\)with the sign of

A1,i​\(S1\)−A1,j​\(S1\)\.A\_\{1,i\}\(S\_\{1\}\)\-A\_\{1,j\}\(S\_\{1\}\)\.Ties form a separate ordering category\. Of the28,00028\{,\}000within\-group trajectory pairs,11,69311\{,\}693change their relative ordering, corresponding to41\.76%41\.76\\%\. Thus,S1−S0S\_\{1\}\-S\_\{0\}andS1S\_\{1\}do not merely rescale the same corrective preference; they change the relative ordering of corrective actions within a group\.

### E\.4Optimization Behavior of the Credit Variants

The fixed\-trajectory audit above compares the learning signals directly\. The corresponding training runs also differ in optimization behavior\. Table[16](https://arxiv.org/html/2610.14573#A5.T16)summarizes the four variants\.

Table 16:Optimization behavior observed for the four factorized credit variants\.VariantStable UntilFirst Abnormal StepObserved BehaviorA125–Completed normallyB∼81\\sim 8182Late length/termination pathologyC2425–26Early zero\-signal collapseD2526Early zero\-signal collapseVariant A completes all 125 training iterations without the length or termination failure observed in the alternative runs\. Variant B behaves normally through the checkpoints used for the A–B comparison but develops a late\-training length pathology\. Its mean Stage 1 completion length is 482 tokens at step 80, 965 tokens at step 81, and 3,663 tokens at step 82\. At step 83 it increases to 8,067 tokens, with98\.4%98\.4\\%of Stage 1 completions reaching the generation limit and no Stage 1 completion terminating normally at`\\end\{document\}`\.

Variants C and D show an earlier shared failure regime\. Both use absolute Stage 1 creditS1S\_\{1\}but different Stage 0 credits\. Their execution behavior collapses around step 26\. Table[17](https://arxiv.org/html/2610.14573#A5.T17)shows the transition around this point\.

Table 17:Training diagnostics around the C/D collapse\. “Hit Max” is the fraction of Stage 1 generations reaching the token limit, “End Doc” is the fraction terminating normally at\\end\{document\}, andA0/A1A\_\{0\}/A\_\{1\}nonzero counts the trajectories receiving nonzero group\-relative advantages\. Each training iteration contains 64 complete trajectories\.VariantStepStage 1TokensHit MaxEnd DocStage 0CompileStage 1CompileFFA0A\_\{0\}NonzeroA1A\_\{1\}NonzeroC25272331%45%1011524848C26563969%0%006400C27742591%0%006400D25131311%80%3839236457D26704186%0%006400D29–318192100%0%006400From step 26 onward, both C and D enter an all\-zero execution regime\. All 64 trajectories in the observed training iteration fail at both stages, so the fixed\-zero execution score gives identical zero returns within the group\. The resulting group\-relative advantages, policy loss, and gradient norm are therefore zero, while Stage 1 generations become substantially longer and normal document termination disappears\.

At the corresponding steps, A and B remain executable and retain nonzero training signals on the same sample groups\. C and D use different Stage 0 credits but share the absolute Stage 1 definition, making it the common credit choice associated with the observed early failure regime\.

### E\.5Summary

The ablations and fixed\-trajectory audit show that the two credit choices affect different parts of learning\. Using downstream Stage 0 credit changes which initial programs are favored according to the quality reached after the corrective action\. Using improvement\-relative Stage 1 credit produces substantially different corrective preferences from absolute final\-score credit, changing advantage signs and rankings for a large fraction of the same sampled trajectories\.

These differences also appear in training behavior\. Variant A combines downstream Stage 0 credit with improvement\-relative Stage 1 credit and completes the full training run, while the alternative variants exhibit distinct correction behavior or optimization failures\.

## Appendix FState\-Dependent Correction and Post\-Execution Behavior

This section expands the state\-dependent correction analysis in Section[4\.5](https://arxiv.org/html/2610.14573#S4.SS5)\. We first report the complete statistics behind Figure[4](https://arxiv.org/html/2610.14573#S4.F4), then examine the same behavior across nearby checkpoints\. We finally analyze revision and repair routing, execution transitions, and the optional selection rule applied when both outputs are available\.

### F\.1Full State\-Dependent Correction Analysis

Figure[4](https://arxiv.org/html/2610.14573#S4.F4)analyzes the Dev\-selected checkpoint at step 100\. As in the main text, we restrict this analysis to paired\-success examples for which both Stage 0 and Stage 1 produce valid renders\. We partition examples according to their Stage 0 RSim and define

Δ​M=M⁡\(I1,I⋆\)−M⁡\(I0,I⋆\)\.\\Delta M=M\(I\_\{1\},I^\{\\star\}\)\-M\(I\_\{0\},I^\{\\star\}\)\.We separately report

P⁡\(S1\>S0\),P\(S\_\{1\}\>S\_\{0\}\),which denotes the fraction of examples with a strictly higher Stage 1 RSim\. This quantity uses the thresholdΔ​RSim\>0\\Delta\\mathrm\{RSim\}\>0, whereas the correction\-outcome categories used in Section[4\.3](https://arxiv.org/html/2610.14573#S4.SS3)use a±0\.005\\pm 0\.005neutral region\.

Table 18:Complete state\-dependent correction statistics for our method at checkpoint 100 on Dev\-150\. The analysis is restricted to paired\-success examples\.Δ\\Deltavalues denote Stage 1 minus Stage 0\. The95%95\\%intervals are bootstrap intervals over examples within each bucket\. Improve, neutral, and degrade use the±0\.005\\pm 0\.005RSim threshold from the main learning\-dynamics analysis, whereasP⁡\(S1\>S0\)P\(S\_\{1\}\>S\_\{0\}\)uses the strictΔ​RSim\>0\\Delta\\mathrm\{RSim\}\>0criterion\.Stage 0 RSimnnΔ\\DeltaRSim95%95\\%CIP⁡\(S1\>S0\)P\(S\_\{1\}\>S\_\{0\}\)ImproveNeutralDegrade\[0\.0,0\.4\)\[0\.0,0\.4\)17\+0\.0260\+0\.0260\[\+0\.0103,\+0\.0442\]\[\+0\.0103,\+0\.0442\]76\.5%58\.8%29\.4%11\.8%\[0\.4,0\.5\)\[0\.4,0\.5\)29\+0\.0029\+0\.0029\[−0\.0103,\+0\.0168\]\[\-0\.0103,\+0\.0168\]55\.2%41\.4%20\.7%37\.9%\[0\.5,0\.6\)\[0\.5,0\.6\)39\+0\.0021\+0\.0021\[−0\.0087,\+0\.0128\]\[\-0\.0087,\+0\.0128\]53\.8%48\.7%7\.7%43\.6%\[0\.6,0\.7\)\[0\.6,0\.7\)26−0\.0026\-0\.0026\[−0\.0287,\+0\.0194\]\[\-0\.0287,\+0\.0194\]53\.8%53\.8%15\.4%30\.8%\[0\.7,1\.0\]\[0\.7,1\.0\]30−0\.0235\-0\.0235\[−0\.0593,\+0\.0026\]\[\-0\.0593,\+0\.0026\]23\.3%20\.0%33\.3%46\.7%The full statistics reinforce the pattern shown in the main figure\. The weakest intermediate states receive the largest average correction: for Stage 0 RSim below0\.40\.4, Stage 1 improves RSim by0\.02600\.0260on average and achieves a higher RSim in76\.5%76\.5\\%of paired examples\. The average gain decreases as Stage 0 quality increases\. For the highest\-quality bucket, the mean change becomes−0\.0235\-0\.0235, and only23\.3%23\.3\\%of examples obtain a strictly higher Stage 1 RSim\.

The same contrast is visible under metrics that are not used to train the policy\. Table[19](https://arxiv.org/html/2610.14573#A6.T19)reports the corresponding DINO, DreamSim, and CLIPImg changes\. All four metrics improve on average in the weakest bucket, whereas all four decrease in the highest\-quality bucket\.

Table 19:Cross\-metric Stage 1 changes at checkpoint 100\. RSim is the optimization reward; DINO, DreamSim, and CLIPImg are evaluation\-only metrics\.Stage 0 RSimΔ\\DeltaRSimΔ\\DeltaDINOΔ\\DeltaDreamSimΔ\\DeltaCLIPImg\[0\.0,0\.4\)\[0\.0,0\.4\)\+0\.0260\+0\.0260\+0\.0195\+0\.0195\+0\.0352\+0\.0352\+0\.0191\+0\.0191\[0\.4,0\.5\)\[0\.4,0\.5\)\+0\.0029\+0\.0029\+0\.0002\+0\.0002−0\.0001\-0\.0001\+0\.0015\+0\.0015\[0\.5,0\.6\)\[0\.5,0\.6\)\+0\.0021\+0\.0021−0\.0166\-0\.0166−0\.0093\-0\.0093−0\.0033\-0\.0033\[0\.6,0\.7\)\[0\.6,0\.7\)−0\.0026\-0\.0026−0\.0222\-0\.0222−0\.0189\-0\.0189\+0\.0030\+0\.0030\[0\.7,1\.0\]\[0\.7,1\.0\]−0\.0235\-0\.0235−0\.0044\-0\.0044−0\.0133\-0\.0133−0\.0109\-0\.0109The intermediate buckets are less uniform across metrics, while the endpoint contrast is consistent: weak intermediate states leave more room for correction, whereas additional modification of already strong renders is more likely to degrade the result\.

### F\.2State Dependence across Checkpoints

The main analysis uses step 100 because it is the checkpoint selected on Dev\-150 for final evaluation\. We repeat the same analysis at steps 75 and 125 using identical RSim buckets and definitions\.

Table[20](https://arxiv.org/html/2610.14573#A6.T20)summarizes the weakest and strongest buckets\. Across all three later checkpoints, weak Stage 0 states receive a larger average gain and are more likely to obtain a higher Stage 1 RSim than strong Stage 0 states\.

Table 20:State\-dependent correction across later checkpoints of our method\. “Weak” denotes Stage 0 RSim below0\.40\.4and “strong” denotes Stage 0 RSim in\[0\.7,1\.0\]\[0\.7,1\.0\]\. All statistics use paired\-success examples and the same fixed bucket boundaries\.CheckpointWeakΔ\\DeltaRSimWeakP⁡\(S1\>S0\)P\(S\_\{1\}\>S\_\{0\}\)StrongΔ\\DeltaRSimStrongP⁡\(S1\>S0\)P\(S\_\{1\}\>S\_\{0\}\)Step 75\+0\.0412\+0\.041258\.3%\+0\.0045\+0\.004528\.0%Step 100\+0\.0260\+0\.026076\.5%−0\.0235\-0\.023523\.3%Step 125\+0\.0337\+0\.033781\.3%−0\.0158\-0\.015828\.0%The exact magnitude changes over training, but the separation between weak and strong intermediate states is preserved\. At step 75, the strongest bucket still has a small positive averageΔ\\DeltaRSim, but Stage 1 achieves a higher RSim in only28\.0%28\.0\\%of those examples\. At steps 100 and 125, the strongest bucket has a negative average change\.

### F\.3Revision and Repair Routing during Training

The Stage 1 route is determined independently for every trajectory by its own Stage 0 execution result\. A valid Stage 0 render enters visual revision; an unsuccessful execution enters repair\.

Across the completed training run, the 1,000 target\-image groups contain 8,000 complete trajectories\. Of these, 7,447 trajectories \(93\.1%93\.1\\%\) enter visual revision and 553 \(6\.9%6\.9\\%\) enter repair\. The distribution across groups is shown in Table[21](https://arxiv.org/html/2610.14573#A6.T21)\.

Table 21:Routing composition of the 1,000 training groups\. Each group contains eight independently sampled complete trajectories\. A mixed group contains at least one revision trajectory and at least one repair trajectory\.Group compositionGroupsFractionRevision only73873\.8%Mixed revision / repair25925\.9%Repair only30\.3%All groups1,000100%The 259 mixed groups also show that a training group contains independently sampled Stage 0 actions rather than one shared Stage 0 state followed by multiple corrective candidates\. Each Stage 1 action operates on the execution state produced by its own Stage 0 program\.

### F\.4Execution Transitions and Repair

We next examine how the corrective action changes execution status\. We denote the four possible transitions bySS\\mathrm\{SS\},SF\\mathrm\{SF\},FS\\mathrm\{FS\}, andFF\\mathrm\{FF\}\. TheFS\\mathrm\{FS\}transition corresponds to execution repair, whereasSF\\mathrm\{SF\}denotes an initially valid program that becomes invalid after correction\.

Table 22:Execution transitions for the initializer and the Dev\-selected checkpoint of our method\. “Repair success” isP⁡\(S1​succeeds∣S0​fails\)P\(S\_\{1\}\\text\{ succeeds\}\\mid S\_\{0\}\\text\{ fails\}\)\.SplitPolicySSSFFSFFStage 0 failuresRepair successDev\-150Initializer1230126273\.7%Dev\-150Step 100141414520\.0%Final\-945Initializer780561541603\.8%Final\-945Step 1008631314556920\.3%Training substantially improves Stage 0 executability, reducing Final\-945 Stage 0 failures from 160 for the initializer to 69 at checkpoint 100\. Among the remaining failed states, Stage 1 repairs 14 of 69 programs, compared with 6 of 160 for the initializer\. The improvement is therefore already visible before correction is invoked, while the second action adds repair capability for the failed states that remain\.

The oppositeSF\\mathrm\{SF\}transition also remains possible\. On Final\-945, 13 initially successful Stage 0 programs fail after Stage 1\. This motivates retaining the initial output when the corrective action makes execution worse\.

### F\.5Execution\-Aware Selection

For correction\-enabled inference, both Stage 0 and Stage 1 are available\. Because the effect of Stage 1 depends on the intermediate state, we additionally evaluate the selection rule defined in Appendix[C\.4](https://arxiv.org/html/2610.14573#A3.SS4)\. The rule chooses only between the two programs already produced and requires no additional model generation\.

Table[23](https://arxiv.org/html/2610.14573#A6.T23)compares Stage 0, raw Stage 1, and the selected output at the fixed checkpoint 100\.

Table 23:Stage 0, raw Stage 1, and execution\-aware selection for our method at checkpoint 100\. Visual metrics use the fixed\-zero convention\.SplitOutputRenderRSimDINODreamSimCLIPImgDev\-150Stage 096\.67%0\.54790\.82250\.75540\.8668Stage 194\.67%0\.53580\.79940\.73480\.8485Selected97\.33%0\.56460\.83260\.76770\.8774Final\-945Stage 092\.70%0\.53820\.80140\.73300\.8372Stage 192\.80%0\.54460\.80460\.74010\.8409Selected94\.18%0\.56490\.82210\.75930\.8583On Dev\-150, Stage 0 is already strong at checkpoint 100, while raw Stage 1 is lower on the aggregate metrics\. The state\-dependent analysis nevertheless shows that the corrective action remains useful for weaker intermediate states\. Selection retains those gains while keeping Stage 0 when it is preferable, raising selected RSim\-FZ to0\.56460\.5646\.

On Final\-945, raw Stage 1 is slightly higher than Stage 0 under all four visual metrics, and selection provides a further improvement: render success rises from92\.80%92\.80\\%for raw Stage 1 to94\.18%94\.18\\%, while RSim\-FZ increases from0\.54460\.5446to0\.56490\.5649\. As reported in Appendix[D\.4](https://arxiv.org/html/2610.14573#A4.SS4), the paired bootstrap interval for each Stage 1\-to\-Selected improvement excludes zero on the fixed Final\-945 sample set\.

Stage 0 remains the single\-call output of the trained policy\. The corrective action and selection provide an optional additional inference mode when execution and a second model call are available\.

### F\.6Summary

The extended analysis supports three observations\. First, correction is state dependent: weak intermediate renders receive larger gains, while strong renders are more susceptible to degradation after further modification\. Second, the same separation appears across nearby checkpoints rather than only at the Dev\-selected checkpoint\. Third, training improves the initial action while also producing visual revision and execution\-repair behavior in the corrective action; this second action can be used optionally at inference time\.