I’ll be at the poster today in Hall A, 5–7:45pm KST. Come say hi if you want to talk about synthetic data, self-play, or RL!

Joint work w/ wonderful collaborators @zeynebnkaya @ShirleyYXWu @tengyuma @tatsu_hashimoto @lschmidt3

Paper: https://t.co/ei6vq44RE1

Anchored Self-Play for Code Repair

Source: https://arxiv.org/html/2607.03523 Zeyneb KayaShirley WuTengyu MaTatsunori HashimotoLudwig Schmidt

Abstract

Code repair is an important capability for language models (LMs): given a buggy program and unit tests, an LM must produce a fixed program that passes the tests. Because code repair data is limited, we aim to scale supervision by using an LM to generate bug–fix tasks. We proposegenerator–fixer self-play, in which a single model is trained with reinforcement learning to generate bugs and fix them. As the fixer improves, the generator adapts to produce more difficult bugs, yielding an automatic curriculum. To test whether this curriculum generalizes, we introduceBugSourceBench, a repair benchmark spanning realistic bug sources: bugs in human-written code, LM-generated code, and human-edited LM-generated code. OnBugSourceBench, we find that self-play drifts toward difficult but unrealistic bugs, improving on synthetic bugs but degrading on human-authored ones. We proposeAnchored Self-Play (ASP), which anchors self-play with a small reference set by adding a code-embedding similarity reward for generation and mixing reference bugs into fixer training. Across bug sources,ASPachieves the best fix rates, improving average fix rate over standard self-play by+24%+24\%relative /+7.0+7.0pp absolute, with gains on bugs from both LMs and humans.

code repair, program repair, self-play, large language models

Figure 1:Anchored self-play for code repair.*Left:*In generator–fixer self-play, the generator edits a correct program to produce a bug and the fixer repairs it; unit tests reward bug validity and repair correctness. Because unit tests verify pass/fail behavior but not realism, self-play can drift toward unrealistic test-failing bugs.*Right:*BugSourceBenchevaluates repair on the same programming tasks while varying the bug source, covering human-written bugs, human-edited LM bugs, and LM-generated bugs.Anchored Self-Play (ASP)mitigates this drift by anchoring training to a small reference set, using an embedding-similarity reward for bug generation and reference-bug mixing for fixer training.## 1Introduction

Code repair is an important capability for language models (LMs) used in programming workflows(Xuet al.,2022; Jimenezet al.,2023). Given a buggy program and accompanying unit tests, an LM must produce a fix that passes the tests. However, high-quality, real-world repair data is limited(Justet al.,2014; Widyasariet al.,2020; Le Goueset al.,2015; Madeiralet al.,2019; Olivaet al.,2025).

We ask whether high-quality supervision for code repair can be scaled using LMs to generate bug–fix tasks for training, with unit tests providing automatic verification. We study an open-ended generation setting in which an LM can apply arbitrary text edits to correct code. This allows for more diverse and adaptive synthetic data. Ideally, as the fixer improves over training, the generator produces bugs that are increasingly challenging and realistic, forming an automatic curriculum.

We instantiate this idea withgenerator–fixer self-play(Figure1). A single model is trained with reinforcement learning to alternate between generating a bug and fixing it. The generator is rewarded for producing valid, appropriately difficult bugs (tests fail), and the fixer is rewarded for producing correct repairs (tests pass).

A key challenge in generator–fixer self-play is distribution drift. Unit tests verify functional behavior, but they do not verify realism: many edits can break tests, but few resemble bugs encountered in practice. As training progresses, the generator can drift toward difficult but unrealistic bugs, improving repair on self-generated tasks while degrading on real-world ones. Prior work mitigates this by constraining bug generation to repository histories, deletions, or mutation operators(Weiet al.,2025; Forrestet al.,2009; Allamaniset al.,2021). However, this limits the diversity and scale of synthetic training data and does not address the open-ended generation setting we consider here.

To evaluate whether self-play improves code repair beyond its own synthetic distribution, we introduceBugSourceBench, a benchmark for cross-source repair generalization. In LM-assisted programming, repair models may encounter bugs from multiple sources: human-written code, LM-generated code, and human edits to LM-generated code.BugSourceBenchtherefore spans human-authored bugs, human edits of LM-generated bugs, and LM-generated bugs from weaker and stronger code models. Crucially,BugSourceBenchholds the repair task fixed while varying only the bug source.

OnBugSourceBench, we find that standard self-play exhibits distribution drift: it improves on LM-generated bugs but regresses on human-authored bugs. We proposeAnchored Self-Play (ASP)(Figure1), which anchors self-play to a small reference set sampled from the target bug sources.ASPuses this reference set in two complementary ways: a code-embedding similarity reward guides the generator toward target-like bugs, and reference-bug mixing exposes the fixer to realistic bugs throughout training. Anchoring stabilizes self-play and improves cross-source repair:ASPachieves the best overall fix rate, improving average fix rate by24%24\%relative /+7.0+7.0pp absolute over standard self-play, with gains on both LM-generated bugs (100%100\%relative /+11+11pp absolute) and human-originated bugs (7.1%7.1\%relative /+3.4+3.4pp absolute).

Our contributions are:

• •We formulate generator–fixer self-play for code repair and identify distribution drift as a key failure mode of unit-test-only self-play.
• •We introduce and releaseBugSourceBench, a controlled multi-source benchmark spanning human-written, human-edited LM, and LM-generated bugs.
• •We proposeAnchored Self-Play (ASP), which reduces drift by combining embedding-similarity guidance for bug generation with reference-mixed fixer training.

2Problem Formulation

We study code repair with unit-test feedback. Our goal is to train repair models using unit tests as the correctness signal, while evaluating whether the learned fixer generalizes across realistic sources of bugs.

Code repair.

Letxxdenote a programming task, consisting of natural-language instructions, input/output specifications, constraints, and a unit-test suite. Given a candidate programcc, running the tests produces a binary verifierv​(x,c)∈{0,1}v(x,c)\in\{0,1\}, wherev​(x,c)=1v(x,c)=1if and only ifccpasses all tests for taskxx, together with test outputo​(x,c)o(x,c), such as compilation errors, failed assertions, or stack traces.

We define a valid bug for taskxxas an executable programbbthat fails at least one test:

A repair model, orfixer,πF\pi_{F}maps the task, buggy program, and test output to a distribution over candidate repairs:

y∼πF(⋅∣x,b,o(x,b)).y\sim\pi_{F}(\cdot\mid x,b,o(x,b)).A repair succeeds if the repaired program passes the unit tests, i.e.,v​(x,y)=1v(x,y)=1. In our main experiments, the fixer outputs a full corrected program rather than a diff.

Evaluation across bug sources.

In deployment, bugs may come from heterogeneous sources: human programmers, LM coding assistants, or human edits of LM-generated code. We therefore evaluate repair under a family of bug-source distributions. For each sources∈𝒮s\in\mathcal{S}, letPs(⋅∣x)P_{s}(\cdot\mid x)denote the distribution over valid bugs for taskxx. The repair performance of fixerπF\pi_{F}on sourcessis

Perf​(πF;s)=𝔼x∼𝒟​𝔼b∼Ps(⋅∣x)​𝔼y∼πF(⋅∣x,b,o(x,b))​[v​(x,y)],\mathrm{Perf}(\pi_{F};s)=\mathbb{E}_{x\sim\mathcal{D}}\;\mathbb{E}_{b\sim P_{s}(\cdot\mid x)}\;\mathbb{E}_{y\sim\pi_{F}(\cdot\mid x,b,o(x,b))}\left[v(x,y)\right],where𝒟\mathcal{D}is the evaluation distribution over tasks. In our benchmark,𝒟\mathcal{D}is the uniform distribution over a fixed held-out task set, and different bug sources share the same underlying tasks.

We summarize cross-source performance by averaging over bug sources:

Perfavg​(πF)=1|𝒮|​∑s∈𝒮Perf​(πF;s).\mathrm{Perf}_{\mathrm{avg}}(\pi_{F})=\frac{1}{|\mathcal{S}|}\sum_{s\in\mathcal{S}}\mathrm{Perf}(\pi_{F};s).This criterion evaluates whether a fixer improves broadly across realistic bug sources, rather than only on the distribution induced by a particular bug generator. We next instantiate𝒮\mathcal{S}withBugSourceBench, a controlled benchmark spanning multiple bug sources in LM-assisted programming.

3BugSourceBench: Controlled Bug-Source Evaluation

We introduceBugSourceBenchto evaluate whether code-repair methods generalize across realistic sources of bugs. The benchmark is designed to isolate bug-source shift: all sources share the same programming tasks, specifications, and unit tests, and differ only in the buggy implementation provided to the repair model.

3.1Benchmark Construction

We buildBugSourceBenchfrom BigCodeBench(Zhuoet al.,2024), a code-generation benchmark emphasizing realistic library and API usage. Each BigCodeBench task contains (i) natural-language programming instructions, (ii) unit tests that define correctness, and (iii) a reference implementation that passes those tests. We convert each task into a repair instance by preserving the original prompt and unit tests and replacing the reference implementation with a buggy program.

Task structure.

ABugSourceBenchexample consists of programming instructionsxx, a buggy programbb, and the accompanying unit-test verifierv​(x,⋅)v(x,\cdot). All buggy programs inBugSourceBenchexecute under the test harness and fail at least one unit test. At evaluation, the model receives(x,b)(x,b)and unit-test feedbacko​(x,b)\mathrm{o}(x,b)(e.g., failing tests and truncated error traces) and must output a corrected programyythat passes all tests.

Bug sources.

BugSourceBenchcontains four bug-source variants that reflect common sources of errors in LM-assisted programming. All variants are constructed on the same underlying tasks; only the buggy implementation differs.

• •Human.Annotators introduce bugs into each task’s human-written reference solution. They are instructed to make 1–4 localized edits that preserve executability while causing at least one unit test to fail. We encourage realistic developer mistakes, such as off-by-one errors, wrong constants, missing edge cases, or API misuse, rather than syntax-breaking edits.
• •Human-Edited LM.To model human-in-the-loop errors, we first promptgpt-5-minito solve the task and retain an incorrect program that executes but fails at least one unit test. Annotators then edit this draft while keeping it executable and incorrect. This source captures mistakes that can arise when developers modify, integrate, or partially correct LM-generated code.
• •LM Errors (Qwen-7B).We promptQwen2.5-Coder-7B-Instructto solve each task, not to generate bugs, and retain incorrect programs that execute but fail at least one unit test. This source captures errors produced by a weaker code LM under standard code-generation prompting.
• •LM Errors (gpt-oss-20b).We apply the same procedure withgpt-oss-20b, yielding errors from a stronger code model.

Together, these variants cover three realistic bug-source families: human-written bugs, human-edited LM bugs, and unedited LM-generated bugs. Because the task and tests are fixed across sources,BugSourceBenchdirectly measures how repair performance changes with the origin of the bug.

Repair interface.

We evaluate repair models using a full-program repair interface with unit-test feedback. That is, the model receives the task description, buggy program, and truncated test output, and returns a complete corrected program. InAppendicesAand3, we compare this interface with full-program repair without test traces and diff-based patching. Test feedback improves repair performance, while diff-based repair often underperforms due to brittle formatting and patch application. We therefore use the test-trace interface in all main experiments.

Benchmark analyses.

We provide per-source examples inSectionA.1and full construction details, including filtering criteria and sampling budgets, inSectionA.2. InSectionA.4, we evaluate frontier models onBugSourceBenchand show that repair is distinct from code generation: models often solve tasks from scratch that they fail to repair, and vice versa. We also analyze bug-source structure by categorizing bugs into coarse error types and by measuringkkNN source purity usingvoyage-code-3embeddings. These analyses reveal systematic source-dependent failure patterns and strong within-source clustering.

BugSourceBenchlets us test whether synthetic bug–fix training improves repair across realistic bug sources, rather than only on the distribution induced by a particular generator. We next describe generator–fixer self-play and then introduceAnchored Self-Play (ASP), which anchors self-play to a small reference set to reduce bug-source drift.

4Self-Play for Code Repair

We now describe the training procedures used to scale code-repair supervision. We first introduce generator–fixer self-play, which uses unit-test outcomes as rewards. We then show that this unit-test-only objective can drift toward unrealistic bug distributions, and introduceAnchored Self-Play (ASP)to anchor self-play to reference bug sources.

4.1Generator–Fixer Self-Play

For each training task, we assume access to the programming specificationxx, unit tests, and a correct reference implementationc⋆c^{\star}. We train a single policyπθ\pi_{\theta}to play two roles: ageneratorGGthat edits correct programs into buggy ones, and afixerFFthat repairs the resulting bugs (Figure1). We denote the corresponding role-conditioned distributions byπG\pi_{G}andπF\pi_{F}.

The generator samples a candidate buggy program

b∼πG(⋅∣x,c⋆),b\sim\pi_{G}(\cdot\mid x,c^{\star}),and we run the unit tests to obtain test outputo​(x,b)o(x,b). We callbbvalidif it executes under the test harness and fails at least one unit test. Invalid bugs receive a penalty and are not used for fixer training. For valid bugs, the fixer samples one or more candidate repairs conditioned on the task, buggy program, and test output:

y∼πF(⋅∣x,b,o(x,b)).y\sim\pi_{F}(\cdot\mid x,b,o(x,b)).

4.2Correctness and Difficulty Rewards

Fixer reward.

The fixer is rewarded for producing repairs that pass the unit tests. For a candidate repairyy, we use

rF​(x,b,y)=v​(x,y),r^{\textsc{F}}(x,b,y)=v(x,y),wherev​(x,y)=1v(x,y)=1if and only ifyypasses all tests for taskxx.

Generator reward.

The generator should produce bugs that are valid, challenging, and still solvable by the current fixer. A single repair attempt gives a noisy estimate of difficulty, so we estimate the fix rate of a generated bug usingKKindependent repair attempts. For a valid bug(x,b)(x,b), we sample

y(1),…,y(K)∼πF(⋅∣x,b,o(x,b))y^{(1)},\ldots,y^{(K)}\sim\pi_{F}(\cdot\mid x,b,o(x,b))and define

ρ​(x,b)=1K​∑k=1Kv​(x,y(k)).\rho(x,b)=\frac{1}{K}\sum_{k=1}^{K}v(x,y^{(k)}).Bugs withρ​(x,b)=1\rho(x,b)=1are too easy, while bugs withρ​(x,b)=0\rho(x,b)=0provide little useful training signal because the current fixer cannot repair them. We therefore reward valid bugs whose fix rate falls in an intermediate difficulty band:

rbaseG​(x,b)={−1,b​is invalid,1,ρ​(x,b)∈[ρℓ,ρh],−α,ρ​(x,b)∈{0,1},0,otherwise.r^{\textsc{G}}_{\mathrm{base}}(x,b)=\begin{cases}-1,&b\text{ is invalid},\\ 1,&\rho(x,b)\in[\rho_{\ell},\rho_{h}],\\ -\alpha,&\rho(x,b)\in\{0,1\},\\ 0,&\text{otherwise.}\end{cases}

4.3Optimization

We optimize the generator–fixer loop with GRPO. For each taskxx, we sampleG=4G=4candidate bugsbi∼πG(⋅∣x,c⋆)b_{i}\sim\pi_{G}(\cdot\mid x,c^{\star}). For each valid bug(x,bi)(x,b_{i}), we sampleK=4K=4independent repair attempts

yi(1:K)∼πF(⋅∣x,bi,o(x,bi))y_{i}^{(1:K)}\sim\pi_{F}(\cdot\mid x,b_{i},o(x,b_{i}))and compute the empirical fix rate

ρ​(x,bi)=1K​∑k=1Kv​(x,yi(k)).\rho(x,b_{i})=\frac{1}{K}\sum_{k=1}^{K}v(x,y_{i}^{(k)}). LetRG​(x,b)R^{\textsc{G}}(x,b)denote the generator reward andRF​(x,b,y)R^{\textsc{F}}(x,b,y)denote the fixer reward. In standard self-play,RFR^{\textsc{F}}is the unit-test pass indicator andRGR^{\textsc{G}}is the difficulty-shaped reward fromSection4.2.ASPmodifies this objective by adding a reference-similarity term toRGR^{\textsc{G}}and by mixing reference bugs into fixer training (Section4.5).

Generator update.

For each taskxx, we compute group-normalized advantages over theGGsampled bugs:

A^iG\displaystyle\hat{A}^{\textsc{G}}_{i}=RG​(x,bi)−μG​(x)σG​(x)+ϵ,\displaystyle=\frac{R^{\textsc{G}}(x,b_{i})-\mu^{\textsc{G}}(x)}{\sigma^{\textsc{G}}(x)+\epsilon},μG​(x),σG​(x)\displaystyle\mu^{\textsc{G}}(x),\sigma^{\textsc{G}}(x)computed over​i∈{1,…,G}.\displaystyle\text{ computed over }i\in\{1,\dots,G\}.We then updateπG(⋅∣x,c⋆)\pi_{G}(\cdot\mid x,c^{\star})using the clipped GRPO objective.

Fixer update.

For each valid generated bugbib_{i}, we compute group-normalized advantages over theKKrepair attempts:

A^i,kF\displaystyle\hat{A}^{\textsc{F}}_{i,k}=RF​(x,bi,yi(k))−μF​(x,bi)σF​(x,bi)+ϵ,\displaystyle=\frac{R^{\textsc{F}}(x,b_{i},y_{i}^{(k)})-\mu^{\textsc{F}}(x,b_{i})}{\sigma^{\textsc{F}}(x,b_{i})+\epsilon},μF​(x,bi),σF​(x,bi)\displaystyle\mu^{\textsc{F}}(x,b_{i}),\sigma^{\textsc{F}}(x,b_{i})computed over​k∈{1,…,K}.\displaystyle\text{ computed over }k\in\{1,\dots,K\}.We updateπF(⋅∣x,bi,o(x,bi))\pi_{F}(\cdot\mid x,b_{i},o(x,b_{i}))with the same clipped GRPO objective, computing the loss only on tokens generated by the policy and masking prompt tokens.

4.4Distribution Drift Under Standard Self-Play

The rewards above encourage bugs that are valid and appropriately difficult, but they do not encourage bugs to be realistic. Unit tests can identify whether a candidate bug changes program behavior, but they do not indicate whether the edit resembles a bug written by a human programmer or produced by an LM coding assistant. As a result, the generator can drift toward idiosyncratic test-failing edits that remain useful under the self-play reward but differ from realistic bug sources.

Figure2(a)shows that standard self-play produces the intended co-evolutionary dynamics: as the generator trains, its bugs become harder for a fixed fixer, and as the fixer trains, it improves on bugs from a fixed generator. However,Figure2(b)shows that these gains do not reliably transfer to realistic bug sources. Performance improves early but later degrades, especially on human-originated bugs. This pattern is consistent with distribution drift: the fixer improves on the generator’s evolving synthetic distribution while losing robustness to target bug sources. We therefore modify the self-play objective to anchor generation and fixer training to reference bug distributions.

(a)Co-evolution of standard self-play.With a fixed fixer checkpoint, fix rate declines over generator training, indicating that the generator produces harder bugs. With a fixed generator checkpoint, fix rate increases as the fixer trains, indicating improved repair on generated bugs. (b)Standard self-play exhibits distribution drift.Fix rate improves early but later regresses, most strongly on human-originated bugs. This suggests overfitting to the generator’s shifting synthetic bug distribution.

Figure 2:Self-play training dynamics and distribution drift.

4.5Anchoring Self-Play to Reference Bug Sources

Anchored Self-Play (ASP)assumes access to a small reference set of valid bugs,𝒟ref\mathcal{D}_{\mathrm{ref}}, drawn from the training tasks and disjoint from evaluation. The reference set provides a weak realism signal for self-play. We use it in two complementary ways: (i)reference mixing, which exposes the fixer to reference-source bugs during training, and (ii)similarity-guided shaping, which nudges the generator toward reference-like edits.

4.5.1Reference Mixing for Fixer Training

For a training task with an associated reference bugbref∈𝒟refb^{\mathrm{ref}}\in\mathcal{D}_{\mathrm{ref}}, we replace the generated bug with probabilitypmixp_{\mathrm{mix}}and train the fixer on(x,bref,o​(x,bref)).(x,b^{\mathrm{ref}},o(x,b^{\mathrm{ref}})).On these mixed episodes, we update only the fixer. We do not update the generator because the reference bug was not sampled fromπG\pi_{G}; empirically, updating the generator on mixed episodes reduced performance (Table10).

4.5.2Similarity-Guided Generator Shaping

Reference mixing improves fixer training but does not directly change the generator’s distribution. To guide generation toward realistic bugs, we add an auxiliary reward based on similarity to the reference set.

Edit embeddings.

For each generated bugbb, we compute the unified diff between the reference solutionc⋆c^{\star}andbb. We embed this diff with a code embedding model,voyage-code-3, and compute its average cosine similarity to thekknearest reference edit embeddings. We map the resulting score to[0,1][0,1]and denote it bysim01​(b)\mathrm{sim}_{01}(b).

Centered similarity reward.

The similarity reward should act as a shaping term: it should bias the generator toward reference-like edits without overwhelming the unit-test reward. Because the absolute scale of embedding similarities can vary across batches and embedding models, we subtract a running baseline. At training steptt, we maintain

Bt←β​Bt−1+(1−β)​𝔼b∼batch​[sim01​(b)],B_{t}\leftarrow\beta B_{t-1}+(1-\beta)\,\mathbb{E}_{b\sim\mathrm{batch}}\!\left[\mathrm{sim}_{01}(b)\right],and define

δt​(b)=sim01​(b)−Bt.\delta_{t}(b)=\mathrm{sim}_{01}(b)-B_{t}.Thus,δt​(b)\delta_{t}(b)measures whether a generated bug is more or less reference-like than the generator’s current average output. For valid generated bugs, the final generator reward is

rG​(x,b)=rbaseG​(x,b)+λ​clip⁡(δt​(b)),r^{\textsc{G}}(x,b)=r^{\textsc{G}}_{\mathrm{base}}(x,b)+\lambda\,\operatorname{clip}\!\left(\delta_{t}(b)\right),whereclip⁡(⋅)\operatorname{clip}(\cdot)clips the shaping term to a fixed range. This centered reward is more stable than using an uncentered similarity score, since it is less sensitive to shifts in the absolute magnitude of embedding similarities. Hyperparameters are given inSectionB.1.

5Experimental Setup

Figure 3:Fix rates across bug sources onBugSourceBench.Standard self-play yields inconsistent gains across bug sources, consistent with distribution drift.ASPimproves repair across every source and achieves the best overall fix rate, outperforming standard self-play by+7.0+7.0pp /24%24\%relative. Gains hold for both LM-generated bugs (+11+11pp /100%100\%relative) and human-originated bugs (+3.4+3.4pp /7.1%7.1\%relative).Figure 4:Anchoring stabilizes self-play and improves cross-source repair.We plot fix rate versus training step for Fixer-Only, standard Self-Play, andASPon held-out bugs from eachBugSourceBenchsource and their average. Standard self-play improves early but later degrades on realistic bug sources, especially Human and Human-Edited LM bugs. In contrast,ASPyields higher and more stable performance across sources.##### Data and splits.

We train on900900BigCodeBench tasks(Zhuoet al.,2024). For evaluation, we use127127held-out tasks that appear in everyBugSourceBenchsplit. Thus, all bug sources share the same underlying tasks, specifications, and unit tests; only the buggy implementation differs. For validation and checkpoint selection, we use8181additional tasks from allBugSourceBenchsplits, disjoint from both train and test. We report results on four bug sources:BugSourceBench-Human,BugSourceBench-Human-Edited LM,BugSourceBench-Qwen-7B, andBugSourceBench-gpt-oss-20b.

Reference pool.

ASPuses a reference pool of900900bugs sampled from the training splits ofBugSourceBench, with equal representation from the four bug sources. This pool serves two purposes: it provides reference edit embeddings for the generator’s similarity reward, and it supplies reference bugs for fixer training through reference mixing. For parity, the Fixer-Only baseline also mixes the same reference bugs into fixer training, but does not use the similarity-based generator reward.

Initialization.

Unless otherwise stated, both generator and fixer are initialized fromQwen2.5-Coder-7B-Instruct(Huiet al.,2024). The two roles share a single set of weights and are trained with GRPO.

Comparisons.

We compareASPagainst three main baselines:

• •Base Model:the pretrained model used directly as a fixer, without post-training.
• •Fixer-Only:a frozen base-model generator samples bugs, while only the fixer is trained with GRPO. Reference bugs are mixed into fixer batches for parity withASP.
• •Self-Play:the generator and fixer are trained jointly with GRPO using the unit-test-only self-play objective.

Training and evaluation protocol.

For each training task, the generator samples candidate buggy programs. We retain programs that execute under the test harness and fail at least one unit test. The fixer receives the task description, buggy code, and truncated unit-test output, and is rewarded for producing a program that passes the tests. During training, the fixer samples up toK=4K=4repairs per bug. At test time, we use a single repair attempt per bug and decode greedily with temperature0.00.0. Full hyperparameters and prompts are provided inAppendixB; additional baselines are reported inAppendixE.

6Main Results

Figure3comparesASPwith standard self-play and fixer-only training onBugSourceBench.ASPachieves the best fix rate on every bug source, improving average fix rate by24%24\%relative /+7.0+7.0pp absolute over standard self-play and by29%29\%relative /+8.2+8.2pp absolute over Fixer-Only. The largest gains are on LM-generated bugs:+145%+145\%relative /+12.6+12.6pp absolute onQwen-7Bbugs, and+69%+69\%relative /+8.7+8.7pp absolute ongpt-oss-20bbugs. Anchoring also improves both Human and Human-Edited LM splits relative to standard self-play, with gains of+17.5%+17.5\%relative /+5.5+5.5pp absolute on Human-Edited LM and+2.0%+2.0\%relative /+1.3+1.3pp absolute on Human.

Standard self-play improves the average fix rate over Fixer-Only by+1.2+1.2pp, but its gains are not consistent across sources: it improves some splits while degrading onQwen-7Bbugs by−1.5-1.5pp. In contrast,ASPimproves all sources simultaneously. The Base Model and Fixer-Only have similar average performance, but Fixer-Only shifts performance toward LM-generated bugs: it improvesQwen-7Bfrom7.1%7.1\%to10.2%10.2\%andgpt-oss-20bfrom7.0%7.0\%to12.6%12.6\%, while leaving Human performance essentially unchanged (30.8%30.8\%vs.30.7%30.7\%). These results show why evaluating only one bug source can be misleading: training can improve one source while failing to improve, or even hurting, another.

We provide pass@kkresults inSectionD.1. We also show thatASPcan improve larger 30B+-parameter code models when used as a test-time fixer inSectionD.5.

ASPproduces more reference-like bugs.

We use embedding-based diagnostics to verify that the similarity reward changes the generator’s bug distribution. As shown inFigure9, the meankk-NN similarity between generated bugs and the reference pool increases underASP, indicating that similarity shaping moves generations toward the target bug sources. We also stratify benchmark performance by generation-similarity quantiles and find thatASPoutperforms standard self-play within each bin. Thus, the gains are not explained solely by generating higher-similarity bugs; anchoring also improves the fixer’s robustness within comparable similarity regimes. Additional analyses show thatASPimproves consistently across semantic bug categories (SectionD.4), and qualitative examples are provided inFigure8andAppendixD.

7Ablations

7.1Anchoring Components

Table1(a)ablates the two components ofASP: reference mixing for fixer training and the embedding-similarity reward for bug generation. Starting from standard self-play, each component alone improves average fix rate modestly. Combining them yields a substantially larger gain, suggesting that the two components address complementary failure modes. The similarity reward steers the generator toward reference-like bugs, while reference mixing keeps the fixer exposed to realistic bugs as the generator’s distribution evolves.

(a)Ablation of anchoring components.Reference mixing exposes the fixer to reference bugs during training, while similarity shaping adds an embedding-based reward that guides the generator toward reference-like edits. The fullASPmethod combines both and achieves the highest average fix rate acrossBugSourceBenchsplits. (b)Effect of reference-pool composition.We vary which bug sources are included in the reference pool. Human-only references yield the strongest performance on Human bugs, while LM-only references shift improvements toward LM-originated bugs. The mixed reference pool used byASPgives the best overall performance and the strongest gains on LM-generated sources, illustrating that the reference pool steers which bug patterns self-play emphasizes.

7.2Reference-Pool Composition

Table1(b)varies the sources used to construct the reference pool. The choice of reference bugs affects which bug patterns self-play emphasizes. Human-only references give the highest fix rate on Human bugs, while LM-only references improve LM-originated bugs relative to standard self-play and give the highest score on Human-Edited LM. The mixed pool used byASPachieves the best overall fix rate and the strongest performance on both LM-generated sources. These results suggest that the reference pool can steer anchoring toward particular deployment sources, while a diverse pool provides the best overall performance.

7.3Reference Set Size

Figure5varies the number of reference bugs used for anchoring, sampled uniformly across Human, Human-Edited LM,Qwen-7B, andgpt-oss-20bsources.ASPimproves with as few as5050reference examples, indicating that anchoring is sample-efficient. Performance continues to improve as the reference set grows, suggesting that larger pools provide broader coverage of realistic bug patterns.

Additional robustness ablations inAppendixDandAppendixEshow thatASP’s gains are robust across embedding models andkk-NN pooling parameters (Table9), base models (Table12), shared versus decoupled generator/fixer weights (Table11), and task distributions (Table13).

Figure 5:Reference set scaling.We vary the number of reference bugs used for anchoring and report fix rates on eachBugSourceBenchsplit and their average.ASPimproves with small reference sets and continues to benefit from larger pools, with the largest gains on LM-generated bug sources. Human and Human-Edited LM performance remain strong across reference-set sizes.

8Related Work

Code repair and bug-source variation.

Automated program repair is commonly evaluated on curated real-world bug datasets such as Defects4J, BugsInPy, ManyBugs, and Bears, as well as short unit-testable benchmarks such as QuixBugs(Justet al.,2014; Widyasariet al.,2020; Le Goueset al.,2015; Madeiralet al.,2019; Linet al.,2017). Related code-generation benchmarks, including HumanEval, MBPP, APPS, EvalPlus, and LiveCodeBench, show that fluent model outputs often fail functional tests(Chenet al.,2021; Austinet al.,2021; Hendryckset al.,2021; Liuet al.,2023; Jainet al.,2024), while repository-level benchmarks such as SWE-bench evaluate end-to-end issue resolution with realistic context and tooling(Jimenezet al.,2023; Yanget al.,2025a; Phamet al.,2025). Across these settings, performance is sensitive to the bug distribution: human-written bugs, synthetic mutations, and errors in model-generated code induce different failure modes(Heet al.,2022; Xuet al.,2022; Sonwaneet al.,2025; Douet al.,2025; Yanget al.,2025b).BugSourceBenchcomplements prior benchmarks by holding the task, specification, and tests fixed while varying only the bug source, enabling controlled evaluation of cross-source generalization.

Synthetic bug generation.

Prior work scales repair supervision by generating synthetic bugs, often under constraints that preserve realism. BugLab uses predefined mutation operators for self-supervised bug localization and repair(Allamaniset al.,2021; Forrestet al.,2009); repository-level systems use grounding from repository structure, tests, issue context, or patch provenance(Yanget al.,2025a; Phamet al.,2025; Weiet al.,2025; Yeet al.,2023; Zirak and Hemmati,2024); and Break-it-Fix-it trains paired bug introducers and fixers using a critic such as a parser or compiler(Yasunaga and Liang,2021; Long and Rinard,2016; Chenet al.,2019). In contrast, we study open-ended bug generation, where the generator can apply arbitrary text edits and unit tests verify failure but only weakly constrain realism. Our method adds a soft realism signal through embedding similarity to reference bugs while preserving open-ended generation.

Self-play and curricula.

Self-play has been used to generate curricula by proposing tasks near a learner’s frontier and learning from rollouts(Bengioet al.,2009; Silveret al.,2017; Chenget al.,2024; Kubaet al.,2025; Chenet al.,2024; Zhaoet al.,2025; Huanget al.,2025). In language and code domains, related work uses self-generated data for reasoning, theorem proving, adaptive testing, open-ended task design, and proposer–solver frameworks for synthesis, testing, or verification(Poesiaet al.,2024; Dong and Ma,2025; Chenet al.,2025; Liuet al.,2025; Yuet al.,2025; Ribeiro and Lundberg,2022; Colaset al.,2022; Parker-Holderet al.,2023; Teodorescuet al.,2023; Pourcelet al.,2024; Linet al.,2025; Wanget al.,2025; Wilfet al.,2025). We apply this curriculum perspective to code repair and identify a failure mode of unit-test-only self-play: without an explicit realism signal, the generator can produce difficult test-failing edits that do not resemble realistic bugs.

9Discussion

We study whether unit tests can scale code-repair supervision through open-ended synthetic data generation. Generator–fixer self-play creates an adaptive curriculum by training a model to generate bugs that fail tests and repair bugs to pass them. However, we find that unit-test-only self-play can drift toward valid but unrealistic bugs, improving on its own generated distribution while weakening generalization to realistic bug sources.

Anchored Self-Play (ASP)mitigates this drift by anchoring self-play to a small reference set, using an embedding-similarity reward for bug generation and reference mixing for fixer training. UsingBugSourceBench, a controlled benchmark spanning human-written, human-edited LM, and LM-generated bugs, we show thatASPimproves average repair over standard self-play and reduces bug-source drift. These results highlight the need for explicit realism signals in unit-test-only self-play. Future work could explore stronger realism objectives, such as learned bug-style critics, preference-based rewards, or richer anchoring signals beyond code embeddings.

Limitations.

Our experiments focus on Python function-level repair with unit-test feedback. This controlled setting isolates bug-source shift, but does not capture repository-level challenges such as multi-file fault localization, dependency management, build-system interaction, or long-horizon tool use. Our realism signal also relies on a frozen code-embedding model and a finite reference pool; learned critics or human preference data may provide stronger signals in future work.

Impact Statement

This work aims to improve the reliability of language-model-based code repair. By studying generalization across bug sources, our results can help build automated programming tools that are more robust to realistic errors. As with other automated software-engineering systems, generated fixes should be validated before deployment. We do not anticipate additional societal risks beyond those associated with broader use of machine learning in software engineering.

Software and Data

Acknowledgements

The authors thank Yangjun Ruan, Neil Band, Kaiyue Wen, Luke Bailey, Thomas Chen, and Arvind Mahankali for valuable discussions and feedback on this work. We also thank the anonymous reviewers for their constructive feedback. This work made use of computational resources provided by the Stanford Marlowe team(Kapferet al.,2025). CC was supported by the National Science Foundation Graduate Research Fellowship Program under Grant No. DGE-2146755.

Appendix ABugSourceBench

A.1Examples

BugSourceBench TaskID 1063Performs Principal Component Analysis (PCA) on the sum of rows of a 2D numpy array and plots the explained variance ratio.Note that:the title of the plot is set to ”Explained Variance Ratio of Principal Components”. The function assumes that ’arr’ is a valid 2D numpy array. Only the first principal component is considered in this analysis. The plot illustrates the proportion of the dataset’s variance that lies along the axis of this first principal component.The function should output with:ax (matplotlib.axes.Axes) – an Axes object from matplotlib.

Human⬇importnumpyasnpfrommatplotlibimportpyplotaspltfromsklearn.decompositionimportPCAdeftask_func(arr):row_sums=arr.sum(axis=1)pca=PCA(n_components=10)pca.fit(row_sums.reshape(-1,1))_,ax=plt.subplots()ax.bar([0],pca.explained_variance_ratio)ax.set_title(“ExplainedVarianceRatioofPrincipalComponents”)ax.set_xticks([0])ax.set_xticklabels([“PC1”])returnax

Human-Edited LM⬇importnumpyasnpfrommatplotlibimportpyplotaspltfromsklearn.decompositionimportPCAdeftask_func(arr):ifarr.size==0:raiseValueError(“Inputarrayisempty.”)summed=np.sum(arr,axis=1)X=summed.reshape(-1,1)pca=PCA(n_components=1)pca.fit(X)fig,ax=plt.subplots()ax.bar([1],pca.explained_variance_,color=“tab:blue”)ax.set_xticks([1])ax.set_xticklabels([“PC1”])ax.set_ylabel(“ExplainedVarianceRatio”)ax.set_title(“ExplainedVarianceRatioofPrincipalComponents”)returnax

LM Errors (Qwen-7B)⬇importnumpyasnpfrommatplotlibimportpyplotaspltfromsklearn.decompositionimportPCAdeftask_func(arr):iflen(arr.shape)!=2:raiseValueError(“Inputmustbea2Dnumpyarray”)summed_data=np.sum(arr,axis=0)reshaped_data=summed_data.reshape(1,-1)pca=PCA(n_components=1)pca.fit(reshaped_data)explained_variance_ratio=pca.explained_variance_ratio_fig,ax=plt.subplots()ax.bar(range(1),explained_variance_ratio,tick_label=[’PC1’])ax.set_title(’ExplainedVarianceRatioofPrincipalComponents’)returnax

LM Errors (gpt-oss-20b)⬇importnumpyasnpfrommatplotlibimportpyplotaspltfromsklearn.decompositionimportPCAdeftask_func(arr):row_sums=arr.sum(axis=0)X=row_sums.reshape(1,-1)pca=PCA(n_components=1)pca.fit(X)evr=pca.explained_variance_ratio_[0]fig,ax=plt.subplots()ax.bar([1],[evr],width=0.5,color=’skyblue’)ax.set_title(“ExplainedVarianceRatioofPrincipalComponents”)ax.set_xlabel(“PrincipalComponent”)ax.set_ylabel(“ExplainedVarianceRatio”)ax.set_xticks([1])ax.set_ylim(0,1)returnax

Figure 6:BugSourceBenchexample showing buggy programs from different sources for the same taskxx. The bug patterns differ qualitatively across sources. The human-written bug changes PCA parameters and plotting behavior; the human-edited LM bug uses explained variance rather than explained-variance ratio and changes the plotted index; Qwen-7B and gpt-oss-20b both make model-like errors involving the aggregation axis and PCA input shape. These examples illustrate why evaluating a single bug source can miss source-specific repair failures.

A.2Construction

Overview.

BugSourceBenchis built from1,1141{,}114BigCodeBench-style programming tasks. Each task provides natural-language instructions, unit tests that define correctness, and a reference implementation that passes those tests. ABugSourceBenchinstance pairs a task with a buggy program that executes under the test harness and fails at least one unit test. AllBugSourceBenchvariants share the same underlying tasks and differ only in how the buggy program is produced, enabling controlled comparisons across bug sources.

Bug validity criteria.

We first remove2626BigCodeBench tasks whose reference solutions do not pass the provided unit tests, leaving1,1141{,}114tasks for bug construction. We accept a candidate program as a valid bug if it executes under the test harness and fails at least one unit test. We reject candidates that fail due to interpreter or runtime errors identified by pattern matching over test output, such asSyntaxError,ImportError, orNameError. Assertion failures from the unit tests are treated as valid test failures.

Task structure.

All datasets are converted to a commonBugSourceBenchschema. We store function bodies only, with 4-space indentation, for bothbuggyandcanonical_solution. Each example contains:

• •task_id: identifier for the underlying task, shared across bug sources.
• •instruct_prompt: natural-language problem statement, including input/output specifications and constraints.
• •buggy: buggy function body that executes but fails at least one unit test.
• •canonical_solution: reference correct function body, used for dataset construction and evaluation but never provided to the fixer.
• •test: unit-test harness used to evaluate candidate repairs.
• •complete_prompt: full prompt used in the default repair interface, including instructions, buggy code, and test feedback.
• •code_prompt: code-only prompt segment containing the function context.
• •entry_point: function name called by the test harness.
• •doc_struct: structured metadata extracted from the problem statement, when available.
• •libs: libraries or modules required by the task.
• •test_output: truncated feedback from runningbuggyontest, used for analysis and as repair context.

This schema lets us swap bug sources while holding the task, specification, and tests fixed.

Bug sources.

We construct four bug-source variants, each providing a different buggy program for the same underlying BigCodeBench tasks. For fair comparison, we retain only task IDs that appear in all variants.

• •BugSourceBench-Human.Starting from each task’s reference solution, two annotators introduce 1–4 localized edits that preserve executability while causing at least one unit test to fail.
• •BugSourceBench-Human-Edited LM.For each task, we promptgpt-5-minito solve the original BigCodeBench problem, without asking it to generate a bug. We resample up to1616times until obtaining an executable but incorrect program. Annotators then edit this draft, for example by changing indices, conditions, or initializations, while keeping it executable and incorrect. Tasks for which no executable failing draft is found within the sampling budget are removed.
• •BugSourceBench-Qwen-7B.We promptQwen2.5-Coder-7B-Instructto solve each task and retain one sampled program that executes but fails at least one unit test, resampling up to1616times.
• •BugSourceBench-gpt-oss-20b.We apply the same procedure withgpt-oss-20b, yielding errors from a stronger code model.

Task alignment and splits.

To ensure controlled evaluation, we compute the intersection of task IDs across all bug-source variants separately for each split. We provide a large test split,test_all, with517517examples per source (20682068total instances), and a smallertestsplit with127127examples per source (508508total instances). The main experiments use the smallertestsplit. We also construct training splits by taking task IDs outsidetestandtest_alland intersecting them with BigCodeBench-trainto avoid leakage. These training splits are used to form the reference pools forAnchored Self-Play (ASP).

A.3Bug Source Analyses

We characterize bugs along two axes: semantic bug type and embedding-space source structure. For semantic type, we use five coarse categories:Logic Error, where the algorithm or reasoning is incorrect;Wrong Value, where an identifier, literal, constant, return value, or boundary condition is wrong;Missing Edge Case, where special cases or validations are missing or mishandled;API Misuse, where a library or framework API is used incorrectly; andOther, for errors not captured by the previous categories. We label each buggy program with GPT-4o conditioned on the task specification, reference solution, buggy code, and unit-test traces.Figure7reports the resulting type distribution and an embedding-based source-clustering analysis.

(a)Bug-type profiles by source. (b)kk-NN source clustering in embedding space.

Figure 7:Characterizing bug sources.We label each buggy program with a coarse semantic category using GPT-4o and report the distribution for eachBugSourceBenchsplit. Bug types vary systematically by source: human-edited bugs skew toward logic errors, gpt-oss-20b bugs toward edge-case and constraint violations, and Qwen-7B bugs toward type and API mistakes. We also embed each reference-to-bug diff withvoyage-code-3and compute the fraction of itskknearest neighbors, excluding same-task neighbors, that come from each source. Diagonal dominance indicates that bugs cluster by source in embedding space.Table 2:Test failure rates by bug source.We report the mean fraction of unit tests failed by each bug and the percentage of bugs that fail all tests.##### Failure-rate analysis.

Table2reports how often bugs from each source fail the unit tests.Humanbugs have the highest mean failure rate and the largest fraction of bugs that fail all tests.Human-Edited LMbugs are more subtle on average, often passing a larger fraction of the test suite.

A.4Evaluation of Frontier Models onBugSourceBench

Table3reports fix rates for frontier models across bug sources and repair interfaces.

Table 3:Repair interface comparison.Fix rate (%) acrossBugSourceBenchbug sources forRepair(full repair),+Tests(repair with unit-test traces), andDiff(patch output).Codegenis accuracy on the original code-generation tasks.##### Bug-source shift affects repair performance.

Fix rates vary substantially across bug sources. Bugs from incorrect model-generated solutions, especially Qwen-7B and gpt-oss-20b, are consistently harder to repair than human-written or human-edited LM bugs. This supports the need for controlled cross-source evaluation: conclusions from one bug source may not transfer to another.

A.5Ablation of Repair Interfaces forBugSourceBench

We compare four interfaces:

• •Code generation:generate a complete solution from the task description.
• •Full repair:generate a corrected program given the buggy code.
• •Diff repair:output a patch to modify the buggy code.
• •Repair with test traces:generate a corrected program given the buggy code and unit-test error traces.

Results are shown inTable3. Test traces improve repair performance, while diff repair often underperforms due to brittle formatting and patch application. For diff repair, we use a custom patch applier with fuzzy context matching. We use repair with test traces as the default interface in the main experiments because it performs well and matches common test-driven debugging workflows.

Debugging is distinct from code generation.

Solving a task from scratch and repairing an existing solution succeed on different instances.Table4reports the fraction of examples where one interface succeeds and the other fails. For example, onBugSourceBench-Human-Edited LM, Sonnet solves41.0%41.0\%of tasks from scratch that it fails to repair, while repairing43.9%43.9\%of tasks that it fails to solve from scratch. Similar patterns hold across models and bug sources, suggesting that repair requires capabilities, such as fault localization, that are not captured by end-to-end code generation alone.

Table 4:Repair and code-generation disagreement rates.Entries report the percentage of examples where one interface succeeds and the other fails.

Appendix BTraining Details

B.1Anchored Self-Play (ASP)and Self-Play Hyperparameters

We train aQwen2.5-Coder-7B-Instructpolicy with GRPO using the following settings.

• •Optimization and regularization.Learning rate1×10−61\times 10^{-6}. PPO-style clipping with ratio0.280.28.
• •Batching.Training batch size6464. Validation batch size256256. PPO minibatch size3232. Dynamic batch sizing enabled. Maximum PPO token budget of30,00030{,}000tokens per GPU.
• •Sequence lengths.Maximum prompt length81928192tokens. Maximum response length20482048tokens.
• •Rollouts.Asynchronous rollouts. Sampling temperature0.60.6for training and validation. Top-pp0.950.95for validation. Samples per prompt:n=4n=4for training andn=1n=1for validation.
• •Systems settings.Gradient checkpointing enabled.
• •Training schedule and logging.1010epochs total.
• •Parallelism and hardware.One node with88GPUs. Tensor parallel size11and sequence parallel size11.

Self-play loop.

For each task, we sampleG=4G=4candidate bugs. For each bug, we sampleK=4K=4independent repair attempts and estimate its solve rateρ\rho. The generator receives a band-shaped difficulty reward withρℓ=0.25\rho_{\ell}=0.25,ρh=0.75\rho_{h}=0.75, invalid-bug reward−1.0-1.0, and extreme-case penaltyα=0.2\alpha=0.2. The fixer receives failing test output as context. Generator and fixer advantages are normalized separately.

Anchoring.

ForAnchored Self-Play (ASP), the reference pool contains bugs from all fourBugSourceBenchsources:Human,Human-Edited LM,Qwen-7B, andgpt-oss-20b. We usevoyage-code-3to embed diffs between the reference solution and buggy program. The generator similarity reward useskk-NN scoring withk=5k=5, margin scoring with temperature5.05.0, reward weightλ=0.20\lambda=0.20, and an exponential-moving-average baseline with decayβ=0.99\beta=0.99. For reference mixing,20%20\%of fixer-training samples are drawn from the reference pool.

B.2Fixer-Only Hyperparameters

We train aQwen2.5-Coder-7B-Instructfixer with GRPO while keeping the generator frozen. To approximately match compute with Self-Play andASP, we usen=16n=16actor rollouts per training prompt; validation usesn=1n=1.

Data and prompting.

Fixer-Only is trained on a mixture of BigCodeBench tasks and the same reference bug-source training splits available toASP. During repair, the fixer receives failed unit-test output when available. At validation time, we evaluate both repair and standard code generation.

• •Optimization and regularization.Learning rate1×10−61\times 10^{-6}. PPO-style clipping with ratio0.280.28.
• •Batching.Training batch size6464. Validation batch size256256. PPO minibatch size3232. Dynamic batch sizing enabled. Maximum PPO token budget of24,00024{,}000tokens per GPU.
• •Sequence lengths.Maximum prompt length81928192tokens. Maximum response length20482048tokens.
• •Rollouts.Asynchronous rollouts. Sampling temperature0.60.6for training and validation. Top-pp0.950.95for both the frozen generator and validation sampling. Samples per prompt:n=16n=16for training andn=1n=1for validation.
• •Frozen generator configuration.Generator model: Qwen2.5-Coder-7B-Instruct. Generation temperature0.60.6and top-pp0.950.95.
• •Training schedule and logging.1010epochs total.
• •Parallelism and hardware.One node with88GPUs. Tensor parallel size11and sequence parallel size11.

Compute.

All runs use one node with88H100 GPUs and take approximately4848hours for120120training steps.

Appendix CPrompts

We use two role-specific prompts: abug generatorprompt that injects faults into a correct reference solution, and abug fixerprompt that repairs a buggy program using unit-test feedback when available. We report the prompts verbatim. Placeholders such as<PROBLEM>,<CORRECT_CODE>,<BUGGY_CODE>, and<FAILED_TEST_OUTPUT>are filled at runtime.

Bug generator prompt Bug fixer prompt

Appendix D Additional Evaluations D\.1 Pass@k Results We report pass@kk under temperature sampling in Table 5, averaging over all BugSourceBench sources\. ASP outperforms all baselines at each sampling budget, and the gap increases at larger kk\. Table 5: Pass@kk on BugSourceBench\. Results are averaged across bug sources\. Method pass@1 pass@5 pass@10 Base Model 25\.8 37\.3 43\.8 Fixer\-Only 27\.9 40\.4 46\.2 Self\-Play 29\.1 41\.5 48\.3 ASP \(Ours\) 36\.1 50\.7 57\.2 D\.2 Qualitative Examples of Generated Bugs Figure 8 compares bugs generated before and after training with ASP\. BugSourceBench TaskID 953 Generates random data points to plot bar charts for each in a given list of plot names, then saves them in a specified directory\. This function takes a list of plot names, for each generating 10 random data points in \[0, 1\) to create a bar chart, then saves the bar charts as \.png files in the specified directory, creating the directory if it does not exist\. \>\>\>\> task\_func\(\[’First Plot’, ’Second Plot’\], ’\./another\_folder/’\) \[’First\_Plot\.png’, ’Second\_Plot\.png’\] Note that: This function deduplicates mystrings while maintaining its original order\. Random data points for bar charts are generated in the range \[0, 1\)\. Each bar chart contains 10 data points\. The function should raise the exception for: FileNotFoundError: If the provided directory path does not exist and cannot be created\. The function should output with: list: Names of the files where the plots are saved\. Each file corresponds to a title from mystrings\. BASE ⬇ import numpy as np import matplotlib\.pyplot as plt import os def task\_func\(mystrings, folder\_path, seed=None\): if seed is not None: np\.random\.seed\(seed\) saved\_plots = \[\] processed\_names = set\(\) if not os\.path\.exists\(folder\_path\): os\.mkdir\(folder\_path\) for name in mystrings: if name in processed\_names: continue data = np\.random\.rand\(10\) plt\.bar\(range\(len\(data\)\), data\) plt\.title\(name\) file\_name = name\.replace\(" ", "\_"\) \+ "\.png" plt\.savefig\(os\.path\.join\(folder\_path, file\_name\)\) saved\_plots\.append\(file\_name\) processed\_names\.add\(name\) return saved\_plots ASP ⬇ import numpy as np import matplotlib\.pyplot as plt import os def task\_func\(mystrings, folder\_path, seed=None\): if seed is not None: np\.random\.seed\(seed\) saved\_plots = \[\] if not os\.path\.exists\(folder\_path\): raise Exception\("Directory could not be created\."\) for name in mystrings: data = np\.random\.rand\(10\) plt\.bar\(range\(len\(data\)\), data\) plt\.title\(name\) file\_name = name\.replace\(" ", "\_"\) \+ "\.png" plt\.savefig\(os\.path\.join\(folder\_path, file\_name\)\) saved\_plots\.append\(file\_name\) return \["error"\] Figure 8: BugSourceBench example comparing bugs generated by the base model and by the ASP\-trained generator\. The base generator produces a simple synthetic error, while the ASP\-trained generator introduces more semantic failures, including missing de\-duplication, improper error handling, and an incorrect return value\. D\.3 Similarity of Generated Bugs Mean similarity over training\. For each generator checkpoint, we sample synthetic bugs on held\-out BugSourceBench tasks\. We embed each reference\-to\-bug diff and compute its mean top\-kk cosine similarity to a target bug pool, excluding same\-task matches when available\. Averaging this score across generated bugs gives a similarity trajectory that measures how the generator distribution moves relative to the target source over training\. Fix rate by generation\-similarity quantile\. To test whether downstream gains are explained only by producing more target\-like bugs, we bin tasks by the generator’s similarity to the target pool using shared quantile boundaries across methods\. Within each bin, we evaluate fixer performance on the corresponding held\-out BugSourceBench instances\. This matched\-similarity diagnostic compares ASP and Self\-Play at comparable levels of generator\-target similarity, allowing us to test whether ASP improves robustness beyond shifting the generator toward higher\-similarity bugs\. \(a\) Similarity\-guided shaping increases target\-likeness of generated bugs\. \(b\) Performance on test tasks by generated similarity\. Figure 9: We sample n=3n=3 bugs from the generator for each held\-out BugSourceBench task and compute the kk\-NN embedding similarity of generated bugs to target\-source bugs\. Similarity\-guided shaping in ASP increases target similarity over training relative to standard self\-play \(left\)\. We then bin tasks by generated\-bug similarity, sample k=3k=3 repair attempts per task, and report fix rate with 95% confidence intervals \(right\)\. D\.4 Evaluation by Semantic Bug Type We group bugs by semantic type and report fix rates in Table 6\. ASP improves most categories, suggesting that its gains are not limited to a single error pattern\. Table 6: Fix rate \(%\) by semantic bug type\. Fix rates are reported in percent\. Model Logic Wrong Val\. Edge Case API Misuse Other Qwen\-7B \(base\) 14 37 5 20 33 Fixer\-Only 12 37 5 14 28 Self\-Play 16 37 9 18 44 ASP \(Ours\) 21 36 12 29 38 D\.5 Evaluation as a Test\-Time Fixer We evaluate whether ASP\-trained fixers can improve larger code\-generation models at test time\. We use DeepSeek\-Coder\-33B and Qwen2\.5\-Coder\-32B as initial coders, generate candidate solutions, and then apply 7B fixer models for rr repair rounds\. Table 7 reports pass rates with kk samples\. ASP generally outperforms the base, Fixer\-Only, and Self\-Play fixers, and can improve over self\-repair by the larger coder\. Table 7: Test\-time fixer results\. Pass rate \(%\) using large coders with fixer models over rr repair rounds and kk samples\. The No repair row is the coder alone \(r=0r\{=\}0\); all other rows apply the listed fixer\. r=1r\{=\}1 r=2r\{=\}2 Fixer k=1k\{=\}1 k=2k\{=\}2 k=1k\{=\}1 k=2k\{=\}2 Coder: DeepSeek\-33B No repair: k=1k\{=\}1: 29\.5 k=2k\{=\}2: 37\.0 Qwen\-7B \(base\) 35\.4 40\.9 36\.2 47\.2 Fixer\-Only 34\.6 41\.7 35\.4 41\.7 Self\-Repair \(Qwen\-32B\) 34\.6 42\.5 37\.8 49\.6 Self\-Play 35\.4 42\.5 37\.8 46\.5 ASP \(Ours\) 36\.2 45\.7 43\.3 53\.5 Coder: Qwen\-32B No repair: k=1k\{=\}1: 30\.3 k=2k\{=\}2: 37\.0 Qwen\-7B \(base\) 33\.1 40\.9 36\.2 44\.1 Fixer\-Only 33\.9 40\.2 37\.0 43\.3 Self\-Repair \(DeepSeek\-33B\) 35\.4 40\.9 44\.9 57\.5 Self\-Play 33\.9 43\.3 37\.8 48\.0 ASP \(Ours\) 38\.6 44\.1 44\.9 54\.3 Appendix E Additional Experiments E\.1 Additional Baselines We compare against additional baselines that test alternative ways of improving repair\. RL for code generation\. To test whether improvements come from better general code\-generation ability rather than repair\-specific training, we train the base model to generate solutions from task specifications using binary unit\-test rewards\. We use the same task splits and comparable training settings as in the repair experiments\. Self\-play with constrained bug generation\. As an alternative to open\-ended bug generation, we evaluate a constrained generator based on AST mutations\. We identify valid mutation sites in the reference implementation and ask the generator to select mutations that preserve syntactic validity while producing failing programs\. We then train with the same self\-play procedure\. Supervised fine\-tuning for repair\. We train a fixer with supervised fine\-tuning on BugSourceBench training instances from all four sources, using a masked cross\-entropy loss on the reference correct solutions\. Alternate embeddings for ASP\. To test sensitivity to the embedding model, we run ASP with CodeBERT embeddings instead of voyage\-code\-3\. Table 8: Additional baselines\. For BugSourceBench splits, we report repair fix rate \(%\)\. For Codegen, we report pass rate on the original code\-generation task\. Results in Table 8 show that RL post\-training for code generation improves code\-generation pass rate but does not improve repair, and often degrades it\. Fixer SFT improves the LM\-generated splits slightly but substantially degrades Human and Human\-Edited LM performance\. The CodeBERT variant of ASP remains strong, suggesting that the method is not tied to a single embedding model\. E\.2 Additional Ablations Embedding and similarity ablations\. We ablate the kk\-NN pooling parameter and the embedding model used for the similarity reward in \(Table 9\)\. ASP is stable across moderate values of kk and performs similarly with voyage\-code\-3 and CodeBERT embeddings\. Table 9: Embedding and kk\-NN ablations\. We vary the kk\-NN pooling parameter using voyage\-code\-3 embeddings and compare against CodeBERT\. Updating the generator on mixed episodes\. Reference mixing replaces a generated bug with a reference bug for fixer training\. We do not update the generator on these mixed episodes because the bug was not sampled from the generator; applying a generator update would be off\-policy or require additional fixer rollouts\. Table 10 shows that updating the generator on mixed episodes reduces average fix rate from 36\.1%36\.1\\% to 33\.6%33\.6\\%, supporting our choice to update only the fixer on reference\-mixed episodes\. Table 10: Updating the generator on reference\-mixed episodes\. Fix rates are reported on BugSourceBench splits\. Decoupled generator and fixer\. We test whether drift and ASP’s gains depend on sharing weights between generator and fixer\. Table 11 compares shared and decoupled variants\. Drift is not removed by decoupling, and ASP remains strong with separate generator and fixer weights\. This suggests that drift arises from the unit\-test\-only reward signal rather than from weight sharing alone\. Table 11: Shared vs\. decoupled generator/fixer weights\. We compare standard Self\-Play and ASP with shared or separate generator and fixer models\. Alternate base model\. We replicate the main experiments using DeepSeek\-Coder\-6\.7B\-Instruct as the base model\. As shown in Table 12, Self\-Play yields limited gains and slightly degrades human\-originated bugs, while ASP improves all sources and gains \+8\.9\+8\.9 pp over Self\-Play on average\. Table 12: Alternate base model\. Results using DeepSeek\-Coder\-6\.7B\-Instruct as the base model\. External benchmarks and task distributions\. We also evaluate on DebugBench \(Tian et al\., 2024\), a debugging benchmark derived from LeetCode\-style problems\. We use the 1,3401\{,\}340 Python buggy programs and the same fixer prompt format as in BugSourceBench\. Table 13\(a\) shows that ASP transfers to this external distribution and outperforms standard Self\-Play\. \(a\) Fix rates on DebugBench\. Models are trained on BigCodeBench\-based tasks and evaluated on DebugBench Python bugs\. \(b\) Held\-out DebugBench\. Models are trained and evaluated on DebugBench splits\. Table 13: DebugBench transfer and in\-domain training\. ASP improves over standard Self\-Play both when transferred from BigCodeBench\-based training to DebugBench and when trained directly on DebugBench\. References M\. Allamanis, H\. Jackson\-Flux, and M\. Brockschmidt \(2021\) Self\-supervised bug detection and repair\. In Advances in Neural Information Processing Systems, Vol\. 34, pp\. 27865–27876\. External Links: Link Cited by: §1, §8\. J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. Cai, M\. Terry, Q\. Le, et al\. \(2021\) Program synthesis with large language models\. In arXiv preprint arXiv:2108\.07732, Cited by: §8\. Y\. Bengio, J\. Louradour, R\. Collobert, and J\. Weston \(2009\) Curriculum learning\. In Proceedings of the 26th Annual International Conference on Machine Learning, pp\. 41–48\. Cited by: §8\. L\. Chen, M\. Prabhudesai, K\. Fragkiadaki, H\. Liu, and D\. Pathak \(2025\) Self\-questioning language models\. arXiv preprint arXiv:2508\.03682\. Cited by: §8\. M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. d\. O\. Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman, et al\. \(2021\) Evaluating large language models trained on code\. In arXiv preprint arXiv:2107\.03374, Cited by: §8\. Z\. Chen, M\. Tufano, J\. Pantiuchina, C\. Watson, G\. B\. Jiang, and D\. Poshyvanyk \(2019\) SequenceR: sequence\-to\-sequence learning for end\-to\-end program repair\. In Proceedings of the 2019 IEEE/ACM 41st International Conference on Software Engineering \(ICSE\), Cited by: §8\. Z\. Chen, Y\. Deng, H\. Yuan, K\. Ji, and Q\. Gu \(2024\) Self\-play fine\-tuning converts weak language models to strong language models\. arXiv preprint arXiv:2401\.01335\. Cited by: §8\. J\. Cheng, X\. Liu, C\. Wang, X\. Gu, Y\. Lu, D\. Zhang, Y\. Dong, J\. Tang, H\. Wang, and M\. Huang \(2024\) Spar: self\-play with tree\-search refinement to improve instruction\-following in large language models\. arXiv preprint arXiv:2412\.11605\. Cited by: §8\. C\. Colas, T\. Karch, O\. Sigaud, and P\. Oudeyer \(2022\) Autotelic agents with intrinsically motivated goal\-conditioned reinforcement learning: a short survey\. External Links: 2012\.09830, Link Cited by: §8\. K\. Dong and T\. Ma \(2025\) STP: self\-play llm theorem provers with iterative conjecturing and proving\. External Links: 2502\.00212, Link Cited by: §8\. S\. Dou, H\. Jia, S\. Wu, H\. Zheng, M\. Wu, Y\. Tao, M\. Zhang, M\. Chai, J\. Fan, Z\. Xi, R\. Zheng, Y\. Wu, M\. Wen, T\. Gui, Q\. Zhang, X\. Qiu, and X\. Huang \(2025\) What’s wrong with your code generated by large language models? an extensive study\. External Links: 2407\.06153, Link Cited by: §8\. S\. Forrest, T\. Nguyen, W\. Weimer, and C\. Le Goues \(2009\) A genetic programming approach to automated software repair\. In Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation \(GECCO\), Cited by: §1, §8\. J\. He, L\. Beurer\-Kellner, and M\. Vechev \(2022\) On distribution shift in learning\-based bug detectors\. In International conference on machine learning, pp\. 8559–8580\. Cited by: §8\. D\. Hendrycks, S\. Basart, S\. Kadavath, M\. Mazeika, A\. Arora, E\. Guo, C\. Burns, S\. Puranik, H\. He, D\. Song, et al\. \(2021\) Measuring coding challenge competence with apps\. In Advances in Neural Information Processing Systems, Cited by: §8\. C\. Huang, W\. Yu, X\. Wang, H\. Zhang, Z\. Li, R\. Li, J\. Huang, H\. Mi, and D\. Yu \(2025\) R\-zero: self\-evolving reasoning llm from zero data\. arXiv preprint arXiv:2508\.05004\. Cited by: §8\. B\. Hui, J\. Yang, Z\. Cui, J\. Yang, D\. Liu, L\. Zhang, T\. Liu, J\. Zhang, B\. Yu, K\. Lu, et al\. \(2024\) Qwen2\.5\-Coder technical report\. arXiv preprint arXiv:2409\.12186\. Cited by: §5\. N\. Jain, K\. Han, A\. Gu, W\. Li, F\. Yan, T\. Zhang, S\. Wang, A\. Solar\-Lezama, K\. Sen, and I\. Stoica \(2024\) LiveCodeBench: holistic and contamination free evaluation of large language models for code\. External Links: 2403\.07974, Link Cited by: §8\. C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. Narasimhan \(2023\) Swe\-bench: can language models resolve real\-world github issues?\. arXiv preprint arXiv:2310\.06770\. Cited by: §1, §8\. R\. Just, D\. Jalali, and M\. D\. Ernst \(2014\) Defects4J: a database of existing faults to enable controlled testing studies for java programs\. In Proceedings of the 2014 international symposium on software testing and analysis, pp\. 437–440\. Cited by: §1, §8\. C\. Kapfer, K\. Stine, B\. Narasimhan, C\. Mentzel, and E\. Candès \(2025\) Marlowe: stanford’s gpu\-based computational instrument\. Zenodo\. External Links: Document, Link Cited by: Acknowledgements\. J\. G\. Kuba, M\. Gu, Q\. Ma, Y\. Tian, V\. Mohan, and J\. Chen \(2025\) Language self\-play for data\-free training\. arXiv preprint arXiv:2509\.07414\. Cited by: §8\. C\. Le Goues, N\. Holtschulte, E\. K\. Smith, Y\. Brun, P\. Devanbu, S\. Forrest, and W\. Weimer \(2015\) The manybugs and introclass benchmarks for automated repair of c programs\. IEEE Transactions on Software Engineering 41 \(12\), pp\. 1236–1256\. Cited by: §1, §8\. D\. Lin, J\. Koppel, A\. Chen, and A\. Solar\-Lezama \(2017\) QuixBugs: a multi\-lingual program repair benchmark set based on the quixey challenge\. In Proceedings Companion of the 2017 ACM SIGPLAN international conference on systems, programming, languages, and applications: software for humanity, pp\. 55–56\. Cited by: §8\. Z\. Lin, S\. Shen, J\. Shang, J\. Weston, and Y\. Nie \(2025\) Learning to solve and verify: a self\-play framework for code and test generation\. arXiv preprint arXiv:2502\.14948\. Cited by: §8\. B\. Liu, C\. Jin, S\. Kim, W\. Yuan, W\. Zhao, I\. Kulikov, X\. Li, S\. Sukhbaatar, J\. Lanchantin, and J\. Weston \(2025\) Spice: self\-play in corpus environments improves reasoning\. arXiv preprint arXiv:2510\.24684\. Cited by: §8\. J\. Liu, C\. S\. Xia, Y\. Wang, and L\. Zhang \(2023\) Is your code generated by ChatGPT really correct? rigorous evaluation of large language models for code generation\. In Advances in Neural Information Processing Systems, Vol\. 36\. Note: arXiv:2305\.01210 External Links: Link Cited by: §8\. F\. Long and M\. Rinard \(2016\) Automatic patch generation by learning correct code\. In Proceedings of the 43rd Annual ACM SIGPLAN\-SIGACT Symposium on Principles of Programming Languages \(POPL\), Cited by: §8\. F\. Madeiral, S\. Urli, M\. Maia, and M\. Monperrus \(2019\) Bears: an extensible Java bug benchmark for automatic program repair studies\. In Proceedings of the 26th IEEE International Conference on Software Analysis, Evolution and Reengineering \(SANER\), pp\. 468–478\. External Links: Document, Link Cited by: §1, §8\. G\. A\. Oliva, G\. K\. Rajbahadur, A\. Bhatia, H\. Zhang, Y\. Chen, Z\. Chen, A\. Leung, D\. Lin, B\. Chen, and A\. E\. Hassan \(2025\) Spice: an automated swe\-bench labeling pipeline for issue clarity, test coverage, and effort estimation\. arXiv preprint arXiv:2507\.09108\. Cited by: §1\. J\. Parker\-Holder, M\. Jiang, M\. Dennis, M\. Samvelyan, J\. Foerster, E\. Grefenstette, and T\. Rocktäschel \(2023\) Evolving curricula with regret\-based environment design\. External Links: 2203\.01302, Link Cited by: §8\. M\. V\. Pham, H\. N\. Phan, H\. N\. Phan, C\. L\. Chi, T\. N\. Nguyen, and N\. D\. Bui \(2025\) SWE\-synth: synthesizing verifiable bug\-fix data to enable large language models in resolving real\-world bugs\. arXiv preprint arXiv:2504\.14757\. Cited by: §8, §8\. G\. Poesia, D\. Broman, N\. Haber, and N\. D\. Goodman \(2024\) Learning formal mathematics from intrinsic motivation\. External Links: 2407\.00695, Link Cited by: §8\. J\. Pourcel, C\. Colas, G\. Molinaro, P\. Oudeyer, and L\. Teodorescu \(2024\) ACES: generating a diversity of challenging programming puzzles with autotelic generative models\. Advances in Neural Information Processing Systems 37, pp\. 67627–67662\. Cited by: §8\. M\. T\. Ribeiro and S\. Lundberg \(2022\) Adaptive testing and debugging of NLP models\. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), Dublin, Ireland, pp\. 3253–3267\. External Links: Document, Link Cited by: §8\. D\. Silver, J\. Schrittwieser, K\. Simonyan, I\. Antonoglou, A\. Huang, A\. Guez, T\. Hubert, L\. Baker, M\. Lai, A\. Bolton, et al\. \(2017\) Mastering the game of go without human knowledge\. nature 550 \(7676\), pp\. 354–359\. Cited by: §8\. A\. Sonwane, I\. White, H\. Lee, M\. Pereira, L\. Caccia, M\. Kim, Z\. Shi, C\. Singh, A\. Sordoni, M\. Côté, and X\. Yuan \(2025\) BugPilot: complex bug generation for efficient learning of swe skills\. External Links: 2510\.19898, Link Cited by: §8\. L\. Teodorescu, C\. Colas, M\. Bowers, T\. Carta, and P\. Oudeyer \(2023\) Codeplay: autotelic learning through collaborative self\-play in programming environments\. In IMOL 2023: Intrinsically Motivated Open\-ended Learning Workshop at NeurIPS 2023, External Links: Link Cited by: §8\. R\. Tian, Y\. Ye, Y\. Qin, X\. Cong, Y\. Lin, Y\. Pan, Y\. Wu, H\. Hui, W\. Liu, Z\. Liu, and M\. Sun \(2024\) DebugBench: evaluating debugging capability of large language models\. External Links: 2401\.04621, Link Cited by: §E\.2\. Y\. Wang, L\. Yang, Y\. Tian, K\. Shen, and M\. Wang \(2025\) CURE: co\-evolving coders and unit testers via reinforcement learning\. In The Thirty\-ninth Annual Conference on Neural Information Processing Systems, Cited by: §8\. Y\. Wei, Z\. Sun, E\. McMilin, J\. Gehring, D\. Zhang, G\. Synnaeve, D\. Fried, L\. Zhang, and S\. Wang \(2025\) Toward training superintelligent software agents through self\-play swe\-rl\. arXiv preprint arXiv:2512\.18552\. Cited by: §1, §8\. R\. Widyasari, S\. Q\. Sim, C\. Lok, H\. Qi, J\. Phan, Q\. Tay, C\. Tan, F\. Wee, J\. E\. Tan, Y\. Yieh, et al\. \(2020\) Bugsinpy: a database of existing bugs in python programs to enable controlled testing and debugging studies\. In Proceedings of the 28th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, pp\. 1556–1560\. Cited by: §1, §8\. A\. Wilf, P\. Aggarwal, B\. Parno, D\. Fried, L\. Morency, P\. P\. Liang, and S\. Welleck \(2025\) Propose, solve, verify: self\-play through formal verification\. arXiv preprint arXiv:2512\.18160\. Cited by: §8\. F\. F\. Xu, U\. Alon, G\. Neubig, and V\. J\. Hellendoorn \(2022\) A systematic evaluation of large language models of code\. In Proceedings of the 6th ACM SIGPLAN international symposium on machine programming, pp\. 1–10\. Cited by: §1, §8\. J\. Yang, K\. Leret, C\. E\. Jimenez, A\. Wettig, K\. Khandpur, Y\. Zhang, B\. Hui, O\. Press, L\. Schmidt, and D\. Yang \(2025a\) SWE\-smith: scaling data for software engineering agents\. arXiv preprint arXiv:2504\.21798\. Cited by: §8, §8\. W\. Yang, H\. Wang, Z\. Liu, X\. Li, Y\. Yan, S\. Wang, Y\. Gu, M\. Yu, Z\. Liu, and G\. Yu \(2025b\) COAST: enhancing the code debugging ability of llms through communicative agent based data synthesis\. External Links: 2408\.05006, Link Cited by: §8\. M\. Yasunaga and P\. Liang \(2021\) Break\-it\-fix\-it: unsupervised learning for program repair\. In International Conference on Machine Learning \(ICML\), Cited by: §8\. H\. Ye, M\. Martinez, X\. Luo, T\. Zhang, and M\. Monperrus \(2023\) SelfAPR: self\-supervised program repair with test execution diagnostics\. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering, ASE ’22, New York, NY, USA\. External Links: ISBN 9781450394758, Link, Document Cited by: §8\. W\. Yu, Z\. Liang, C\. Huang, K\. Panaganti, T\. Fang, H\. Mi, and D\. Yu \(2025\) Guided self\-evolving llms with minimal human supervision\. arXiv preprint arXiv:2512\.02472\. Cited by: §8\. A\. Zhao, Y\. Wu, Y\. Yue, T\. Wu, Q\. Xu, M\. Lin, S\. Wang, Q\. Wu, Z\. Zheng, and G\. Huang \(2025\) Absolute zero: reinforced self\-play reasoning with zero data\. arXiv preprint arXiv:2505\.03335\. Cited by: §8\. T\. Y\. Zhuo, M\. C\. Vu, J\. Chim, H\. Hu, W\. Yu, R\. Widyasari, I\. N\. B\. Yusuf, H\. Zhan, J\. He, I\. Paul, et al\. \(2024\) Bigcodebench: benchmarking code generation with diverse function calls and complex instructions\. arXiv preprint arXiv:2406\.15877\. Cited by: §3\.1, §5\. A\. Zirak and H\. Hemmati \(2024\) Improving automated program repair with domain adaptation\. ACM Trans\. Softw\. Eng\. Methodol\. 33 \(3\)\. External Links: ISSN 1049\-331X, Link, Document Cited by: §8\.