Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models

# Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models
Source: [https://arxiv.org/html/2607.12463](https://arxiv.org/html/2607.12463)
Yubo Wang1,5111Equal contribution\. Corresponding authors: yubo\.wang\.sunny@gmail\.com, hustchenwenhu@gmail\.comJiarong Liang1111Equal contribution\. Corresponding authors: yubo\.wang\.sunny@gmail\.com, hustchenwenhu@gmail\.comYuxuan Zhang2Xuye Liu1Cong Wei1,3 Yuyu Zhang4Ping Nie1Wenhu Chen1,5111Equal contribution\. Corresponding authors: yubo\.wang\.sunny@gmail\.com, hustchenwenhu@gmail\.com 1University of Waterloo,2University of British Columbia,3NVIDIA,4Verdent AI,5Vector Institute

###### Abstract

Coding agents must integrate external tool returns into ongoing reasoning—a capability that standard left\-to\-right pretraining on code exposes only in its forward direction\. We observe that theaction→\\rightarrowobservation→\\rightarrowcontinuationloop of a coding agent is structurally isomorphic to a function call site, where a caller binds arguments, a callee returns a value computed elsewhere, and downstream code consumes that value\. This conditioning structure exists at internet scale in ordinary code\. We exploit it through*function\-aware fill\-in\-the\-middle \(FIM\) mid\-training*: a self\-supervised objective that masks functions selected via program dependency graph analysis and a complexity–inferability double criterion\. We mid\-train Qwen2\.5\-Coder\-Instruct \(7B/14B\) and Qwen3\-8B on a 2\.6B\-token decontaminated corpus drawn from 968 GitHub repositories, then apply existing agentic post\-training pipelines\. Mid\-training improves SWE\-Bench\-Verified by\+2\.8/\+3\.0\+2\.8/\+3\.0at 7B/14B and by\+3\.2\+3\.2on Qwen3\-8B; SWE\-Bench\-Lite gains are\+3\.7/\+4\.0/\+5\.4\+3\.7/\+4\.0/\+5\.4on the same models\. The improvement holds across two post\-training pipelines \(R2E\-Gym, SWE\-Smith\) and on a non\-Qwen2\.5 base \(Qwen3\-8B with SWE\-Lego\)\. Beyond in\-domain gains, mid\-training also mitigates the capability erosion that agentic post\-training otherwise inflicts on non\-agent coding \(e\.g\., LiveCodeBench\) and non\-coding tool\-use benchmarks \(τ\\tau\-bench, BFCL\): although the mid\-training corpus contains Python code only, the function\-call inductive bias survives post\-training and yields consistent gains\.

![Refer to caption](https://arxiv.org/html/2607.12463v1/x1.png)Figure 1:Left:A function call site and a single step of a coding agent are*structurally similar*, decomposing into the same four stages: context, call/action, return/observation, continuation\.Middle:We exploit this analogy via function\-aware FIM mid\-training\. A functionBBis selected from the program dependency graph using complexity \(H^\\hat\{H\}\) and inferability \(I^\\hat\{I\}\) scores; the model is then mid\-trained to fill inBB’s body together with a CoT rationale, given the surrounding file as an FIM\-formatted prompt\.Right:Mid\-training yields consistent gains across both Qwen2\.5\-Coder\-Instruct \(7B, 14B\) and Qwen3 \(8B\) on SWE\-Bench\-Verified \(solid bars\) and SWE\-Bench\-Lite \(hatched bars\)\.## 1Introduction

Coding agents that resolve real software engineering issues have moved from research demos to deployed systems\(Jimenezet al\.,[2023](https://arxiv.org/html/2607.12463#bib.bib1); Yanget al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib2); Wanget al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib3)\)\. Their progress, however, has been driven almost entirely by scaling synthetic agent\-trajectory data during post\-training: pipelines such as SWE\-Gym\(Panet al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib4)\), R2E\-Gym\(Jainet al\.,[2025](https://arxiv.org/html/2607.12463#bib.bib5)\), and SWE\-Smith\(Yanget al\.,[2025b](https://arxiv.org/html/2607.12463#bib.bib6)\)curate or synthesize trajectories that imitate human or LLM behavior on issue\-resolution tasks\. The base model these pipelines start from is typically a code LLM trained with next\-token prediction \(and, in some cases, random\-span FIM\) on internet\-scale code\(Huiet al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib8); Guoet al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib9); Liet al\.,[2023](https://arxiv.org/html/2607.12463#bib.bib10)\)\. Between these two stages lies a training\-time gap: the base model is rarely optimized for the conditioning structure that agentic post\-training will later demand\. We treat this gap as an opportunity for a dedicated*mid\-training*stage that aligns the base model with agent\-relevant inductive biases before agent\-specific data is introduced\.

Our central observation is that the inductive bias required by a coding agent already exists in ordinary code, but in a shape that left\-to\-right pretraining systematically under\-exposes\. At each step an agent maintains a historyhth\_\{t\}, samples an actionat∼π​\(at∣ht\)a\_\{t\}\\sim\\pi\(a\_\{t\}\\mid h\_\{t\}\), receives an observationot\+1o\_\{t\+1\}produced by an external process, and continues conditioned on the entire trace\. This four\-part decomposition—context, action, externally\-computed return, continuation—is precisely the decomposition of a function call site: pre\-call code that establishes intent and binds arguments; the call itself; a return value produced by code outside the immediate scope; and downstream code that consumes the return value \(Figure[1](https://arxiv.org/html/2607.12463#S0.F1), left\)\. A model trained to reason*bidirectionally*about function\-level dependencies must learn to reconstruct a callee’s behavior from caller context and downstream usage, which is the same competence required to predict an agent’s continuation given a history and a tool return\. The correspondence is structural rather than literal—FIM training conditions on a given suffix while agent inference generates one—but it suggests that representations induced by the former should transfer to the latter, an empirical question we address in Section[3](https://arxiv.org/html/2607.12463#S3)\.

The fill\-in\-the\-middle objective is not new\(Bavarianet al\.,[2022](https://arxiv.org/html/2607.12463#bib.bib14)\); recent code LLMs mix random\-span FIM into pretraining\(Huiet al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib8); Guoet al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib9); Liet al\.,[2023](https://arxiv.org/html/2607.12463#bib.bib10)\)\. Random\-span FIM is nonetheless poorly aligned with agentic conditioning for three reasons\.*\(i\) Span boundaries are syntactically arbitrary:*most masked spans cut through expressions or partial statements and carry weak signal about function\-level dependencies\.*\(ii\) There is no reasoning supervision:*the model fills the span directly, with no intermediate rationale mirroring an agent’s think\-then\-act pattern\.*\(iii\) The objective is dissolved into pretraining:*by the time post\-training begins, any FIM\-conferred structural prior has been amortized across trillions of unrelated tokens\. We address all three points: masking targets are selected at function granularity via program dependency graph analysis with a base\-model\-agnostic complexity–inferability double criterion \(Section[2\.3](https://arxiv.org/html/2607.12463#S2.SS3)\); chain\-of\-thought rationales are embedded*inside*the FIM middle span so the model produces reasoning consistent with the eventual code \(Section[2\.4](https://arxiv.org/html/2607.12463#S2.SS4)\); and the objective is applied at a dedicated mid\-training stage, concentrating its signal immediately before agentic post\-training\.

We evaluate this recipe along three axes of robustness\.*On the Qwen2\.5\-Coder\-Instruct series*, mid\-training improves SWE\-Bench\-Verified by\+2\.8\+2\.8and\+3\.0\+3\.0points at 7B and 14B respectively, indicating that the structural prior is not absorbed by larger pretrained models in the practical deployment range\.*Across two post\-training pipelines*, mid\-training improves both R2E\-Gym and SWE\-Smith on the same 7B base, with the SWE\-Smith pairing yielding\+5\.3\+5\.3points on SWE\-Bench\-Verified\.*On a non\-Qwen2\.5 base*, mid\-training transfers to Qwen3\-8B \(paired with SWE\-Lego\) for a\+3\.2\+3\.2\-point gain on SWE\-Bench\-Verified; this single comparison varies the post\-training pipeline jointly with the base model and should be read as evidence that the prior is not specific to a single Qwen2\.5\-Coder\+\+R2E\-Gym/SWE\-Smith combination, rather than as a guarantee across families\.

A second set of results probes our motivating hypothesis\. All checkpoints are evaluated after the full pipeline \(post\-training alone, or mid\-training followed by post\-training\), since FIM\-only checkpoints degrade instruction\-following and are not comparable to instruction\-tuned baselines\. Agentic post\-training alone substantially regresses non\-target capabilities at 14B, dropping LiveCodeBench, BFCL, andτ\\tau\-bench by double\-digit margins in some cases; adding mid\-training before the same post\-training restores\+11\.1\+11\.1on LiveCodeBench,\+2\.4\+2\.4on BFCL, and\+3\.9\+3\.9onτ\\tau\-bench\. Since the mid\-training corpus contains only Python code with no tool\-use data, the cross\-domain recovery is direct evidence for the function\-call/tool\-call isomorphism\.

Contributions\.\(1\)Framing function call sites as the internet\-scale analogue of the agent action–observation–continuation loop, motivating function\-granularity FIM as a self\-supervised prior for agent capability\.\(2\)A function\-aware FIM mid\-training pipeline combining program dependency graph analysis, a base\-model\-agnostic complexity–inferability double criterion, and CoT rationales embedded inside the FIM middle span\.\(3\)Robustness validation along three axes—two model sizes \(Qwen2\.5\-Coder\-Instruct 7B/14B\), two post\-training pipelines \(R2E\-Gym, SWE\-Smith\), and one alternative base \(Qwen3\-8B with SWE\-Lego\)—with consistent in\-domain gains in every configuration\.\(4\)A direct test of the motivating hypothesis: the same Python\-only corpus also yields gains onτ\\tau\-bench, BFCL, and LiveCodeBench, evidence that the function\-call inductive bias survives post\-training and transfers across task families\.\(5\)Open release of the 968\-repository decontaminated corpus \(400​K400\\mathrm\{K\}FIM samples,2\.6​B2\.6\\mathrm\{B\}tokens\), the selection pipeline, and mid\-training checkpoints\.

## 2Method

### 2\.1Motivation: Function Calls as Agent\-Like Structures

A coding agent at stepttsamplesat∼π​\(at∣ht\)a\_\{t\}\\sim\\pi\(a\_\{t\}\\mid h\_\{t\}\), observesot\+1∼p​\(ot\+1∣ht,at\)o\_\{t\+1\}\\sim p\(o\_\{t\+1\}\\mid h\_\{t\},a\_\{t\}\), and continues\. Function calls mirror this loop \(Figure[1](https://arxiv.org/html/2607.12463#S0.F1), left\): pre\-call context, call, return, and downstream usage align with history, action, observation, and continuation\. This isomorphism motivates a fill\-in\-the\-middle \(FIM\) objective drawn from code\. Random\-span FIM\(Huiet al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib8); Guoet al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib9); Liet al\.,[2023](https://arxiv.org/html/2607.12463#bib.bib10)\)captures it only incidentally; our*function\-aware*variant selects masking targets by program structure and contextual predictability\.

### 2\.2Data Collection and Decontamination

We curate a corpus of968968Python repositories from GitHub\. Starting from∼2,000\\sim\\\!2\{,\}000candidates retrieved by combining a star\-count threshold with ten topic queries, we apply manual quality filtering and remove every repository whose origin overlaps with the source repositories of SWE\-Bench\(Jimenezet al\.,[2023](https://arxiv.org/html/2607.12463#bib.bib1)\)\(verified by repository name and any known fork\)\. To eliminate test\-time leakage, we restrict each repository to commits whose timestamp precedes the earliest base\-commit used in SWE\-Bench\-Verified and SWE\-Bench\-Lite\. The filtered corpus yields≈400​K\\approx\\\!400\\mathrm\{K\}FIM samples \(≈2\.6​B\\approx\\\!2\.6\\mathrm\{B\}tokens under the Qwen2\.5\-Coder tokenizer\):320​K320\\mathrm\{K\}single\-function,60​K60\\mathrm\{K\}pair, and20​K20\\mathrm\{K\}triple targets\. Full statistics, topic\-category breakdown, and license inventory are reported in Appendix[A](https://arxiv.org/html/2607.12463#A1)\.

### 2\.3Function\-Aware FIM Target Selection

Given each source file, our pipeline produces a set of mask targets together with the corresponding masked file\. Each target is a single function or a connected group of22–33functions; the multi\-function variant is studied in Section[3\.4](https://arxiv.org/html/2607.12463#S3.SS4)\. The pipeline has four stages: dependency\-graph construction, complexity scoring, inferability scoring, and threshold\-based selection\. Figure[2](https://arxiv.org/html/2607.12463#S2.F2)illustrates these stages on a small running example—aCalculatorclass with two top\-level helpers\. The full single\-function selection algorithm is given in Appendix[B\.1](https://arxiv.org/html/2607.12463#A2.SS1), the multi\-function extension in Appendix[B\.7](https://arxiv.org/html/2607.12463#A2.SS7), and a numerical walkthrough of every quantity shown in Figure[2](https://arxiv.org/html/2607.12463#S2.F2)\(b\) in Appendix[B\.6](https://arxiv.org/html/2607.12463#A2.SS6)\.

![Refer to caption](https://arxiv.org/html/2607.12463v1/x2.png)Figure 2:Function\-aware FIM target selection on a small calculator example\.\(a\)Program dependency graph parsed from the AST: solid arrows are call edgesℰcall\\mathcal\{E\}\_\{\\mathrm\{call\}\}, dashed lines are sibling edgesℰsib\\mathcal\{E\}\_\{\\mathrm\{sib\}\}between same\-class methods\.\(b\)Stacked bars decompose the complexity scoreH^=0\.40\\hat\{H\}\\\!=\\\!0\.40\(Eq\.[1](https://arxiv.org/html/2607.12463#S2.E1); LoC, CC, depth\) and the inferability scoreI^=0\.48\\hat\{I\}\\\!=\\\!0\.48\(Eq\.[2](https://arxiv.org/html/2607.12463#S2.E2); five context signals\) for𝙲𝚊𝚕𝚌𝚞𝚕𝚊𝚝𝚘𝚛\.𝚝𝚘𝚝𝚊𝚕\\mathtt\{Calculator\.total\}, yieldingFIM≈0\.22≥τ=0\.08\\mathrm\{FIM\}\\\!\\approx\\\!0\.22\\\!\\geq\\\!\\tau\\\!=\\\!0\.08\(a hyperparameter; see Appendix[B](https://arxiv.org/html/2607.12463#A2)for the full list\)\.#### 2\.3\.1Program Dependency Graph

For each file we parse its AST and extract the set𝒱\\mathcal\{V\}of function nodes \(top\-level functions and class methods, identified by qualified names\)\. Two edge sets are constructed:*call edges*ℰcall\\mathcal\{E\}\_\{\\mathrm\{call\}\}between caller and callee, and*sibling edges*ℰsib\\mathcal\{E\}\_\{\\mathrm\{sib\}\}between methods of the same class \(capturing intra\-class coupling that flows through shared instance state rather than direct calls\)\. Call resolution handles common Python idioms \(direct invocations, class instantiation,self/clsmethod calls\) with a short\-name fallback against the qualified\-name index; Appendix[B\.2](https://arxiv.org/html/2607.12463#A2.SS2)details the procedure\.

#### 2\.3\.2Complexity ScoreH^\\hat\{H\}

For eachv∈𝒱v\\in\\mathcal\{V\}we define

H^​\(v\)=wℓ​ϕ​\(LoC​\(v\),cℓ\)\+wc​ϕ​\(CC​\(v\),cc\)\+wd​ϕ​\(D​\(v\),cd\),\\hat\{H\}\(v\)\\;=\\;w\_\{\\ell\}\\,\\phi\\\!\\left\(\\mathrm\{LoC\}\(v\),\\,c\_\{\\ell\}\\right\)\+w\_\{c\}\\,\\phi\\\!\\left\(\\mathrm\{CC\}\(v\),\\,c\_\{c\}\\right\)\+w\_\{d\}\\,\\phi\\\!\\left\(\\mathrm\{D\}\(v\),\\,c\_\{d\}\\right\),\(1\)whereLoC​\(v\)\\mathrm\{LoC\}\(v\)is lines of code,CC​\(v\)\\mathrm\{CC\}\(v\)is McCabe cyclomatic complexity,D​\(v\)\\mathrm\{D\}\(v\)is the maximum nesting depth of control\-flow constructs, andϕ​\(x,c\)=min⁡\(x/c,2\)\\phi\(x,c\)\\\!=\\\!\\min\(x/c,\\,2\)normalizes each quantity by a soft cap\. Caps and weights are listed in Appendix[B\.3](https://arxiv.org/html/2607.12463#A2.SS3)\.

#### 2\.3\.3Inferability ScoreI^\\hat\{I\}

A target should be*recoverable*from the surrounding context\.I^​\(v\)\\hat\{I\}\(v\)aggregates five context\-derived signals that approximate the mutual information betweenvv’s body and the rest of the file:

I^​\(v\)=α​Ccaller​\(v\)\+β​Ccallee​\(v\)\+γ​Csig​\(v\)\+δ​Cdoc​\(v\)\+ε​Cclass​\(v\)\.\\hat\{I\}\(v\)\\;=\\;\\alpha\\,C\_\{\\mathrm\{caller\}\}\(v\)\+\\beta\\,C\_\{\\mathrm\{callee\}\}\(v\)\+\\gamma\\,C\_\{\\mathrm\{sig\}\}\(v\)\+\\delta\\,C\_\{\\mathrm\{doc\}\}\(v\)\+\\varepsilon\\,C\_\{\\mathrm\{class\}\}\(v\)\.\(2\)CcallerC\_\{\\mathrm\{caller\}\}scores call\-site argument specificity,CcalleeC\_\{\\mathrm\{callee\}\}counts intra\-file functions called byvv,CsigC\_\{\\mathrm\{sig\}\}aggregates type annotations and name descriptiveness,CdocC\_\{\\mathrm\{doc\}\}indicates docstring presence, andCclassC\_\{\\mathrm\{class\}\}counts in\-class siblings; full component formulas and weights are in Appendix[B\.4](https://arxiv.org/html/2607.12463#A2.SS4)\. Each component is a hand\-designed proxy: a learned predictability score would couple selection to a particular reference model and complicate the cross\-base\-model generalization analysis \(Section[3\.2](https://arxiv.org/html/2607.12463#S3.SS2)\)\.

#### 2\.3\.4Single\-Function Score

We combineH^\\hat\{H\}andI^\\hat\{I\}in a harmonic\-mean\-like form, scaled by a one\-sided difficulty penaltyρ​\(Δ​\(v\)\)\\rho\(\\Delta\(v\)\):

FIM​\(v\)=H^​\(v\)​I^​\(v\)H^​\(v\)\+I^​\(v\)\+ϵ⋅ρ​\(Δ​\(v\)\)\.\\mathrm\{FIM\}\(v\)\\;=\\;\\frac\{\\hat\{H\}\(v\)\\,\\hat\{I\}\(v\)\}\{\\hat\{H\}\(v\)\+\\hat\{I\}\(v\)\+\\epsilon\}\\cdot\\rho\\\!\\left\(\\Delta\(v\)\\right\)\.\(3\)The harmonic\-mean form forces bothH^\\hat\{H\}andI^\\hat\{I\}to be large simultaneously, penalizing imbalance;ρ\\rhodown\-weights targets that remain hard even given full context, which would otherwise be unlearnable noise\. Hard filters on length and dunder methods, the form ofρ\\rho, and the thresholdτ=0\.08\\tau\\\!=\\\!0\.08used throughout this work are documented in Appendix[B\.5](https://arxiv.org/html/2607.12463#A2.SS5)\.

#### 2\.3\.5Multi\-Function Group Selection

Real\-world code patches frequently span multiple related functions\(Jimenezet al\.,[2023](https://arxiv.org/html/2607.12463#bib.bib1)\), motivating an extension that masks*groups*ofk=2k\\\!=\\\!2ork=3k\\\!=\\\!3structurally connected functions; we use this variant in our main recipe and ablate it in Section[3\.4](https://arxiv.org/html/2607.12463#S3.SS4)\. A group scoreFIM​\(G\)\\mathrm\{FIM\}\(G\)multiplies a coupling term, the harmonic\-mean\-like product of group\-levelH^​\(G\)\\hat\{H\}\(G\)andI^​\(G\)\\hat\{I\}\(G\), and a difficulty penalty, withI^​\(G\)\\hat\{I\}\(G\)recomputed under joint masking so that intra\-group references cannot inflate the score\. The eight topology patterns \(caller\-callee, co\-callee, sibling\-coupled, mutual\-call, call\-chain, hub, fan\-in, class\-triad\), the full equations, Algorithm[2](https://arxiv.org/html/2607.12463#alg2), and a worked pair example are in Appendix[B\.7](https://arxiv.org/html/2607.12463#A2.SS7)\.

### 2\.4Chain\-of\-Thought Augmentation

For each selected target we run a three\-stage pipeline\.Generate\.Gemini\-3\-Flash sees only the masked file and produces a step\-by\-step rationale together with a candidate function body, with no access to the ground\-truth body \(Appendix[B\.9](https://arxiv.org/html/2607.12463#A2.SS9)\)\.Filter\.A separate Gemini\-3\-Flash judge scores the \(rationale, candidate body\) pair against the ground\-truth body on feasibility and five quality dimensions; we keep the top\-scoring∼400\\sim\\\!400K samples \(Appendix[B\.10](https://arxiv.org/html/2607.12463#A2.SS10)\)\.Format\.Each retained pair is placed inside the FIM middle span, rationale before body \(Appendix[B\.11](https://arxiv.org/html/2607.12463#A2.SS11)\):

<fim\_prefix\>⟨\\langleprefix⟩\\rangle<fim\_suffix\>⟨\\langlesuffix⟩\\rangle<fim\_middle\>⟨\\langlerationale⟩\\rangle⟨\\langlebody⟩\\rangle

The model is thus trained to produce reasoning followed by consistent code, mirroring the think\-then\-act structure of an agent step\. The ground\-truth body serves only as a filter anchor and never appears in the training target\. Section[3\.4](https://arxiv.org/html/2607.12463#S3.SS4)isolates the contribution of FIM structure from CoT distillation via a*self\-CoT*variant in which the model under training generates its own rationales\.

## 3Experiments

### 3\.1Setup

Benchmarks\.We evaluate on three groups\.*Coding\-agent benchmarks*: SWE\-Bench\-Verified \(500500\) and SWE\-Bench\-Lite \(300300\)\(Jimenezet al\.,[2023](https://arxiv.org/html/2607.12463#bib.bib1)\), our primary in\-domain target\.*Non\-agent coding benchmarks*: LiveCodeBench\(Jainet al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib15)\), OJBench\(Wanget al\.,[2025](https://arxiv.org/html/2607.12463#bib.bib16)\), and FullStackBench\-EN\(Chenget al\.,[2024b](https://arxiv.org/html/2607.12463#bib.bib17)\); these probe pure code generation and serve as a regression check\.*Tool\-use and OOD agent benchmarks*: Terminal\-Bench 2\.0\(Merrillet al\.,[2026](https://arxiv.org/html/2607.12463#bib.bib18)\),τ\\tau\-bench\(Yaoet al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib19)\), and BFCL\(Patilet al\.,[2025](https://arxiv.org/html/2607.12463#bib.bib20)\); the latter two contain no Python code\-editing trajectories and test whether the function\-call inductive bias transfers across task families\.

Mid\-training and post\-training pipeline\.We mid\-train Qwen2\.5\-Coder\-Instruct \(7B/14B\)\(Huiet al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib8)\)and Qwen3\-8B\(Yanget al\.,[2025a](https://arxiv.org/html/2607.12463#bib.bib13)\)on the selected FIM corpus using the standard FIM loss on the middle span \(rationale plus body\), packed to the model’s native context length and using its native FIM sentinel tokens\. We then apply existing agentic post\-training pipelines without modification: R2E\-Gym\(Jainet al\.,[2025](https://arxiv.org/html/2607.12463#bib.bib5)\)or SWE\-Smith\(Yanget al\.,[2025b](https://arxiv.org/html/2607.12463#bib.bib6)\)for the Qwen2\.5\-Coder runs, and SWE\-Lego\(Taoet al\.,[2026](https://arxiv.org/html/2607.12463#bib.bib7)\)for Qwen3\-8B\. For Qwen3\-8B we train SWE\-Lego for22epochs rather than the official44to prevent overfitting in our setup\. Hyperparameters and token budgets are in Appendix[C](https://arxiv.org/html/2607.12463#A3)\.

Evaluation protocol\.All numbers are means over three independent evaluation seeds on the final checkpoint of each pipeline, with “%” omitted in table cells and std bands in parentheses\.*Baseline*is base\+\+post\-training;*ours*is base\+\+FIM mid\-training\+\+identical post\-training\. We do not evaluate mid\-training\-only checkpoints because FIM\-only models have degraded instruction\-following and cannot be compared fairly with instruction\-tuned baselines—every reported gain therefore*survives*subsequent post\-training\. The agent harness is fixed by the post\-training pipeline: R2E\-Gym uses an OpenHands fork\(Wanget al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib3); Jainet al\.,[2025](https://arxiv.org/html/2607.12463#bib.bib5)\), SWE\-Smith uses SWE\-agent\(Yanget al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib2)\), and the Qwen3\-8B runs use OpenHands directly \(SWE\-Lego data exceeds the 32K context of the Qwen2\.5\-Coder models\)\.

Models and baselines\.For Qwen2\.5\-Coder\-Instruct at 7B/14B\(Huiet al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib8)\)we report \(i\) the instruction\-tuned base, \(ii\) post\-training with R2E\-Gym reproduced under our setup, \(iii\) the official R2E\-Gym numbers \(in grey, where available\), \(iv\) our recipe of FIM mid\-training followed by R2E\-Gym post\-training, and at 7B additionally the analogous SWE\-Smith\(Yanget al\.,[2025b](https://arxiv.org/html/2607.12463#bib.bib6)\)variants\. Qwen3\-8B uses the SWE\-Lego\(Taoet al\.,[2026](https://arxiv.org/html/2607.12463#bib.bib7)\)pipeline; the cross\-base\-model comparison thus varies the post\-training pipeline simultaneously, a confound we discuss in Section[3\.2](https://arxiv.org/html/2607.12463#S3.SS2)\.

### 3\.2Main Results on SWE Agent Benchmarks

Table 1:Main results on coding agent benchmarks\. All numbers are percentages, means over three independent seeds with std bands;*Average*is the unweighted mean of Verified and Lite\. Bolded rows are our recipe; rows tagged*officially reported*are quoted from the corresponding original publications\. SWE\-Lego is post\-trained for22epochs \(vs\.44in the official release\) to prevent overfitting\.Table[1](https://arxiv.org/html/2607.12463#S3.T1)summarizes in\-domain agent results\.

Consistent gains on the Qwen2\.5\-Coder\-Instruct series\.Holding the post\-training pipeline fixed at R2E\-Gym, FIM mid\-training improves SWE\-Bench\-Verified by\+2\.80\+2\.80on 7B\-Instruct and\+3\.00\+3\.00on 14B\-Instruct, with matching directional gains on SWE\-Bench\-Lite \(\+3\.67/\+4\.00\+3\.67/\+4\.00\)\. Both Qwen2\.5\-Coder\-Instruct sizes benefit from the same mid\-training corpus and recipe, indicating that the structural prior is not absorbed by the larger pretrained checkpoint within this family\.

Transfer across post\-training pipelines\.Replacing R2E\-Gym with SWE\-Smith on the same 7B base yields\+5\.30\+5\.30points on Verified, larger than the\+2\.80\+2\.80under R2E\-Gym; on Lite the SWE\-Smith pairing gains only\+0\.50\+0\.50, smaller than\+3\.67\+3\.67for the R2E\-Gym pairing\. The two pipelines together indicate that mid\-training is not tuned to a single post\-training data distribution, though the magnitude of its benefit depends on the baseline pipeline it composes with\.

Transfer to a non\-Qwen2\.5 base\.Switching to Qwen3\-8B paired with SWE\-Lego, mid\-training improves Verified by\+3\.20\+3\.20and Lite by\+5\.40\+5\.40, comparable to the Qwen2\.5\-Coder\-Instruct gains\. This single comparison varies the post\-training pipeline jointly with the base model, so the result should be read as “not specific to the Qwen2\.5\-Coder\-Instruct\+\+R2E\-Gym/SWE\-Smith pairing” rather than as a guarantee across base\-model families\.

### 3\.3Capability Preservation and Cross\-Domain Transfer

A natural concern with task\-specialized post\-training is that it erodes capabilities the base model already had\. We further evaluate the 14B model on six additional benchmarks to assess capability preservation\. To save compute, we restrict this controlled comparison \(instruct base vs\. post\-training only vs\. mid\-training\+\+post\-training\) to R2E\-Gym as the post\-training dataset \(Table[2](https://arxiv.org/html/2607.12463#S3.T2)\)\.

Table 2:Capability preservation and cross\-domain transfer at 14B with R2E\-Gym\. Bold marks the better trained variant per column \(post\-only vs\. ours\); the Instruct row is shown as a reference ceiling\. All cells are percentages\. “Terminal” denotes Terminal\-Bench 2\.0\.Agentic post\-training has a substantial hidden capability cost\.R2E\-Gym alone reduces every non\-agent and tool\-use benchmark relative to the Instruct ceiling: LiveCodeBench by13\.1013\.10, BFCL by7\.407\.40, FullStackBench\-EN by6\.086\.08,τ\\tau\-bench by2\.302\.30, and OJBench by2\.402\.40\. Averaged across the six benchmarks the post\-trained model loses4\.814\.81points relative to Instruct—the implicit cost paid for SWE\-Bench gains, rarely highlighted in agent papers\.

Mid\-training largely closes the regression gap\.Adding FIM mid\-training before the same post\-training restores LiveCodeBench by\+11\.10\+11\.10, OJBench by\+1\.94\+1\.94\(within0\.460\.46of the Instruct ceiling\), and FullStackBench\-EN by\+0\.53\+0\.53\. The six\-benchmark average rises from16\.0416\.04to19\.5619\.56\(\+3\.52\+3\.52over post\-training only\), while the SWE\-Bench\-Verified gain on the same base is preserved \(Table[1](https://arxiv.org/html/2607.12463#S3.T1)\)\. Mid\-training therefore improves the cost–benefit profile of agentic post\-training: in\-domain target metrics improve and the bulk of off\-distribution erosion is undone in the same training run\.

Cross\-domain transfer to non\-coding tool use\.τ\\tau\-bench and BFCL contain no Python code\-editing data, and our mid\-training corpus contains no tool\-use trajectories\. Mid\-training nevertheless improvesτ\\tau\-bench by\+3\.90\+3\.90and BFCL by\+2\.40\+2\.40over post\-training alone, with a consistent recovery on Terminal\-Bench 2\.0 \(\+1\.25\+1\.25\)\. Because the corpus carries no tool\-use signal, the only mechanism is a structural prior installed at mid\-training that survives post\-training, which is the direct evidence for the function\-call/tool\-call isomorphism\(Section[2\.1](https://arxiv.org/html/2607.12463#S2.SS1)\)\.

### 3\.4Ablation Studies

We run three controlled ablations on Qwen2\.5\-Coder\-7B\-Instruct with R2E\-Gym post\-training and a shared mid\-training\-free baseline\. The first isolates the role of the chain\-of\-thought rationale; the second isolates the role of the function\-aware selection pipeline; the third varies mask granularity \(single\-function vs\. multi\-function groups\)\. Due to compute constraints, ablations run at 7B only\.

Table 3:Ablations on the 7B model with R2E\-Gym post\-training\.\(A\)rationale source\.\(B\)function\-selection algorithm\.\(C\)mask granularity \(single vs\. multi\-function groups\)\. All blocks share the baseline \(*w/o mid\-train*\) and a controlled200200K\-target budget; CoT is fixed to Gemini\-3\-Flash in \(B\) and \(C\)\. Bold marks the best configuration per block; the80%/15%/5%80\\%/15\\%/5\\%mixture in \(C\) is the recipe used in our main results \(Table[1](https://arxiv.org/html/2607.12463#S3.T1)\), which is trained on the full corpus rather than the200200K budget here\. Absolute numbers across all blocks therefore lie below the main\-table recipe; the relative orderings within each block, rather than the absolute values, are the object of comparison\.FIM structure contributes independently of CoT distillation\.Block \(A\) addresses the concern that gains stem mainly from distilling a frontier teacher\. Removing the rationale entirely \(*no CoT*\) already lifts the average by\+1\.18\+1\.18—roughly half of the\+2\.43\+2\.43\-point Gemini\-3 gain—direct evidence that the function\-aware FIM signal does substantive work before any reasoning supervision is added\. Replacing Gemini\-3 with rationales from the model under training \(*self\-CoT*\) reaches14\.8514\.85, recovering1\.681\.68of the2\.432\.43\-point gap; the residual0\.750\.75points attributable to a frontier teacher are real but modest\. The recipe is therefore not a thinly disguised distillation pipeline\.

Function\-selection algorithm is the dominant lever\.Block \(B\) varies the selection algorithm with CoT and budget held fixed\.*Random*masking sets a floor at13\.9513\.95, and*Gemini\-selected*reaches15\.0515\.05: frontier judgment on*which*function to mask helps but is not sufficient\. Restricting candidates to functions with at least one PDG neighbor \(*PDG only*\) reaches14\.8514\.85; addingH^\\hat\{H\}orI^\\hat\{I\}on top yields15\.0515\.05and15\.3515\.35—intrinsic difficulty and contextual recoverability each contribute beyond the structural filter\. Combining them \(*Full*\) reaches15\.6015\.60, confirming thatH^\\hat\{H\}andI^\\hat\{I\}are not redundant\.

Mask granularity: pair masking helps, triples saturate\.Block \(C\) mixes single\-function targets with multi\-function groups \(Section[2\.3\.5](https://arxiv.org/html/2607.12463#S2.SS3.SSS5)\)\. Adding15%15\\%pair targets raises the average from15\.6015\.60to15\.9015\.90; substituting5%5\\%triple targets instead is essentially neutral \(15\.7015\.70\)\. The full80%/15%/5%80\\%/15\\%/5\\%mix achieves16\.1016\.10\. This matches the analysis in Section[4\.2](https://arxiv.org/html/2607.12463#S4.SS2): training on cross\-function dependencies disproportionately helps tasks whose gold patches span multiple functions, but the marginal return from larger groups diminishes as coupling becomes harder to maintain under joint masking\.

## 4Analysis

The previous section established that FIM mid\-training improves end\-task metrics\. We now ask*how*the resulting agent behaves differently and*where*along the trajectory the gains accrue\. Both analyses use the 14B configuration with R2E\-Gym, comparing the post\-training\-only baseline \(*R2E\-Gym*\) against our recipe \(*FIM\-Midtrain\+\+R2E\-Gym*\)\. The Lite analogue, the full action\-type distribution, the no\-patch mechanism, and a concrete trajectory contrast are deferred to Appendix[D](https://arxiv.org/html/2607.12463#A4)\.

### 4\.1Recovery from Negative Observations

A trajectory contains a*negative observation*if any tool output matches a fixed set of error patterns \(stack traces, “No replacement was performed”, shell errors, etc\.; full list in Appendix[B](https://arxiv.org/html/2607.12463#A2)\)\. The fraction of such trajectories is essentially identical across checkpoints \(88\.8%88\.8\\%baseline vs\.91\.8%91\.8\\%ours\), so the agents see comparable amounts of negative feedback\. The*recovery rate*—the fraction of error\-containing trajectories that nonetheless terminate with a passing patch—rises from24\.8%24\.8\\%to28\.8%28\.8\\%\(\+4\.0\+4\.0pp;\+16%\+16\\%relative; Table[4](https://arxiv.org/html/2607.12463#S4.T4)\)\. This is precisely the capability our framing predicts mid\-training should support: continuing productively after an external return contradicts the model’s prior expectation\. Mid\-training also shifts the agent toward an iterate\-and\-verify policy, increasing edits per solved task from3\.33\.3to7\.47\.4and trajectory length from15\.115\.1to23\.623\.6steps; action\-type breakdowns are in Appendix[D](https://arxiv.org/html/2607.12463#A4)\.

Table 4:Headline trajectory metrics on SWE\-Bench\-Verified\. Full metrics in Appendix[D](https://arxiv.org/html/2607.12463#A4)\.
### 4\.2The Gain Concentrates on Multi\-Function Reasoning

The most direct test of the isomorphism argument \(Section[2\.1](https://arxiv.org/html/2607.12463#S2.SS1)\) is whether mid\-training preferentially helps tasks that require reasoning about cross\-function dependencies inside a file\. We stratify the500500Verified tasks by the shape of the gold patch \(Figure[5](https://arxiv.org/html/2607.12463#A4.F5), Appendix[D](https://arxiv.org/html/2607.12463#A4)\)\. On the8888tasks whose gold patch modifies≥2\\geq 2functions within a single file, the baseline solves13\.6%13\.6\\%on average and ours solves22\.7%22\.7\\%, an absolute gain of\+9\.1\+9\.1pp—more than4×4\{\\times\}the gain on the341341single\-function tasks \(\+2\.1\+2\.1pp\)\. Per\-instance head\-to\-head on this bucket shows ours uniquely solves about twice as many tasks as the baseline\. Multi\-file tasks \(n=71n\{=\}71\) are not differentially helped, which we attribute to a granularity mismatch: our FIM operates within files \(Appendix[D](https://arxiv.org/html/2607.12463#A4)\)\. The slice where mid\-training helps most is precisely the slice where the agent must follow control\- and data\-dependencies between caller\-callee or sibling functions inside a file—the same structure exposed by function\-aware FIM masking\. A complementary outcome\-distribution breakdown \(Figure[6](https://arxiv.org/html/2607.12463#A4.F6)\) shows the\+15\+15\-task gain is driven by an order\-of\-magnitude reduction in*no\-patch*failures alongside small reductions in localization errors; the FIM signal conditions the model to produce a non\-empty span between prefix and suffix—a disposition that survives subsequent post\-training\.

## 5Related Work

Mid\-training and continued pretraining\.A growing body of work\(Tuet al\.,[2025](https://arxiv.org/html/2607.12463#bib.bib27); Gururanganet al\.,[2020](https://arxiv.org/html/2607.12463#bib.bib28)\)identifies a stage between pretraining and post\-training in which a model is further trained on a curated corpus to install inductive biases hard to acquire from generic web text or fine\-tuning\. MiniCPM\(Huet al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib21)\), OLMo\(Groeneveldet al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib22)\), and DeepSeek\-V3\(Liuet al\.,[2024a](https://arxiv.org/html/2607.12463#bib.bib29)\)schedule such a stage near the end of pretraining; Code\-Llama\(Roziereet al\.,[2023](https://arxiv.org/html/2607.12463#bib.bib12)\)follows the same staging philosophy for context\-length generalization\. Recent work shows*when*specialized data is introduced matters as much as how much is used\(Huanget al\.,[2026](https://arxiv.org/html/2607.12463#bib.bib31); Akteret al\.,[2025](https://arxiv.org/html/2607.12463#bib.bib30)\)\. We adopt this philosophy but target an*agent\-oriented*structural prior through function\-aware FIM\.

Fill\-in\-the\-middle and structure\-aware code objectives\.FIM\(Bavarianet al\.,[2022](https://arxiv.org/html/2607.12463#bib.bib14); Friedet al\.,[2022](https://arxiv.org/html/2607.12463#bib.bib32)\)is a defining ingredient of modern code LLM pretraining, used in Code\-Llama\(Roziereet al\.,[2023](https://arxiv.org/html/2607.12463#bib.bib12)\), StarCoder\-2\(Lozhkovet al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib11)\), CodeGen\(Nijkampet al\.,[2022](https://arxiv.org/html/2607.12463#bib.bib34)\), Qwen\-Coder\(Huiet al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib8)\), and DeepSeek\-Coder\(Guoet al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib9)\)\. A recent line moves beyond random spans toward*structure\-aware*masking: AST\-T5\(Gonget al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib35)\)and AST\-FIM\(Gonget al\.,[2025](https://arxiv.org/html/2607.12463#bib.bib36)\)mask AST subtrees, the latter reporting up to\+5\+5pts on real\-edit FIM benchmarks; Horizon\-Length Prediction\(Dinget al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib37)\)adds a planning signal; Instruction\-aware FIM\(Sunet al\.,[2025](https://arxiv.org/html/2607.12463#bib.bib38)\)extends the FIM tuple with a developer\-comment slot\. Repository\-level retrieval methods such as GraphCoder\(Liuet al\.,[2024b](https://arxiv.org/html/2607.12463#bib.bib39)\)and DRACO\(Chenget al\.,[2024a](https://arxiv.org/html/2607.12463#bib.bib40)\)exploit program dependencies, but at inference time\. Our recipe differs along three axes: targets are chosen at*function granularity*via PDG analysis with a complexity–inferability criterion, so the masked region is a unit of agent\-relevant reasoning rather than a syntactic subtree; an explicit*chain\-of\-thought*sits inside the FIM middle; and the objective lives in a*dedicated mid\-training stage*\.

Coding agent foundation models\.Progress has advanced on three fronts\. Benchmarks have moved from SWE\-Bench\(Jimenezet al\.,[2023](https://arxiv.org/html/2607.12463#bib.bib1)\)to broader, contamination\-resistant suites including Multi\-SWE\-Bench\(Zanet al\.,[2025](https://arxiv.org/html/2607.12463#bib.bib41)\)and SWE\-Bench\-Pro\(Denget al\.,[2025](https://arxiv.org/html/2607.12463#bib.bib42)\)\. Scaffolds such as SWE\-agent\(Yanget al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib2)\), OpenHands\(Wanget al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib3)\), and Agentless\(Xiaet al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib43)\)expose different action interfaces\. Trajectory\-centric post\-training pipelines—SWE\-Gym\(Panet al\.,[2024](https://arxiv.org/html/2607.12463#bib.bib4)\), R2E\-Gym\(Jainet al\.,[2025](https://arxiv.org/html/2607.12463#bib.bib5)\), SWE\-Smith\(Yanget al\.,[2025b](https://arxiv.org/html/2607.12463#bib.bib6)\), SWE\-Lego\(Taoet al\.,[2026](https://arxiv.org/html/2607.12463#bib.bib7)\), and Skywork\-SWE\(Zenget al\.,[2025](https://arxiv.org/html/2607.12463#bib.bib44)\)—curate agent trajectories on top of these scaffolds, with recent extensions replacing or augmenting SFT with RL\(Golubevet al\.,[2025](https://arxiv.org/html/2607.12463#bib.bib45); Weiet al\.,[2025](https://arxiv.org/html/2607.12463#bib.bib46)\)\. We use R2E\-Gym, SWE\-Smith, and SWE\-Lego unmodified atop a mid\-trained checkpoint; the novelty lies one stage earlier—a self\-supervised signal extracted from structure already present in source code, which mitigates the off\-domain capability erosion trajectory\-only post\-training inflicts\.

Distillation from frontier models\.Our recipe uses Gemini\-3\-Flash to generate the CoT rationales embedded in each FIM middle, placing it within distillation from a strong teacher\(Mukherjeeet al\.,[2023](https://arxiv.org/html/2607.12463#bib.bib23); Gandhiet al\.,[2023](https://arxiv.org/html/2607.12463#bib.bib25); Weiet al\.,[2023](https://arxiv.org/html/2607.12463#bib.bib47)\)and CoT distillation\(Hoet al\.,[2023](https://arxiv.org/html/2607.12463#bib.bib48); Hsiehet al\.,[2023](https://arxiv.org/html/2607.12463#bib.bib49); Chenet al\.,[2025](https://arxiv.org/html/2607.12463#bib.bib50)\)\. Recent work shows diversity and structural alignment of the rationale matter as much as teacher strength\(Chenet al\.,[2025](https://arxiv.org/html/2607.12463#bib.bib50)\); consistent with this, our CoT\-source ablation \(Section[3\.4](https://arxiv.org/html/2607.12463#S3.SS4)\) confirms the recipe is not distillation\-bound\.

## 6Limitations and Discussion

We close by stating four limitations that scope our claims\. \(i\)Python\-only corpus and evaluation\.The mid\-training corpus and the in\-domain agent benchmarks are exclusively Python; cross\-language evidence comes only indirectly through FullStackBench\-EN \(Section[3\.3](https://arxiv.org/html/2607.12463#S3.SS3)\), and transfer to Java, C\+\+, or Rust is left to future work\. \(ii\)Teacher dependency for CoT\.The default recipe relies on Gemini\-3\-Flash; the CoT\-source ablation \(Section[3\.4](https://arxiv.org/html/2607.12463#S3.SS4)\) shows self\-generated rationales recover most of the gain, but a fully open\-source replication requires a comparably strong open teacher\. \(iii\)Partial cross\-base validation\.Our cross\-base evidence comes from a single non\-Qwen2\.5\-Coder configuration \(Qwen3\-8B with SWE\-Lego\), which simultaneously varies the post\-training pipeline; the result indicates the recipe is not tied to one pretraining/post\-training combination rather than guaranteeing transfer across all base families\. \(iv\)Modularity assumption\.Function\-aware FIM presupposes modular code; on monolithic scripts, generated code, or notebooks, the pipeline yields fewer eligible targets, a regime we do not study systematically\.

## 7Conclusion

A single step of a coding agent and a single function call site share the same four\-part structure—context, action, externally produced return, continuation—making source code an internet\-scale supply of agent\-relevant signal\. We turn this into function\-aware FIM mid\-training: a self\-supervised stage that masks targets selected via program dependency graph analysis and a complexity–inferability double criterion, with chain\-of\-thought rationales embedded inside the FIM middle\. Across model size, post\-training pipeline, and base\-model family, the recipe delivers consistent gains on coding\-agent benchmarks, and the same Python\-only corpus transfers to non\-coding tool\-use benchmarks \(τ\\tau\-bench, BFCL\)\. Future work includes extending the selection to non\-Python languages and composing mid\-training with RL post\-training\.

## References

- Front\-loading reasoning: the synergy between pretraining and post\-training data\.arXiv preprint arXiv:2510\.03264\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p1.1)\.
- M\. Bavarian, H\. Jun, N\. Tezak, J\. Schulman, C\. McLeavey, J\. Tworek, and M\. Chen \(2022\)Efficient training of language models to fill in the middle\.Cited by:[§1](https://arxiv.org/html/2607.12463#S1.p3.1),[§5](https://arxiv.org/html/2607.12463#S5.p2.1)\.
- X\. Chen, Z\. Sun, G\. Wenjin, M\. Zhang, Y\. Chen, Y\. Sun, H\. Su, Y\. Pan, D\. Klakow, W\. Li,et al\.\(2025\)Unveiling the key factors for distilling chain\-of\-thought reasoning\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 15094–15119\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p4.1)\.
- W\. Cheng, Y\. Wu, and W\. Hu \(2024a\)Dataflow\-guided retrieval augmentation for repository\-level code completion\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 7957–7977\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p2.1)\.
- Y\. Cheng, J\. Chen, J\. Chen, L\. Chen, L\. Chen, W\. Chen, Z\. Chen, S\. Geng, A\. Li, B\. Li,et al\.\(2024b\)Fullstack bench: evaluating llms as full stack coders\.Cited by:[§3\.1](https://arxiv.org/html/2607.12463#S3.SS1.p1.3)\.
- X\. Deng, J\. Da, E\. Pan, Y\. Y\. He, C\. Ide, K\. Garg, N\. Lauffer, A\. Park, N\. Pasari, C\. Rane,et al\.\(2025\)Swe\-bench pro: can ai agents solve long\-horizon software engineering tasks?\.arXiv preprint arXiv:2509\.16941\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p3.1)\.
- Y\. Ding, H\. Ding, S\. Wang, Q\. Sun, V\. Kumar, and Z\. Wang \(2024\)Horizon\-length prediction: advancing fill\-in\-the\-middle capabilities for code generation with lookahead planning\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p2.1)\.
- D\. Fried, A\. Aghajanyan, J\. Lin, S\. Wang, E\. Wallace, F\. Shi, R\. Zhong, W\. Yih, L\. Zettlemoyer, and M\. Lewis \(2022\)Incoder: a generative model for code infilling and synthesis\.arXiv preprint arXiv:2204\.05999\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p2.1)\.
- S\. Gandhi, P\. Von Platen, and A\. M\. Rush \(2023\)Distil\-whisper: robust knowledge distillation via large\-scale pseudo labelling\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p4.1)\.
- A\. Golubev, M\. Trofimova, S\. Polezhaev, I\. Badertdinov, M\. Nekrashevich, A\. Shevtsov, S\. Karasik, S\. Abramov, A\. Andriushchenko, F\. Fisin,et al\.\(2025\)Training long\-context, multi\-turn software engineering agents with reinforcement learning\.arXiv preprint arXiv:2508\.03501\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p3.1)\.
- L\. Gong, A\. Cheung, M\. Elhoushi, and S\. Wang \(2025\)Structure\-aware fill\-in\-the\-middle pretraining for code\.arXiv preprint arXiv:2506\.00204\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p2.1)\.
- L\. Gong, M\. Elhoushi, and A\. Cheung \(2024\)Ast\-t5: structure\-aware pretraining for code generation and understanding\.arXiv preprint arXiv:2401\.03003\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p2.1)\.
- D\. Groeneveld, I\. Beltagy, E\. Walsh, A\. Bhagia, R\. Kinney, O\. Tafjord, A\. Jha, H\. Ivison, I\. Magnusson, Y\. Wang,et al\.\(2024\)OLMo: accelerating the science of language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 15789–15809\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p1.1)\.
- D\. Guo, Q\. Zhu, D\. Yang, Z\. Xie, K\. Dong, W\. Zhang, G\. Chen, X\. Bi, Y\. Wu, Y\. Li,et al\.\(2024\)DeepSeek\-Coder: when the large language model meets programming – the rise of code intelligence\.Cited by:[§1](https://arxiv.org/html/2607.12463#S1.p1.1),[§1](https://arxiv.org/html/2607.12463#S1.p3.1),[§2\.1](https://arxiv.org/html/2607.12463#S2.SS1.p1.3),[§5](https://arxiv.org/html/2607.12463#S5.p2.1)\.
- S\. Gururangan, A\. Marasović, S\. Swayamdipta, K\. Lo, I\. Beltagy, D\. Downey, and N\. A\. Smith \(2020\)Don’t stop pretraining: adapt language models to domains and tasks\.InProceedings of the 58th annual meeting of the association for computational linguistics,pp\. 8342–8360\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p1.1)\.
- N\. Ho, L\. Schmid, and S\. Yun \(2023\)Large language models are reasoning teachers\.InProceedings of the 61st annual meeting of the association for computational linguistics \(volume 1: long papers\),pp\. 14852–14882\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p4.1)\.
- C\. Hsieh, C\. Li, C\. Yeh, H\. Nakhost, Y\. Fujii, A\. Ratner, R\. Krishna, C\. Lee, and T\. Pfister \(2023\)Distilling step\-by\-step\! outperforming larger language models with less training data and smaller model sizes\.InFindings of the Association for Computational Linguistics: ACL 2023,pp\. 8003–8017\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p4.1)\.
- S\. Hu, Y\. Tu, X\. Han, C\. He, G\. Cui, X\. Long, Z\. Zheng, Y\. Fang, Y\. Huang, W\. Zhao,et al\.\(2024\)Minicpm: unveiling the potential of small language models with scalable training strategies\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p1.1)\.
- J\. Huang, J\. Qin, D\. Yin, W\. Liu, Y\. Yu, X\. Sun, and W\. Zhang \(2026\)ReMiT: rl\-guided mid\-training for iterative llm evolution\.arXiv preprint arXiv:2602\.03075\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p1.1)\.
- B\. Hui, J\. Yang, Z\. Cui, J\. Yang, D\. Liu, L\. Zhang, T\. Liu, J\. Zhang, B\. Yu, K\. Lu,et al\.\(2024\)Qwen2\.5\-Coder technical report\.Cited by:[§1](https://arxiv.org/html/2607.12463#S1.p1.1),[§1](https://arxiv.org/html/2607.12463#S1.p3.1),[§2\.1](https://arxiv.org/html/2607.12463#S2.SS1.p1.3),[§3\.1](https://arxiv.org/html/2607.12463#S3.SS1.p2.2),[§3\.1](https://arxiv.org/html/2607.12463#S3.SS1.p4.1),[§5](https://arxiv.org/html/2607.12463#S5.p2.1)\.
- N\. Jain, K\. Han, A\. Gu, W\. Li, F\. Yan, T\. Zhang, S\. Wang, A\. Solar\-Lezama, K\. Sen, and I\. Stoica \(2024\)Livecodebench: holistic and contamination free evaluation of large language models for code\.Cited by:[§3\.1](https://arxiv.org/html/2607.12463#S3.SS1.p1.3)\.
- N\. Jain, J\. Singh, M\. Shetty, L\. Zheng, K\. Sen, and I\. Stoica \(2025\)R2E\-Gym: procedural environments and hybrid verifiers for scaling open\-weights SWE agents\.Cited by:[§1](https://arxiv.org/html/2607.12463#S1.p1.1),[§3\.1](https://arxiv.org/html/2607.12463#S3.SS1.p2.2),[§3\.1](https://arxiv.org/html/2607.12463#S3.SS1.p3.3),[Table 1](https://arxiv.org/html/2607.12463#S3.T1.8.4.3),[§5](https://arxiv.org/html/2607.12463#S5.p3.1)\.
- C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. R\. Narasimhan \(2023\)SWE\-bench: can language models resolve real\-world GitHub issues?\.InThe twelfth international conference on learning representations,Cited by:[§1](https://arxiv.org/html/2607.12463#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.12463#S2.SS2.p1.7),[§2\.3\.5](https://arxiv.org/html/2607.12463#S2.SS3.SSS5.p1.6),[§3\.1](https://arxiv.org/html/2607.12463#S3.SS1.p1.3),[§5](https://arxiv.org/html/2607.12463#S5.p3.1)\.
- R\. Li, L\. B\. Allal, Y\. Zi, N\. Muennighoff, D\. Kocetkov, C\. Mou, M\. Marone, C\. Akiki, J\. Li, J\. Chim,et al\.\(2023\)StarCoder: may the source be with you\!\.Cited by:[§1](https://arxiv.org/html/2607.12463#S1.p1.1),[§1](https://arxiv.org/html/2607.12463#S1.p3.1),[§2\.1](https://arxiv.org/html/2607.12463#S2.SS1.p1.3)\.
- A\. Liu, B\. Feng, B\. Xue, B\. Wang, B\. Wu, C\. Lu, C\. Zhao, C\. Deng, C\. Zhang, C\. Ruan,et al\.\(2024a\)Deepseek\-v3 technical report\.arXiv preprint arXiv:2412\.19437\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p1.1)\.
- W\. Liu, A\. Yu, D\. Zan, B\. Shen, W\. Zhang, H\. Zhao, Z\. Jin, and Q\. Wang \(2024b\)Graphcoder: enhancing repository\-level code completion via code context graph\-based retrieval and language model\.arXiv preprint arXiv:2406\.07003\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p2.1)\.
- A\. Lozhkov, R\. Li, L\. B\. Allal, F\. Cassano, J\. Lamy\-Poirier, N\. Tazi, A\. Tang, D\. Pykhtar, J\. Liu, Y\. Wei,et al\.\(2024\)Starcoder 2 and the stack v2: the next generation\.arXiv preprint arXiv:2402\.19173\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p2.1)\.
- M\. A\. Merrill, A\. G\. Shaw, N\. Carlini, B\. Li, H\. Raj, I\. Bercovich, L\. Shi, J\. Y\. Shin, T\. Walshe, E\. K\. Buchanan,et al\.\(2026\)Terminal\-bench: benchmarking agents on hard, realistic tasks in command line interfaces\.Cited by:[§3\.1](https://arxiv.org/html/2607.12463#S3.SS1.p1.3)\.
- S\. Mukherjee, A\. Mitra, G\. Jawahar, S\. Agarwal, H\. Palangi, and A\. Awadallah \(2023\)Orca: progressive learning from complex explanation traces of gpt\-4\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p4.1)\.
- E\. Nijkamp, B\. Pang, H\. Hayashi, L\. Tu, H\. Wang, Y\. Zhou, S\. Savarese, and C\. Xiong \(2022\)Codegen: an open large language model for code with multi\-turn program synthesis\.arXiv preprint arXiv:2203\.13474\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p2.1)\.
- J\. Pan, X\. Wang, G\. Neubig, N\. Jaitly, H\. Ji, A\. Suhr, and Y\. Zhang \(2024\)Training software engineering agents and verifiers with SWE\-Gym\.Cited by:[§1](https://arxiv.org/html/2607.12463#S1.p1.1),[§5](https://arxiv.org/html/2607.12463#S5.p3.1)\.
- S\. G\. Patil, H\. Mao, F\. Yan, C\. C\. Ji, V\. Suresh, I\. Stoica, and J\. E\. Gonzalez \(2025\)The berkeley function calling leaderboard \(bfcl\): from tool use to agentic evaluation of large language models\.InForty\-second International Conference on Machine Learning,Cited by:[§3\.1](https://arxiv.org/html/2607.12463#S3.SS1.p1.3)\.
- B\. Roziere, J\. Gehring, F\. Gloeckle, S\. Sootla, I\. Gat, X\. E\. Tan, Y\. Adi, J\. Liu, R\. Sauvestre, T\. Remez,et al\.\(2023\)Code Llama: open foundation models for code\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p1.1),[§5](https://arxiv.org/html/2607.12463#S5.p2.1)\.
- Z\. Sun, C\. Yang, C\. Peng, P\. Gao, X\. Du, L\. Li, and D\. Lo \(2025\)Bridging developer instructions and code completion through instruction\-aware fill\-in\-the\-middle paradigm\.arXiv preprint arXiv:2509\.24637\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p2.1)\.
- C\. Tao, J\. Chen, Y\. Jiang, K\. Kou, S\. Wang, R\. Wang, X\. Li, S\. Yang, Y\. Du, J\. Dai,et al\.\(2026\)SWE\-Lego: pushing the limits of supervised fine\-tuning for software issue resolving\.Cited by:[§3\.1](https://arxiv.org/html/2607.12463#S3.SS1.p2.2),[§3\.1](https://arxiv.org/html/2607.12463#S3.SS1.p4.1),[Table 1](https://arxiv.org/html/2607.12463#S3.T1.40.36.3),[§5](https://arxiv.org/html/2607.12463#S5.p3.1)\.
- C\. Tu, X\. Zhang, R\. Weng, R\. Li, C\. Zhang, Y\. Bai, H\. Yan, J\. Wang, and X\. Cai \(2025\)A survey on llm mid\-training\.arXiv preprint arXiv:2510\.23081\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p1.1)\.
- X\. Wang, B\. Li, Y\. Song, F\. F\. Xu, X\. Tang, M\. Zhuge, J\. Pan, Y\. Song, B\. Li, J\. Singh,et al\.\(2024\)OpenHands: an open platform for AI software developers as generalist agents\.Cited by:[§1](https://arxiv.org/html/2607.12463#S1.p1.1),[§3\.1](https://arxiv.org/html/2607.12463#S3.SS1.p3.3),[§5](https://arxiv.org/html/2607.12463#S5.p3.1)\.
- Z\. Wang, Y\. Liu, Y\. Wang, W\. He, B\. Gao, M\. Diao, Y\. Chen, K\. Fu, F\. Sung, Z\. Yang,et al\.\(2025\)Ojbench: a competition level code benchmark for large language models\.Cited by:[§3\.1](https://arxiv.org/html/2607.12463#S3.SS1.p1.3)\.
- Y\. Wei, Z\. Sun, E\. McMilin, J\. Gehring, D\. Zhang, G\. Synnaeve, D\. Fried, L\. Zhang, and S\. Wang \(2025\)Toward training superintelligent software agents through self\-play swe\-rl\.arXiv preprint arXiv:2512\.18552\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p3.1)\.
- Y\. Wei, Z\. Wang, J\. Liu, Y\. Ding, and L\. Zhang \(2023\)Magicoder: empowering code generation with oss\-instruct\.arXiv preprint arXiv:2312\.02120\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p4.1)\.
- C\. S\. Xia, Y\. Deng, S\. Dunn, and L\. Zhang \(2024\)Agentless: demystifying llm\-based software engineering agents\.arXiv preprint arXiv:2407\.01489\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p3.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025a\)Qwen3 technical report\.Cited by:[§3\.1](https://arxiv.org/html/2607.12463#S3.SS1.p2.2)\.
- J\. Yang, C\. E\. Jimenez, A\. Wettig, K\. Lieret, S\. Yao, K\. Narasimhan, and O\. Press \(2024\)SWE\-agent: agent–computer interfaces enable automated software engineering\.Vol\.37,pp\. 50528–50652\.Cited by:[§1](https://arxiv.org/html/2607.12463#S1.p1.1),[§3\.1](https://arxiv.org/html/2607.12463#S3.SS1.p3.3),[§5](https://arxiv.org/html/2607.12463#S5.p3.1)\.
- J\. Yang, K\. Lieret, C\. E\. Jimenez, A\. Wettig, K\. Khandpur, Y\. Zhang, B\. Hui, O\. Press, L\. Schmidt, and D\. Yang \(2025b\)SWE\-smith: scaling data for software engineering agents\.Cited by:[§1](https://arxiv.org/html/2607.12463#S1.p1.1),[§3\.1](https://arxiv.org/html/2607.12463#S3.SS1.p2.2),[§3\.1](https://arxiv.org/html/2607.12463#S3.SS1.p4.1),[Table 1](https://arxiv.org/html/2607.12463#S3.T1.18.14.3),[§5](https://arxiv.org/html/2607.12463#S5.p3.1)\.
- S\. Yao, N\. Shinn, P\. Razavi, and K\. Narasimhan \(2024\)τ\\tau\-Bench: a benchmark for tool\-agent\-user interaction in real\-world domains\.Cited by:[§3\.1](https://arxiv.org/html/2607.12463#S3.SS1.p1.3)\.
- D\. Zan, Z\. Huang, W\. Liu, H\. Chen, L\. Zhang, S\. Xin, L\. Chen, Q\. Liu, X\. Zhong, A\. Li,et al\.\(2025\)Multi\-swe\-bench: a multilingual benchmark for issue resolving\.arXiv preprint arXiv:2504\.02605\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p3.1)\.
- L\. Zeng, Y\. Li, Y\. Xiao, C\. Li, C\. Y\. Liu, R\. Yan, T\. Wei, J\. He, X\. Song, Y\. Liu,et al\.\(2025\)Skywork\-swe: unveiling data scaling laws for software engineering in llms\.arXiv preprint arXiv:2506\.19290\.Cited by:[§5](https://arxiv.org/html/2607.12463#S5.p3.1)\.

## Appendix ACorpus Details

This appendix expands on the data\-collection summary \(Section[2\.2](https://arxiv.org/html/2607.12463#S2.SS2)\)\. We report category coverage, per\-property statistics, and the license inventory of the968968released repositories\.

![Refer to caption](https://arxiv.org/html/2607.12463v1/x3.png)Figure 3:Distribution of the968968source repositories across ten topic categories\. The corpus is dominated by reference implementations, scientific computing, and small frameworks; compiler and networking/security tails are kept by design to maintain coverage diversity\.Table 5:Mid\-training corpus statistics after decontamination and quality filtering\. Token counts use the Qwen2\.5\-Coder tokenizer\. All numeric quantities reflect the released corpus\.### A\.1License Inventory

Figure[4](https://arxiv.org/html/2607.12463#A1.F4)shows the license distribution across the968968released repositories\. The corpus is dominated by permissive licenses \(MIT, Apache\-2\.0, BSD\-3\); the remainder is split among copyleft \(GPL, AGPL, LGPL, MPL\) and Creative Commons families\. Every license in the corpus permits at least non\-commercial research use, and all small categories are aggregated in this chart as “Other research\-permissive licenses\.” The per\-repository license is released alongside the repository list\.

![Refer to caption](https://arxiv.org/html/2607.12463v1/x4.png)Figure 4:License distribution of the968968\-repository corpus\. Permissive licenses \(MIT, Apache\-2\.0, BSD\) account for over80%80\\%of the corpus\. Small categories \(LGPL, ISC, Boost, MIT\-0, Unlicense, CC0, CC BY, CC BY\-NC, etc\.\) are aggregated as “Other research\-permissive licenses,” all of which permit at least non\-commercial research use\.

## Appendix BAlgorithmic Details

This appendix collects the full algorithmic details summarized in Section[2\.3](https://arxiv.org/html/2607.12463#S2.SS3)\. The complete Python implementation of both the single\-function and multi\-function selection pipelines, including all default hyperparameters listed below, is provided in the supplementary material\.

### B\.1Single\-Function FIM Target Selection

Algorithm[1](https://arxiv.org/html/2607.12463#alg1)summarizes the per\-file pipeline that produces single\-function FIM targets\. It composes the program dependency graph \(Section[2\.3\.1](https://arxiv.org/html/2607.12463#S2.SS3.SSS1)\), the complexity scoreH^\\hat\{H\}\(Section[2\.3\.2](https://arxiv.org/html/2607.12463#S2.SS3.SSS2)\), the inferability scoreI^\\hat\{I\}\(Section[2\.3\.3](https://arxiv.org/html/2607.12463#S2.SS3.SSS3)\), the difficulty penaltyρ\\rho\(Appendix[B\.5](https://arxiv.org/html/2607.12463#A2.SS5)\), and the hard filters described below\.

Algorithm 1Single\-function FIM target selection for one file\.1:Source file

ss; thresholds

τd,τFIM,τH^\\tau\_\{d\},\\tau\_\{\\mathrm\{FIM\}\},\\tau\_\{\\hat\{H\}\}; file\-line bounds

\[Lmin,Lmax\]\[L\_\{\\min\},L\_\{\\max\}\]; LoC bounds

\[ℓmin,ℓmax\]\[\\ell\_\{\\min\},\\ell\_\{\\max\}\]
2:if

Lines​\(s\)∉\[Lmin,Lmax\]\\mathrm\{Lines\}\(s\)\\notin\[L\_\{\\min\},L\_\{\\max\}\]then

3:return

∅\\emptyset⊳\\trianglerightfile\-level pre\-filter

4:endif

5:

T←ParseAST​\(s\)T\\leftarrow\\mathrm\{ParseAST\}\(s\)
6:

𝒱,ℰcall,ℰsib←BuildPDG​\(T\)\\mathcal\{V\},\\mathcal\{E\}\_\{\\mathrm\{call\}\},\\mathcal\{E\}\_\{\\mathrm\{sib\}\}\\leftarrow\\mathrm\{BuildPDG\}\(T\)⊳\\trianglerightSection[2\.3\.1](https://arxiv.org/html/2607.12463#S2.SS3.SSS1)

7:

𝒞←∅\\mathcal\{C\}\\leftarrow\\emptyset
8:foreach

v∈𝒱v\\in\\mathcal\{V\}do

9:if

vvis a dunder methodor

LoC​\(v\)∉\[ℓmin,ℓmax\]\\mathrm\{LoC\}\(v\)\\notin\[\\ell\_\{\\min\},\\ell\_\{\\max\}\]then

10:continue

11:endif

12:Compute

H^​\(v\)\\hat\{H\}\(v\)and

I^​\(v\)\\hat\{I\}\(v\)⊳\\trianglerightEqs\. \([1](https://arxiv.org/html/2607.12463#S2.E1)\), \([2](https://arxiv.org/html/2607.12463#S2.E2)\)

13:if

H^​\(v\)<τH^\\hat\{H\}\(v\)<\\tau\_\{\\hat\{H\}\}thencontinue

14:endif

15:

Δ​\(v\)←max⁡\(0,H^​\(v\)−I^​\(v\)\)/\(H^​\(v\)\+ϵ\)\\Delta\(v\)\\leftarrow\\max\(0,\\hat\{H\}\(v\)\-\\hat\{I\}\(v\)\)/\(\\hat\{H\}\(v\)\+\\epsilon\)
16:

FIM​\(v\)←H^​\(v\)​I^​\(v\)H^​\(v\)\+I^​\(v\)\+ϵ⋅ρ​\(Δ​\(v\)\)\\mathrm\{FIM\}\(v\)\\leftarrow\\dfrac\{\\hat\{H\}\(v\)\\,\\hat\{I\}\(v\)\}\{\\hat\{H\}\(v\)\+\\hat\{I\}\(v\)\+\\epsilon\}\\cdot\\rho\(\\Delta\(v\)\)
17:if

FIM​\(v\)≥τFIM\\mathrm\{FIM\}\(v\)\\geq\\tau\_\{\\mathrm\{FIM\}\}then

18:

𝒞←𝒞∪\{v\}\\mathcal\{C\}\\leftarrow\\mathcal\{C\}\\cup\\\{v\\\}
19:endif

20:endfor

21:return

𝒞\\mathcal\{C\}⊳\\trianglerightranked byFIM\\mathrm\{FIM\}descending

We useLmin=50L\_\{\\min\}=50,Lmax=1800L\_\{\\max\}=1800,ℓmin=10\\ell\_\{\\min\}=10,ℓmax=200\\ell\_\{\\max\}=200,τH^=0\.15\\tau\_\{\\hat\{H\}\}=0\.15, andτFIM=0\.08\\tau\_\{\\mathrm\{FIM\}\}=0\.08throughout\.

### B\.2Program Dependency Graph: Call Resolution

For everyCallnode inside the body ofu∈𝒱u\\in\\mathcal\{V\}, we attempt to resolve its callee to somev∈𝒱v\\in\\mathcal\{V\}, adding a directed edgeu→vu\\to vinℰcall\\mathcal\{E\}\_\{\\mathrm\{call\}\}\. Resolution handles three Python idioms:

- •Direct module\-level invocations:foo\(\.\.\.\)is resolved by qualified\-name lookup in the same module\.
- •Class instantiation:ClassName\(\.\.\.\)is resolved toClassName\.\_\_init\_\_\.
- •Method calls:self\.m\(\.\.\.\)orcls\.m\(\.\.\.\)inside class bodies is resolved within the enclosing class scope\.

When a call cannot be resolved exactly we fall back to short\-name matching against the qualified\-name index, which recovers most cross\-module references while admitting a controlled false\-positive rate\. Sibling edgesℰsib\\mathcal\{E\}\_\{\\mathrm\{sib\}\}are constructed by enumerating all pairs of methods belonging to the same class\.

### B\.3Complexity Score: Caps and Weights

The complexity scoreH^​\(v\)\\hat\{H\}\(v\)defined in Eq\. \([1](https://arxiv.org/html/2607.12463#S2.E1)\) uses caps\(cℓ,cc,cd\)=\(50,10,5\)\(c\_\{\\ell\},c\_\{c\},c\_\{d\}\)=\(50,10,5\)and weights\(wℓ,wc,wd\)=\(0\.4,0\.4,0\.2\)\(w\_\{\\ell\},w\_\{c\},w\_\{d\}\)=\(0\.4,0\.4,0\.2\)\. The capcℓ=50c\_\{\\ell\}=50on lines of code matches the median target length in our corpus; the capcc=10c\_\{c\}=10on McCabe complexity is roughly twice the median over our filtered functions; the capcd=5c\_\{d\}=5on nesting depth covers the tail of practical Python code\. The cap value of22inϕ​\(x,c\)=min⁡\(x/c,2\)\\phi\(x,c\)=\\min\(x/c,2\)allows genuinely complex functions to score above the median without letting outliers dominate\.

### B\.4Inferability Score: Component Formulas

The five components ofI^​\(v\)\\hat\{I\}\(v\)\(Eq\.[2](https://arxiv.org/html/2607.12463#S2.E2)\) are defined below\. Default mixing weights are\(α,β,γ,δ,ε\)=\(0\.30,0\.25,0\.20,0\.10,0\.15\)\(\\alpha,\\beta,\\gamma,\\delta,\\varepsilon\)=\(0\.30,0\.25,0\.20,0\.10,0\.15\)\. All numerical constants used inside the components are listed in Table[6](https://arxiv.org/html/2607.12463#A2.T6); the formulas reference them directly\.

Caller signalCcallerC\_\{\\mathrm\{caller\}\}\.For each in\-file calleruuofvvwe compute a per\-caller specificitysp​\(u,v\)\\mathrm\{sp\}\(u,v\)by scanning every call site insideuuthat targetsvv\(matched by short name\)\. At each matching site we accumulatebsite=0\.50b\_\{\\mathrm\{site\}\}=0\.50for the call itself, plusηlit=0\.15\\eta\_\{\\mathrm\{lit\}\}=0\.15per literal\-constant argument,ηname=0\.05\\eta\_\{\\mathrm\{name\}\}=0\.05per name argument,ηother=0\.08\\eta\_\{\\mathrm\{other\}\}=0\.08per other expression argument,ηkw=0\.12\\eta\_\{\\mathrm\{kw\}\}=0\.12per keyword argument, and abfound=0\.10b\_\{\\mathrm\{found\}\}=0\.10“found” bonus\. The per\-caller specificity is clipped atcapcaller=1\.5\\mathrm\{cap\}\_\{\\mathrm\{caller\}\}=1\.5, with a fallback value of0\.50\.5when the caller does not actually invokevvby name\. Summing over callers and normalizing,Ccaller=min⁡\(∑usp​\(u,v\)/ncaller,1\)C\_\{\\mathrm\{caller\}\}=\\min\\\!\\left\(\\sum\_\{u\}\\mathrm\{sp\}\(u,v\)\\,/\\,n\_\{\\mathrm\{caller\}\},\\,1\\right\), with normalization constantncaller=3n\_\{\\mathrm\{caller\}\}=3\.

Callee signalCcalleeC\_\{\\mathrm\{callee\}\}\.Ccallee=min⁡\(\|IntCallees​\(v\)\|/ncallee,1\)C\_\{\\mathrm\{callee\}\}=\\min\(\|\\mathrm\{IntCallees\}\(v\)\|/n\_\{\\mathrm\{callee\}\},\\,1\)whereIntCallees​\(v\)\\mathrm\{IntCallees\}\(v\)is the set of intra\-file functions thatvvcalls, andncallee=4n\_\{\\mathrm\{callee\}\}=4\. A function that orchestrates known helpers is more recoverable than one calling only opaque externals\.

Signature signalCsigC\_\{\\mathrm\{sig\}\}\.A monotone aggregate of four sub\-bonuses, capped at11:\+0\.30\+0\.30if a return\-type annotation is present,\+0\.25\+0\.25if any parameter carries a type annotation,\+min⁡\(nameparts/5,0\.25\)\+\\min\(\\mathrm\{nameparts\}/5,\\,0\.25\)for the descriptiveness of the function name \(split on\_\), and\+min⁡\(\|params¬self\|/6,0\.20\)\+\\min\(\|\\mathrm\{params\}\_\{\\neg\\mathrm\{self\}\}\|/6,\\,0\.20\)for the count of non\-self/clsparameters\.

Documentation signalCdocC\_\{\\mathrm\{doc\}\}\.Cdoc=0\.5C\_\{\\mathrm\{doc\}\}=0\.5if a docstring is present, otherwise0\. We do not score docstring informativeness, both for robustness and because Section[2\.4](https://arxiv.org/html/2607.12463#S2.SS4)generates a richer textual rationale\.

Class signalCclassC\_\{\\mathrm\{class\}\}\.For methods,Cclass=min⁡\(sib​\(v\)/nsib,1\)C\_\{\\mathrm\{class\}\}=\\min\(\\mathrm\{sib\}\(v\)/n\_\{\\mathrm\{sib\}\},\\,1\)wherensib=5n\_\{\\mathrm\{sib\}\}=5, plus abinit=0\.30b\_\{\\mathrm\{init\}\}=0\.30bonus when a distinct\_\_init\_\_exists in the same class; the total is clipped at11\. Module\-level functions receiveCclass=0C\_\{\\mathrm\{class\}\}=0\.

Table 6:Numerical constants inside theI^\\hat\{I\}components\.ComponentConstantValueCcallerC\_\{\\mathrm\{caller\}\}call\-site basebsiteb\_\{\\mathrm\{site\}\}0\.500\.50literal\-arg incrementηlit\\eta\_\{\\mathrm\{lit\}\}0\.150\.15name\-arg incrementηname\\eta\_\{\\mathrm\{name\}\}0\.050\.05other\-arg incrementηother\\eta\_\{\\mathrm\{other\}\}0\.080\.08keyword\-arg incrementηkw\\eta\_\{\\mathrm\{kw\}\}0\.120\.12found bonusbfoundb\_\{\\mathrm\{found\}\}0\.100\.10per\-caller capcapcaller\\mathrm\{cap\}\_\{\\mathrm\{caller\}\}1\.51\.5expected\-callers normncallern\_\{\\mathrm\{caller\}\}33CcalleeC\_\{\\mathrm\{callee\}\}expected\-fan\-out normncalleen\_\{\\mathrm\{callee\}\}44CsigC\_\{\\mathrm\{sig\}\}return\-type bonus\+0\.30\+0\.30param\-type bonus\+0\.25\+0\.25name\-parts termmin⁡\(parts/5,0\.25\)\\min\(\\mathrm\{parts\}/5,\\,0\.25\)non\-self params termmin⁡\(\|p¬self\|/6,0\.20\)\\min\(\|p\_\{\\neg\\mathrm\{self\}\}\|/6,\\,0\.20\)CdocC\_\{\\mathrm\{doc\}\}docstring present0\.50\.5CclassC\_\{\\mathrm\{class\}\}sibling normnsibn\_\{\\mathrm\{sib\}\}55\_\_init\_\_bonusbinitb\_\{\\mathrm\{init\}\}0\.300\.30
### B\.5DifficultyΔ\\Delta, Penaltyρ\\rho, and Hard Filters

The penaltyρ​\(Δ​\(v\)\)\\rho\(\\Delta\(v\)\)in Eq\. \([3](https://arxiv.org/html/2607.12463#S2.E3)\) is built from the residual difficultyΔ​\(v\)\\Delta\(v\), defined as the share of complexity that is not explained by context:

Δ​\(v\)=max⁡\(0,H^​\(v\)−I^​\(v\)\)H^​\(v\)\+ϵ∈\[0,1\)\.\\Delta\(v\)\\;=\\;\\frac\{\\max\\\!\\left\(0,\\,\\hat\{H\}\(v\)\-\\hat\{I\}\(v\)\\right\)\}\{\\hat\{H\}\(v\)\+\\epsilon\}\\;\\in\\;\[0,1\)\.\(4\)The penaltyρ\\rhois one\-sided: targets withΔ​\(v\)≤τd\\Delta\(v\)\\leq\\tau\_\{d\}are left intact, while targets above the threshold are damped by a Gaussian factor:

ρ​\(Δ\)=\{1,Δ≤τd,exp⁡\(−\(Δ−τd\)22​σ2\),Δ\>τd\.\\rho\(\\Delta\)\\;=\\;\\begin\{cases\}1,&\\Delta\\leq\\tau\_\{d\},\\\\\[2\.0pt\] \\exp\\\!\\left\(\-\\dfrac\{\(\\Delta\-\\tau\_\{d\}\)^\{2\}\}\{2\\sigma^\{2\}\}\\right\),&\\Delta\>\\tau\_\{d\}\.\\end\{cases\}\(5\)For single\-function selection we useτd=0\.50\\tau\_\{d\}=0\.50andσ=0\.20\\sigma=0\.20\. The asymmetry reflects an asymmetry in the underlying objective: trivially easy targets are already eliminated by the hard filters below, so we do not need to additionally reward high inferability; conversely, targets that remain hard even with full context become unlearnable noise and must be down\-weighted\.

Hard filters\.Independent of the smooth score, we discard*\(i\)*files outside the line\-count window\[Lmin,Lmax\]=\[50,1800\]\[L\_\{\\min\},L\_\{\\max\}\]=\[50,1800\]entirely;*\(ii\)*functions withLoC​\(v\)∉\[10,200\]\\mathrm\{LoC\}\(v\)\\notin\[10,200\]to control training\-instance length;*\(iii\)*dunder methods \(\_\_init\_\_,\_\_repr\_\_, etc\.\), which serve as scaffolding rather than reasoning targets;*\(iv\)*functions withH^​\(v\)<τH^=0\.15\\hat\{H\}\(v\)<\\tau\_\{\\hat\{H\}\}=0\.15to remove trivial cases; and*\(v\)*functions withFIM​\(v\)<τFIM=0\.08\\mathrm\{FIM\}\(v\)<\\tau\_\{\\mathrm\{FIM\}\}=0\.08\. A file with no surviving target contributes nothing to the corpus\.

### B\.6Worked Example: ScoringCalculator\.total

To make the numbers in Figure[2](https://arxiv.org/html/2607.12463#S2.F2)\(b\) reproducible, this subsection walks through every quantity that goes intoH^\\hat\{H\},I^\\hat\{I\}, and the final FIM score for the running example\. The target function is:

```
def total(self) -> int:
    """Sum positive integer entries."""
    s, n = 0, 0
    for v in self.history:
        if is_int(v):
            if v > 0:
                s = add(s, v)
                n += 1
    if n == 0: return 0
    return s
```

Lines of code \(LoC = 10\)\.Following Python’sastconvention,LoC​\(v\)=end\_lineno−\(lineno−1\)\\mathrm\{LoC\}\(v\)=\\texttt\{end\\\_lineno\}\-\(\\texttt\{lineno\}\-1\), the number of source lines from thedefkeyword through the last statement, both inclusive\. Counting the ten lines above \(including the docstring\) givesLoC=10\\mathrm\{LoC\}=10\.

Cyclomatic complexity \(CC = 5\)\.The McCabe counter starts at11and adds11per branching node:If/IfExp,For/While,ExceptHandler,\(values−1\)\(\\text\{values\}\{\-\}1\)perBoolOp, and one per generator inside a comprehension\. ForCalculator\.total: base \(\+1\+1\), theforloop \(\+1\+1\),if is\_int\(\+1\+1\),if v\>0\(\+1\+1\),if n==0\(\+1\+1\), totallingCC=5\\mathrm\{CC\}=5\. Tuple unpacking andn\+=1are not branches\.

Maximum nesting depth \(D = 3\)\.Depth increments only on structural containers \(If,For,While,AsyncFor,With,AsyncWith,Try\)\. The deepest path intotalisfor→\\toif is\_int→\\toif v\>0, givingD=3\\mathrm\{D\}=3\.

Complexity scoreH^=0\.40\\hat\{H\}=0\.40\.Plugging into Eq\.[1](https://arxiv.org/html/2607.12463#S2.E1)with the default\(wℓ,wc,wd\)=\(0\.4,0\.4,0\.2\)\(w\_\{\\ell\},w\_\{c\},w\_\{d\}\)=\(0\.4,0\.4,0\.2\)and\(cℓ,cc,cd\)=\(50,10,5\)\(c\_\{\\ell\},c\_\{c\},c\_\{d\}\)=\(50,10,5\):

wℓ​ϕ​\(10,50\)\\displaystyle w\_\{\\ell\}\\,\\phi\(10,50\)=0\.4⋅0\.20=0\.08,wc​ϕ​\(5,10\)=0\.4⋅0\.50=0\.20,\\displaystyle=0\.4\\\!\\cdot\\\!0\.20=0\.08,\\qquad w\_\{c\}\\,\\phi\(5,10\)=0\.4\\\!\\cdot\\\!0\.50=0\.20,wd​ϕ​\(3,5\)\\displaystyle w\_\{d\}\\,\\phi\(3,5\)=0\.2⋅0\.60=0\.12,H^​\(total\)=0\.08\+0\.20\+0\.12=0\.40\.\\displaystyle=0\.2\\\!\\cdot\\\!0\.60=0\.12,\\quad\\hat\{H\}\(\\texttt\{total\}\)=0\.08\+0\.20\+0\.12=0\.40\.
Inferability scoreI^=0\.48\\hat\{I\}=0\.48\.*\(i\) Caller specificity\.*The only in\-file caller isCalculator\.mean, which invokesself\.total\(\)with no arguments and no keywords\. Per Table[6](https://arxiv.org/html/2607.12463#A2.T6),sp=bsite\+bfound=0\.50\+0\.10=0\.60\\mathrm\{sp\}=b\_\{\\mathrm\{site\}\}\+b\_\{\\mathrm\{found\}\}=0\.50\+0\.10=0\.60\. Aggregating and normalizing byncaller=3n\_\{\\mathrm\{caller\}\}=3,Ccaller=min⁡\(0\.60/3,1\.0\)=0\.20C\_\{\\mathrm\{caller\}\}=\\min\(0\.60/3,1\.0\)=0\.20andα​Ccaller=0\.06\\alpha\\,C\_\{\\mathrm\{caller\}\}=0\.06\.*\(ii\) Callee count\.*totalcallsis\_intandaddin\-file \(self\.history\.append\(v\)does not count becauseappendis not in𝒱\\mathcal\{V\}\), soCcallee=min⁡\(2/ncallee,1\.0\)=min⁡\(2/4,1\.0\)=0\.50C\_\{\\mathrm\{callee\}\}=\\min\(2/n\_\{\\mathrm\{callee\}\},1\.0\)=\\min\(2/4,1\.0\)=0\.50andβ​Ccallee=0\.13\\beta\\,C\_\{\\mathrm\{callee\}\}=0\.13\.*\(iii\) Signature\.*Sub\-bonuses\+0\.30\+0\.30for\-\> int;\+min⁡\(1/5,0\.25\)=0\.20\+\\min\(1/5,0\.25\)=0\.20for the descriptive nametotal;0for parameter type annotations;0for non\-selfparameters:Csig=0\.50C\_\{\\mathrm\{sig\}\}=0\.50andγ​Csig=0\.10\\gamma\\,C\_\{\\mathrm\{sig\}\}=0\.10\.*\(iv\) Documentation\.*A docstring is present, soCdoc=0\.5C\_\{\\mathrm\{doc\}\}=0\.5andδ​Cdoc=0\.05\\delta\\,C\_\{\\mathrm\{doc\}\}=0\.05\.*\(v\) Class context\.*totalsits among three siblings; the base score ismin⁡\(3/nsib,1\.0\)=min⁡\(3/5,1\.0\)=0\.60\\min\(3/n\_\{\\mathrm\{sib\}\},1\.0\)=\\min\(3/5,1\.0\)=0\.60, plus thebinit=0\.30b\_\{\\mathrm\{init\}\}=0\.30bonus for the distinct\_\_init\_\_ofCalculator, givingCclass=0\.90C\_\{\\mathrm\{class\}\}=0\.90andε​Cclass=0\.14\\varepsilon\\,C\_\{\\mathrm\{class\}\}=0\.14\. Summing:I^​\(total\)=0\.06\+0\.13\+0\.10\+0\.05\+0\.14=0\.48\\hat\{I\}\(\\texttt\{total\}\)=0\.06\+0\.13\+0\.10\+0\.05\+0\.14=0\.48\.

Final FIM score and selection\.Δ​\(total\)=max⁡\(0,−0\.08\)/0\.40=0\\Delta\(\\texttt\{total\}\)=\\max\(0,\-0\.08\)/0\.40=0, soρ​\(Δ\)=1\\rho\(\\Delta\)=1andFIM​\(total\)=\(0\.40⋅0\.48\)/\(0\.40\+0\.48\)≈0\.22≥τFIM=0\.08\\mathrm\{FIM\}\(\\texttt\{total\}\)=\(0\.40\\\!\\cdot\\\!0\.48\)/\(0\.40\\\!\+\\\!0\.48\)\\approx 0\.22\\geq\\tau\_\{\\mathrm\{FIM\}\}=0\.08, selecting it as a mask target\. The four short helpers are removed before scoring by the LoC hard filter;Calculator\.\_\_init\_\_is removed by the dunder filter\.

### B\.7Multi\-Function Group Score

Group score\.For a groupG=\{v1,…,vk\}G=\\\{v\_\{1\},\\dots,v\_\{k\}\\\}enumerated from one of eight topology patterns,

FIM​\(G\)=Coup​\(G\)⋅H^​\(G\)​I^​\(G\)H^​\(G\)\+I^​\(G\)\+ϵ⋅ρG​\(Δ​\(G\)\),\\mathrm\{FIM\}\(G\)\\;=\\;\\mathrm\{Coup\}\(G\)\\cdot\\frac\{\\hat\{H\}\(G\)\\,\\hat\{I\}\(G\)\}\{\\hat\{H\}\(G\)\+\\hat\{I\}\(G\)\+\\epsilon\}\\cdot\\rho^\{G\}\\\!\\left\(\\Delta\(G\)\\right\),\(6\)whereH^​\(G\)=1k​∑vH^​\(v\)\\hat\{H\}\(G\)=\\tfrac\{1\}\{k\}\\sum\_\{v\}\\hat\{H\}\(v\)averages individual complexities,Δ​\(G\)\\Delta\(G\)is defined analogously to the single\-function case,ρG\\rho^\{G\}uses group\-specific parametersτdG=0\.55\\tau\_\{d\}^\{G\}=0\.55andσG=0\.20\\sigma^\{G\}=0\.20\(slightly more permissive than the single\-functionτd=0\.50\\tau\_\{d\}=0\.50, reflecting that joint masking inherently lowersI^\\hat\{I\}\), and the coupling termCoup​\(G\)∈\[0,1\]\\mathrm\{Coup\}\(G\)\\in\[0,1\]is a normalized count of intra\-group edges plus a shared\-state ratio:

Coup​\(G\)=wc​\|EGcall\|k​\(k−1\)\+ws​\|EGsib\|\(k2\)\+wst​Jacc¯G,\\mathrm\{Coup\}\(G\)\\;=\\;w\_\{\\mathrm\{c\}\}\\,\\frac\{\|E^\{\\text\{call\}\}\_\{G\}\|\}\{k\(k\\\!\-\\\!1\)\}\+w\_\{\\mathrm\{s\}\}\\,\\frac\{\|E^\{\\text\{sib\}\}\_\{G\}\|\}\{\\binom\{k\}\{2\}\}\+w\_\{\\mathrm\{st\}\}\\,\\overline\{\\mathrm\{Jacc\}\}\_\{G\},\(7\)withJacc¯G\\overline\{\\mathrm\{Jacc\}\}\_\{G\}the mean Jaccard similarity over pairs inGGbetween sets of accessed instance attributes \(self\.x\)\. We use\(wc,ws,wst\)=\(0\.50,0\.20,0\.30\)\(w\_\{\\mathrm\{c\}\},w\_\{\\mathrm\{s\}\},w\_\{\\mathrm\{st\}\}\)=\(0\.50,0\.20,0\.30\)\. The coupling term enters multiplicatively because, unlike the single\-function case, the value of a multi\-function target derives specifically from cross\-function dependencies; a weakly coupled group is indistinguishable fromkkindependent samples\.

Group inferability\.I^​\(G\)\\hat\{I\}\(G\)is recomputed under the assumption that*all*members ofGGare masked simultaneously: caller, callee, and sibling contributions from insideGGare excluded; docstrings of group members contribute zero \(they are part of the masked body\); and the\_\_init\_\_bonus inCclassC\_\{\\mathrm\{class\}\}is awarded only when\_\_init\_\_is itself outsideGG\. Signature information is retained because we do not mask function signatures\. The remaining constants\(bsite,ηlit,…\)\(b\_\{\\mathrm\{site\}\},\\eta\_\{\\mathrm\{lit\}\},\\dots\)in Table[6](https://arxiv.org/html/2607.12463#A2.T6)are unchanged\. This re\-computation prevents groups from receiving spurious credit for intra\-group references that disappear under joint masking\.

Topology taxonomy\.A candidate group must form a connected subgraph inℰcall∪ℰsib\\mathcal\{E\}\_\{\\mathrm\{call\}\}\\cup\\mathcal\{E\}\_\{\\mathrm\{sib\}\}\. We enumerate eight patterns:

- •*k=2k=2*:*caller\-callee*\(directA→BA\\\!\\to\\\!Bedge\);*co\-callee*\(two callees of a common in\-file caller\);*sibling\-coupled*\(same\-class methods sharing≥1\\geq 1instance attribute\);*mutual\-call*\(A⇌BA\\\!\\rightleftharpoons\\\!B\)\.
- •*k=3k=3*:*call\-chain*\(A→B→CA\\\!\\to\\\!B\\\!\\to\\\!C\);*hub*\(A→B,A→CA\\\!\\to\\\!B,\\ A\\\!\\to\\\!C\);*fan\-in*\(B→A,C→AB\\\!\\to\\\!A,\\ C\\\!\\to\\\!A\);*class\-triad*\(three pairwise state\-sharing methods of the same class\)\.

Group selection\.We score every candidate group emitted by the topology enumerator and discard those failing any of the following defaults: each member must satisfyLoC∈\[10,200\]\\mathrm\{LoC\}\\in\[10,200\]and not be a dunder;Coup​\(G\)≥τcoupG=0\.15\\mathrm\{Coup\}\(G\)\\geq\\tau^\{G\}\_\{\\mathrm\{coup\}\}=0\.15;H^​\(G\)≥τH^G=0\.15\\hat\{H\}\(G\)\\geq\\tau^\{G\}\_\{\\hat\{H\}\}=0\.15; total LoC ratio of the group relative to the file does not exceedθ2=0\.30\\theta\_\{2\}=0\.30fork=2k=2orθ3=0\.40\\theta\_\{3\}=0\.40fork=3k=3; and the score clears a size\-specific floor,FIM​\(G\)≥τFIM,2G=0\.04\\mathrm\{FIM\}\(G\)\\geq\\tau^\{G\}\_\{\\mathrm\{FIM,2\}\}=0\.04for pairs andFIM​\(G\)≥τFIM,3G=0\.03\\mathrm\{FIM\}\(G\)\\geq\\tau^\{G\}\_\{\\mathrm\{FIM,3\}\}=0\.03for triples\. Remaining candidates are sorted byFIM​\(G\)\\mathrm\{FIM\}\(G\)and selected greedily under non\-overlap: a function may belong to at most one selected group per file\. Per\-file caps\(N2,N3\)=\(5,3\)\(N\_\{2\},N\_\{3\}\)=\(5,3\)prevent any single file from dominating the corpus\.

Algorithm 2Multi\-function group FIM target selection for one file\.1:Source file

ss; per\-function scores

H^​\(v\),I^​\(v\)\\hat\{H\}\(v\),\\hat\{I\}\(v\); thresholds

τcoupG=0\.15,τH^G=0\.15,τFIM,2G=0\.04,τFIM,3G=0\.03\\tau^\{G\}\_\{\\mathrm\{coup\}\}=0\.15,\\;\\tau^\{G\}\_\{\\hat\{H\}\}=0\.15,\\;\\tau^\{G\}\_\{\\mathrm\{FIM,2\}\}=0\.04,\\;\\tau^\{G\}\_\{\\mathrm\{FIM,3\}\}=0\.03; LoC ratio caps

\(θ2,θ3\)=\(0\.30,0\.40\)\(\\theta\_\{2\},\\theta\_\{3\}\)=\(0\.30,0\.40\); per\-file caps

\(N2,N3\)=\(5,3\)\(N\_\{2\},N\_\{3\}\)=\(5,3\)
2:

𝒱,ℰcall,ℰsib←BuildPDG​\(ParseAST​\(s\)\)\\mathcal\{V\},\\mathcal\{E\}\_\{\\mathrm\{call\}\},\\mathcal\{E\}\_\{\\mathrm\{sib\}\}\\leftarrow\\mathrm\{BuildPDG\}\(\\mathrm\{ParseAST\}\(s\)\)
3:

𝒢←∅\\mathcal\{G\}\\leftarrow\\emptyset
4:foreach topology pattern

P∈\{caller\-callee, co\-callee, sibling\-coupled, mutual\-call,P\\in\\\{\\text\{caller\-callee, co\-callee, sibling\-coupled, mutual\-call,\}call\-chain, hub, fan\-in, class\-triad\}\\quad\\text\{call\-chain, hub, fan\-in, class\-triad\}\\\}do

5:Enumerate all connected subgraphs

G⊆𝒱G\\subseteq\\mathcal\{V\}matching

PPin

ℰcall∪ℰsib\\mathcal\{E\}\_\{\\mathrm\{call\}\}\\cup\\mathcal\{E\}\_\{\\mathrm\{sib\}\}
6:foreach candidate

GGdo

7:ifany

v∈Gv\\\!\\in\\\!Gfails the per\-function filteror

LoC​\_​ratio​\(G,s\)\>θ\|G\|\\mathrm\{LoC\\\_ratio\}\(G,s\)\>\\theta\_\{\|G\|\}or

Coup​\(G\)<τcoupG\\mathrm\{Coup\}\(G\)<\\tau^\{G\}\_\{\\mathrm\{coup\}\}or

H^​\(G\)<τH^G\\hat\{H\}\(G\)<\\tau^\{G\}\_\{\\hat\{H\}\}then

8:continue

9:endif

10:Recompute

I^​\(G\)\\hat\{I\}\(G\)under joint masking⊳\\trianglerightexclude intra\-group references;Cdoc=0C\_\{\\mathrm\{doc\}\}\\\!=\\\!0

11:

Δ​\(G\)←max⁡\(0,H^​\(G\)−I^​\(G\)\)/\(H^​\(G\)\+ϵ\)\\Delta\(G\)\\leftarrow\\max\(0,\\hat\{H\}\(G\)\-\\hat\{I\}\(G\)\)/\(\\hat\{H\}\(G\)\+\\epsilon\)
12:

FIM​\(G\)←Coup​\(G\)⋅H^​\(G\)​I^​\(G\)H^​\(G\)\+I^​\(G\)\+ϵ⋅ρG​\(Δ​\(G\)\)\\mathrm\{FIM\}\(G\)\\leftarrow\\mathrm\{Coup\}\(G\)\\cdot\\dfrac\{\\hat\{H\}\(G\)\\,\\hat\{I\}\(G\)\}\{\\hat\{H\}\(G\)\+\\hat\{I\}\(G\)\+\\epsilon\}\\cdot\\rho^\{G\}\(\\Delta\(G\)\)
13:if

FIM​\(G\)≥τFIM,\|G\|G\\mathrm\{FIM\}\(G\)\\geq\\tau^\{G\}\_\{\\mathrm\{FIM\},\|G\|\}then

14:

𝒢←𝒢∪\{\(G,P,FIM​\(G\)\)\}\\mathcal\{G\}\\leftarrow\\mathcal\{G\}\\cup\\\{\(G,P,\\mathrm\{FIM\}\(G\)\)\\\}
15:endif

16:endfor

17:endfor

18:Sort

𝒢\\mathcal\{G\}by

FIM​\(G\)\\mathrm\{FIM\}\(G\)descending

19:

𝒮←∅\\mathcal\{S\}\\leftarrow\\emptyset
20:foreach

\(G,P,FIM​\(G\)\)∈𝒢\(G,P,\\mathrm\{FIM\}\(G\)\)\\in\\mathcal\{G\}do

21:ifany

v∈Gv\\in Galready covered by

𝒮\\mathcal\{S\}orper\-size cap

\(N2\(N\_\{2\}for

k=2k\\\!=\\\!2,

N3N\_\{3\}for

k=3\)k\\\!=\\\!3\)exceededthen

22:continue

23:endif

24:

𝒮←𝒮∪\{G\}\\mathcal\{S\}\\leftarrow\\mathcal\{S\}\\cup\\\{G\\\}
25:endfor

26:return

𝒮\\mathcal\{S\}

### B\.8Worked Example: Acaller\-calleePair

To illustrate the group score, consider a small file with two top\-level functions:

```
def normalize(text: str) -> str:
    """Lowercase and strip punctuation."""
    out = []
    for ch in text:
        if ch.isalnum() or ch.isspace():
            out.append(ch.lower())
    return "".join(out)

def word_freq(doc: str) -> dict[str, int]:
    """Count tokens of ‘doc‘ after normalization."""
    cleaned = normalize(doc)
    counts = {}
    for tok in cleaned.split():
        counts[tok] = counts.get(tok, 0) + 1
    return counts
```

Topology and pre\-checks\.The pairG=\{normalize,word\_freq\}G\\\!=\\\!\\\{\\texttt\{normalize\},\\texttt\{word\\\_freq\}\\\}matches the*caller\-callee*pattern with one call edgeword\_freq→normalize\\texttt\{word\\\_freq\}\\\!\\to\\\!\\texttt\{normalize\}\. They are top\-level \(no class\), so the sibling component ofCoup\\mathrm\{Coup\}is zero and the shared\-state component is zero \(noself\.\_accesses\)\. The combined LoC is roughly1414lines, well under theθ2=0\.30\\theta\_\{2\}\\\!=\\\!0\.30file\-ratio cap\.

Coupling\.With one call edge in a pair,\|EGcall\|/\(k​\(k−1\)\)=1/2\|E^\{\\text\{call\}\}\_\{G\}\|/\(k\(k\\\!\-\\\!1\)\)=1/2\. Plugging into Eq\.[7](https://arxiv.org/html/2607.12463#A2.E7)with\(wc,ws,ws​t\)=\(0\.50,0\.20,0\.30\)\(w\_\{c\},w\_\{s\},w\_\{st\}\)=\(0\.50,0\.20,0\.30\):Coup​\(G\)=0\.50⋅0\.5\+0\+0=0\.25\\mathrm\{Coup\}\(G\)=0\.50\\cdot 0\.5\+0\+0=0\.25, comfortably above theτcoupG=0\.15\\tau^\{G\}\_\{\\mathrm\{coup\}\}=0\.15floor\.

GroupH^\\hat\{H\}\.Per\-function complexities \(computed as in Section[2\.3\.2](https://arxiv.org/html/2607.12463#S2.SS3.SSS2)\) come out to roughlyH^​\(normalize\)≈0\.36\\hat\{H\}\(\\texttt\{normalize\}\)\\\!\\approx\\\!0\.36andH^​\(word\_freq\)≈0\.30\\hat\{H\}\(\\texttt\{word\\\_freq\}\)\\\!\\approx\\\!0\.30, givingH^​\(G\)≈0\.33\\hat\{H\}\(G\)\\\!\\approx\\\!0\.33\.

GroupI^\\hat\{I\}under joint masking\.Both functions are masked, so the call edge between them no longer contributes toCcallerC\_\{\\mathrm\{caller\}\}orCcalleeC\_\{\\mathrm\{callee\}\}\. Signature information is retained: both have descriptive names and full type annotations \(strongCsigC\_\{\\mathrm\{sig\}\}\); docstrings vanish under joint masking \(Cdoc=0C\_\{\\mathrm\{doc\}\}=0\)\. With no surviving caller/callee/class signal, the group inferability collapses toI^​\(G\)≈0\.18\\hat\{I\}\(G\)\\\!\\approx\\\!0\.18, dominated by the signature term\.

Final group FIM\.The harmonic\-mean\-like productH^​I^/\(H^\+I^\)≈\(0\.33⋅0\.18\)/0\.51≈0\.116\\hat\{H\}\\hat\{I\}/\(\\hat\{H\}\+\\hat\{I\}\)\\\!\\approx\\\!\(0\.33\\\!\\cdot\\\!0\.18\)/0\.51\\\!\\approx\\\!0\.116;Δ​\(G\)=\(0\.33−0\.18\)/0\.33≈0\.45<τdG=0\.55\\Delta\(G\)\\\!=\\\!\(0\.33\-0\.18\)/0\.33\\\!\\approx\\\!0\.45<\\tau\_\{d\}^\{G\}=0\.55soρG=1\\rho^\{G\}=1; andCoup​\(G\)=0\.25\\mathrm\{Coup\}\(G\)=0\.25:FIM​\(G\)≈0\.25×0\.116≈0\.029\\mathrm\{FIM\}\(G\)\\approx 0\.25\\\!\\times\\\!0\.116\\\!\\approx\\\!0\.029\. This example sits below the pair floorτFIM,2G=0\.04\\tau^\{G\}\_\{\\mathrm\{FIM,2\}\}\\\!=\\\!0\.04and would therefore*not*be selected—the toy module is too small for the joint\-masking inferability to remain large enough\. Real corpus pairs that pass the threshold typically come from class\-method pairs with non\-trivial sibling and shared\-state coupling, whereI^​\(G\)\\hat\{I\}\(G\)retains substantialCclassC\_\{\\mathrm\{class\}\}contribution even under joint masking\. The walkthrough nonetheless shows the mechanism: joint masking strictly reducesI^\\hat\{I\}relative to the single\-function average, and the coupling factor must compensate\.

### B\.9Chain\-of\-Thought Generation Prompt Template

Each FIM rationale–implementation pair is produced by Gemini\-3\-Flash in a single forward pass: given a Python source file with one function body redacted, the model is asked to first reason about what the body should do and then write a candidate implementation\. Crucially, the model is*not*shown the ground\-truth body at this stage; both the rationale and the predicted body must be derived from the surrounding context alone\. The \(rationale, predicted body\) pairs that survive the quality filter \(Appendix[B\.10](https://arxiv.org/html/2607.12463#A2.SS10)\) are then used to assemble mid\-training samples \(Appendix[B\.11](https://arxiv.org/html/2607.12463#A2.SS11)\)\.

> \[System\]You are an expert Python programmer\. \[User\]Below is a Python file where one function’s body has been replaced with\# <MASKED\_FUNCTION\_BODY\>\. Complete the masked function from the surrounding context\. Procedure\.Analyze the visible context—imports, sibling functions, classes, call sites, and helpers invoked by the target—then reason step by step about what the function must do given its signature, type hints, and usage pattern\. Only after this analysis, write an implementation consistent with the file’s coding style\. Output format\.Return two sections in order:\#\#\# Reasoning\(a step\-by\-step natural\-language analysis\) and\#\#\# Implementation\(a fencedpythonblock containing the function body only, without the signature\)\. <file with target body redacted\> The function to complete:<function name\>\.

For brevity, the skeleton condenses three pieces of the production prompt that govern output formatting rather than the task itself: \(i\) the explicit three\-step task breakdown \(analyze context, reason, implement\) and the four sub\-bullets under the reasoning step \(signature semantics, intra\-file usage, callee dependencies, overall purpose\), which we collapse into a singleProcedureparagraph and which direct the model toward the same context sources scored byI^\\hat\{I\}\(Section[2\.3\.3](https://arxiv.org/html/2607.12463#S2.SS3.SSS3)\); \(ii\) the formatting checklist enforcing body\-only output, four\-space indentation, and explicit docstring handling; and \(iii\) the verbatim Markdown headers and code\-fence delimiters that make the two sections programmatically separable\. The condensed material specifies*how*the response is formatted but does not change*what*is being elicited\. We sample with the Gemini API’s near\-greedy configuration \(temperature=0=0,topK=1=1\); the verbatim prompt and all post\-processing logic will be released with the corpus to support exact replication\.

### B\.10Completion\-Quality Filtering Prompt Template

Each candidate FIM sample, paired with a trial completion, is screened by an LLM judge that assesses both whether the masked function is recoverable from its surrounding context and the quality of the completion against the ground truth\. Filtering is performed by Gemini\-3\-Flash using the following prompt skeleton:

> \[System\]You are an expert code reviewer\. \[User\]A model was asked to complete a masked function from its surrounding code context\. Evaluate the result along two axes: \(i\) whether the function is feasible to complete from context alone, and \(ii\) the quality of the completion compared with the ground truth\. The source file with the target body redacted \(shown as<MASKED\>\): <file with target body redacted\> The function name, the ground\-truth body, and the model’s completion: <function name\>,<ground\-truth body\>,<model completion\> Part 1 – Feasibility\.Mark the function*infeasible*if it depends on uncommon external APIs, external conventions, or magic constants that cannot be inferred from the visible context; otherwise mark it*feasible*\. Part 2 – Quality\.Score the completion on a11–55scale along five dimensions—*correctness*,*executability*,*API usage*,*readability*, and*completeness*—each with a one\-sentence justification, then assign an overall11–55score\. Output\.Return a JSON object containing the feasibility verdict, the five component scores with reasons, the overall score, and ashould\_discardflag set totruewhen the function is infeasible, whenexecutability=1=1, or when bothoverall\_score≤2\\leq 2andexecutability≤2\\leq 2\.

For brevity, the skeleton above omits three pieces of the production prompt that anchor the scores rather than alter the procedure: \(i\) the per\-level rubric for each of the five dimensions \(e\.g\.,*correctness*55= “functionally identical,”33= “partially correct, some important cases wrong,”11= “completely incorrect”\), which fixes a consistent scale across samples; \(ii\) the expanded feasibility checklist enumerating concrete infeasibility factors \(reliance on niche libraries, knowledge of external conventions, insufficient context, system\-specific magic constants\) and corresponding feasibility indicators; and \(iii\) the full JSON schema with named fields for every component reason, key similarities/differences, and a free\-textdiscard\_reason\. The omitted material specifies*how*each axis is graded but does not change*what*is graded; We sample with the Gemini API’s near\-greedy configuration \(temperature=0=0,topK=1=1\)\. The verbatim prompt together with all post\-processing logic will be released with the corpus to support exact replication\.

### B\.11Mid\-Training Sample Formatting Template

After upstream CoT generation \(Appendix[B\.9](https://arxiv.org/html/2607.12463#A2.SS9)\) and quality filtering \(Appendix[B\.10](https://arxiv.org/html/2607.12463#A2.SS10)\), each retained sample is serialized into a chat\-style turn pair: the user turn poses the FIM completion task on the file with one function body redacted, and the assistant turn carries Gemini’s rationale followed by Gemini’s predicted body, both produced in the upstream pass\. The ground\-truth body from the source repository is used only by the filter to admit the sample; it does not appear in the training target\.

> \[System\]You are an expert Python programmer\. \[User\]Below is a Python file where one function’s body has been replaced with\# <MASKED\_FUNCTION\_BODY\>\. Complete the masked function from the surrounding context\.\[Procedure and output\-format instructions identical to Appendix[B\.9](https://arxiv.org/html/2607.12463#A2.SS9)\.\] <file with target body redacted\> The function to complete:<function name\>\. \[Assistant\] \#\#\# Reasoning <Gemini\-produced rationale\> \#\#\# Implementation ‘‘‘python <Gemini\-produced implementation\> ‘‘‘

### B\.12Negative\-Observation Patterns

The recovery\-rate analysis \(Section[4\.1](https://arxiv.org/html/2607.12463#S4.SS1)\) flags a tool output as a*negative observation*when it matches any of the following patterns: Python stack\-trace prefixes \(Traceback \(most recent call last\):\); error class names \(SyntaxError,IndentationError,NameError,ImportError,AttributeError,TypeError,ValueError\); shell error markers \(command not found,permission denied,No such file or directory\); and harness\-specific failure strings \(No replacement was performed,Patch did not apply,tests failed\)\. Patterns are matched case\-insensitively\.

## Appendix CTraining Hyperparameters

We useLlamaFactoryfor FIM mid\-training, R2E\-Gym post\-training, and SWE\-Lego post\-training, andtorchtunefor SWE\-Smith post\-training\. All runs use AdamW with bf16 mixed precision and a cosine learning\-rate schedule\. For R2E\-Gym and SWE\-Smith we follow the official release scripts; for SWE\-Lego we use22epochs instead of the official44to prevent overfitting \(all other hyperparameters unchanged\)\. Tables[7](https://arxiv.org/html/2607.12463#A3.T7)and[8](https://arxiv.org/html/2607.12463#A3.T8)list the exact settings; effective \(global\) batch sizes assume88GPUs\.

Table 7:FIM mid\-training hyperparameters, applied uniformly to all three base models \(Qwen2\.5\-Coder\-7B\-Instruct, Qwen2\.5\-Coder\-14B\-Instruct, and Qwen3\-8B\)\.Table 8:Agentic post\-training hyperparameters for the three pipelines\. R2E\-Gym and SWE\-Smith follow their official released scripts; SWE\-Lego follows the official recipe except for the epoch count\.†\\daggerDeviates from the official SWE\-Lego recipe \(44epochs\); reduced to22to prevent overfitting on the FIM\-midtrained Qwen3\-8B base\.

### C\.1Compute Resources

All experiments are run on a single node of88NVIDIA H100 80 GB GPUs\. Reproducing the full set of results in this paper—FIM mid\-training of three base models \(Qwen2\.5\-Coder\-7B/14B\-Instruct and Qwen3\-8B\), the three agentic post\-training pipelines \(R2E\-Gym, SWE\-Smith, SWE\-Lego\) and their corresponding mid\-trained variants, the data/CoT/granularity ablations of Section[3\.4](https://arxiv.org/html/2607.12463#S3.SS4), and the multi\-seed evaluation sweeps—takes roughly3030days of wall\-clock time on the88\-GPU node, i\.e\.≈5,760\\approx\\\!5\{,\}760GPU\-hours in total\. Preliminary and discarded runs not reported in this paper account for an additional∼30%\{\\sim\}30\\%of compute\.

## Appendix DExtended Behavioral Analysis \(SWE\-Bench\-Verified\)

This appendix collects the trajectory\-level material that supplements the headline analysis in Section[4](https://arxiv.org/html/2607.12463#S4)\. Unless stated otherwise, statistics are run\-means over three independent evaluation runs of each checkpoint on SWE\-Bench\-Verified \(500500instances per run\)\.

### D\.1Pass Rate by Gold\-Patch Shape

Figure[5](https://arxiv.org/html/2607.12463#A4.F5)reports the pass\-rate stratification used in Section[4\.2](https://arxiv.org/html/2607.12463#S4.SS2): the largest gain \(\+9\.1\+9\.1pp\) is on multi\-function single\-file tasks \(n=88n\{=\}88\), more than4×4\{\\times\}the\+2\.1\+2\.1pp gain on single\-function tasks \(n=341n\{=\}341\); multi\-file tasks \(n=71n\{=\}71\) remain hard for both checkpoints \(∼11\.3%\\sim\\\!11\.3\\%each\)\.

![Refer to caption](https://arxiv.org/html/2607.12463v1/x5.png)Figure 5:Pass rate on SWE\-Bench\-Verified \(1414B \+ R2E\-Gym\) stratified by gold\-patch shape, run\-means over three evaluation runs per checkpoint\.
### D\.2Full Trajectory Metrics

Table[9](https://arxiv.org/html/2607.12463#A4.T9)reports the full trajectory metrics summarized in Table[4](https://arxiv.org/html/2607.12463#S4.T4), including unsolved\-task breakdowns and output/context summaries\.

Table 9:Trajectory\-level behavioral metrics on SWE\-Bench\-Verified \(14B, R2E\-Gym\), run\-means over three evaluation runs of each checkpoint\. Steps, edits, and errors\-encountered are means per task\. “Empty patch” is the fraction of*failed*trajectories whose finaloutput\_patchis empty; “Loc\. correct” is the fraction of all trajectories that locate at least one file overlapping the gold patch; “Ctx @ success” is the mean prompt context length on solved trajectories\.Steps / taskEdits / taskErrors enc\.RecoverySettingsolvedunsolvedsolvedunsolvedsolvedunsolvedrate \(%\)\+ R2E\-Gym15\.122\.43\.35\.43\.55\.824\.8\+ FIM\-Midtrain \+ R2E\-Gym23\.632\.77\.410\.96\.28\.128\.8Output and context summariesEmpty patch \(% failed\)Loc\. correct \(% all\)Ctx @ success \(tok\)Pass \(%\)\+ R2E\-Gym3\.03\.070\.670\.611,82611\{,\}82626\.2\+ FIM\-Midtrain \+ R2E\-Gym0\.3\\mathbf\{0\.3\}73\.4\\mathbf\{73\.4\}𝟏𝟔,𝟑𝟗𝟕\\mathbf\{16\{,\}397\}29\.2
### D\.3Iterate\-and\-Verify: Action\-Type Distribution

Mid\-training shifts the action\-type distribution toward an iterate\-and\-verify policy\. The share ofsearchactions falls from15\.3%15\.3\\%to11\.0%11\.0\\%, while the share ofexecute\_bashactions rises from19\.5%19\.5\\%to24\.6%24\.6\\%: the agent spends a smaller fraction of its budget on passive repository lookup and a larger fraction on actively running scripts and tests, then refining its edit\. The cost is more steps—ours hits the step cap on∼45%\\sim\\\!45\\%of trajectories vs\.7%7\\%for the baseline—but this cost is paid on the unsolved tail; on the solved set the additional steps translate into more correct patches\.

### D\.4Failure\-Mode Breakdown

We label every failed trajectory using a deterministic patch\-quality cascade:*no\-patch*\(output diff is empty\);*localization error*\(non\-empty patch but zero file overlap with the gold patch\);*patch error*\(file overlap with gold patch but tests fail\)\. Figure[6](https://arxiv.org/html/2607.12463#A4.F6)reports the outcome distribution: across the three runs, mid\-training cuts no\-patch failures by an order of magnitude \(∼11→∼1\\sim\\\!11\\\!\\to\\\!\\sim\\\!1trajectories per run\), trims localization errors slightly \(∼131→∼126\\sim\\\!131\\\!\\to\\\!\\sim\\\!126\), and leaves the patch\-error count essentially flat \(∼227→∼227\\sim\\\!227\\\!\\to\\\!\\sim\\\!227\)\. The\+15\+15\-task gain in solved count is therefore primarily driven by the no\-patch mode collapse\.

![Refer to caption](https://arxiv.org/html/2607.12463v1/x6.png)Figure 6:Outcome distribution per evaluation run on SWE\-Bench\-Verified \(14B, R2E\-Gym\), averaged over three runs\.
### D\.5No\-Patch Mode: Mechanism

The baseline no\-patch failures recovered by mid\-training are distributed across all task buckets and are not concentrated in any single repository\. In every recovered case the baseline emitted<finish\>without applying anystr\_replace, while ours applied at least one edit\. We interpret this as a direct consequence of the FIM training signal: under FIM the model is always conditioned to produce a non\-empty token span between the prefix and suffix, and this disposition survives the post\-training pipeline\.

### D\.6Multi\-File Tasks Are Not Differentially Helped

On the7171Verified tasks whose gold patch spans≥2\\geq 2files, both checkpoints solve∼11\.3%\\sim\\\!11\.3\\%on average and per\-instance head\-to\-head on this bucket is balanced\. We attribute this to a granularity mismatch: our function\-aware FIM mid\-training operates at the function level within files, so cross\-file coordination is not directly trained for\. Extending the selection algorithm to cross\-file function pairs is a natural follow\-up\.

### D\.7A Concrete Contrast

The shift in termination behavior is clearest on cases where the baseline quits prematurely\. Onscikit\-learn\-26323, for instance, the baseline trajectory runs for a single step in every run, immediately emits<finish\>with an empty edit and the wrong target file, and is scored as a failure; on the same task our agent runs∼41\\sim\\\!41steps, applies on the order of2020str\_replaceoperations, observes multiple negative tool returns, and recovers to a passing patch\. Pooling across the three runs, on the order of∼16\\sim\\\!16of the∼55\\sim\\\!55tasks that ours uniquely solves on Verified have this signature: the baseline emits<finish\>with zero file overlap with the gold patch \(often before any negative observation has even arrived\), while ours iterates past intermediate signals and converges on a correct edit\. Mid\-training does not enable a categorically new capability on these instances; it changes the agent’s stopping policy\.

## Appendix EBehavioral Analysis on SWE\-Bench\-Lite

This appendix replicates the Verified analysis on the SWE\-Bench\-Lite test split \(300300instances\)\. The same trajectory\-parsing pipeline and failure\-mode taxonomy are used; statistics are run\-means over three independent evaluation runs of each checkpoint\.

The qualitative trends from Verified hold on Lite: mid\-training raises the recovery rate from18\.8%18\.8\\%to22\.1%22\.1\\%, eliminates the no\-patch failure mode entirely \(baseline averages∼8\\sim\\\!8no\-patch failures per run, ours averages0\), and increases edit iteration on solved trajectories from4\.44\.4to7\.87\.8str\_replaceoperations per task\. The*multi\-function*concentration of gain documented on Verified is not visible on Lite because Lite contains no multi\-file tasks and only5454multi\-function single\-file tasks; the bulk of Lite is single\-function single\-file tasks, on which the gain is\+4\.9\+4\.9pp \(19\.5%→24\.4%19\.5\\%\\\!\\to\\\!24\.4\\%\)\.

Table 10:Trajectory\-level behavioral metrics on SWE\-Bench\-Lite \(14B, R2E\-Gym\), run\-means over three evaluation runs of each checkpoint\. Definitions match Table[9](https://arxiv.org/html/2607.12463#A4.T9)\.Steps / taskEdits / taskErrors enc\.RecoverySettingsolvedunsolvedsolvedunsolvedsolvedunsolvedrate \(%\)\+ R2E\-Gym17\.521\.34\.45\.24\.85\.418\.8\+ FIM\-Midtrain \+ R2E\-Gym24\.631\.77\.811\.15\.77\.522\.1Output and context summariesEmpty patch \(% failed\)Loc\. correct \(% all\)Ctx @ success \(tok\)Pass \(%\)\+ R2E\-Gym3\.33\.365\.065\.013,95813\{,\}95818\.0\+ FIM\-Midtrain \+ R2E\-Gym0\.0\\mathbf\{0\.0\}71\.0\\mathbf\{71\.0\}𝟏𝟖,𝟏𝟎𝟒\\mathbf\{18\{,\}104\}22\.0Failure\-mode counts on Lite move in the same direction as on Verified: the no\-patch mode is eliminated \(∼8→0\\sim\\\!8\\\!\\to\\\!0trajectories per run\), localization errors fall by about1111, and patch errors rise by about77, reflecting the same iterate\-and\-verify shift\.

![Refer to caption](https://arxiv.org/html/2607.12463v1/x7.png)Figure 7:Pass rate on SWE\-Bench\-Lite stratified by gold\-patch shape, averaged over three evaluation runs per checkpoint\. Lite contains no multi\-file tasks; the multi\-function single\-file bucket is small \(n=54n\{=\}54\) and shows no gain on this slice, with the\+4\.0\+4\.0pp end\-task improvement coming entirely from the single\-function single\-file bucket \(n=246n\{=\}246,\+4\.9\+4\.9pp\)\.Per\-instance head\-to\-head\.Aggregating per\-task outcomes across the three runs \(run\-majority vote per checkpoint\), the differential favors ours on Lite by a∼1\.8×\\sim\\\!1\.8\\timesratio overall and by a∼2\.1×\\sim\\\!2\.1\\timesratio on the single\-function single\-file bucket\. On the multi\-function single\-file bucket the differential is balanced, consistent with the small absolute number of such tasks in Lite\.

## NeurIPS Paper Checklist

1. 1\.Claims
2. Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
3. Answer:\[Yes\]
4. Justification: The abstract and Section[1](https://arxiv.org/html/2607.12463#S1)state the function\-call/agent\-step correspondence, the function\-aware FIM mid\-training method, and the three\-axis robustness claim, each of which is supported by the experiments in Section[3\.2](https://arxiv.org/html/2607.12463#S3.SS2)\(Table[1](https://arxiv.org/html/2607.12463#S3.T1)\) and Section[3\.3](https://arxiv.org/html/2607.12463#S3.SS3)\(Table[2](https://arxiv.org/html/2607.12463#S3.T2)\)\.
5. Guidelines: - •The answer\[N/A\]means that the abstract and introduction do not include the claims made in the paper\. - •The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations\. A\[No\]or\[N/A\]answer to this question will not be perceived well by the reviewers\. - •The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings\. - •It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper\.
6. 2\.Limitations
7. Question: Does the paper discuss the limitations of the work performed by the authors?
8. Answer:\[Yes\]
9. Justification: Section[6](https://arxiv.org/html/2607.12463#S6)explicitly enumerates four limitations: Python\-only corpus and evaluation, teacher dependency for CoT generation, partial cross\-base validation \(only one non\-Qwen2\.5\-Coder configuration\), and the modularity assumption underlying function\-aware selection\.
10. Guidelines: - •The answer\[N/A\]means that the paper has no limitation while the answer\[No\]means that the paper has limitations, but those are not discussed in the paper\. - •The authors are encouraged to create a separate “Limitations” section in their paper\. - •The paper should point out any strong assumptions and how robust the results are to violations of these assumptions \(e\.g\., independence assumptions, noiseless settings, model well\-specification, asymptotic approximations only holding locally\)\. The authors should reflect on how these assumptions might be violated in practice and what the implications would be\. - •The authors should reflect on the scope of the claims made, e\.g\., if the approach was only tested on a few datasets or with a few runs\. In general, empirical results often depend on implicit assumptions, which should be articulated\. - •The authors should reflect on the factors that influence the performance of the approach\. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting\. Or a speech\-to\-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon\. - •The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size\. - •If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness\. - •While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper\. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community\. Reviewers will be specifically instructed to not penalize honesty concerning limitations\.
11. 3\.Theory assumptions and proofs
12. Question: For each theoretical result, does the paper provide the full set of assumptions and a complete \(and correct\) proof?
13. Answer:\[N/A\]
14. Justification: The paper is empirical and contains no theorems or formal theoretical results; the function\-call/agent\-step correspondence motivating the method is described as a structural analogy rather than a formal claim \(Section[2\.1](https://arxiv.org/html/2607.12463#S2.SS1)\)\.
15. Guidelines: - •The answer\[N/A\]means that the paper does not include theoretical results\. - •All the theorems, formulas, and proofs in the paper should be numbered and cross\-referenced\. - •All assumptions should be clearly stated or referenced in the statement of any theorems\. - •The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition\. - •Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material\. - •Theorems and Lemmas that the proof relies upon should be properly referenced\.
16. 4\.Experimental result reproducibility
17. Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper \(regardless of whether the code and data are provided or not\)?
18. Answer:\[Yes\]
19. Justification: Section[2](https://arxiv.org/html/2607.12463#S2)fully specifies the selection pipeline, scoring functions, and FIM format, with single\- and multi\-function selection algorithms, hyperparameters, and CoT prompt templates given in Appendix[B](https://arxiv.org/html/2607.12463#A2)and corpus statistics in Appendix[A](https://arxiv.org/html/2607.12463#A1); Section[3\.1](https://arxiv.org/html/2607.12463#S3.SS1)specifies models, post\-training pipelines \(R2E\-Gym, SWE\-Smith, SWE\-Lego\), evaluation protocol, and seeds\.
20. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •If the paper includes experiments, a\[No\]answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not\. - •If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable\. - •Depending on the contribution, reproducibility can be accomplished in various ways\. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model\. In general\. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model \(e\.g\., in the case of a large language model\), releasing of a model checkpoint, or other means that are appropriate to the research performed\. - •While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution\. For example 1. \(a\)If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm\. 2. \(b\)If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully\. 3. \(c\)If the contribution is a new model \(e\.g\., a large language model\), then there should either be a way to access this model for reproducing the results or a way to reproduce the model \(e\.g\., with an open\-source dataset or instructions for how to construct the dataset\)\. 4. \(d\)We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility\. In the case of closed\-source models, it may be that access to the model is limited in some way \(e\.g\., to registered users\), but it should be possible for other researchers to have some path to reproducing or verifying the results\.
21. 5\.Open access to data and code
22. Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
23. Answer:\[No\]
24. Justification: We will release the 968\-repository corpus and training code at camera\-ready time; we do not include them with this submission to preserve double\-blind anonymity\. Algorithm pseudocode for both single\- and multi\-function selection is in Appendix[B](https://arxiv.org/html/2607.12463#A2)\.
25. Guidelines: - •The answer\[N/A\]means that paper does not include experiments requiring code\. - • - •While we encourage the release of code and data, we understand that this might not be possible, so\[No\]is an acceptable answer\. Papers cannot be rejected simply for not including code, unless this is central to the contribution \(e\.g\., for a new open\-source benchmark\)\. - •The instructions should contain the exact command and environment needed to run to reproduce the results\. See the NeurIPS code and data submission guidelines \([https://neurips\.cc/public/guides/CodeSubmissionPolicy](https://neurips.cc/public/guides/CodeSubmissionPolicy)\) for more details\. - •The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc\. - •The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines\. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why\. - •At submission time, to preserve anonymity, the authors should release anonymized versions \(if applicable\)\. - •Providing as much information as possible in supplemental material \(appended to the paper\) is recommended, but including URLs to data and code is permitted\.
26. 6\.Experimental setting/details
27. Question: Does the paper specify all the training and test details \(e\.g\., data splits, hyperparameters, how they were chosen, type of optimizer\) necessary to understand the results?
28. Answer:\[Yes\]
29. Justification: Section[3\.1](https://arxiv.org/html/2607.12463#S3.SS1)describes the mid\-training and post\-training setup, benchmarks, evaluation protocol \(means over three seeds\), and agent harness; the full hyperparameter list and token budgets are in Appendix[C](https://arxiv.org/html/2607.12463#A3), and selection\-pipeline thresholds, weights, and filters are in Appendix[B](https://arxiv.org/html/2607.12463#A2)\.
30. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them\. - •The full details can be provided either with the code, in appendix, or as supplemental material\.
31. 7\.Experiment statistical significance
32. Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
33. Answer:\[Yes\]
34. Justification: Table[1](https://arxiv.org/html/2607.12463#S3.T1)reports means over three independent evaluation seeds with±\\pmstandard\-deviation bands on the main coding\-agent benchmarks \(SWE\-Bench\-Verified and SWE\-Bench\-Lite\), as described in the evaluation protocol in Section[3\.1](https://arxiv.org/html/2607.12463#S3.SS1)\.
35. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The authors should answer\[Yes\]if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper\. - •The factors of variability that the error bars are capturing should be clearly stated \(for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions\)\. - •The method for calculating the error bars should be explained \(closed form formula, call to a library function, bootstrap, etc\.\) - •The assumptions made should be given \(e\.g\., Normally distributed errors\)\. - •It should be clear whether the error bar is the standard deviation or the standard error of the mean\. - •It is OK to report 1\-sigma error bars, but one should state it\. The authors should preferably report a 2\-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified\. - •For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range \(e\.g\., negative error rates\)\. - •If error bars are reported in tables or plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text\.
36. 8\.Experiments compute resources
37. Question: For each experiment, does the paper provide sufficient information on the computer resources \(type of compute workers, memory, time of execution\) needed to reproduce the experiments?
38. Answer:\[Yes\]
39. Justification: Appendix[C\.1](https://arxiv.org/html/2607.12463#A3.SS1)reports the compute budget: all runs use a single node of88NVIDIA H100 80 GB GPUs, and reproducing the full set of experiments \(mid\-training of three base models, the three post\-training pipelines and their mid\-trained variants, the ablations, and the multi\-seed evaluation sweeps\) takes≈5,760\\approx\\\!5\{,\}760GPU\-hours \(≈30\\approx\\\!30days on the88\-GPU node\)\.
40. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage\. - •The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute\. - •The paper should disclose whether the full research project required more compute than the experiments reported in the paper \(e\.g\., preliminary or failed experiments that didn’t make it into the paper\)\.
41. 9\.Code of ethics
43. Answer:\[Yes\]
44. Justification: The work involves no human subjects, no PII, and uses only permissively\-licensed open\-source code \(license inventory in Appendix[A\.1](https://arxiv.org/html/2607.12463#A1.SS1)\); benchmarks used \(SWE\-Bench, LiveCodeBench, OJBench, FullStackBench\-EN, Terminal\-Bench 2\.0,τ\\tau\-bench, BFCL\) are publicly available academic resources\.
45. Guidelines: - •The answer\[N/A\]means that the authors have not reviewed the NeurIPS Code of Ethics\. - •If the authors answer\[No\], they should explain the special circumstances that require a deviation from the Code of Ethics\. - •The authors should make sure to preserve anonymity \(e\.g\., if there is a special consideration due to laws or regulations in their jurisdiction\)\.
46. 10\.Broader impacts
47. Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
48. Answer:\[No\]
49. Justification: The paper focuses on technical contributions and does not include a dedicated broader\-impacts section; the work is incremental over existing code LLMs and agentic post\-training pipelines, with positive impact from improved coding\-agent reliability and dual\-use risks comparable to those of the underlying base models, briefly touched on in Section[6](https://arxiv.org/html/2607.12463#S6)\.
50. Guidelines: - •The answer\[N/A\]means that there is no societal impact of the work performed\. - •If the authors answer\[N/A\]or\[No\], they should explain why their work has no societal impact or why the paper does not address societal impact\. - •Examples of negative societal impacts include potential malicious or unintended uses \(e\.g\., disinformation, generating fake profiles, surveillance\), fairness considerations \(e\.g\., deployment of technologies that could make decisions that unfairly impact specific groups\), privacy considerations, and security considerations\. - •The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments\. However, if there is a direct path to any negative applications, the authors should point it out\. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate Deepfakes for disinformation\. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster\. - •The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from \(intentional or unintentional\) misuse of the technology\. - •If there are negative societal impacts, the authors could also discuss possible mitigation strategies \(e\.g\., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML\)\.
51. 11\.Safeguards
52. Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse \(e\.g\., pre\-trained language models, image generators, or scraped datasets\)?
53. Answer:\[N/A\]
54. Justification: The released artifacts are a fine\-tuning corpus of permissively\-licensed open\-source Python code and code\-completion/agent fine\-tuned models that do not introduce generative capabilities for harmful content beyond what the underlying Qwen2\.5\-Coder/Qwen3 base models already provide; no scraped private data or high\-misuse\-risk content is released\.
55. Guidelines: - •The answer\[N/A\]means that the paper poses no such risks\. - •Released models that have a high risk for misuse or dual\-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters\. - •Datasets that have been scraped from the Internet could pose safety risks\. The authors should describe how they avoided releasing unsafe images\. - •We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort\.
56. 12\.Licenses for existing assets
57. Question: Are the creators or original owners of assets \(e\.g\., code, data, models\), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
58. Answer:\[Yes\]
59. Justification: All base models \(Qwen2\.5\-Coder, Qwen3\), post\-training pipelines \(R2E\-Gym, SWE\-Smith, SWE\-Lego, OpenHands, SWE\-agent\), and benchmarks \(SWE\-Bench, LiveCodeBench, OJBench, FullStackBench\-EN, Terminal\-Bench 2\.0,τ\\tau\-bench, BFCL\) are cited with their original publications, and the per\-repository license inventory of the 968\-repo corpus is in Appendix[A\.1](https://arxiv.org/html/2607.12463#A1.SS1)\.
60. Guidelines: - •The answer\[N/A\]means that the paper does not use existing assets\. - •The authors should cite the original paper that produced the code package or dataset\. - •The authors should state which version of the asset is used and, if possible, include a URL\. - •The name of the license \(e\.g\., CC\-BY 4\.0\) should be included for each asset\. - •For scraped data from a particular source \(e\.g\., website\), the copyright and terms of service of that source should be provided\. - •If assets are released, the license, copyright information, and terms of use in the package should be provided\. For popular datasets,[paperswithcode\.com/datasets](https://arxiv.org/html/2607.12463v1/paperswithcode.com/datasets)has curated licenses for some datasets\. Their licensing guide can help determine the license of a dataset\. - •For existing datasets that are re\-packaged, both the original license and the license of the derived asset \(if it has changed\) should be provided\. - •If this information is not available online, the authors are encouraged to reach out to the asset’s creators\.
61. 13\.New assets
62. Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
63. Answer:\[No\]
64. Justification: The 968\-repository decontaminated corpus, selection pipeline, and mid\-trained checkpoints are documented in Section[2\.2](https://arxiv.org/html/2607.12463#S2.SS2)and Appendix[A](https://arxiv.org/html/2607.12463#A1)\(corpus statistics, topic categories, licenses\) but are not released with this submission to preserve double\-blind anonymity; they will be released with full documentation at camera\-ready time\.
65. Guidelines: - •The answer\[N/A\]means that the paper does not release new assets\. - •Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates\. This includes details about training, license, limitations, etc\. - •The paper should discuss whether and how consent was obtained from people whose asset is used\. - •At submission time, remember to anonymize your assets \(if applicable\)\. You can either create an anonymized URL or include an anonymized zip file\.
66. 14\.Crowdsourcing and research with human subjects
67. Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation \(if any\)?
68. Answer:\[N/A\]
69. Justification: The paper involves no crowdsourcing or research with human subjects; the corpus is constructed from publicly available GitHub repositories and chain\-of\-thought rationales are generated by an LLM \(Gemini\-3\-Flash\), not by human annotators\.
70. Guidelines: - •The answer\[N/A\]means that the paper does not involve crowdsourcing nor research with human subjects\. - •Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper\. - •According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector\.
71. 15\.Institutional review board \(IRB\) approvals or equivalent for research with human subjects
72. Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board \(IRB\) approvals \(or an equivalent approval/review based on the requirements of your country or institution\) were obtained?
73. Answer:\[N/A\]
74. Justification: The paper does not involve research with human subjects, so IRB approval is not applicable\.
75. Guidelines: - •The answer\[N/A\]means that the paper does not involve crowdsourcing nor research with human subjects\. - •Depending on the country in which research is conducted, IRB approval \(or equivalent\) may be required for any human subjects research\. If you obtained IRB approval, you should clearly state this in the paper\. - •We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution\. - •For initial submissions, do not include any information that would break anonymity \(if applicable\), such as the institution conducting the review\.
76. 16\.Declaration of LLM usage
77. Question: Does the paper describe the usage of LLMs if it is an important, original, or non\-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does*not*impact the core methodology, scientific rigor, or originality of the research, declaration is not required\.
78. Answer:\[Yes\]
79. Justification: Section[2\.4](https://arxiv.org/html/2607.12463#S2.SS4)describes the use of Gemini\-3\-Flash to generate chain\-of\-thought rationales and Gemini\-3\-Flash to score\(CoT,completion\)\(\\text\{CoT\},\\text\{completion\}\)pairs for filtering, with the full prompt templates provided in Appendix[B\.9](https://arxiv.org/html/2607.12463#A2.SS9); LLMs are also used as the trained models themselves \(Qwen2\.5\-Coder, Qwen3\) per Section[3\.1](https://arxiv.org/html/2607.12463#S3.SS1)\.
80. Guidelines: - •The answer\[N/A\]means that the core method development in this research does not involve LLMs as any important, original, or non\-standard components\. - •Please refer to our LLM policy in the NeurIPS handbook for what should or should not be described\.