DHRCL:Training Code LLMs with Dense Hierarchical Rewards and Curriculum Learning

# DHRCL: Training Code LLMs with Dense Hierarchical Rewards and Curriculum Learning
Source: [https://arxiv.org/html/2607.26457](https://arxiv.org/html/2607.26457)
###### Abstract

Reinforcement learning is a natural post\-training paradigm for code\-oriented large language models because generated programs can be evaluated through parsing, execution, unit tests, and structural analysis\. However, existing methods often rely on sparse outcome rewards or statically combine heterogeneous dense signals, even though syntax validity, executability, functional correctness, and structural organization describe different and progressively dependent programming capabilities\. We proposeDHRCL, a reinforcement learning framework with Dense Hierarchical Rewards and Curriculum Learning\. DHRCL decomposes feedback into syntax validation, execution success, unit\-test pass rate, and AST\-based structural similarity, and organizes these signals through a three\-stage Syntax, Execution, Pass & Structural curriculum\. Stage duration is determined automatically from recent validation trends rather than manually specified capability thresholds\. We further introduce stage\-aware probability\-based token credit redistribution\. The mechanism follows a consolidation\-to\-refinement principle: it emphasizes established token patterns during syntax\-oriented optimization, applies uniform propagation for non\-local execution feedback, and allocates more credit or blame to less\-established token decisions during final functional optimization\. Under a unified Qwen3\-8B and KodCode protocol, the experiments compare DHRCL with binary, pass\-rate, reward\-model\-based, and verifiable dense\-reward baselines\. We further evaluate DHRCL across Qwen3\-4B, Qwen3\-8B, and Qwen3\-14B backbones, showing that its advantage remains consistent as model capacity increases\.

## Introduction

Large language models \(LLMs\) have made code generation a central application of modern language models\(Chenet al\.[2021](https://arxiv.org/html/2607.26457#bib.bib32); Rozièreet al\.[2023](https://arxiv.org/html/2607.26457#bib.bib33)\), while realistic benchmarks increasingly emphasize execution\-based validation\(Zhuoet al\.[2025](https://arxiv.org/html/2607.26457#bib.bib34); Wanget al\.[2023](https://arxiv.org/html/2607.26457#bib.bib35)\)\. Code is well suited to reinforcement learning \(RL\) because generated programs admit automatically verifiable feedback from parsing, execution, tests, and structural analysis\. Prior work exploits compiler and unit\-test outcomes\(Leet al\.[2022](https://arxiv.org/html/2607.26457#bib.bib3); Shojaeeet al\.[2023](https://arxiv.org/html/2607.26457#bib.bib4); Liuet al\.[2023a](https://arxiv.org/html/2607.26457#bib.bib5)\), but sparse rewards provide little guidance when most samples fail, while static combinations of dense rewards ignore the prerequisite relations among syntax validity, executability, and functional correctness\.

We proposeDHRCL, a code\-LLM RL framework withDenseHierarchicalRewards andCurriculumLearning\. DHRCL combines syntax validation, execution success, unit\-test pass rate, and AST\-based structural similarity in a Syntax, Execution, Pass & Structural curriculum\. Stage duration is selected automatically from held\-out validation trends rather than manually chosen capability thresholds\. DHRCL also redistributes trajectory\-level credit across tokens according to a consolidation\-to\-refinement principle: confidence\-oriented weighting in the syntax stage, uniform weighting for non\-local execution feedback, and uncertainty\-oriented weighting in the final stage\.

Our contributions are:

- •A three\-stage hierarchical reward curriculum coordinating syntax, execution, functional, and structural feedback while retaining functional correctness as the primary objective\.
- •Trend\-based automatic stage progression that adapts stage duration without fixed syntax or execution thresholds\.
- •Stage\-aware token credit redistribution with controlled ablations across reward composition, curriculum scheduling, and token weighting\.

## Related Work

### RL\-Based Optimization for Code LLMs

RL has become a prominent post\-training paradigm for aligning code LLMs with sequence\-level functional correctness\(Wanget al\.[2024](https://arxiv.org/html/2607.26457#bib.bib1); Jianget al\.[2026](https://arxiv.org/html/2607.26457#bib.bib2)\)\. Early policy\-gradient methods such as REINFORCE\(Williams[1992](https://arxiv.org/html/2607.26457#bib.bib9)\)enable optimization beyond token likelihood, and have been adapted to code generation through actor\-critic and PPO\-style frameworks\. CodeRL\(Leet al\.[2022](https://arxiv.org/html/2607.26457#bib.bib3)\)uses execution feedback in an actor\-critic setting, while PPOCoder\(Shojaeeet al\.[2023](https://arxiv.org/html/2607.26457#bib.bib4)\), RLTF\(Liuet al\.[2023a](https://arxiv.org/html/2607.26457#bib.bib5)\), and RLEF\(Gehringet al\.[2025](https://arxiv.org/html/2607.26457#bib.bib12)\)extend PPO\(Schulmanet al\.[2017](https://arxiv.org/html/2607.26457#bib.bib8)\)to compiler\-guided, online, and iterative feedback scenarios\. Related preference\- and group\-based methods further improve scalable post\-training: DPO\(Rafailovet al\.[2023](https://arxiv.org/html/2607.26457#bib.bib13)\)and Focused\-DPO\(Zhanget al\.[2025b](https://arxiv.org/html/2607.26457#bib.bib14)\)avoid explicit critic learning, while GRPO\(Shaoet al\.[2024](https://arxiv.org/html/2607.26457#bib.bib6); DeepSeek\-AI[2025](https://arxiv.org/html/2607.26457#bib.bib7)\), DAPO\(Yuet al\.[2025](https://arxiv.org/html/2607.26457#bib.bib10)\), and GSPO\(Zhenget al\.[2025](https://arxiv.org/html/2607.26457#bib.bib15)\)improve critic\-free or sequence\-level policy optimization\. These methods strengthen the RL optimization backbone, whereas our work focuses on how code\-specific rewards are constructed and organized\.

### Code\-Specific Reward Design

Reward design is central to RL\-based code generation because programs can be verified through compilation, execution, tests, and structural analysis\. Existing rewards include verifiable execution or compiler feedback\(Leet al\.[2022](https://arxiv.org/html/2607.26457#bib.bib3); Shojaeeet al\.[2023](https://arxiv.org/html/2607.26457#bib.bib4); Liuet al\.[2023a](https://arxiv.org/html/2607.26457#bib.bib5); Gehringet al\.[2025](https://arxiv.org/html/2607.26457#bib.bib12)\), structured or dense signals such as AST/DFG consistency, partial test success, and task\-specific code similarity\(Zhaoet al\.[2025](https://arxiv.org/html/2607.26457#bib.bib16); Wanget al\.[2026b](https://arxiv.org/html/2607.26457#bib.bib17); Wuet al\.[2026](https://arxiv.org/html/2607.26457#bib.bib18)\), as well as learned reward models and process\-level supervision\(Zenget al\.[2025](https://arxiv.org/html/2607.26457#bib.bib19); Daiet al\.[2024](https://arxiv.org/html/2607.26457#bib.bib20); Yeet al\.[2025](https://arxiv.org/html/2607.26457#bib.bib21)\)\. Recent preference\- and test\-construction methods further refine supervision through test\-based preferences or adversarial test generation\(Zhanget al\.[2024](https://arxiv.org/html/2607.26457#bib.bib25),[2025a](https://arxiv.org/html/2607.26457#bib.bib24); Wuet al\.[2025](https://arxiv.org/html/2607.26457#bib.bib22); Wanget al\.[2026a](https://arxiv.org/html/2607.26457#bib.bib23)\)\. Overall, these studies enrich code rewards with verifiable, dense, local, or adaptive feedback, but heterogeneous objectives are usually combined statically\. This leaves stage\-wise reward scheduling across syntax, execution, tests, and structure underexplored\.

### Curriculum Learning for Code LLMs

Curriculum learning organizes optimization from easier to harder learning units\. In code generation, prior work has explored curricula over completion subtasks, data complexity, code blocks, and test cases\. StepCoder\(Douet al\.[2024](https://arxiv.org/html/2607.26457#bib.bib26)\)decomposes full program synthesis into code\-completion subtasks with fine\-grained optimization; data\-level curricula order training samples by code complexity to improve small code LMs\(Naïret al\.[2024](https://arxiv.org/html/2607.26457#bib.bib28)\); AST\-guided block splitting improves fill\-in\-the\-middle DPO alignment\(Renet al\.[2025](https://arxiv.org/html/2607.26457#bib.bib27)\); and TAROT\(Parket al\.[2026](https://arxiv.org/html/2607.26457#bib.bib29)\)constructs tiered test suites with capability\-adaptive curriculum policies\. These methods mainly schedule training units rather than heterogeneous reward objectives\. In contrast, DHRCL organizes syntax, execution, functional, and structural rewards into a stage\-aware curriculum and further propagates them with stage\-aware token credit redistribution\.

## Method

![Refer to caption](https://arxiv.org/html/2607.26457v1/Figures/pipeline.jpg)Figure 1:Overall pipeline of DHRCL\.We proposeDHRCL, a reinforcement learning framework for code\-oriented LLM post\-training\. As shown in Figure[1](https://arxiv.org/html/2607.26457#Sx3.F1), DHRCL contains two coupled components\. First, a dense hierarchical reward decomposes code feedback into syntax validity, execution validity, partial functional correctness, and AST\-based structural alignment\. Second, a stage\-wise curriculum changes the primary optimization emphasis according to the policy’s current bottleneck, while probability\-aware token credit redistribution controls how each trajectory\-level update is allocated across generated tokens\. The predefined capability order is Syntax, Execution, Pass & Structural, whereas each stage duration is determined automatically from held\-out validation trends\.

### Dense Hierarchical Reward

Outcome\-level rewards such as binary test success or aggregate pass rate often cannot distinguish locally malformed code, runtime failures, and partially correct executable programs\. DHRCL therefore combines four complementary reward components\. The first three directly describe program validity and functional behavior\. The fourth is an auxiliary, reference\-conditioned structural signal; it is not treated as semantic equivalence or as a complete measure of code quality\.

#### Syntax Validation Rate

We use a parser\-based syntax reward to provide partial feedback for locally well\-formed code\. Given a generated programyy, an error\-tolerant parser extracts syntactic unitsℬ​\(y\)=\{bi\}i=1m\\mathcal\{B\}\(y\)=\\\{b\_\{i\}\\\}\_\{i=1\}^\{m\}\. A unit is valid if its subtree contains no parser error node, and the reward is

rsyntax​\(y\)=∑i=1mwi​vi∑i=1mwi,r\_\{\\mathrm\{syntax\}\}\(y\)=\\frac\{\\sum\_\{i=1\}^\{m\}w\_\{i\}v\_\{i\}\}\{\\sum\_\{i=1\}^\{m\}w\_\{i\}\},whereviv\_\{i\}is the validity indicator andwiw\_\{i\}is the token length\. We setrsyntax​\(y\)=0r\_\{\\mathrm\{syntax\}\}\(y\)=0if no unit is extracted\. Implementation details are provided in the Supplementary Document, Section “Implementation Details of Syntax Reward\.”

#### Execution Success

While syntax validity ensures that a program can be parsed at least locally, it does not guarantee successful execution\. Generated code may still trigger runtime errors, exceed time limits, or cause memory\-related failures\. To capture runtime validity, we introduce an execution success reward:

rexec=𝕀​\(execution success\),r\_\{\\text\{exec\}\}=\\mathbb\{I\}\(\\text\{execution success\}\),where𝕀​\(⋅\)\\mathbb\{I\}\(\\cdot\)denotes the indicator function\. The reward is 1 if the generated program executes successfully under the evaluation environment, and 0 otherwise\. This component encourages the model to move beyond syntactic well\-formedness and learn programs with valid runtime behavior\.

#### Pass Rate

The final goal of code generation is to solve the programming problem, which is typically evaluated by unit tests\. Instead of using a binary all\-or\-nothing reward, we use the fraction of passed test cases as a dense functional correctness signal:

rpass=npassntest,r\_\{\\text\{pass\}\}=\\frac\{n\_\{\\text\{pass\}\}\}\{n\_\{\\text\{test\}\}\},wherenpassn\_\{\\text\{pass\}\}is the number of passed test cases andntestn\_\{\\text\{test\}\}is the total number of test cases\. This reward differentiates partially correct solutions and provides a smoother optimization signal than binary test success\.

#### AST\-Based Structural Similarity

Unit\-test feedback evaluates functional behavior but does not distinguish the structural organization of generated programs\. We therefore introduce an auxiliary reference\-conditioned AST reward\. For a generated programyyand a verified reference solutionyrefy^\{\\mathrm\{ref\}\}, let𝒩​\(AST​\(⋅\)\)\\mathcal\{N\}\(\\mathrm\{AST\}\(\\cdot\)\)denote the structural normalization used by our implementation\. For programs that can be strictly parsed, we compute

rAST​\(y\)=SimAST​\(𝒩​\(AST​\(y\)\),𝒩​\(AST​\(yref\)\)\),r\_\{\\mathrm\{AST\}\}\(y\)=\\mathrm\{Sim\}\_\{\\mathrm\{AST\}\}\\left\(\\mathcal\{N\}\(\\mathrm\{AST\}\(y\)\),\\mathcal\{N\}\(\\mathrm\{AST\}\(y^\{\\mathrm\{ref\}\}\)\)\\right\),whereSimAST∈\[0,1\]\\mathrm\{Sim\}\_\{\\mathrm\{AST\}\}\\in\[0,1\]is the implemented AST\-similarity function\. If the generated program cannot be strictly parsed, we setrAST​\(y\)=0r\_\{\\mathrm\{AST\}\}\(y\)=0\. Implementation details of the AST structural reward are provided in the Supplementary Document, Section “Implementation Details of the AST Structural Reward\.”

AST similarity is not interpreted as semantic equivalence, maintainability, or computational efficiency\. Instead, it supplies a weak structural prior derived from a verified implementation\. Functional correctness remains directly supervised by the unit\-test pass\-rate reward, while the AST signal encourages exploration of structurally coherent implementation patterns\. The reward\-component ablation compares the full formulation against an otherwise identical variant withoutrASTr\_\{\\mathrm\{AST\}\}, thereby isolating its contribution to Pass@1, structural similarity, and the observed characteristics of accepted outputs\.

### Curriculum Learning for Code Generation

Although the above rewards provide dense supervision, combining them with fixed weights ignores prerequisite relations among programming capabilities\. DHRCL uses three consecutive stages: Syntax, Execution, and Pass & Structural\. The curriculum changes the primary optimization emphasis rather than completely enabling or disabling a reward\. In particular,rpassr\_\{\\mathrm\{pass\}\}remains active throughout training because functional correctness is always the final task objective, while previously introduced validity signals remain auxiliary components in later stages\.

#### Syntax Stage

For theii\-th sampled trajectory, the Syntax\-stage reward is

ri=rpass,i\+rsyntax,i\+αsyntax​rexec,i\+αsyntax​rAST,i,r\_\{i\}=r\_\{\\mathrm\{pass\},i\}\+r\_\{\\mathrm\{syntax\},i\}\+\\alpha\_\{\\mathrm\{syntax\}\}r\_\{\\mathrm\{exec\},i\}\+\\alpha\_\{\\mathrm\{syntax\}\}r\_\{\\mathrm\{AST\},i\},where

αsyntax=1n​∑i=1nrsyntax,i\.\\alpha\_\{\\mathrm\{syntax\}\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}r\_\{\\mathrm\{syntax\},i\}\.Here,nnis the number of trajectories in the rollout batch\. The batch\-level coefficient acts as a competence\-dependent gate\. When syntax validity is weak, downstream execution and structural signals receive less weight; as syntax validity improves, they are introduced more strongly\. The pass\-rate reward is retained without attenuation\.

#### Execution Stage

After syntax\-oriented optimization, the primary emphasis shifts to executable behavior:

ri=rpass,i\+rexec,i\+αexec​rsyntax,i\+αexec​rAST,i,r\_\{i\}=r\_\{\\mathrm\{pass\},i\}\+r\_\{\\mathrm\{exec\},i\}\+\\alpha\_\{\\mathrm\{exec\}\}r\_\{\\mathrm\{syntax\},i\}\+\\alpha\_\{\\mathrm\{exec\}\}r\_\{\\mathrm\{AST\},i\},with

αexec=1n​∑i=1nrexec,i\.\\alpha\_\{\\mathrm\{exec\}\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}r\_\{\\mathrm\{exec\},i\}\.Runtime validity often depends on interactions among definitions, control flow, data types, API calls, and memory or time constraints\. This stage therefore prioritizes whole\-program execution feedback while retaining syntax and structural signals as auxiliary terms\.

#### Pass & Structural Stage

In the final stage, the primary objective combines functional correctness with the auxiliary structural prior:

ri=rpass,i\+rAST,i\+αpass​rsyntax,i\+αpass​rexec,i,r\_\{i\}=r\_\{\\mathrm\{pass\},i\}\+r\_\{\\mathrm\{AST\},i\}\+\\alpha\_\{\\mathrm\{pass\}\}r\_\{\\mathrm\{syntax\},i\}\+\\alpha\_\{\\mathrm\{pass\}\}r\_\{\\mathrm\{exec\},i\},where

αpass=1n​∑i=1nrpass,i\.\\alpha\_\{\\mathrm\{pass\}\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}r\_\{\\mathrm\{pass\},i\}\.The pass\-rate term is the direct functional objective\. The AST term is an auxiliary reference\-conditioned signal and does not replace execution\-based correctness\. Final checkpoint selection is based on validation Pass@1, with AST similarity used only as a tie\-breaker between checkpoints with identical validation Pass@1\.

#### Trend\-Based Automatic Stage Progression

Fixed absolute thresholds can immediately advance a strong initialization or indefinitely retain a weak one, even when both policies exhibit meaningful learning progress\. DHRCL instead determines stage transitions from the recent trend of the primary held\-out validation metric\. Letms\(k\)m\_\{s\}^\{\(k\)\}be the primary validation metric of stagessat thekk\-th evaluation, wherekkindexes validation observations:

ms\(k\)=\{rsyntaxval,\(k\),s=Syntax,rexecval,\(k\),s=Execution,Pass​@​1val\(k\),s=Pass & Structural\.m\_\{s\}^\{\(k\)\}=\\begin\{cases\}r\_\{\\mathrm\{syntax\}\}^\{\\mathrm\{val\},\(k\)\},&s=\\text\{Syntax\},\\\\ r\_\{\\mathrm\{exec\}\}^\{\\mathrm\{val\},\(k\)\},&s=\\text\{Execution\},\\\\ \\mathrm\{Pass@1\}\_\{\\mathrm\{val\}\}^\{\(k\)\},&s=\\text\{Pass \\& Structural\}\.\\end\{cases\}After at least 12 validation observations have been collected in the current stage, we use the most recent eight observations,

ℋs\(k\)=\{ms\(k−7\),…,ms\(k\)\},\\mathcal\{H\}\_\{s\}^\{\(k\)\}=\\left\\\{m\_\{s\}^\{\(k\-7\)\},\\ldots,m\_\{s\}^\{\(k\)\}\\right\\\},and fit the linear model

ms\(t\)=as\+bs​t\+ϵt\.m\_\{s\}^\{\(t\)\}=a\_\{s\}\+b\_\{s\}t\+\\epsilon\_\{t\}\.We perform a one\-sided test of

H0:bs≤0againstH1:bs\>0\.H\_\{0\}:b\_\{s\}\\leq 0\\qquad\\text\{against\}\\qquad H\_\{1\}:b\_\{s\}\>0\.The policy remains in the current stage while the recent window supports a positive trend\. A transition is triggered only after the positive\-trend test fails at two consecutive evaluations, which reduces sensitivity to a single noisy validation point\. We then restore the checkpoint with the highest primary validation metric observed in that stage and advance to the next stage\. The same rule is applied to validation Pass@1 in the final stage for early stopping\.

#### Stage\-Aware Probability\-Based Token Credit Redistribution

Group\-based RL assigns a trajectory\-level advantageAiA\_\{i\}to all generated tokens in trajectoryτi\\tau\_\{i\}\. The unweighted token contribution is

gi,j=Ai​∇θlog⁡πθ​\(yi,j∣xi,yi,<j\)\.g\_\{i,j\}=A\_\{i\}\\nabla\_\{\\theta\}\\log\\pi\_\{\\theta\}\\left\(y\_\{i,j\}\\mid x\_\{i\},y\_\{i,<j\}\\right\)\.DHRCL redistributes this trajectory\-level update across tokens:

gi,j′=βi,j​gi,j\.g^\{\\prime\}\_\{i,j\}=\\beta\_\{i,j\}g\_\{i,j\}\.Let

pi,j=πθold​\(yi,j∣xi,yi,<j\)p\_\{i,j\}=\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\\left\(y\_\{i,j\}\\mid x\_\{i\},y\_\{i,<j\}\\right\)be the detached rollout\-policy probability of tokenyi,jy\_\{i,j\}\. Token probability is interpreted as how established a decision is under the current policy, not as a token\-correctness label\.

During the Syntax Stage, we use confidence\-oriented weighting,

βi,jconf=pi,j1\|τi\|​∑k=1\|τi\|pi,k\.\\beta\_\{i,j\}^\{\\mathrm\{conf\}\}=\\frac\{p\_\{i,j\}\}\{\\frac\{1\}\{\|\\tau\_\{i\}\|\}\\sum\_\{k=1\}^\{\|\\tau\_\{i\}\|\}p\_\{i,k\}\}\.For positive syntax advantages, this consolidates recurrent valid patterns; for negative advantages, it more strongly suppresses recurrent high\-probability patterns associated with syntax failure\. The goal is to reduce the influence of incidental low\-probability samples during early syntax\-oriented optimization\.

During the Execution Stage, we use

βi,jexec=1\.\\beta\_\{i,j\}^\{\\mathrm\{exec\}\}=1\.Execution validity frequently depends on non\-local interactions, and token probability alone does not reliably localize runtime responsibility\. Uniform weighting therefore avoids introducing an unsupported local bias\.

During the Pass & Structural Stage, we use uncertainty\-oriented weighting,

βi,junc=1−pi,j1\|τi\|​∑k=1\|τi\|\(1−pi,k\)\.\\beta\_\{i,j\}^\{\\mathrm\{unc\}\}=\\frac\{1\-p\_\{i,j\}\}\{\\frac\{1\}\{\|\\tau\_\{i\}\|\}\\sum\_\{k=1\}^\{\|\\tau\_\{i\}\|\}\(1\-p\_\{i,k\}\)\}\.At this stage, the optimization bottleneck shifts toward decisions that remain less established under the current policy\. The weighting does not indiscriminately increase their probability: the sign ofAiA\_\{i\}determines whether a low\-probability decision receives stronger positive credit or stronger negative blame\.

All three weighting rules have trajectory\-level mean one:

1\|τi\|​∑j=1\|τi\|βi,j=1\.\\frac\{1\}\{\|\\tau\_\{i\}\|\}\\sum\_\{j=1\}^\{\|\\tau\_\{i\}\|\}\\beta\_\{i,j\}=1\.The mechanism therefore preserves the average contribution of a trajectory while changing how its gradient budget is allocated across generated tokens\. Overall, the design follows a consolidation\-to\-refinement principle: confidence\-oriented consolidation in the syntax stage, uniform propagation for non\-local execution feedback, and uncertainty\-oriented refinement in the final stage\. Trajectory rewards are normalized within each prompt\-level rollout group before advantage estimation\.

## Experiments

We evaluate DHRCL under a controlled Qwen3\-8B and KodCode setting and additionally examine scale robustness using Qwen3\-4B, Qwen3\-8B, and Qwen3\-14B backbones\.

### Experimental Setup

All trainable methods are initialized from the corresponding backbone checkpoint and trained on KodCode\(Xuet al\.[2025](https://arxiv.org/html/2607.26457#bib.bib40)\)under a matched RL protocol\. The detailed ablations use Qwen3\-8B\(Team[2025](https://arxiv.org/html/2607.26457#bib.bib39)\)\. The cross\-scale comparison uses Qwen3\-4B, Qwen3\-8B, and Qwen3\-14B, which share the same dense Transformer family and tokenizer while differing in capacity\. A held\-out KodCode validation split is used exclusively for trend\-based stage progression, early stopping, and checkpoint selection\. Final evaluation uses HumanEval, HumanEval\+, BigCodeBench\-Full, BigCodeBench\-Hard, LiveCodeBench V6, and CodeElo\(Chenet al\.[2021](https://arxiv.org/html/2607.26457#bib.bib32); Liuet al\.[2023b](https://arxiv.org/html/2607.26457#bib.bib41); Zhuoet al\.[2025](https://arxiv.org/html/2607.26457#bib.bib34); Jainet al\.[2025](https://arxiv.org/html/2607.26457#bib.bib42); Quanet al\.[2025](https://arxiv.org/html/2607.26457#bib.bib43)\)\. Reference solutions are used only by the AST\-based reward and are never included in rollout prompts\. All trainable configurations are repeated with three random seeds\. Tables report mean±\\pmstandard deviation, and pairwise method differences are additionally assessed with problem\-level paired bootstrap confidence intervals\.

We compare DHRCL with GRPO, GRPO\-PassRate, AceCoder, and VeRPO\(Shaoet al\.[2024](https://arxiv.org/html/2607.26457#bib.bib6); Zenget al\.[2025](https://arxiv.org/html/2607.26457#bib.bib19); Wanget al\.[2026b](https://arxiv.org/html/2607.26457#bib.bib17)\)\. Each method retains its core reward formulation while using the same policy initialization, training corpus, rollout budget, optimization budget, decoding configuration, and execution sandbox at a given backbone scale\.

Pass@1 is the primary functional metric\. Syntax Acc\. and Exec Acc\. describe intermediate program validity\. AST Sim\., normalized execution time \(NET\), and normalized test memory usage \(NTMU\) are complementary diagnostics for accepted outputs\. Because NET and NTMU are computed over each method’s own accepted subset, they are descriptive statistics rather than strictly paired efficiency comparisons\. For training\-process analysis, we report Step Time, GPU Hours, Iterations to Target, Avg\. DGR, and Reward Var\. Metric definitions and additional implementation details are provided in the Supplementary Document, Sections “Program Validity and Accepted\-Output Diagnostics” and “Training\-Efficiency and Reward\-Signal Metrics\.”

### Comparison Experiments

Table 1:Main comparison under the unified Qwen3\-8B and KodCode setting\. Trainable results are mean±\\pmstandard deviation over three seeds\.Table 2:Training\-efficiency and reward\-signal statistics on Qwen3\-8B\. Values are mean±\\pmstandard deviation over three seeds\.Table 3:Cross\-scale Pass@1 comparison across Qwen3\-4B, Qwen3\-8B, and Qwen3\-14B\. Trainable results are mean±\\pmstandard deviation over three seeds\.Table[1](https://arxiv.org/html/2607.26457#Sx4.T1)reports the controlled Qwen3\-8B comparison\. DHRCL achieves an average Pass@1 of50\.3±0\.250\.3\\pm 0\.2, outperforming VeRPO, the strongest baseline, by 1\.1 points\. A problem\-level paired bootstrap comparison gives a 95% confidence interval of\[0\.6,1\.5\]\[0\.6,1\.5\]for this average improvement\. The gain is accompanied by higher syntax and execution validity, with Syntax Acc\. and Exec Acc\. reaching92\.8±0\.292\.8\\pm 0\.2and82\.7±0\.382\.7\\pm 0\.3, respectively\. DHRCL also obtains the highest mean AST similarity, while its accepted outputs exhibit lower average NET and NTMU under the adopted evaluation protocol\.

Because all baselines are retrained under the unified Qwen3\-8B and KodCode protocol, the comparison reflects their adapted performance under matched data, model, and training budgets rather than their originally reported absolute results\. Table[3](https://arxiv.org/html/2607.26457#Sx4.T3)shows that DHRCL retains the highest average Pass@1 across Qwen3\-4B, Qwen3\-8B, and Qwen3\-14B\. Its average improvement over VeRPO is 1\.3, 1\.1, and 0\.7 points, with paired\-bootstrap 95% confidence intervals of\[0\.8,1\.8\]\[0\.8,1\.8\],\[0\.6,1\.5\]\[0\.6,1\.5\], and\[0\.3,1\.2\]\[0\.3,1\.2\], respectively\. The observed margin narrows with model capacity while remaining positive under the reported paired\-bootstrap analysis\. Results are not uniformly dominant on every benchmark: for Qwen3\-14B, VeRPO is marginally higher on HumanEval, while DHRCL retains larger gains on BigCodeBench\-Hard, LiveCodeBench, and CodeElo\.

### Ablation Studies

This subsection isolates the three main components of DHRCL: reward composition, curriculum scheduling, and probability\-aware token credit redistribution\. Table[4](https://arxiv.org/html/2607.26457#Sx4.T4)summarizes controlled ablations for reward composition, curriculum scheduling, and probability\-aware token credit redistribution\.

Table 4:Controlled ablations on Qwen3\-8B\. Avg\. Pass@1 is reported as mean±\\pmstandard deviation over three seeds\.##### Reward hierarchy\.

For the reward\-component ablation, the curriculum rule, probability\-aware token weighting, optimizer, rollout budget, and training budget are held fixed; only the included reward components change\. The comparison betweenPass \+ Syntax \+ ExecandFull Rewardisolates the AST\-based structural reward\. Adding the AST reward increases average Pass@1 from 49\.7 to 50\.3 and AST similarity from 0\.62 to 0\.68\. We regard the structural\-alignment gain as its primary effect, while the smaller Pass@1 improvement suggests a secondary functional benefit\. NET and NTMU are reported as descriptive statistics of accepted outputs because the solved\-problem subsets differ across variants\.

##### Curriculum scheduling\.

All curriculum variants use the same full reward, probability\-aware weighting, optimizer, rollout budget, and training budget\. They differ only in reward scheduling and stage progression\.Fixed Sumjointly optimizes all components from the beginning,Fixed\-stagechanges stages after predefined training iterations, and DHRCL uses the recent validation trend\. This comparison isolates whether automatically adapting stage duration is preferable to static aggregation or a fixed schedule\.

##### Probability\-aware token credit redistribution\.

All token\-weighting variants use the same hierarchical reward and trend\-based curriculum\. They differ only in how the trajectory\-level gradient budget is distributed among generated tokens\. No Weighting uses uniform propagation in all stages; Early\-only and Late\-only apply one side of the proposed ordering; and Reversed applies uncertainty weighting early and confidence weighting late\. The full method evaluates the proposed consolidation\-to\-uniform\-to\-refinement ordering\. The comparison therefore isolates whether stage\-dependent token weighting improves the allocation of the trajectory\-level gradient budget\.

### Further Analysis

We use Qwen3\-8B for detailed diagnostics because it is the primary ablation setting; cross\-scale effectiveness is established in Table[3](https://arxiv.org/html/2607.26457#Sx4.T3)\.

![Refer to caption](https://arxiv.org/html/2607.26457v1/Figures/training_dynamics.png)Figure 2:Training dynamics of DHRCL on Qwen3\-8B\.##### Training dynamics\.

Figure[2](https://arxiv.org/html/2607.26457#Sx4.F2)shows the evolution of held\-out validation metrics throughout DHRCL training on Qwen3\-8B\. Syntax Accuracy improves most rapidly in the first stage, Execution Accuracy rises more clearly after the first transition, and average Pass@1 and AST Similarity obtain their largest gains after the second transition\. Raw measurements fluctuate because of rollout and optimization stochasticity, whereas the moving averages reveal the longer\-term stage\-wise progression\. Across three seeds, the median transition iterations are 140/280 for Qwen3\-8B, 180/340 for Qwen3\-4B, and 120/260 for Qwen3\-14B, indicating that the selected stage duration varies with model scale\.

![Refer to caption](https://arxiv.org/html/2607.26457v1/Figures/dgr_curve.png)Figure 3:Degenerate group ratio on Qwen3\-8B\.
##### Reward\-signal discriminability\.

DGR measures how often a rollout group receives nearly identical rewards and therefore little relative\-advantage information\. As shown in Figure[3](https://arxiv.org/html/2607.26457#Sx4.F3), binary GRPO has the highest DGR, partial\-test and verifiable rewards reduce it, and AceCoder is lowest because its learned reward model produces continuously varying scores\. DHRCL achieves the second\-lowest DGR without an external reward model, indicating that its hierarchical feedback distinguishes more candidates within each rollout group than the binary, pass\-rate, and verifiable dense\-reward alternatives considered here\.

Additional error\-type breakdowns are provided in the Supplementary Document, Section “Error\-Type Analysis\.” They show that DHRCL reduces invalid programs while executable\-but\-incorrect solutions remain the dominant residual failure mode\.

## Conclusion

We introduced DHRCL, which coordinates syntax, execution, test, and AST\-based structural feedback through a trend\-adaptive curriculum and stage\-aware token credit redistribution\. Controlled Qwen3\-8B experiments and a Qwen3\-4B/8B/14B scale study show consistent gains in functional correctness, intermediate validity, structural alignment, reward discriminability, and convergence efficiency\.

## Limitations

AST similarity is an auxiliary reference\-conditioned signal rather than semantic equivalence; token probability does not identify semantic correctness; the three\-stage capability order remains manually specified; and the scale study is limited to the Qwen3 family\. NET and NTMU are descriptive statistics over each method’s accepted subset\. Broader cross\-family and cross\-corpus validation remains future work\.

## Supplementary Material

## Overview

This supplementary document provides additional implementation details, evaluation protocols, metric definitions, and extended ablation results\.

## Appendix ABaseline Adaptation Protocol

All trainable baselines are retrained under a matched protocol rather than evaluated using released policy checkpoints\. At each backbone scale, GRPO, GRPO\-PassRate, VeRPO, AceCoder, and DHRCL use the same policy initialization, KodCode training split, rollout budget, optimization budget, decoding configuration, execution sandbox, and checkpoint\-selection protocol\. The reported results therefore characterize these unified adaptations rather than reproducing the original papers’ absolute performance\.

Each baseline retains its defining reward formulation\. GRPO uses binary complete\-test outcomes, GRPO\-PassRate uses the fraction of passed test cases, VeRPO uses its difficulty\-aware verifiable reward and policy objective, and AceCoder uses the released AceCodeRM as an external reward model\. Baseline\-specific hyperparameters and reward normalization are selected under the same held\-out validation protocol and comparable tuning budgets\.

## Appendix BTraining Configuration

Experiments are conducted on 8 NVIDIA A100 80GB GPUs for the Qwen3\-8B setting\. The Qwen3\-4B experiments use 4 NVIDIA A100 80GB GPUs, while the Qwen3\-14B experiments use 16 NVIDIA A100 80GB GPUs\. Each rollout batch contains 32 programming problems with 8 candidate solutions per problem\. Training\-time generation uses temperatureT=1\.0T=1\.0, top\-p=1\.0p=1\.0, top\-k=−1k=\-1, and a maximum response length of 16384 tokens\. Evaluation uses the protocol described below underBenchmark Evaluation Protocol\.

All methods use the same maximum training\-iteration budget at a given backbone scale\. Each trainable configuration is run with three random seeds\. The main performance results report mean and standard deviation, while supplementary diagnostic tables report seed means unless otherwise stated\. All policies are optimized with full\-parameter AdamW using a peak learning rate of1×10−61\\times 10^\{\-6\},β1=0\.9\\beta\_\{1\}=0\.9,β2=0\.95\\beta\_\{2\}=0\.95, and weight decay0\.10\.1\. We use a 3% linear warmup followed by cosine decay, gradient clipping at a global norm of 1\.0, one policy\-update epoch per rollout batch, bfloat16 mixed precision, and FSDP full sharding\. The same optimizer and stabilization settings are used for all trainable methods at a given backbone scale\.

For GRPO\-style updates, we use Clip\-Higher withϵlow=0\.2\\epsilon\_\{\\mathrm\{low\}\}=0\.2andϵhigh=0\.28\\epsilon\_\{\\mathrm\{high\}\}=0\.28and omit an explicit KL penalty\. For VeRPO, the adapted implementation usesα=2\.0\\alpha=2\.0andβ=1\.0\\beta=1\.0; the trajectory\-length decay is inactive in the single\-turn setting\.

For DHRCL, validation is performed every two training iterations on a fixed held\-out validation set using a fixed decoding protocol and fixed random seeds\. After at least 12 validation observations have been collected in the current stage, a one\-sided linear\-trend test at significance level0\.050\.05is applied to the most recent eight observations\. A stage transition is triggered only when the test fails to support a positive trend at two consecutive evaluations\. The best checkpoint of the current stage is then restored before training advances\. The monitored metric is syntax reward in the Syntax Stage, execution reward in the Execution Stage, and validation Pass@1 in the final stage\. All curriculum, reward, and token\-weighting ablations use this same rule\.

## Appendix CDataset Splits

We use KodCode as the RL training corpus\. The KodCode training split is used for rollout generation, reward computation, and policy optimization\. From the training corpus, we reserve a held\-out validation split for curriculum\-stage switching, early stopping, reward\-curve logging, and training\-stability analysis\. The validation split is never used for policy\-gradient updates and is not included in final benchmark reporting\.

Final evaluation is conducted only on external benchmark suites, including HumanEval, HumanEval\+, BigCodeBench\-Full, BigCodeBench\-Hard, LiveCodeBench V6, and CodeElo\. These benchmarks serve as the test sets for reporting final Pass@1, intermediate validity, and accepted\-output diagnostic metrics\. This separation ensures that curriculum decisions are made on held\-out in\-domain validation problems, while final results are measured on external evaluation benchmarks\.

Reference solutions in KodCode are used only for computing reward components that require structural comparison, such as AST similarity\. They are not included in model inputs and are not exposed to the policy during rollout generation\. During rollout generation, the policy only receives the programming problem prompt and generates candidate solutions under the specified decoding configuration\.

## Appendix DBenchmark Evaluation Protocol

All methods are evaluated under the same benchmark protocol, decoding configuration, and execution environment\. During benchmark evaluation, we samplen=8n=8candidate solutions for each problem and report Pass@1\. Evaluation\-time generation uses temperatureT=0\.6T=0\.6, top\-p=0\.95p=0\.95, and top\-k=20k=20\. The maximum response length is set to 16384 tokens for both training and evaluation, following the VeRPO\-aligned protocol and reducing truncation effects on long\-form code\-generation tasks\. All generated programs are executed in the same sandbox with fixed timeout, memory constraints, and unit\-test protocol\.

For each benchmark, the same prompt format, decoding parameters, execution sandbox, and pass/fail criteria are used across all methods\. For method comparisons, we compute problem\-level paired bootstrap confidence intervals with 10,000 resamples, preserving the pairing of predictions on each benchmark problem\. This ensures that differences in final results reflect differences in training objectives rather than differences in evaluation configuration\. NET and NTMU are computed only on accepted programs\. Because each method may solve a different subset of problems, they are interpreted as descriptive accepted\-output diagnostics rather than strictly paired efficiency comparisons\.

## Appendix EProgram Validity and Accepted\-Output Diagnostics

Pass@1 is the primary functional metric\. Syntax Acc\. is a binary program\-level metric obtained with a strict language parser, whereas the training\-time syntax reward provides partial credit over local units using an error\-tolerant parser\. Exec Acc\. is the fraction of generated programs that finish without runtime exceptions, timeout, or memory\-limit failure\.

AST Sim\. is computed between a generated program and a verified reference after applying the exact normalization and similarity procedure specified below underImplementation Details of the AST Structural Reward\. The metric describes reference\-conditioned structural alignment and is not interpreted as semantic equivalence\. A generated program that cannot be strictly parsed receives zero AST reward during training\.

Table 5: Error\-type distribution under the unified Qwen3\-8B evaluation protocol\. All categories are mutually exclusive, and each row sums to 100%\.

Table 6: Reward\-component ablation\.

Table 7: Curriculum and token\-credit ablations\.

NET is the execution time of an accepted generated program normalized by the corresponding aggregate GRPO value\. NTMU is the peak test memory usage of an accepted generated program normalized in the same way\. Both metrics are computed over each method’s own accepted outputs\. Because the solved\-problem subsets can differ, NET and NTMU are descriptive computational characteristics under the adopted evaluation protocol, not strictly paired evidence that one method is universally more efficient\.

We keep these evidence levels separate: Pass@1 measures functional correctness; Syntax Acc\. and Exec Acc\. diagnose intermediate validity; AST Sim\. measures structural alignment; and NET/NTMU describe the observed computation of accepted outputs\.

## Appendix FTraining\-Efficiency and Reward\-Signal Metrics

We report training\-log metrics to compare computational cost, convergence efficiency, reward informativeness, and reward dispersion across RL methods\.

Step Time is the average wall\-clock time per training iteration, including rollout generation, execution, reward computation, log\-probability recomputation, and policy update\. GPU Hours measures the GPU\-side cost required to reach the target validation performance\. It is computed from the average wall\-clock time per training iteration, the number of iterations required to reach the target, and the number of GPUs used during training\. All values are normalized by GRPO\. For methods that do not reach the target within the fixed training budget, we report the GPU Hours consumed under the full budget and mark Iterations to Target as N\.R\.

Iterations to Target measures how many training iterations are required to reach a fixed validation\-performance threshold\. We set this threshold to the final validation Pass@1 achieved by GRPO under the same training budget\. GRPO therefore serves as the reference method with normalized Iterations to Target equal to1\.00×1\.00\\times\. For methods that do not reach this threshold within the training budget, we report N\.R\. A lower value indicates faster convergence to the same validation\-performance level\.

Avg\. DGR denotes the average degenerate group ratio over logged training iterations\. For each prompt, the policy samples a rollout groupG=\{τi\}i=1NG=\\\{\\tau\_\{i\}\\\}\_\{i=1\}^\{N\}with scalar rewards\{R​\(τi\)\}i=1N\\\{R\(\\tau\_\{i\}\)\\\}\_\{i=1\}^\{N\}\. A rollout group is considered degenerate if

Std​\(\{R​\(τi\)\}i=1N\)<10−2\.\\mathrm\{Std\}\\left\(\\\{R\(\\tau\_\{i\}\)\\\}\_\{i=1\}^\{N\}\\right\)<10^\{\-2\}\.The degenerate group ratio at a training iteration is the proportion of degenerate groups among all prompt\-level rollout groups in that iteration\. Avg\. DGR is the average of this ratio over logged training iterations\. A lower Avg\. DGR indicates that a larger proportion of rollout groups provide non\-trivial relative\-advantage signals for policy optimization\.

Reward Var\. denotes the average within\-group reward variance over logged training iterations\. It is used to characterize reward dispersion rather than as a metric to be minimized\. Extremely low reward variance often indicates uninformative rewards, while excessively high variance may make optimization unstable\.

In the reward\-signal analysis, DGR is used to compare how often different reward designs fail to distinguish sampled candidates within the same prompt\-level rollout group\. Binary pass/fail rewards tend to produce high DGR when most sampled programs fail all tests\. Pass\-rate and difficulty\-aware rewards reduce this issue by assigning different scores to partially correct programs\. AceCoder achieves the lowest DGR through its external learned reward model\. DHRCL achieves the second\-lowest DGR without an external reward model because programs may differ in syntax validity, execution status, unit\-test pass rate, and structural similarity\. This makes the rollout group more likely to provide useful relative\-advantage signals during group\-based policy optimization\.

## Appendix GImplementation Details of the AST Structural Reward

The AST reward is implemented for Python programs using Python 3\.11’s built\-inastparser\. Before comparison, Markdown fences, comments, docstrings, formatting information, and source\-location attributes are removed\. User\-defined identifiers are consistently renamed in order of first occurrence, while operators, control\-flow node types, library API names, and the constants−1\-1,0, and11are preserved\. Other string and numeric literals are mapped to type\-level placeholders\.

For normalized treesAAandBB, structural similarity is computed from unit\-cost tree\-edit distance:

SimAST​\(A,B\)=max⁡\(0,1−TED​\(A,B\)\|A\|\+\|B\|\),\\mathrm\{Sim\}\_\{\\mathrm\{AST\}\}\(A,B\)=\\max\\left\(0,1\-\\frac\{\\mathrm\{TED\}\(A,B\)\}\{\|A\|\+\|B\|\}\\right\),where\|A\|\|A\|and\|B\|\|B\|denote the numbers of AST nodes\. Each problem uses its verified reference solution; when multiple verified references are available, the maximum similarity is used\.

For unparsable programs, the training reward is zero\. For parsable programs, the returned similarity is bounded in\[0,1\]\[0,1\]\. This signal is used only as reference\-conditioned structural supervision; unit\-test feedback remains the direct functional objective\.

## Appendix HImplementation Details of Syntax Reward

During RL training, generated programs often contain local syntax errors, so a strict AST parser may provide no partial feedback\. We therefore use Tree\-sitter as an error\-tolerant parser to obtain partial syntax trees after removing Markdown code fences and normalizing indentation when possible\.

We extract statement\- and block\-level syntactic units from the parse tree, including function definitions, class definitions, control\-flow statements, assignments, expressions, and return statements\. Lexical leaves, comments, and overly short fragments are excluded from reward computation\. A unit is marked as syntactically valid if its subtree does not contain parser\-reported error nodes\. The final syntax reward is the length\-weighted fraction of valid units:

rsyntax​\(y\)=∑i=1mwi​vi∑i=1mwi\.r\_\{\\mathrm\{syntax\}\}\(y\)=\\frac\{\\sum\_\{i=1\}^\{m\}w\_\{i\}v\_\{i\}\}\{\\sum\_\{i=1\}^\{m\}w\_\{i\}\}\.We use token length aswiw\_\{i\}so that larger program regions contribute proportionally more to the reward\. If the parser cannot extract any statement\-level unit, the syntax reward is set to zero\.

This implementation makes the training reward robust to early\-stage generations that are only partially well\-formed\. It also distinguishes the training\-time syntax reward from the evaluation\-time Syntax Acc\. metric: the former gives partial credit over local syntactic units, whereas the latter checks whether the complete generated program is syntactically valid\.

## Appendix IError\-Type Analysis

To complement the aggregate benchmark scores in the main paper, we categorize each generated program by its final execution outcome\. The five categories are mutually exclusive: a sample is assigned to the first applicable category among syntax error, runtime error, timeout, wrong answer, and accepted\. Consequently, every row in Table[E](https://arxiv.org/html/2607.26457#A5)sums to 100%\. For each seed, the error\-type percentages are computed separately for each benchmark and then macro\-averaged across the six benchmarks, assigning equal weight to each benchmark\. The reported values are subsequently averaged over three seeds\. Under this aggregation, the Accepted proportion corresponds to the average Pass@1 reported in the main paper\.

Compared with the base model, DHRCL reduces syntax errors from 8\.0% to 4\.8%, runtime errors from 9\.7% to 5\.9%, and timeouts from 4\.8% to 2\.9%\. Relative to VeRPO, DHRCL also lowers all three invalid\-program categories and increases the accepted proportion from 49\.2% to 50\.3%\. Executable\-but\-incorrect programs remain the dominant residual failure mode after syntax and execution failures are reduced\.

## Appendix JDetailed Ablation Results

Table[E](https://arxiv.org/html/2607.26457#A5)reports the benchmark\-level reward\-component ablation, while Table[E](https://arxiv.org/html/2607.26457#A5)summarizes the curriculum\-scheduling and token\-credit ablations\. Stage Transitions report the median first/second transition iterations over three seeds\. Together, these results isolate the contributions of reward composition, trend\-based stage progression, and stage\-aware probability\-based token credit redistribution\.

##### Reward components\.

As shown in Table[E](https://arxiv.org/html/2607.26457#A5), adding syntax and execution feedback improves intermediate program validity, while the full reward obtains the highest average Pass@1 and AST similarity\. The comparison betweenPass \+ Syntax \+ ExecandFull Rewardassociates the AST component with a 0\.6\-point increase in average Pass@1 and a clearer improvement in AST similarity\. We therefore interpret structural alignment as the primary direct effect and the Pass@1 gain as a possible secondary benefit\.

##### Curriculum and token credit\.

Table[E](https://arxiv.org/html/2607.26457#A5)shows that trend\-based stage progression outperforms both fixed reward aggregation and predefined stage switching\. Among token\-credit variants, the complete consolidation\-to\-refinement ordering achieves the highest average Pass@1, the lowest DGR, and the smallest normalized training cost\. The reversed variant performs worse than either the early\-only or late\-only variants, supporting the importance of the proposed stage ordering\.

## References

- M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. de Oliveira Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman, A\. Ray, R\. Puri, G\. Krueger, M\. Petrov, H\. Khlaaf, G\. Sastry, P\. Mishkin, B\. Chan, S\. Gray, N\. Ryder, M\. Pavlov, A\. Power, L\. Kaiser, M\. Bavarian, C\. Winter, P\. Tillet, F\. P\. Such, D\. Cummings, M\. Plappert, F\. Chantzis, E\. Barnes, A\. Herbert\-Voss, W\. H\. Guss, A\. Nichol, A\. Paino, N\. Tezak, J\. Tang, I\. Babuschkin, S\. Balaji, S\. Jain, W\. Saunders, C\. Hesse, A\. N\. Carr, J\. Leike, J\. Achiam, V\. Misra, E\. Morikawa, A\. Radford, M\. Knight, M\. Brundage, M\. Murati, K\. Mayer, P\. Welinder, B\. McGrew, D\. Amodei, S\. McCandlish, I\. Sutskever, and W\. Zaremba \(2021\)Evaluating large language models trained on code\.CoRRabs/2107\.03374\.External Links:[Link](https://arxiv.org/abs/2107.03374),2107\.03374Cited by:[Introduction](https://arxiv.org/html/2607.26457#Sx1.p1.1),[Experimental Setup](https://arxiv.org/html/2607.26457#Sx4.SSx1.p1.1)\.
- N\. Dai, Z\. Wu, R\. Zheng, Z\. Wei, W\. Shi, X\. Jin, G\. Liu, C\. Dun, L\. Huang, and L\. Yan \(2024\)Process supervision\-guided policy optimization for code generation\.CoRRabs/2410\.17621\.External Links:[Link](https://doi.org/10.48550/arXiv.2410.17621),[Document](https://dx.doi.org/10.48550/ARXIV.2410.17621),2410\.17621Cited by:[Code\-Specific Reward Design](https://arxiv.org/html/2607.26457#Sx2.SSx2.p1.1)\.
- DeepSeek\-AI \(2025\)DeepSeek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.CoRRabs/2501\.12948\.External Links:[Link](https://doi.org/10.48550/arXiv.2501.12948),[Document](https://dx.doi.org/10.48550/ARXIV.2501.12948),2501\.12948Cited by:[RL\-Based Optimization for Code LLMs](https://arxiv.org/html/2607.26457#Sx2.SSx1.p1.1)\.
- S\. Dou, Y\. Liu, H\. Jia, L\. Xiong, E\. Zhou, W\. Shen, J\. Shan, C\. Huang, X\. Wang, X\. Fan, Z\. Xi, Y\. Zhou, T\. Ji, R\. Zheng, Q\. Zhang, X\. Huang, and T\. Gui \(2024\)StepCoder: improve code generation with reinforcement learning from compiler feedback\.CoRRabs/2402\.01391\.External Links:[Link](https://doi.org/10.48550/arXiv.2402.01391),[Document](https://dx.doi.org/10.48550/ARXIV.2402.01391),2402\.01391Cited by:[Curriculum Learning for Code LLMs](https://arxiv.org/html/2607.26457#Sx2.SSx3.p1.1)\.
- J\. Gehring, K\. Zheng, J\. Copet, V\. Mella, T\. Cohen, and G\. Synnaeve \(2025\)RLEF: grounding code llms in execution feedback with reinforcement learning\.InForty\-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13\-19, 2025,A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, and J\. Zhu \(Eds\.\),Proceedings of Machine Learning Research\.External Links:[Link](https://proceedings.mlr.press/v267/gehring25a.html)Cited by:[RL\-Based Optimization for Code LLMs](https://arxiv.org/html/2607.26457#Sx2.SSx1.p1.1),[Code\-Specific Reward Design](https://arxiv.org/html/2607.26457#Sx2.SSx2.p1.1)\.
- N\. Jain, K\. Han, A\. Gu, W\. Li, F\. Yan, T\. Zhang, S\. Wang, A\. Solar\-Lezama, K\. Sen, and I\. Stoica \(2025\)LiveCodeBench: holistic and contamination free evaluation of large language models for code\.External Links:[Link](https://openreview.net/forum?id=chfJJYC3iL)Cited by:[Experimental Setup](https://arxiv.org/html/2607.26457#Sx4.SSx1.p1.1)\.
- J\. Jiang, F\. Wang, J\. Shen, S\. Kim, and S\. H\. Kim \(2026\)A survey on large language models for code generation\.ACM Trans\. Softw\. Eng\. Methodol\.35\(2\),pp\. 58:1–58:72\.External Links:[Link](https://doi.org/10.1145/3747588),[Document](https://dx.doi.org/10.1145/3747588)Cited by:[RL\-Based Optimization for Code LLMs](https://arxiv.org/html/2607.26457#Sx2.SSx1.p1.1)\.
- H\. Le, Y\. Wang, A\. D\. Gotmare, S\. Savarese, and S\. C\. Hoi \(2022\)CodeRL: mastering code generation through pretrained models and deep reinforcement learning\.InAdvances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 \- December 9, 2022,S\. Koyejo, S\. Mohamed, A\. Agarwal, D\. Belgrave, K\. Cho, and A\. Oh \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper%5C_files/paper/2022/hash/8636419dea1aa9fbd25fc4248e702da4-Abstract-Conference.html)Cited by:[Introduction](https://arxiv.org/html/2607.26457#Sx1.p1.1),[RL\-Based Optimization for Code LLMs](https://arxiv.org/html/2607.26457#Sx2.SSx1.p1.1),[Code\-Specific Reward Design](https://arxiv.org/html/2607.26457#Sx2.SSx2.p1.1)\.
- J\. Liu, Y\. Zhu, K\. Xiao, Q\. Fu, X\. Han, W\. Yang, and D\. Ye \(2023a\)RLTF: reinforcement learning from unit test feedback\.Trans\. Mach\. Learn\. Res\.2023\.External Links:[Link](https://openreview.net/forum?id=hjYmsV6nXZ)Cited by:[Introduction](https://arxiv.org/html/2607.26457#Sx1.p1.1),[RL\-Based Optimization for Code LLMs](https://arxiv.org/html/2607.26457#Sx2.SSx1.p1.1),[Code\-Specific Reward Design](https://arxiv.org/html/2607.26457#Sx2.SSx2.p1.1)\.
- J\. Liu, C\. S\. Xia, Y\. Wang, and L\. Zhang \(2023b\)Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation\.External Links:[Link](http://papers.nips.cc/paper%5C_files/paper/2023/hash/43e9d647ccd3e4b7b5baab53f0368686-Abstract-Conference.html)Cited by:[Experimental Setup](https://arxiv.org/html/2607.26457#Sx4.SSx1.p1.1)\.
- M\. Naïr, K\. M\. Yamani, L\. S\. L’Hadj, and R\. Baghdadi \(2024\)Curriculum learning for small code language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 4: Student Research Workshop\), ACL 2024, Bangkok, Thailand, August 11\-16, 2024,X\. Fu and E\. Fleisig \(Eds\.\),pp\. 531–542\.External Links:[Link](https://doi.org/10.18653/v1/2024.acl-srw.44),[Document](https://dx.doi.org/10.18653/V1/2024.ACL-SRW.44)Cited by:[Curriculum Learning for Code LLMs](https://arxiv.org/html/2607.26457#Sx2.SSx3.p1.1)\.
- C\. Park, J\. Jiang, F\. Wang, S\. Paul, J\. Shen, J\. Tang, and J\. Li \(2026\)TAROT: test\-driven and capability\-adaptive curriculum reinforcement fine\-tuning for code generation with large language models\.CoRRabs/2602\.15449\.External Links:[Link](https://doi.org/10.48550/arXiv.2602.15449),[Document](https://dx.doi.org/10.48550/ARXIV.2602.15449),2602\.15449Cited by:[Curriculum Learning for Code LLMs](https://arxiv.org/html/2607.26457#Sx2.SSx3.p1.1)\.
- S\. Quan, J\. Yang, B\. Yu, B\. Zheng, D\. Liu, A\. Yang, X\. Ren, B\. Gao, Y\. Miao, Y\. Feng, Z\. Wang, J\. Yang, Z\. Cui, Y\. Fan, Y\. Zhang, B\. Hui, and J\. Lin \(2025\)CodeElo: benchmarking competition\-level code generation of llms with human\-comparable elo ratings\.CoRRabs/2501\.01257\.External Links:[Link](https://doi.org/10.48550/arXiv.2501.01257),[Document](https://dx.doi.org/10.48550/ARXIV.2501.01257),2501\.01257Cited by:[Experimental Setup](https://arxiv.org/html/2607.26457#Sx4.SSx1.p1.1)\.
- R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. Finn \(2023\)Direct preference optimization: your language model is secretly a reward model\.InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper%5C_files/paper/2023/hash/a85b405ed65c6477a4fe8302b5e06ce7-Abstract-Conference.html)Cited by:[RL\-Based Optimization for Code LLMs](https://arxiv.org/html/2607.26457#Sx2.SSx1.p1.1)\.
- H\. Ren, Z\. Lu, W\. Shi, H\. Hou, Y\. Yang, K\. Wang, A\. Zhou, J\. Pan, M\. Zhan, and H\. Li \(2025\)Alignment with fill\-in\-the\-middle for enhancing code generation\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, EMNLP 2025, Suzhou, China, November 4\-9, 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),pp\. 8304–8320\.External Links:[Link](https://doi.org/10.18653/v1/2025.emnlp-main.419),[Document](https://dx.doi.org/10.18653/V1/2025.EMNLP-MAIN.419)Cited by:[Curriculum Learning for Code LLMs](https://arxiv.org/html/2607.26457#Sx2.SSx3.p1.1)\.
- B\. Rozière, J\. Gehring, F\. Gloeckle, S\. Sootla, I\. Gat, X\. E\. Tan, Y\. Adi, J\. Liu, T\. Remez, J\. Rapin, A\. Kozhevnikov, I\. Evtimov, J\. Bitton, M\. Bhatt, C\. Canton\-Ferrer, A\. Grattafiori, W\. Xiong, A\. Défossez, J\. Copet, F\. Azhar, H\. Touvron, L\. Martin, N\. Usunier, T\. Scialom, and G\. Synnaeve \(2023\)Code llama: open foundation models for code\.CoRRabs/2308\.12950\.External Links:[Link](https://doi.org/10.48550/arXiv.2308.12950),[Document](https://dx.doi.org/10.48550/ARXIV.2308.12950),2308\.12950Cited by:[Introduction](https://arxiv.org/html/2607.26457#Sx1.p1.1)\.
- J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. Klimov \(2017\)Proximal policy optimization algorithms\.CoRRabs/1707\.06347\.External Links:[Link](http://arxiv.org/abs/1707.06347),1707\.06347Cited by:[RL\-Based Optimization for Code LLMs](https://arxiv.org/html/2607.26457#Sx2.SSx1.p1.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo \(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.CoRRabs/2402\.03300\.External Links:[Link](https://doi.org/10.48550/arXiv.2402.03300),[Document](https://dx.doi.org/10.48550/ARXIV.2402.03300),2402\.03300Cited by:[Appendix E](https://arxiv.org/html/2607.26457#A5.5.5.5.7.2.1),[RL\-Based Optimization for Code LLMs](https://arxiv.org/html/2607.26457#Sx2.SSx1.p1.1),[Experimental Setup](https://arxiv.org/html/2607.26457#Sx4.SSx1.p2.1),[Table 1](https://arxiv.org/html/2607.26457#Sx4.T1.18.16.16.15),[Table 2](https://arxiv.org/html/2607.26457#Sx4.T2.14.12.12.9)\.
- P\. Shojaee, A\. Jain, S\. Tipirneni, and C\. K\. Reddy \(2023\)Execution\-based code generation using deep reinforcement learning\.Trans\. Mach\. Learn\. Res\.2023\.External Links:[Link](https://openreview.net/forum?id=0XBuaxqEcG)Cited by:[Introduction](https://arxiv.org/html/2607.26457#Sx1.p1.1),[RL\-Based Optimization for Code LLMs](https://arxiv.org/html/2607.26457#Sx2.SSx1.p1.1),[Code\-Specific Reward Design](https://arxiv.org/html/2607.26457#Sx2.SSx2.p1.1)\.
- Q\. Team \(2025\)Qwen3 technical report\.CoRRabs/2505\.09388\.External Links:[Link](https://doi.org/10.48550/arXiv.2505.09388),[Document](https://dx.doi.org/10.48550/ARXIV.2505.09388),2505\.09388Cited by:[Appendix E](https://arxiv.org/html/2607.26457#A5.5.5.5.6.1.1),[Experimental Setup](https://arxiv.org/html/2607.26457#Sx4.SSx1.p1.1),[Table 1](https://arxiv.org/html/2607.26457#Sx4.T1.4.2.2.3)\.
- A\. Wang, Y\. Yan, N\. Zhou, Z\. Lu, W\. Lu, J\. Xiao, Y\. Zhuang, and Y\. Shen \(2026a\)Code\-a1: adversarial evolving of code LLM and test LLM via reinforcement learning\.CoRRabs/2603\.15611\.External Links:[Link](https://doi.org/10.48550/arXiv.2603.15611),[Document](https://dx.doi.org/10.48550/ARXIV.2603.15611),2603\.15611Cited by:[Code\-Specific Reward Design](https://arxiv.org/html/2607.26457#Sx2.SSx2.p1.1)\.
- J\. Wang, Z\. Zhang, Y\. He, Y\. Song, T\. Shi, Y\. Li, H\. Xu, K\. Wu, G\. Qian, Q\. Chen, and L\. He \(2024\)Enhancing code llms with reinforcement learning in code generation: A survey\.CoRRabs/2412\.20367\.External Links:[Link](https://doi.org/10.48550/arXiv.2412.20367),[Document](https://dx.doi.org/10.48550/ARXIV.2412.20367),2412\.20367Cited by:[RL\-Based Optimization for Code LLMs](https://arxiv.org/html/2607.26457#Sx2.SSx1.p1.1)\.
- L\. Wang, X\. Wu, X\. Hu, Y\. Liu, Y\. Fan, K\. Yu, Q\. Weng, W\. Xi, and X\. Li \(2026b\)VeRPO: verifiable dense reward policy optimization for code generation\.CoRRabs/2601\.03525\.External Links:[Link](https://doi.org/10.48550/arXiv.2601.03525),[Document](https://dx.doi.org/10.48550/ARXIV.2601.03525),2601\.03525Cited by:[Appendix E](https://arxiv.org/html/2607.26457#A5.5.5.5.10.5.1),[Code\-Specific Reward Design](https://arxiv.org/html/2607.26457#Sx2.SSx2.p1.1),[Experimental Setup](https://arxiv.org/html/2607.26457#Sx4.SSx1.p2.1),[Table 1](https://arxiv.org/html/2607.26457#Sx4.T1.60.58.58.15),[Table 2](https://arxiv.org/html/2607.26457#Sx4.T2.38.36.36.9)\.
- Z\. Wang, S\. Zhou, D\. Fried, and G\. Neubig \(2023\)Execution\-based evaluation for open\-domain code generation\.InFindings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6\-10, 2023,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Findings of ACL,pp\. 1271–1290\.External Links:[Link](https://doi.org/10.18653/v1/2023.findings-emnlp.89),[Document](https://dx.doi.org/10.18653/V1/2023.FINDINGS-EMNLP.89)Cited by:[Introduction](https://arxiv.org/html/2607.26457#Sx1.p1.1)\.
- R\. J\. Williams \(1992\)Simple statistical gradient\-following algorithms for connectionist reinforcement learning\.Mach\. Learn\.8,pp\. 229–256\.External Links:[Link](https://doi.org/10.1007/BF00992696),[Document](https://dx.doi.org/10.1007/BF00992696)Cited by:[RL\-Based Optimization for Code LLMs](https://arxiv.org/html/2607.26457#Sx2.SSx1.p1.1)\.
- H\. Wu, Y\. Yao, W\. Yu, and N\. Zhang \(2026\)ReCode: updating code API knowledge with reinforcement learning\.InFortieth AAAI Conference on Artificial Intelligence, Thirty\-Eighth Conference on Innovative Applications of Artificial Intelligence, Sixteenth Symposium on Educational Advances in Artificial Intelligence, AAAI 2026, Singapore, January 20\-27, 2026,S\. Koenig, C\. Jenkins, and M\. E\. Taylor \(Eds\.\),pp\. 33908–33916\.External Links:[Link](https://doi.org/10.1609/aaai.v40i40.40683),[Document](https://dx.doi.org/10.1609/AAAI.V40I40.40683)Cited by:[Code\-Specific Reward Design](https://arxiv.org/html/2607.26457#Sx2.SSx2.p1.1)\.
- J\. Wu, H\. Li, X\. Zhang, X\. Liu, Y\. Huang, J\. Luo, Y\. Zhang, Z\. Li, R\. Chu, Y\. Yang, and S\. Li \(2025\)Teaching your models to understand code via focal preference alignment\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, EMNLP 2025, Suzhou, China, November 4\-9, 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),pp\. 14003–14023\.External Links:[Link](https://doi.org/10.18653/v1/2025.emnlp-main.707),[Document](https://dx.doi.org/10.18653/V1/2025.EMNLP-MAIN.707)Cited by:[Code\-Specific Reward Design](https://arxiv.org/html/2607.26457#Sx2.SSx2.p1.1)\.
- Z\. Xu, Y\. Liu, Y\. Yin, M\. Zhou, and R\. Poovendran \(2025\)KodCode: A diverse, challenging, and verifiable synthetic dataset for coding\.pp\. 6980–7008\.External Links:[Link](https://doi.org/10.18653/v1/2025.findings-acl.365),[Document](https://dx.doi.org/10.18653/V1/2025.FINDINGS-ACL.365)Cited by:[Experimental Setup](https://arxiv.org/html/2607.26457#Sx4.SSx1.p1.1)\.
- Y\. Ye, T\. Zhang, W\. Jiang, and H\. Huang \(2025\)Process\-supervised reinforcement learning for code generation\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, EMNLP 2025, Suzhou, China, November 4\-9, 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),pp\. 14213–14226\.External Links:[Link](https://doi.org/10.18653/v1/2025.emnlp-main.719),[Document](https://dx.doi.org/10.18653/V1/2025.EMNLP-MAIN.719)Cited by:[Code\-Specific Reward Design](https://arxiv.org/html/2607.26457#Sx2.SSx2.p1.1)\.
- Q\. Yu, Z\. Zhang, R\. Zhu, Y\. Yuan, X\. Zuo, Y\. Yue, T\. Fan, G\. Liu, L\. Liu, X\. Liu, H\. Lin, Z\. Lin, B\. Ma, G\. Sheng, Y\. Tong, C\. Zhang, M\. Zhang, W\. Zhang, H\. Zhu, J\. Zhu, J\. Chen, J\. Chen, C\. Wang, H\. Yu, W\. Dai, Y\. Song, X\. Wei, H\. Zhou, J\. Liu, W\. Ma, Y\. Zhang, L\. Yan, M\. Qiao, Y\. Wu, and M\. Wang \(2025\)DAPO: an open\-source LLM reinforcement learning system at scale\.CoRRabs/2503\.14476\.External Links:[Link](https://doi.org/10.48550/arXiv.2503.14476),[Document](https://dx.doi.org/10.48550/ARXIV.2503.14476),2503\.14476Cited by:[RL\-Based Optimization for Code LLMs](https://arxiv.org/html/2607.26457#Sx2.SSx1.p1.1)\.
- H\. Zeng, D\. Jiang, H\. Wang, P\. Nie, X\. Chen, and W\. Chen \(2025\)ACECODER: acing coder RL via automated test\-case synthesis\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), ACL 2025, Vienna, Austria, July 27 \- August 1, 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),pp\. 12023–12040\.External Links:[Link](https://aclanthology.org/2025.acl-long.587/)Cited by:[Appendix E](https://arxiv.org/html/2607.26457#A5.5.5.5.9.4.1),[Code\-Specific Reward Design](https://arxiv.org/html/2607.26457#Sx2.SSx2.p1.1),[Experimental Setup](https://arxiv.org/html/2607.26457#Sx4.SSx1.p2.1),[Table 1](https://arxiv.org/html/2607.26457#Sx4.T1.46.44.44.15),[Table 2](https://arxiv.org/html/2607.26457#Sx4.T2.30.28.28.9)\.
- D\. Zhang, S\. Diao, X\. Zou, and H\. Peng \(2024\)PLUM: preference learning plus test cases yields better code language models\.CoRRabs/2406\.06887\.External Links:[Link](https://doi.org/10.48550/arXiv.2406.06887),[Document](https://dx.doi.org/10.48550/ARXIV.2406.06887),2406\.06887Cited by:[Code\-Specific Reward Design](https://arxiv.org/html/2607.26457#Sx2.SSx2.p1.1)\.
- K\. Zhang, G\. Li, Y\. Dong, J\. Xu, J\. Zhang, J\. Su, Y\. Liu, and Z\. Jin \(2025a\)CodeDPO: aligning code models with self generated and verified source code\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), ACL 2025, Vienna, Austria, July 27 \- August 1, 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),pp\. 15854–15871\.External Links:[Link](https://aclanthology.org/2025.acl-long.771/)Cited by:[Code\-Specific Reward Design](https://arxiv.org/html/2607.26457#Sx2.SSx2.p1.1)\.
- K\. Zhang, G\. Li, J\. Li, Y\. Dong, and Z\. Jin \(2025b\)Focused\-dpo: enhancing code generation through focused preference optimization on error\-prone points\.InFindings of the Association for Computational Linguistics, ACL 2025, Vienna, Austria, July 27 \- August 1, 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Findings of ACL,pp\. 9578–9591\.External Links:[Link](https://aclanthology.org/2025.findings-acl.498/)Cited by:[RL\-Based Optimization for Code LLMs](https://arxiv.org/html/2607.26457#Sx2.SSx1.p1.1)\.
- Y\. Zhao, W\. Fu, S\. Li, Y\. Hu, X\. Guo, and Y\. Jin \(2025\)Enhancing LLM performance on hardware design generation task via reinforcement learning\.InIEEE International Symposium on Circuits and Systems, ISCAS 2025, London, United Kingdom, May 25\-28, 2025,pp\. 1–5\.External Links:[Link](https://doi.org/10.1109/ISCAS56072.2025.11043832),[Document](https://dx.doi.org/10.1109/ISCAS56072.2025.11043832)Cited by:[Code\-Specific Reward Design](https://arxiv.org/html/2607.26457#Sx2.SSx2.p1.1)\.
- C\. Zheng, S\. Liu, M\. Li, X\. Chen, B\. Yu, C\. Gao, K\. Dang, Y\. Liu, R\. Men, A\. Yang, J\. Zhou, and J\. Lin \(2025\)Group sequence policy optimization\.CoRRabs/2507\.18071\.External Links:[Link](https://doi.org/10.48550/arXiv.2507.18071),[Document](https://dx.doi.org/10.48550/ARXIV.2507.18071),2507\.18071Cited by:[RL\-Based Optimization for Code LLMs](https://arxiv.org/html/2607.26457#Sx2.SSx1.p1.1)\.
- T\. Y\. Zhuo, M\. C\. Vu, J\. Chim, H\. Hu, W\. Yu, R\. Widyasari, I\. N\. B\. Yusuf, H\. Zhan, J\. He, I\. Paul, S\. Brunner, C\. Gong, J\. Hoang, A\. R\. Zebaze, X\. Hong, W\. Li, J\. Kaddour, M\. Xu, Z\. Zhang, P\. Yadav, and et al\. \(2025\)BigCodeBench: benchmarking code generation with diverse function calls and complex instructions\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,External Links:[Link](https://openreview.net/forum?id=YrycTjllL0)Cited by:[Introduction](https://arxiv.org/html/2607.26457#Sx1.p1.1),[Experimental Setup](https://arxiv.org/html/2607.26457#Sx4.SSx1.p1.1)\.