Understanding Curriculum Learning in Large Language Models via Cross-Difficulty Optimization Dynamics

# Understanding Curriculum Learning in Large Language Models via Cross-Difficulty Optimization Dynamics
Source: [https://arxiv.org/html/2608.17268](https://arxiv.org/html/2608.17268)
1\]Fudan University\\correspondence

###### Abstract

###### Abstract

Curriculum learning has been widely adopted in the post\-training of large language models by organizing training data from easy to hard\. However, its effectiveness varies substantially across reasoning tasks, suggesting that no single curriculum is universally optimal and raising a fundamental question: what determines when curriculum learning works? In this paper, we answer this question by analyzing the optimization dynamics induced by different curriculum schedules\. We show that the transfer relationship between different difficulty levels characterizes the optimization dynamics induced by curriculum learning, which in turn explains the effectiveness of different curriculum schedules, and formalize this relationship as Relative Transfer, a principled measure of cross\-difficulty knowledge transfer\. Based on this measurement, we derive Transfer\-aware Dynamic Curriculum Sampling \(TDCS\), which dynamically adjusts the sampling distribution according to the estimated transfer relationship throughout training\. Extensive experiments on multiple reasoning benchmarks demonstrate that TDCS consistently outperforms representative scheduling strategies across different tasks, model scales, and training paradigms\. More importantly, our work provides a unified optimization\-based explanation of curriculum learning through cross\-difficulty transfer\.

††footnotetext:∗Equal Contribution\.
†Corresponding authors\.## 1Introduction

Large language models \(LLMs\) have achieved remarkable success across a wide range of reasoning tasks through supervised fine\-tuning \(SFT\)\. As post\-training datasets continue to grow in both scale and diversity, how to effectively organize training data has become increasingly important for improving optimization performance and model generalization\. Consequently, training data scheduling has emerged as a fundamental component of LLM post\-training, aiming to determine the order in which training examples are presented throughout optimization\[[30](https://arxiv.org/html/2608.17268#bib.bib2),[20](https://arxiv.org/html/2608.17268#bib.bib3),[15](https://arxiv.org/html/2608.17268#bib.bib4),[5](https://arxiv.org/html/2608.17268#bib.bib12)\]\.

Among various scheduling strategies, Curriculum Learning \(CL\)\[[1](https://arxiv.org/html/2608.17268#bib.bib5),[24](https://arxiv.org/html/2608.17268#bib.bib7),[22](https://arxiv.org/html/2608.17268#bib.bib9),[35](https://arxiv.org/html/2608.17268#bib.bib14),[23](https://arxiv.org/html/2608.17268#bib.bib16)\], which organizes training examples from easy to hard, is arguably the most widely adopted paradigm\. Originally proposed in traditional machine learning, curriculum learning has demonstrated consistent optimization and generalization benefits across numerous learning problems\[[11](https://arxiv.org/html/2608.17268#bib.bib15)\]\. Motivated by its success, many recent LLM post\-training methods have incorporated curriculum learning by ranking training samples according to predefined difficulty metrics and progressively exposing the model to increasingly challenging examples\[[31](https://arxiv.org/html/2608.17268#bib.bib6),[28](https://arxiv.org/html/2608.17268#bib.bib8),[13](https://arxiv.org/html/2608.17268#bib.bib13)\]\. Consequently, fixed easy\-to\-hard curricula are commonly adopted as the default scheduling strategy in existing LLM post\-training pipelines\.

Although curriculum learning has been widely adopted, its effectiveness varies substantially across reasoning tasks\. Existing studies have documented this phenomenon through large\-scale empirical evaluations across different models, tasks, and difficulty metrics, they demonstrate that curriculum learning may outperform random sampling in some scenarios while becoming ineffective or even detrimental in others\[[9](https://arxiv.org/html/2608.17268#bib.bib10),[7](https://arxiv.org/html/2608.17268#bib.bib11),[26](https://arxiv.org/html/2608.17268#bib.bib17),[30](https://arxiv.org/html/2608.17268#bib.bib2)\]\. Although these studies reveal the limitations of fixed curriculum schedules, they primarily provide empirical observations and analyses of difficulty metrics, leaving a fundamental question unanswered:*what determines whether curriculum learning is effective?*Without understanding the underlying mechanism, it remains difficult to design more effective curriculum strategies beyond empirical trial and error\.

In this paper, we revisit curriculum learning by analyzing the optimization dynamics induced by different curriculum schedules\. Rather than treating curriculum learning as a predefined training heuristic, we seek to understand the optimization mechanism that determines when and why a curriculum schedule succeeds\. To this end, we first conduct a systematic empirical study across multiple reasoning benchmarks\. Our results reveal that no fixed scheduling strategy consistently performs best across different reasoning tasks, suggesting that the effectiveness of curriculum learning is fundamentally task\-dependent rather than universally optimal\.

To answer this question, we investigate the optimization dynamics induced by curriculum learning\. Specifically, we analyze how optimization on one difficulty level influences the optimization of other difficulty levels throughout training: optimizing one difficulty level may either facilitate or interfere with the optimization of others, and the overall transfer relationship determines whether a fixed curriculum schedule is effective\. Based on a first\-order optimization analysis, we formalize this transfer relationship as*Relative Transfer*, a principled measure of cross\-difficulty knowledge transfer\. This analysis provides a unified explanation of when curriculum learning succeeds or fails across different reasoning tasks, going beyond previous empirical observations\.

Building upon this theoretical understanding, we deriveTransfer\-aware Dynamic Curriculum Sampling \(TDCS\), an adaptive curriculum learning framework that dynamically adjusts the sampling distribution according to the estimated transfer relationship throughout training\. Instead of following a predefined easy\-to\-hard schedule, TDCS allocates training samples based on the estimated transfer benefits across difficulty levels\. Extensive experiments demonstrate that the resulting sampling strategy consistently outperforms existing fixed scheduling strategies across multiple reasoning benchmarks and model scales, while further generalizing to downstream self\-improvement settings\.

Our main contributions are summarized as follows:

- •We conduct a systematic empirical study of curriculum learning for LLM reasoning and demonstrate that no fixed scheduling strategy consistently performs best across different reasoning tasks\.
- •We provide a theoretical explanation of when curriculum learning succeeds or fails by analyzing cross\-difficulty knowledge transfer, and formalize this mechanism through Relative Transfer\.
- •Based on the proposed transfer analysis, we derive Transfer\-aware Dynamic Curriculum Sampling \(TDCS\), which consistently outperforms existing fixed scheduling strategies across multiple reasoning benchmarks, model scales, and self\-improvement settings\.

## 2Related Work

### 2\.1Curriculum Learning

Curriculum learning organizes training samples from easy to hard according to predefined difficulty measures to facilitate optimization\[[1](https://arxiv.org/html/2608.17268#bib.bib5),[17](https://arxiv.org/html/2608.17268#bib.bib18),[10](https://arxiv.org/html/2608.17268#bib.bib19)\]\. Numerous variants have since been proposed, including self\-paced learning and mentor\-guided curriculum design\[[8](https://arxiv.org/html/2608.17268#bib.bib20),[14](https://arxiv.org/html/2608.17268#bib.bib21),[19](https://arxiv.org/html/2608.17268#bib.bib22)\]\.

Recently, curriculum learning has been widely adopted in LLM post\-training\. Existing methods construct curricula based on reasoning complexity, model confidence, training loss, or estimated sample difficulty, demonstrating improved performance on reasoning and instruction\-following tasks\[[9](https://arxiv.org/html/2608.17268#bib.bib10),[7](https://arxiv.org/html/2608.17268#bib.bib11),[12](https://arxiv.org/html/2608.17268#bib.bib23),[25](https://arxiv.org/html/2608.17268#bib.bib24),[18](https://arxiv.org/html/2608.17268#bib.bib25)\]\. However, they mainly focus on designing difficulty metrics or curriculum schedules\.

In contrast, our work studies the optimization mechanism underlying curriculum learning\. Rather than proposing another predefined schedule, we explain when and why different curriculum schedules become effective through cross\-difficulty optimization analysis\.

### 2\.2Adaptive Data Scheduling

Instead of relying on a predefined curriculum, adaptive data scheduling dynamically adjusts the sampling distribution according to the optimization state throughout training\[[27](https://arxiv.org/html/2608.17268#bib.bib26),[4](https://arxiv.org/html/2608.17268#bib.bib27),[16](https://arxiv.org/html/2608.17268#bib.bib28)\]\. Existing methods estimate sample importance using signals such as training loss, uncertainty, gradient information, or reinforcement learning objectives, and adaptively allocate training resources to improve optimization efficiency\[[21](https://arxiv.org/html/2608.17268#bib.bib29),[34](https://arxiv.org/html/2608.17268#bib.bib30),[3](https://arxiv.org/html/2608.17268#bib.bib31),[33](https://arxiv.org/html/2608.17268#bib.bib1)\]\.

While these methods improve training efficiency through adaptive sample selection, their scheduling decisions are primarily driven by optimization signals that reflect the current training state\. In contrast, our work focuses on the transfer relationship between different difficulty levels, providing a transfer\-based criterion for curriculum scheduling rather than relying solely on optimization heuristics\.

## 3Empirical Observation

Table 1:Difficulty definition for different tasks\.TaskDifficulty DefinitionSudokuNumber of blank cellsKodCodeGPT pass rateiGSMReasoning steps
Table 2:Performance comparison of scheduling strategies\.TaskCurriculumMixRandomSudoku0\.2050\.1900\.164iGSM0\.3500\.3900\.353Code0\.5800\.5820\.607

![Refer to caption](https://arxiv.org/html/2608.17268v1/figures/acc_sudoku_1.png)\(a\)
![Refer to caption](https://arxiv.org/html/2608.17268v1/figures/acc_igsm_1.png)\(b\)
![Refer to caption](https://arxiv.org/html/2608.17268v1/figures/acc_code_1.png)\(c\)

Figure 1:Stepwise accuracy change of different schedules\.To investigate the effectiveness of curriculum learning in large language models, we conduct preliminary experiments on three reasoning benchmarks with naturally defined difficulty levels: Sudoku, KodCode\[[29](https://arxiv.org/html/2608.17268#bib.bib34)\], and iGSM\[[32](https://arxiv.org/html/2608.17268#bib.bib35)\], covering logical reasoning, code generation, and mathematical reasoning, respectively\.

Following the difficulty definitions in Table[2](https://arxiv.org/html/2608.17268#S3.T2), the training data are partitioned into four difficulty levels for Sudoku and KodCode, and five difficulty levels for iGSM\. We compare three representative training schedules:Curriculum, which trains the model from easy to hard;Random, which uniformly samples training examples from all difficulty levels; andMix, which follows the curriculum schedule while replacing half of the samples at each stage with randomly sampled examples\. All methods are trained under the same training budget\.

Table[2](https://arxiv.org/html/2608.17268#S3.T2)reports the final performance of the three scheduling strategies\. Surprisingly, Curriculum does not consistently outperform the alternatives\. Instead, Curriculum performs best on Sudoku, Mix achieves the highest accuracy on iGSM, and Random performs best on KodCode\. These results indicate that curriculum learning is not universally beneficial for large language models, and no single scheduling strategy consistently achieves the best performance across different reasoning tasks\.

The final performance, however, only reflects the optimization outcome\. To obtain a more complete view, we further compare the training dynamics of different scheduling strategies in Figure[1](https://arxiv.org/html/2608.17268#S3.F1)\. The three tasks exhibit markedly different optimization behaviors\. While a single scheduling strategy consistently dominates throughout training on Sudoku and KodCode, the optimal strategy on iGSM changes as training progresses\.

These observations suggest that the effectiveness of curriculum learning cannot be fully explained by the curriculum schedule itself\. Instead, they raise an important question: what optimization mechanism determines whether a curriculum schedule succeeds? To answer this question, we next investigate the optimization process induced by curriculum learning, focusing on how optimization on one difficulty level influences the optimization of others\.

## 4Optimization Analysis of Curriculum Learning

### 4\.1Modeling Knowledge Transfer

The empirical observations in Section 3 show that no single scheduling strategy consistently performs best across different reasoning tasks\. Understanding the underlying optimization mechanism is therefore essential for explaining the effectiveness of curriculum learning\.

In this section, rather than analyzing the curriculum schedule itself, we investigate the optimization dynamics induced by curriculum learning\. Specifically, we study how optimization on one difficulty level affects the optimization objective of another\. We refer to this cross\-difficulty interaction as knowledge transfer\.

Assume that the current optimization step is performed on examples from difficulty leveljj\. We are interested in quantifying how this update affects the loss of another difficulty levelii\. Such cross\-difficulty interaction characterizes the knowledge transfer during curriculum learning\.

Let the model parameter at training stepttbe denoted bywtw\_\{t\}\. After performing one gradient descent step on data with difficulty leveljj, the model is updated as

wt\+1=wt−η​gj,w\_\{t\+1\}=w\_\{t\}\-\\eta g\_\{j\},\(1\)
whereη\\etais the learning rate andgj=∇Lj​\(wt\)g\_\{j\}=\\nabla L\_\{j\}\(w\_\{t\}\)is the gradient computed on difficulty leveljj\.

We then analyze the loss of difficulty leveliiafter this update\. Applying the first\-order Taylor expansion gives

Li\(wt\+1\)≈Li\(wt\)\+∇Li\(wt\)T\(wt\+1−wt\)\.L\_\{i\}\(w\_\{t\+1\}\)\\approx L\_\{i\}\(w\_\{t\}\)\+\\nabla L\_\{i\}\(w\_\{t\}\)^\{T\}\(w\_\{t\+1\}\-w\_\{t\}\)\.\(2\)
Since

∇Li​\(wt\)=gi,\\nabla L\_\{i\}\(w\_\{t\}\)=g\_\{i\},\(3\)
substituting Eq\. \(1\) into Eq\. \(2\) yields

Li​\(wt\+1\)≈Li​\(wt\)−η​giT​gj\.L\_\{i\}\(w\_\{t\+1\}\)\\approx L\_\{i\}\(w\_\{t\}\)\-\\eta g\_\{i\}^\{T\}g\_\{j\}\.\(4\)
Therefore, the expected loss variation on difficulty leveliiafter optimizing difficulty leveljjcan be approximated by

Δ​Li=Li​\(wt\+1\)−Li​\(wt\)≈−η​giT​gj\.\\Delta L\_\{i\}=L\_\{i\}\(w\_\{t\+1\}\)\-L\_\{i\}\(w\_\{t\}\)\\approx\-\\eta g\_\{i\}^\{T\}g\_\{j\}\.\(5\)
Eq\. \(5\) shows that the influence of optimizing one difficulty level on another is determined by the projection of one gradient onto the descent direction induced by the other\. WhengiT​gj\>0g\_\{i\}^\{T\}g\_\{j\}\>0, optimizing difficulty leveljjreduces the loss of difficulty levelii, indicating positive knowledge transfer\. In contrast, negative values imply that optimizing one difficulty level increases the loss of another, resulting in optimization conflicts\.

However, the quantitygiT​gjg\_\{i\}^\{T\}g\_\{j\}alone does not provide a fair comparison of transfer across different target difficulty levels, since its magnitude is also influenced by the scale of the target gradientgig\_\{i\}\. To compare transfer effects independently of gradient magnitude, we measure the loss reduction achieved by optimizing difficulty leveljj*relative*to that achieved by directly optimizing difficulty leveliiitself\.

When optimizing the target difficulty levelii, Eq\. \(5\) gives an expected loss reduction proportional togiT​gig\_\{i\}^\{T\}g\_\{i\}\. In contrast, optimizing difficulty leveljjreduces the same loss by an amount proportional togiT​gjg\_\{i\}^\{T\}g\_\{j\}\. Their ratio therefore naturally measures the relative transfer effect from difficulty leveljjto difficulty levelii, which we define as theRelative Transfer

Re⁡\(i,j\)=giT​gjgiT​gi\.\\mathrm\{Re\}\(i,j\)=\\frac\{g\_\{i\}^\{T\}g\_\{j\}\}\{g\_\{i\}^\{T\}g\_\{i\}\}\.\(6\)
Intuitively,Re⁡\(i,j\)\\mathrm\{Re\}\(i,j\)measures the effectiveness of optimizing difficulty leveljjrelative to directly optimizing difficulty levelii\. A value ofRe⁡\(i,j\)=1\\mathrm\{Re\}\(i,j\)=1indicates that optimizing difficulty leveljjis expected to reduce the loss of difficulty leveliias much as directly optimizingiiitself\. Values between00and11indicate partial positive transfer, values larger than11indicate even stronger transfer than self\-optimization, while negative values imply optimization conflicts\.

### 4\.2Transfer Analysis

![Refer to caption](https://arxiv.org/html/2608.17268v1/figures/transfer_bar.png)Figure 2:Transfer from the current difficulty to other difficulties\. Sudoku exhibits consistently strong hard\-to\-easy transfer, while iGSM and Code show much weaker transfer\.Eq\. \(6\) characterizes the knowledge transfer between arbitrary difficulty levels\. In the following analysis, we focus on the final curriculum stage and examine the transfer from the current hardest difficulty level to all previously learned difficulty levels\.

Figure[2](https://arxiv.org/html/2608.17268#S4.F2)visualizes the transfer from the current hardest difficulty level to all previously learned difficulty levels on each task\. Sudoku exhibits consistently strong positive transfer, whereas iGSM and KodCode show much weaker transfer to earlier difficulty levels\.

These transfer patterns provide an explanation for the empirical observations in Section 3\. For Sudoku, strong transfer from the hardest difficulty continues to reduce the losses of previously learned difficulties\. Consequently, optimizing only the hardest examples is sufficient to maintain performance across all difficulty levels, making a pure curriculum schedule effective\. In contrast, the much weaker transfer observed on iGSM and KodCode suggests that optimizing only the hardest examples is insufficient to maintain performance on earlier difficulties, motivating the inclusion of additional replay from previous difficulty levels\.

These findings suggest that the optimal sampling strategy should adapt to the observed cross\-difficulty transfer relationship rather than follow a predefined curriculum schedule\. This observation directly motivates the transfer\-aware dynamic curriculum sampling strategy introduced in the next section\.

## 5Transfer\-aware Dynamic Curriculum Sampling

Based on the transfer analysis in Section 4, the optimal curriculum should adapt to the observed cross\-difficulty transfer relationship rather than follow a predefined easy\-to\-hard schedule\. This observation naturally leads to three design principles\. First, the amount of training allocated to the current difficulty should increase with its transfer capability\. Second, replay should focus on difficulty levels that receive insufficient transfer\. Third, when a harder difficulty provides exceptionally strong positive transfer, it should also be incorporated into training\. Based on these principles, we derive Transfer\-aware Dynamic Curriculum Sampling \(TDCS\)\. The overall procedure is summarized in Algorithm 1\.

#### Current Difficulty Ratio Mapping\.

Let the current training stage correspond to difficulty levelkk, referred to as the*current difficulty*throughout this section\. The first step is to determine how much training should remain on the current difficulty\. This directly follows from the transfer analysis in Section 4: if the current difficulty can effectively optimize previously learned difficulties, replay becomes less necessary\. Intuitively, if the current difficulty can effectively transfer knowledge to previously learned difficulty levels, more training should remain on the current difficulty; otherwise, additional replay is required\.

Since difficulty levels that can already be effectively optimized through knowledge transfer require little additional replay, we first identify the difficulty levels that receive insufficient transfer from the current difficulty\. Specifically, we select the difficulty levels whose Relative Transfer is smaller than a thresholdτe\\tau\_\{e\},

𝒮=\{i≠k∣R​e​\(i,k\)<τe\}\.\\mathcal\{S\}=\\\{i\\neq k\\mid Re\(i,k\)<\\tau\_\{e\}\\\}\.\(7\)
We then compute the average Relative Transfer

R¯=1\|𝒮\|​∑i∈𝒮R​e​\(i,k\),\\bar\{R\}=\\frac\{1\}\{\|\\mathcal\{S\}\|\}\\sum\_\{i\\in\\mathcal\{S\}\}Re\(i,k\),\(8\)which summarizes the overall transfer capability of the current difficulty to the difficulty levels that require additional replay\.

Finally, the average transfer is mapped to the sampling ratio of the current difficulty through a sigmoid function,

ρk=11\+exp⁡\[−α⁡\(R¯−β\)\],\\rho\_\{k\}=\\frac\{1\}\{1\+\\exp\[\-\\alpha\(\\bar\{R\}\-\\beta\)\]\},\(9\)whereα\\alphaandβ\\betacontrol the slope and midpoint of the mapping, respectively\. The sigmoid function provides a smooth transition between replay\-oriented and current\-difficulty\-oriented sampling\. Consequently, stronger Relative Transfer leads to a larger sampling ratio for the current difficulty, while weaker transfer encourages more replay of previous difficulty levels\.

#### Difficulty Allocation\.

Given the sampling ratio of the current difficulty, the remaining probability\(1−ρk\)\(1\-\\rho\_\{k\}\)is allocated to the selected difficulty levels\. To prioritize difficulty levels that receive weaker knowledge transfer, we adopt a reverse exponential weighting strategy,

pi=\(1−ρk\)​exp⁡\(−λ​R​e​\(i,k\)\)∑j∈𝒮exp⁡\(−λ​R​e​\(j,k\)\),p\_\{i\}=\(1\-\\rho\_\{k\}\)\\frac\{\\exp\(\-\\lambda Re\(i,k\)\)\}\{\\sum\_\{j\\in\\mathcal\{S\}\}\\exp\(\-\\lambda Re\(j,k\)\)\},\(10\)where difficulty levels with weaker transfer receive larger sampling probabilities, encouraging additional replay for difficulty levels that are less likely to benefit from the current optimization\.

#### Harder Difficulty Adjustment\.

The previous steps determine the sampling distribution according to the estimated Relative Transfer\. Nevertheless, when a harder difficulty provides exceptionally strong positive transfer, allocating additional training to that difficulty can further improve subsequent optimization\. Therefore, if the Relative Transfer from the current difficulty to a harder difficulty exceeds a thresholdτh\\tau\_\{h\}, i\.e\.,

R​e​\(k,j\)\>τh,Re\(k,j\)\>\\tau\_\{h\},\(11\)a portion of the current sampling probability is reassigned to the harder difficulty in proportion to its Relative Transfer\. Finally, all sampling probabilities are normalized to obtain the sampling distribution for the next training stage\.

Algorithm 1Transfer\-aware Dynamic Curriculum Sampling

0:Relative Transfer matrix

𝐑𝐞\\mathbf\{Re\}, current difficulty

kk
0:Sampling distribution

𝐩\\mathbf\{p\}
1:

𝒮←\{i≠k∣R​e​\(i,k\)<τe\}\\mathcal\{S\}\\leftarrow\\\{i\\neq k\\mid Re\(i,k\)<\\tau\_\{e\}\\\}// Re Filter

2:

R¯←\|𝒮\|−1​∑i∈𝒮R​e​\(i,k\)\\bar\{R\}\\leftarrow\|\\mathcal\{S\}\|^\{\-1\}\\sum\_\{i\\in\\mathcal\{S\}\}Re\(i,k\)// Mean Transfer

3:

ρ←Sigmoid⁡\(R¯\)\\rho\\leftarrow\\mathrm\{Sigmoid\}\(\\bar\{R\}\)// Ratio mapping

4:

pk←ρp\_\{k\}\\leftarrow\\rho
5:for all

i∈𝒮i\\in\\mathcal\{S\}do

6:

pi∝e−β​R​e​\(i,k\)p\_\{i\}\\propto e^\{\-\\beta Re\(i,k\)\}// Difficulty allocation

7:endfor

9:if

R​e​\(k,j\)\>τhRe\(k,j\)\>\\tau\_\{h\}then

10:

pj∝R​e​\(k,j\)p\_\{j\}\\propto Re\(k,j\)// harder adjustment

11:

pk←pk−pjp\_\{k\}\\leftarrow p\_\{k\}\-p\_\{j\}
12:endif

13:endfor

14:Normalize

𝐩\\mathbf\{p\}

Table 3:Main results on three reasoning benchmarks\. The best result is shown inbold\.CategoryMethodSudokuiGSMKodCodeFixed ScheduleRandom0\.1640\.3530\.607Curriculum0\.2050\.3500\.580Mix0\.1900\.3900\.582Adaptive ScheduleOurs0\.2310\.4280\.619![Refer to caption](https://arxiv.org/html/2608.17268v1/figures/sudoku_contrast.png)\(a\)
![Refer to caption](https://arxiv.org/html/2608.17268v1/figures/igsm_contrast.png)\(b\)
![Refer to caption](https://arxiv.org/html/2608.17268v1/figures/code_contrast.png)\(c\)

Figure 3:Training dynamics of the proposed method and the strongest fixed scheduling baseline during supervised fine\-tuning\.

## 6Experiments

### 6\.1Experimental Settings

#### Datasets\.

We evaluate our method on three reasoning benchmarks:Sudoku,KodCode\[[29](https://arxiv.org/html/2608.17268#bib.bib34)\], andiGSM\[[32](https://arxiv.org/html/2608.17268#bib.bib35)\], covering logical reasoning, code generation, and mathematical reasoning, respectively\. Following the difficulty definitions introduced in Section 3, Sudoku and KodCode are partitioned into four difficulty levels, while iGSM is partitioned into five difficulty levels\. For self\-improvement, we use GSM8K and KodCode dataset\.

#### Models and Baselines\.

We conduct the main experiments on Qwen2\.5\-1\.5B\. To evaluate the generalization ability of the proposed method, we further consider Qwen2\.5\-3B, Qwen2\.5\-7B\[[6](https://arxiv.org/html/2608.17268#bib.bib32)\], and Llama3\.2\-3B\[[2](https://arxiv.org/html/2608.17268#bib.bib33)\]\. We compare TDCS with three representative scheduling strategies introduced in Section 3:*Curriculum*,*Random*, and*Mix*\.

#### Implementation Details\.

All methods are trained under the same training data budget using LoRA fine\-tuning\. Unless otherwise specified, all methods are trained for 4 epochs with a learning rate of1×10−51\\times 10^\{\-5\}and a LoRA rank of 64\. All remaining hyperparameters are kept identical across different methods to ensure a fair comparison\.

### 6\.2Main Results

Table[3](https://arxiv.org/html/2608.17268#S5.T3)summarizes the final performance on the three reasoning benchmarks\. Our method consistently achieves the best performance across all tasks, outperforming the strongest fixed scheduling strategy by 2\.6%, 3\.8%, and 1\.2% on Sudoku, iGSM, and KodCode, respectively\. These results demonstrate that no single fixed curriculum is universally optimal, while dynamically adjusting the sampling distribution according to the estimated cross\-difficulty transfer leads to consistently better optimization\.

Figure[3](https://arxiv.org/html/2608.17268#S5.F3)further compares the training dynamics between TDCS and the strongest baseline\. On Sudoku and iGSM, TDCS consistently maintains superior performance throughout training and converges to higher final accuracy\. On KodCode, although the advantage is less pronounced during the intermediate stages, our method still achieves the best final performance\. This observation suggests that transfer\-aware curriculum adjustment not only improves the final optimization result but also provides a more stable training process across different reasoning tasks\.

To better understand how TDCS achieves these improvements, Figure[4](https://arxiv.org/html/2608.17268#S6.F4)visualizes the evolution of the sampling distribution during training\. The sampling behavior differs substantially across reasoning tasks, reflecting their distinct transfer characteristics\. Specifically, TDCS rapidly shifts the sampling distribution toward harder difficulties on Sudoku, where stronger cross\-difficulty transfer reduces the need for replay\. In contrast, iGSM maintains a moderate replay ratio throughout training, while KodCode preserves considerably larger replay ratios because of its weaker transfer capability\. These observations are highly consistent with the transfer analysis in Section 4, demonstrating that TDCS dynamically adapts its curriculum according to the estimated Relative Transfer rather than following a predefined scheduling strategy\.

![Refer to caption](https://arxiv.org/html/2608.17268v1/figures/sampling_distribution_sudoku.png)\(a\)
![Refer to caption](https://arxiv.org/html/2608.17268v1/figures/sampling_distribution_igsm.png)\(b\)
![Refer to caption](https://arxiv.org/html/2608.17268v1/figures/sampling_distribution_code.png)\(c\)

Figure 4:Evolution of the sampling distribution produced by TDCS during supervised fine\-tuning on the three reasoning benchmarks\. Each stacked bar represents the sampling ratio assigned to different difficulty levels at a representative training stage, while dashed boxes indicate the current difficulty\.
### 6\.3Generalization

#### Generalization Across Model Scales\.

Table 4:Generalization across different model scales\. Numbers in parentheses denote the absolute accuracy improvement over the strongest baseline\.ModelSudokuiGSMBaselineOursBaselineOursQwen2\.5\-3B0\.4280\.446 \(\+1\.8%\)0\.5250\.546 \(\+2\.1%\)Qwen2\.5\-7B0\.4380\.497 \(\+5\.9%\)0\.7660\.819 \(\+5\.3%\)Llama3\.2\-3B0\.4050\.433 \(\+2\.8%\)0\.6730\.875 \(\+20\.8%\)To evaluate whether the proposed transfer\-aware sampling strategy generalizes across different model scales and architectures, we further conduct experiments on Qwen2\.5\-3B\-Instruct, Qwen2\.5\-7B\-Instruct, and Llama3\.2\-3B\-Instruct\. As shown in Table[4](https://arxiv.org/html/2608.17268#S6.T4), our method consistently outperforms the strongest fixed scheduling strategy on all evaluated models\. Notably, the improvements are observed across both the Qwen and Llama model families, indicating that the proposed sampling strategy does not rely on a specific pretrained model or parameter scale\. These results suggest that cross\-difficulty knowledge transfer is a general optimization property, making the proposed transfer\-aware curriculum broadly applicable to different LLMs\.

#### Generalization Across Training Paradigms\.

Table 5:Self\-improvement results under TDCS\.TaskCurriculumMixRandomOursGSM8K0\.6120\.6010\.6010\.622Code0\.3470\.3380\.3350\.352We further evaluate our method in a self\-improvement setting, where the model is fine\-tuned using its own generated responses instead of manually curated training data\. In this setting, model answer accuracy is used as the difficulty metric to construct the curriculum\. As shown in Table[5](https://arxiv.org/html/2608.17268#S6.T5), our method consistently outperforms all fixed scheduling strategies on both GSM8K and KodCode\. These results demonstrate that the proposed transfer\-aware sampling strategy is compatible with different training paradigms and remains effective even when difficulty is estimated from the model’s own predictions\.

### 6\.4Ablation Study

Table 6:Ablation study of TDCS components\.HAEDASudokuiGSMCode×\\times✓\\checkmark0\.2130\.4160\.612✓\\checkmark×\\times0\.2200\.3900\.589✓\\checkmark✓\\checkmark0\.2310\.4280\.619To evaluate the contribution of each component, we conduct ablation experiments on Qwen2\.5\-1\.5B\. In Table[6](https://arxiv.org/html/2608.17268#S6.T6),HAdenotes the*Harder Adjustment*module, andEDAdenotes the*Exponential Difficulty Allocation*module, which replaces uniform probability allocation with transfer\-aware exponential weighting over the selected difficulty levels\.

As shown in Table[6](https://arxiv.org/html/2608.17268#S6.T6), removing either component consistently degrades performance on all three benchmarks, confirming that both components contribute to the effectiveness of TDCS\.

RemovingHAresults in the largest performance drop on Sudoku, where stronger positive transfer from harder to easier difficulty levels makes harder\-difficulty adjustment particularly beneficial\. In contrast, removingEDAconsistently reduces performance across all tasks, indicating that allocating replay according to the estimated transfer relationship is more effective than uniform allocation\.

These results are consistent with the transfer analysis in Section 4 and demonstrate that both components are necessary for effectively exploiting cross\-difficulty knowledge transfer\.

## 7Conclusion

In this paper, we revisited curriculum learning by analyzing the optimization dynamics induced by different curriculum schedules\. We introduced*Relative Transfer*, a principled measure that explains when curriculum learning succeeds by characterizing cross\-difficulty knowledge transfer\. Based on this analysis, we derived Transfer\-aware Dynamic Curriculum Sampling \(TDCS\), which dynamically adjusts the sampling distribution according to the estimated transfer relationship and consistently outperforms representative curriculum scheduling strategies\. We hope that the proposed transfer\-based perspective provides a useful foundation for understanding curriculum learning\. In future work, we plan to extend the proposed transfer analysis to broader optimization settings and explore more general transfer\-aware data scheduling methods\.

## Appendix

## 8Dataset Details

This section provides additional details of the datasets used in our experiments, including dataset statistics, difficulty definitions, and representative examples\.

### 8\.1Dataset Statistics

Table[7](https://arxiv.org/html/2608.17268#S8.T7)summarizes the datasets used in this work\. We report the number of training and evaluation samples, the number of difficulty levels, and the corresponding difficulty criterion for each benchmark\.

Table 7:Statistics of the datasets used in this work\.DatasetTrainTest\#LevelsDifficultySudoku800\*4epochs10004Blank cellsiGSM1000\*4epochs2005Reasoning stepsKodCode800\*4epochs2004GPT pass rate
### 8\.2Dataset Examples

This section shows representative examples from the three benchmarks\. For each example, we present the input, the expected output, and its corresponding difficulty level\.

### 8\.3Sudoku Example

InputFill the Sudoku\. Replace each "?" with a digit from 1 to 9\.Rules:Each row, each column, and each 3x3 box must contain the digits 1\-9 exactly once\.Output ONLY the final completed 9x9 grid in the same format\.Grid:\[\["?", 2, 7, 9, 3, 1, 4, 8, 5\], \[9, 3, 4, 5, 8, 6, 7, 1, 2\], \[1, 8, 5, 2, 7, "?", 6, "?", 3\], \[7, "?", "?", 4, 6, "?", "?", 3, 8\], \["?", 4, 6, "?", 9, 8, 5, 2, "?"\], \[8, "?", 9, 3, 2, 7, 1, 6, 4\], \[5, 9, 8, 7, "?", 3, 2, 4, 6\], \[4, 6, 1, 8, "?", "?", 3, "?", 9\], \[2, "?", 3, 6, 4, 9, 8, 5, 1\]\]Output\[\[6, 2, 7, 9, 3, 1, 4, 8, 5\], \[9, 3, 4, 5, 8, 6, 7, 1, 2\], \[1, 8, 5, 2, 7, 4, 6, 9, 3\], \[7, 1, 2, 4, 6, 5, 9, 3, 8\], \[3, 4, 6, 1, 9, 8, 5, 2, 7\], \[8, 5, 9, 3, 2, 7, 1, 6, 4\], \[5, 9, 8, 7, 1, 3, 2, 4, 6\], \[4, 6, 1, 8, 5, 2, 3, 7, 9\], \[2, 7, 3, 6, 4, 9, 8, 5, 1\]\]Difficulty Level: 1

### 8\.4iGSM Example

InputThe number of each NetVersity’s Microbiology Lab equals 4 times as much as each NetLearn Institute’s Zoology Lab\. The number of each NetVersity’s Zoology Lab equals 19 more than each NetLearn Institute’s Microbiology Lab\. The number of each NetLearn Institute’s Microbiology Lab equals 4\. The number of each NetLearn Institute’s Zoology Lab equals each NetVersity’s Zoology Lab\. How many Microbiology Lab does NetVersity have?OutputDefine NetLearn Institute’s Microbiology Lab as G; so G = 4\. Define NetVersity’s Zoology Lab as S; so S = 19 \+ G = 19 \+ 4 = 0\. Define NetVersity’s Microbiology Lab as P; so P = 4 \* S = 4 \* 0 = 0\. Answer: 0Difficulty Level: 4

### 8\.5KodCode Example

Inputdef permute\(nums\): """ Generates all possible permutations of a given list using backtracking\.:param nums: List of elements to permute:return: A list of all permutations\>\>\> permute\(\[\]\) == \[\[\]\]\>\>\> permute\(\[1\]\) == \[\[1\]\]\>\>\> permute\(\[1, 2\]\) == \[\[1, 2\], \[2, 1\]\]\>\>\> result = permute\(\[1, 2, 3\]\)\>\>\> expected = \[\[1, 2, 3\], \[1, 3, 2\], \[2, 1, 3\], \[2, 3, 1\], \[3, 1, 2\], \[3, 2, 1\]\]\>\>\> sorted\(result\) == sorted\(expected\)\>\>\> result = permute\(\[1, 1\]\)\>\>\> expected = \[\[1, 1\], \[1, 1\]\]\>\>\> sorted\(result\) == sorted\(expected\)"""Output

```
def permute(nums):
   """
   Generates all possible permutations of a     given list using
backtracking.
   :param nums: List of elements to permute
   :return: A list of all permutations
   """
   results = []
   def backtrack(path):
     if not nums:
       results.append(path)
       return
     for i in range(len(nums)):
       backtrack(path + [nums[i]])
       nums.pop(i)
   backtrack([])
   return results
```

Difficulty Level: 1

## 9Training Details

This section summarizes the implementation details used in all experiments\. We fine\-tune the 7B model on 8 NVIDIA RTX 5090 GPUs, and all models smaller than 7B on 8 NVIDIA RTX 4090 GPUs\.

Table 8:Training hyperparameters used in all experiments\.HyperparameterValueFine\-tuning methodLoRALoRA rank64LoRA alpha128LoRA dropout0\.05Target modulesallOptimizerAdamWLearning rate5e\-5SchedulerConstantEpochs4Batch size / GPU4/8Gradient accumulation1Gradient checkpointingYesMaximum sequence length512During the training procedure of dynamic sampling curriculum learning, we sample and reweigh every difficulty per 25 steps, using 8 examples each difficulty level to calculate the Re matrix, which ensures evaluation accuracy while have a very low extra computation cost\.

### 9\.1Hyperparameter Sensitivity Analysis

We conduct a sensitivity analysis of the two hyperparameters in our transfer\-aware dynamic curriculum sampling strategy, namelyτe\\tau\_\{e\}andτh\\tau\_\{h\}\. The thresholdτe\\tau\_\{e\}determines which easier difficulty levels are considered for replay, whileτh\\tau\_\{h\}controls when a harder difficulty level receives additional sampling probability based on its relative transferability\. The results are reported in Table[9](https://arxiv.org/html/2608.17268#S9.T9)\.

The results reveal that the two hyperparameters affect different aspects of the sampling strategy\. As shown in the left table, varyingτe\\tau\_\{e\}mainly affects the iGSM and Sudoku benchmarks\. In particular, a lowerτe\\tau\_\{e\}changes the set of easier problems selected for replay, which has a more noticeable effect on tasks where replaying easier examples is beneficial\. This effect is especially apparent on Sudoku, while the performance on Code remains relatively stable across the tested values\.

In contrast,τh\\tau\_\{h\}has a stronger influence on Sudoku, as shown in the right table\. This is consistent with the characteristics of Sudoku, where the transfer relationship between easier and harder difficulty levels can be more pronounced\. Increasing or decreasingτh\\tau\_\{h\}changes when additional probability is allocated to harder difficulty levels, thereby directly affecting the balance between easier and harder examples\.

Importantly, the middle configuration in each table corresponds to the hyperparameter values used by our final method, namelyτe=0\.5\\tau\_\{e\}=0\.5andτh=0\.85\\tau\_\{h\}=0\.85\. These settings achieve consistently strong performance across all three benchmarks, suggesting that they provide a robust balance between replaying easier problems and allocating additional probability to harder problems\. We therefore useτe=0\.5\\tau\_\{e\}=0\.5andτh=0\.85\\tau\_\{h\}=0\.85throughout the main experiments\.

Table 9:Sensitivity analysis ofτe\\tau\_\{e\}andτh\\tau\_\{h\}\. The middle row in each table corresponds to the configuration used in our final method\.\(a\) Varyingτe\\tau\_\{e\}τe\\tau\_\{e\}iGSMCodeSudoku0\.70\.4120\.6030\.2120\.50\.4280\.6190\.2310\.30\.4310\.6100\.231
\(b\) Varyingτh\\tau\_\{h\}τh\\tau\_\{h\}iGSMCodeSudoku1\.050\.4280\.6190\.2280\.850\.4280\.6190\.2310\.550\.4050\.5980\.235

## References

- \[1\]Y\. Bengio, J\. Louradour, R\. Collobert, and J\. Weston\(2009\)Curriculum learning\.InProceedings of the 26th Annual International Conference on Machine Learning,ICML ’09,New York, NY, USA,pp\. 41–48\.External Links:ISBN 9781605585161,[Link](https://doi.org/10.1145/1553374.1553380),[Document](https://dx.doi.org/10.1145/1553374.1553380)Cited by:[§1](https://arxiv.org/html/2608.17268#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.17268#S2.SS1.p1.1)\.
- \[2\]A\. G\. et\.al\.\(2024\)The llama 3 herd of models\.External Links:2407\.21783,[Link](https://arxiv.org/abs/2407.21783)Cited by:[§6\.1](https://arxiv.org/html/2608.17268#S6.SS1.SSS0.Px2.p1.1)\.
- \[3\]Y\. Fu, F\. Hamman, and S\. Dutta\(2025\)T\-shirt: token\-selective hierarchical data selection for instruction tuning\.InAdvances in Neural Information Processing Systems,D\. Belgrave, C\. Zhang, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, and N\. Chen \(Eds\.\),Vol\.38,pp\. 113932–113958\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2025/file/a59ff5f7384176b5d14a5ded77c4aa4f-Paper-Conference.pdf)Cited by:[§2\.2](https://arxiv.org/html/2608.17268#S2.SS2.p1.1)\.
- \[4\]Z\. Hammoudeh and D\. Lowd\(2024\)Training data influence analysis and estimation: a survey\.Machine Learning113\(5\),pp\. 2351–2403\.External Links:ISSN 1573\-0565,[Link](http://dx.doi.org/10.1007/s10994-023-06495-7),[Document](https://dx.doi.org/10.1007/s10994-023-06495-7)Cited by:[§2\.2](https://arxiv.org/html/2608.17268#S2.SS2.p1.1)\.
- \[5\]X\. Hu, Y\. Tang, H\. Li, H\. Chen, Q\. Zhang, Z\. Shen, Y\. Zhang, H\. Wang, and J\. Zhao\(2026\)Fine\-grained data ordering improves fine\-tuning for large language models\.InFindings of the Association for Computational Linguistics: ACL 2026,M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 20406–20418\.External Links:[Link](https://aclanthology.org/2026.findings-acl.1021/),[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.1021),ISBN 979\-8\-89176\-395\-1Cited by:[§1](https://arxiv.org/html/2608.17268#S1.p1.1)\.
- \[6\]B\. Hui, J\. Yang, Z\. Cui, J\. Yang, D\. Liu, L\. Zhang, T\. Liu, J\. Zhang, B\. Yu, K\. Lu, K\. Dang, Y\. Fan, Y\. Zhang, A\. Yang, R\. Men, F\. Huang, B\. Zheng, Y\. Miao, S\. Quan, Y\. Feng, X\. Ren, X\. Ren, J\. Zhou, and J\. Lin\(2024\)Qwen2\.5\-coder technical report\.External Links:2409\.12186,[Link](https://arxiv.org/abs/2409.12186)Cited by:[§6\.1](https://arxiv.org/html/2608.17268#S6.SS1.SSS0.Px2.p1.1)\.
- \[7\]Y\. Jia, C\. Zhang, X\. Diao, X\. Yuan, Z\. Ouyang, C\. Ma, and S\. Vosoughi\(2026\)What makes a good curriculum? disentangling the effects of data ordering on LLM mathematical reasoning\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 34472–34488\.External Links:[Link](https://aclanthology.org/2026.acl-long.1591/),[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.1591),ISBN 979\-8\-89176\-390\-6Cited by:[§1](https://arxiv.org/html/2608.17268#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.17268#S2.SS1.p2.1)\.
- \[8\]L\. Jiang, D\. Meng, Q\. Zhao, S\. Shan, and A\. Hauptmann\(2015\)Self\-paced curriculum learning\.Proceedings of the AAAI Conference on Artificial Intelligence29\(1\)\.External Links:[Link](https://ojs.aaai.org/index.php/AAAI/article/view/9608),[Document](https://dx.doi.org/10.1609/aaai.v29i1.9608)Cited by:[§2\.1](https://arxiv.org/html/2608.17268#S2.SS1.p1.1)\.
- \[9\]J\. Kim and J\. Lee\(2024\)Strategic data ordering: enhancing large language model performance through curriculum learning\.External Links:2405\.07490,[Link](https://arxiv.org/abs/2405.07490)Cited by:[§1](https://arxiv.org/html/2608.17268#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.17268#S2.SS1.p2.1)\.
- \[10\]S\. Narvekar, B\. Peng, M\. Leonetti, J\. Sinapov, M\. E\. Taylor, and P\. Stone\(2020\)Curriculum learning for reinforcement learning domains: a framework and survey\.Journal of Machine Learning Research21\(181\),pp\. 1–50\.External Links:[Link](http://jmlr.org/papers/v21/20-212.html)Cited by:[§2\.1](https://arxiv.org/html/2608.17268#S2.SS1.p1.1)\.
- \[11\]X\. PENG, C\. Xia, X\. Yang, C\. Xiong, C\. Wu, and C\. Xing\(2025\)ReGenesis: llms can grow into reasoning generalists via self\-improvement\.InInternational Conference on Learning Representations,Y\. Yue, A\. Garg, N\. Peng, F\. Sha, and R\. Yu \(Eds\.\),Vol\.2025,pp\. 62484–62519\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/file/9c77f2ce42151b2c2e26d2cf47f99564-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2608.17268#S1.p2.1)\.
- \[12\]S\. Rampp, M\. Milling, A\. Triantafyllopoulos, and B\. W\. Schuller\(2024\)Does the definition of difficulty matter? scoring functions and their role for curriculum learning\.External Links:2411\.00973,[Link](https://arxiv.org/abs/2411.00973)Cited by:[§2\.1](https://arxiv.org/html/2608.17268#S2.SS1.p2.1)\.
- \[13\]L\. Ranaldi, G\. Pucci, and A\. Freitas\(2024\)Does theOrdermatter? Curriculum learning over languages\.InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING 2024\),N\. Calzolari, M\. Kan, V\. Hoste, A\. Lenci, S\. Sakti, and N\. Xue \(Eds\.\),Torino, Italia,pp\. 5212–5220\.External Links:[Link](https://aclanthology.org/2024.lrec-main.464/)Cited by:[§1](https://arxiv.org/html/2608.17268#S1.p2.1)\.
- \[14\]Z\. Ren, D\. Dong, H\. Li, and C\. Chen\(2018\)Self\-paced prioritized curriculum learning with coverage penalty in deep reinforcement learning\.IEEE Transactions on Neural Networks and Learning Systems29\(6\),pp\. 2216–2226\.External Links:[Document](https://dx.doi.org/10.1109/TNNLS.2018.2790981)Cited by:[§2\.1](https://arxiv.org/html/2608.17268#S2.SS1.p1.1)\.
- \[15\]N\. Sachdeva, B\. Coleman, W\. Kang, J\. Ni, L\. Hong, E\. H\. Chi, J\. Caverlee, J\. McAuley, and D\. Z\. Cheng\(2024\)How to train data\-efficient llms\.External Links:2402\.09668,[Link](https://arxiv.org/abs/2402.09668)Cited by:[§1](https://arxiv.org/html/2608.17268#S1.p1.1)\.
- \[16\]A\. Silva, R\. Chopra, and M\. Gombolay\(2022\)Cross\-loss influence functions to explain deep network representations\.InProceedings of The 25th International Conference on Artificial Intelligence and Statistics,G\. Camps\-Valls, F\. J\. R\. Ruiz, and I\. Valera \(Eds\.\),Proceedings of Machine Learning Research, Vol\.151,pp\. 1–17\.External Links:[Link](https://proceedings.mlr.press/v151/silva22a.html)Cited by:[§2\.2](https://arxiv.org/html/2608.17268#S2.SS2.p1.1)\.
- \[17\]P\. Soviany, R\. T\. Ionescu, P\. Rota, and N\. Sebe\(2022\)Curriculum learning: a survey\.External Links:2101\.10382,[Link](https://arxiv.org/abs/2101.10382)Cited by:[§2\.1](https://arxiv.org/html/2608.17268#S2.SS1.p1.1)\.
- \[18\]C\. Tao and S\. Xing1\(2026\)Dynamic curriculum learning over difficulty heterogeneity\.InKnowledge Science, Engineering and Management: 19th International Conference, KSEM 2026, Beijing, China, July 17–19, 2026, Proceedings, Part I,pp\. 158\.Cited by:[§2\.1](https://arxiv.org/html/2608.17268#S2.SS1.p2.1)\.
- \[19\]H\. Wang, A\. H\. Li, Y\. Hu, S\. Zhang, H\. Kobayashi, J\. Zhang, H\. Zhu, C\. Hang, and P\. Ng\(2026\)DSMentor: curriculum\-guided inference with online memory for data\-science LLM agents\.InProceedings of the First Workshop on Structured Understanding, Retrieval, and Generation in the LLM Era \(SURGeLLM 2026\),V\. Gupta, K\. Ding, H\. Kokel, Y\. Zhao, A\. Agarwal, Y\. Wang, M\. Glass, Y\. Zhang, K\. Srinivas, X\. Chen, O\. Hassanzadeh, Q\. Zhu, S\. Chang, and Y\. Luo \(Eds\.\),San Diego, California, United States,pp\. 190–208\.External Links:[Link](https://aclanthology.org/2026.surgellm-1.12/),[Document](https://dx.doi.org/10.18653/v1/2026.surgellm-1.12),ISBN 979\-8\-89176\-406\-4Cited by:[§2\.1](https://arxiv.org/html/2608.17268#S2.SS1.p1.1)\.
- \[20\]H\. Wang, Z\. Hao, J\. Luo, C\. Wei, Y\. Shu, L\. Liu, Q\. Lin, H\. Dong, and J\. Chen\(2026\)Scheduling your llm reinforcement learning with reasoning trees\.External Links:2510\.24832,[Link](https://arxiv.org/abs/2510.24832)Cited by:[§1](https://arxiv.org/html/2608.17268#S1.p1.1)\.
- \[21\]J\. Wang, X\. Lin, R\. Qiao, P\. W\. Koh, C\. Foo, and B\. K\. H\. Low\(2025\)NICE data selection for instruction tuning in LLMs with non\-differentiable evaluation metric\.InProceedings of the 42nd International Conference on Machine Learning,A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, and J\. Zhu \(Eds\.\),Proceedings of Machine Learning Research, Vol\.267,pp\. 63662–63689\.External Links:[Link](https://proceedings.mlr.press/v267/wang25bm.html)Cited by:[§2\.2](https://arxiv.org/html/2608.17268#S2.SS2.p1.1)\.
- \[22\]X\. Wang, Y\. Chen, and W\. Zhu\(2022\)A survey on curriculum learning\.IEEE Transactions on Pattern Analysis and Machine Intelligence44\(9\),pp\. 4555–4576\.External Links:[Document](https://dx.doi.org/10.1109/TPAMI.2021.3069908)Cited by:[§1](https://arxiv.org/html/2608.17268#S1.p2.1)\.
- \[23\]X\. Wang, Y\. Zhou, H\. Chen, and W\. Zhu\(2024\)Curriculum learning: theories, approaches, applications, tools, and future directions in the era of large language models\.InCompanion Proceedings of the ACM Web Conference 2024,WWW ’24,New York, NY, USA,pp\. 1306–1310\.External Links:ISBN 9798400701726,[Link](https://doi.org/10.1145/3589335.3641257),[Document](https://dx.doi.org/10.1145/3589335.3641257)Cited by:[§1](https://arxiv.org/html/2608.17268#S1.p2.1)\.
- \[24\]D\. Weinshall, G\. Cohen, and D\. Amir\(2018\)Curriculum learning by transfer learning: theory and experiments with deep networks\.InProceedings of the 35th International Conference on Machine Learning,J\. Dy and A\. Krause \(Eds\.\),Proceedings of Machine Learning Research, Vol\.80,pp\. 5238–5246\.External Links:[Link](https://proceedings.mlr.press/v80/weinshall18a.html)Cited by:[§1](https://arxiv.org/html/2608.17268#S1.p2.1)\.
- \[25\]Z\. H\. Wong, H\. Yang, Q\. Yao, and Y\. Wang\(2026\)Robust heterogeneous network representation learning by multifaceted curriculum training\.Neural Networks196,pp\. 108438\.External Links:ISSN 0893\-6080,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.neunet.2025.108438),[Link](https://www.sciencedirect.com/science/article/pii/S089360802501319X)Cited by:[§2\.1](https://arxiv.org/html/2608.17268#S2.SS1.p2.1)\.
- \[26\]M\. Wu, Q\. Qian, W\. Liu, X\. Wang, Z\. Huang, D\. Liang, L\. Miao, S\. Dou, C\. Lv, Z\. Wang, Z\. Xu, L\. Chen, T\. Li, X\. Zheng, and X\. Huang\(2025\)Progressive mastery: customized curriculum learning with guided prompting for mathematical reasoning\.External Links:2506\.04065,[Link](https://arxiv.org/abs/2506.04065)Cited by:[§1](https://arxiv.org/html/2608.17268#S1.p3.1)\.
- \[27\]M\. Xia, S\. Malladi, S\. Gururangan, S\. Arora, and D\. Chen\(2024\)LESS: selecting influential data for targeted instruction tuning\.External Links:2402\.04333,[Link](https://arxiv.org/abs/2402.04333)Cited by:[§2\.2](https://arxiv.org/html/2608.17268#S2.SS2.p1.1)\.
- \[28\]B\. Xu, L\. Zhang, Z\. Mao, Q\. Wang, H\. Xie, and Y\. Zhang\(2020\)Curriculum learning for natural language understanding\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,D\. Jurafsky, J\. Chai, N\. Schluter, and J\. Tetreault \(Eds\.\),Online,pp\. 6095–6104\.External Links:[Link](https://aclanthology.org/2020.acl-main.542/),[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.542)Cited by:[§1](https://arxiv.org/html/2608.17268#S1.p2.1)\.
- \[29\]Z\. Xu, Y\. Liu, Y\. Yin, M\. Zhou, and R\. Poovendran\(2025\)KodCode: a diverse, challenging, and verifiable synthetic dataset for coding\.InFindings of the Association for Computational Linguistics: ACL 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 6980–7008\.External Links:[Link](https://aclanthology.org/2025.findings-acl.365/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.365),ISBN 979\-8\-89176\-256\-5Cited by:[§3](https://arxiv.org/html/2608.17268#S3.p1.1),[§6\.1](https://arxiv.org/html/2608.17268#S6.SS1.SSS0.Px1.p1.1)\.
- \[30\]Z\. Xu, R\. Zhang, Y\. Chuang, X\. Lou, H\. A\. D\. Le, O\. Gal, A\. S\. Szalay, Z\. Xu, G\. Wang, and V\. Braverman\(2026\)Learning at the right pace: adaptive data scheduling improves llm reinforcement learning\.External Links:2606\.22305,[Link](https://arxiv.org/abs/2606.22305)Cited by:[§1](https://arxiv.org/html/2608.17268#S1.p1.1),[§1](https://arxiv.org/html/2608.17268#S1.p3.1)\.
- \[31\]C\. Yang, Z\. Xiang, Y\. Tang, Z\. Teng, C\. Huang, F\. Long, Y\. Liu, and J\. Su\(2026\)TTCS: test\-time curriculum synthesis for self\-evolving\.External Links:2601\.22628,[Link](https://arxiv.org/abs/2601.22628)Cited by:[§1](https://arxiv.org/html/2608.17268#S1.p2.1)\.
- \[32\]T\. Ye, Z\. Xu, Y\. Li, and Z\. Allen\-Zhu\(2025\)Physics of language models: part 2\.2, how to learn from mistakes on grade\-school math problems\.InInternational Conference on Learning Representations,Y\. Yue, A\. Garg, N\. Peng, F\. Sha, and R\. Yu \(Eds\.\),Vol\.2025,pp\. 78136–78147\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/file/c239bac713017b0b2257b7622bf8aab3-Paper-Conference.pdf)Cited by:[§3](https://arxiv.org/html/2608.17268#S3.p1.1),[§6\.1](https://arxiv.org/html/2608.17268#S6.SS1.SSS0.Px1.p1.1)\.
- \[33\]Z\. Ye, X\. Li, Q\. Li, Q\. Ai, Y\. Zhou, W\. Shen, D\. Yan, and Y\. Liu\(2025\)Learning llm\-as\-a\-judge for preference alignment\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 3537–3564\.Cited by:[§2\.2](https://arxiv.org/html/2608.17268#S2.SS2.p1.1)\.
- \[34\]D\. Zhang, Q\. Dai, and H\. Peng\(2025\)The best instruction\-tuning data are those that fit\.InAdvances in Neural Information Processing Systems,D\. Belgrave, C\. Zhang, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, and N\. Chen \(Eds\.\),Vol\.38,pp\. 141172–141208\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2025/file/cea04322465ad2f261f08e5b47ba9e7a-Paper-Conference.pdf)Cited by:[§2\.2](https://arxiv.org/html/2608.17268#S2.SS2.p1.1)\.
- \[35\]Y\. Zhang, A\. Mohamed, H\. Abdine, G\. Shang, and M\. Vazirgiannis\(2026\)Beyond random sampling: efficient language model pretraining via curriculum learning\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\),V\. Demberg, K\. Inui, and L\. Marquez \(Eds\.\),Rabat, Morocco,pp\. 5776–5794\.External Links:[Link](https://aclanthology.org/2026.eacl-long.271/),[Document](https://dx.doi.org/10.18653/v1/2026.eacl-long.271),ISBN 979\-8\-89176\-380\-7Cited by:[§1](https://arxiv.org/html/2608.17268#S1.p2.1)\.