SLIM-RL: Risk-Budgeted Random-Masking RL for Diffusion LLMs Without Trajectory Slicing

# Risk-Budgeted Random-Masking RL for Diffusion LLMs Without Trajectory Slicing
Source: [https://arxiv.org/html/2607.00208](https://arxiv.org/html/2607.00208)
Ruikang Zhao Technical University of Denmark &Zhenting Wang MBZUAI Institute of Foundation Models &Han Gao Iowa State University &Ligong Han\* Red Hat AI Innovation & MIT–IBM Watson AI Lab

###### Abstract

Reinforcement learning for diffusion large language models \(dLLMs\) has largely moved to trajectory\-aware methods\. The current state of the art, TraceRL, holds that random masking is mismatched with the model’s inference trajectory, and it reconstructs that trajectory during training by slicing each rollout into up to⌈K/s⌉\\lceil K/s\\rceiltrajectory\-aligned training samples, a cost that grows with the block sizeKK\. We show that this mismatch can be mitigated without reconstructing the trajectory\. Our method,SLIM\-RL, bounds the commit risk of each rollout step with aτ\\tau\-budget decoder, reducing aggregate commit risk in the training data\. During optimization, SLIM\-RL trains on these risk\-controlled rollouts with a trace\-free random\-masking objective that adapts variance\-reduction tools, combining sequence\-level importance sampling, deterministic quadrature over masking levels under a mean\-preserving, monotonically decreasing per\-block mask schedule that we introduce\. On SDAR\-4B, SLIM\-RL matches TraceRL’s best MATH500 accuracy on only0\.46×0\.46\\timesits training samples at block size 16, improving over TraceRL by6\.32%6\.32\\%on MATH500 and11\.05%11\.05\\%on GSM8K under matched dynamic sampling\. At block size 4, the44B SLIM\-RL surpasses the larger LLaDA\-8B and Dream\-7B dLLMs on math, exceeding LLaDA\-8B by10\.76%10\.76\\%on MATH500 while staying below the autoregressive Qwen2\.5\-7B\. On code, it improves over TraceRL by4\.20%4\.20\\%on MBPP and3\.65%3\.65\\%on HumanEval\. Theτ\\tau\-budget decoder transfers training\-free across LLaDA, Dream, and SDAR\. The source code is available at https://github\.com/laolaorkkkkk/SLIM\-RL \.

## 1Introduction

Diffusion Large Language Models \(dLLMs\)\(Nie et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib19); Sahoo et al\.,[2024](https://arxiv.org/html/2607.00208#bib.bib23); Ye et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib31)\)generate text by iteratively denoising a masked sequence, refining many positions at once instead of strictly left to right\. Block\-wise variants\(Arriola et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib1); Cheng et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib6)\)decode consecutive blocks autoregressively while denoising within each block, which restores KV\-cache reuse\. Reinforcement learning has become the standard way to improve their reasoning\(Zhao et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib33); Wang et al\.,[2025b](https://arxiv.org/html/2607.00208#bib.bib28); Hu et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib13); He et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib11); Zhu et al\.,[2025b](https://arxiv.org/html/2607.00208#bib.bib36); Tang et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib25); Liu et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib17); Zhu et al\.,[2025a](https://arxiv.org/html/2607.00208#bib.bib35)\)\.

Early work applies RL to randomly masked targets\(Zhao et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib33); Yang et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib30)\), but random masking yields high\-variance gradient estimates\(Zhu et al\.,[2025a](https://arxiv.org/html/2607.00208#bib.bib35)\), and recent work has therefore moved toward trajectory\-aware methods\(Huang et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib14); Wang et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib26)\)\. The current state of the art, TraceRL\(Wang et al\.,[2025b](https://arxiv.org/html/2607.00208#bib.bib28)\), trains not on random masks but on the model’s exact decoding trajectory, on the view that under random masking the post\-training objective is mismatched with the trajectory the model follows at inference\.

Preserving the trajectory is not free\. To train on each decoding step in order, TraceRL slices a single rollout into up to⌈K/s⌉\\lceil K/s\\rceiltrajectory\-aligned training samples, one forward each, whereKKis the block size ands≥1s\{\\geq\}1a shrinkage factor\(Wang et al\.,[2025b](https://arxiv.org/html/2607.00208#bib.bib28)\)\. At full fidelity \(s=1s\{=\}1\) one rollout becomes up toKKsamples, so the data grows withKK\. Raising the shrinkagessbounds the cost but aggregatesssconsecutive decoding steps into one slice and discards their internal order, trading trajectory fidelity for cost\. A larger block therefore makes a trade\-off between slicing cost and trajectory fidelity, which we quantify in Section[4\.5](https://arxiv.org/html/2607.00208#S4.SS5)\.

The question is whether reconstructing the exact trajectory is necessary\. Without slicing, a dLLM\-RL run still faces two weaknesses at two independent stages, rollout generation and optimization\. Rollouts are generated with dynamic sampling\(Wu et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib29); Yu et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib32)\), the common rollout decoder across dLLM\-RL methods including TraceRL, which commits every token whose confidence exceeds a fixed thresholdτ\\tau\. Because this rule is pointwise, a step that unmasks many positions only marginally aboveτ\\tauover\-commits and injects several errors into the rollout\. At optimization, random masking makes the policy\-gradient estimate high\-variance\(Zhu et al\.,[2025a](https://arxiv.org/html/2607.00208#bib.bib35)\)\.

In this work, we address both stages and call the resulting recipeSLIM\-RL, which combines three components developed in Section[3](https://arxiv.org/html/2607.00208#S3)\. Aτ\\tau\-budget dynamic\-unmasking decoder, a single\-pass training\-free rule shared by rollout and inference, caps each step’s cumulative confidence\-based uncertainty\. A variance\-reduced framework then updates the policy, built from a sequence\-level length\-normalized ratio\(Zheng et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib34)\), deterministic quadrature over masking levels\(Rojas et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib22)\), and an unnormalized advantage\(Liu et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib18)\)\. A mean\-preserving, monotonically decreasing per\-block mask schedule front\-loads the policy\-gradient signal onto the earliest, most\-conditioned block\.

On SDAR\-4B, SLIM\-RL outperforms trajectory\-aware TraceRL on math at block size 16 and on both math and code at block size 4, where the 4B model also surpasses the larger LLaDA\-8B and Dream\-7B diffusion models, and reaches TraceRL’s best accuracy on under half the training data \(Table[6](https://arxiv.org/html/2607.00208#A1.T6)\)\. The margin widens with block size, and Section[4](https://arxiv.org/html/2607.00208#S4)reports the per\-benchmark results\.

Our contributions are summarized as follows:

- •We show that reconstructing the exact decoding trajectory is not required to match trajectory\-aware RL at equal training cost\. With rollout commit risk bounded and the masking objective variance\-reduced, trace\-free random masking is on par with trajectory\-aligned slicing at block size 4 and outperforms it at block size 16\.
- •We introduce aτ\\tau\-budget dynamic\-unmasking decoder that commits only the largest low\-uncertainty subset of positions whose cumulative uncertainty∑i\(1−pi\)\\sum\_\{i\}\(1\-p\_\{i\}\)stays within a calibrated budgetm​\(1−τ\)m\(1\-\\tau\), in contrast to dynamic sampling’s per\-position threshold\. It is training\-free and transfers across LLaDA, Dream, and SDAR\.
- •We introduce a mean\-preserving, monotonically decreasing per\-block mask schedule that concentrates the policy\-gradient signal on the earliest, most\-conditioned block, adding5\.91%5\.91\\%on MATH500 over random masking\.
- •On SDAR\-4B at block size 16, SLIM\-RL achieves6\.32%6\.32\\%and11\.05%11\.05\\%higher accuracy than TraceRL on MATH500 and GSM8K under matched decoding, and reaches TraceRL’s best accuracy on0\.46×0\.46\\timesthe training data\. At block size 4, it achieves higher accuracy than the larger LLaDA\-8B and Dream\-7B diffusion models across math and code, and improves over TraceRL on code by4\.20%4\.20\\%on MBPP and3\.65%3\.65\\%on HumanEval\. On SDAR\-1\.7B, SLIM\-RL reaches TraceRL’s best on0\.76×0\.76\\timesthe training data\.

## 2Preliminaries

### 2\.1Diffusion Large Language Models

#### Forward and reverse processes\.

Given a promptxxand a clean responsey=\(y1,…,yL\)y=\(y^\{1\},\\ldots,y^\{L\}\), a diffusion large language model \(dLLM\)\(Nie et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib19); Ye et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib31)\)defines a forward corruption process and a learned reverse generation process\. The forward process independently replaces each token with the mask symbol\[MASK\]\[\\mathrm\{MASK\}\]at masking levelt∈\[0,1\]t\\in\[0,1\]:

qt​\(yt∣y\)=∏i=1Lqt​\(yti∣yi\),q\_\{t\}\(y\_\{t\}\\mid y\)=\\prod\_\{i=1\}^\{L\}q\_\{t\}\(y\_\{t\}^\{i\}\\mid y^\{i\}\),\(1\)where each token is independently masked according toqt​\(yti=\[MASK\]∣yi\)=tq\_\{t\}\(y\_\{t\}^\{i\}\{=\}\[\\mathrm\{MASK\}\]\\mid y^\{i\}\)=tandqt​\(yti=yi∣yi\)=1−tq\_\{t\}\(y\_\{t\}^\{i\}\{=\}y^\{i\}\\mid y^\{i\}\)=1\-t, so thaty0y\_\{0\}is the clean response andy1y\_\{1\}is fully masked\. For reverse process, starting fromy1y\_\{1\}, the model iteratively predicts masked positions and resamples a less\-noisedyry\_\{r\}fromyty\_\{t\}\(r<tr<t\) untily0y\_\{0\}is recovered\.

#### Block\-wise dLLMs\.

Vanilla dLLMs denoise the full response under bidirectional attention, which prevents KV\-cache reuse and is expensive for long responses\. Block\-wise dLLMs\(Arriola et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib1); Cheng et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib6)\)partition the response intoBBconsecutive blocksY1,…,YBY\_\{1\},\\ldots,Y\_\{B\}of sizeKK, whereYb=\(y\(b−1\)​K\+1,…,yb​K\)Y\_\{b\}=\(y^\{\(b\-1\)K\+1\},\\ldots,y^\{bK\}\)\. Attention is bidirectional within a block and causal across blocks, so blocks are generated sequentially while tokens within each block are denoised in parallel\. This yields the block\-causal factorization

πθ​\(y∣x\)=∏b=1Bπθ​\(Yb∣x,Y<b\)\\pi\_\{\\theta\}\(y\\mid x\)=\\prod\_\{b=1\}^\{B\}\\pi\_\{\\theta\}\(Y\_\{b\}\\mid x,Y\_\{<b\}\)\(2\)which preserves KV\-cache compatibility across blocks while retaining intra\-block parallelism\.

#### Confidence\-driven decoding\.

At each denoising stepss, the model scores every masked position by its max\-confidencepi=maxv∈𝒱⁡pθ​\(y0i=v∣ys,x\)p\_\{i\}=\\max\_\{v\\in\\mathcal\{V\}\}p\_\{\\theta\}\(y\_\{0\}^\{i\}=v\\mid y\_\{s\},x\), whereysy\_\{s\}is the partially denoised sequence at stepss, and commits positioniionly whenpi\>τp\_\{i\}\>\\taufor a fixed thresholdτ\\tau\. Aggressive thresholds \(smallτ\\tau\) commit more tokens per step but inject errors that propagate through later denoising\. Conservative thresholds preserve quality but require more refinement steps, eroding the parallelism advantage of dLLMs\.

### 2\.2GRPO for dLLMs

For each promptx∼𝒟x\\sim\\mathcal\{D\}, the old policy samples a group ofGGresponses\{yj\}j=1G∼πθold\(⋅∣x\)\\\{y\_\{j\}\\\}\_\{j=1\}^\{G\}\\sim\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(\\cdot\\mid x\)with scalar rewards\{rj\}j=1G\\\{r\_\{j\}\\\}\_\{j=1\}^\{G\}and the standardized advantageA^j=\(rj−mean⁡\{ri\}\)/std⁡\{ri\}\\hat\{A\}\_\{j\}=\(r\_\{j\}\-\\operatorname\{mean\}\\\{r\_\{i\}\\\}\)/\\operatorname\{std\}\\\{r\_\{i\}\\\}\. Standard GRPO\(Shao et al\.,[2024](https://arxiv.org/html/2607.00208#bib.bib24)\)defines a token\-level importance ratioρjk​\(θ\)=πθ​\(yjk∣x,yj<k\)/πθold​\(yjk∣x,yj<k\)\\rho\_\{j\}^\{k\}\(\\theta\)=\\pi\_\{\\theta\}\(y\_\{j\}^\{k\}\\mid x,y\_\{j\}^\{<k\}\)/\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(y\_\{j\}^\{k\}\\mid x,y\_\{j\}^\{<k\}\)and maximizes the per\-token clipped surrogate1G​∑j1\|yj\|​∑kmin⁡\(ρjk​A^j,clip⁡\(ρjk,1−ϵ,1\+ϵ\)​A^j\)\\frac\{1\}\{G\}\\sum\_\{j\}\\frac\{1\}\{\|y\_\{j\}\|\}\\sum\_\{k\}\\min\\\!\\bigl\(\\rho\_\{j\}^\{k\}\\hat\{A\}\_\{j\},\\operatorname\{clip\}\(\\rho\_\{j\}^\{k\},1\{\-\}\\epsilon,1\{\+\}\\epsilon\)\\hat\{A\}\_\{j\}\\bigr\), regularized byβ​DKL​\[πθ∥πref\]\\beta D\_\{\\mathrm\{KL\}\}\[\\pi\_\{\\theta\}\\,\\\|\\,\\pi\_\{\\mathrm\{ref\}\}\]toward a reference policy\.

For dLLMs, the per\-token conditionalπθ​\(yjk∣x,yj<k\)\\pi\_\{\\theta\}\(y\_\{j\}^\{k\}\\mid x,y\_\{j\}^\{<k\}\)is intractable in closed form because the model parameterizespθ\(⋅∣yt,x\)p\_\{\\theta\}\(\\cdot\\mid y\_\{t\},x\)at masked positions rather than via a left\-to\-right factorization\. Random\-masking dLLM\-RL methods\(Gong et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib9); Yang et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib30)\)factorize the sequence probability into per\-token denoising terms and approximate the ratio on sampled corruptionsy~j​\(t\)\\widetilde\{y\}\_\{j\}\(t\)of each response at masking levelt∼U​\[0,1\]t\\sim\\mathrm\{U\}\[0,1\]:

ρjk​\(θ\)≈pθ​\(yjk∣x,y~j​\(t\)\)pθold​\(yjk∣x,y~j​\(t\)\),k∈𝒰j​\(t\),\\rho\_\{j\}^\{k\}\(\\theta\)\\;\\approx\\;\\frac\{p\_\{\\theta\}\\bigl\(y\_\{j\}^\{k\}\\mid x,\\widetilde\{y\}\_\{j\}\(t\)\\bigr\)\}\{p\_\{\\theta\_\{\\mathrm\{old\}\}\}\\bigl\(y\_\{j\}^\{k\}\\mid x,\\widetilde\{y\}\_\{j\}\(t\)\\bigr\)\},\\quad k\\in\\mathcal\{U\}\_\{j\}\(t\),\(3\)where𝒰j​\(t\)\\mathcal\{U\}\_\{j\}\(t\)denotes the masked positions iny~j​\(t\)\\widetilde\{y\}\_\{j\}\(t\)\.

## 3Method

SLIM\-RL separates dLLM RL into two stages: risk\-controlled rollout collection, then trace\-free low\-variance optimization\. Aτ\\tau\-budget decoder \(Section[3\.1](https://arxiv.org/html/2607.00208#S3.SS1)\) shapes the rollouts by bounding the cumulative uncertainty committed at each denoising step; a low\-variance random\-masking objective then updates the policy on them \(Sections[3\.2](https://arxiv.org/html/2607.00208#S3.SS2)and[3\.3](https://arxiv.org/html/2607.00208#S3.SS3)\), with a mean\-preserving decreasing per\-block mask schedule \(Section[3\.4](https://arxiv.org/html/2607.00208#S3.SS4)\) concentrating the gradient signal on the earliest, most\-conditioned block\.

### 3\.1τ\\tau\-Budget Dynamic Unmasking

τ\\tau\-budget dynamic unmasking is training\-free and used for both RL rollout generation and test\-time inference\. At denoising stepss, letℳs\\mathcal\{M\}\_\{s\}denote the currently masked positions in the active block, and letpi=maxv∈𝒱⁡pθ​\(y0i=v∣ys,x\)p\_\{i\}=\\max\_\{v\\in\\mathcal\{V\}\}p\_\{\\theta\}\(y\_\{0\}^\{i\}=v\\mid y\_\{s\},x\)be the model’s max\-confidence at positionii\. The threshold\-based candidate set

𝒜s=\{i∈ℳs:pi\>τ\}\\mathcal\{A\}\_\{s\}=\\\{i\\in\\mathcal\{M\}\_\{s\}:p\_\{i\}\>\\tau\\\}\(4\)is the set of positions standard dynamic sampling would commit at stepss\. We assign each candidate the confidence\-based uncertaintyui=1−piu\_\{i\}=1\-p\_\{i\}and define the cumulative step\-level uncertainty of a set𝒮⊆𝒜s\\mathcal\{S\}\\subseteq\\mathcal\{A\}\_\{s\}asU​\(𝒮\)=∑i∈𝒮ui=∑i∈𝒮\(1−pi\)U\(\\mathcal\{S\}\)=\\sum\_\{i\\in\\mathcal\{S\}\}u\_\{i\}=\\sum\_\{i\\in\\mathcal\{S\}\}\(1\-p\_\{i\}\)\. Given a risk\-budget schedulem≥1m\\geq 1, we set the step\-level budgetℬs=m​\(1−τ\)\\mathcal\{B\}\_\{s\}=m\(1\-\\tau\)\. We sort the candidates by ascending uncertaintyui1≤ui2≤⋯≤ui\|𝒜s\|u\_\{i\_\{1\}\}\\leq u\_\{i\_\{2\}\}\\leq\\cdots\\leq u\_\{i\_\{\|\\mathcal\{A\}\_\{s\}\|\}\}and commit the largest prefix of this ordering whose cumulative uncertainty stays within the budget:

ks\\displaystyle k\_\{s\}=max⁡\{k:∑r=1kuir≤ℬs\},\\displaystyle=\\max\\left\\\{k:\\sum\_\{r=1\}^\{k\}u\_\{i\_\{r\}\}\\leq\\mathcal\{B\}\_\{s\}\\right\\\},\(5\)𝒞s\\displaystyle\\mathcal\{C\}\_\{s\}=\{i1,…,iks\}\.\\displaystyle=\\\{i\_\{1\},\\ldots,i\_\{k\_\{s\}\}\\\}\.
![Refer to caption](https://arxiv.org/html/2607.00208v1/x1.png)Figure 1:Dynamic sampling versusτ\\tau\-budget unmasking at one denoising step\.Both rules start from the same partially denoised block and the same above\-threshold candidate set𝒜s\\mathcal\{A\}\_\{s\}, the three positions whose confidence exceedsτ=0\.9\\tau\{=\}0\.9\.*\(A\)*Dynamic sampling commits every candidate, finalizing all three including the position with confidence0\.940\.94\.*\(B\)*τ\\tau\-budget assigns each candidate the uncertaintyui=1−piu\_\{i\}\{=\}1\{\-\}p\_\{i\}, sorts ascending, and commits only the largest prefix whose cumulative uncertainty stays within the budgetℬs=m​\(1−τ\)=0\.10\\mathcal\{B\}\_\{s\}\{=\}m\(1\{\-\}\\tau\)\{=\}0\.10\.The remaining positions stay masked for subsequent denoising steps\. If𝒜s=∅\\mathcal\{A\}\_\{s\}=\\emptyset, we commit the single masked position with the highest confidence\. Figure[1](https://arxiv.org/html/2607.00208#S3.F1)contrastsτ\\tau\-budget with dynamic sampling on a single denoising step, and Algorithm[2](https://arxiv.org/html/2607.00208#alg2)gives the full single\-pass decoder\.

#### Interpretation: a calibrated wrong\-commit budget\.

LetWrongCommits=∑i∈𝒞s𝟏​\{y^i≠yi∗\}\\mathrm\{WrongCommit\}\_\{s\}=\\sum\_\{i\\in\\mathcal\{C\}\_\{s\}\}\\mathbf\{1\}\\\{\\hat\{y\}\_\{i\}\\neq y\_\{i\}^\{\\ast\}\\\}count committed positions that disagree with the correct token\. If max\-confidence approximated the probability that the argmax is correct, then𝔼​\[WrongCommits\]≈∑i∈𝒞s\(1−pi\)=U​\(𝒞s\)≤ℬs\\mathbb\{E\}\[\\mathrm\{WrongCommit\}\_\{s\}\]\\approx\\sum\_\{i\\in\\mathcal\{C\}\_\{s\}\}\(1\-p\_\{i\}\)=U\(\\mathcal\{C\}\_\{s\}\)\\leq\\mathcal\{B\}\_\{s\}, so the budget caps a calibrated proxy for the expected wrong commitments per step\. Pointwise thresholding instead commits all of𝒜s\\mathcal\{A\}\_\{s\}, whose aggregate uncertainty grows with the number of accepted tokens, so many marginally\-above\-threshold positions can together inject large step\-level risk\. The calibration assumption holds only approximately, so we treat it as motivation andτ\\tau\-budget rollouts commit fewer expected\-wrong tokens per step than dynamic sampling \(Figure[5](https://arxiv.org/html/2607.00208#A1.F5)\)\.

### 3\.2Training Objective: Variance\-Reduced Random Masking

After rollouts are collected, SLIM\-RL does not reconstruct the token\-level decoding order\. The intrinsic weakness of random masking for RL is a high\-variance policy gradient\. The first source of variance is the granularity of the importance ratio\. A token\-level ratio attaches a response\-level advantage to many independently clipped per\-position ratios\. As argued by GSPO\(Zheng et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib34)\), such token\-level importance weights introduce high\-variance training noise that can accumulate with response length\. We therefore use a sequence\-level ratio, which is computed for the whole response at a given masking level\.

For a masking leveltt, lety~j​\(t\)\\widetilde\{y\}\_\{j\}\(t\)be a randomly masked version ofyjy\_\{j\}, let𝒰j​\(t\)\\mathcal\{U\}\_\{j\}\(t\)denote its masked positions, and letMj​\(t\)=\|𝒰j​\(t\)\|M\_\{j\}\(t\)=\|\\mathcal\{U\}\_\{j\}\(t\)\|\. The normalized sequence denoising log\-score is

ℓθj​\(t\)=1Mj​\(t\)​∑i∈𝒰j​\(t\)log⁡pθ​\(yji∣x,y~j​\(t\)\),\\ell\_\{\\theta\}^\{j\}\(t\)=\\frac\{1\}\{M\_\{j\}\(t\)\}\\sum\_\{i\\in\\mathcal\{U\}\_\{j\}\(t\)\}\\log p\_\{\\theta\}\\left\(y\_\{j\}^\{i\}\\mid x,\\widetilde\{y\}\_\{j\}\(t\)\\right\),\(6\)whereyjiy\_\{j\}^\{i\}is the clean token at masked positionii\.

The corresponding sequence\-level denoising ratio is the geometric mean of the per\-position denoising ratios over the masked positions,

ρj\(t\)​\(θ\)=exp⁡\(ℓθj​\(t\)−ℓθoldj​\(t\)\)=exp⁡\(1Mj​\(t\)​∑i∈𝒰j​\(t\)log⁡pθ​\(yji∣x,y~j​\(t\)\)pθold​\(yji∣x,y~j​\(t\)\)\)\.\\rho\_\{j\}^\{\(t\)\}\(\\theta\)=\\exp\\\!\\left\(\\ell\_\{\\theta\}^\{j\}\(t\)\-\\ell\_\{\\theta\_\{\\mathrm\{old\}\}\}^\{j\}\(t\)\\right\)=\\exp\\\!\\left\(\\frac\{1\}\{M\_\{j\}\(t\)\}\\sum\_\{i\\in\\mathcal\{U\}\_\{j\}\(t\)\}\\log\\frac\{p\_\{\\theta\}\(y\_\{j\}^\{i\}\\mid x,\\widetilde\{y\}\_\{j\}\(t\)\)\}\{p\_\{\\theta\_\{\\mathrm\{old\}\}\}\(y\_\{j\}^\{i\}\\mid x,\\widetilde\{y\}\_\{j\}\(t\)\)\}\\right\)\.\(7\)

### 3\.3Deterministic Quadrature and Unnormalized Advantage

The second source of variance is the masking leveltt, over which the random masking objective is an expectation\. Prior dLLM\-RL estimates this expectation stochastically, drawingttat random for each update\(Gong et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib9); Yang et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib30)\), andRojas et al\. \([2025](https://arxiv.org/html/2607.00208#bib.bib22)\)show that this random time is the dominant source of the estimator’s variance\. We instead evaluate the expectation deterministically with Gauss–Legendre quadrature over\(0,1\)\(0,1\)\. AtQQ\-point nodes and weights\{\(tq,ωq\)\}q=1Q\\\{\(t\_\{q\},\\omega\_\{q\}\)\\\}\_\{q=1\}^\{Q\},ρj\(tq\)​\(θ\)\\rho\_\{j\}^\{\(t\_\{q\}\)\}\(\\theta\)is the ratio of Eq\.[7](https://arxiv.org/html/2607.00208#S3.E7)att=tqt\{=\}t\_\{q\}, which removes the variance of the randomtt\. We also use the unnormalized advantage to remove bias\(Liu et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib18)\), and letAjA\_\{j\}denote the response\-level advantage assigned toyjy\_\{j\}\.

Aj=rj−1G​∑i=1Gri\.A\_\{j\}=r\_\{j\}\-\\frac\{1\}\{G\}\\sum\_\{i=1\}^\{G\}r\_\{i\}\.\(8\)
#### Final objective\.

The final objective combines the per\-quadrature\-node sequence ratiosρj\(tq\)​\(θ\)\\rho\_\{j\}^\{\(t\_\{q\}\)\}\(\\theta\), the unnormalized advantageAjA\_\{j\}of Eq\.[8](https://arxiv.org/html/2607.00208#S3.E8), and a KL regularizer toward the old policy\.

𝒥SLIM​\(θ\)=𝔼​\[1G​∑j=1G∑q=1Qωq​ψϵ​\(ρj\(tq\)​\(θ\),Aj\)−β​DKL​\(πθ∥πθold\)\]\.\\mathcal\{J\}\_\{\\mathrm\{SLIM\}\}\(\\theta\)=\\mathbb\{E\}\\Biggl\[\\frac\{1\}\{G\}\\sum\_\{j=1\}^\{G\}\\sum\_\{q=1\}^\{Q\}\\omega\_\{q\}\\,\\psi\_\{\\epsilon\}\\\!\\left\(\\rho\_\{j\}^\{\(t\_\{q\}\)\}\(\\theta\),\\,A\_\{j\}\\right\)\-\\beta\\,D\_\{\\mathrm\{KL\}\}\\\!\\left\(\\pi\_\{\\theta\}\\,\\\|\\,\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\\right\)\\Biggr\]\.\(9\)
We define the clipped policy objective term asψϵ​\(ρ,A\)=min⁡\(ρ​A,clip⁡\(ρ,1−ϵ,1\+ϵ\)​A\)\\psi\_\{\\epsilon\}\(\\rho,A\)=\\min\\\!\\left\(\\rho A,\\,\\operatorname\{clip\}\(\\rho,1\-\\epsilon,1\+\\epsilon\)A\\right\)\. Algorithm[1](https://arxiv.org/html/2607.00208#alg1)details the training pipeline, from risk\-budgeted rollout collection to the quadrature\-based policy update\.

### 3\.4Monotonically Decreasing Per\-Block Masking Schedule

Vanilla random\-masking RL masks each position independently\. We instead apply a monotonically decreasing per\-block schedulep1≥p2≥⋯≥pBp\_\{1\}\\geq p\_\{2\}\\geq\\cdots\\geq p\_\{B\}across theBBblocksY1,…,YBY\_\{1\},\\dots,Y\_\{B\}of a response, masking blockYbY\_\{b\}at ratepbp\_\{b\}\. Under the block\-causal factorization of Eq\. \([2](https://arxiv.org/html/2607.00208#S2.E2)\) the earliest block conditions every later block, so masking it most concentrates the learning signal on the positions the rest of the response depends on, while later blocks stay progressively less masked\. The per\-block rates average to the global masking leveltt\(1B​∑bpb=t\\tfrac\{1\}\{B\}\\sum\_\{b\}p\_\{b\}=t\) and the explicit cosine form is given in Appendix[B\.1](https://arxiv.org/html/2607.00208#A2.SS1)\. It reaches32\.9132\.91on MATH500 against27\.0027\.00for uniform masking \(Table[2](https://arxiv.org/html/2607.00208#A1.T2)\)\.

## 4Experiments

### 4\.1Setup

#### Data, models, and training\.

Our base model is the block\-wise dLLM SDAR\-4B\-Chat\(Cheng et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib6)\)with block size 16 and block size 4, SDAR\-1\.7B\-Chat is at block size 4\. FollowingWang et al\. \([2025b](https://arxiv.org/html/2607.00208#bib.bib28)\)we train on the MATH set\(Hendrycks et al\.,[2021](https://arxiv.org/html/2607.00208#bib.bib12)\)\(level 3–5\) and PrimeIntellect\-verified coding problems\(Jaghouar et al\.,[2024](https://arxiv.org/html/2607.00208#bib.bib15)\), keeping a separate model per task\. On SDAR\-4B\-Chat this runs for160160steps at block size 16 and100100steps at block size 4, and the SDAR\-1\.7B\-Chat model at block size 4 trains to convergence\. The code model continues from the 4B math model at block size 4 and also trains to convergence\. Each RL step samples128128prompts withG=8G\{=\}8responses each at temperature1\.01\.0\. We useQ=3Q\{=\}3Gauss–Legendre nodes per step, AdamW at constant learning rate1×10−61\{\\times\}10^\{\-6\}, clipϵ=0\.1\\epsilon\{=\}0\.1\(the range\[1−0\.1,1\+0\.1\]\[1\{\-\}0\.1,1\{\+\}0\.1\]\), and KL coefficientβ=0\.01\\beta\{=\}0\.01\(k3k\_\{3\}estimator\), on8×8\\timesA100 \(40 GB\) GPUs across two nodes of four \(full configuration in Appendix[B](https://arxiv.org/html/2607.00208#A2), Table[7](https://arxiv.org/html/2607.00208#A2.T7), and the evaluation details in Appendix[A\.5](https://arxiv.org/html/2607.00208#A1.SS5)\)\.

#### Evaluation\.

We evaluate on two math benchmarks, MATH500\(Hendrycks et al\.,[2021](https://arxiv.org/html/2607.00208#bib.bib12)\)and GSM8K\(Cobbe et al\.,[2021](https://arxiv.org/html/2607.00208#bib.bib7)\), and two code benchmarks, MBPP\(Austin et al\.,[2021b](https://arxiv.org/html/2607.00208#bib.bib3)\)and HumanEval\(Chen et al\.,[2021](https://arxiv.org/html/2607.00208#bib.bib5)\)\. Generation uses length256256, temperature1\.01\.0, and the two decoders, dynamic sampling andτ\\tau\-budget at both block size 16 and block size 4\. We compare against an autoregressive reference, Qwen2\.5\-7B\-Instruct\(Qwen et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib21)\); the full\-attention dLLMs LLaDA\-8B\-Instruct\(Nie et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib19)\)and Dream\-7B\-Instruct\(Ye et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib31)\); and two RL baselines, random\-masking and TraceRL\.

### 4\.2Main Results

Table 1:Main results \(accuracy, %\)\. For each benchmark the paired sub\-columns dynamic andτ\\tauare two test\-time decoders applied to every row: dynamic sampling and ourτ\\tau\-budget decoder\. Ours = SLIM\-RL, which collects its training rollouts with theτ\\tau\-budget decoder; the dynamic andτ\\taucolumns are test\-time choices, independent of the rollout decoder\. Upper group: block size 16 \(math RL\); lower groups: block size 4 \(math RL, then code RL continued from math\)\.#### SLIM\-RL substantially outperforms the RL baselines on math\.

On the SDAR\-4B\-Chat base at block size 16, SLIM\-RL reaches31\.3631\.36on MATH500 and71\.0471\.04on GSM8K under dynamic sampling, and32\.9132\.91and75\.0375\.03underτ\\tau\-budget, well above TraceRL at25\.0425\.04and59\.9959\.99and random\-masking RL, which barely moves off the base at12\.8212\.82and47\.7147\.71\. At block size 4 it stays ahead of TraceRL on both math sets under dynamic sampling, and the 4B model surpasses the larger LLaDA\-8B and Dream\-7B diffusion models, exceeding LLaDA\-8B by10\.76%10\.76\\%on MATH500 while remaining below the autoregressive Qwen2\.5\-7B \(Table[1](https://arxiv.org/html/2607.00208#S4.T1)\)\. The advantage also carries to a smaller model, where on SDAR\-1\.7B at block size 4, SLIM\-RL again outperforms TraceRL on MATH500 \(36\.7136\.71vs\.35\.2935\.29under dynamic sampling\) and matches it on GSM8K, with cross\-scale data\-efficiency reported in Section[4\.5](https://arxiv.org/html/2607.00208#S4.SS5)\.

#### SLIM\-RL also outperforms TraceRL on code\.

Under code RL trained from the block size 4 math model, SLIM\-RL improves over TraceRL by4\.20%4\.20\\%on MBPP and3\.65%3\.65\\%on HumanEval under dynamic sampling, while preserving math accuracy\. Block size 4 parallelism \(tokens\-per\-forward, TPF\) and the decode\-time block\-size change are reported in Appendix[A\.3](https://arxiv.org/html/2607.00208#A1.SS3)\.

### 4\.3Ablation Study

Each component contributes as a leave\-one\-out drop from the full recipe on SDAR\-4B\-Chat at block size 16, measured on MATH500 under theτ\\tau\-budget decoder where the full recipe peaks at32\.9132\.91\(Table[2](https://arxiv.org/html/2607.00208#A1.T2)\)\. The largest drop is the quadrature, where using two Gauss–Legendre quadrature nodes instead of three lowers it to26\.5626\.56\. Generating the rollouts with dynamic sampling instead ofτ\\tau\-budget lowers accuracy to27\.0927\.09\. The sequence\-level importance ratio attains the full\-recipe32\.9132\.91, whereas a token\-level ratio degrades it to28\.0928\.09\. Replacing the monotonically decreasing per\-block schedule with uniform random masking lowers accuracy to27\.0027\.00\. Theτ\\tau\-budget rollout’s benefit does not transfer to TraceRL\. Trained onτ\\tau\-budget rollouts, TraceRL reaches only22\.9122\.91on MATH500, below the26\.0926\.09of its usual dynamic sampling rollouts, so the gain is specific to trace\-free random masking\.

### 4\.4Scaling Block Size

SLIM\-RL’s advantage over TraceRL grows with the block size\. With both methods trained natively at each block size \(Table[3](https://arxiv.org/html/2607.00208#A1.T3)\), the two are on par at block size 4, where TraceRL keeps the exact \(s=1s\{=\}1\) trajectory at low slicing cost and SLIM\-RL matches it on math to within0\.33%0\.33\\%on MATH500 and0\.23%0\.23\\%on GSM8K under matched decoding\. At block size 16 the math margin widens to6\.32%6\.32\\%and11\.05%11\.05\\%\. The same widening shows on code benchmarks\. TraceRL loses code ability at block size 16, its MBPP falling to35\.9635\.96and HumanEval to44\.6544\.65from the base’s38\.8038\.80and54\.4854\.48, while SLIM\-RL holds near the base at38\.8438\.84and53\.3953\.39\(Table[1](https://arxiv.org/html/2607.00208#S4.T1)\)\. Random\-masking RL improves far less than TraceRL and SLIM\-RL on math at every block size, while on code it stays near the base\.

### 4\.5Training Efficiency: Trace\-Free Optimization Avoids Trajectory Slicing

#### Trace\-free optimization attains TraceRL’s accuracy at a constant per\-rollout cost\.

On SDAR\-4B the baseline produces3\.553\.55samples per response at block size 16 \(s=4s\{=\}4\) and4\.004\.00at block size 4 \(s=1s\{=\}1\), both higher than our constantQ=3Q\{=\}3; the full trajectory \(s=1s\{=\}1\) at block size 16 instead costs12\.9212\.92, with18\.5%18\.5\\%of responses reaching the full1616samples\.111Measured over two passes of MATH500 \(10001000responses\) under the exact TraceRL slicing\.At block size 16, plain random\-masking RL is near the base model while SLIM\-RL overtakes TraceRL, reaching its best MATH500 accuracy on only0\.46×0\.46\\timesthe training data \(Table[6](https://arxiv.org/html/2607.00208#A1.T6), Figure[2](https://arxiv.org/html/2607.00208#A1.F2)\)\. At block size 4 and1\.71\.7B, it reaches TraceRL’s best on0\.81×0\.81\\timesand0\.76×0\.76\\timesthe data, and uses less total data at every scale,0\.85×0\.85\\times,0\.70×0\.70\\times, and0\.74×0\.74\\times\. At block size 4 the baseline keeps the full trajectory and both methods are near their best accuracy, where SLIM\-RL still stays slightly ahead\.

#### SLIM\-RL also trains to a more parallel model\.

Beyond accuracy, SLIM\-RL commits more tokens\-per\-forward than TraceRL at matched decoding\. At block size 4 it reaches TPF1\.941\.94vs\.1\.721\.72on MATH500 under dynamic sampling and leads on every benchmark \(Table[5](https://arxiv.org/html/2607.00208#A1.T5)\), and at block size 16 it sustains the higher TPF throughout training \(Figure[4](https://arxiv.org/html/2607.00208#A1.F4)\)\. Because the decoder is fixed, this TPF gain comes from the trained model, not from theτ\\tau\-budget rule, which atm=1m\{=\}1slightly lowers TPF in exchange for accuracy\. The other panels of Figure[4](https://arxiv.org/html/2607.00208#A1.F4)report the training dynamics, where generation length shortens for all methods, and the mask ratio for Random\-masking and ours stays near0\.50\.5while TraceRL’s stays near0\.250\.25\.

#### Theτ\\tau\-budget decoder transfers across architectures\.

Because it reads only per\-position confidences, theτ\\tau\-budget decoder transfers training\-free across LLaDA, Dream, and SDAR, a drop\-in replacement for dynamic sampling \(Table[4](https://arxiv.org/html/2607.00208#A1.T4)\)\. On the same model its more conservative commits trade a little throughput for accuracy, leaving tokens\-per\-forward just below dynamic sampling\. Dream\-7B is the exception, whereτ\\tau\-budget instead raises tokens\-per\-forward above dynamic sampling on every benchmark except GSM8K, so it is faster while its code accuracy also rises\.

## 5Related Work

#### Reinforcement learning for dLLMs\.

Existing RL for dLLMs either reconstructs the decoding trajectory or trains on randomly masked corruptions of the completed response\. On the trajectory side, TraceRL\(Wang et al\.,[2025b](https://arxiv.org/html/2607.00208#bib.bib28)\)optimizes the objective on the exact decoding trace, DCoLT\(Huang et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib14)\)rewards the full denoising trajectory through outcome\-based RL, and d2\(Wang et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib26)\)estimates the likelihood of the decoding trajectory\. The masking view instead discards the decoding order, with d1\(Zhao et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib33)\)factorizing the policy through a mean\-field decomposition, DiffuCoder’s coupled\-GRPO\(Gong et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib9)\)pairing complementary masking realizations, and MMaDA’s UniGRPO\(Yang et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib30)\)adapting GRPO to masked diffusion\. Closest to our objective,Rojas et al\. \([2025](https://arxiv.org/html/2607.00208#bib.bib22)\)replace the random masking level with Gaussian quadrature on the diffusion ELBO, and ESPO\(Ou et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib20)\)reduces the ratio to a single sequence\-level weight per response\. SLIM\-RL stays on the random\-masking side and controls its gradient variance by recombining existing components, a sequence\-level length\-normalized ratio\(Zheng et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib34)\), deterministic quadrature over the masking level\(Rojas et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib22)\), and an unnormalized advantage\(Liu et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib18)\)\. Both act on the training objective alone, whereas SLIM\-RL pairs it with theτ\\tau\-budget rollout decoder and the mean\-preserving per\-block mask schedule, controlling rollout commit risk together with optimization variance\.

#### Diffusion language models\.

Diffusion language models\(Austin et al\.,[2021a](https://arxiv.org/html/2607.00208#bib.bib2); Gong et al\.,[2022](https://arxiv.org/html/2607.00208#bib.bib8); Li et al\.,[2022](https://arxiv.org/html/2607.00208#bib.bib16)\)differ in architecture and in how they commit tokens at inference\. Full\-attention dLLMs \(LLaDA\(Nie et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib19); Bie et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib4)\), Dream\(Ye et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib31)\), MMaDA\(Yang et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib30)\)\) scale iterative denoising to the 7–8B range but cannot natively reuse KV caches across denoising steps\. Block\-wise hybrids \(Block Diffusion\(Arriola et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib1)\), SDAR\(Cheng et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib6)\)\) generate autoregressively across blocks while denoising the active block in parallel, restoring KV\-cache reuse and serving as the substrate for recent dLLM RL\(Wang et al\.,[2025b](https://arxiv.org/html/2607.00208#bib.bib28); Hu et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib13); Zhu et al\.,[2025b](https://arxiv.org/html/2607.00208#bib.bib36)\)\. A separate line accelerates inference by adaptively committing tokens at each denoising step\. Fast\-dLLM\(Wu et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib29)\)and Dimple\(Yu et al\.,[2025](https://arxiv.org/html/2607.00208#bib.bib32)\)commit every position whose confidence exceeds a threshold \(dynamic sampling\), D2F\(Wang et al\.,[2025a](https://arxiv.org/html/2607.00208#bib.bib27)\)adds discrete diffusion forcing for faster\-than\-autoregressive inference, and S2D2\(Han et al\.,[2026](https://arxiv.org/html/2607.00208#bib.bib10)\)adds training\-free self\-speculation\. Dynamic sampling bounds only per\-position confidence, so the wrong\-commit risk of a step grows with the number of positions committed\. Theτ\\tau\-budget decoder \(Section[3\.1](https://arxiv.org/html/2607.00208#S3.SS1)\) instead bounds the cumulative step\-level risk directly in a single pass\.

## 6Conclusion

We asked whether reconstructing the exact decoding trajectory is necessary to match TraceRL at equal training cost in diffusion language models, and found that it is not\. SLIM\-RL is trace\-free, pairing aτ\\tau\-budget rollout decoder that caps a confidence\-based proxy for step\-level commit risk with a variance\-reduced random\-masking objective built from sequence\-level ratios, deterministic quadrature, and a decreasing per\-block mask schedule\. On block\-causal SDAR\-4B, this recipe outperforms TraceRL on both math and code, reaching its best accuracy on0\.46×0\.46\\timesthe training data\. Whether exact trajectory reconstruction remains worthwhile at larger blocks or longer responses is left to future work\.

## References

- Arriola et al\. \(2025\)Marianne Arriola, Aaron Gokaslan, Justin Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sahoo, and Volodymyr Kuleshov\.Block diffusion: Interpolating between autoregressive and diffusion language models\.In*International Conference on Learning Representations*, volume 2025, pp\. 50726–50753, 2025\.
- Austin et al\. \(2021a\)Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg\.Structured denoising diffusion models in discrete state\-spaces\.*Advances in neural information processing systems*, 34:17981–17993, 2021a\.
- Austin et al\. \(2021b\)Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al\.Program synthesis with large language models\.*arXiv preprint arXiv:2108\.07732*, 2021b\.
- Bie et al\. \(2025\)Tiwei Bie, Maosong Cao, Kun Chen, Lun Du, Mingliang Gong, Zhuochen Gong, Yanmei Gu, Jiaqi Hu, Zenan Huang, Zhenzhong Lan, Chengxi Li, Chongxuan Li, Jianguo Li, Zehuan Li, Huabin Liu, Lin Liu, Guoshan Lu, Xiaocheng Lu, Yuxin Ma, Jianfeng Tan, Lanning Wei, Ji\-Rong Wen, Yipeng Xing, Xiaolu Zhang, Junbo Zhao, Da Zheng, Jun Zhou, Junlin Zhou, Zhanchao Zhou, Liwang Zhu, and Yihong Zhuang\.Llada2\.0: Scaling up diffusion language models to 100b, 2025\.URL[https://arxiv\.org/abs/2512\.15745](https://arxiv.org/abs/2512.15745)\.
- Chen et al\. \(2021\)Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al\.Evaluating large language models trained on code\.*arXiv preprint arXiv:2107\.03374*, 2021\.
- Cheng et al\. \(2026\)Shuang Cheng, Yihan Bian, Dawei Liu, Yuhua Jiang, Yihao Liu, Linfeng Zhang, Qian Yao, Zhongbo Tian, Wenhai Wang, Qipeng Guo, et al\.Sdar: A synergistic diffusion\-autoregression paradigm for scalable sequence generation\.In*Findings of the Association for Computational Linguistics: ACL 2026*, pp\. 22058–22075, 2026\.
- Cobbe et al\. \(2021\)Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al\.Training verifiers to solve math word problems\.*arXiv preprint arXiv:2110\.14168*, 2021\.
- Gong et al\. \(2022\)Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and LingPeng Kong\.Diffuseq: Sequence to sequence text generation with diffusion models\.*arXiv preprint arXiv:2210\.08933*, 2022\.
- Gong et al\. \(2025\)Shansan Gong, Ruixiang Zhang, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong, and Yizhe Zhang\.Diffucoder: Understanding and improving masked diffusion models for code generation\.*arXiv preprint arXiv:2506\.20639*, 2025\.
- Han et al\. \(2026\)Ligong Han, Hao Wang, Han Gao, Kai Xu, and Akash Srivastava\.S2d2: Fast decoding for diffusion llms via training\-free self\-speculation\.*arXiv preprint arXiv:2603\.25702*, 2026\.
- He et al\. \(2025\)Haoyu He, Katrin Renz, Yong Cao, and Andreas Geiger\.Mdpo: Overcoming the training\-inference divide of masked diffusion language models\.*arXiv preprint arXiv:2508\.13148*, 2025\.
- Hendrycks et al\. \(2021\)Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt\.Measuring mathematical problem solving with the math dataset\.*arXiv preprint arXiv:2103\.03874*, 2021\.
- Hu et al\. \(2026\)Yanzhe Hu, Yijie Jin, Pengfei Liu, Kai Yu, and Zhijie Deng\.Lightningrl: Breaking the accuracy\-parallelism trade\-off of block\-wise dllms via reinforcement learning\.*arXiv preprint arXiv:2603\.13319*, 2026\.
- Huang et al\. \(2026\)Zemin Huang, Zhiyang Chen, Zijun Wang, Tiancheng Li, and Guo\-Jun Qi\.Reinforcing the diffusion chain of lateral thought with diffusion language models\.*Advances in Neural Information Processing Systems*, 38:152677–152710, 2026\.
- Jaghouar et al\. \(2024\)Sami Jaghouar, Jack Min Ong, Manveer Basra, Fares Obeid, Jannik Straube, Michael Keiblinger, Elie Bakouch, Lucas Atkins, Maziyar Panahi, Charles Goddard, Max Ryabinin, and Johannes Hagemann\.Intellect\-1 technical report, 2024\.URL[https://arxiv\.org/abs/2412\.01152](https://arxiv.org/abs/2412.01152)\.
- Li et al\. \(2022\)Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto\.Diffusion\-lm improves controllable text generation\.*Advances in neural information processing systems*, 35:4328–4343, 2022\.
- Liu et al\. \(2026\)Jiawei Liu, Xiting Wang, Yuanyuan Zhong, Defu Lian, and Yu Yang\.Efficient and stable reinforcement learning for diffusion language models\.*arXiv preprint arXiv:2602\.08905*, 2026\.
- Liu et al\. \(2025\)Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin\.Understanding r1\-zero\-like training: A critical perspective\.*arXiv preprint arXiv:2503\.20783*, 2025\.
- Nie et al\. \(2026\)Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji\-Rong Wen, and Chongxuan Li\.Large language diffusion models\.*Advances in Neural Information Processing Systems*, 38:50608–50646, 2026\.
- Ou et al\. \(2025\)Jingyang Ou, Jiaqi Han, Minkai Xu, Shaoxuan Xu, Jianwen Xie, Stefano Ermon, Yi Wu, and Chongxuan Li\.Principled rl for diffusion llms emerges from a sequence\-level perspective\.*arXiv preprint arXiv:2512\.03759*, 2025\.
- Qwen et al\. \(2025\)Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu\.Qwen2\.5 technical report, 2025\.URL[https://arxiv\.org/abs/2412\.15115](https://arxiv.org/abs/2412.15115)\.
- Rojas et al\. \(2025\)Kevin Rojas, Jiahe Lin, Kashif Rasul, Anderson Schneider, Yuriy Nevmyvaka, Molei Tao, and Wei Deng\.Improving reasoning for diffusion language models via group diffusion policy optimization\.*arXiv preprint arXiv:2510\.08554*, 2025\.
- Sahoo et al\. \(2024\)Subham S Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, and Volodymyr Kuleshov\.Simple and effective masked diffusion language models\.*Advances in Neural Information Processing Systems*, 37:130136–130184, 2024\.
- Shao et al\. \(2024\)Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al\.Deepseekmath: Pushing the limits of mathematical reasoning in open language models\.*arXiv preprint arXiv:2402\.03300*, 2024\.
- Tang et al\. \(2026\)Xiaohang Tang, Rares Dolga, Sangwoong Yoon, and Ilija Bogunovic\.wd1: Weighted policy optimization for reasoning in diffusion language models, 2026\.URL[https://arxiv\.org/abs/2507\.08838](https://arxiv.org/abs/2507.08838)\.
- Wang et al\. \(2026\)Guanghan Wang, Gilad Turok, Yair Schiff, Marianne Arriola, and Volodymyr Kuleshov\.d2: Improving reasoning in diffusion language models via trajectory likelihood estimation, 2026\.URL[https://arxiv\.org/abs/2509\.21474](https://arxiv.org/abs/2509.21474)\.
- Wang et al\. \(2025a\)Xu Wang, Chenkai Xu, Yijie Jin, Jiachun Jin, Hao Zhang, and Zhijie Deng\.Diffusion llms can do faster\-than\-ar inference via discrete diffusion forcing\.*arXiv preprint arXiv:2508\.09192*, 2025a\.
- Wang et al\. \(2025b\)Yinjie Wang, Ling Yang, Bowen Li, Ye Tian, Ke Shen, and Mengdi Wang\.Revolutionizing reinforcement learning framework for diffusion large language models\.*arXiv preprint arXiv:2509\.06949*, 2025b\.
- Wu et al\. \(2025\)Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, and Enze Xie\.Fast\-dllm: Training\-free acceleration of diffusion llm by enabling kv cache and parallel decoding\.*arXiv preprint arXiv:2505\.22618*, 2025\.
- Yang et al\. \(2026\)Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, and Mengdi Wang\.Mmada: Multimodal large diffusion language models\.*Advances in Neural Information Processing Systems*, 38:138867–138907, 2026\.
- Ye et al\. \(2025\)Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong\.Dream 7b: Diffusion large language models\.*arXiv preprint arXiv:2508\.15487*, 2025\.
- Yu et al\. \(2025\)Runpeng Yu, Xinyin Ma, and Xinchao Wang\.Dimple: Discrete diffusion multimodal large language model with parallel decoding\.*arXiv preprint arXiv:2505\.16990*, 2025\.
- Zhao et al\. \(2026\)Siyan Zhao, Devaansh Gupta, Qinqing Zheng, and Aditya Grover\.d1: Scaling reasoning in diffusion large language models via reinforcement learning\.*Advances in Neural Information Processing Systems*, 38:56729–56762, 2026\.
- Zheng et al\. \(2025\)Chujie Zheng, Shixuan Liu, Mingze Li, Xiong\-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, et al\.Group sequence policy optimization\.*arXiv preprint arXiv:2507\.18071*, 2025\.
- Zhu et al\. \(2025a\)Fengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Hu, Jun Zhou, Jianfei Chen, Yankai Lin, Ji\-Rong Wen, and Chongxuan Li\.Llada 1\.5: Variance\-reduced preference optimization for large language diffusion models, 2025a\.URL[https://arxiv\.org/abs/2505\.19223](https://arxiv.org/abs/2505.19223)\.
- Zhu et al\. \(2025b\)Ying Zhu, Jiaxin Wan, Xiaoran Liu, Siyang He, Qiqi Wang, Xu Guo, Tianyi Liang, Zengfeng Huang, Ziwei He, and Xipeng Qiu\.Dirl: An efficient post\-training framework for diffusion language models\.*arXiv preprint arXiv:2512\.22234*, 2025b\.

## Appendix AAdditional Experimental Results

This appendix collects the parallelism/TPF, cross\-architecture, block size 4, code\-RL, cross\-scale, and training\-dynamics results referenced from the main text\. All numbers are at response length256256\.

### A\.1Ablation and Block\-Size Tables

These tables support the analysis in Sections[4\.3](https://arxiv.org/html/2607.00208#S4.SS3)–[4\.4](https://arxiv.org/html/2607.00208#S4.SS4): Table[2](https://arxiv.org/html/2607.00208#A1.T2)gives the leave\-one\-out component ablations and Table[3](https://arxiv.org/html/2607.00208#A1.T3)reports the block\-size dependence of the advantage over TraceRL\.

Table 2:Leave\-one\-out ablations of SLIM\-RL on MATH500 and ablation to TraceRL\.Table 3:Block\-size dependence of SLIM\-RL’s advantage over TraceRL \(dynamic sampling\); block size 4 uses models trained natively at block size 4\.
### A\.2Cross\-Architecture Decoder Transfer and Parallelism

Becauseτ\\tau\-budget depends only on per\-position confidences, it applies unchanged to full\-attention \(LLaDA, Dream\) and block\-causal \(SDAR\) diffusion models as a training\-free, drop\-in replacement for dynamic sampling \(Table[4](https://arxiv.org/html/2607.00208#A1.T4)\)\. Full SLIM\-RL is excluded because it already trains withτ\\tau\-budget, so we include its dynamic\-sampling\-trained ablation in its place\. On parallelism, at the conservative budgetm=1m\{=\}1theτ\\tau\-budget tokens\-per\-forward is uniformly slightly below dynamic sampling across LLaDA and every SDAR model in Table[4](https://arxiv.org/html/2607.00208#A1.T4), the small throughput cost it pays for its accuracy gains, so we report it as an accuracy decoder rather than a speed one\. Dream\-7B at block size 4 is the one architecture whereτ\\tau\-budget speeds up TPF on MATH500, MBPP, and HumanEval, with only GSM8K nearly the same at2\.612\.61versus2\.602\.60; on the two code sets accuracy rises in step, MBPP from36\.9336\.93to37\.3337\.33at TPF4\.454\.45to4\.594\.59and HumanEval from35\.5735\.57to36\.1836\.18at TPF5\.465\.46to5\.725\.72\. For Dream the decoder is both faster and more accurate on code, while on MATH500 accuracy is nearly unchanged and on GSM8K it drops slightly\.

Table 4:Training\-freeτ\\tau\-budget decoding across architectures\.
### A\.3Block Size 4 and Code RL

Table[5](https://arxiv.org/html/2607.00208#A1.T5)reports the full block size 4 results: the two methods reach comparable accuracy, with SLIM\-RL slightly ahead, and replacing dynamic sampling withτ\\tau\-budget decoding leaves accuracy nearly unchanged\. The code\-RL results, with code RL continued from the block size 4 math\-RL model, are reported in the main\-text Table[1](https://arxiv.org/html/2607.00208#S4.T1)\(lower group\)\.

Table 5:Native block size 4 evaluation of the SDAR\-4B family \(both methods trained and decoded at block size 4\)\.
### A\.4Training Dynamics and Cross\-Scale

At1\.71\.7B scale \(Figure[3](https://arxiv.org/html/2607.00208#A1.F3), Table[6](https://arxiv.org/html/2607.00208#A1.T6)\), SLIM\-RL stays ahead of TraceRL on MATH500 \(36\.7136\.71vs\.35\.2935\.29\) and reaches TraceRL’s best accuracy on0\.76×0\.76\\timesthe training data, a data\-efficiency gain that holds at a second model scale\. Figure[4](https://arxiv.org/html/2607.00208#A1.F4)shows training dynamics: SLIM\-RL maintains a stable tokens\-per\-forward while TraceRL’s declines, so SLIM\-RL ends higher, and generation length shortens for all methods\.

![Refer to caption](https://arxiv.org/html/2607.00208v1/x2.png)Figure 2:MATH500 accuracy versus cumulative training samples at block size 16\. Random masking \(gray\) is near the base model; SLIM\-RL \(blue\) overtakes TraceRL \(red\), reaching TraceRL’s best accuracy on0\.46×0\.46\\timesthe training samples\.Table 6:Training cost of SLIM\-RL relative to the TraceRL baseline \(TraceRL=1\.00×=1\.00\\times;<1<1is cheaper\)\. SLIM\-RL needs less data to reach TraceRL’s best MATH500 accuracy, and less total\-run data, at every scale\.![Refer to caption](https://arxiv.org/html/2607.00208v1/x3.png)Figure 3:Cross\-scale training dynamics at1\.71\.7B and block size 4\. SLIM\-RL \(blue\) overtakes TraceRL \(red\) on training accuracy over cumulative data, reaching TraceRL’s best MATH500 accuracy on0\.76×0\.76\\timesthe training samples \(Table[6](https://arxiv.org/html/2607.00208#A1.T6)\)\.![Refer to caption](https://arxiv.org/html/2607.00208v1/x4.png)Figure 4:Training dynamics at block size 16\.*Left\.*SLIM\-RL maintains a stable tokens\-per\-forward \(TPF\) while TraceRL’s declines, so SLIM\-RL ends higher\.*Middle\.*Generation length shortens over training for all methods\.*Right\.*Mask ratio: SLIM\-RL and random masking hold near0\.50\.5while TraceRL stays near0\.250\.25\.![Refer to caption](https://arxiv.org/html/2607.00208v1/x5.png)Figure 5:Expected wrong commitments per step,∑i\(1−pi\)\\sum\_\{i\}\(1\-p\_\{i\}\)over committed positions\. Theτ\\tau\-budget rollouts \(SLIM\-RL, blue\) commit fewer than TraceRL’s dynamic sampling rollouts,0\.170\.17vs\.0\.220\.22at block size 16 and0\.150\.15vs\.0\.210\.21at block size 4\.
### A\.5Evaluation Details

We report the average accuracy overnnsampled responses per task\. For the SDAR family we setn=9n\{=\}9on MATH500,n=3n\{=\}3on GSM8K, andn=9n\{=\}9on both MBPP and HumanEval\. A coding response counts as correct only when it passes every functional test\. Each diffusion model is evaluated under both decoders, dynamic sampling and theτ\\tau\-budget decoder of Section[3\.1](https://arxiv.org/html/2607.00208#S3.SS1)\.

For the SDAR\-4B\-Chat and SDAR\-1\.7B\-Chat models, we keep the pretrained block\-diffusion decoding, a response length of256256, temperature1\.01\.0, top\-p=1\.0p\{=\}1\.0, and top\-k=0k\{=\}0\. Dynamic sampling unmasks every position above the confidence thresholdτ=0\.9\\tau\{=\}0\.9, and theτ\\tau\-budget decoder reuses thatτ\\tauwith budget multiplierm=1m\{=\}1\. The 4B models are evaluated at block size1616and block size44, and the 1\.7B model at block size44\.

For the full\-attention baselines, we use temperature0\.10\.1and samplen=3n\{=\}3responses per task across all four benchmarks\. LLaDA\-8B\-Instruct decodes at block size3232and Dream\-7B\-Instruct at block size44, both over a length\-256256response\. Both baselines run dynamic sampling and theτ\\tau\-budget decoder at thresholdτ=0\.95\\tau\{=\}0\.95withm=1m\{=\}1\.

Qwen2\.5\-7B\-Instruct is the autoregressive reference\. We samplen=3n\{=\}3responses at temperature0\.10\.1, and raise the generation length to512512\.

## Appendix BAlgorithm Pipeline

Algorithm[1](https://arxiv.org/html/2607.00208#alg1)details the SLIM\-RL training loop\. Each outer step samplesGGrollouts viaτ\\tau\-budget dynamic unmasking \(Algorithm[2](https://arxiv.org/html/2607.00208#alg2)\), scores them with a verifiable rewardr​\(⋅\)r\(\\cdot\), forms the unnormalized advantageAjA\_\{j\}\(Eq\.[8](https://arxiv.org/html/2607.00208#S3.E8)\), evaluates the sequence\-level ratioρj\(tq\)​\(θ\)\\rho\_\{j\}^\{\(t\_\{q\}\)\}\(\\theta\)\(Eq\.[7](https://arxiv.org/html/2607.00208#S3.E7)\) atQQGauss–Legendre quadrature nodes per response, and updatesθ\\thetaon the clipped objective \(Eq\.[9](https://arxiv.org/html/2607.00208#S3.E9)\)\.

Algorithm 1SLIM\-RL Training1:Input:

2:1\) Prompt set

𝒟\\mathcal\{D\}; reward

r​\(x,y\)r\(x,y\); policy

πθ\\pi\_\{\\theta\}; old policy

πθold\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\.

3:2\) Outer steps

TT; rollouts

GG; update epochs

EE; clip range

ϵ\\epsilon; KL weight

β\\beta; learning rate

η\\eta\.

4:3\) Block size

KK; confidence threshold

τ\\tau; risk\-budget schedule

mm; quadrature nodes

QQ\.

5:Initialize

θ\\thetaand Gauss–Legendre nodes/weights

\{\(tq,ωq\)\}q=1Q\\\{\(t\_\{q\},\\omega\_\{q\}\)\\\}\_\{q=1\}^\{Q\}on

\(0,1\)\(0,1\)\.

6:for

n=1n=1to

TTdo

7:

πθold←πθ\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\\leftarrow\\pi\_\{\\theta\};

𝒟grp←∅\\mathcal\{D\}\_\{\\rm grp\}\\leftarrow\\emptyset
8:Rollout collection with risk\-budgeted decoding

9:forprompt minibatch

𝒳∼𝒟\\mathcal\{X\}\\sim\\mathcal\{D\}do

10:for all

x∈𝒳x\\in\\mathcal\{X\}do

11:Sample

\{yj\}j=1G\\\{y\_\{j\}\\\}\_\{j=1\}^\{G\}by

TauBudgetDecode​\(πθold,x,K,m,τ\)\\textsc\{TauBudgetDecode\}\(\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\},x,K,m,\\tau\)\.

12:Evaluate

rj←r​\(x,yj\)r\_\{j\}\\leftarrow r\(x,y\_\{j\}\)for

j=1,…,Gj=1,\\ldots,G\.

13:Compute

Aj←rj−1G​∑i=1GriA\_\{j\}\\leftarrow r\_\{j\}\-\\frac\{1\}\{G\}\\sum\_\{i=1\}^\{G\}r\_\{i\}\.

14:

𝒟grp←𝒟grp∪\{\(x,\{yj\}j=1G,\{Aj\}j=1G\)\}\\mathcal\{D\}\_\{\\rm grp\}\\leftarrow\\mathcal\{D\}\_\{\\rm grp\}\\cup\\\{\(x,\\\{y\_\{j\}\\\}\_\{j=1\}^\{G\},\\\{A\_\{j\}\\\}\_\{j=1\}^\{G\}\)\\\}\.

15:endfor

16:endfor

17:Policy optimization with sequence\-level ratios

18:for

e=1e=1to

EEdo

19:Sample minibatch

𝒢⊂𝒟grp\\mathcal\{G\}\\subset\\mathcal\{D\}\_\{\\rm grp\}\.

20:Build masked corruptions

y~j​\(tq\)\\widetilde\{y\}\_\{j\}\(t\_\{q\}\)for all

\(x,yj\)∈𝒢\(x,y\_\{j\}\)\\in\\mathcal\{G\}and nodes

qq, with per\-block rates from the decreasing schedule \(Section[3\.4](https://arxiv.org/html/2607.00208#S3.SS4)\)\.

21:Compute sequence log\-scores

ℓθj,q\\ell\_\{\\theta\}^\{j,q\}and

ℓθoldj,q\\ell\_\{\\theta\_\{\\mathrm\{old\}\}\}^\{j,q\}by Eq\.[6](https://arxiv.org/html/2607.00208#S3.E6)\.

22:Compute sequence ratios

ρj\(tq\)​\(θ\)\\rho\_\{j\}^\{\(t\_\{q\}\)\}\(\\theta\)by Eq\.[7](https://arxiv.org/html/2607.00208#S3.E7)\.

23:Update

θ←θ\+η​∇θ𝒥SLIM​\(θ\)\\theta\\leftarrow\\theta\+\\eta\\nabla\_\{\\theta\}\\mathcal\{J\}\_\{\\mathrm\{SLIM\}\}\(\\theta\)using Eq\.[9](https://arxiv.org/html/2607.00208#S3.E9)\.

24:endfor

25:endfor

26:Trained policy

πθ\\pi\_\{\\theta\}\.

Algorithm 2TauBudgetDecode\(τ\\tau\-Budget Dynamic Unmasking\)1:Input:

2:1\) Policy

πθ\\pi\_\{\\theta\}; prompt

xx\.

3:2\) Block size

KK; risk\-budget schedule

mm; confidence threshold

τ\\tau\.

4:Initialize:

y←x∥\[MASK\]Ly\\leftarrow x\\mathbin\{\\\|\}\[\\mathrm\{MASK\}\]^\{L\}; block index

b←1b\\leftarrow 1\.

5:whilesome block remains maskeddo

6:

s←1s\\leftarrow 1
7:whileblock

bbhas masked positionsdo

8:For masked

iiin block

bb:

pi←maxv∈𝒱⁡pθ​\(y0i=v∣y,x\)p\_\{i\}\\leftarrow\\max\_\{v\\in\\mathcal\{V\}\}p\_\{\\theta\}\(y\_\{0\}^\{i\}=v\\mid y,x\)\.

9:Candidate set

𝒜s←\{i:pi\>τ\}\\mathcal\{A\}\_\{s\}\\leftarrow\\\{i:p\_\{i\}\>\\tau\\\}\. // Eq\.[4](https://arxiv.org/html/2607.00208#S3.E4)

10:if

𝒜s=∅\\mathcal\{A\}\_\{s\}=\\emptysetthen

11:Commit single

arg⁡maxi⁡pi\\arg\\max\_\{i\}p\_\{i\}to break tie;continue\.

12:endif

13:Uncertainties

ui←1−piu\_\{i\}\\leftarrow 1\-p\_\{i\}for

i∈𝒜si\\in\\mathcal\{A\}\_\{s\}; sort

ui1≤ui2≤⋯≤ui\|𝒜s\|u\_\{i\_\{1\}\}\\leq u\_\{i\_\{2\}\}\\leq\\cdots\\leq u\_\{i\_\{\|\\mathcal\{A\}\_\{s\}\|\}\}\.

14:Step budget

ℬs←m​\(1−τ\)\\mathcal\{B\}\_\{s\}\\leftarrow m\(1\-\\tau\)\.

15:

ks←max⁡\{k:∑r=1kuir≤ℬs\}k\_\{s\}\\leftarrow\\max\\\{k:\\sum\_\{r=1\}^\{k\}u\_\{i\_\{r\}\}\\leq\\mathcal\{B\}\_\{s\}\\\}; committed set

𝒞s←\{i1,…,iks\}\\mathcal\{C\}\_\{s\}\\leftarrow\\\{i\_\{1\},\\ldots,i\_\{k\_\{s\}\}\\\}\. // Eq\.[5](https://arxiv.org/html/2607.00208#S3.E5)

16:For

i∈𝒞si\\in\\mathcal\{C\}\_\{s\}:

y^i←arg⁡maxv∈𝒱⁡pθ​\(y0i=v∣y,x\)\\hat\{y\}\_\{i\}\\leftarrow\\arg\\max\_\{v\\in\\mathcal\{V\}\}p\_\{\\theta\}\(y\_\{0\}^\{i\}=v\\mid y,x\); update

yy\.

17:

s←s\+1s\\leftarrow s\+1
18:endwhile

19:

b←b\+1b\\leftarrow b\+1// move to next block

20:endwhile

21:Output:decoded response

yy\.

Table 7:SLIM\-RL hyperparameters for the SDAR\-4B\-Chat math\-RL runs\.### B\.1Per\-Block Mask Schedule

The monotonically decreasing per\-block schedule \(Section[3\.4](https://arxiv.org/html/2607.00208#S3.SS4)\) is a cosine schedule centered at the masking leveltt:

pb=t\+δ2​cos⁡\(π​ξb\),ξb=b−1B−1,p\_\{b\}=t\+\\tfrac\{\\delta\}\{2\}\\cos\(\\pi\\xi\_\{b\}\),\\qquad\\xi\_\{b\}=\\frac\{b\-1\}\{B\-1\},\(10\)whereξb\\xi\_\{b\}is the normalized position of blockbbamong theBBblocks of a response\. The rate decreases fromt\+δ2t\+\\tfrac\{\\delta\}\{2\}at the first block tot−δ2t\-\\tfrac\{\\delta\}\{2\}at the last\. Sincecos⁡\(π​ξb\)\\cos\(\\pi\\xi\_\{b\}\)is antisymmetric aboutξb=12\\xi\_\{b\}=\\tfrac\{1\}\{2\}, the per\-block rates average tottexactly \(1B​∑bpb=t\\tfrac\{1\}\{B\}\\sum\_\{b\}p\_\{b\}=t\), so the schedule only reallocates a fixed expected mask count across blocks\. The spread is bounded byδ≤2​min⁡\(t,1−t\)\\delta\\leq 2\\min\(t,1\-t\)to keep everypb∈\[0,1\]p\_\{b\}\\in\[0,1\]; we setδ=0\.2\\delta=0\.2and apply the schedule at each Gauss–Legendre quadrature nodett\.