GRPO post-training makes LLMs accurate but verbose: on 20 MATH-500 problems, a distilled 1.5B model and a PhD volunteer both score 100% - but the model spends 1,658 tokens where the human needs 264. Existing fixes are content-agnostic: they favor short outputs or decay credit by position, without measuring what each token actually contributes.

IAPO (UVA + LinkedIn) assigns each token an advantage based on how much it reduces uncertainty about the final answer, given everything written so far. Tokens that move the answer forward get amplified; redundant ones get suppressed. A confidence-based exploration term keeps correct high-confidence tokens reinforced while pushing incorrect completions toward alternatives. All three signals drop into GRPO’s objective unchanged.

The estimator is the clever part: an early-exit prompt elicits answer distributions from intermediate reasoning states, with KV-cache reuse keeping the cost practical. Result: up to 36% shorter reasoning on Qwen2.5-7B-Instruct while improving accuracy on GSM8K, MATH, and MATH-500.

Made a short visual breakdown - one diagram per trick. Swipe through.

paper - https://arxiv.org/abs/2602.19049 code - https://github.com/YinhanHe123/IAPO… full summary pdf - https://gist.github.com/vukrosic/f59a3c48b1a97761f90427a2b9b0885c…

Every Sunday I run a hands-on live AI research with 1 on 1 help: https://skool.com/become-ai-researcher-2669/about…

Information-Aware Policy Optimization for Token-Efficient Reasoning

Source: https://arxiv.org/html/2602.19049 Yaochen ZhuMingjia ShiWendy ZhengLin SuXiaoqing WangQi GuoJundong Li

Abstract

Large language models increasingly rely on long chains of thought to improve accuracy, yet such gains come with substantial inference-time costs. We revisit token-efficient post-training and argue that existing sequence-level reward-shaping methods offer limited control over how reasoning effort is allocated across tokens. To bridge the gap, we proposeIAPO, an information-theoretic post-training framework that assigns token-wise advantages based on each token’s conditional mutual information (MI) with the final answer. This yields an explicit, principled mechanism for identifying informative reasoning steps and suppressing low-utility exploration. We provide a theoretical analysis showing that ourIAPOcan induce monotonic reductions in reasoning verbosity without harming correctness. Empirically,IAPOconsistently improves reasoning accuracy while reducing reasoning length by up to 47%, outperforming existing token-efficient RL methods across various reasoning datasets. Our results demonstrate that information-aware advantage shaping is a powerful and general direction for token-efficient post-training. The code is available athttps://github.com/YinhanHe123/IAPO.

1Introduction

Figure 1:Reasoning verbosity of RL post-trained LLMs. (a) Comparison of reasoning length between LLM (DeepSeekR1-Distilled-Qwen-1.5B(DeepSeek-AI,2025)) and a human volunteer on math problems(Lightmanet al.,2024). (b) Illustration of why the reasoning generated by the LLM are unnecessarily verbose.Reinforcement learning (RL)-based post-training has become one of the most effective tools for strengthening the reasoning capabilities of large language models (LLMs)(Kumaret al.,2025). Recently, group relative policy optimization (GRPO)(Shaoet al.,2024)and its variants have proven especially powerful: by contrasting multiple sampled solutions and explicitly rewarding reasoning chains that lead to better answers, these methods incentivize deeper, more structured multi-step reasoning, thus delivering substantial gains across diverse reasoning benchmarks.

However, GRPO also comes with an often-overlooked cost. Post-trained LLMs tend to producemorereasoning than necessary. While additional reasoning steps enable more computational power of the base LLM, the generated content could be redundant, circular, or uninformative. As shown in Fig.1, DeepSeekR1-Distilled-Qwen-1.5B(DeepSeek-AI,2025)and a Ph.D.-level volunteer both achieve perfect accuracy on 20 MATH-500 problems(Lightmanet al.,2024), yet the model generates substantially longer reasoning (1,658 vs. 264 tokens on average). Such verbosity inflates inference latency and computational cost, which scale quadratically with sequence length, while offering diminishing returns in correctness. This mismatch motivates the central question:Can we retain the reasoning strength achieved by RL post-training while reducing unnecessary reasoning tokens?

Several recent approaches have attempted to answer this question by shaping advantages in GRPO(Shaoet al.,2024)to favor brevity(Tanet al.,2025; Shrivastavaet al.,2025; Lee and Tong,2025; Yuet al.,2025). However, the semantics of the generated reasoning are overall overlooked. Generally, existing methods fall into two broad categories. Length-based methods(Shrivastavaet al.,2025; Yuet al.,2025)favor shorter outputs by assigning uniformly higher advantages to all tokens in concise outputs, regardless of whether they are actually informative. Position-based methods(Daiet al.,2025; Lee and Tong,2025; Yuet al.,2025)penalize later tokens via position-dependent advantage decay, even if some later tokens can be crucial for producing correct answers. Both paradigms share a fundamental limitation:they are content-agnostic, i.e., they determine a token’s advantage by completion length or token position, instead of the information it contributes to the correctness of the final answer. Consequently, current approaches struggle to distinguish essential reasoning from redundant tokens.

To overcome this limitation, we proposeInformation-AwarePolicyOptimization(IAPO), a novel post-training framework that brings information-theoretic awareness into intra-completion token-level advantages for policy optimization. Specifically,IAPOconsists of two modules. Theinformation-aware advantage shaping moduleassigns token-level advantages by quantifying each token’s contribution to answer correctness through its conditional mutual information (MI) with the final answer, conditioned on the preceding partial completion. This module provides a principled mechanism for identifying informative reasoning steps and reducing low-utility generation. Theefficient conditional MI estimation moduleefficiently estimates token-level conditional MI values utilizing an early-exit–based conditional MI estimator, together with KV-cache preloading and chunk-wise forwarding techniques, to significantly reduce the computational overhead of MI estimation, making our proposedIAPOcomputationally tractable at scale.

Empirically,IAPOdelivers substantial gains in token efficiency while preserving strong reasoning performance. Applied to Qwen2.5-7B-Instruct(Yanget al.,2024; Team,2024),IAPOachieves a47%reduction in reasoning length relative to the base model and a 43% improvement over the best existing token-efficient RL baselines on GSM8K(Cobbeet al.,2021), without sacrificing accuracy.

Our contributions are as follows. (1)Problem Formulation.We formalize token-efficient post-training as maximizing the ratio of expected task accuracy to completion length, capturing the objective of achieving high reasoning performance with minimal token consumption. (2)Theoretical Framework.We proposeIAPO, a token-level advantage-shaping framework that assigns advantages based on conditional MI between tokens and final answers. We provide a theoretical analysis demonstrating howIAPOreduces the expected completion length while maintaining model accuracy. (3)Efficient Module Design.We introduce an early-exit conditional MI estimator along with KV-cache preloading and chunk-wise forwarding techniques, making token-level MI computation tractable for modern LLMs at scale. (4)Extensive Empirical Validation.Across multiple reasoning datasets and model scales,IAPOconsistently achieves state-of-the-art token efficiency compared with existing token-efficient RL post-training baselines for LLMs.

Conflict of Interest Disclosure.

Lin Su, Xiaoqing Wang, and Qi Guo are employed by LinkedIn Inc.; the remaining authors are affiliated with the University of Virginia. This work does not evaluate or rely on any product, model, or service developed by LinkedIn, and the authors declare no financial conflicts of interest.

2Preliminaries and Problem Definition

Preliminaries.Letq∼Qq\sim Qdenote an input query (e.g., a math problem), and letoobe a completion sampled from an LLM policyπθ\pi_{\theta}. Post-training aims to optimize aπθ\pi_{\theta}which maximize the expected rewardsrr(a value quantifying the quality ofoo), i.e.,maxθ⁡𝔼q∼Q,o∼πθ​(o|q)​[r​(o)]\max_{\theta}{\mathbb{E}_{q\sim Q,o\sim\pi_{\theta}(o|q)}[r(o)]}.

For each queryqq, GRPO(Shaoet al.,2024)samples a group ofGGcompletions{oi}i=1G\{o_{i}\}_{i=1}^{G}from a frozen policyπθold\pi_{\theta_{\mathrm{old}}}. Each completionoio_{i}receives a rewardri∈ℝr_{i}\in\mathbb{R}from a reward model. GRPO computes advantages as group-wise normalized rewardsA~i=ri−mean​(𝐫)std​(𝐫),\tilde{A}_{i}=\frac{r_{i}-\mathrm{mean}(\mathbf{r})}{\mathrm{std}(\mathbf{r})},where𝐫={ri}i=1G\mathbf{r}=\{r_{i}\}_{i=1}^{G}. The normalized advantage is then uniformly assigned to all token positionstts inoio_{i}as token-wise advantagesA~i,t=A~i,∀t.\tilde{A}_{i,t}=\tilde{A}_{i},\forall t.The GRPO objective maximizes a clipped policy-gradient surrogate(Grondmanet al.,2012)with KL regularization:

JGRPO(θ)=𝔼q,{oi}[1G∑i=1G1|oi|∑t=1|oi|min(ρi,t(θ)A~i,t,\displaystyle J_{\mathrm{GRPO}}(\theta)=\mathbb{E}_{q,\{o_{i}\}}\Bigg[\frac{1}{G}\sum_{i=1}^{G}\frac{1}{|o_{i}|}\sum_{t=1}^{|o_{i}|}\min\Big(\rho_{i,t}(\theta)\,\tilde{A}_{i,t},clip(ρi,t(θ),1−ε,1+ε)A~i,t)−βDKL(πθ∥πref)],\displaystyle\mathrm{clip}(\rho_{i,t}(\theta),1-\varepsilon,1+\varepsilon)\,\tilde{A}_{i,t}\Big)-\beta\,D_{\mathrm{KL}}(\pi_{\theta}\,\|\,\pi_{\mathrm{ref}})\Bigg],whereρi,t​(θ)=πθ​(oi,t∣q,oi,<t)πθold​(oi,t∣q,oi,<t).\rho_{i,t}(\theta)=\frac{\pi_{\theta}(o_{i,t}\mid q,o_{i,<t})}{\pi_{\theta_{\mathrm{old}}}(o_{i,t}\mid q,o_{i,<t})}.While GRPO(Shaoet al.,2024)effectively improves reasoning accuracy, its uniform token-level advantage fails to distinguish informative tokens from redundant ones, often leading to verbose completions.

Problem 1(Token-Efficient Post-Training).

Given a query–answer distribution(Q,Y)(Q,Y), we aim to learn a policyπθ\pi_{\theta}that maximizes reasoning accuracy per generated token for samples(q,y)(q,y)drawn from(Q,Y)(Q,Y):

maxθ\displaystyle\max_{\theta}𝔼q∼Q,o∼πθ​[𝕀​{o​is​correct}]𝔼q∼Q,o∼πθ​[|o|]\displaystyle\frac{\mathbb{E}_{q\sim Q,\,o\sim\pi_{\theta}}\left[\mathbb{I}\{o\mathrm{~is~correct}\}\right]}{\mathbb{E}_{q\sim Q,\,o\sim\pi_{\theta}}\left[|o|\right]}(1)s.t.𝔼q∼Q,o∼πθ​[𝕀​{o​is​correct}]≥τ,\displaystyle\mathbb{E}_{q\sim Q,\,o\sim\pi_{\theta}}\left[\mathbb{I}\{o\mathrm{~is~correct}\}\right]\geq\tau,

Here,τ\taudenotes a minimum effectiveness threshold that prevents degenerated policies, e.g., trivially short but incorrect completions, from being mistakenly regarded as token-efficient. This formulation captures the central objective of token-efficient reasoning: preserving task-relevant information while eliminating redundant token generation.

3Proposed Methodology

Figure 2:Illustration of the information-aware advantage shaping module, wheresi,js_{i,j}andci,jc_{i,j}are token-wise advantages of the informativeness level and exploration adjustment of thejjth token in theiith completionoio_{i}of the completions group{oi}i=1G\{o_{i}\}_{i=1}^{G}.We first present an overview of theIAPOframework, followed by an illustration of its two critical modules: the information-aware advantage shaping module and the efficient conditional MI estimation module.

3.1Overview

We address the token-efficient post-training problem defined in Problem1by explicitly encouraging models to generateinformativetokens while suppressing redundant ones. We proposeIAPOframework (Fig.2), composed of two modules: Theinformation-aware advantage shaping moduleassigns token-level advantages based on each token’s informativeness (measured via conditional MI w.r.t. the final answer). To prevent training collapse and encourage beneficial exploration, this module incorporates an exploration adjustment metric. However, computing token informativeness faces two fundamental challenges: (1)inaccessible conditional MIand (2)significant computational overheadfor MI estimation. We address these challenges with the proposedefficient conditional MI estimation module, which tackles the first challenge via anearly-exit–based conditional MI estimatorand mitigates the second through ourKV-cache preloadingandchunk-wise forwardingtechniques.

3.2Information-Aware Advantage Shaping Module

3.2.1Informativeness Level of a Token

To quantify how much information a tokenoi,to_{i,t}from a completionoio_{i}contributes toward producing the final answeryiy_{i}, we measure its informativeness using the conditional mutual information (MI)I​(yi;oi,t∣q,oi,<t)I(y_{i};o_{i,t}\mid q,o_{i,<t}). We adopt this metric for two reasons: (1)Semantic alignment:From an information-theoretic perspective(Ash,2012; Brillouin,2013), conditional MI measures the reduction in uncertainty of the final answer after observingoi,to_{i,t}, conditioned on the preceding context (i.e.,oi,<to_{i,<t}). This directly reflects how semantically informativeoi,to_{i,t}is for determiningyiy_{i}. (2)Selective decomposition:If an entire completionoio_{i}has MII​(yi;oi∣q)I(y_{i};o_{i}\mid q)with the answer, the chain rule of MI givesI​(yi;oi∣q)=∑tI​(y;oi,t∣q,oi,<t)I(y_{i};o_{i}\mid q)=\sum_{t}I(y;o_{i,t}\mid q,o_{i,<t}). Thus, token-level conditional MI provides a principled decomposition of the total information contribution across the sequence. For any token budgetkk, selecting thekktokens with the highest conditional MI yields the subsequence that best preserves the mutual-information signal of the original completion. Formally,o^ik=arg⁡maxo~ik⊂oi,|o~ik|=k⁡I​(yi;o~ik∣q)\hat{o}_{i}^{k}=\arg\max_{\tilde{o}_{i}^{k}\subset o_{i},\,|\tilde{o}_{i}^{k}|=k}I(y_{i};\tilde{o}_{i}^{k}\mid q). We therefore incorporatesi,ts_{i,t}as the token-level informativeness score in the advantage shaping ofIAPO, where

si,t=I​(yi;ot∣qi,o<t).s_{i,t}=I(y_{i};o_{t}\mid q_{i},o_{<t}).(2)

3.2.2Token Exploration Adjustments

While a token’s informativeness level quantifies how much it contributes to producing the correct answer, relying solely on this metric for RL-based LLM post-training raises two potential concerns. First, aggressively optimizing for informativeness alone may lead to premature trajectory collapse, in which the model converges to overly concise reasoning patterns that sacrifice its reasoning accuracy. Second, the token-wise informativeness metric solely evaluates tokens’ contributions within the current reasoning trajectory, neglecting that the model also needs to explore alternative, potentially more effective reasoning paths.

To address the challenges, we introduce an exploration adjustment term that introduce token-level advantages to explore other potentially effective reasoning paths, thus further maintaining the reasoning accuracy. Specifically, we define the token-level exploration advantage as:

ci,t={πθold​(oi,t|q,oi,<t),if​oi​is correct,−πθold​(oi,t|q,oi,<t),if​oi​is incorrect,c_{i,t}=\begin{cases}\pi_{\theta_{\text{old}}}(o_{i,t}|q,o_{i,<t}),&\text{if }o_{i}\text{ is correct},\\ -\pi_{\theta_{\text{old}}}(o_{i,t}|q,o_{i,<t}),&\text{if }o_{i}\text{ is incorrect},\end{cases}(3)whereπθold​(oi,t|q,oi,<t)\pi_{\theta_{\text{old}}}(o_{i,t}|q,o_{i,<t})denotes the probability assigned by the frozen policyπθold\pi_{\theta_{\text{old}}}of the LLM to tokenoi,to_{i,t}before the current policy update step. Intuitively,πθold​(oi,t∣q,oi,<t)\pi_{\theta_{\mathrm{old}}}(o_{i,t}\mid q,o_{i,<t})reflects how expected or uncertain the model is about generating tokenoto_{t}. Assigning a positive advantage ofπθold​(oi,t∣q,oi,<t)\pi_{\theta_{\mathrm{old}}}(o_{i,t}\mid q,o_{i,<t})for correct completions amplifies tokens that the model is already confident about, whichreduces the entropyof the policy around high-confidence states and thereforesupresses exploration. Conversely, assigning a negative advantage of−πθold​(oi,t∣q,oi,<t)-\pi_{\theta_{\mathrm{old}}}(o_{i,t}\mid q,o_{i,<t})for incorrect completions inverts this signal: tokens the model is confident about now reduce the advantage, pushing the model toward less probable alternatives. Thisincreases the entropyof the policy and thereforeencourages exploration. Please see the formal theoretical analysis in Section4.2.

3.2.3Token-wise Advantage Assignment

In contrast to the vanilla GRPO(Shaoet al.,2024)that assigns a uniform advantage to all tokens within a completion,IAPOassigns the advantage of a token using three components:sequence-level reward,token-level informativeness level, andtoken-level exploration adjustment. Specifically, for completionoio_{i}with rewardrir_{i},IAPOassigns advantage

A~i,t=norm​(ri,𝐫)⏟seq-level reward+α​norm​(si,t,𝐬i)⏟token-level info.+β​norm​(ci,t,𝐜i)⏟token-level explo.\tilde{A}_{i,t}=\underbrace{\mathrm{norm}(r_{i},\mathbf{r})}_{\text{seq-level reward}}+\alpha\underbrace{\mathrm{norm}(s_{i,t},\mathbf{s}_{i})}_{\text{token-level info.}}+\beta\underbrace{\mathrm{norm}(c_{i,t},\mathbf{c}_{i})}_{\text{token-level explo.}}(4)to tokenoi,to_{i,t}, wherenorm​(x,𝐯)=[x−mean​(𝐯)]/std​(𝐯)\mathrm{norm}(x,\mathbf{v})=[x-\mathrm{mean}(\mathbf{v})]/\mathrm{std}(\mathbf{v}),𝐫={ri}i=1G\mathbf{r}=\{r_{i}\}_{i=1}^{G},𝐬i={si,t}t=1|oi|\mathbf{s}_{i}=\{s_{i,t}\}_{t=1}^{|o_{i}|}, and𝐜i={ci,t}t=1|oi|\mathbf{c}_{i}=\{c_{i,t}\}_{t=1}^{|o_{i}|}. SubstitutingA~i,t\tilde{A}_{i,t}into the GRPO objective yields theIAPOupdate. This design provably reduces expected completion length (see Section4) while achieving satisfying accuracy. Next, we analyze the fundamental challenges of conditional MI computation and design a tailored module to tackle them.

3.3Efficient Conditional MI Estimation Module

3.3.1Early-exit-based Conditional MI Estimator

Figure 3:Illustration of early-exit-based conditional MI estimator. We highlight “o<to_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}<}t}” and “o≤to_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\leq}t}” to emphasis the inclusion and exclusion of the current examined tokenoto_{t}in the partial completions.The informativeness metric proposed in Section3.2.1, defined as the conditional MII​(yi;oi,t∣q,oi,<t)I(y_{i};o_{i,t}\mid q,o_{i,<t}), is not directly observable during training. This is because, computing this quantity requires access to the actual posterior distributionp​(yi∣q,oi,≤t)p(y_{i}\mid q,o_{i,\leq t}), which cannot be obtained directly from the model’s autoregressive generation process. To address this challenge, we leverage the information-theoretic relationship between conditional MI and entropy:

I​(yi;oi,t∣q,oi,<t)=H​(yi∣q,oi,<t)−H​(yi∣q,oi,≤t).I(y_{i};o_{i,t}\mid q,o_{i,<t})=H(y_{i}\mid q,o_{i,<t})-H(y_{i}\mid q,o_{i,\leq t}).This formulation expresses the informativeness of the tokenoi,to_{i,t}as the reduction in uncertainty about the answeryiy_{i}after observingoi,to_{i,t}, given the preceding context. To estimate this value efficiently, we propose an early-exit–based approach that leverages the model’s ability to generate answers from intermediate reasoning states. For each prefix(q,oi,≤t)(q,o_{i,\leq t}), we append alightweight answer-generation postfix prompt(e.g., “</think><answer>”) that instructs the model to immediately produce the final answer without further reasoning. Specifically, we obtain two answer distributions. First, before tokenoi,to_{i,t}, we feed the prefix(q,oi,<t)(q,o_{i,<t})concatenated with the postfix prompt to the LLM and extract the probability distribution over answers from the logits at the final token position, yieldingH​(yi∣q,oi,<t)H(y_{i}\mid q,o_{i,<t}). Second, after generating the tokenoto_{t}, we similarly feed(q,oi,≤t)(q,o_{i,\leq t})with the postfix prompt to obtainH​(yi∣q,oi,≤t)H(y_{i}\mid q,o_{i,\leq t}). The conditional MI is then approximated asH​(yi∣q,oi,<t)−H​(yi∣q,oi,≤t).H(y_{i}\mid q,o_{i,<t})-H(y_{i}\mid q,o_{i,\leq t}).

This early-exit estimator provides a principled approximation by measuring how much the model’s uncertainty about the final answer decreases after generating tokenoi,to_{i,t}. Tokens that substantially reduce answer entropy receive higher informativeness level scores, while redundant tokens that provide little new information receive lower scores.

3.3.2Training Acceleration Techniques

Naively estimating the conditional MI for all tokens in a completion is computationally prohibitive: Each token requires two separate forward passes through the LLM, i.e., one before and one after generating the token, to compute the entropy differenceH​(yi∣q,oi,<t)−H​(yi∣q,oi,≤t)H(y_{i}\mid q,o_{i,<t})-H(y_{i}\mid q,o_{i,\leq t}). For a completion of lengthLL, the cumulative time complexity becomesO​(∑l=1Ll2​d)=O​(L3​d)O(\sum_{l=1}^{L}l^{2}d)=O(L^{3}d), whereddis the model’s embedding dimension. This cubic scaling makes the approach computationally intractable for long reasoning.

KV-Cache Preloading.

Figure 4:Illustration of the naive implementation and KV cache preloading technique in conditional MI estimation. We highlight the key, query, and values of the prompt postfix inred. We show the time complexities in the right, whereKKis the length of the prompt postfix,NNis|oi||o_{i}|,LLis|q|+|oi||q|+|o_{i}|, andddis the embedding dimension of the LLM. The technique is significantly faster than the naive implementation since the conditionK≪LK\ll Lholds.We proposeKV-cache preloadingfor fast conditional MI estimation. Specifically, we first performa single forward passover the full completion(q,oi)(q,o_{i})and store the resulting key–value (KV) caches at all transformer layers. These cached states fully encode the autoregressive prefixes(q,oi,≤t)(q,o_{i,\leq t})for every token positiontt. Crucially, when estimating the entropy at an intermediate position, the model isnotre-invoked on the textual prefix(q,oi,<t)(q,o_{i,<t}). Instead, the forward pass takes as input only the preloaded KV cache corresponding to the prefix and the short postfix prompt (e.g., “</think><answer>”) used to elicit an answer distribution. Therefore, all attention computations over the shared prefix tokens are reused exactly, and the cost of each entropy evaluation dependsonlyon the postfix length, rather than the full prefix length. This design eliminates redundant prefix processing entirely of conditional MI estimation from requiring full forward passes per token to lightweight cache continuations.

Chunk-wise LLM Forwarding.While KV-cache preloading removes redundant prefix computation, invoking the LLM sequentially for each token position would still incur nontrivial overhead. To further improve efficiency, we introduce chunk-wise LLM forwarding. This technique partitions each completion into contiguous token chunks and estimates conditional MIs for all tokens within a chunk simultaneously using a single batched forward pass. For all token positions within each chunk, we reuse the same KV cache states for the shared prefix and batch the corresponding postfix continuations. Chunk-wise forwarding amortizes the costs of model invocations and memory accesses across multiple tokens, substantially reducing per-token overhead.

Remark on Time Complexity Improvements.The proposed KV-cache preloading technique substantially reduces computational overhead. By caching intermediate states, we eliminate redundant prefix computations and reduce the complexity toO​((K3+L2)​d)O((K^{3}+L^{2})d), whereKKdenotes the length of the postfix prompt (“</think><answer>”) andLLis|o|+|q||o|+|q|. SinceK≪LK\ll Lin practice, this represents a substantial improvement over the naiveO​(L3​d)O(L^{3}d)approach.

Our chunk-wise LLM forwarding further amortizes costs by batching token evaluations. By partitioning the sequence intoCCchunks and processing multiple positions simultaneously, we achieve an effective complexity ofO​(K3​d⋅L/C+K2​L2​d/(2​C)+L2​d)O(K^{3}d\cdot L/C+K^{2}L^{2}d/(2C)+L^{2}d). This design reduces the per-token overhead by a factor ofCCwhile maintaining exact computation of informativeness level scores, makingIAPOpractical for training with long reasoning sequences.

4Theoretical Analysis

Here, we show that our proposedIAPOreduces completion lengths and enables well-adjusted RL exploration, thereby maintaining the reasoning accuracy of the LLMs.

4.1Completion Lengths Reduction

We show that, under a small policy step, the difference in expected completion length betweenIAPOand vanilla GRPO(Shaoet al.,2024)depends on the covariance between the completion lengthL​(o)L(o)and a key quantityS​(o)S(o). Here,S​(o)S(o)captures the total informativeness-weighted policy gradient signal accumulated across all tokens inoo.

Theorem 4.1.

Given an LLMπ0\pi_{0}, letLGRPOL_{\text{GRPO}}andLIAPOL_{\text{{IAPO}}}denote the expected completion lengths underπGRPO\pi_{\text{GRPO}}andπIAPO\pi_{\text{{IAPO}}}, which are one-step updated policy models given by GRPO andIAPOupon the original policyπ0\pi_{0}, respectively. For sufficiently small step sizeη\eta, we have

LIAPO−LGRPO∝Covo∼pGRPO​(L​(o),S​(o)),L_{\text{{IAPO}}}-L_{\text{GRPO}}\;\propto\;\mathrm{Cov}_{o\sim p_{\text{GRPO}}}\bigl(L(o),\,S(o)\bigr),(5)whereS​(o)S(o)is the informativeness-weighted accumulated token-level gradient induced by theIAPOadvantage.

Significantly,S​(o)S(o)is directly related to the average token-wise informativeness (see AppendixBfor details): when a completion contains a high proportion of informative tokens,S​(o)S(o)is large; conversely, if it includes many low-informativeness tokens (e.g., redundant verifications),S​(o)S(o)is small. Thus,S​(o)S(o)can be characterized by the average token-wise informativeness. Therefore, we can justify that the completion lengthL​(o)L(o)and the termS​(o)S(o)are negatively correlated, as evidenced by the negative correlation between the average token-wise informativeness andL​(o)L(o).

Corollary 4.2.

Given a queryqqand an LLM, if the average informativeness per token decreases monotonically as a function of completion length, then for sufficiently small policy update step size, we haveLIAPO<LGRPO.L_{\text{{IAPO}}}<L_{\text{GRPO}}.

This assumption is mild and natural because, as completions grow longer, they tend to accumulate more low-informativeness tokens, leadingS​(o)S(o)to decrease. This establishes the negative covariance in Theorem4.1, ensuring thatIAPOreduces the expected completion length.

4.2Exploration Adjustment

IAPOadditionally incorporates a token-level termci,t=πθold​(oi,t∣q,oi,<t)c_{i,t}=\pi_{\theta_{\text{old}}}(o_{i,t}\mid q,o_{i,<t})to regulate exploration. To analyze how it affects exploration, we define the degree of exploration of an LLMπ0\pi_{0}under an input sequences:=(q,oi,<t)s:=(q,o_{i,<t})as the entropy of its prediction distribution, i.e.,H(π0(⋅∣s))H(\pi_{0}(\cdot\mid s)).

For the policyπIAPO\pi_{\texttt{IAPO}}obtained by one policy-gradient update fromπ0\pi_{0}, it holds that(Cuiet al.,2025)

H(πIAPO(⋅∣s))−H(π0(⋅∣s))\displaystyle H(\pi_{\texttt{IAPO}}(\cdot\mid s))-H(\pi_{0}(\cdot\mid s))(6)≈−η​Covot∼π0(⋅∣s)​(log⁡π0​(ot∣s),A​(s,ot)).\displaystyle\approx-\eta\,\mathrm{Cov}_{o_{t}\sim\pi_{0}(\cdot\mid s)}\big(\log\pi_{0}(o_{t}\mid s),A(s,o_{t})\big).whereη\etais the policy update step size,A​(s,ot)A(s,o_{t})is the advantage assigned to tokenoto_{t}given the contextss. When the completion prefixed byssyields a correct answer, we setA​(s,ot)=π0​(ot∣s)A(s,o_{t})=\pi_{0}(o_{t}\mid s). In this case, we have the covariance

Covot∼π0(⋅∣s)​(log⁡π0​(ot∣s),π0​(ot∣s))>0,\mathrm{Cov}_{o_{t}\sim\pi_{0}(\cdot\mid s)}\big(\log\pi_{0}(o_{t}\mid s),\pi_{0}(o_{t}\mid s)\big)>0,which, by Eq. (6), impliesH(πIAPO(⋅∣s))<H(π0(⋅∣s))H(\pi_{\texttt{IAPO}}(\cdot\mid s))<H(\pi_{0}(\cdot\mid s)). Thus, exploration is suppressed, encouraging the policy to remain close to the current correct reasoning trajectory.

Conversely, when the completion is incorrect, we setA​(s,ot)=−π0​(ot∣s)A(s,o_{t})=-\pi_{0}(o_{t}\mid s), which reverses the sign of the covariance term. As a result, Eq. (6) yieldsH(πIAPO(⋅∣s))>H(π0(⋅∣s))H(\pi_{\texttt{IAPO}}(\cdot\mid s))>H(\pi_{0}(\cdot\mid s)), thereby increasing the policy entropy and promoting exploration of alternative potentially effective reasoning trajectories that may lead to correct answers.

5Empirical Study

We first introduce the experiment setup. Then, we present the evaluation results to answer the following research questions:RQ1: How well canIAPOimprove the reasoning efficiency compared with the state-of-the-art token-efficient LLM post-training baselines?RQ2: To what extent does each component ofIAPOcontribute to the overall reasoning token-efficiency?RQ3: How do hyperparameters such as the coefficient between thesi,ts_{i,t}andci,tc_{i,t}affectIAPO’s performance?RQ4: Is there a case study to show howIAPOis more token-efficient compared to other baselines?

5.1Experiment Settings

Here, we introduce our experiment settings. For more details such as hardware information, please refer to AppendixC.

Datasets.We evaluate on three representative mathematics reasoning datasets commonly used for LLM post-training. GSM8K(Cobbeet al.,2021)contains grade-school–level arithmetic word problems requiring multi-step numerical reasoning. MATH-500(Lightmanet al.,2024)is a competition-level mathematics dataset. DAPO-Math-17k(Yuet al.,2025)is a large-scale, diverse math dataset featuring longer solutions than GSM8K and MATH-500.

Baselines.We adopt state-of-the-art LLM post-training methods for token efficiency as baselines. Specifically, (1)DAPO(Yuet al.,2025)utilizes an overlong reward shaping technique that assigns zero advantage to overlong sequences. (2)GFPO(Shrivastavaet al.,2025)samples more completions during group sampling and only assigns rewards to the shortest completions. (3)GTPO(Tanet al.,2025)rewards tokens with high entropy in correct completions while penalizing tokens with low entropy in incorrect completions. (4)S-GRPO(Lee and Tong,2025)assigns zero advantage to tokens in a completion whose index is larger than a threshold.

Figure 5:Ablation Study forIAPO.Evaluation Metrics.We follow well-established work(Yuet al.,2025)and adopt Pass@kkas the metric for evaluating model effectiveness, which is the percentage of evaluation samples that the LLM answers correctly withkktrials. Correspondingly, we utilize Length@kk, the average length of thekkcompletions across all test queries. Finally, we design a token-efficiency metric, Ratio@kk, which is the ratio of Pass@kkto Length@kk, indicating how effective each token is at producing the correct answer.

Implementation Details.We optimize the LLMs using AdamW(Loshchilov and Hutter,2019)with a learning rate of1×10−61\times 10^{-6}and a decay factor of0.50.5. The completion group sizeGG(see Section2) is set to88, and the coefficient for KL-divergenceDKL​(πθ∥πref)D_{\mathrm{KL}}(\pi_{\theta}\,\|\,\pi_{\mathrm{ref}})is fixed at0.0010.001. Please see additional implementation details in AppendixC.

5.2Effectiveness & Efficiency ofIAPO

Table 1:Comparison ofIAPOwith baselines across three mathematical reasoning datasets and three LLMs in various scales. P@kk, L@kk, and R@kkdenote Pass@kk, Length@kk, and the ratio Pass@kk/Length@kk, respectively. The optimal R@kks are inbold, the runner-ups areunderlined. Results are shown fork∈{16,32}k\in\{16,32\}; for additional results withk∈{2,4,8}k\in\{2,4,8\}, please see AppendixC.2.2.MethodP@16P@32L@16L@32R@16R@32Dataset GSM8K(Cobbeet al.,2021)Qwen2.5-0.5B-Instruct0.4261±0.00990.5722±0.0081148.04±3.51149.64±6.012.88×10−32.88\times 10^{-3}3.82×10−33.82\times 10^{-3}+ DAPO(Yuet al.,2025)0.8605±0.00060.9050±0.0004172.95±0.27172.53±0.094.98×10−34.98\times 10^{-3}5.25×10−35.25\times 10^{-3}+ GFPO(Shrivastavaet al.,2025)0.8590±0.00560.9052±0.0043216.26±0.31215.99±0.123.97×10−33.97\times 10^{-3}4.19×10−34.19\times 10^{-3}+ GTPO(Tanet al.,2025)0.8519±0.00410.9085±0.0028243.17±0.26243.21±0.323.50×10−33.50\times 10^{-3}3.74×10−33.74\times 10^{-3}+ S-GRPO(Lee and Tong,2025)0.8489±0.00770.9017±0.0009159.70±0.13159.49±0.205.32×10−3¯\underline{5.32\times 10^{-3}}5.65×10−3¯\underline{5.65\times 10^{-3}}+IAPO(ours)0.8519±0.00840.8979±0.0068150.37±0.22150.15±0.245.67×𝟏𝟎−𝟑\mathbf{5.67\times 10^{-3}}5.98×𝟏𝟎−𝟑\mathbf{5.98\times 10^{-3}}Qwen2.5-1.5B-Instruct0.8160±0.00900.8941±0.0029152.66±1.53152.59±3.765.35×10−35.35\times 10^{-3}5.86×10−3¯\underline{5.86\times 10^{-3}}+ DAPO(Yuet al.,2025)0.9479±0.00590.9664±0.0050169.31±0.37169.80±0.265.60×10−3¯\underline{5.60\times 10^{-3}}5.69×10−35.69\times 10^{-3}+ GFPO(Shrivastavaet al.,2025)0.9510±0.00290.9725±0.0022203.85±0.65204.05±0.584.67×10−34.67\times 10^{-3}4.77×10−34.77\times 10^{-3}+ GTPO(Tanet al.,2025)0.9497±0.00280.9674±0.0012262.63±0.62262.49±0.783.62×10−33.62\times 10^{-3}3.69×10−33.69\times 10^{-3}+ S-GRPO(Lee and Tong,2025)0.9558±0.00320.9735±0.0027181.07±0.08180.85±0.165.28×10−35.28\times 10^{-3}5.38×10−35.38\times 10^{-3}+IAPO(ours)0.9512±0.00340.9707±0.0028163.32±0.46163.51±0.565.82×𝟏𝟎−𝟑\mathbf{5.82\times 10^{-3}}5.94×𝟏𝟎−𝟑\mathbf{5.94\times 10^{-3}}Qwen2.5-7B-Instruct0.9793±0.00090.9851±0.0016157.08±1.76156.43±1.056.23×10−36.23\times 10^{-3}6.30×10−36.30\times 10^{-3}+ DAPO(Yuet al.,2025)0.9778±0.00260.9816±0.0026160.36±0.73159.83±0.216.10×10−36.10\times 10^{-3}6.14×10−36.14\times 10^{-3}+ GFPO(Shrivastavaet al.,2025)0.9798±0.00160.9853±0.0009160.36±0.99160.50±0.576.11×10−36.11\times 10^{-3}6.14×10−36.14\times 10^{-3}+ GTPO(Tanet al.,2025)0.9765±0.00060.9826±0.0006192.99±0.10192.85±0.105.06×10−35.06\times 10^{-3}5.10×10−35.10\times 10^{-3}+ S-GRPO(Lee and Tong,2025)0.9790±0.00200.9843±0.0004147.67±0.70147.40±0.426.63×10−3¯\underline{6.63\times 10^{-3}}6.68×10−3¯\underline{6.68\times 10^{-3}}+IAPO(ours)0.9735±0.00120.9805±0.000984.03±0.1283.97±0.041.16×𝟏𝟎−𝟐\mathbf{1.16\times 10^{-2}}1.17×𝟏𝟎−𝟐\mathbf{1.17\times 10^{-2}}Dataset MATH-500(Lightmanet al.,2024)Qwen2.5-0.5B-Instruct0.3199±0.01720.4478±0.0048253.22±7.36257.55±6.551.26×10−31.26\times 10^{-3}1.74×10−31.74\times 10^{-3}+ DAPO(Yuet al.,2025)0.4714±0.01720.5488±0.0048162.41±1.57162.30±1.312.90×10−32.90\times 10^{-3}3.38×10−33.38\times 10^{-3}+ GFPO(Shrivastavaet al.,2025)0.5118±0.00480.5690±0.0095161.56±1.66161.55±1.903.17×10−3¯\underline{3.17\times 10^{-3}}3.52×10−33.52\times 10^{-3}+ GTPO(Tanet al.,2025)0.4714±0.02520.5354±0.0082472.01±1.23472.58±0.209.99×10−49.99\times 10^{-4}1.13×10−31.13\times 10^{-3}+ S-GRPO(Lee and Tong,2025)0.4680±0.04230.5253±0.0378103.37±0.39103.31±0.124.53×𝟏𝟎−𝟑\mathbf{4.53\times 10^{-3}}5.08×𝟏𝟎−𝟑\mathbf{5.08\times 10^{-3}}+IAPO(ours)0.5118±0.01720.5960±0.0218164.42±0.09164.96±0.413.11×10−33.11\times 10^{-3}3.61×10−3¯\underline{3.61\times 10^{-3}}Qwen2.5-1.5B-Instruct0.5152±0.01430.6330±0.0390296.65±3.09296.89±3.281.74×10−31.74\times 10^{-3}2.13×10−32.13\times 10^{-3}+ DAPO(Yuet al.,2025)0.6869±0.00820.7508±0.0190327.34±1.24326.89±1.112.10×10−32.10\times 10^{-3}2.30×10−32.30\times 10^{-3}+ GFPO(Shrivastavaet al.,2025)0.6465±0.01650.7003±0.0095374.95±2.54374.47±0.711.72×10−31.72\times 10^{-3}1.87×10−31.87\times 10^{-3}+ GTPO(Tanet al.,2025)0.5084±0.01720.5421±0.0126496.05±0.69495.83±0.331.02×10−31.02\times 10^{-3}1.09×10−31.09\times 10^{-3}+ S-GRPO(Lee and Tong,2025)0.7273±0.01430.7576±0.0218276.75±1.02278.08±0.372.63×10−3¯\underline{2.63\times 10^{-3}}2.72×10−3¯\underline{2.72\times 10^{-3}}+IAPO(ours)0.7172±0.01650.7576±0.0165265.12±1.35264.28±0.382.71×𝟏𝟎−𝟑\mathbf{2.71\times 10^{-3}}2.87×𝟏𝟎−𝟑\mathbf{2.87\times 10^{-3}}Qwen2.5-7B-Instruct0.6734±0.01260.7003±0.0190341.46±2.43340.73±1.951.97×10−31.97\times 10^{-3}2.06×10−32.06\times 10^{-3}+ DAPO(Yuet al.,2025)0.7811±0.00480.8148±0.0048331.61±1.34331.22±0.542.36×10−32.36\times 10^{-3}2.46×10−3¯\underline{2.46\times 10^{-3}}+ GFPO(Shrivastavaet al.,2025)0.7340±0.00480.7811±0.0126344.24±0.39344.59±0.252.13×10−32.13\times 10^{-3}2.27×10−32.27\times 10^{-3}+ GTPO(Tanet al.,2025)0.6128±0.01260.6498±0.0252453.00±0.47453.14±0.561.35×10−31.35\times 10^{-3}1.43×10−31.43\times 10^{-3}+ S-GRPO(Lee and Tong,2025)0.8013±0.01260.8215±0.0172335.58±0.44335.48±0.372.39×10−3¯\underline{2.39\times 10^{-3}}2.45×10−32.45\times 10^{-3}+IAPO(ours)0.7744±0.01260.8047±0.0048303.04±1.30302.24±0.752.56×𝟏𝟎−𝟑\mathbf{2.56\times 10^{-3}}2.66×𝟏𝟎−𝟑\mathbf{2.66\times 10^{-3}}Dataset DAPO-Math-17k(Yuet al.,2025)Qwen2.5-0.5B-Instruct0.0867±0.02490.1667±0.0094374.85±8.83378.65±8.642.31×10−42.31\times 10^{-4}4.40×10−44.40\times 10^{-4}+ DAPO(Yuet al.,2025)0.2467±0.00940.2733±0.009432.59±0.1832.22±0.127.57×10−37.57\times 10^{-3}8.48×10−38.48\times 10^{-3}+ GFPO(Shrivastavaet al.,2025)0.2067±0.04110.2667±0.049916.76±0.1916.79±0.131.23×𝟏𝟎−𝟐\mathbf{1.23\times 10^{-2}}1.59×𝟏𝟎−𝟐\mathbf{1.59\times 10^{-2}}+ GTPO(Tanet al.,2025)0.1400±0.01630.1733±0.009415.87±0.0215.86±0.018.82×10−38.82\times 10^{-3}1.09×10−21.09\times 10^{-2}+ S-GRPO(Lee and Tong,2025)0.1733±0.01890.2667±0.009417.57±0.1017.64±0.039.86×10−39.86\times 10^{-3}1.51×10−2¯\underline{1.51\times 10^{-2}}+IAPO(ours)0.1933±0.00940.2333±0.009417.52±0.2417.48±0.091.10×10−2¯\underline{1.10\times 10^{-2}}1.33×10−21.33\times 10^{-2}Qwen2.5-1.5B-Instruct0.1800±0.01630.2333±0.0249387.59±9.38386.86±6.844.64×10−44.64\times 10^{-4}6.03×10−46.03\times 10^{-4}+ DAPO(Yuet al.,2025)0.2133±0.02490.2800±0.028338.36±0.9738.35±0.935.56×10−3¯\underline{5.56\times 10^{-3}}7.30×10−37.30\times 10^{-3}+ GFPO(Shrivastavaet al.,2025)0.2133±0.03400.2733±0.009464.51±1.0664.59±1.223.31×10−33.31\times 10^{-3}4.23×10−34.23\times 10^{-3}+ GTPO(Tanet al.,2025)0.2000±0.01630.2533±0.018918.88±0.0418.87±0.021.06×𝟏𝟎−𝟐\mathbf{1.06\times 10^{-2}}1.34×𝟏𝟎−𝟐\mathbf{1.34\times 10^{-2}}+ S-GRPO(Lee and Tong,2025)0.2200±0.03270.2800±0.016360.36±0.7560.11±0.523.64×10−33.64\times 10^{-3}4.66×10−34.66\times 10^{-3}+IAPO(ours)0.2133±0.04110.3200±0.000041.39±0.5741.46±0.565.15×10−35.15\times 10^{-3}7.72×10−3¯\underline{7.72\times 10^{-3}}Qwen2.5-7B-Instruct0.3267±0.03400.3800±0.0566449.39±4.05446.26±2.147.27×10−47.27\times 10^{-4}8.52×10−48.52\times 10^{-4}+ DAPO(Yuet al.,2025)0.3467±0.02490.4200±0.0327445.48±1.03444.91±0.797.78×10−47.78\times 10^{-4}9.44×10−49.44\times 10^{-4}+ GFPO(Shrivastavaet al.,2025)0.3933±0.01890.4800±0.0327424.36±0.55424.10±0.499.27×10−4¯\underline{9.27\times 10^{-4}}1.13×10−3¯\underline{1.13\times 10^{-3}}+ GTPO(Tanet al.,2025)0.1667±0.02490.1933±0.0340519.29±0.26519.03±0.113.21×10−43.21\times 10^{-4}3.72×10−43.72\times 10^{-4}+ S-GRPO(Lee and Tong,2025)0.4067±0.01890.4667±0.0094398.34±0.13398.02±0.281.02×𝟏𝟎−𝟑\mathbf{1.02\times 10^{-3}}1.17×𝟏𝟎−𝟑\mathbf{1.17\times 10^{-3}}+IAPO(ours)0.3867±0.01890.4267±0.0249430.06±1.68429.62±0.948.99×10−48.99\times 10^{-4}9.93×10−49.93\times 10^{-4}We answerRQ1by evaluating the effectiveness and efficiency of our proposedIAPO, as shown in Table1. We observe that: (1)Efficiency.When achieving comparable Pass@kkperformance,IAPOconsistently generates the fewest tokens across datasets and model scales, outperforming all baselines in terms of reasoning length. (2)Effectiveness.IAPOattains the highest reasoning effectiveness (Pass@kk) across almost all experimental settings. In particular, we achieve a Pass@32 gain of more than 0.06 over the runner-up in configurations such as the Qwen2.5-1.5B-Instruct(Team,2024)model and MATH-500(Lightmanet al.,2024)dataset. (3)Effectiveness–Efficiency Trade-off.When evaluated using the token-efficiency metric Ratio@kk,IAPOachieves the best (or runner-up) across all settings.

Remark.We also evaluateIAPOon non-mathematical reasoning tasks (e.g.,commonsense reasoning) to assess its generalizability; see AppendixC.2.1. Our results show thatIAPOalso achieves the best token efficiency on these tasks. Additionally, beyond measuring completion length as an efficiency indicator, we also measurewall-clock inference timeand find that post-training withIAPOreduces inference time by more than 11% (see AppendixC.2.3).

5.3Ablation Study

Here, we addressRQ2by measuring the contribution of different components of our proposedIAPOusing two variants: (1)IAPO-NI: removes the token-wise advantage assignment based on token informativeness level (the conditional MI term). (2)IAPO-NE: replaces the early-exit-based informativeness level estimator with next-token entropy reduction term, i.e.,H​(oi,t∣q,oi,<t)−H​(oi,t+1∣q,oi,≤t)H(o_{i,t}\mid q,o_{i,<t})-H(o_{i,t+1}\mid q,o_{i,\leq t}), where the entropies are those of the LLM’s probability distributions when generatingoi,to_{i,t}andoi,t+1o_{i,t+1}given{q,oi,<t}\{q,o_{i,<t}\}and{q,oi,≤t}\{q,o_{i,\leq t}\}, respectively. We report Ratio@32 for these variants using Qwen2.5-0.5B-Instruct(Team,2024; Yanget al.,2024)in Fig.5. We observe that: (1)IAPO’s token efficiency degrades when removing the informativeness level term or substituting it with next-token prediction entropies. (2)IAPO-NEhas higher token efficiency thanIAPO-NI, showing that incorporating conditional MI is promising.

5.4Parameter Analysis

Figure 6:Parameter Analysis forIAPO.We answerRQ3by examining the effects of two key hyperparameters inIAPO, i.e., theα\alphaandβ\betain Equ.4. We report Pass@32 and Length@32 while varyingα\alpha(orβ\beta) in{10−6,10−4,10−2,1}\{10^{-6},10^{-4},10^{-2},1\}, with the other coefficient fixed at10−610^{-6}, in Fig.6(a) and Fig.6(b), respectively on Qwen2.5-0.5B-Instruct(Team,2024)on MATH-500 dataset(Lightmanet al.,2024). We observe that (1) Asα\alphaincreases, reasoning length decreases monotonically before10−210^{-2}, demonstrating the effectiveness of the informativeness level term in reducing verbosity. Although Pass@32 sometimes decreases, its degradation is substantially slower, indicating a favorable effectiveness-efficiency trade-off. (2) Asβ\betaincreases, Pass@32 can significantly improve, but reasoning length also grows sharply. This suggests that a largerβ\betaencourages exploration of reasoning, enhancing accuracy at the cost of substantially increased token consumption.

Figure 7:Case study: We show reasoning trajectories generated by Qwen2.5-7B-Instruct(Team,2024)post-trained byIAPOand baselines for the same query from GSM8K(Cobbeet al.,2021)dataset.IAPOproduces a substantially shorter yet sufficient completion by concentrating generation on high-informativeness tokens.

5.5Case Study

We answerRQ4by examining reasoning completions generated by different methods on a query from the GSM8K dataset(Cobbeet al.,2021), shown in Fig.7. We make two key observations: (1)IAPOproducessubstantially shorter reasoningby concentrating on informative tokens. For the query in Fig.7,IAPOgenerates the correct answer with only 15 tokens, which is 3.4× to 7× shorter than baselines (51–105 tokens). (2)Baselines exhibit distinct verbosity patterns. GFPO(Shrivastavaet al.,2025)restates problem givens (10 tokens) and relations (14 tokens). GTPO(Tanet al.,2025)repeatedly restate givens or its high-level reasoning strategies. DAPO(Yuet al.,2025)and S-GRPO(Lee and Tong,2025)share similar reasoning logic; they verbosely identify what quantities the problem requires.

6Related Work

6.1RL-based Post-training for LLMs

RL-based post-training methods improve LLM reasoning by optimizing sequence-level rewards, typically derived from correctness or preference signals, using policy-gradient algorithms such as PPO(Schulmanet al.,2017)and GRPO(Shaoet al.,2024). Variants including RLHF(Ouyanget al.,2022; Christianoet al.,2017), RLAIF(Leeet al.,2024,2024), and DPO(Rafailovet al.,2023)simplify reward modeling or replace human feedback with AI-based evaluators, improving scalability and stability(Baiet al.,2022; Rafailovet al.,2023). More recent reasoning-focused approaches apply outcome-based or verifier-guided rewards, encouraging correct final answers or stepwise validity without explicit supervision of reasoning chains(Uesatoet al.,2022; Lightmanet al.,2024). Although these methods enhance reasoning, their reasoning processes are verbose, consuming large amounts of tokens and generation time.

6.2Reasoning Efficiency of LLMs

Two main approaches improve LLM reasoning efficiency. The first(Cheng and Van Durme,2024; Suet al.,2025; Xuet al.,2025; Shenet al.,2025)replaces verbose chain-of-thought with continuous embeddings—implicit reasoning tokens—bypassing explicit text generation and reducing reasoning length. While efficient, these methods sacrifice interpretability by using embeddings rather than explicit text, limiting their use in high-stakes scenarios that require transparent rationales. The second approach, including ours, reduces reasoning length via advantage shaping under GRPO(Shaoet al.,2024). However, existing methods uniformly favor shorter completions(Shrivastavaet al.,2025; Yuet al.,2025)or apply heuristic position-based decay(Daiet al.,2025; Lee and Tong,2025), without distinguishing informative from redundant tokens during optimization. Consequently, they fail to align reasoning accuracy with token efficiency.

7Conclusion

We presentIAPO, a token-efficient post-training framework that assigns advantages based on information theoretic measurements, achieving up to 36% token reduction while preserving reasoning accuracy.IAPOprovides a principled foundation for efficient LLM reasoning optimization.

Acknowledgements

This work was supported in part by the National Science Foundation (NSF) under Grants IIS-2144209, IIS-2223769, BCS-2228534, and CMMI-2411248, and by the Office of Naval Research (ONR) under Grant N000142412636.

Impact Statement

This paper advances machine learning by improving the efficiency of large language model reasoning. By reducing reasoning token consumption by up to 47% while maintaining accuracy,IAPOlowers the compute and energy required for LLM inference, supporting more sustainable AI deployment and reducing the carbon footprint of large-scale usage. The resulting cost reductions also make advanced reasoning capabilities more accessible to researchers, organizations, and individuals with limited computational budgets, helping to democratize AI and lower barriers to entry for educational, scientific, and startup applications. Beyond efficiency,IAPO’s information-theoretic framework offers principled insights into which reasoning steps contribute most to correct answers, supporting more interpretable AI systems and better oversight. Finally, the methodological contributions of information-aware advantage shaping and efficient conditional mutual information estimation are general and may benefit other areas of machine learning beyond language model reasoning.

References

• R. B. Ash (2012)Information theory.Courier Corporation.Cited by:§3.2.1.
• Y. Bai, S. Kadavath, S. Kundu, A. Askell, J. Kernion, A. Jones, A. Chen, A. Goldie, A. Mirhoseini, C. McKinnon,et al.(2022)Constitutional ai: harmlessness from ai feedback.arXiv.Cited by:§6.1.
• L. Brillouin (2013)Science and information theory.Courier Corporation.Cited by:§3.2.1.
• J. Chen and N. Jiang (2019)Information-theoretic considerations in batch reinforcement learning.InICML,Cited by:§A.1.
• J. Cheng and B. Van Durme (2024)Compressed chain of thought: efficient reasoning through dense representations.arXiv.Cited by:§6.2.
• P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei (2017)Deep reinforcement learning from human preferences.NeurIPS.Cited by:§6.1.
• K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman (2021)Training verifiers to solve math word problems.arXiv.Cited by:§C.1,§C.1,§C.4,§1,Figure 7,Figure 7,§5.1,§5.5,Table 1.
• G. Cui, Y. Zhang, J. Chen, L. Yuan, Z. Wang, Y. Zuo, H. Li, Y. Fan, H. Chen, W. Chen,et al.(2025)The entropy mechanism of reinforcement learning for reasoning language models.arXiv.Cited by:§4.2.
• M. Dai, C. Yang, and Q. Si (2025)S-grpo: early exit via reinforcement learning in reasoning models.arXiv.Cited by:§1,§6.2.
• DeepSeek-AI (2025)DeepSeek-r1: incentivizing reasoning capability in llms via reinforcement learning.Nature645,pp. 633–638.Cited by:Figure 1,Figure 1,§1.
• I. Grondman, L. Busoniu, G. A. Lopes, and R. Babuska (2012)A survey of actor-critic reinforcement learning: standard and natural policy gradients.IEEE Trans. Syst. Man Cybern. Part C.Cited by:§2.
• K. Kumar, T. Ashraf, O. Thawakar, R. M. Anwer, H. Cholakkal, M. Shah, M. Yang, P. H. Torr, F. S. Khan, and S. Khan (2025)Llm post-training: a deep dive into reasoning large language models.arXiv.Cited by:§1.
• W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica (2023)Efficient memory management for large language model serving with pagedattention.InSOSP,Cited by:§C.1.
• A. Lee and H. Tong (2025)Token-efficient rl for llm reasoning.arXiv.Cited by:§A.2,Table 2,§1,§5.1,§5.5,Table 1,Table 1,Table 1,Table 1,Table 1,Table 1,Table 1,Table 1,Table 1,§6.2.
• H. Lee, S. Phatale, H. Mansoor, T. Mesnard, J. Ferret, K. R. Lu, C. Bishop, E. Hall, V. Carbune, A. Rastogi,et al.(2024)RLAIF vs. rlhf: scaling reinforcement learning from human feedback with ai feedback.InICML,Cited by:§6.1.
• H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe (2024)Let’s verify step by step.InICLR,Cited by:§C.1,§C.1,Figure 1,Figure 1,§1,§5.1,§5.2,§5.4,Table 1,§6.1.
• I. Loshchilov and F. Hutter (2019)Decoupled weight decay regularization.InICLR,Cited by:§5.1.
• L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray,et al.(2022)Training language models to follow instructions with human feedback.NeurIPS.Cited by:§6.1.
• R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn (2023)Direct preference optimization: your language model is secretly a reward model.NeurIPS.Cited by:§6.1.
• J. Rasley, S. Rajbhandari, O. Ruwase, and Y. He (2020)Deepspeed: system optimizations enable training deep learning models with over 100 billion parameters.InSIGKDD,Cited by:§C.1.
• J. Schmidhuber (2015)On learning to think: algorithmic information theory for novel combinations of reinforcement learning controllers and recurrent neural world models.arXiv.Cited by:§A.1.
• J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel (2016)High-dimensional continuous control using generalized advantage estimation.InICLR,Cited by:§A.2.
• J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov (2017)Proximal policy optimization algorithms.arXiv.Cited by:§A.2,§6.1.
• Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu,et al.(2024)Deepseekmath: pushing the limits of mathematical reasoning in open language models.arXiv.Cited by:§A.2,§1,§1,§2,§2,§3.2.3,§4.1,§6.1,§6.2.
• Z. Shen, H. Yan, L. Zhang, Z. Hu, Y. Du, and Y. He (2025)Codi: compressing chain-of-thought into continuous space via self-distillation.InEMNLP,pp. 677–693.Cited by:§6.2.
• V. Shrivastava, A. Awadallah, V. Balachandran, S. Garg, H. Behl, and D. Papailiopoulos (2025)Sample more to think less: group filtered policy optimization for concise reasoning.arXiv.Cited by:Table 2,§1,§5.1,§5.5,Table 1,Table 1,Table 1,Table 1,Table 1,Table 1,Table 1,Table 1,Table 1,§6.2.
• S. Still and D. Precup (2012)An information-theoretic approach to curiosity-driven reinforcement learning.Theory in Biosciences.Cited by:§A.1.
• D. Su, H. Zhu, Y. Xu, J. Jiao, Y. Tian, and Q. Zheng (2025)Token assorted: mixing latent and text tokens for improved language model reasoning.InICML,Cited by:§6.2.
• A. Talmor, J. Herzig, N. Lourie, and J. Berant (2019)Commonsenseqa: a question answering challenge targeting commonsense knowledge.InNAACL,Cited by:§C.2.1,Table 2,Table 2.
• H. Tan, J. Pan, J. Lin, T. Chen, Z. Zheng, Z. Tang, and H. Yang (2025)Gtpo and grpo-s: token and sequence-level reward shaping with policy entropy.arXiv.Cited by:Table 2,§1,§5.1,§5.5,Table 1,Table 1,Table 1,Table 1,Table 1,Table 1,Table 1,Table 1,Table 1.
• Q. Team (2024)Qwen2.5: a party of foundation models.External Links:LinkCited by:§C.1,§C.1,§C.1,§C.2.3,§C.4,Table 2,Table 2,§1,Figure 7,Figure 7,§5.2,§5.3,§5.4.
• H. Tran, Z. Yao, and H. Yu (2025)Exploiting tree structure for credit assignment in rl training of llms.arXiv.Cited by:§A.2.
• J. Uesato, N. Kushman, R. Kumar, F. Song, N. Siegel, L. Wang, A. Creswell, G. Irving, and I. Higgins (2022)Solving math word problems with process-and outcome-based feedback.arXiv.Cited by:§6.1.
• S. Wang, L. Yu, C. Gao, C. Zheng, S. Liu, R. Lu, K. Dang, X. Chen, J. Yang, Z. Zhang,et al.(2025)Beyond the 80/20 rule: high-entropy minority tokens drive effective reinforcement learning for llm reasoning.arXiv.Cited by:§A.2.
• G. Xie, Y. Shi, H. Tian, T. Yao, and X. Zhang (2025)CAPO: towards enhancing llm reasoning through generative credit assignment.arXiv.Cited by:§A.2.
• Y. Xu, X. Guo, Z. Zeng, and C. Miao (2025)SoftCoT: soft chain-of-thought for efficient reasoning with llms.InACL,pp. 23336–23351.Cited by:§6.2.
• A. Yang, B. Yang, B. Hui, B. Zheng, B. Yu, C. Zhou, C. Li, C. Li, D. Liu, F. Huang, G. Dong, H. Wei, H. Lin, J. Tang, J. Wang, J. Yang, J. Tu, J. Zhang, J. Ma, J. Xu, J. Zhou, J. Bai, J. He, J. Lin, K. Dang, K. Lu, K. Chen, K. Yang, M. Li, M. Xue, N. Ni, P. Zhang, P. Wang, R. Peng, R. Men, R. Gao, R. Lin, S. Wang, S. Bai, S. Tan, T. Zhu, T. Li, T. Liu, W. Ge, X. Deng, X. Zhou, X. Ren, X. Zhang, X. Wei, X. Ren, Y. Fan, Y. Yao, Y. Zhang, Y. Wan, Y. Chu, Y. Liu, Z. Cui, Z. Zhang, and Z. Fan (2024)Qwen2 technical report.arXiv preprint arXiv:2407.10671.Cited by:§C.1,§C.1,§C.1,§1,§5.3.
• Z. Yang, Z. Guo, Y. Huang, X. Liang, Y. Wang, and J. Tang (2025)TreeRPO: tree relative policy optimization.arXiv.Cited by:§A.2.
• Q. Yu, Z. Zhang, R. Zhu, Y. Yuan, X. Zuo, Y. Yue, W. Dai, T. Fan, G. Liu, L. Liu,et al.(2025)Dapo: an open-source llm reinforcement learning system at scale.arXiv.Cited by:§C.1,§C.1,Table 2,§1,§5.1,§5.1,§5.1,§5.5,Table 1,Table 1,Table 1,Table 1,Table 1,Table 1,Table 1,Table 1,Table 1,Table 1,§6.2.
• Z. Zhang, Z. Shan, K. Song, Y. Li, and K. Ren (2025)Linking process to outcome: conditional reward modeling for llm reasoning.arXiv.Cited by:§A.2.

Appendix

Organization.This appendix is organized as follows: SectionApresents supplementary related work on the application of information theory in reinforcement learning and natural language research, providing a broader context for our information-theoretic approach to token-efficient reasoning. SectionBpresents the theoretical foundations and proofs, including the complete derivation of Theorem4.1, demonstrating howIAPOreduces expected completion length through covariance-based analysis, along with Corollary4.2establishing conditions under whichIAPOachieves shorter completions than GRPO. SectionCprovides supplementary experimental results, including detailed implementation specifications (hardware configuration, distributed computing setup, hyperparameters), comprehensive evaluation results across additional metrics (Pass@2, Pass@4, Pass@8 with corresponding Length@kkand Ratio@kkmeasurements), training dynamics visualizations showing the evolution of correctness rewards, completion lengths, and efficiency ratios throughout the post-training process, and extensive case studies with token-level heatmaps demonstrating how our conditional mutual information estimator successfully identifies informative tokens while assigning low scores to redundant content such as meta-commentary and repetitive verifications. These supplementary materials provide complete transparency into our experimental methodology and offer deeper insights intoIAPO’s mechanisms for achieving token-efficient reasoning.

Appendix ASupplementary Related Work

A.1Application of Information Theory in Reinforcement Learning

Information theory has been used to analyze fundamental questions in reinforcement learning.Chen and Jiang (2019)use information theory to prove lower bounds in batch RL, showing that certain assumptions about data coverage and function approximation are necessary: without them, no algorithm can learn efficiently even with unlimited data.Still and Precup (2012)shows that Boltzmann exploration naturally emerges when you minimize the information cost (mutual information) of a policy while maintaining a target reward level.Schmidhuber (2015)proposes that when a model and controller share algorithmic mutual information, the controller can learn more efficiently by querying the model. While these works apply information theory to understand what’s possible in RL (lower bounds), how to explore efficiently, and how to leverage learned models, they focus on different problems than token efficiency in LLM reasoning. Our work takes a different direction: we use conditional mutual information to measure how much each token contributes to getting the correct answer, which lets us train models to generate shorter but still accurate reasoning chains.

A.2Token-Level Advantage Assignment in RL post-training for LLMs

Traditional RL-based post-training methods such as PPO(Schulmanet al.,2017)and GRPO(Shaoet al.,2024)face a fundamental challenge in credit assignment: determining which specific tokens contribute to the final outcome. While standard GRPO assigns uniform sequence-level advantages to all tokens(Shaoet al.,2024), recent work has explored more sophisticated token-level strategies. PPO employs a learned value network for token-level advantages via GAE(Schulmanet al.,2016), but this introduces complexity and overfitting risks. Tree-based approaches like TEMPO(Tranet al.,2025)and TreePO(Yanget al.,2025)exploit prefix structures in grouped rollouts to compute nonparametric values at branching points, while methods like T-SPMO(Lee and Tong,2025)apply token-level prefix matching for fine-grained credit assignment. Another line of work focuses on entropy-based selection:Wanget al.(2025)identifies high-entropy “forking tokens” as critical decision points. Process Reward Models (PRMs)(Zhanget al.,2025)provide step-wise supervision by evaluating intermediate reasoning, with recent work like CAPO(Xieet al.,2025)leveraging LLMs as generative PRMs for token-level critique. In contrast to these approaches that use entropy, tree structure, or learned value functions as proxies for importance, ourIAPOdirectly quantifies each token’s contribution through conditional mutual information with the final answer, providing an explicit, principled mechanism that measures how much a token reduces uncertainty about correctness rather than merely reflecting generation uncertainty.

Appendix BTheory

Theorem 4.1(Change in expected length = covariance).

LetLGRPOL_{\mathrm{GRPO}}andLIAPOL_{\texttt{IAPO}}denote the expected completion length of a queryqqunder the GRPO andIAPOupdated policies, respectively:

LGRPO≜𝔼o∼πθGRPO​[L​(o)],LIAPO≜𝔼o∼πθIAPO​[L​(o)].L_{\mathrm{GRPO}}\triangleq\mathbb{E}_{o\sim\pi_{\theta_{\mathrm{GRPO}}}}[L(o)],\qquad L_{\texttt{IAPO}}\triangleq\mathbb{E}_{o\sim\pi_{\theta_{\texttt{IAPO}}}}[L(o)].LetθIAPO=θGRPO+η​Δ​θs\theta_{\texttt{IAPO}}=\theta_{\mathrm{GRPO}}+\eta\,\Delta\theta_{s}, and define, for a trajectoryoo,

gt​(q,oi,t)≜∇θlog⁡πθ​(oi,t∣q)|θ=θGRPO​Δ​θs,S​(o)≜∑t=1L​(o)gt​(q,oi,t).g_{t}(q,o_{i,t})\triangleq\nabla_{\theta}\log\pi_{\theta}(o_{i,t}\mid q)\Big|_{\theta=\theta_{\mathrm{GRPO}}}\Delta\theta_{s},\qquad S(o)\triangleq\sum_{t=1}^{L(o)}g_{t}(q,o_{i,t}).Then, for sufficiently small step sizeη\eta, we have the first-order relation

1η​(LIAPO−LGRPO)≈dd​η​𝔼o∼pη​[L​(o)]|η=0=Covo∼pGRPO​(L​(o),S​(o)),\tfrac{1}{\eta}\bigl(L_{\texttt{IAPO}}-L_{\mathrm{GRPO}}\bigr)\;\approx\;\tfrac{d}{d\eta}\mathbb{E}_{o\sim p_{\eta}}[L(o)]\Big|_{\eta=0}\;=\;\mathrm{Cov}_{o\sim p_{\mathrm{GRPO}}}\bigl(L(o),\,S(o)\bigr),(7)wherepGRPO​(o)=pθGRPO​(o)p_{\mathrm{GRPO}}(o)=p_{\theta_{\mathrm{GRPO}}}(o)is the trajectory distribution under GRPO andpηp_{\eta}denotes the trajectory distribution induced byθη=θGRPO+η​Δ​θs\theta_{\eta}=\theta_{\mathrm{GRPO}}+\eta\,\Delta\theta_{s}. In particular, the sign of the change in expected completion length is determined by the covariance betweenL​(o)L(o)andS​(o)S(o)under the GRPO policy.

Proof.

We first analyze the parameters relationship betweenθIAPO\theta_{\texttt{IAPO}}andθGRPO\theta_{\mathrm{GRPO}}:

θIAPO=\displaystyle\theta_{\texttt{IAPO}}={}θ+η​1G​∑i=1G1|oi|​∑t=1|oi|A^i,t​∇θlog⁡πθ​(oi,t∣q,oi,<t)\displaystyle\theta+\eta\,\tfrac{1}{G}\sum_{i=1}^{G}\tfrac{1}{|o_{i}|}\sum_{t=1}^{|o_{i}|}\hat{A}_{i,t}\,\nabla_{\theta}\log\pi_{\theta}(o_{i,t}\mid q,o_{i,<t})(8)=\displaystyle={}θ+η​1G​∑i=1G1|oi|​∑t=1|oi|ri−mean​(𝐫)std​(𝐫)​∇θlog⁡πθ​(oi,t∣q,oi,<t)\displaystyle\theta+\eta\,\tfrac{1}{G}\sum_{i=1}^{G}\tfrac{1}{|o_{i}|}\sum_{t=1}^{|o_{i}|}\tfrac{r_{i}-\mathrm{mean}(\mathbf{r})}{\mathrm{std}(\mathbf{r})}\,\nabla_{\theta}\log\pi_{\theta}(o_{i,t}\mid q,o_{i,<t})+η​1G​∑i=1G1|oi|​∑t=1|oi|si,t−mean​(𝐬i)std​(𝐬i)​∇θlog⁡πθ​(oi,t∣q,oi,<t)\displaystyle{}+\eta\,\tfrac{1}{G}\sum_{i=1}^{G}\tfrac{1}{|o_{i}|}\sum_{t=1}^{|o_{i}|}\tfrac{s_{i,t}-\mathrm{mean}(\mathbf{s}_{i})}{\mathrm{std}(\mathbf{s}_{i})}\,\nabla_{\theta}\log\pi_{\theta}(o_{i,t}\mid q,o_{i,<t})=\displaystyle={}θGRPO+η​1G​∑i=1G1|oi|​∑t=1|oi|si,t−mean​(𝐬i)std​(𝐬i)​∇θlog⁡πθ​(oi,t∣q,oi,<t).\displaystyle\theta_{\mathrm{GRPO}}+\eta\,\tfrac{1}{G}\sum_{i=1}^{G}\tfrac{1}{|o_{i}|}\sum_{t=1}^{|o_{i}|}\tfrac{s_{i,t}-\mathrm{mean}(\mathbf{s}_{i})}{\mathrm{std}(\mathbf{s}_{i})}\,\nabla_{\theta}\log\pi_{\theta}(o_{i,t}\mid q,o_{i,<t}).TheIAPOparameters can be written as

θIAPO=θGRPO+η​Δ​θs,Δ​θs=1G​∑i=1G1|oi|​∑t=1|oi|βi,t​∇θlog⁡πθ​(oi,t∣q,oi,<t)|θ=θGRPO,\theta_{\texttt{IAPO}}=\theta_{\mathrm{GRPO}}+\eta\,\Delta\theta_{s},\qquad\Delta\theta_{s}=\tfrac{1}{G}\sum_{i=1}^{G}\tfrac{1}{|o_{i}|}\sum_{t=1}^{|o_{i}|}\beta_{i,t}\,\nabla_{\theta}\log\pi_{\theta}(o_{i,t}\mid q,o_{i,<t})\Big|_{\theta=\theta_{\mathrm{GRPO}}},(9)whereβi,t=si,t−mean​(𝐬i)std​(𝐬i)\beta_{i,t}=\tfrac{s_{i,t}-\mathrm{mean}(\mathbf{s}_{i})}{\mathrm{std}(\mathbf{s}_{i})}is the normalized informativeness score.

We treatη\etaas a small scalar so that theIAPOupdate is a small perturbation ofθGRPO\theta_{\mathrm{GRPO}}. For any(q,oi,t)(q,o_{i,t}), a first-order Taylor expansion of the log-policy aroundθGRPO\theta_{\mathrm{GRPO}}gives

log⁡πθIAPO​(oi,t∣q)≈log⁡πθGRPO​(oi,t∣q)+η​∇θlog⁡πθ​(oi,t∣q)|θ=θGRPO​Δ​θs.\log\pi_{\theta_{\texttt{IAPO}}}(o_{i,t}\mid q)\;\approx\;\log\pi_{\theta_{\mathrm{GRPO}}}(o_{i,t}\mid q)+\eta\,\nabla_{\theta}\log\pi_{\theta}(o_{i,t}\mid q)\Big|_{\theta=\theta_{\mathrm{GRPO}}}\!\Delta\theta_{s}.(10)Exponentiating and linearizingex≈1+xe^{x}\approx 1+xyields

πθIAPO​(oi,t∣q)≈πθGRPO​(oi,t∣q)​(1+η​gt​(q,oi,t)),\pi_{\theta_{\texttt{IAPO}}}(o_{i,t}\mid q)\;\approx\;\pi_{\theta_{\mathrm{GRPO}}}(o_{i,t}\mid q)\bigl(1+\eta\,g_{t}(q,o_{i,t})\bigr),(11)where we define

gt​(q,oi,t)≜∇θlog⁡πθ​(oi,t∣q)|θ=θGRPO​Δ​θs.g_{t}(q,o_{i,t})\;\triangleq\;\nabla_{\theta}\log\pi_{\theta}(o_{i,t}\mid q)\Big|_{\theta=\theta_{\mathrm{GRPO}}}\!\Delta\theta_{s}.(12)For a trajectoryo=(o1,…,oL​(o))o=(o_{1},\dots,o_{L(o)})we have

pθ​(o)=∏t=1L​(o)πθ​(oi,t∣q),p_{\theta}(o)=\prod_{t=1}^{L(o)}\pi_{\theta}(o_{i,t}\mid q),(13)so substituting (11) and linearizing the product gives

pIAPO​(o)\displaystyle p_{\texttt{IAPO}}(o)=∏t=1L​(o)πθIAPO​(oi,t∣q)≈∏t=1L​(o)[πθGRPO​(oi,t∣q)​(1+η​gt​(q,oi,t))]\displaystyle=\prod_{t=1}^{L(o)}\pi_{\theta_{\texttt{IAPO}}}(o_{i,t}\mid q)\;\approx\;\prod_{t=1}^{L(o)}\Bigl[\pi_{\theta_{\mathrm{GRPO}}}(o_{i,t}\mid q)\bigl(1+\eta\,g_{t}(q,o_{i,t})\bigr)\Bigr](14)=∏t=1L​(o)πθGRPO​(oi,t∣q)⏟=pGRPO​(o)​∏t=1L​(o)(1+η​gt​(q,oi,t))\displaystyle=\underbrace{\prod_{t=1}^{L(o)}\pi_{\theta_{\mathrm{GRPO}}}(o_{i,t}\mid q)}_{=\,p_{\mathrm{GRPO}}(o)}\prod_{t=1}^{L(o)}\bigl(1+\eta\,g_{t}(q,o_{i,t})\bigr)≈pGRPO​(o)​(1+η​∑t=1L​(o)gt​(q,oi,t))=pGRPO​(o)​(1+η​S​(o)),\displaystyle\approx p_{\mathrm{GRPO}}(o)\bigl(1+\eta\textstyle\sum_{t=1}^{L(o)}g_{t}(q,o_{i,t})\bigr)\;=\;p_{\mathrm{GRPO}}(o)\bigl(1+\eta\,S(o)\bigr),where we definedS​(o)≜∑t=1L​(o)gt​(q,oi,t)S(o)\triangleq\sum_{t=1}^{L(o)}g_{t}(q,o_{i,t}).

The expression in (14) isunnormalized. Let

p~η​(o)≜pGRPO​(o)​(1+η​S​(o)),Z​(η)≜∑op~η​(o)=∑opGRPO​(o)​(1+η​S​(o)).\tilde{p}_{\eta}(o)\triangleq p_{\mathrm{GRPO}}(o)\bigl(1+\eta\,S(o)\bigr),\qquad Z(\eta)\triangleq\sum_{o}\tilde{p}_{\eta}(o)=\sum_{o}p_{\mathrm{GRPO}}(o)\bigl(1+\eta\,S(o)\bigr).Then the properly normalized trajectory distribution is

pη​(o)≜p~η​(o)Z​(η)=pGRPO​(o)​(1+η​S​(o))1+η​𝔼o∼pGRPO​[S​(o)].p_{\eta}(o)\;\triangleq\;\tfrac{\tilde{p}_{\eta}(o)}{Z(\eta)}\;=\;\tfrac{p_{\mathrm{GRPO}}(o)\bigl(1+\eta\,S(o)\bigr)}{1+\eta\,\mathbb{E}_{o\sim p_{\mathrm{GRPO}}}[S(o)]}.(15)For smallη\eta, we can again linearize the denominator to obtain

pη​(o)\displaystyle p_{\eta}(o)≈pGRPO​(o)​(1+η​S​(o))​(1−η​𝔼o′∼pGRPO​[S​(o′)])\displaystyle\approx p_{\mathrm{GRPO}}(o)\bigl(1+\eta\,S(o)\bigr)\bigl(1-\eta\,\mathbb{E}_{o^{\prime}\sim p_{\mathrm{GRPO}}}[S(o^{\prime})]\bigr)(16)≈pGRPO​(o)​(1+η​(S​(o)−𝔼o′∼pGRPO​[S​(o′)])).\displaystyle\approx p_{\mathrm{GRPO}}(o)\bigl(1+\eta\bigl(S(o)-\mathbb{E}_{o^{\prime}\sim p_{\mathrm{GRPO}}}[S(o^{\prime})]\bigr)\bigr).Now consider the expected completion length underpηp_{\eta}:

𝔼o∼pη​[L​(o)]=∑opη​(o)​L​(o).\mathbb{E}_{o\sim p_{\eta}}[L(o)]=\sum_{o}p_{\eta}(o)\,L(o).Using (16) and keeping only first-order terms inη\eta, we obtain

𝔼o∼pη​[L​(o)]\displaystyle\mathbb{E}_{o\sim p_{\eta}}[L(o)]≈∑opGRPO​(o)​(1+η​(S​(o)−𝔼o′∼pGRPO​[S​(o′)]))​L​(o)\displaystyle\approx\sum_{o}p_{\mathrm{GRPO}}(o)\bigl(1+\eta(S(o)-\mathbb{E}_{o^{\prime}\sim p_{\mathrm{GRPO}}}[S(o^{\prime})])\bigr)L(o)(17)=𝔼o∼pGRPO​[L​(o)]+η​𝔼o∼pGRPO​[(S​(o)−𝔼o′∼pGRPO​[S​(o′)])​L​(o)]\displaystyle=\mathbb{E}_{o\sim p_{\mathrm{GRPO}}}[L(o)]+\eta\,\mathbb{E}_{o\sim p_{\mathrm{GRPO}}}\Bigl[\bigl(S(o)-\mathbb{E}_{o^{\prime}\sim p_{\mathrm{GRPO}}}[S(o^{\prime})]\bigr)L(o)\Bigr]=LGRPO+η​(𝔼o∼pGRPO​[L​(o)​S​(o)]−𝔼o∼pGRPO​[L​(o)]​𝔼o∼pGRPO​[S​(o)])\displaystyle=L_{\mathrm{GRPO}}+\eta\,\Bigl(\mathbb{E}_{o\sim p_{\mathrm{GRPO}}}[L(o)\,S(o)]-\mathbb{E}_{o\sim p_{\mathrm{GRPO}}}[L(o)]\,\mathbb{E}_{o\sim p_{\mathrm{GRPO}}}[S(o)]\Bigr)=LGRPO+η​Covo∼pGRPO​(L​(o),S​(o)).\displaystyle=L_{\mathrm{GRPO}}+\eta\,\mathrm{Cov}_{o\sim p_{\mathrm{GRPO}}}\bigl(L(o),\,S(o)\bigr).By definition ofpηp_{\eta}, theIAPOexpected length corresponds toη\etaevaluated at the small update step used in Eq. (9), so to first order inη\eta, we have

LIAPO=𝔼o∼pη​[L​(o)]≈LGRPO+η​Covo∼pGRPO​(L​(o),S​(o)).L_{\texttt{IAPO}}=\mathbb{E}_{o\sim p_{\eta}}[L(o)]\approx L_{\mathrm{GRPO}}+\eta\,\mathrm{Cov}_{o\sim p_{\mathrm{GRPO}}}\bigl(L(o),\,S(o)\bigr).Rearranging yields

1η​(LIAPO−LGRPO)≈Covo∼pGRPO​(L​(o),S​(o)),\tfrac{1}{\eta}\bigl(L_{\texttt{IAPO}}-L_{\mathrm{GRPO}}\bigr)\approx\mathrm{Cov}_{o\sim p_{\mathrm{GRPO}}}\bigl(L(o),\,S(o)\bigr),which is exactly Eq. (7). This completes the proof. ∎

Corollary 4.2(Keyword-aligned updates shrink expected length).

Assume that, under the GRPO policyπθGRPO\pi_{\theta_{\mathrm{GRPO}}}: (i) the token-wise scoressi,ts_{i,t}satisfy that informative tokens have strictly larger scores than non-informative tokens, and (ii) for trajectoriesoowith comparable semantic quality (same labelYY), longer trajectories contain strictly more non-informative tokens. Then there existsη0>0\eta_{0}>0such that for all0<η<η00<\eta<\eta_{0}, theIAPOupdate satisfies

LIAPO<LGRPO.L_{\texttt{IAPO}}\,<\,L_{\mathrm{GRPO}}.

Proof.

By Theorem4.1we have, for smallη\eta,

LIAPO=LGRPO+η​Covo∼pGRPO​(L​(o),S​(o))+o​(η).L_{\texttt{IAPO}}=L_{\mathrm{GRPO}}+\eta\,\mathrm{Cov}_{o\sim p_{\mathrm{GRPO}}}\bigl(L(o),\,S(o)\bigr)+o(\eta).Conditions (i)–(ii), together with the within-trajectory normalizationβi,t=si,t−mean​(𝐬i)std​(𝐬i)\beta_{i,t}=\tfrac{s_{i,t}-\mathrm{mean}(\mathbf{s}_{i})}{\mathrm{std}(\mathbf{s}_{i})}, imply that, at fixed semantic quality, trajectories with larger lengthL​(o)L(o)have strictly smaller total informativeness score mass∑tβi,t\sum_{t}\beta_{i,t}, and hence strictly smallerS​(o)S(o). ThereforeL​(o)L(o)andS​(o)S(o)are negatively correlated underpGRPOp_{\mathrm{GRPO}}, i.e.,Cov​(L​(o),S​(o))<0\mathrm{Cov}(L(o),S(o))<0. For sufficiently smallη\eta, the first-order term dominates the higher-order terms, which yieldsLIAPO<LGRPOL_{\texttt{IAPO}}<L_{\mathrm{GRPO}}. ∎

Appendix CSupplementary Experiments

C.1Implementation Details

Remarks on Reasoning Length Measurement.In our work, we employ different length measurement approaches depending on the context and purpose of each analysis. In Fig.1, we compare model-generated reasoning with human-written solutions. Since human solutions are not processed through any LLM tokenizer, we use whitespace-delimited word counts for both human and model outputs to enable fair comparison. In Fig.7, we present reasoning trajectories as natural text to illustrate qualitative differences in verbosity. For interpretability and alignment with the displayed text, we report whitespace-delimited word counts rather than subword tokens provided by the LLMs’ tokenizers. In contrast, all quantitative results reported in Table1use the native tokenizer of the corresponding LLM, as this reflects actual computational cost and ensures fair comparison across methods applied to the same base model.

Data Preprocessing.We use the full GSM8K(Cobbeet al.,2021)and MATH-500(Lightmanet al.,2024)datasets. Due to the large scale of DAPO-Math-17k(Yuet al.,2025)(1.79M samples), we subsample the first 7,000 examples for training and validation. Specifically, the training sets contain 7,473, 400, and 5,600 samples for GSM8K, MATH-500, and DAPO-Math-17k, respectively, while the corresponding validation sets contain 1,319, 100, and 50 samples. For DAPO-Math-17k, the validation set is randomly sampled from the remaining 1,400 examples after the training split.

Training Configurations & Implementation Details.In addition to the key settings described in the main paper, we provide further details of our post-training procedure. All baselines andIAPOare trained with gradient accumulation steps of 6 for Qwen2.5-0.5B-Instruct and Qwen2.5-1.5B-Instruct, and 8 for Qwen2.5-7B-Instruct(Yanget al.,2024; Team,2024). Gradient clipping is applied with a maximum norm of 1.0. We tune both the surprise and confidence coefficients over{10−6,10−4,10−2,1}\{10^{-6},10^{-4},10^{-2},1\}and report the best-performing configuration for each dataset–model combination. For Qwen2.5-0.5B-Instruct and Qwen2.5-1.5B-Instruct(Yanget al.,2024; Team,2024), the per-GPU batch size is set to 32, corresponding to32/8=432/8=4data items per device for group completion generation. For Qwen2.5-7B-Instruct(Yanget al.,2024; Team,2024), the per-GPU batch size is reduced to 16, corresponding to16/8=216/8=2data items per device. We utilize theH​(ot|q,o<t)H(o_{t}|q,o_{<t})as substitution for theπθ​(ot|q,o<t)\pi_{\theta}(o_{t}|q,o_{<t})for better practical significance. We train 10,000 steps for DAPO-Math-17k(Yuet al.,2025)dataset, 2 epochs for GSM8K(Cobbeet al.,2021)and 152 epochs for MATH-500 dataset(Lightmanet al.,2024). We take the model checkpoints with the top validation correctness reward and conduct evaluation on them, and report the evaluation results of the checkpoint with the highest evaluation R@32 score.

Hardware Information.Post-training of Qwen2.5-0.5B-Instruct and Qwen2.5-1.5B-Instruct(Yanget al.,2024; Team,2024)is conducted using 4 NVIDIA H100 GPUs (80GB HBM3), while Qwen2.5-7B-Instruct(Yanget al.,2024; Team,2024)is trained using 8 H100 GPUs. All evaluations are performed using 2 H100 GPUs across all model sizes.

Distributed Computing Configurations.We adopt DeepSpeed ZeRO Stage 2(Rasleyet al.,2020)for post-training Qwen2.5-0.5B-Instruct and Qwen2.5-1.5B-Instruct(Yanget al.,2024; Team,2024), and ZeRO Stage 3(Rasleyet al.,2020)for Qwen2.5-7B-Instruct(Yanget al.,2024; Team,2024). For completion generation, we employ vLLM(Kwonet al.,2023)in colocation mode, with GPU memory utilization set to 0.4 and tensor parallelism set to 2.

C.2Effectiveness & Efficiency

C.2.1Token Efficiency on Non-Math Reasoning Datasets

Although LLM post-training methods are primarily benchmarked on mathematical reasoning datasets, we also report the performance of ourIAPOmethod on a commonsense reasoning dataset, namely CommonsenseQA(Talmoret al.,2019). In this dataset, each prompt presents a commonsense question followed by multiple-choice options. For example, a prompt may be: “The sanctions against the school were a punishing blow, and they seemed to what the efforts the school had made to change? A. ignore, B. enforce, C. authoritarian, D. yell at, E. avoid.” The LLM is expected to reason about the question and output the letter corresponding to its chosen answer. We report Pass@K, Length@K, and Ratio@K in Table2. We observe thatIAPOachieves the second-highest token-efficiency, as measured by Ratio@16 and Ratio@32. At the same time,IAPOdelivers substantially higher effectiveness—approximately a 20% improvement in P@16 and P@32—demonstrating that our method achieves promising post-training performance.

Table 2:Comparison ofIAPOwith various baselines on CommonsenseQA(Talmoret al.,2019)and Qwen2.5-0.5B-Instruct(Team,2024). P@kk, L@kk, and R@kkdenote Pass@kk, Length@kk, and the ratio Pass@kk/Length@kk, respectively.

C.2.2Extended Evaluation on Pass@k and Length@k

We evaluate ourIAPOand the baselines on the three adopted datasets and LLMs with Pass@2, Length@2, Ratio@2, Pass@4, Length@4, Ratio@4, Pass@8, Length@8, Ratio@8 as well. The results of Pass@2, Length@2, and Ratio@2 are shown in Table3, the results of Pass@4, Length@4, and Ratio@4 are shown in Table4, the results of Pass@8, Length@8 and Ratio@8 are shown in Table5. Our evaluation results align with those of the main paper that our proposedIAPOachieves a generally optimal effectiveness-efficiency trade-off when tested across LLMs and reasoning datasets.

Table 3:Effectiveness and efficiency ofIAPOand baselines evaluated with P@2, L@2, and [email protected]@20.6515±0.00660.6027±0.00490.6138±0.00890.6113±0.01470.6247±[email protected]±2.59217.02±1.56241.73±1.55159.72±0.97151.00±[email protected]×10−33.78\times 10^{-3}2.78×10−32.78\times 10^{-3}2.54×10−32.54\times 10^{-3}3.83×10−33.83\times 10^{-3}4.14×10−34.14\times 10^{-3}[email protected]±0.00940.0400±0.01630.0400±0.01630.0733±0.02490.1000±[email protected]±0.5816.86±0.5515.93±0.1917.64±0.0317.43±[email protected]×10−32.67\times 10^{-3}2.37×10−32.37\times 10^{-3}2.51×10−32.51\times 10^{-3}4.16×10−34.16\times 10^{-3}5.74×10−35.74\times 10^{-3}[email protected]±0.01260.2896±0.01260.2323±0.03590.2559±0.01260.2761±[email protected]±2.93164.27±5.21475.90±1.82102.58±1.36163.65±[email protected]×10−31.80\times 10^{-3}1.76×10−31.76\times 10^{-3}4.88×10−44.88\times 10^{-4}2.49×10−32.49\times 10^{-3}1.69×10−31.69\times 10^{-3}[email protected]±0.00530.8279±0.00320.8198±0.00340.8426±0.00850.8261±[email protected]±1.98202.02±0.76261.40±0.31180.13±0.83160.20±[email protected]×10−34.87\times 10^{-3}4.10×10−34.10\times 10^{-3}3.14×10−33.14\times 10^{-3}4.68×10−34.68\times 10^{-3}5.16×10−35.16\times 10^{-3}[email protected]±0.00000.0600±0.03270.0800±0.01630.0933±0.00940.0733±[email protected]±1.3364.59±4.9818.94±0.1362.01±2.6141.08±[email protected]×10−31.04\times 10^{-3}9.29×10−49.29\times 10^{-4}4.22×10−34.22\times 10^{-3}1.50×10−31.50\times 10^{-3}1.78×10−31.78\times 10^{-3}[email protected]±0.01900.4949±0.02180.3502±0.00950.5421±0.01260.5488±[email protected]±3.31374.32±8.07495.30±3.07276.59±3.55264.53±[email protected]×10−31.66\times 10^{-3}1.32×10−31.32\times 10^{-3}7.07×10−47.07\times 10^{-4}1.96×10−31.96\times 10^{-3}2.07×10−32.07\times 10^{-3}[email protected]±0.00060.9449±0.00280.9338±0.00340.9419±0.00220.9179±[email protected]±3.20156.16±1.96193.27±0.60144.39±0.1484.14±[email protected]×10−36.03\times 10^{-3}6.05×10−36.05\times 10^{-3}4.83×10−34.83\times 10^{-3}6.52×10−36.52\times 10^{-3}1.09×10−21.09\times 10^{-2}[email protected]±0.02830.2667±0.00940.1200±0.01630.2600±0.00000.2600±[email protected]±2.66420.55±3.39518.77±1.87400.40±9.52425.78±[email protected]×10−45.40\times 10^{-4}6.34×10−46.34\times 10^{-4}2.31×10−42.31\times 10^{-4}6.49×10−46.49\times 10^{-4}6.11×10−46.11\times 10^{-4}[email protected]±0.01260.6094±0.00950.5185±0.02520.6700±0.02380.6801±[email protected]±3.79341.66±2.71452.15±1.19334.14±2.25299.38±[email protected]×10−31.96\times 10^{-3}1.78×10−31.78\times 10^{-3}1.15×10−31.15\times 10^{-3}2.00×10−32.00\times 10^{-3}2.27×10−32.27\times 10^{-3}Table 4:Effectiveness and efficiency ofIAPOand baselines evaluated with P@4, L@4, and [email protected]@40.7374±0.00560.7028±0.01250.7159±0.00950.7111±0.00590.7165±[email protected]±1.65215.23±1.60242.17±0.15159.34±0.88150.32±[email protected]×10−34.28\times 10^{-3}3.27×10−33.27\times 10^{-3}2.96×10−32.96\times 10^{-3}4.46×10−34.46\times 10^{-3}4.77×10−34.77\times 10^{-3}[email protected]±0.01890.0533±0.00940.0667±0.01890.1133±0.01890.1200±[email protected]±0.4516.60±0.2815.86±0.1117.60±0.1317.39±[email protected]×10−34.43\times 10^{-3}3.21×10−33.21\times 10^{-3}4.21×10−34.21\times 10^{-3}6.44×10−36.44\times 10^{-3}6.90×10−36.90\times 10^{-3}[email protected]±0.00820.3535±0.00820.2997±0.03900.3232±0.01650.3704±[email protected]±1.16162.88±2.78474.21±0.53101.98±1.77162.34±[email protected]×10−32.07\times 10^{-3}2.17×10−32.17\times 10^{-3}6.32×10−46.32\times 10^{-4}3.17×10−33.17\times 10^{-3}2.28×10−32.28\times 10^{-3}[email protected]±0.00860.8855±0.00220.8812±0.00290.8964±0.00040.8888±[email protected]±1.09202.70±1.28262.67±0.88180.57±0.71161.83±[email protected]×10−35.21\times 10^{-3}4.37×10−34.37\times 10^{-3}3.36×10−33.36\times 10^{-3}4.96×10−34.96\times 10^{-3}5.49×10−35.49\times 10^{-3}[email protected]±0.01630.0933±0.01890.1133±0.00940.1200±0.02830.1067±[email protected]±1.0665.52±0.5418.91±0.0860.56±1.1141.84±[email protected]×10−33.02\times 10^{-3}1.42×10−31.42\times 10^{-3}5.99×10−35.99\times 10^{-3}1.98×10−31.98\times 10^{-3}2.55×10−32.55\times 10^{-3}[email protected]±0.02520.5657±0.02180.4343±0.02180.6128±0.01260.5960±[email protected]±3.74372.52±7.55495.57±0.41278.09±3.86264.57±[email protected]×10−31.83\times 10^{-3}1.52×10−31.52\times 10^{-3}8.76×10−48.76\times 10^{-4}2.20×10−32.20\times 10^{-3}2.25×10−32.25\times 10^{-3}[email protected]±0.00190.9636±0.00220.9530±0.00110.9618±0.00140.9441±[email protected]±0.93158.05±2.04193.05±0.13145.81±0.8884.04±[email protected]×10−36.08\times 10^{-3}6.10×10−36.10\times 10^{-3}4.94×10−34.94\times 10^{-3}6.60×10−36.60\times 10^{-3}1.12×10−21.12\times 10^{-2}[email protected]±0.02490.3133±0.00940.1333±0.02490.3000±0.01630.2867±[email protected]±3.16424.78±0.74518.73±0.86401.46±6.58427.52±[email protected]×10−45.98\times 10^{-4}7.38×10−47.38\times 10^{-4}2.57×10−42.57\times 10^{-4}7.47×10−47.47\times 10^{-4}6.71×10−46.71\times 10^{-4}[email protected]±0.00950.6566±0.01650.5623±0.01720.7172±0.01650.7138±[email protected]±2.13342.70±2.41453.33±2.31334.08±1.80300.63±[email protected]×10−32.13\times 10^{-3}1.92×10−31.92\times 10^{-3}1.24×10−31.24\times 10^{-3}2.15×10−32.15\times 10^{-3}2.37×10−32.37\times 10^{-3}Table 5:Effectiveness and efficiency ofIAPOand baselines evaluated with P@8, L@8, and [email protected]@80.8067±0.00190.7923±0.00870.7930±0.00590.7902±0.00530.7928±[email protected]±0.21216.56±1.86242.81±0.22159.81±0.35150.55±[email protected]×10−34.66\times 10^{-3}3.66×10−33.66\times 10^{-3}3.27×10−33.27\times 10^{-3}4.94×10−34.94\times 10^{-3}5.27×10−35.27\times 10^{-3}[email protected]±0.00940.1600±0.02830.1133±0.03400.1400±0.03270.1800±[email protected]±0.2216.56±0.0115.82±0.0417.49±0.1217.36±[email protected]×10−35.67\times 10^{-3}9.66×10−39.66\times 10^{-3}7.16×10−37.16\times 10^{-3}8.00×10−38.00\times 10^{-3}1.04×10−21.04\times 10^{-2}[email protected]±0.01720.4478±0.02380.4108±0.03720.3872±0.01260.4276±[email protected]±1.64162.86±0.74473.62±1.17103.93±0.66165.15±[email protected]×10−32.44\times 10^{-3}2.75×10−32.75\times 10^{-3}8.67×10−48.67\times 10^{-4}3.73×10−33.73\times 10^{-3}2.59×10−32.59\times 10^{-3}[email protected]±0.00410.9232±0.00090.9242±0.00330.9348±0.00160.9282±[email protected]±0.59203.80±0.61262.51±0.38180.73±0.16162.77±[email protected]×10−35.45\times 10^{-3}4.53×10−34.53\times 10^{-3}3.52×10−33.52\times 10^{-3}5.17×10−35.17\times 10^{-3}5.70×10−35.70\times 10^{-3}[email protected]±0.02830.1533±0.00940.1600±0.01630.1733±0.03400.1600±[email protected]±0.6765.56±1.2018.88±0.0360.20±0.5841.76±[email protected]×10−34.11\times 10^{-3}2.34×10−32.34\times 10^{-3}8.47×10−38.47\times 10^{-3}2.88×10−32.88\times 10^{-3}3.83×10−33.83\times 10^{-3}[email protected]±0.00000.6162±0.02180.4714±0.01720.6835±0.01260.6599±[email protected]±2.35375.96±6.42495.66±0.84277.13±2.27265.36±[email protected]×10−32.00\times 10^{-3}1.64×10−31.64\times 10^{-3}9.51×10−49.51\times 10^{-4}2.47×10−32.47\times 10^{-3}2.49×10−32.49\times 10^{-3}[email protected]±0.00120.9742±0.00120.9669±0.00090.9725±0.00250.9623±[email protected]±0.55160.06±1.45193.23±0.02147.33±0.6983.95±[email protected]×10−36.08\times 10^{-3}6.09×10−36.09\times 10^{-3}5.00×10−35.00\times 10^{-3}6.60×10−36.60\times 10^{-3}1.15×10−21.15\times 10^{-2}[email protected]±0.00940.3467±0.02490.1533±0.02490.3533±0.03400.3333±[email protected]±1.63423.35±0.95518.86±0.35398.63±1.65429.34±[email protected]×10−46.88\times 10^{-4}8.19×10−48.19\times 10^{-4}2.95×10−42.95\times 10^{-4}8.86×10−48.86\times 10^{-4}7.76×10−47.76\times 10^{-4}[email protected]±0.01260.7205±0.01260.5892±0.00480.7677±0.00820.7542±[email protected]±1.89344.19±0.51453.86±1.12335.72±1.30301.11±[email protected]×10−32.24\times 10^{-3}2.09×10−32.09\times 10^{-3}1.30×10−31.30\times 10^{-3}2.29×10−32.29\times 10^{-3}2.50×10−32.50\times 10^{-3}

C.2.3Wall-clock Inference Time

We further report the average wall-clock inference time required to generate 32 completions per query on Qwen2.5-7B-Instruct(Team,2024)across all three datasets (Fig.8). Notably, the model post-trained withIAPOachieves a substantially lower inference time—up to a 17.7% reduction—compared to the original model.

Figure 8:Wall-clock LLM inference time before and afterIAPOtraining.

C.3Training Dynamics

Here we present the training dynamics of (i) the mean correctness reward of model completions (with a reward of 1 for correct completions and -1 for incorrect ones), (ii) the mean completion length, and (iii) the ratio between correctness reward and completion length. These dynamics indicate that our method consistently achieves optimal token efficiency throughout the post-training process compared with the state-of-the-art baselines.

Figure 9:The dynamics of correctness reward during the LLM post-training process.Figure 10:The dynamics of completion lengths during the LLM post-training process.Figure 11:The dynamics of the ratio between the mean correctness reward and the mean completion lengths during the LLM post-training process.

C.4Case Study

We examine two research questions in this subsection: (1)Generalizability of Conciseness: We present additional case studies demonstrating that ourIAPOachieves optimal token-efficiency compared with state-of-the-art baselines (see TableLABEL:tab:_extended_case_studies). The samples show that our method exhibits minimal self-commentary, virtually no failed intermediate derivations, and no redundant verifications, indicating thatIAPOachieves token-efficiency improvements by addressing all three factors contributing to verbosity in current RL post-trained LLMs. (2)Informative Token Identification via Conditional MI: We examine whether the conditional MI estimated by our early-exit estimator effectively captures informative tokens for producing correct answers. Specifically, we plot (in Fig.12) token-wise heatmaps for completions generated by Qwen2.5-0.5B-Instruct(Team,2024)on the GSM8K dataset(Cobbeet al.,2021), where color intensity represents the conditional MI value of each token. The conditional MI successfully identifies informative tokens such as key numbers in reasoning and logical conjunctions (e.g., “since”, “proceed”), while assigning low values to redundant tokens such as restatements of givens and meta-commentary.

(a) (b)

Figure 12:Case study on the effectiveness of the conditional MI in identifying informative tokens. Each subplot is a reasoning completion generated by Qwen2.5-0.5B-Instruct on a question from the GSM8K dataset. (Part 1 of 13)(a) (b)

Figure 13:Case study on the effectiveness of the conditional MI in identifying informative tokens. Each subplot is a reasoning completion generated by Qwen2.5-0.5B-Instruct on a question from the GSM8K dataset. (Part 2 of 13)(a) (b)

Figure 14:Case study on the effectiveness of the conditional MI in identifying informative tokens. Each subplot is a reasoning completion generated by Qwen2.5-0.5B-Instruct on a question from the GSM8K dataset. (Part 3 of 13)(a) (b)

Figure 15:Case study on the effectiveness of the conditional MI in identifying informative tokens. Each subplot is a reasoning completion generated by Qwen2.5-0.5B-Instruct on a question from the GSM8K dataset. (Part 4 of 13)(a) (b)

Figure 16:Case study on the effectiveness of the conditional MI in identifying informative tokens. Each subplot is a reasoning completion generated by Qwen2.5-0.5B-Instruct on a question from the GSM8K dataset. (Part 5 of 13)(a) (b)

Figure 17:Case study on the effectiveness of the conditional MI in identifying informative tokens. Each subplot is a reasoning completion generated by Qwen2.5-0.5B-Instruct on a question from the GSM8K dataset. (Part 6 of 13)(a) (b)

Figure 18:Case study on the effectiveness of the conditional MI in identifying informative tokens. Each subplot is a reasoning completion generated by Qwen2.5-0.5B-Instruct on a question from the GSM8K dataset. (Part 7 of 13)(a) (b)

Figure 19:Case study on the effectiveness of the conditional MI in identifying informative tokens. Each subplot is a reasoning completion generated by Qwen2.5-0.5B-Instruct on a question from the GSM8K dataset. (Part 8 of 13)(a) (b)

Figure 20:Case study on the effectiveness of the conditional MI in identifying informative tokens. Each subplot is a reasoning completion generated by Qwen2.5-0.5B-Instruct on a question from the GSM8K dataset. (Part 9 of 13)(a) (b)

Figure 21:Case study on the effectiveness of the conditional MI in identifying informative tokens. Each subplot is a reasoning completion generated by Qwen2.5-0.5B-Instruct on a question from the GSM8K dataset. (Part 10 of 13)(a) (b)

Figure 22:Case study on the effectiveness of the conditional MI in identifying informative tokens. Each subplot is a reasoning completion generated by Qwen2.5-0.5B-Instruct on a question from the GSM8K dataset. (Part 11 of 13)(a) (b)

Figure 23:Case study on the effectiveness of the conditional MI in identifying informative tokens. Each subplot is a reasoning completion generated by Qwen2.5-0.5B-Instruct on a question from the GSM8K dataset. (Part 12 of 13)(a) (b)

Figure 24:Case study on the effectiveness of the conditional MI in identifying informative tokens. Each subplot is a reasoning completion generated by Qwen2.5-0.5B-Instruct on a question from the GSM8K dataset. (Part 13 of 13)Table 6:Comparison of completions across different methodsProblemGFPOGTPOSGRPOLeeDAPOIAPORobe problemFor the robe, it requires 2 bolts of blue fiber. Since the amount of white fiber needed is half that of the blue fiber, we calculate half of 2 bolts, which is 1 bolt. Adding the blue and white fibers together, we get 2 + 1 = 3 bolts in total. Answer: 3To determine the total number of bolts needed for the robe, we first look at the amount of each type of fiber required. The problem states that a robe requires 2 bolts of blue fiber and half that much white fiber. Since half of 2 is 1, the robe will require 1 bolt of white fiber. This means the total number of bolts needed is the sum of the bolts of blue fiber and the bolts of white fiber. So, 2 (blue fiber) + 1 (white fiber) = 3 bolts in total. The robe will need a total of 3 bolts of fiber. Answer: 3First, we calculate how much white fiber is needed, which is half the amount of blue fiber. Since the robe takes 2 bolts of blue fiber, the amount of white fiber needed is 2 / 2 = 1 bolt. The total amount of fiber needed is the sum of the blue and white fibers, so 2 + 1 = 3 bolts. Answer: 3First, we need to determine how much white fiber is required. Since the amount of white fiber is half of the blue fiber, we have 2/2 = 1 bolt of white fiber. Then, we add the bolts of blue fiber and white fiber together: 2 + 1 = 3 bolts in total. Answer: 32 divided by 2 is 1, so the total is 2 + 1 = 3. Answer: 3Duck eggs problemFirst, we calculate the total number of eggs Janet has per day which is 16. Janet consumes 3 eggs for breakfast every day, leaving 16 - 3 = 13 eggs. Then she uses 4 eggs to bake muffins for her friends, leaving 13 - 4 = 9 eggs. Since she sells the remaining eggs at the farmers’ market for $2 each, she makes 9 * $2 = $18 per day from selling eggs. Answer: 18To find out how much money Janet makes every day at the farmers’ market, let’s first determine how many eggs she has left after consuming some for her personal use. Janet’s ducks lay 16 eggs per day. She eats 3 eggs for breakfast, so the number of eggs left after breakfast is 16 - 3 = 13 eggs. She also uses 4 eggs to bake muffins for her friends, so the number of eggs left after baking is 13 - 4 = 9 eggs. These 9 eggs are what she sells at the farmers’ market. Since each egg is sold for $2, the amount of money she makes from selling 9 eggs is 9 * 2 = 18 dollars. Therefore, Janet makes $18 every day at the farmers’ market. Answer: 18Janet’s ducks lay 16 eggs per day. She eats 3 eggs every morning for breakfast, so the number of eggs used for breakfast is 3. She also uses 4 eggs per day to bake muffins for her friends, so the number of eggs used for baking is 4. The total number of eggs used every day is 3 + 4 = 7 eggs. The number of eggs left to sell at the farmers’ market is 16 - 7 = 9 eggs. She sells the remaining eggs at $2 per egg, so the amount she makes from selling the eggs is 9 * 2 = 18 dollars. Answer: 18Janet’s ducks lay 16 eggs per day. She eats 3 eggs every morning for breakfast and uses 4 eggs every day to bake muffins for her friends. In total, she uses 3 + 4 = 7 eggs every day. The number of eggs left to sell at the farmers’ market is 16 - 7 = 9 eggs. She sells each egg for $2, so the amount she makes every day at the farmers’ market is 9 * $2 = $18. Answer: 18Janet gets 16 - 3 - 4 = 9 eggs per day. At $2 per egg, she makes 9 * 2 = 18 dollars. Answer: 18Sprints problemTo find out how many total meters James runs in a week, we first determine how many meters he runs in one session. Since he runs 3 sprints each session and each sprint is 60 meters, he runs 3 * 60 = 180 meters per session. Since he does this 3 times a week, the total meters he runs in a week is 180 * 3 = 540 meters. Answer: 540Let’s break down the problem step by step. First, we need to find out how many meters James runs in a single sprint session. Since each sprint is 60 meters and he does 3 sprints per session, the total distance he runs in one session is 3 * 60 = 180 meters. Next, we need to find out how many meters he runs in a week. Since he does these sprint sessions 3 times a week, the total distance he runs in a week is 180 * 3 = 540 meters. Therefore, James runs 540 meters in a week. Answer: 540First, we calculate the number of meters James runs in one session. He runs 3 sprints and each sprint is 60 meters, so in one session, he runs 3 * 60 = 180 meters. Then we calculate the total for the week. He runs 3 times a week, so the total meters he runs in a week is 180 * 3 = 540 meters. Answer: 540First, we calculate the total distance James runs in one sprint session. Since he runs 60 meters each sprint and does 3 sprints per session, the distance covered in one session is 60 * 3 = 180 meters. Next, since he does 3 sessions a week, the total distance he runs in a week is 180 * 3 = 540 meters. Answer: 540Each sprint is 60 meters, and he does 3 sprints 3 times a week, so 3 * 60 * 3 = 540 meters. Answer: 540Glasses problemTo solve this, we need to calculate the total cost of 16 glasses, where every second glass is at 60% of the price. First, let’s determine the cost of one pair of glasses: the first glass is $5 and the second is 60% of $5, which is $3. The total for one pair is $5 + $3 = $8. Since Kylar is buying 16 glasses, which form 8 pairs, we multiply the cost of one pair by 8: $8 * 8 = $64. Answer: 64To determine the total cost of 16 glasses, we need to consider the pricing scheme. The first glass is priced at $5, and every second glass is priced at 60% of $5, which is $5 * 0.6 = $3. Therefore, the cost of a pair (1 + 1) of glasses is $5 + $3 = $8. Since Kylar is buying 16 glasses, we can imagine the sequence in pairs, and since 16 is an even number, there are exactly 8 pairs (because 16 / 2 = 8). So, for 8 pairs, the total cost is 8 * 8 = 64. Hence, Kylar needs to pay $64 for 16 glasses. Answer: 64First, we need to determine the cost for one pair of glasses (two glasses). The first glass costs $5 and the second glass costs 60% of $5, which is $5 * 0.6 = $3. So the total cost for one pair of glasses is $5 + $3 = $8. Kylar wants to buy 16 glasses, which is 8 pairs of glasses since 16 / 2 = 8. The total cost for 8 pairs of glasses at $8 each is 8 * $8 = $64. Answer: 64First, let’s calculate the cost of one pair of glasses, where the second glass costs 60% of $5. So, the cost of the second glass is 0.60 * $5 = $3. Therefore, the cost of one pair of glasses is $5 + $3 = $8. Since Kylar wants to buy 16 glasses, and he gets a discounted second glass for every pair, he will have 8 pairs of glasses. The total cost will be 8 * $8 = $64. Answer: 64Every second pair of glasses costs 0.6 * 5 = $3. For 16 glasses, we can group them into 8 pairs, so 8 * (5 + 3) = 8 * 8 = 64. Answer: 64Chicken feed problemFirst, we need to calculate how many cups of feed Wendi provides to each chicken in the morning and afternoon. In the morning, she gives 15 cups, and in the afternoon, she gives 25 cups. Since she feeds each chicken 15/20 = 0.75 cups in the morning, and 25/20 = 1.25 cups in the afternoon, the total amount each chicken gets in these two meals is 0.75 + 1.25 = 2 cups. Since each chicken should get 3 cups of feed per day, the final meal must make up the difference: 3 - 2 = 1 cup of feed per chicken. With 20 chickens, Wendi needs to provide 20 * 1 = 20 cups of feed in the final meal. Answer: 20Wendi gives her chickens 15 cups of feed in the morning and 25 cups of feed in the afternoon. First, we need to determine how many cups of feed each chicken gets per day. Since there are three feedings a day and each chicken should get 3 cups of feed per day, the total feed required for all 20 chickens per day is 20 * 3 = 60 cups. In the first two feedings (morning and afternoon), Wendi has already provided 15 + 25 = 40 cups of feed. To find out how much feed is needed for the final feeding, we subtract the amount already given from the total daily requirement: 60 - 40 = 20 cups. Therefore, Wendi needs to give her chickens 20 cups of feed in the final feeding of the day. Answer: 20First, we calculate the total amount of feed given to the chickens in a day. Wendi gives 15 cups in the morning and 25 cups in the afternoon, so the total for these two meals is 15 + 25 = 40 cups. Each chicken is given 3 cups of feed per day. Since there are three meals, the total daily feed needed is 3 * 20 * 3 = 180 cups. We already know the morning and afternoon feed totals, so the feed for the final meal is 180 - 40 = 140 cups. Answer: 140First, we need to calculate the total amount of feed that each chicken consumes daily. Since each chicken gets 3 cups of feed and there are 20 chickens, the daily requirement is 20 * 3 = 60 cups of feed. Wendi gives her chickens 15 cups in the morning and 25 cups in the afternoon, so the total feed given in these two meals is 15 + 25 = 40 cups. To find out how much feed is needed in the final meal, we subtract: 60 - 40 = 20 cups. Answer: 20First, calculate the total daily feed required: 20 * 3 = 60 cups. Then, the remaining feed is 60 - (15 + 25) = 60 - 40 = 20 cups. Answer: 20Lemon tree problemTo determine how many years it will take for Carlos to start earning money from his lemon tree, we first need to calculate the net income each year. Each year, the tree grows 7 lemons, and he sells them for $1.5 each, resulting in a revenue of7×1.5=10.57\times 1.5=10.5dollars. The cost to water and feed the tree each year is $3. Therefore, the net income each year is10.5−3=7.510.5-3=7.5dollars. The initial cost to plant the tree is $90. To find out how many years it will take to recover this cost, we divide:90/7.5=1290/7.5=12. Thus, it will take 12 years. Answer: 12Let’s break down the problem step by step. First, the initial cost of planting the lemon tree is $90. Next, each year, the tree will grow 7 lemons, which Carlos can sell at $1.5 per lemon. The cost to water and feed the tree each year is $3. To determine how many years it will take before Carlos starts earning money, we need to find the point where his total earnings from selling lemons exceed the initial planting cost plus the yearly costs. The profit each year from selling lemons is 7 * 1.5 = 10.5 dollars. Each year, the net profit (after subtracting the yearly cost) is 10.5 - 3 = 7.5 dollars. Now, we need to find out how many years it will take for the total net profit to exceed the initial planting cost of $90. We can set up the equation: 7.5 * n = 90, where n is the number of years. Solving for n, we get n = 90 / 7.5 = 12. Therefore, it will take 12 years for Carlos to start earning money on the lemon tree. Answer: 12The tree costs $90 to plant. Each year the tree produces 7 lemons at $1.5 each, so revenue is7×1.5=10.57\times 1.5=10.5dollars. Annual cost is $3. Net profit per year is10.5−3=7.510.5-3=7.5dollars. Years to break even:90/7.5=1290/7.5=12years. Answer: 12To determine how many years it will take for Carlos to start earning money on the lemon tree, we need to calculate the costs and income over time. Let’s denote the number of years bynn. Annual income from lemons:7×1.5=10.57\times 1.5=10.5dollars. Annual cost: $3. Net annual profit:7.5​n−907.5n-90. We want the smallestnnsuch that net profit is positive:7.5​n−90>07.5n-90>0. Solving:7.5​n>907.5n>90, thusn>12n>12. Sincennmust be a whole number, we round up to 13. Therefore, it will take 13 years. Answer: 13First, we calculate the net profit: 7 * 1.5 - 3 = 10.5 - 3 = 7.5. To break even, we need 90 / 7.5 = 12 years. Answer: 12Sheep problemTo solve this problem, we start by determining the number of sheep in each city. If Seattle has 20 sheep, then Charleston has4×20=804\times 20=80sheep. Toulouse has2×80=1602\times 80=160sheep. Total:20+80+160=26020+80+160=260sheep. Answer: 260Let’s start by figuring out how many sheep each city has based on the given relationships. We know that Seattle has 20 sheep. The problem states that Charleston has 4 times as many sheep as Seattle. Since Seattle has 20 sheep, we can calculate the number of sheep in Charleston as 4 * 20 = 80 sheep. Next, we are told that Toulouse has twice as many sheep as Charleston. Since Charleston has 80 sheep, we can calculate the number of sheep in Toulouse as 2 * 80 = 160 sheep. Now, to find the total number of sheep in Toulouse, Charleston, and Seattle together, we add up the sheep from each city: 160 (Toulouse) + 80 (Charleston) + 20 (Seattle) = 260 sheep. Therefore, the total number of sheep in all three cities is 260. Answer: 260First, calculate Charleston:4×20=804\times 20=80sheep. Next, Toulouse:2×80=1602\times 80=160sheep. Finally, sum:160+80+20=260160+80+20=260sheep. Answer: 260First, Charleston has4×20=804\times 20=80sheep. Next, Toulouse has2×80=1602\times 80=160sheep. Now, we add:160+80+20=260160+80+20=260sheep. Answer: 260Seattle has 20 sheep. Charleston has 4 * 20 = 80 sheep. Toulouse has 2 * 80 = 160 sheep. Therefore, 20 + 80 + 160 = 260. Answer: 260Eliza’s earningsEliza earns40×10=40040\times 10=400dollars for first 40 hours. For 5 overtime hours at10×1.2=1210\times 1.2=12dollars/hour, she earns5×12=605\times 12=60dollars. Total:400+60=460400+60=460dollars. Answer: 460Let’s break down the problem step by step to calculate Eliza’s total earnings. First, we need to determine her pay for the regular hours. Eliza’s regular pay rate is $10 per hour. She worked 45 hours this week, and the first 40 hours are considered regular hours, so for the first 40 hours, her pay is 40 * 10 = 400 dollars. Next, we calculate her overtime pay. Any hours worked beyond 40 are considered overtime, and the overtime pay rate is 1.2 times the regular rate. The overtime rate is therefore 10 * 1.2 = 12 dollars per hour. She worked 45 - 40 = 5 overtime hours. The pay for these 5 overtime hours is 5 * 12 = 60 dollars. Finally, we add her regular pay and overtime pay to find her total earnings: 400 + 60 = 460 dollars. Therefore, Eliza’s total earnings for this week are 460 dollars. Answer: 460Eliza worked 45 hours. Regular pay for first 40 hours:40×10=40040\times 10=400dollars. Remaining 5 hours are overtime at1.2×10=121.2\times 10=12dollars/hour. Overtime earnings:5×12=605\times 12=60dollars. Total:400+60=460400+60=460dollars. Answer: 460Eliza’s regular pay:40×10=40040\times 10=400dollars. Overtime hours:45−40=545-40=5hours. Overtime rate:1.2×10=121.2\times 10=12dollars/hour. Overtime pay:5×12=605\times 12=60dollars. Total:400+60=460400+60=460dollars. Answer: 460Eliza’s regular pay is 40 * $10 = $400. For the 5 hours of overtime, her pay is 5 * (10 * 1.2) = $60. Thus, her total earnings are $400 + $60 = $460. Answer: 460PastriesCost of 3 dozen donuts:3×68=2043\times 68=204dollars. Cost of 2 dozen mini cupcakes:2×80=1602\times 80=160dollars. Cost of 6 dozen mini cheesecakes:6×55=3306\times 55=330dollars. Total:204+160+330=694204+160+330=694dollars. Answer: 694To find the total cost of the items Toula bought, we need to calculate the cost for each type of pastry and then sum these costs. First, we calculate the cost of the donuts. Toula bought 3 dozen donuts, and each dozen costs $68. Therefore, the cost of the donuts is 3 dozen * $68 per dozen = 3 * 68 = $204. Next, we calculate the cost of the mini cupcakes. Toula bought 2 dozen mini cupcakes, and each dozen costs $80. Thus, the cost of the mini cupcakes is 2 dozen * $80 per dozen = 2 * 80 = $160. Finally, we calculate the cost of the mini cheesecakes. Toula bought 6 dozen mini cheesecakes, and each dozen costs $55. Therefore, the cost of the mini cheesecakes is 6 dozen * $55 per dozen = 6 * 55 = $330. To find the total cost, we add these amounts together: $204 + $160 + $330 = $694. Therefore, the total cost of the pastries is $694. Answer: 694First, donuts:3×68=2043\times 68=204dollars. Next, mini cupcakes:2×80=1602\times 80=160dollars. Then, mini cheesecakes:6×55=3306\times 55=330dollars. Total:204+160+330=694204+160+330=694dollars. Answer: 694First, donuts:3×68=2043\times 68=204dollars. Next, mini cupcakes:2×80=1602\times 80=160dollars. Then, mini cheesecakes:6×55=3306\times 55=330dollars. Total:204+160+330=694204+160+330=694dollars. Answer: 694Calculating the cost for each item: 3 * 68 = 204, 2 * 80 = 160, and 6 * 55 = 330. The total cost is 204 + 160 + 330 = 694. Answer: 694DownloadsFirst month: 60 downloads. Second month:3×60=1803\times 60=180downloads. Third month reduced by 30%:0.30×180=540.30\times 180=54, so180−54=126180-54=126downloads. Total:60+180+126=36660+180+126=366downloads. Answer: 366Let’s break down the problem step by step. First, we know the number of downloads in the first month is 60. In the second month, the number of downloads is three times the first month’s downloads. So, we calculate the downloads for the second month as 3 * 60 = 180. In the third month, the downloads are reduced by 30%, meaning 70% of the second month’s downloads. To find 70% of 180, we calculate 0.70 * 180 = 126. Therefore, the third month’s downloads are 126. To find the total downloads over the three months, we sum the downloads from each month: 60 + 180 + 126. Adding these together, we get 60 + 180 + 126 = 366. Therefore, the total downloads over the three months are 366. Answer: 366First month: 60. Second month:3×60=1803\times 60=180. Third month:180−(30/100×180)=180−54=126180-(30/100\times 180)=180-54=126. Total:60+180+126=36660+180+126=366. Answer: 366First month: 60 downloads. Second month:3×60=1803\times 60=180downloads. Third month 30% reduction:0.30×180=540.30\times 180=54downloads, so180−54=126180-54=126downloads. Total:60+180+126=36660+180+126=366downloads. Answer: 366In the second month, 60 * 3 = 180. In the third month, 180 - (180 * 0.3) = 180 - 54 = 126. Total downloads = 60 + 180 + 126 = 366. Answer: 366Dance classContemporary dance:0.20×20=40.20\times 20=4students. Remaining:20−4=1620-4=16students. Jazz dance:0.25×16=40.25\times 16=4students. Hip-hop:16−4=1216-4=12students. Percentage:(12/20)×100=60%(12/20)\times 100=60\%. Answer: 60To find the percentage of students in the hip-hop dance class, let’s start by determining how many students are in each type of dance class. First, 20% of the 20 students signed up for contemporary dance. To calculate this, we multiply 20 by 20%, which is 20 * 0.20 = 4 students. This means 4 students are in the contemporary dance class. After signing up 4 students for contemporary dance, the number of students remaining is 20 - 4 = 16 students. Next, 25% of the remaining 16 students signed up for jazz dance. To calculate this, we multiply 16 by 25%, which is 16 * 0.25 = 4 students. This means 4 students are in the jazz dance class. After signing up 4 students for jazz dance, the number of students remaining for hip-hop is 16 - 4 = 12 students. To find the percentage of the total students who signed up for hip-hop, we divide the number of hip-hop students by the total number of students and multiply by 100. This is (12 / 20) * 100 = 60%. Therefore, 60% of the students signed up for hip-hop dance. Answer: 60Contemporary dance:20×0.2=420\times 0.2=4students. Remaining:20−4=1620-4=16students. Jazz dance:16×0.25=416\times 0.25=4students. Remaining:16−4=1216-4=12students (hip-hop). Percentage:(12/20)×100=60%(12/20)\times 100=60\%. Answer: 60Contemporary dance:(20/100)×20=4(20/100)\times 20=4students. Remaining:20−4=1620-4=16students. Jazz dance:(25/100)×16=4(25/100)\times 16=4students. Hip-hop:16−4=1216-4=12students. Percentage:(12/20)×100=60%(12/20)\times 100=60\%. Answer: 60First, 20% of 20 students = 4, so 20 - 4 = 16. Then, 25% of 16 = 4, so 16 - 4 = 12. Thus, 12 out of 20 students are in hip-hop, which is 12/20 = 60%. Answer: 60File download40% of 200 GB is 80 GB. At 2 GB/minute, this takes80/2=4080/2=40minutes. After restart (20 minutes), she downloads full 200 GB:200/2=100200/2=100minutes. Total:40+20+100=16040+20+100=160minutes. Answer: 160Carla is downloading a 200 GB file at a rate of 2 GB per minute. This means that the total download time without interruptions would be 200 GB / 2 GB/minute = 100 minutes. However, she encounters an interruption 40% into the download. To find out how much data she has downloaded by 40%, we calculate 40% of 200 GB: 200 * 0.4 = 80 GB. Since she is downloading at a rate of 2 GB per minute, it would take her 80 GB / 2 GB/minute = 40 minutes to download 80 GB. At this point, Windows forces a restart, and the download process restarts from the beginning. After the restart, the total data to be downloaded is 200 GB again, and the download process has to start from the beginning. During the restart, it takes 20 minutes to download 80 GB. But since she is already at 40 minutes and the Windows restart takes 20 minutes, the download restarts and then needs to complete the remaining 120 GB (200 GB - 80 GB). The rate of download is still 2 GB/minute, so it takes her 120 GB / 2 GB/minute = 60 minutes to download the remaining 120 GB. Therefore, the total time taken to download the file is the sum of the initial 40 minutes, the restart time of 20 minutes, and the restart download time of 60 minutes, which is 40 + 20 + 60 = 120 minutes. So, the total time taken to download the 200 GB file is 120 minutes. Answer: 120File size: 200 GB. 40% is200×0.4=80200\times 0.4=80GB. Time to download 80 GB:80/2=4080/2=40minutes. Restart takes 20 minutes. Total until restart:40+20=6040+20=60minutes. Then she restarts from beginning, downloading full 200 GB:200/2=100200/2=100minutes. Total:40+20+100=16040+20+100=160minutes. Answer: 16040% of 200 GB is0.4×200=800.4\times 200=80GB. Time:80/2=4080/2=40minutes. After restart, she downloads full 200 GB:200/2=100200/2=100minutes. Plus 20 minutes for restart. Total:40+20+100=16040+20+100=160minutes. Answer: 16040% of 200 GB is 200 * 0.4 = 80 GB. 80 GB / 2 = 40 minutes. The remaining 200 - 80 = 120 GB takes 120 / 2 = 60 minutes. The total time is 40 + 20 + 60 = 120 minutes. Answer: 120DrivingJohn drives3×60=1803\times 60=180miles away. Returning: 2 hours in traffic (0 miles), 0.5 hours at 30 mph (0.5×30=150.5\times 30=15miles), remaining time4−2−0.5=1.54-2-0.5=1.5hours at 80 mph (1.5×80=1201.5\times 80=120miles). Total return:15+120=13515+120=135miles. Distance from home:180−135=45180-135=45miles. Answer: 45John drives at a speed of 60 mph for 3 hours, so the distance he drives away from home is 60 * 3 = 180 miles. On the return trip, John spends 2 hours in standstill traffic, which means he doesn’t cover any distance during this time. After the traffic, he drives for the remaining time at different speeds. The total time for the return trip is 4 hours. Since he spent 2 hours in traffic, he has 4 - 2 = 2 hours left. During the first part of the remaining time, John drives at 30 mph for half an hour (0.5 hours). The distance covered during this time is 30 * 0.5 = 15 miles. For the rest of the return trip, he drives at 80 mph. The remaining time after driving at 30 mph is 2 - 0.5 = 1.5 hours. The distance covered during this time is 80 * 1.5 = 120 miles. Therefore, the total distance John covers on the return trip is 0 (from the 2 hours in traffic) + 15 (from driving at 30 mph) + 120 (from driving at 80 mph) = 135 miles. Since John drove 180 miles away from home, and he covered 135 miles on the return trip, the distance he is from home is 180 - 135 = 45 miles. Answer: 45Initial distance:3×60=1803\times 60=180miles. After traffic and 0.5 hours at 30 mph (0.5×30=150.5\times 30=15miles), he’s180−15=165180-15=165miles away. Remaining time:4−2−0.5=1.54-2-0.5=1.5hours at 80 mph:80×1.5=12080\times 1.5=120miles. Total return:15+120=13515+120=135miles. Distance from home:180−135=45180-135=45miles. Answer: 45Initial distance:3×60=1803\times 60=180miles. Return: 2 hours traffic (0 miles), 0.5 hours at 30 mph (0.5×30=150.5\times 30=15miles), 1.5 hours at 80 mph (1.5×80=1201.5\times 80=120miles). Total return:15+120=13515+120=135miles. Distance from home:180−135=45180-135=45miles. Answer: 45First, John drives 3 * 60 = 180 miles. Turning around, he spends 2 * 0 = 0 miles (in standstill traffic), then 0.5 * 30 = 15 miles, and finally 3.5 * 80 = 280 miles. So, the total distance from home is 180 + 15 + 280 = 475 miles. Answer: 475House flippingPurchase: $80,000. Repairs: $50,000. Total investment: $130,000. Value increase:80,000×1.5=120,00080,000\times 1.5=120,000dollars. New value:80,000+120,000=200,00080,000+120,000=200,000dollars. Profit:200,000−130,000=70,000200,000-130,000=70,000dollars. Answer: 70000Josh initially buys a house for $80,000 and then spends an additional $50,000 on repairs. The total cost for Josh is the sum of the house price and the repair cost. So, let’s calculate the total cost: 80,000 + 50,000 = 130,000 dollars. Next, we need to find the gain in value from the repairs. The repair cost of $50,000 increased the value of the house by 150%. To find out the new value of the house after the 150% increase in value, we first understand that a 150% increase means the value is tripled (since 100% + 150% = 250% of the repair cost, or 2.5 times the repair cost). Thus, the increase in value due to repairs is 50,000 * 1.5 = 75,000 dollars. Adding this increase to the original house purchase price of $80,000, the new value of the house is 80,000 + 75,000 = 155,000 dollars. Josh needs to sell the house to make a profit. The profit is calculated by subtracting the total cost from the new value of the house: 155,000 - 130,000 = 25,000 dollars. Therefore, Josh made a profit of $25,000. Answer: 25000Total spent:80,000+50,000=130,00080,000+50,000=130,000dollars. Value increase:80,000×1.5=120,00080,000\times 1.5=120,000dollars. New value:80,000+120,000=200,00080,000+120,000=200,000dollars. Profit:200,000−130,000=70,000200,000-130,000=70,000dollars. Answer: 70000Initial cost before repairs: $80,000. Repairs: $50,000. Total cost: $130,000. Increase of 150% means value goes up by1.5×130,000=195,0001.5\times 130,000=195,000dollars. New value:130,000+195,000=325,000130,000+195,000=325,000dollars. Profit:325,000−130,000=195,000325,000-130,000=195,000dollars. Answer: 195000First, calculate the increased value: $80,000 * 1.5 = $120,000. The new value of the house is $80,000 + $50,000 = $130,000. The profit is 130,000 \- \(80,000 + $50,000) = $130,000 - $130,000 = $0. Answer: 0Vacuum cleanersLetxxbe total vacuum cleaners. Soldx/3x/3at green house. Remaining:2​x/32x/3. Sold 2 more at red house:2​x/3−22x/3-2left. Sold half at orange house, leaving:(1/2)​(2​x/3−2)=5(1/2)(2x/3-2)=5. Solving:2​x/3−2=102x/3-2=10,2​x/3=122x/3=12,x=18x=18. Answer: 18Melanie sold some of her vacuum cleaners and has 5 left. Let’s break down her sales process step-by-step. Suppose Melanie started with x vacuum cleaners. First, she sold a third of the vacuum cleaners at the green house, which means she sold x/3 vacuum cleaners there. After selling at the green house, she had x - x/3 = 2x/3 vacuum cleaners left. Next, she sold 2 more to the red house, so she then had 2x/3 - 2 vacuum cleaners left. Finally, she sold half of what was left at the orange house, which means she sold (1/2)(2x/3 - 2) vacuum cleaners there. Therefore, the number of vacuum cleaners she has left after the orange house sale is (1/2)(2x/3 - 2). We know that after the sales at the green, red, and orange houses, she has 5 vacuum cleaners left. So, we can set up the equation: (1/2)(2x/3 - 2) = 5. To solve for x, we can multiply both sides by 2 to clear the fraction: 2x/3 - 2 = 10. Next, we add 2 to both sides to isolate the term with x: 2x/3 = 12. Then, we multiply both sides by 3/2 to solve for x: x = 12 * 3/2 = 18. So, Melanie started with 18 vacuum cleaners. We can verify this: if she started with 18, after selling a third at the green house, she sold 18/3 = 6 vacuum cleaners, leaving her with 12. Then, she sold 2 more to the red house, leaving her with 12 - 2 = 10. Lastly, she sold half of the 10 left at the orange house, which is 5, leaving her with 5. This confirms our solution. Therefore, Melanie started with 18 vacuum cleaners. Answer: 18Letxxbe total. Soldx/3x/3at green house. After green house:2​x/32x/3left. After red house:2​x/3−22x/3-2left. Sold half at orange house:(1/2)​(2​x/3−2)(1/2)(2x/3-2). After orange house:2​x/3−2−(1/2)​(2​x/3−2)=52x/3-2-(1/2)(2x/3-2)=5. Simplifying:2​x/3−2−(2​x/6−1)=52x/3-2-(2x/6-1)=5,2​x/3−2−x/3+1=52x/3-2-x/3+1=5,x/3−1=5x/3-1=5,x/3=6x/3=6,x=18x=18. Answer: 18Letxxbe total. Soldx/3x/3at green house. After green:2​x/32x/3left. After red:2​x/3−22x/3-2left. Sold half at orange:(1/2)​(2​x/3−2)=x/3−1(1/2)(2x/3-2)=x/3-1. After orange:2​x/3−2−(x/3−1)=x/3−1=52x/3-2-(x/3-1)=x/3-1=5. Solving:x/3=6x/3=6,x=18x=18. Answer: 18Let’s denote the total number of vacuum cleaners byxx. She soldx3\frac{x}{3}at the green house, and then 2 more to the red house, leaving2​x3−2\frac{2x}{3}-2. The remaining12​(2​x3−2)=5\frac{1}{2}\left(\frac{2x}{3}-2\right)=5, simplifying tox3−1=5\frac{x}{3}-1=5. So,x3=6\frac{x}{3}=6andx=18x=18. Answer: 18