Dynamic Important Example Mining for Reinforcement Finetuning
Summary
DIEM introduces a dynamic framework for reinforcement fine-tuning that adaptively selects and reweights training examples to enhance policy improvement and stabilize optimization.
View Cached Full Text
Cached at: 09/01/26, 11:48 AM
Paper page - Dynamic Important Example Mining for Reinforcement Finetuning
Source: https://huggingface.co/papers/2608.29252 Authors:
,
,
,
,
,
,
,
,
,
,
Abstract
DIEM adaptively selects and reweights training examples during reinforcement fine-tuning by estimating each sample’s marginal contribution to policy improvement and constraining batch updates to stabilize optimization.
Reinforcement fine-tuning(RFT) is increasingly used to strengthen the reasoning abilities of large models, yet its effectiveness is bound by how training data are selected and used. Most data-centric RFT methods rely on static or heuristic sample selection, implicitly assuming a sample’s value is fixed over training. This overlooks thenon-stationary dynamicsof policy learning and can lead to suboptimal updates. We propose Dynamic Important Example Mining (DIEM), a principled and fully automated framework that makes data utilization adaptive throughout RFT. DIEM integrates two components into each optimization step: (i) agradient-alignment importance estimatorthat efficiently approximates each sample’s marginal contribution topolicy improvement; and (ii) aconstrained batch reweightingscheme that maximizes aggregate utility while preserving the update’s gradient magnitude to stabilize optimization. Across several reasoning benchmarks, DIEM consistently outperforms strong static and dynamic baselines. The code will be released via https://github.com/hrtan/DIEM.
View arXiv pageView PDFGitHub0Add to collection
Get this paper in your agent:
hf papers read 2608\.29252
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2608.29252 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2608.29252 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2608.29252 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Diff Mining: Logit Differences Reveal Finetuning Objectives
The paper introduces Diff Mining, a framework for identifying finetuning objectives in language models by analyzing logit differences between finetuned and base models, enabling interpretable auditing of learned behaviors.
Dynamic Sampling that Adapts: Self-Aware Iterative Data Persistent Optimization for Mathematical Reasoning
SAI-DPO introduces a dynamic sampling framework that adapts training data to a model's evolving capabilities during mathematical reasoning tasks, using self-aware difficulty metrics and knowledge semantic alignment to achieve state-of-the-art efficiency with less data on benchmarks like AIME24 and AMC23.
AEM: Adaptive Entropy Modulation for Multi-Turn Agentic Reinforcement Learning
This paper introduces AEM, a supervision-free method for agentic reinforcement learning that adapts entropy dynamics at the response level to improve exploration-exploitation trade-offs. It demonstrates performance gains on benchmarks like ALFWorld and SWE-bench by aligning uncertainty estimation with action granularity.
Evolution Fine-Tuning: Learning to Discover Across 371 Optimization Tasks
This paper introduces Evolution Fine-Tuning (EFT), a mid-training paradigm that teaches LLMs to evolve solutions across optimization tasks by converting evolutionary search trajectories into supervision. The authors construct the Finch Collection dataset of 156K trajectories across 371 tasks and show that fine-tuned models generalize to held-out tasks and match state-of-the-art performance on several benchmarks.
DRIFT: Decoupled Rollouts and Importance-Weighted Fine-Tuning for Efficient Multi-Turn Optimization
This paper proposes DRIFT, a framework that combines offline trajectories with importance-weighted supervised fine-tuning to efficiently achieve multi-turn interactive learning performance comparable to reinforcement learning.