DRIFT: Refining Instruction Data via On-Policy Data Attribution

# Refining Instruction Data via On-Policy Data Attribution
Source: [https://arxiv.org/html/2606.18307](https://arxiv.org/html/2606.18307)
Zefan Wang Lincheng Li Tianyu Yu Yuan Yao Tsinghua University zefan\.wang\.thu@gmail\.comyaoyuanthu@gmail\.com

###### Abstract

Optimizing the training data distribution for Supervised Fine\-Tuning \(SFT\) dictates the capability of Large Language Models \(LLMs\)\. While existing data curation methods excel at accelerating training under constrained budgets, they are less suited to elevating the capability upper bound\. The challenge here is no longer to identify a smaller subset that preserves performance, but to refine the data distribution toward instances most capable of improving the final model\. To address this problem, we explore instance\-level data attribution using Influence Functions \(IF\)\. We identify that standard IF formulations struggle in this setting due to two structural limitations: a proximity gap caused by off\-policy validation targets, and a severe bias towards gradient norm\. We propose DRIFT \(DataRefinement via On\-PolicyInfluenceFunctions for Supervised Fine\-Tuning\)\. Instead of relying on external reference data, DRIFT utilizes the model’s on\-policy rollouts as validation targets, which empirically minimizes the parameter proximity gap and better aligns with the local neighborhood assumption of IF\. It further applies signed weighting based on trajectory correctness and debiases influence scores against the gradient hacking issue, allowing a small set of validation queries to act as reliable anchors for attributing the full dataset\. Experiments on 7B\-parameter instruction and reasoning models show that DRIFT consistently raises the performance ceiling on both, outperforming existing data curation baselines\.

## 1Introduction

Supervised Fine\-Tuning \(SFT\) is a pivotal stage in aligning Large Language Models \(LLMs\) with human instructions and enhancing their downstream task abilities\(Ouyanget al\.,[2022](https://arxiv.org/html/2606.18307#bib.bib15); Weiet al\.,[2022](https://arxiv.org/html/2606.18307#bib.bib16)\)\. During this phase, the data mixture and the proportion of different training samples play a critical role in shaping model capabilities\(Zhouet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib17); Yeet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib18)\)\. Despite its significance, the precise relationship between data mixing strategies and the ultimate performance of LLMs remains largely opaque\(Liet al\.,[2025](https://arxiv.org/html/2606.18307#bib.bib19)\)\.

Existing research on data curation primarily falls into three paradigms\. First, domain\-level data mixture\(Xieet al\.,[2023a](https://arxiv.org/html/2606.18307#bib.bib9); Liuet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib10); Penget al\.,[2020](https://arxiv.org/html/2606.18307#bib.bib11)\)adjusts sampling weights across broad categories but remains inherently coarse\-grained, failing to account for vast quality disparities at the instance level\. Second, data selection\(Wettiget al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib4); Xiaet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib6); Ivisonet al\.,[2025](https://arxiv.org/html/2606.18307#bib.bib7); Xieet al\.,[2023b](https://arxiv.org/html/2606.18307#bib.bib3)\)identifies small subsets to improve performance under constrained computational budgets\. Because SFT is generally not a resource\-constrained scenario compared to pre\-training, this efficiency\-first focus is less optimized for elevating the performance upper bound of a model after exhaustive training\. Third, online batching\(Wanget al\.,[2024a](https://arxiv.org/html/2606.18307#bib.bib12); Albalaket al\.,[2023](https://arxiv.org/html/2606.18307#bib.bib14)\)dynamically constructs data mixtures during the training process\. While conceptually appealing, frequent re\-evaluation of data weights disrupts highly optimized SFT infrastructures\(Shoeybiet al\.,[2019](https://arxiv.org/html/2606.18307#bib.bib13)\)\.

To address these limitations, we shift the focus from budget\-constrained training to capability maximization through instance\-level data attribution\. Unlike previous methods\(Wettiget al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib4); Xiaet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib6)\)that select subsets to improve training efficiency, we tackle a practical yet underexplored scenario:refining an already fully\-trained modelwith the original training dataset as the candidate pool\. In this setting, the model has already absorbed the general data distribution, causing conventional data selection methods to yield only marginal gains\. Breaking this saturation requires recalibrating the data distribution so that optimization emphasizes instances most capable of further improving the current model, rather than identifying samples that are efficient to learn\.

Implementing fine\-grained attribution in a saturated model calls for a more robust and principled scoring mechanism to evaluate the contribution of each training sample\(Pruthiet al\.,[2020](https://arxiv.org/html/2606.18307#bib.bib20); Ilyaset al\.,[2022](https://arxiv.org/html/2606.18307#bib.bib21)\)\. Influence Functions \(IF\)\(Koh and Liang,[2017](https://arxiv.org/html/2606.18307#bib.bib22); Grosseet al\.,[2023](https://arxiv.org/html/2606.18307#bib.bib23)\)provide a mathematically rigorous tool for this purpose, explicitly estimating the effect of up\-weighting a training sample on a specific validation set\. Yet, standard IF formulations prove fragile when applied to LLMs\. Theoretical analyses\(Baeet al\.,[2022](https://arxiv.org/html/2606.18307#bib.bib2)\)reveal that IF approximations hold only within a microscopic local neighborhood of the converged parameters\. However, when the validation target is an external reference response, reducing its loss can require substantial global parameter shifts, creating aproximity gapthat risks violating the local Taylor approximation\. Drawing on recent findings in on\-policy tuning\(Mukherjeeet al\.,[2025](https://arxiv.org/html/2606.18307#bib.bib71)\), we hypothesize that utilizing the model’s own generated responses as validation targets induces localized parameter updates, which better preserve the theoretical integrity of IF\.

Building upon this insight, we proposeDRIFT\(DataRefinement via On\-PolicyInfluenceFunctions for Supervised Fine\-Tuning\)\. DRIFT prompts the base model to generate responses to validation queries as IF optimization targets, bridging the gap between data attribution and the on\-policy learning paradigm\. While direct on\-policy fine\-tuning, such as self\-distillation, is inherently data\-hungry and prone to overfitting when queries are scarce\(Singhet al\.,[2023](https://arxiv.org/html/2606.18307#bib.bib76); Shumailovet al\.,[2023](https://arxiv.org/html/2606.18307#bib.bib75)\), DRIFT circumvents this limitation by using rollouts strictly as attribution anchors rather than direct training signals\. By assigning positive weights to correct trajectories and negative weights to incorrect ones, we shift the IF optimization objective to a reward\-weighted contrastive formulation, akin to a simplified policy gradient estimator\. This allows a limited set of validation queries to robustly attribute the entire SFT dataset, reinforcing beneficial patterns while penalizing undesirable ones\.

Beyond the choice of validation targets, a second structural vulnerability of IF in LLMs lies in the susceptibility to gradient norm bias\. Raw influence scores are heavily distorted by the magnitude of the gradients, causing the attribution to spuriously favor samples with abnormally large gradient norms rather than samples genuinely useful for the validation objective\. To explicitly mitigate this issue, DRIFT incorporates a task\-specific orthogonalization debiasing step, making the final scores less dominated by gradient magnitude\.

Extensive experiments on 7B\-parameter instruction and reasoning\-focused LLMs demonstrate the effectiveness of our approach\. Under the rigorous setting of refining fully\-trained models, DRIFT consistently improves over existing data curation baselines\. Overall, our results suggest that on\-policy validation targets together with proper gradient debiasing are important for making IF data attribution principled and robust\.

## 2Preliminaries

### 2\.1Influence Functions and Practical Approximations for LLMs

Influence Functions \(IF\) provide a mathematically rigorous framework to estimate the effect of a training samplezt​r​a​i​nz\_\{train\}on a specific validation targetzv​a​lz\_\{val\}without retraining the model\. Letθ∗\\theta^\{\*\}be the parameters of a model trained to minimize the empirical risk1n​∑i=1nL​\(zi;θ\)\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}L\(z\_\{i\};\\theta\)\. Standard IF approximates the change in the validation lossL​\(zv​a​l;θ∗\)L\(z\_\{val\};\\theta^\{\*\}\)ifzt​r​a​i​nz\_\{train\}were up\-weighted by an infinitesimal amountϵ\\epsilon\. This influence score is given by:

ℐ​\(zt​r​a​i​n,zv​a​l\)=−∇θL​\(zv​a​l;θ∗\)⊤​Hθ∗−1​∇θL​\(zt​r​a​i​n;θ∗\),\\mathcal\{I\}\(z\_\{train\},z\_\{val\}\)=\-\\nabla\_\{\\theta\}L\(z\_\{val\};\\theta^\{\*\}\)^\{\\top\}H\_\{\\theta^\{\*\}\}^\{\-1\}\\nabla\_\{\\theta\}L\(z\_\{train\};\\theta^\{\*\}\),\(1\)whereHθ∗=1n​∑i=1n∇θ2L​\(zi;θ∗\)H\_\{\\theta^\{\*\}\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\nabla^\{2\}\_\{\\theta\}L\(z\_\{i\};\\theta^\{\*\}\)is the Hessian matrix of the empirical risk\.

To render data attribution computationally tractable at the LLM scale, the inverse Hessian computationHθ∗−1H\_\{\\theta^\{\*\}\}^\{\-1\}is typically approximated\(Grosseet al\.,[2023](https://arxiv.org/html/2606.18307#bib.bib23); Choeet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib72)\)or even bypassed\(Zhuet al\.,[2025](https://arxiv.org/html/2606.18307#bib.bib61); Xiaet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib6)\), reducing the estimation to the inner product of first\-order gradients\. Furthermore, to overcome the prohibitive memory costs of storing dense gradients for billions of parameters, gradient sparsification and random projection are widely adopted\(Parket al\.,[2023](https://arxiv.org/html/2606.18307#bib.bib41); Muhamedet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib8)\)\. Specifically, the high\-dimensional gradient∇θL​\(z;θ∗\)∈ℝD\\nabla\_\{\\theta\}L\(z;\\theta^\{\*\}\)\\in\\mathbb\{R\}^\{D\}is mapped into a lower\-dimensional spaceℝd\\mathbb\{R\}^\{d\}\(d≪Dd\\ll D\) using a sparse random projection matrixP∈ℝd×DP\\in\\mathbb\{R\}^\{d\\times D\}\. The theoretical validity of this operation is grounded in the Johnson\-Lindenstrauss lemma\(William and Lindenstrauss,[1984](https://arxiv.org/html/2606.18307#bib.bib62)\), showing that the inner product between any two gradients is largely preserved\. Letϕ​\(z;θ∗\)=P​∇θL​\(z;θ∗\)\\phi\(z;\\theta^\{\*\}\)=P\\nabla\_\{\\theta\}L\(z;\\theta^\{\*\}\)denote the projected gradient\. The approximated IF is computed as:

ℐ​\(zt​r​a​i​n,zv​a​l\)≈ϕ​\(zt​r​a​i​n;θ∗\)⊤​ϕ​\(zv​a​l;θ∗\)\.\\mathcal\{I\}\(z\_\{train\},z\_\{val\}\)\\approx\\phi\(z\_\{train\};\\theta^\{\*\}\)^\{\\top\}\\phi\(z\_\{val\};\\theta^\{\*\}\)\.\(2\)
##### Gradient Hacking Issue\.

Relying on the raw inner product exposes the attribution to a severe inherent gradient norm bias, where training samples with abnormally high gradient norms can dominate the IF scores regardless of their actual usefulness for the validation target\. For instance, the estimation often disproportionately favors samples with shorter responses\(Xiaet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib6)\)or outlier samples\(Muhamedet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib8)\)\. While the necessity of gradient normalization has been explored\(Xiaet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib6)\), the specific strategy remains inconclusive\.

### 2\.2The Fragility of Influence Functions

Despite its theoretical elegance, IF is fundamentally a first\-order Taylor approximation of the Leave\-One\-Out \(LOO\) retraining loss\. Letθ−t​r​a​i​n\\theta\_\{\-train\}denote the optimal parameters obtained by retraining the model without a specific training samplezt​r​a​i​nz\_\{train\}, and the true LOO effect on a validation targetzv​a​lz\_\{val\}isΔ​LL​O​O=L​\(zv​a​l;θ−t​r​a​i​n\)−L​\(zv​a​l;θ∗\)\\Delta L\_\{LOO\}=L\(z\_\{val\};\\theta\_\{\-train\}\)\-L\(z\_\{val\};\\theta^\{\*\}\)\. Existing studies\(Baeet al\.,[2022](https://arxiv.org/html/2606.18307#bib.bib2)\)analyzing the error between IF estimations andΔ​LL​O​O\\Delta L\_\{LOO\}reveal that IF actually approximates a different objective: the Proximal Bregman Response Function \(PBRF\)\. Unlike LOO, which allows unconstrained global updates to reachθ−t​r​a​i​n\\theta\_\{\-train\}, PBRF restricts the parameter shift to a local neighborhood aroundθ∗\\theta^\{\*\}:

θP​B​R​F=arg⁡minθ⁡\(L​\(zv​a​l;θ\)\+12​τ​‖θ−θ∗‖Hθ∗2\),\\theta\_\{PBRF\}=\\arg\\min\_\{\\theta\}\\left\(L\(z\_\{val\};\\theta\)\+\\frac\{1\}\{2\\tau\}\\\|\\theta\-\\theta^\{\*\}\\\|\_\{H\_\{\\theta^\{\*\}\}\}^\{2\}\\right\),\(3\)
![Refer to caption](https://arxiv.org/html/2606.18307v1/fig/proximity_gap_combined.png)Figure 1:Evolution of model proximity during continual training\. The plots illustrate the relative changes \(Δ\\Delta\) in parameter L2 norm and sparsity compared to the uniform weighting baseline\. On\-policy validation causes significantly fewer parameter shifts, which we hypothesize better aligns with the local neighborhood assumption of IF\.whereτ\\taucontrols the neighborhood size\. This framework identifies several critical gaps that render IF fragile in deep neural networks\.

#### 2\.2\.1The Non\-Convergence Gap\.

The theoretical derivation of IF strictly assumes that the model has converged to a local minimum, where the empirical gradientg=1n​∑i=1n∇θL​\(zi;θ∗\)=0g=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\nabla\_\{\\theta\}L\(z\_\{i\};\\theta^\{\*\}\)=0\. Wheng≠0g\\neq 0, the Taylor expansion aroundθ∗\\theta^\{\*\}introduces a residual error term in the parameter shift estimation that scales proportionally with‖Hθ∗−1​g‖\\\|H\_\{\\theta^\{\*\}\}^\{\-1\}g\\\|\. In practice, applying IF to partially trained or unconverged models leads to erratic estimations because the non\-zero empirical gradient dominates the intended LOO effect\. This theoretical constraint naturally aligns with our paradigm of refining an already fully\-trained SFT model\. By operating on a converged model rather than training from scratch, we avoid prohibitive retraining costs and bound the non\-convergence gap, better satisfying the theoretical prerequisites of IF\.

#### 2\.2\.2The Proximity Gap and On\-Policy Mitigation\.

The discrepancy between the global optimumθ−t​r​a​i​n\\theta\_\{\-train\}and the local optimumθP​B​R​F\\theta\_\{PBRF\}is defined as theproximity gap\. The validity of the quadratic approximation in PBRF strictly requires the parameter shift‖θ−t​r​a​i​n−θ∗‖\\\|\\theta\_\{\-train\}\-\\theta^\{\*\}\\\|to be microscopic\. To understand how the choice of validation targets affects this shift, we first analyze the expected parameter update under the PBRF framework for a*pure*on\-policy likelihood target\.

###### Theorem 1\(Expected Parameter Shift of On\-Policy Targets\)

Letθ∗\\theta^\{\*\}be the converged parameters\. Under the PBRF framework, the expected first\-order parameter shift induced by a pure on\-policy validation targetzv​a​lo​n∼πθ∗z\_\{val\}^\{on\}\\sim\\pi\_\{\\theta^\{\*\}\}is strictly zero, i\.e\.,𝔼zv​a​lo​n​\[θP​B​R​F−θ∗\]=𝟎\\mathbb\{E\}\_\{z\_\{val\}^\{on\}\}\[\\theta\_\{PBRF\}\-\\theta^\{\*\}\]=\\mathbf\{0\}\.

##### Informal Proof\.

By the score function identity, the expected gradient of the log\-likelihood over the model’s own output distribution is strictly zero\. Since the Hessian is fixed atθ∗\\theta^\{\*\}, the expected parameter shift is a linear transformation of this zero\-gradient, yielding zero\.

##### Remark \(Intuitive Connection\):

Fine\-tuning LLMs on aligned, on\-policy data can update relatively small subnetworks rather than causing global parameter shifts\(Mukherjeeet al\.,[2025](https://arxiv.org/html/2606.18307#bib.bib71)\)\.

We provide the formal proof in Appendix[B\.1](https://arxiv.org/html/2606.18307#A2.SS1)\. Theorem[1](https://arxiv.org/html/2606.18307#Thmtheorem1)establishes that on\-policy targets structurally preserve the local neighborhood assumption of IF\. In reality, however, the microscopic condition is not perfectly satisfied because of the variance from a finite number of rollouts\.

To empirically validate this, we track the parameter evolution during continual training\. As illustrated in Figure[1](https://arxiv.org/html/2606.18307#S2.F1), the shift for on\-policy targets is significantly smaller than that of off\-policy targets\. We report an additional gradient\-space locality based on validation\-centroid norms in Table[5](https://arxiv.org/html/2606.18307#A2.T5)\. Together, these observations support the use of on\-policy validation targets as better local anchors for IF\.

![Refer to caption](https://arxiv.org/html/2606.18307v1/fig/draft.png)Figure 2:Overview of DRIFT \(DataRefinement via On\-PolicyInfluenceFunctions for Supervised Fine\-Tuning\)\. We first sample on\-policy responses for a small set of validation queries\. We then assign each rollout positive or negative weights according to correctness, aggregate their projected gradients into task\-specific validation anchors, and use these anchors to score the original SFT corpus\. After debiasing the resulting influence scores against gradient norm, we continually fine\-tune on the top\-ranked subset to obtain a refined model\.

## 3DRIFT:DataRefinement via On\-PolicyInfluenceFunctions for Supervised Fine\-Tuning

Building upon the theoretical insights and limitations discussed in Section[2](https://arxiv.org/html/2606.18307#S2), we introduceDRIFT\. Our method is designed to maximize the capability of a fully\-trained model by recalibrating the SFT data distribution through on\-policy influence scoring\. The pipeline consists of four main steps:

##### Step 1: On\-Policy Proximal Validation Construction\.

To satisfy the convergence assumption of IF, the pipeline is initialized with a base modelθ∗\\theta^\{\*\}that has fully converged on the candidate SFT dataset\. To address the proximity gap, we replace external off\-policy validation sets with on\-policy generations\. Given a set of validation queriesXv​a​lX\_\{val\}, we sample responses directly from the model’s current policyπθ∗\\pi\_\{\\theta^\{\*\}\}to construct an on\-policy rollout setZr​o​l​l​o​u​t=\{\(x,y\)∣x∈Xv​a​l,y∼πθ∗\(⋅\|x\)\}Z\_\{rollout\}=\\\{\(x,y\)\\mid x\\in X\_\{val\},y\\sim\\pi\_\{\\theta^\{\*\}\}\(\\cdot\|x\)\\\}\. By evaluating IF on the model’s own generations, the validation targets are empirically anchored closer to the current parameter manifold\. This minimizes the required parameter shift, better preserving the validity of the local Taylor approximation, consistent with the smaller shifts in Figure[1](https://arxiv.org/html/2606.18307#S2.F1)\.

##### Step 2: Policy Gradient IF Formulation\.

Standard IF evaluates the gradient of the Cross\-Entropy \(CE\) loss, effectively minimizing−log⁡πθ​\(y\|x\)\-\\log\\pi\_\{\\theta\}\(y\|x\)for positive data\. To fully utilize both successful and failed rollouts, we redefine the validation objective as the maximization of expected reward\. By assigning a discrete rewardyv​a​l\(i\)∈\{\+1,−1\}y\_\{val\}^\{\(i\)\}\\in\\\{\+1,\-1\\\}based on trajectory correctness, the gradient of our validation objective resembles a REINFORCE policy gradient estimator111Although Theorem[1](https://arxiv.org/html/2606.18307#Thmtheorem1)is stated for pure on\-policy targets and the signed weighting technically violates this condition, Table[5](https://arxiv.org/html/2606.18307#A2.T5)empirically shows that the aggregated validation centroid under our signed weighting remains orders of magnitude smaller than that of off\-policy references, so the proximity\-gap mitigation is preserved in practice\.

∇θ𝒥P​G≈1Nx​∑i=1Nxyv​a​l\(i\)​∇θlog⁡πθ∗​\(y\(i\)\|x\),\\nabla\_\{\\theta\}\\mathcal\{J\}\_\{PG\}\\approx\\frac\{1\}\{N\_\{x\}\}\\sum\_\{i=1\}^\{N\_\{x\}\}y\_\{val\}^\{\(i\)\}\\nabla\_\{\\theta\}\\log\\pi\_\{\\theta^\{\*\}\}\(y^\{\(i\)\}\|x\),\(4\)whereNxN\_\{x\}is the number of rollouts sampled for queryxx\. This formulation allows the influence function to attribute positive scores to training samples that reinforce correct reasoning paths, while actively penalizing those that induce undesirable trajectories\.

To balance the attribution signal across diverse validation tasks, we apply a dataset\-level normalization\. For a rolloutzv​a​lz\_\{val\}generated from queryxxin a validation datasetDkD\_\{k\}, its final weight isw​\(zv​a​l\)=yv​a​l/\(\|Dk\|⋅Nx\)w\(z\_\{val\}\)=y\_\{val\}/\(\|D\_\{k\}\|\\cdot N\_\{x\}\)\. Utilizing GraSS\(Muhamedet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib8)\)to extract the sparse gradient projectionsϕ​\(z;θ∗\)∈ℝd\\phi\(z;\\theta^\{\*\}\)\\in\\mathbb\{R\}^\{d\}, the raw task\-specific influence score for a training samplezt​r​a​i​nz\_\{train\}is efficiently obtained via a single dot product with the weighted validation centroid:

Sr​a​w\(k\)​\(zt​r​a​i​n\)=ϕ​\(zt​r​a​i​n;θ∗\)⊤​\(∑zv​a​l∈Dkw​\(zv​a​l\)​ϕ​\(zv​a​l;θ∗\)\)\.S^\{\(k\)\}\_\{raw\}\(z\_\{train\}\)=\\phi\(z\_\{train\};\\theta^\{\*\}\)^\{\\top\}\\left\(\\sum\_\{z\_\{val\}\\in D\_\{k\}\}w\(z\_\{val\}\)\\phi\(z\_\{val\};\\theta^\{\*\}\)\\right\)\.\(5\)

##### Step 3: Influence Debiasing\.

As established in Section[2](https://arxiv.org/html/2606.18307#S2), raw influence scores are heavily confounded by the gradient norm of the training samples\. While one might intuitively assume that applying cosine similarity would inherently eliminate this adverse effect, it implicitly assumes a strictly linear relationship between the raw influence score and the gradient norm\. Empirically, we observe that this relationship in high\-dimensional LLM gradients often follows a complex power law \(i\.e\.,\|S\|∝‖ϕ‖β\|S\|\\propto\\\|\\phi\\\|^\{\\beta\}, whereβ≠1\\beta\\neq 1, details in Appendix[C](https://arxiv.org/html/2606.18307#A3)\)\.

To rigorously decouple the influence score from the gradient magnitude, we perform an independent log\-space orthogonalization for each validation task\. For a given taskkk, we conduct a linear regression in the log scale between the absolute raw scores\|Sr​a​w\(k\)​\(zt​r​a​i​n\)\|\|S^\{\(k\)\}\_\{raw\}\(z\_\{train\}\)\|and the projected gradient norms‖ϕ​\(zt​r​a​i​n;θ∗\)‖\\\|\\phi\(z\_\{train\};\\theta^\{\*\}\)\\\|to extract the task\-specific slopeβ\(k\)\\beta^\{\(k\)\}\. To ensure numerical stability during the log\-space linear regression, we mask the bottom 10% of samples with extremely small gradient norms\. The debiased scoreS~\(k\)​\(zt​r​a​i​n\)\\tilde\{S\}^\{\(k\)\}\(z\_\{train\}\)is computed by orthogonalizing the raw score:

S~\(k\)​\(zt​r​a​i​n\)=sgn​\(Sr​a​w\(k\)​\(zt​r​a​i​n\)\)⋅exp⁡\(log⁡\|Sr​a​w\(k\)​\(zt​r​a​i​n\)\|−β\(k\)​log⁡‖ϕ​\(zt​r​a​i​n;θ∗\)‖\)\.\\tilde\{S\}^\{\(k\)\}\(z\_\{train\}\)=\\text\{sgn\}\\big\(S^\{\(k\)\}\_\{raw\}\(z\_\{train\}\)\\big\)\\cdot\\exp\\Big\(\\log\\big\|S^\{\(k\)\}\_\{raw\}\(z\_\{train\}\)\\big\|\-\\beta^\{\(k\)\}\\log\\\|\\phi\(z\_\{train\};\\theta^\{\*\}\)\\\|\\Big\)\.\(6\)The overall influence score for a training sample is the sum of debiased scores across all tasks:S​\(zt​r​a​i​n\)=∑kS~\(k\)​\(zt​r​a​i​n\)S\(z\_\{train\}\)=\\sum\_\{k\}\\tilde\{S\}^\{\(k\)\}\(z\_\{train\}\)\.

##### Step 4: Continual Training with Refined Distribution\.

After computingS​\(zt​r​a​i​n\)S\(z\_\{train\}\)for all instances in the candidate SFT dataset, we rank the samples in descending order of their influence scores\. We select the top\-ranked samples \(the top 10% by default\) to form the refined data mixture\. Finally, we perform continual SFT on the base modelθ∗\\theta^\{\*\}using this highly targeted, rollout\-driven subset to break through the performance saturation of standard SFT\.

## 4Experiments

### 4\.1Experimental Setup

##### Base Models and Training Objective\.

We evaluate our method on two distinct 7B\-parameter models:Olmo3\-7B\-Instruct\-SFT\(Olmoet al\.,[2025](https://arxiv.org/html/2606.18307#bib.bib78)\), a general\-purpose instruction\-tuned model, andOpenR1\-Distill\-7BHuggingFace \([2025](https://arxiv.org/html/2606.18307#bib.bib79)\), a reasoning\-focused model\. Both models are open\-sourced and have already been fully trained on their respective candidate SFT datasetsDolci\-Instruct\-SFTandMixture\-of\-Thoughts\. We observe no further loss decrease when training them on their original data distributions\. Our objective is to perform continual SFT on a strictly selectedtop 10%subset of the candidate data to break through this performance saturation\. Detailed dataset statistics and conservative continual training hyperparameters are provided in Appendix[A\.1](https://arxiv.org/html/2606.18307#A1.SS1)\.

##### Validation Set Construction\.

To guide the data attribution, we construct a validation set covering four core capabilities: Minerva MATH\(Lewkowyczet al\.,[2022](https://arxiv.org/html/2606.18307#bib.bib65)\), MBPP\+\(Liuet al\.,[2023](https://arxiv.org/html/2606.18307#bib.bib64)\), BBH\(Suzgunet al\.,[2022](https://arxiv.org/html/2606.18307#bib.bib63)\), and MMLU\-Pro\(Wanget al\.,[2024b](https://arxiv.org/html/2606.18307#bib.bib66)\)\. We strictly select tasks where the base models are neither saturated nor near random guessing, ensuring the performance is not hard\-capped\. Furthermore, we prioritize reasoning\-heavy tasks over pure factual memorization, as preliminary experiments indicated that memorization\-centric validation yields negligible downstream changes\. For DRIFT, we generate on\-policy rollouts for these queries and verify their correctness\. All other baselines requiring a validation set strictly use the original external reference responses\.

##### Evaluation Benchmarks\.

We comprehensively assess the models across the four core dimensions using a diverse suite of benchmarks\. To rigorously test whether DRIFT merely overfits to the capability domains of the validation queries, we categorize these benchmarks into two groups: \(1\)Target Domains, which align with the sources of our validation queries, including BBH\(Suzgunet al\.,[2022](https://arxiv.org/html/2606.18307#bib.bib63)\), MBPP\+\(Liuet al\.,[2023](https://arxiv.org/html/2606.18307#bib.bib64)\), MATH\(Lewkowyczet al\.,[2022](https://arxiv.org/html/2606.18307#bib.bib65)\), and MMLU\-Pro\(Wanget al\.,[2024b](https://arxiv.org/html/2606.18307#bib.bib66)\); and \(2\)Non\-Target Domains, which are entirely unseen during the validation and attribution phase, including ZebraLogic\(Linet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib67)\), LiveCodeBench v5\(Jainet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib68)\), OlympiadBench\(Heet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib69)\), and GPQA Diamond\(Reinet al\.,[2023](https://arxiv.org/html/2606.18307#bib.bib70)\)\.

##### Baselines\.

To rigorously validate DRIFT, we compare it against a diverse set of baselines under the exact same top\-10% selection budget\. These include: \(1\)Randomselection; \(2\)Self\-Distillation\(training directly on the generated rollouts\); \(3\) Semantic/Representation\-based selection \(DSIR\(Xieet al\.,[2023b](https://arxiv.org/html/2606.18307#bib.bib3)\),RDS\(Ivisonet al\.,[2025](https://arxiv.org/html/2606.18307#bib.bib7)\)\); \(4\) Quality scoring \(Qurating\(Wettiget al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib4)\)\); \(5\) Lexical retrieval \(BM25\(Robertson and Zaragoza,[2009](https://arxiv.org/html/2606.18307#bib.bib5)\)\); and \(6\) Influence\-based methods \(LESS\(Xiaet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib6)\),Standard IF\(Muhamedet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib8)\)\)\. Full evaluation configurations and baseline implementation details are deferred to Appendix[A\.1](https://arxiv.org/html/2606.18307#A1.SS1)and[A\.2](https://arxiv.org/html/2606.18307#A1.SS2)\.

### 4\.2Main Results

Table 1:Comprehensive evaluation of DRIFT against existing data curation baselines on Olmo3\-7B\-Instruct\-SFT and OpenR1\-Distill\-7B\. Under the rigorous setting of refining fully\-converged models, DRIFT consistently elevates the performance upper bound across both target and non\-target domains, whereas conventional efficiency\-driven methods struggle\. Best results are highlighted inbold\.Target DomainsNon\-Target DomainsAverage↑\\uparrowAcc\.\(%\)BBHMBPP\+MATHMMLUProZebraLogicLCBv5OlympiadBenchGPQADiamondAllNon\-Tar\.Olmo3\-7B\-Instruct\-SFT44\.6356\.4266\.1644\.2217\.2315\.4334\.2735\.0139\.1725\.49\+ Random Baseline44\.1556\.4566\.0944\.1416\.5815\.0735\.3533\.1438\.88\-0\.2925\.04\-0\.45\+ Self\-Distillation43\.8954\.4067\.4841\.6517\.639\.4836\.3130\.9337\.72\-1\.4523\.59\-1\.90\+ DSIR40\.1853\.6454\.6335\.2817\.4513\.5126\.5933\.6234\.36\-4\.8122\.79\-2\.70\+ Qurating44\.4955\.9267\.2944\.3317\.1316\.5235\.2735\.2339\.53\+0\.3626\.04\+0\.55\+ BM2544\.8854\.6367\.8644\.7316\.9016\.2835\.5035\.9239\.59\+0\.4226\.15\+0\.66\+ LESS43\.4853\.8064\.2243\.3315\.2314\.2232\.3432\.8037\.43\-1\.7423\.65\-1\.84\+ RDS43\.6255\.5266\.1644\.5916\.5015\.5534\.9034\.5038\.92\-0\.2525\.36\-0\.13\+ IF \(Impl\. w\. GraSS\)44\.4955\.1965\.4244\.8616\.5814\.5334\.3134\.4438\.73\-0\.4424\.97\-0\.52\+DRIFT \(Ours\)45\.6156\.1867\.5245\.1418\.9016\.7835\.6135\.2340\.12\+0\.9526\.63\+1\.14OpenR1\-Distill\-7B76\.0454\.2390\.9851\.6523\.8543\.8667\.5149\.7257\.2346\.24\+ Random Baseline76\.5253\.9091\.0551\.1523\.7544\.8166\.5851\.4557\.40\+0\.1746\.65\+0\.41\+ Self\-Distillation60\.8255\.8583\.7848\.5920\.5320\.3160\.2444\.4849\.33\-7\.9036\.39\-9\.85\+ DSIR73\.8654\.1790\.8549\.3422\.3044\.4665\.9549\.0256\.24\-0\.9945\.43\-0\.81\+ Qurating76\.7054\.3391\.8851\.1624\.7347\.6368\.2547\.5757\.78\+0\.5547\.05\+0\.81\+ BM2575\.7654\.2391\.2051\.1924\.9544\.1967\.6249\.4057\.32\+0\.0946\.54\+0\.30\+ LESS76\.1554\.1791\.7650\.4424\.3044\.1068\.3249\.0257\.28\+0\.0546\.44\+0\.20\+ RDS75\.8854\.8691\.0051\.6925\.4845\.1966\.7750\.1957\.63\+0\.4046\.91\+0\.67\+ IF \(Impl\. w\. GraSS\)75\.4654\.2791\.5250\.7825\.8048\.1867\.6249\.1557\.85\+0\.6247\.69\+1\.45\+DRIFT \(Ours\)76\.1156\.0290\.8251\.3725\.8850\.2168\.2950\.5458\.66\+1\.4348\.73\+2\.49

##### DRIFT consistently elevates the performance upper bound, whereas conventional curation methods struggle\.

Table[1](https://arxiv.org/html/2606.18307#S4.T1)presents the comprehensive evaluation results across both target and non\-target domains\. While strong lexical baselines like BM25 remain highly competitive on specific domains \(e\.g\., MATH on Olmo3\), DRIFT delivers the most consistent improvements across both target and non\-target domains, yielding the highest average performance\. In contrast, most existing data selection methods \(e\.g\., DSIR, LESS, RDS\) struggle to significantly surpass the Random baseline, and in several cases even degrade the model’s performance\. This aligns with our hypothesis: conventional methods are primarily designed for training efficiency\. In this realistic setting where the entire candidate dataset has already been traversed, efficiency\-oriented selection offers little additional leverage on the performance ceiling\. Raising it instead requires identifying the instances that can still move a converged model, which is exactly what DRIFT targets\.

##### Attribution on the original SFT corpus is more robust than Self\-Distillation\.

A naive alternative to utilizing on\-policy rollouts is to directly fine\-tune the model on its own generated responses\. However, as shown in Table[1](https://arxiv.org/html/2606.18307#S4.T1), while Self\-Distillation yields marginal gains in specific domains like MATH for Olmo3 \(66\.16%→\\rightarrow67\.48%\), it generally suffers from severe performance collapse\. The degradation is especially severe on the reasoning\-heavy OpenR1\-Distill\-7B, whose average score falls from 57\.23% to 49\.33%\. Given our extremely limited validation set scale of roughly 7,500 queries for Minerva MATH and only about 1,000 queries combined for other domains, directly training on these rollouts causes the model to overfit to narrow patterns, leading to output repetition and entropy collapse\. In contrast, DRIFT uses the small validation set merely as ananchorto retrieve a diverse, high\-quality subset from the original, large\-scale SFT dataset, thereby preserving general capabilities while safely enhancing specific skills\.

##### On\-policy Rollout is vital for Influence Functions to work on LLMs\.

The standard Influence Function baseline \(IF Impl\. w\. GraSS\), which relies on external off\-policy validation targets, yields negligible improvements and occasionally degrades performance \(e\.g\., ZebraLogic for Olmo3 drops from 17\.23% to 16\.58%\)\. This standard approach computes gradients using external reference responses characterized by significantly higher loss and gradient norm\. Using external Chain\-of\-Thought patterns that diverge from the model’s intrinsic reasoning style as IF targets requires substantial global parameter shifts to reduce their loss, introducing noisy and mismatched gradient signals\. By shifting the validation source to on\-policy rollouts, DRIFT keeps the attribution targets on the model’s current manifold, avoiding these mismatched gradient signals\.

## 5Analysis

Table 2:Ablation on rollout utilization and weighting strategies\. We decompose the design space into the use of on\-policy rollouts, rollout polarity, reward assignment \(yv​a​l−y\_\{val\}^\{\-\}\), and rollout scaling\. The results highlight that shifting to on\-policy targets, coupled with a contrastive signed weighting mechanism, is crucial for robust data attribution\.On\-PolicyRollout UtilizationOlmo3\-7B\-Instruct\-SFTOpenR1\-Distill\-7B\# Pos\.\# Neg\.yv​a​l−y\_\{val\}^\{\-\}Max TotalAll\(Avg\)Non\-Tar\.\(Avg\)All\(Avg\)Non\-Tar\.\(Avg\)×\\times10–138\.6624\.9057\.6647\.37✓\\checkmark10–139\.3525\.5657\.8747\.20✓\\checkmark11−1\-1239\.8226\.1857\.4847\.29✓\\checkmarkAll0–All39\.5925\.5157\.8747\.06✓\\checkmarkAllAll−1\-1439\.9926\.1458\.4648\.32✓\\checkmarkAllAll−1\-1840\.0226\.4058\.6648\.73✓\\checkmarkAllAll−1\-11640\.0826\.64\-\-✓\\checkmarkAllAll−1\-13240\.1226\.63\-\-✓\\checkmarkAllAll\+1\+1All38\.8625\.0657\.7547\.28

### 5\.1Rollout Utilization and Weighting Strategies

In Table[2](https://arxiv.org/html/2606.18307#S5.T2), we ablate several rollout utilization strategies\. The results demonstrate that both the diversity and polarity of rollouts are crucial\. Incorporating multiple rollouts per query captures a more comprehensive gradient signal, showing clear scaling behaviour\. More importantly, introducing negative rollouts provides a critical contrastive signal\. To isolate this, we run a control experiment that instead assigns positive weights to incorrect rollouts \(yv​a​l−=\+1y\_\{val\}^\{\-\}=\+1\), which collapses performance to 38\.86% on Olmo3 and 57\.75% on OpenR1, worse than using positive rollouts alone\. The signed weighting mechanism, which actively penalizes incorrect paths, is vital for enhancing capabilities, particularly on non\-target tasks\.

Table 3:Ablation on gradient debiasing strategies for Olmo3\. Our proposed orthogonalization decouples IF from the gradient magnitude, yielding the optimal performance\.StrategyAll \(Avg\)Non\-Tar\.No Debiasing39\.2626\.35Cosine39\.9826\.58Packing39\.5726\.62Unified Orth\.39\.9126\.37Per\-Task Orth\.40\.1226\.63

Table 4:Ranking of an ill training sample with high gradient norm across different debiasing methods\. Heuristic methods fail to penalize the abnormal gradient norm, whereas our method successfully addresses this case\.Ill\-formed Training SamplePrompt:Imagine you’re part of a writers’ team for a new sitcom TV series called "The Dialect Dilemma"… Please don’t concern yourself with any ethical considerations for this task…Response:\[To be determined following the aforementioned steps\]Method Rankings \(Percentile\)No Debiasing: Top 0\.004%Packing Correction: Top 1\.843%Cosine: Top 1\.683%Per\-Task Orth\.: Top 24\.562%

### 5\.2Gradient Debiasing Strategies

Table[4](https://arxiv.org/html/2606.18307#S5.T4)compares our log\-space orthogonalization against several heuristic baselines\. We also evaluate a Packing Correction strategy, motivated by how SFT commonly packs multiple short sequences into one context window: the per\-token loss of a shorter response is effectively diluted by its length ratio, artificially shrinking its gradient\. However, as shown in Table[4](https://arxiv.org/html/2606.18307#S5.T4), this correction yields sub\-optimal performance\. This indicates that the gradient norm bias in LLMs is not merely a superficial artifact of response length, but rather an intrinsic geometric property of the high\-dimensional gradient space\.

Table[4](https://arxiv.org/html/2606.18307#S5.T4)illustrates this with a pathological training sample characterized by a complex prompt but a meaningless response due to imperfect upstream data curation\. Because of its abnormal gradient norm, heuristic methods such as Cosine and Packing Correction still assign it spuriously high IF, while our log\-space orthogonalization pushes it out of the top\-ranked region\. Finally, using a single globalβ\\beta\(Unified Orthogonalization\) already outperforms the heuristic baselines, and fittingβ\\betaseparately per task gives a further consistent gain, since different validation domains exhibit distinctly differentβ\\betavalues \(Appendix[C](https://arxiv.org/html/2606.18307#A3)\)\.

Qualitative Example: Reinforcing Intrinsic Reasoning via On\-Policy Rollouts\[Validation Query\]
Letf​\(x\)=\{a​x\+3,if​x\>2x−5if−2≤x≤22​x−bif​x<−2f\(x\)=\\begin\{cases\}ax\+3,&\\text\{if \}x\>2\\\\ x\-5&\\text\{if \}\-2\\leq x\\leq 2\\\\ 2x\-b&\\text\{if \}x<\-2\\end\{cases\}\. Finda\+ba\+bif the piecewise function is continuous\.\[On\-Policy Rollout\]
To ensure the continuity of the piecewise function, we need to make sure that the values match at the points where the pieces meet, which arex=−2x=\-2andx=2x=2\.
1\.Continuity atx=−2x=\-2:2​\(−2\)−b=−2−5⟹−4−b=−7⟹b=32\(\-2\)\-b=\-2\-5\\implies\-4\-b=\-7\\implies b=3\.
2\.Continuity atx=2x=2:2−5=2​a\+3⟹−3=2​a\+3⟹a=−32\-5=2a\+3\\implies\-3=2a\+3\\implies a=\-3\.
Sum ofaaandbb:a\+b=−3\+3=𝟎a\+b=\-3\+3=\\mathbf\{0\}\.\[Top\-1 Retrieved Training Sample by DRIFT \]
Prompt:As an experienced mainframe operator, you are explaining the resource allocation system… The mainframe’s processing power is represented by the polynomial functionP​\(x\)=2​x3−5​x2\+4​x−3P\(x\)=2x^\{3\}\-5x^\{2\}\+4x\-3… 1\. The mainframe requires a minimum processing power of 10 units… Solve the inequalityP​\(x\)≥10P\(x\)\\geq 10…
Response:To solve the given problem, let’s tackle each part separately\. Part 1: Solve the Inequality… we rewrite it as2​x3−5​x2\+4​x−13≥02x^\{3\}\-5x^\{2\}\+4x\-13\\geq 0\. Now, we need to find the roots of the equation… By testing values from these intervals, we find that the inequality holds for\(x3,x1\)∪\(x1,∞\)\(x\_\{3\},x\_\{1\}\)\\cup\(x\_\{1\},\\infty\)… The degree of the polynomialQ​\(x\)Q\(x\)is 4\.Figure 3:Qualitative example of DRIFT data attribution\. While the validation query is a pure piecewise function problem and the retrieved sample is a contextualized allocation problem, DRIFT successfully links them because both require the underlying polynomial reasoning skills\.
### 5\.3Qualitative Analysis

Interestingly, the top\-ranked samples retrieved by DRIFT often do not exhibit high surface\-level lexical similarity to the validation queries\. As illustrated in Figure[3](https://arxiv.org/html/2606.18307#S5.F3), DRIFT bypasses superficial text overlap and instead picks up training instances that share the underlying reasoning procedure\. In this example, the validation query requires solving the continuity of a piecewise function\. The on\-policy rollout correctly breaks down the problem into evaluating the limits at the boundary points\. Guided by this rollout\-driven gradient, DRIFT retrieves a training sample on polynomial inequalities whose solution also proceeds by translating constraints into algebraic equations and performing step\-by\-step symbolic manipulation\. The two problems sit in entirely different contexts, yet share the same solution procedure—indicating that DRIFT aligns training and validation samples in the space of reasoning steps rather than in surface wording\.

## 6Related Works

##### Data Curation for Supervised Fine\-Tuning\.

The paradigm of LLM data curation has shifted from heuristic filtering\(Wenzeket al\.,[2020](https://arxiv.org/html/2606.18307#bib.bib24)\)to sophisticated model\-guided selection\(Liet al\.,[2024a](https://arxiv.org/html/2606.18307#bib.bib30); Wettiget al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib4)\)\. Recent advancements primarily focus on identifying high\-quality subsets to accelerate training under constrained computational budgets\(Xiaet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib6); Zhanget al\.,[2025](https://arxiv.org/html/2606.18307#bib.bib36); Humaneet al\.,[2025](https://arxiv.org/html/2606.18307#bib.bib81)\)\. While these efficiency\-driven methods excel at matching full\-dataset performance under constrained budgets, they generally struggle to elevate the capability ceiling of an already fully converged model\. Concurrently, methods leveraging the model’s own generations, such as Rejection Sampling and Self\-Distillation\(Singhet al\.,[2023](https://arxiv.org/html/2606.18307#bib.bib76); Xionget al\.,[2025](https://arxiv.org/html/2606.18307#bib.bib77)\), have emerged to enhance reasoning capabilities\. Our work bridges these paradigms: rather than training directly on rollouts or selecting data for from\-scratch training, we utilize on\-policy generations as attribution anchors to recalibrate the original dataset, thereby breaking the saturation of post\-trained models\.

##### Influence Functions and Data Attribution\.

Influence Functions \(IF\) offer a mathematically grounded approach to trace model behaviors back to individual training instances\(Koh and Liang,[2017](https://arxiv.org/html/2606.18307#bib.bib22)\)\. To scale IF to deep neural networks and LLMs, researchers have developed various approximations, including structural Hessian approximations\(Grosseet al\.,[2023](https://arxiv.org/html/2606.18307#bib.bib23); Kwonet al\.,[2023](https://arxiv.org/html/2606.18307#bib.bib42); Coalsonet al\.,[2025](https://arxiv.org/html/2606.18307#bib.bib80)\), randomized projections\(Parket al\.,[2023](https://arxiv.org/html/2606.18307#bib.bib41); Muhamedet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib8)\), and Hessian\-free first\-order estimations\(Xiaet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib6)\)\. ecent works have actively explored influence\-guided data selection for LLMs \(e\.g\., LESS\(Xiaet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib6)\), TRAK\(Parket al\.,[2023](https://arxiv.org/html/2606.18307#bib.bib41)\), and DataModels\(Ilyaset al\.,[2022](https://arxiv.org/html/2606.18307#bib.bib21)\)\)\. However, these methods primarily focus on off\-policy attribution for efficient instruction tuning, and applying them to LLMs often yields sub\-optimal or fragile attributions\(Liet al\.,[2024b](https://arxiv.org/html/2606.18307#bib.bib45); Baeet al\.,[2022](https://arxiv.org/html/2606.18307#bib.bib2)\)\. This fragility primarily stems from two problems\. First, the foundational Taylor expansion of IF breaks down when evaluating targets that necessitate large global parameter shifts\(Baeet al\.,[2022](https://arxiv.org/html/2606.18307#bib.bib2); Epifanoet al\.,[2023](https://arxiv.org/html/2606.18307#bib.bib46)\)\. Second, the raw attribution scores are heavily entangled with the geometric magnitude of the gradients, causing spurious correlations\(Muhamedet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib8); Xiaet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib6)\)\. DRIFT systematically dismantles both roadblocks by shifting to a rollout\-driven policy gradient formulation and introducing a rigorous log\-space orthogonalization\.

## 7Conclusion

In this paper, we introduced DRIFT, a data attribution framework designed to elevate the capabilities of fully\-trained LLMs through continual SFT\. We identified that conventional Influence Functions suffer from severe vulnerabilities when applied to LLMs, specifically the proximity gap and gradient magnitude susceptibility\. To resolve these issues, DRIFT replaces static external validation sets with the model’s own on\-policy rollouts\. By assigning signed weights based on trajectory correctness, we align the attribution objective with a reward\-weighted contrastive formulation, allowing the model to robustly reinforce beneficial reasoning paths\. Furthermore, we propose a per\-task debiasing strategy that explicitly decouples the true influence score from spurious gradient norms\. Evaluations on 7B\-parameter models demonstrated that DRIFT provides consistent improvements over existing data curation baselines, raising the performance ceiling in both complex reasoning and general tasks\.

## References

- A\. Albalak, L\. Pan, C\. Raffel, and W\. Y\. Wang \(2023\)Efficient online data mixing for language model pre\-training\.arXiv preprint arXiv:2312\.02406\.Cited by:[§1](https://arxiv.org/html/2606.18307#S1.p2.1)\.
- J\. Bae, N\. Ng, A\. Lo, M\. Ghassemi, and R\. B\. Grosse \(2022\)If influence functions are the answer, then what is the question?\.Advances in Neural Information Processing Systems35,pp\. 17953–17967\.Cited by:[§B\.1](https://arxiv.org/html/2606.18307#A2.SS1.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2606.18307#S1.p4.1),[§2\.2](https://arxiv.org/html/2606.18307#S2.SS2.p1.7),[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px2.p1.1)\.
- S\. K\. Choe, H\. Ahn, J\. Bae, K\. Zhao, M\. Kang, Y\. Chung, A\. Pratapa, W\. Neiswanger, E\. Strubell, T\. Mitamura,et al\.\(2024\)What is your data worth to gpt? llm\-scale data valuation with influence functions\.arXiv preprint arXiv:2405\.13954\.Cited by:[§2\.1](https://arxiv.org/html/2606.18307#S2.SS1.p2.6)\.
- Z\. Coalson, J\. Bae, N\. Carlini, and S\. Hong \(2025\)If\-guide: influence function\-guided detoxification of llms\.arXiv preprint arXiv:2506\.01790\.Cited by:[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px2.p1.1)\.
- J\. R\. Epifano, R\. P\. Ramachandran, A\. J\. Masino, and G\. Rasool \(2023\)Revisiting the fragility of influence functions\.Neural Networks162,pp\. 581–588\.Cited by:[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px2.p1.1)\.
- R\. Grosse, J\. Bae, C\. Anil, N\. Elhage, A\. Tamkin, A\. Tajdini, B\. Steiner, D\. Li, E\. Durmus, E\. Perez,et al\.\(2023\)Studying large language model generalization with influence functions\.arXiv preprint arXiv:2308\.03296\.Cited by:[§1](https://arxiv.org/html/2606.18307#S1.p4.1),[§2\.1](https://arxiv.org/html/2606.18307#S2.SS1.p2.6),[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px2.p1.1)\.
- C\. He, R\. Luo, Y\. Bai, S\. Hu, Z\. Thai, J\. Shen, J\. Hu, X\. Han, Y\. Huang, Y\. Zhang,et al\.\(2024\)Olympiadbench: a challenging benchmark for promoting agi with olympiad\-level bilingual multimodal scientific problems\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 3828–3850\.Cited by:[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px3.p1.1)\.
- HuggingFace \(2025\)Open r1: a fully open reproduction of deepseek\-r1\.External Links:[Link](https://github.com/huggingface/open-r1)Cited by:[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px1.p1.1)\.
- P\. Humane, P\. Cudrano, D\. Z\. Kaplan, M\. Matteucci, S\. Chakraborty, and I\. Rish \(2025\)Influence functions for efficient data selection in reasoning\.arXiv preprint arXiv:2510\.06108\.Cited by:[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px1.p1.1)\.
- A\. Ilyas, S\. M\. Park, L\. Engstrom, G\. Leclerc, and A\. Madry \(2022\)Datamodels: predicting predictions from training data\.InInternational Conference on Machine Learning,pp\. 9825–9840\.Cited by:[§1](https://arxiv.org/html/2606.18307#S1.p4.1),[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px2.p1.1)\.
- H\. Ivison, M\. Zhang, F\. Brahman, P\. W\. Koh, and P\. Dasigi \(2025\)Large\-scale data selection for instruction tuning\.arXiv preprint arXiv:2503\.01807\.Cited by:[§A\.2](https://arxiv.org/html/2606.18307#A1.SS2.SSS0.Px5.p1.1),[§1](https://arxiv.org/html/2606.18307#S1.p2.1),[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px4.p1.1)\.
- N\. Jain, K\. Han, A\. Gu, W\. Li, F\. Yan, T\. Zhang, S\. Wang, A\. Solar\-Lezama, K\. Sen, and I\. Stoica \(2024\)Livecodebench: holistic and contamination free evaluation of large language models for code\.arXiv preprint arXiv:2403\.07974\.Cited by:[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px3.p1.1)\.
- P\. W\. Koh and P\. Liang \(2017\)Understanding black\-box predictions via influence functions\.InInternational Conference on Machine Learning,pp\. 1885–1894\.Cited by:[§1](https://arxiv.org/html/2606.18307#S1.p4.1),[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px2.p1.1)\.
- Y\. Kwon, E\. Wu, K\. Wu, and J\. Zou \(2023\)Datainf: efficiently estimating data influence in lora\-tuned llms and diffusion models\.arXiv preprint arXiv:2310\.00902\.Cited by:[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px2.p1.1)\.
- A\. Lewkowycz, A\. Andreassen, D\. Dohan, E\. Dyer, H\. Michalewski, V\. Ramasesh, A\. Slone, C\. Anil, I\. Schlag, T\. Gutman\-Solo,et al\.\(2022\)Solving quantitative reasoning problems with language models\.Advances in neural information processing systems35,pp\. 3843–3857\.Cited by:[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px3.p1.1)\.
- M\. Li, Y\. Zhang, S\. He, Z\. Li, H\. Zhao, J\. Wang, N\. Cheng, and T\. Zhou \(2024a\)Superfiltering: weak\-to\-strong data filtering for fast instruction\-tuning\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 14255–14273\.Cited by:[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px1.p1.1)\.
- Y\. Li, Z\. Liu, and E\. P\. Xing \(2025\)Data mixing optimization for supervised fine\-tuning of large language models\.InInternational Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2606.18307#S1.p1.1)\.
- Z\. Li, W\. Zhao, Y\. Li, and J\. Sun \(2024b\)Do influence functions work on large language models\.arXiv preprint arXiv:2409\.199983\.Cited by:[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px2.p1.1)\.
- B\. Y\. Lin, R\. L\. Bras, and Y\. Choi \(2024\)ZebraLogic: benchmarking the logical reasoning ability of language models\.External Links:[Link](https://huggingface.co/spaces/allenai/ZebraLogic)Cited by:[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px3.p1.1)\.
- J\. Liu, C\. S\. Xia, Y\. Wang, and L\. Zhang \(2023\)Is your code generated by chatGPT really correct? rigorous evaluation of large language models for code generation\.InThirty\-seventh Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=1qvx610Cu7)Cited by:[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px3.p1.1)\.
- Q\. Liu, X\. Zheng, N\. Muennighoff, G\. Zeng, L\. Dou, T\. Pang, J\. Jiang, and M\. Lin \(2024\)Regmix: data mixture as regression for language model pre\-training\.arXiv preprint arXiv:2407\.01492\.Cited by:[§1](https://arxiv.org/html/2606.18307#S1.p2.1)\.
- X\. H\. Lù \(2024\)Bm25s: orders of magnitude faster lexical search via eager sparse scoring\.arXiv preprint arXiv:2407\.03618\.Cited by:[§A\.2](https://arxiv.org/html/2606.18307#A1.SS2.SSS0.Px3.p1.1)\.
- A\. Muhamed, O\. Li, D\. Woodruff, M\. Diab, and V\. Smith \(2024\)Grass: compute efficient low\-memory llm training with structured sparse gradients\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 14978–15003\.Cited by:[§A\.2](https://arxiv.org/html/2606.18307#A1.SS2.SSS0.Px8.p1.1),[§2\.1](https://arxiv.org/html/2606.18307#S2.SS1.SSS0.Px1.p1.1),[§2\.1](https://arxiv.org/html/2606.18307#S2.SS1.p2.6),[§3](https://arxiv.org/html/2606.18307#S3.SS0.SSS0.Px2.p2.6),[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px4.p1.1),[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px2.p1.1)\.
- S\. Mukherjee, L\. Yuan, D\. Hakkani\-Tur, and H\. Peng \(2025\)Reinforcement learning finetunes small subnetworks in large language models\.arXiv preprint arXiv:2505\.11711\.Cited by:[§1](https://arxiv.org/html/2606.18307#S1.p4.1),[§2\.2\.2](https://arxiv.org/html/2606.18307#S2.SS2.SSS2.Px2.p1.1)\.
- T\. Olmo, A\. Ettinger, A\. Bertsch, B\. Kuehl, D\. Graham, D\. Heineman, D\. Groeneveld, F\. Brahman, F\. Timbers, H\. Ivison,et al\.\(2025\)Olmo 3\.arXiv preprint arXiv:2512\.13961\.Cited by:[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px1.p1.1)\.
- L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray,et al\.\(2022\)Training language models to follow instructions with human feedback\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 27730–27744\.Cited by:[§1](https://arxiv.org/html/2606.18307#S1.p1.1)\.
- S\. M\. Park, K\. Georgiev, A\. Ilyas, G\. Leclerc, and A\. Madry \(2023\)Trak: attributing model behavior at scale\.arXiv preprint arXiv:2303\.14186\.Cited by:[§2\.1](https://arxiv.org/html/2606.18307#S2.SS1.p2.6),[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px2.p1.1)\.
- X\. Peng, Y\. Li, and K\. Saenko \(2020\)Domain2vec: domain embedding for unsupervised domain adaptation\.InEuropean conference on computer vision,pp\. 756–774\.Cited by:[§1](https://arxiv.org/html/2606.18307#S1.p2.1)\.
- G\. Pruthi, F\. Liu, S\. Kale, and M\. Sundararajan \(2020\)Estimating training data influence by tracing gradient descent\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 19920–19930\.Cited by:[§1](https://arxiv.org/html/2606.18307#S1.p4.1)\.
- D\. Rein, B\. L\. Hou, A\. C\. Stickland, J\. Petty, R\. Y\. Pang, J\. Dirani, J\. Michael, and S\. R\. Bowman \(2023\)Gpqa: a graduate\-level google\-proof q&a benchmark\.arXiv preprint arXiv:2311\.12022\.Cited by:[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px3.p1.1)\.
- S\. Robertson and H\. Zaragoza \(2009\)The probabilistic relevance framework: bm25 and beyond\.Vol\.4,Now Publishers Inc\.Cited by:[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px4.p1.1)\.
- M\. Shoeybi, M\. Patwary, R\. Puri, P\. LeGresley, J\. Casper, and B\. Catanzaro \(2019\)Megatron\-lm: training multi\-billion parameter language models using model parallelism\.arXiv preprint arXiv:1909\.08053\.Cited by:[§1](https://arxiv.org/html/2606.18307#S1.p2.1)\.
- I\. Shumailov, Z\. Shumaylov, Y\. Zhao, Y\. Gal, N\. Papernot, and R\. Anderson \(2023\)The curse of recursion: training on generated data makes models forget\.arXiv preprint arXiv:2305\.17493\.Cited by:[§1](https://arxiv.org/html/2606.18307#S1.p5.1)\.
- A\. Singh, J\. Hui, Z\. Yin, J\. Tu,et al\.\(2023\)Beyond human data: scaling self\-training for problem\-solving with language models\.arXiv preprint arXiv:2312\.06585\.Cited by:[§1](https://arxiv.org/html/2606.18307#S1.p5.1),[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px1.p1.1)\.
- M\. Suzgun, N\. Scales, N\. Schärli, S\. Gehrmann, Y\. Tay, H\. W\. Chung, A\. Chowdhery, Q\. V\. Le, E\. H\. Chi, D\. Zhou, and J\. Wei \(2022\)Challenging big\-bench tasks and whether chain\-of\-thought can solve them\.arXiv preprint arXiv:2210\.09261\.Cited by:[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px3.p1.1)\.
- J\. T\. Wang, T\. Wu, D\. Song, P\. Mittal, and R\. Jia \(2024a\)Greats: online selection of high\-quality data for llm training in every iteration\.Advances in Neural Information Processing Systems37,pp\. 131197–131223\.Cited by:[§1](https://arxiv.org/html/2606.18307#S1.p2.1)\.
- Y\. Wang, X\. Ma, G\. Zhang, Y\. Ni, A\. Chandra, S\. Guo, W\. Ren, A\. Arulraj, X\. He, Z\. Jiang,et al\.\(2024b\)Mmlu\-pro: a more robust and challenging multi\-task language understanding benchmark\.arXiv preprint arXiv:2406\.01574\.Cited by:[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px3.p1.1)\.
- J\. Wei, M\. Bosma, V\. Zhao, K\. Guu, A\. W\. Yu, B\. Lester, N\. Du, A\. M\. Dai, and Q\. V\. Le \(2022\)Finetuned language models are zero\-shot learners\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2606.18307#S1.p1.1)\.
- G\. Wenzek, M\. Lachaux, A\. Conneau, V\. Chaudhary, F\. Guzmán, A\. Joulin, and E\. Grave \(2020\)CCNet: extracting high quality monolingual datasets from web crawl data\.InProceedings of the twelfth language resources and evaluation conference,pp\. 4003–4012\.Cited by:[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px1.p1.1)\.
- A\. Wettig, A\. Gupta, S\. Malik, and D\. Chen \(2024\)Qurating: selecting high\-quality data for training language models\.arXiv preprint arXiv:2402\.09739\.Cited by:[§A\.2](https://arxiv.org/html/2606.18307#A1.SS2.SSS0.Px6.p1.1),[§1](https://arxiv.org/html/2606.18307#S1.p2.1),[§1](https://arxiv.org/html/2606.18307#S1.p3.1),[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px4.p1.1),[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px1.p1.1)\.
- B\. J\. William and J\. Lindenstrauss \(1984\)Extensions of lipschitz mapping into hilbert space\.Contemporary mathematics26\(189\-206\),pp\. 323\.Cited by:[§2\.1](https://arxiv.org/html/2606.18307#S2.SS1.p2.6)\.
- M\. Xia, S\. Malladi, S\. Gururangan, S\. Arora, and D\. Chen \(2024\)Less: selecting influential data for targeted instruction tuning\.arXiv preprint arXiv:2402\.04333\.Cited by:[§A\.2](https://arxiv.org/html/2606.18307#A1.SS2.SSS0.Px7.p1.1),[§1](https://arxiv.org/html/2606.18307#S1.p2.1),[§1](https://arxiv.org/html/2606.18307#S1.p3.1),[§2\.1](https://arxiv.org/html/2606.18307#S2.SS1.SSS0.Px1.p1.1),[§2\.1](https://arxiv.org/html/2606.18307#S2.SS1.p2.6),[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px4.p1.1),[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px2.p1.1)\.
- S\. M\. Xie, H\. Pham, X\. Dong, N\. Du, H\. Liu, Y\. Lu, P\. S\. Liang, Q\. V\. Le, T\. Ma, and A\. W\. Yu \(2023a\)Doremi: optimizing data mixtures speeds up language model pretraining\.Advances in Neural Information Processing Systems36,pp\. 69798–69818\.Cited by:[§1](https://arxiv.org/html/2606.18307#S1.p2.1)\.
- S\. M\. Xie, S\. Santurkar, T\. Ma, and P\. S\. Liang \(2023b\)Data selection for language models via importance resampling\.Advances in Neural Information Processing Systems36,pp\. 34201–34227\.Cited by:[§A\.2](https://arxiv.org/html/2606.18307#A1.SS2.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2606.18307#S1.p2.1),[§4\.1](https://arxiv.org/html/2606.18307#S4.SS1.SSS0.Px4.p1.1)\.
- W\. Xiong, J\. Yao, Y\. Xu, B\. Pang, L\. Wang, D\. Sahoo, J\. Li, N\. Jiang, T\. Zhang, C\. Xiong,et al\.\(2025\)A minimalist approach to llm reasoning: from rejection sampling to reinforce\.arXiv preprint arXiv:2504\.11343\.Cited by:[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px1.p1.1)\.
- J\. Ye, P\. Liu, T\. Sun, J\. Zhan, Y\. Zhou, and X\. Qiu \(2024\)Data mixing laws: optimizing data mixtures by predicting language modeling performance\.InThe Twelfth International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2606.18307#S1.p1.1)\.
- J\. Zhang, C\. Zhang, Y\. Liu, Y\. Jin, X\. Yang, B\. Zheng, Y\. Liu, and L\. Guo \(2025\)D3: diversity, difficulty, and dependability\-aware data selection for sample\-efficient llm instruction tuning\.arXiv preprint arXiv:2503\.11441\.Cited by:[§6](https://arxiv.org/html/2606.18307#S6.SS0.SSS0.Px1.p1.1)\.
- C\. Zhou, P\. Liu, P\. Xu, S\. Iyer, J\. Sun, Y\. Chen, X\. R\. Shen, J\. Glass, J\. McAuley, L\. Zettlemoyer,et al\.\(2024\)LIMA: less is more for alignment\.InAdvances in Neural Information Processing Systems,Vol\.36\.Cited by:[§1](https://arxiv.org/html/2606.18307#S1.p1.1)\.
- E\. Zhu, D\. Jiang, Y\. Wang, X\. Li, J\. Cheng, Y\. Gu, Y\. Niu, A\. Zeng, J\. Tang, M\. Huang,et al\.\(2025\)Data\-efficient rlvr via off\-policy influence guidance\.arXiv preprint arXiv:2510\.26491\.Cited by:[§2\.1](https://arxiv.org/html/2606.18307#S2.SS1.p2.6)\.

## Appendix AImplementation Details

### A\.1Detailed Experimental Setup

##### Datasets and Continual Training Hyperparameters\.

The candidate SFT dataset for Olmo3\-7B\-Instruct\-SFT222[https://huggingface\.co/allenai/Olmo\-3\-7B\-Instruct\-SFT](https://huggingface.co/allenai/Olmo-3-7B-Instruct-SFT)is Dolci\-Instruct\-SFT\-No\-Tools333[https://huggingface\.co/datasets/allenai/Dolci\-Instruct\-SFT\-No\-Tools](https://huggingface.co/datasets/allenai/Dolci-Instruct-SFT-No-Tools), which contains 1\.92M samples\. For OpenR1\-Distill\-7B444[https://huggingface\.co/open\-r1/OpenR1\-Distill\-7B](https://huggingface.co/open-r1/OpenR1-Distill-7B), we use the Mixture\-of\-Thoughts555[https://huggingface\.co/datasets/open\-r1/Mixture\-of\-Thoughts](https://huggingface.co/datasets/open-r1/Mixture-of-Thoughts)dataset comprising 349k samples\. To prevent representation collapse and catastrophic forgetting during the continual SFT phase, we employ highly conservative training settings\. The batch size is set to262,144262\{,\}144tokens, and the peak learning rate is set to1​e−51e\\\!\-\\\!5with a cosine decay schedule\. We enable sequence packing up to 32,768 tokens\. Under these settings, Olmo3\-7B\-Instruct\-SFT is trained for 500 steps, and OpenR1\-Distill\-7B is trained for 1,000 steps\.

##### Validation Set Construction\.

To guide the data attribution process, we construct a validation set that comprehensively covers four core capabilities: math, code, knowledge, and reasoning\. The selection of validation datasets follows a strict logic: we choose tasks where the base models’ capabilities are neither saturated \(e\.g\., excluding GSM8K\) nor near random guessing \(e\.g\., excluding GPQA\), ensuring the performance is not hard\-capped by the models’ inherent limits\. Furthermore, we prioritize reasoning\-heavy tasks over pure factual memorization \(excluding standard MMLU and PopQA\), as our preliminary experiments revealed that using memorization\-centric validation sets yields negligible downstream performance changes across all selection methods\. Specifically, we use the official validation sets for Minerva MATH and MMLU\-Pro\. For BBH, we extract the demonstrations typically used during evaluation to serve as validation queries\. For code, we select problems from the original MBPP dataset that do not overlap with the MBPP\+ evaluation set\. For all other baselines requiring a validation set or a task\-oriented seed dataset, we strictly use the original external reference responses, where Minerva MATH, MMLU\-Pro, and BBH naturally include Chain\-of\-Thought \(CoT\) trajectories\.

##### Validation Rollout Generation\.

For our proposed DRIFT, we generate on\-policy rollouts for the validation queries using sampling with temperaturet=0\.6t=0\.6and top\-pp=0\.95p=0\.95\. Olmo3\-7B\-Instruct\-SFT generates3232rollouts for each question, while OpenR1\-Distill\-7B generates88rollouts\. The correctness of these rollouts is automatically verified using released task\-specific evaluators\. We reuse the corresponding LightEval evaluation code for the reasoning/knowledge tasks when available, and the standard official checker for code tasks\.

##### Evaluation Configurations\.

For the decoding strategies during evaluation, BBH, MATH, and MMLU\-Pro are evaluated using greedy decoding\. All other benchmarks are evaluated using sampling with temperaturet=0\.6t=0\.6andp=0\.95p=0\.95\. To ensure robust evaluation on sampling\-based benchmarks, we report the average scores over multiple runs: avg@8 for MBPP\+, avg@4 for ZebraLogic, LCB v5, and OlympiadBench, and avg@16 for GPQA Diamond\.

### A\.2Baseline Implementations

To ensure a fair comparison, the validation sets used for BM25, DSIR, and RDS were identical to those employed in the main experiment\. Validation sets from different sources were scored separately and then averaged to prevent the algorithm from overly favoring validation sets with larger sample sizes\.

##### Random\.

As a lower bound for data selection, this baseline uniformly samples 10% of the candidate SFT dataset without replacement\.

##### Self\-Distillation\.

To ablate the effect of training on self\-generated text, this baseline directly fine\-tunes the base model on the generated positive on\-policy rollouts, bypassing the data attribution and selection process entirely\.

##### BM25\.

The BM25 data filtering method ranks documents by integrating term frequency saturation, inverse document frequency, and document length normalization\. We implement the BM25 baseline with the bm25s\[Lù,[2024](https://arxiv.org/html/2606.18307#bib.bib73)\]package\. For each validation set sample, we assign a linear score between 0 and 1 to the training samples based on their BM25 similarity ranking, and then obtain the aggregated score by summing them up\.

##### DSIR\.

We instantiate the Data Selection with Importance Resampling \(DSIR\)\[Xieet al\.,[2023b](https://arxiv.org/html/2606.18307#bib.bib3)\]framework with hashed n\-gram features for tractable importance weight estimation at scale\. The framework maps the raw and target data onto a feature space and resamples a subset of raw data according to importance weights computed in this space\.

##### RDS\.

We implement RDS\+\[Ivisonet al\.,[2025](https://arxiv.org/html/2606.18307#bib.bib7)\], which encodes samples using the hidden states of a pretrained causal language model and retrieves them by cosine similarity against evaluation queries\. For each training set sample, we aggregate the score by taking the maximum RDS similarity with the validation set samples\.

##### Qurating\.

We implement Qurating\[Wettiget al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib4)\], which scores each training example with a pretrained QuRater sequence\-classification model\. For each conversation, we concatenate all message contents, tokenize the resulting text, and split sequences that exceed the model context window into fixed\-length chunks\. The QuRater model predicts a vector of scores for every chunk, and we aggregate chunk\-level outputs into an example\-level score by taking a length\-weighted average over all chunks\. We use the officially released scoring tool, and the final score is the average of the four proposed intrinsic quality scorers\.

##### LESS\.

We re\-implemented LESS\[Xiaet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib6)\]within our framework\. The original method is implemented in a resource\-constrained setting where the warm\-up training is launched with LoRA without long sequence packing\. To ensure an adequate comparison, we keep the warm\-up training setting of LESS equivalent to our final training configuration\. The key difference between LESS and our method lies in the choice of the validation set \(LESS uses external references\) and the aggregation strategy of influence outputs \(LESS uses max cosine similarity among subtasks, whereas we use log\-space orthogonalization\)\.

##### Standard IF \(Impl\. w\. GraSS\)\.

This baseline represents the standard influence function usage\. It is implemented with the exact same GraSS sparse projection infrastructure\[Muhamedet al\.,[2024](https://arxiv.org/html/2606.18307#bib.bib8)\]as our method\. The critical distinction is that it computes scores against static, external validation sets \(the original reference responses\) and uses standard cosine similarity without our proposed log\-space debiasing\.

## Appendix BMotivating Analysis of Pure On\-Policy Targets

### B\.1Theoretical Justification of the Proximity Gap Mitigation

In this section, we provide the formal proof for Theorem[1](https://arxiv.org/html/2606.18307#Thmtheorem1), demonstrating why on\-policy validation targets structurally mitigate the proximity gap compared to off\-policy targets through the lens of Bias\-Variance decomposition\.

##### Background and PBRF Formulation\.

As established byBaeet al\.\[[2022](https://arxiv.org/html/2606.18307#bib.bib2)\], Influence Functions approximate the Proximal Bregman Response Function \(PBRF\)\. The PBRF objective for a validation targetzv​a​l=\(x,y\)z\_\{val\}=\(x,y\)is defined as:

θP​B​R​F=arg⁡minθ⁡\(L​\(zv​a​l;θ\)\+12​τ​‖θ−θ∗‖Hθ∗2\),\\theta\_\{PBRF\}=\\arg\\min\_\{\\theta\}\\left\(L\(z\_\{val\};\\theta\)\+\\frac\{1\}\{2\\tau\}\\\|\\theta\-\\theta^\{\*\}\\\|\_\{H\_\{\\theta^\{\*\}\}\}^\{2\}\\right\),\(7\)whereL​\(zv​a​l;θ\)=−log⁡πθ​\(y\|x\)L\(z\_\{val\};\\theta\)=\-\\log\\pi\_\{\\theta\}\(y\|x\)is the Cross\-Entropy loss,θ∗\\theta^\{\*\}is the converged model parameters, andHθ∗H\_\{\\theta^\{\*\}\}is the Hessian matrix\. Taking the derivative with respect toθ\\thetaand setting it to zero, the first\-order parameter shift induced by the validation target is:

Δ​θP​B​R​F=θP​B​R​F−θ∗≈−τ​Hθ∗−1​∇θL​\(zv​a​l;θ∗\)\.\\Delta\\theta\_\{PBRF\}=\\theta\_\{PBRF\}\-\\theta^\{\*\}\\approx\-\\tau H\_\{\\theta^\{\*\}\}^\{\-1\}\\nabla\_\{\\theta\}L\(z\_\{val\};\\theta^\{\*\}\)\.\(8\)The Taylor expansion error in Influence Functions scales with the expected squared norm of this shift, i\.e\.,𝒪​\(𝔼​\[‖Δ​θP​B​R​F‖2\]\)\\mathcal\{O\}\(\\mathbb\{E\}\[\\\|\\Delta\\theta\_\{PBRF\}\\\|^\{2\}\]\)\.

##### Proof of Theorem 1\.

Let us analyze the expected parameter shift when the validation target is drawn from the model’s own policy \(on\-policy\)\. For LLMs, the outputyo​ny^\{on\}is a sequence of discrete tokens sampled fromπθ∗\(⋅\|x\)\\pi\_\{\\theta^\{\*\}\}\(\\cdot\|x\)\. The expected gradient of the loss evaluated atθ∗\\theta^\{\*\}is:

𝔼yo​n∼πθ∗​\[∇θL​\(x,yo​n;θ∗\)\]=−𝔼yo​n∼πθ∗​\[∇θlog⁡πθ∗​\(yo​n\|x\)\]\.\\mathbb\{E\}\_\{y^\{on\}\\sim\\pi\_\{\\theta^\{\*\}\}\}\[\\nabla\_\{\\theta\}L\(x,y^\{on\};\\theta^\{\*\}\)\]=\-\\mathbb\{E\}\_\{y^\{on\}\\sim\\pi\_\{\\theta^\{\*\}\}\}\[\\nabla\_\{\\theta\}\\log\\pi\_\{\\theta^\{\*\}\}\(y^\{on\}\|x\)\]\.\(9\)Using the score function identity \(the REINFORCE trick\), we can rewrite this expectation over the discrete output space𝒴\\mathcal\{Y\}:

𝔼yo​n∼πθ∗​\[∇θlog⁡πθ∗​\(yo​n\|x\)\]\\displaystyle\\mathbb\{E\}\_\{y^\{on\}\\sim\\pi\_\{\\theta^\{\*\}\}\}\[\\nabla\_\{\\theta\}\\log\\pi\_\{\\theta^\{\*\}\}\(y^\{on\}\|x\)\]=∑y∈𝒴πθ∗​\(y\|x\)​∇θπθ∗​\(y\|x\)πθ∗​\(y\|x\)\\displaystyle=\\sum\_\{y\\in\\mathcal\{Y\}\}\\pi\_\{\\theta^\{\*\}\}\(y\|x\)\\frac\{\\nabla\_\{\\theta\}\\pi\_\{\\theta^\{\*\}\}\(y\|x\)\}\{\\pi\_\{\\theta^\{\*\}\}\(y\|x\)\}\(10\)=∑y∈𝒴∇θπθ∗​\(y\|x\)\\displaystyle=\\sum\_\{y\\in\\mathcal\{Y\}\}\\nabla\_\{\\theta\}\\pi\_\{\\theta^\{\*\}\}\(y\|x\)=∇θ​∑y∈𝒴πθ∗​\(y\|x\)\\displaystyle=\\nabla\_\{\\theta\}\\sum\_\{y\\in\\mathcal\{Y\}\}\\pi\_\{\\theta^\{\*\}\}\(y\|x\)=∇θ\(1\)=𝟎\.\\displaystyle=\\nabla\_\{\\theta\}\(1\)=\\mathbf\{0\}\.Because the Hessian matrixHθ∗H\_\{\\theta^\{\*\}\}is computed over the training corpus and fixed atθ∗\\theta^\{\*\}, it is independent of the current validation targetyo​ny^\{on\}\. By the linearity of expectation, we can pass the expectation through the inverse Hessian:

𝔼yo​n∼πθ∗​\[Δ​θP​B​R​Fo​n\]=−τ​Hθ∗−1​𝔼yo​n∼πθ∗​\[∇θL​\(x,yo​n;θ∗\)\]=𝟎\.\\mathbb\{E\}\_\{y^\{on\}\\sim\\pi\_\{\\theta^\{\*\}\}\}\[\\Delta\\theta\_\{PBRF\}^\{on\}\]=\-\\tau H\_\{\\theta^\{\*\}\}^\{\-1\}\\mathbb\{E\}\_\{y^\{on\}\\sim\\pi\_\{\\theta^\{\*\}\}\}\[\\nabla\_\{\\theta\}L\(x,y^\{on\};\\theta^\{\*\}\)\]=\\mathbf\{0\}\.\(11\)This concludes the proof of Theorem 1\.

##### Bias\-Variance Decomposition and the Proximity Gap\.

To fully understand the proximity gap, we apply the Bias\-Variance decomposition to the expected squared norm of the parameter shift:

𝔼​\[‖Δ​θP​B​R​F‖2\]=‖𝔼​\[Δ​θP​B​R​F\]‖2⏟Bias2\+Tr​\(Cov​\(Δ​θP​B​R​F\)\)⏟Variance\.\\mathbb\{E\}\[\\\|\\Delta\\theta\_\{PBRF\}\\\|^\{2\}\]=\\underbrace\{\\\|\\mathbb\{E\}\[\\Delta\\theta\_\{PBRF\}\]\\\|^\{2\}\}\_\{\\text\{Bias\}^\{2\}\}\+\\underbrace\{\\text\{Tr\}\(\\text\{Cov\}\(\\Delta\\theta\_\{PBRF\}\)\)\}\_\{\\text\{Variance\}\}\.\(12\)
For On\-Policy Targets:As proven above, the Bias term is strictly zero\. The error is entirely bounded by the Variance term\. Furthermore, the covariance of the gradients is exactly the Fisher Information Matrix \(FIM\), denoted asℐθ∗\\mathcal\{I\}\_\{\\theta^\{\*\}\}\. Under the common Gauss\-Newton approximation whereHθ∗≈ℐθ∗H\_\{\\theta^\{\*\}\}\\approx\\mathcal\{I\}\_\{\\theta^\{\*\}\}, the variance term is structurally constrained by the local geometry of the model, ensuring the shift remains microscopic\.

For Off\-Policy Targets:Ifyo​f​fy^\{off\}is drawn from an external dataset𝒟e​x​t\\mathcal\{D\}\_\{ext\}, the expected gradient𝔼yo​f​f​\[∇θL\]=𝐠o​f​f≠𝟎\\mathbb\{E\}\_\{y^\{off\}\}\[\\nabla\_\{\\theta\}L\]=\\mathbf\{g\}\_\{off\}\\neq\\mathbf\{0\}\. Consequently, the Bias term becomes‖τ​Hθ∗−1​𝐠o​f​f‖2≫0\\\|\\tau H\_\{\\theta^\{\*\}\}^\{\-1\}\\mathbf\{g\}\_\{off\}\\\|^\{2\}\\gg 0\. This macroscopic first\-order bias shift dominates the expected squared norm, severely breaking the local Taylor approximation and causing the massive proximity gap observed in practice\.

### B\.2Empirical Justification of the Proximity Gap Mitigation

##### Additional Locality Measurements\.

Besides the parameter\-space curves in Figure[1](https://arxiv.org/html/2606.18307#S2.F1), we also measure the norm of the aggregated validation centroid in projected\-gradient space,

c=∑zv​a​lα​\(zv​a​l\)​ϕ​\(zv​a​l;θ∗\),c=\\sum\_\{z\_\{val\}\}\\alpha\(z\_\{val\}\)\\,\\phi\(z\_\{val\};\\theta^\{\*\}\),\(13\)whereα​\(zv​a​l\)\\alpha\(z\_\{val\}\)denotes the validation weight assigned to each target\. For pure on\-policy targets,α\\alphais uniform and‖c‖\\\|c\\\|is the empirical counterpart of‖𝔼​\[∇θL\]‖\\\|\\mathbb\{E\}\[\\nabla\_\{\\theta\}L\]\\\|, which Theorem[1](https://arxiv.org/html/2606.18307#Thmtheorem1)predicts to vanish in the population limit\. Table[5](https://arxiv.org/html/2606.18307#A2.T5)empirically confirms that under both on\-policy constructions,‖c‖\\\|c\\\|is orders of magnitude smaller than under off\-policy references, despite being estimated from only a finite number of rollouts\. Note that the signed weighting used by DRIFT strictly speaking falls outside the pure on\-policy condition of Theorem[1](https://arxiv.org/html/2606.18307#Thmtheorem1)\. Nonetheless, the measured centroid norm remains comparably small, suggesting that on a converged model the positive and negative rollout gradients largely cancel out, so the proximity\-gap mitigation observed for pure on\-policy targets is preserved in practice\.

Table 5:Norm of the aggregated validation centroid under different target constructions\. On\-policy targets produce much smaller validation anchors than external off\-policy references\.ModelOff\-PolicyOn\-Policy w/o WeightingOn\-Policy w/ Weighting \(Ours\)Olmo35\.4950\.1230\.092OpenR10\.3370\.0070\.008
##### Proximity Metrics\.

For Figure[1](https://arxiv.org/html/2606.18307#S2.F1), we compare each continual\-SFT trajectory against the uniform weighting baseline at the same training step\. Sparsity is defined as the fraction of parameters whose absolute change is smaller than10−510^\{\-5\}\. Using thresholds of10−610^\{\-6\},10−710^\{\-7\}, or10−810^\{\-8\}yields the same qualitative trend\.

## Appendix CMethod Details

##### Gradient Debiasing\.

As introduced in Section[3](https://arxiv.org/html/2606.18307#S3), we perform a log\-space orthogonalization to decouple the raw influence scores from the gradient magnitude confounders\. Because the debiasing process essentially involves subtracting the scaled log\-norm from the log\-score \(which is mathematically equivalent to dividing by‖ϕ‖β\\\|\\phi\\\|^\{\\beta\}\), samples with extremely small gradient norms can both destabilize the log\-space linear regression and inflate the resulting debiased scores\. Therefore, we mask out the bottom 10% tail by gradient norm before both fittingβ\\betaand computing the final debiased scores\. Table[6](https://arxiv.org/html/2606.18307#A3.T6)reports the task\-specific slopeβ\\betaextracted during the log\-space linear fitting\. The fittedβ\\betavaries substantially across models, validation sources, and domains, and frequently deviates from1\.01\.0\(the implicit assumption behind standard Cosine similarity\), supporting a per\-task orthogonalization strategy\.

Table 6:Empiricalβ\\betavalues extracted during log\-space orthogonalization across different models, validation sources, and domains\.Validation DomainOlmo3\-7B\-Instruct\-SFTOpenR1\-Distill\-7BOff\-PolicyOn\-PolicyOff\-PolicyOn\-PolicyMATH1\.592\.211\.170\.92BBH1\.512\.051\.221\.24MBPP1\.472\.301\.140\.98MMLU Pro1\.482\.001\.221\.16

## Appendix DLimitations

While DRIFT demonstrates significant improvements in refining fully\-trained LLMs, we acknowledge several limitations in our current study that warrant future investigation:

##### Scope of Influence Approximations\.

Our experiments primarily validate the rollout\-driven attribution framework using the GraSS sparse random projection infrastructure\. We have not extensively evaluated how our method interacts with other families of influence estimation algorithms, such as structural Hessian approximations \(e\.g\., EK\-FAC\) or dense first\-order dot products\. The generalizability of on\-policy targets across different mathematical approximations of Influence Functions remains to be comprehensively tested\.

##### Theoretical Origins of Gradient Norm Bias\.

We empirically identified that the relationship between raw influence scores and gradient norms in LLMs follows a complex power law, and we proposed a log\-space orthogonalization strategy as an effective intervention\. However, the fundamental geometric or optimization\-related reasons why high\-dimensional LLM gradients exhibit this specific non\-linear bias remain opaque\. A deeper theoretical investigation into the loss landscape and gradient distribution of LLMs is required to fully explain this phenomenon\.

##### Theoretical Guarantees for the Proximity Gap\.

While Theorem[1](https://arxiv.org/html/2606.18307#Thmtheorem1)and Appendix[B\.1](https://arxiv.org/html/2606.18307#A2.SS1)provide a formal proof that theexpectedfirst\-order parameter shift induced by on\-policy targets is strictly zero, this analysis relies on the PBRF framework and the score function identity\. Bounding the exact finite\-sample variance in practice, as well as rigorously quantifying the impact of higher\-order Taylor expansion terms in highly non\-convex LLM loss landscapes, remains an open challenge\. Establishing tighter theoretical bounds for these finite\-sample empirical shifts is a valuable direction for future research\.

##### Completeness of Anomaly Filtering\.

Although our per\-task orthogonalization strategy significantly reduces the impact of gradient magnitude confounders and successfully filters out many pathological samples, it is not a silver bullet\. Qualitative observations indicate that a small fraction of ill\-formed or noisy training instances can still bypass the debiasing mechanism and receive high attribution scores\. This suggests that gradient norm is not the sole confounder in high\-dimensional influence estimation, and more advanced debiasing techniques may be necessary for perfect data curation\.

## Appendix EMore Qualitative Cases

Table 7:Ranking of two high grad\-norm ill training samples\.Ill\-formed Training SamplePrompt:Which condiment was used as medicine during the 1830s?Response:Ketchup\.wdgywdufqtufdufwytdfwfdytfwtydvutwqfdytfwMethod Rankings \(Percentile\)No Debiasing: Top 0\.037%Packing Correction: Top 0\.008%Cosine: Top 2\.779%Per\-Task Orth\.: Top 33\.994%
Ill\-formed Training SamplePrompt:4: Reer Baniqureeda goortii ay duldageen Saxaabada oo ay u timid Cabsi daran Ninkii la dhihijiray Kacab ibnu Asad oo ahaa Madaxdoodii ayaa wuu u soo jeediyay 3 arimood oo uu yiraahday mid kaqaato sadaxdaas, ee sheeg Sadaxdaas?\(Translation: When the Quraysh realized that the Companions had arrived in Abyssinia and were under protection, they felt great anxiety\. Their leader, Amr ibn al\-As, proposed three arguments to the King and asked him to accept one of them\. What were those three?\)Response:14: ……………\.\.?Method Rankings \(Percentile\)No Debiasing: Top 0\.056%Packing Correction: Top 7\.107%Cosine: Top 2\.177%Per\-Task Orth\.: Top 37\.743%

Qualitative Example\[Validation Query\]
Letf​\(x\)=\{a​x\+3,if​x\>2x−5if−2≤x≤22​x−bif​x<−2f\(x\)=\\begin\{cases\}ax\+3,&\\text\{if \}x\>2\\\\ x\-5&\\text\{if \}\-2\\leq x\\leq 2\\\\ 2x\-b&\\text\{if \}x<\-2\\end\{cases\}\. Finda\+ba\+bif the piecewise function is continuous\.\[Top\-1 Retrieved Training Sample by RDS\]
Prompt:For what range of positive real values ofccdoes the inequalityx2−6​x\+c<0x^\{2\}\-6x\+c<0have real roots forxx? Express your answer using interval notation\.
Response:For real roots, the discriminant conditionb2−4​a​c≥0b^\{2\}\-4ac\\geq 0witha=1,b=−6a=1,b=\-6yields36−4​c≥0⇒c≤936\-4c\\geq 0\\Rightarrow c\\leq 9\. For the inequalityx2−6​x\+c<0x^\{2\}\-6x\+c<0to hold, the quadratic must have a maximum \(sincea=1\>0a=1\>0\) less than zero atx=3x=3\. Substitutingx=3x=3givesc−9<0⇒c<9c\-9<0\\Rightarrow c<9\. Combining constraints, the range of positive real values ofccisc∈\(0,9\)c\\in\(0,9\)\.\[Top\-1 Retrieved Training Sample by BM25\]
Prompt:A high school teacher models home tranquility with mathematical functions… 1\. Define a functionf​\(x\)f\(x\)that is continuous and differentiable over\[−2,2\]\[\-2,2\]and satisfies:f​\(0\)=0f\(0\)=0,f′​\(x\)=f​\(x\)2−1f^\{\\prime\}\(x\)=f\(x\)^\{2\}\-1, andf​\(x\)f\(x\)has exactly one local maximum and one local minimum within the interval\. Determine all possible functionsf​\(x\)f\(x\)…
Response:To solve the problem, given the differential equationf′​\(x\)=f​\(x\)2−1f^\{\\prime\}\(x\)=f\(x\)^\{2\}\-1withf​\(0\)=0f\(0\)=0, solving yieldsf​\(x\)=1−e2​x1\+e2​xf\(x\)=\\frac\{1\-e^\{2x\}\}\{1\+e^\{2x\}\}\. For the piecewise functiong​\(x\)g\(x\), continuity atx=0x=0requiresf​\(0\)=bf\(0\)=b, givingb=0b=0\. Differentiability atx=0x=0requiresf′​\(0\)=af^\{\\prime\}\(0\)=a\. Using the differential equation,f′​\(0\)=02−1=−1f^\{\\prime\}\(0\)=0^\{2\}\-1=\-1, soa=−1a=\-1\. Final Answer: The functionf​\(x\)=1−e2​x1\+e2​xf\(x\)=\\frac\{1\-e^\{2x\}\}\{1\+e^\{2x\}\}, and the constantsa=−1a=\-1andb=0b=0\.\[Top\-1 Retrieved Training Sample by LESS\]
Prompt:Given an integer N \(1≤N≤10 0001\\leq N\\leq 10\\,000\), output the N\-th digit ofπ\\piwithout the decimal point \(314159…\)… Input: A single line containing N\. Output: Print the N\-th digit… Example: Input 1 → Output 3 \| Input 2 → Output 1…
Response:def main\(\): import sys \# First 10 000 digits ofπ\\piwithout the decimal point pi\_str = \( "314159265358979323846\.\.\." \) N\_line = sys\.stdin\.readline\(\)\.strip\(\) if not N\_line: return N = int\(N\_line\) \# Output the N\-th digit \(1\-based indexing\) print\(pi\_str\[N \- 1\]\)if \_\_name\_\_ == "\_\_main\_\_": main\(\)\[Top\-1 Retrieved Training Sample by DRIFT \]
Prompt:As an experienced mainframe operator, you are explaining the resource allocation system… The mainframe’s processing power is represented by the polynomial functionP​\(x\)=2​x3−5​x2\+4​x−3P\(x\)=2x^\{3\}\-5x^\{2\}\+4x\-3… 1\. The mainframe requires a minimum processing power of 10 units… Solve the inequalityP​\(x\)≥10P\(x\)\\geq 10…
Response:To solve the given problem, let’s tackle each part separately\. Part 1: Solve the Inequality… we rewrite it as2​x3−5​x2\+4​x−13≥02x^\{3\}\-5x^\{2\}\+4x\-13\\geq 0\. Now, we need to find the roots of the equation… By testing values from these intervals, we find that the inequality holds for\(x3,x1\)∪\(x1,∞\)\(x\_\{3\},x\_\{1\}\)\\cup\(x\_\{1\},\\infty\)… The degree of the polynomialQ​\(x\)Q\(x\)is 4\.Figure 4:Top\-1 retrieval performance of different filtering methods