Tag
The paper formalizes non-local data attribution in online reinforcement learning via a trajectory-level leave-one-out quantity (RL-LOO) and derives RL-IF, a first-order estimator of data influence, which is then used to build RINSE, a practical data filtering method that improves training efficiency and final performance over PPO on control benchmarks and an RLHF toxicity-mitigation task.
提出了 MOSAIC（Multimodal Orthogonalized Sparse Autoencoders for Influence Characterization），通过共享的稀疏图文潜空间与 EK-FAC 分解，实现视觉语言模型中训练样本到具体文本短语和图像区域的补丁级归因，并在 Qwen 和 Gemma 的多项任务上验证了其有效性。
This paper derives a unified token- and layer-wise decomposition of how learning from one token changes another prediction, connecting data attribution, catastrophic forgetting, and plasticity loss as distinct regimes of the same evolving update-behavior interaction. It yields forward-computable approximations enabling data selection, interference controls, and diagnostics for future learnability across models and training regimes.
This paper resolves a debate on gradient-based data attribution methods for large language models by demonstrating that they primarily track answer format rather than task semantics, challenging their reliability in targeted instruction tuning.
Ramp developed a semantic layer to attribute AI agent spend to objectives and outcomes, moving from monitoring spend to understanding AI ROI.
This paper demonstrates that a small coordinated Wikipedia editing campaign can measurably shape how language models handle topics, using animal welfare as a case study.
DRIFT proposes a method that uses on-policy influence functions to refine training data distribution for supervised fine-tuning of large language models, consistently improving performance ceilings over existing baselines.
The paper proposes NULLs (Natively Unlearnable LLMs), a model class that isolates source-specific contributions in sparsely activated sinks while sharing backbone neurons, enabling clean unlearning of individual data sources without retraining and preserving general language capabilities.
GRASP introduces a geometry-aware, interaction-based method for scalable pretraining data attribution that models subset dynamics, outperforming existing additive approaches by over double the task-level rank correlation while reducing computation costs.
This paper provides the first systematic analysis of error sources in trajectory-based data attribution methods, identifies optimizer mismatch as the dominant error, proposes AdamW-influence to address it, and offers practical guidelines for data selection via a K-step look-ahead framework.