Tag
论文提出 OptEP，一种面向混合专家（MoE）语言模型的后训练专家剪枝方法，通过连续松弛优化专家选择变量并结合重构驱动的权重更新，在 Qwen3、GLM-4.5-Air、Mixtral 等模型上显著优于现有剪枝基线，激进剪枝下精度提升最高达 16.20%。
论文提出 Activation-Space Exploration (ASE)，通过聚类正确与错误回答的激活差异构建潜空间探索向量库，以多样化 RLVR 训练中的 rollout；在 Qwen3-1.7B 和 Qwen3-8B 上较 GRPO 分别提升 4.87 和 1.87 个百分点的 Mean@8。
A Purdue paper introduces a bounded, online step-level repair module for Lean 4 theorem provers that feeds compiler error messages to a pretrained repair model instead of discarding failed tactics, achieving up to a 32% relative gain in solve rate on the LeanDojo benchmark and nearly doubling performance on Lean Workbook.
本文提出 CA-MOPD，通过合并强化学习专家并利用验证器估算的师生能力差距来调整多教师在线蒸馏的监督强度，在数学、代码生成和指令遵循六个基准上超越现有 MOPD 方法。
该论文研究了在均匀扩散语言模型（UDLM）中循环早期 Transformer 层的方法，在不改变训练目标和采样器的前提下，以显著更少的训练 FLOPs 达到相同的验证 NLL，并在推理时降低生成困惑度。
PACE 是一种免微调的后训练四比特量化器，用于线性注意力循环状态，通过降低量化频率（每隔几个 token 才舍入一次）并采用列对齐的量化尺度来减少误差，从而在 4B-35B 参数混合模型上以约三分之二的内存达到与八比特 DAMP 相当的性能。
ALTK-Evolve introduces a learned-memory approach that tops the τ²-bench leaderboard on customer service tasks, achieving state-of-the-art results without relying on a frontier model.
ProVe reverses speculative decoding roles by fine-tuning a large model to propose token blocks verified by a smaller or identical model, achieving up to 2.7x throughput with adjustable speed-accuracy tradeoffs across six benchmarks.
The paper builds teams of persistent, discipline-diverse AI researcher agents with private histories and adaptive allocation of experimental budget to autonomously discover linear attention architectures, outperforming GD-NS at 1.3B scale on language modeling and retrieval tasks.
The paper introduces calibrated accommodation pairs (CAP), a preference dataset that rewards chat models for matching users' dialect while preserving Standard English, and audits how SFT, DPO, and RL stages change a model's tendency to accommodate informal dialect cues across 10 open model lineages.
The paper presents Lafa, an 8B-parameter blockwise continuous flow language model trained with under 30B tokens that matches or surpasses autoregressive quality while achieving 2.54x inference throughput via self-speculative decoding and blockwise parallel decoding.
PAR (Progress-Aligned Recurrence) aligns each loop iteration of a looped language model with a step of the task and rescales hidden states to prevent drift, enabling 16× length extrapolation on swap-S5 state tracking with a 0.5B model reaching 95.5% accuracy, far surpassing RLT and CoT baselines.
该论文因果性地追踪了导致模型「跟随提示线索而放弃正确答案」（sycophancy/caving）的注意力头，发现多种偏差共享同一个在预训练阶段就存在的线索路由电路，而微调与RL后训练只是调节该电路的激活强度与时机。
Yale researchers use influence functions to identify indirect evidence in language model training data across 15 BLiMP grammatical phenomena, showing that removing the top influential sentences lowers held-out accuracy significantly more than removing random controls, and characterizing how this evidence relates to linguistic structure.
论文提出 Causal Edit Serialization (CES) 框架，通过特殊指针令牌与双索引 RoPE，在标准因果注意力掩码下实现插入、删除、替换、复制和移动等局部化文本编辑，并在文法纠错任务上验证了监督与强化学习后训练的可行性。
论文提出“watermarkability”指标来量化语言模型与基于logit的水印规则的兼容性，并通过两阶段方法（目标集条件LoRA加闭环控制器）在保持高水印可靠性的同时显著降低困惑度增加。
提出 First Passage Attention (FPA)，将注意力遗忘机制从路径求和改为路径极值，包含 Forgetting Transformer 作为特例，并在 110M 语言模型上实现 8 倍训练长度的完美 passkey 检索。
The paper introduces ToMStories, an evaluation dataset of ~6,000 synthetic narrative vignettes for testing theory-of-mind inference in 13 open-weight language models, finding that large models match humans on multiple-choice but not open-ended questions and tend to oversimplify implicit psychological states.
The paper identifies user representations as a key bottleneck in user simulation and introduces hindsight distillation, an online training procedure, to improve them. The resulting UserRepGen-4B model generates representations that enable more faithful simulation than those produced by GPT-5.5 across seven datasets.
该论文研究语言模型如何学习动作名词化结构（ANCs），通过构建 96 种参数化的人工语言并从头训练 GPT-2-small 模型，发现模型学习偏好在基本从句配置上与自然语言类型学分布一致，但在补语系统和 ANC 策略上出现分歧。