language-models

norregel 是一个预注册基准，用于测试语言模型能否给出挪威法律中带日期的数值（金额、百分比、年龄限制等）。第二轮预注册实验显示模型对训练截止后生效的数值更易给出过时答案，但该效应在敏感性分析下并不稳健，且部分模型（如 Haiku 4.5）答案大多不可解读。

论文揭示连续扩散语言模型（如ELF）因生成过程缺少逐步离散采样步骤而导致长尾词汇严重缺失（稀有词生成率仅为人类文本的0.30×），并提出训练无关的GRAFT方法，通过在模型未确定时提交稀有词再重新生成序列，将稀有词比例恢复至人类水平的0.82–1.10×。

This paper shows that regenerating target model responses shrinks the gains that draft-training objectives like knowledge distillation provide in speculative decoding, replicating findings across vision-language (Qwen3-VL-8B) and text (Qwen3-4B) settings with multiple controls.

研究发现语义泄漏（semantic leakage）不仅源于概念与实体的直接归属，即便概念只是出现在场景中也会驱动模型生成刻板印象输出，这揭示了一种当前性别偏见评估所忽略的联想性偏见机制。

The paper introduces Common-Support Distillation (CSD), a unified logits-based framework for cross-tokenizer knowledge transfer that constructs a shared support between teacher and student and matches induced distributions with a single objective, outperforming existing cross-tokenizer distillation methods across multiple teacher-student settings.

LIFT presents a new instruction-following benchmark (LIFT-Bench) and training counterpart (LIFT-Train) using scoped constraints on structured document outlines, showing that frontier models struggle on harder subsets and that training with LIFT-Train is less susceptible to reward hacking than existing IFTrain approaches.

This paper identifies and formalizes 'inference-time model shift'—a change in conformal score distributions caused by varying inference configurations like quantization and temperature rather than data distribution changes—showing it breaks split-conformal exchangeability and overstates deployment-time coverage across 8 language models and 4 finite-label benchmarks.

论文提出 Skill Schema Transport (SST) 框架，通过将新领域知识重铸为模型已有技能空间中的可复用程序化 schema 来缓解持续领域适应中的灾难性遗忘，在两个 3B 模型上将 Qwen MATH 流的最大遗忘从 42.41 降至 7.10 个点。

This paper measures exactly how much weight gradient is lost when training looped transformers via truncated backpropagation across nine models, finding that the lost gradient is largely set near the output, diminishes during fine-tuning, and does not clearly outperform measurement-free cost heuristics like Parcae's rule.

本文提出 Discord 与 BRACE 两种方法，利用奖励分歧来校准 MCP 环境中工具使用语言模型的强化学习 rollout 预算分配，并在 Qwen3-VL 多个规模及 BFCL、NESTFUL 基准上验证其优于基线。

文章介绍了一种名为"Next-Concept Modeling"的新范式，主张生成式AI应先生成抽象概念、再生成文字，而非直接以词为输出单位，有望改变现有生成式AI的建模方式。

This paper evaluates small recurrent-depth language models (Ouro-1.4B/2.6B-Thinking, HRM-Text-1B) fine-tuned jointly across 13 clinical NLP datasets, finding they outperform or match similarly sized Qwen3 transformers on most tasks, especially inference and question answering.

本文评估语言模型智能体的上下文压缩方法在'后续问题'上的表现，发现针对当前问题选择的句子在后续问题上并不优于随机删除，而 LLMingua-2 等词级压缩方法在多个基准上表现更佳。

该论文提出了一种新的Jacobian lens变体——entity lens，可在不使用关系信息的情况下预测线性关系可解码性（LRE faithfulness），在OLMo-3-7B的28个关系上取得ρ=0.7355的相关性，表明线性可解码性更多是主体表示本身的属性而非关系的属性。

论文发现语言模型在组合多个任务时错误率显著上升，且无法完全用独立错误累积解释，而是源于隐式的思考预算分配——组合任务时每个任务获得的推理 token 减少，导致长推理任务更脆弱；通过按脆弱性分组可减少 43% token 用量并保持准确率。

H2DF introduces a calibrated target-band objective for language-model unlearning that limits gradient-ascent pressure once forget loss reaches a data-dependent target range, matching GradDiff performance on TOFU while using far less retain replay and improving retention over GA and NPO on MUSE-News.

This paper introduces 'decision sufficiency' as a framework for evaluating whether learned representation interfaces, such as selected sparse autoencoder features, retain enough information for a controller to safely steer language models, and audits Qwen2.5, SmolLM2, and Phi showing measurable information loss in safety-critical decisions.

The paper proposes Guided-AP, a method that uses a few exact activation patching measurements to reveal and correct shared errors in attribution patching estimates, reducing error by up to 26% on single prompts and 75-79% for average effects across prompts in mechanistic interpretability analyses.

This paper introduces GREG, a Graph Reasoning Model that augments Language Reasoning Models with learnable, permutation-equivariant graph positional encodings computed via a GNN and Graph Transformer, eliminating the need for serialized graph descriptions and reducing context complexity to O(N). Trained on small graphs, it achieves size-generalization to graphs with up to 1,000 nodes while remaining robust to node relabelings.

Mass General Brigham researchers present Clinical System One (CS1-4B), a 4B-parameter decision model that performs fast, typed judgments over clinical records, outperforming open decision models up to 27B parameters on 13 of 14 tasks while running on a single desktop GPU. Paired with a generalist LLM agent, it enables longitudinal chart review and clinical trial screening with 690-fold reduced token usage.