robustness

LlamaIndex released BooBench, a playful 179-page Halloween-themed benchmark that 'scares' LLMs and document OCR tools into repeating characters, exposing failure modes where VLMs enter repetition loops on real-world documents with repeated elements like fill leaders and redaction marks.

提出了 Augmentation-Guided Adversarial Training (AGAT) 框架，利用基于增强的局部稳定性 (ALS) 得分选择不稳定邻域的样本进行对抗训练，在仅对 20–30% 样本施加对抗监督的情况下达到相当或更高的鲁棒准确率，同时降低训练成本。

The paper introduces a controlled benchmark of French surface variants (apostrophe omission, diacritic removal, keyboard typos) across seven tasks with 12,600 inputs, showing that accuracy drops and Brier score/NLL rise with perturbation intensity, while calibration combined with input restoration improves reliability.

This paper develops a direction-sensitive, readout-relative account of semantic fragility in language models, showing that heterogeneous task-preserving rewrites can share failure dynamics while lacking a common input-independent failure direction, validated on Qwen3-32B and Gemma 4 31B.

This paper audits residual activation interventions in Qwen3.5-27B on tool-use tasks, finding that while such edits boost target-selection accuracy, the gains fail to generalize to recombined facts and don't extend to complete-action correctness, separating target-selection control from condition-sensitive reasoning.

论文提出 C3L（Concept-Consistent Continual Learning），通过将显著性正则化锚定到离线发现的语义概念上，打破持续学习中的自指性解释漂移循环，并借助 LRP 相关性守恒产生"挤出效应"以抑制虚假相关，在五个基准上取得 SOTA 表现。

TRACE is a training-aware benchmark framework that reorganizes 168 text-vision embedding tasks into General, Adaptability, and Challenge categories, revealing that data adaptation improves adaptability but fails to solve challenging multilingual and visual-document tasks.

该论文提出了 MAMath-Bench，一个面向多智能体数学推理的过程感知基准，包含 1003 道跨 11 个数学类别与文本、图像、音频多模态的题目，从答案正确性、推理过程可靠性与对抗扰动鲁棒性三个维度进行评估，并发现多智能体协作并不总能提升最终准确率且易受对抗推理错误影响。

该论文提出有界续写分析(BCA),通过概率模型检查量化自回归语言模型中隐藏的有害生成概率,以补充传统的攻击成功率(ASR)指标,更好地揭示表面拒绝之下的分布脆弱性。

A Harvard and MIT team propose MPT (Market-Based Post-Training), which jointly learns action policies and self-orchestration through a bidding mechanism in a decentralized agent swarm — no central coordinator needed. Experiments on embodied creation, robot navigation, and deep research tasks validate its significant advantages in scaling, resilience to node failures, and task transfer.

该论文提出了一个受扰动控制的医疗LLM智能体基准测试，包含400个多轮episode，评测在缺失、过时、冲突和迟到信息下的升级/拒答决策能力。结果显示模型在干净环境下准确率达0.97，但在不确定性扰动下漏掉32-97%的必要升级，且基于词汇规则的控制器在未见过的同义改写下召回率降至0。

本文揭示了自我改进智能体中技能-策略共进化的脆弱性陷阱，提出部署导向的评估协议CODA来审计技能漂移的影响，发现最差情况下共进化增益会被削弱52-93%。

Matt Pocock shares that he consistently gets better robustness and easier, more testable future updates when AI coding agents build state machines or reducers, and recommends the practice for frontend development.

该论文提出 REASON 双循环推理框架，通过快速（System 1 式）和中速（System 2 式）循环分层解决残差误差，在分布偏移和未见扰动下以约一半计算量显著提升模型鲁棒性，并优于分级分类器和学习延迟方法。

提出 DecisionRobust 基准，系统评估 JEV 风格决策模型在文本与视觉输入下的对抗鲁棒性，涵盖文本/图像扰动与提示注入等攻击，揭示了模型鲁棒性的多项关键发现。

提出 UnCoBench 基准，研究 LLM 多智能体协作中面对不可靠队友（信息错位、不完整或未响应）时的恢复能力，发现相似的可靠场景性能并不意味着同等的鲁棒性。

BeliefScope 是一个受控黑盒框架，用于区分大语言模型在面对真实证据与无事实增量的定向用户压力时的信念变化来源，并在 Qwen/Llama 等 36 族模型上展示了其结论对评估上下文的依赖性。

该论文提出SSRFT（Supervised Safe-Role Fine-Tuning），首次将安全对齐重新定义为内化预定义安全角色的过程，而非显式拒绝模式，在多种模型上实现了比标准SFT更鲁棒、更可泛化的安全对齐效果，并降低了对良性查询的过度拒绝。

This paper introduces 'alignment–utility asymmetry,' showing that when LLMs adapt to rule-based semantic-preserving input transformations (e.g., letter swaps), task utility is largely retained while harmful-response rates spike sharply—such as GPT-4.1 mini rising from 13.3% to 74.3% and Gemini 3 Flash from 2.3% to 43.0%—revealing that safety alignment fails to generalize with meaning.

该研究发现，大型语言模型在临床文档和推理任务中容易受到无关信息（如寒暄对话、背景语音）的污染，提出临床推理与分心的双编码假设，并建议在临床部署前评估模型对无关信息的抵抗力。