robustness

This paper studies how multimodal claim verification models respond to stylistic text changes induced by LLM rewriting. Evaluating 11 open-weight VLMs (2B–38B), the authors find accuracy is largely robust to natural rewriting and controlled LLM-word injection, though hedging-oriented modifications cause consistent probability shifts across nearly all models.

The paper benchmarks email assistants across five communication-style axes and four English dialect variants, finding that indirect and formal requests degrade retrieval and cause agents to omit required actions rather than make unsupported ones, even when relevant evidence is successfully retrieved.

This paper argues that time series forecasting benchmarks need scenario-grounded stress testing, where test cases include semantic scenarios, explicit failure operators, and measurable difficulty levels to capture structured anomalies like regime shifts and sensor failures that i.i.d. perturbation-based robustness tests miss.

A paper introducing a generator-based evaluation framework to study how Concept Bottleneck Models and prototype-based models respond to geometric-latent versus semantic-concept perturbations, arguing that interpretability does not inherently confer robustness but instead redistributes sensitivity across perturbation spaces.

该论文提出 Robust Nash Alignment，一个面向不确定成对偏好的博弈论对齐框架，通过主-对偶代理博弈与乐观镜面下降-上升算法，实现了对最坏情况胜率的认证下界，并在 LLM 对齐实验中优于基线方法。

JEPA 有一个未被讨论的严重盲点：它严重依赖稳定、缓慢变化的特征。只要在每个 episode 中加入一个固定不变的无关 patch，规划就会崩溃；若让该 patch 不可预测，性能反而几乎恢复正常。

The paper introduces Rhetorical Robustness as an evaluation target for AI scientific reviewers, proposes the RobustReview benchmark with 1,260 manuscript versions, and presents SciCore, a dual-branch reviewer that combines full-manuscript assessment with a science-core-extracted judgment to improve stability while maintaining human alignment.

MIT researchers introduce a benchmark of 6,000+ clinical scenarios and 7,000 physician annotations showing that LLMs recommend more unnecessary care than physicians and are more sensitive to gender and tone perturbations in triage settings.

该论文提出 Stability-Aware Generalization Objective（SAGO）框架，主张 LLM 的泛化应评估为稳定性而非准确率，通过生成一致性、内部激活、置信度和响应镜像等多维度变异测量揭示了现有模型普遍存在且与排名相关的泛化不稳定问题。

BreakingWeb is a benchmark framework from Amazon and Duke that constructs challenging browser-use tasks via controlled, deterministic environment interventions on tasks agents already solve, cutting agent pass rates by 22.9% while humans lose only 10.0%, revealing widespread 'belief failure' where agents falsely declare success.

This arXiv paper evaluates how prompt perturbations affect bias and hallucination in LLMs used for decision-making tasks, finding that perturbations can mitigate these issues in some models, with Claude 3 outperforming GPT-3.5 across most datasets.

This preprint uses adversarial training as a controlled instrument on GPT-2 Small to test whether representational simplicity (SAE decomposability, concentrated attribution) implies causal circuit simplicity. It finds that robust models are more SAE-decomposable, while circuit size is regime-dependent: robustness helps at high faithfulness levels (90-95%) but not below 85%.

The paper introduces a Reliability-aware Cross-sample Enhancement (RCE) framework for robust multimodal sentiment analysis, addressing noise and missing modalities through adaptive information bottleneck and cross-sample enhancement, demonstrating superior performance across various settings.

This paper audits a non-generative 'System-1' model on biosecurity-relevant benchmarks, measuring accuracy, calibration, and sensitivity to answer option order, with implications for AI safety pipelines.

This paper proposes TRACE, a fine-tuning framework for Retrieval-Augmented Generation (RAG) that uses multi-agent debate traces and answer completeness regularization to handle knowledge conflicts, improving robustness against misleading context and reducing incomplete answers.

论文提出 MIST（Misleading Image Stress Test）基准，测试 13 个 VLM judge 在存在对齐、误导或缺失图像时标签的稳定性，发现图像是否出现而非图像内容才是改变判断的主要因素，说明可替代性结论对评测配置同样敏感。

This paper investigates black-box model identification by using a repeated token as degenerate input to map token destinations, showing that non-fixed-point tokens can accurately distinguish model families and measuring robustness across quantization levels.

BAER introduces a backbone-adaptive evidence routing method for robust pairwise LLM judging, achieving higher accuracy across benchmarks by dynamically selecting evidence mechanisms per condition.

BaRe-Mem introduces a Bayesian online reliability memory for multi-agent consultation, enhancing robustness against misleading advisor information and guiding the choice between consultation and autonomous reasoning.

This paper provides a comprehensive survey of robustness in federated learning, covering threat models, aggregation strategies, defensive tactics, and future research directions.