robustness

This paper introduces M2BIND, a benchmark to evaluate whether vision-language models maintain stable visual-linguistic associations across languages. It finds that binding is not language-invariant, with cross-family and cross-script settings causing significant performance collapse and weaker internal causal binding.

This paper introduces the Robust Ambiguity Detection (RAD) framework, which uses model-space and feature-space consistency metrics to quantify predictive ambiguity and guide abstention in machine learning models.

This paper introduces adversarial persuasion, showing that RL-trained persuaders can collapse LLM accuracy to near zero with a single false argument, and that these tactics transfer across models including GPT-4o-mini, highlighting a critical safety vulnerability in LLM agents.

Introduces EnterpriseRAG, a benchmark of 983 expert-validated samples across six domains that evaluates LLM instruction adherence and robustness under non-ideal enterprise retrieval conditions including noise, knowledge gaps, and factual conflicts. Evaluation of 13 LLMs reveals a large gap between per-constraint satisfaction and holistic compliance, highlighting the need for context-aware protocols in production RAG systems.

This paper introduces VialectBench, a benchmark evaluating LLM robustness to six Vietnamese dialect groups across four tasks, finding average performance drops of 2.82% and no model fully dialect-invariant.

Introduces AndroidReality, a perturbation-based framework for evaluating and improving the robustness of mobile agents, with a taxonomy of real-world interface perturbations and a training-free Test-Time Introspective Recovery (TTIR) mechanism.

A paper introducing ILL, an inaudible low-frequency red-teaming method to expose safety vulnerabilities in audio-language models, and DRG, a defense that detects distribution shifts and requests a second recording to recover accuracy.

Introduces Cultivar, a contrastive localized translation benchmark for detecting data contamination and evaluating localization robustness in multilingual translation models. It benchmarks 32 open-weight models and finds that MT-specialised models are less robust, with potential overfitting to FLORES.

An experiment on analog hardware noise shows accuracy collapses at a threshold rather than degrading smoothly, and noise-aware training shifts that threshold significantly.

This arXiv paper introduces CohortHijack, a robustness audit that removes non-target cells from single-cell query cohorts to test how annotation tools can be manipulated without altering the target cell's expression profile. It shows that structured removal and search strategies can change refined labels in popular pipelines while preserving the target, identifying query cohort composition as a vulnerability surface.

Introduces GROM, a gradient-free one-shot machine unlearning method that computes a closed-form additive weight update via ridge-regularized least squares, achieving state-of-the-art forgetting-utility trade-offs on benchmarks like TOFU and WMDP, and resisting quantization-based recovery attacks.

This paper proposes a method to identify spuriously correlated samples after model convergence by measuring prediction fragility under input perturbation, requiring no group labels or early-stopping epochs. Rebalancing training with detected samples improves worst-group accuracy on Waterbirds from 57.3% to 80.8%.

This paper presents a triple-robustness analysis of RAG for multi-hop traceability, varying embedder, corpus, and judge across thousands of runs. It finds that over-citation is architecturally universal in GraphRAG but its faithfulness consequences are corpus-conditional, and LLM-as-judge faithfulness is fragile across retrieval states.

This arXiv paper evaluates theory of mind capabilities in reasoning LLMs, finding increased robustness to prompt variations and task perturbations. The authors interpret gains as evidence for a robustness-based account rather than a new ToM-specific ability.

This paper proposes a framework to elicit intrinsic hallucinations in LLMs using semantically equivalent adversarial perturbations, showing that state-of-the-art models degrade significantly in contextual faithfulness even with meaning-preserving query variations.

This paper introduces ExpG, a mechanism for building and refining adaptive guidance that captures each tool's capability boundaries and best practices, enabling agents to use tools more robustly across diverse runtime conditions. Experiments show consistent improvements in tool selection, tool calling, and response generation, allowing smaller agents to outperform larger ones without ExpG.

Introduces ARB, a matched authorship-rewriting benchmark for evaluating AI-text detectors, showing that detector performance drops significantly when human text is rewritten by an LLM despite high recall on direct LLM-generated text.

This paper introduces CAGE, a certification method that verifies whether tool-using LLM agents remain authorized under typed-return uncertainty, including binding faults and numerical drift. It proves that separate channel certification is insufficient and provides exact and learned-gate variants for practical authorization.

This paper evaluates GRU, LSTM, and Transformer encoder models for classifying Level 2 automated driving systems from vehicle telematics data, and introduces a robustness evaluation framework simulating realistic data corruption.

Analyzes the phenomenon of zero-byte outputs produced by major AI models (GPT, Claude, Gemini, Kimi) across different vendors, revealing a semantic void matrix that affects generation reliability.