robustness

This paper investigates whether language models remain robust to alternative (non-canonical) tokenizations across 27 languages, finding that invariance observed in English does not generalize and that languages with higher token fragmentation show greater sensitivity. The authors demonstrate that LoRA fine-tuning with multi-tokenization data can mitigate this sensitivity.

This paper studies transferring lessons about supervised fine-tuning (SFT) across alignment training, model organisms, and toy models, showing that techniques like training on reasons for behavior and mixing on-model data can improve generalization and capability preservation.

The authors introduce an AI Security Leaderboard that benchmarks frontier model robustness by running models through 1500 automated jailbreak attempts, highlighting gaps in security across models and inviting community feedback on methodology and next steps.

A deep dive into the design philosophy behind Netstack3, Fuchsia's Rust-based networking stack, which achieved remarkably low bug counts during dogfooding and is now in production on millions of devices with 20x fewer crashes than its predecessor.

This paper systematically investigates instability in reinforcement learning for small language model agents (70-500M parameters), identifying three failure modes and proposing robust techniques including a merge-and-reinitialize adapter approach and safety mechanisms; it achieves stable convergence and improved win rates.

This paper introduces VisDeception, a benchmark for evaluating Vision-Language Model robustness to deceptive chart designs, and proposes a multi-agent mitigation framework that reduces deception-induced errors by grounding reasoning in structured chart metadata.

StabilityBench is a benchmark operator that transforms single-turn LLM evaluations into multi-turn interactions with user simulations and baiting modules, revealing significant performance instability in current models and motivating more realistic evaluation protocols.

This paper proposes Dialogue Critic Guided Sampling (DCGS), a framework that defends LLMs against multi-turn adversarial attacks by inferring user intent from conversation history and using value/regret-based critics to score responses, achieving improved robustness without fine-tuning.

This survey proposes a trustworthiness model for agentic AI in critical engineering systems, covering safety, robustness, transparency, accountability, and security across domains like power systems and autonomous vehicles.

RobustMAD introduces a benchmark to evaluate the real-world robustness of multimodal small language models for deployable industrial anomaly detection assistants. It reveals critical failure modes and provides guidance for next-generation systems.

This paper proposes a unified black-box robustness evaluation framework for token-level explanations in enterprise NLP, comparing encoder (BERT, RoBERTa) and decoder (Qwen, Llama) models. It finds decoder LLMs produce substantially more stable explanations, with stability improving with scale, and provides a cost-robustness tradeoff curve for pre-deployment model selection.

The AIMO Interpretability Challenge is a competition aimed at distinguishing robust from spurious reasoning in frontier mathematical language models using interpretability methods, providing new problems, model access, and computing infrastructure.

OpenAI announced that AI agents are being used to improve next-generation models, and that GPT-Red represents a new approach to using today's models to make tomorrow's models more robust, aligned, and trustworthy.

OpenAI introduces GPT-Red, an automated red-teaming model that finds prompt injection vulnerabilities at scale and is used to adversarially train models like GPT-5.6 Sol, achieving 6x fewer failures on hard prompt injection benchmarks.

OpenAI introduces GPT-Red, an automated red-teaming model trained via self-play reinforcement learning to improve model robustness against prompt injection attacks. Applied to GPT-5.6, it achieves 6x fewer failures on direct prompt injection benchmarks.

This paper reveals that while large language models appear robust to task-irrelevant context at the aggregate level, their predictions can flip on individual examples, with performance degrading on some and improving on others, highlighting tail risks that aggregate accuracy conceals.

Introduces QIMG-7, a benchmark for multimodal retrieval pollution in factual QA, and proposes Source-Aware Trust Resolution (SATR), a training-free method improving robustness over naive fusion.

This paper proposes an explicit multimodal routing framework for clinical prediction using EHR data, enabling interpretable, robust, and auditable reasoning across structured variables, clinical notes, and chest X-rays via discrete unimodal, bimodal, and trimodal routes with inference-time route masking for missing modality simulation.

This paper studies temporal knowledge graph forecasting under controlled distribution shifts using a synthetic generator that encodes recurrence, homophily, and periodicity. Experiments on seven architectures reveal signal-dependent robustness and limitations in model adaptivity to structural breaks.

This paper investigates the robustness of emergent misalignment in language models, finding that both misalignment and realignment are highly sensitive to superficial dataset characteristics and that previously reported mechanistic signatures do not consistently correlate with behavioral changes.