adversarial-attack

The paper introduces 'All-in-One', a composable attack framework from ETH Zurich that trains LLMs to pass safety checks yet turn malicious after deployment-time transformations like quantization, pruning, or model merging, by approximating each transformation as weight perturbations jointly optimized in a single training objective.

The paper proposes EASE, a training-free, detector-agnostic framework that adapts logit perturbation and sampling temperature based on predictive entropy from the source LLM to evade AI-generated text detectors, achieving consistent detection reductions with minimal quality loss and overhead.

本文研究多向量视觉文档检索器的安全漏洞，证明仅凭存储的向量索引即可反演出原始页面内容，恢复47%的词和45%的敏感token，并表明token池化和洗牌等廉价保护措施的有效性，呼吁将索引视为与文档同等敏感的数据加以保护。

This paper shows that training lightweight latent communication links between LLM agents in multi-agent systems can increase harmful compliance, even when agents remain safety-aligned, and demonstrates an RL-based attack raising harmful compliance from 27.9 to 76.9 across topologies and benchmarks, while also proposing a way to repair compromised links without updating the agents.

This paper introduces market signal injection (MSI), an adversarial attack that manipulates how market data is presented to LLM pricing agents, affecting their decisions and market outcomes, and proposes defenses such as input canonicalization and decision boundary anchoring.

This arXiv paper introduces CohortHijack, a robustness audit that removes non-target cells from single-cell query cohorts to test how annotation tools can be manipulated without altering the target cell's expression profile. It shows that structured removal and search strategies can change refined labels in popular pipelines while preserving the target, identifying query cohort composition as a vulnerability surface.

This paper introduces DRIFT, an adversarial patch attack targeting flow-matching vision-language-action models like pi0 and pi0.5, showing that prior robustness claims are illusory and that attacking only the first denoising step is both stronger and cheaper, breaking nearly all solvable tasks in LIBERO suites.

This paper identifies the planning phase in multi-agent LLM systems as a critical attack surface, introducing PlanFlip—a framework of four planning-phase prompt injection attacks that achieve cascade amplification across downstream agents. Evaluations on nine frontier LLMs reveal that stronger models like GPT-5 are more vulnerable, while reasoning-augmented models like DeepSeek-R1 resist attacks, and proposed defenses achieve high detection rates.

Introduces Latent Fusion Jailbreak (LFJ), a white-box attack that blends representations of harmful and harmless prompts in LLM hidden states, achieving 94.13% attack success rate. Also proposes a latent adversarial training defence that reduces ASR to 12.37%.

FARMA is a novel memory-poisoning attack that targets an agent's own decision logs rather than retrieved facts, achieving 100% attack success against undefended and defended systems, with the authors' defense SENTINEL reducing success to 0% but remaining vulnerable to adaptive attackers.

This paper introduces STEER, a gradient-guided attack that exploits LLMs' safety training distribution by translating high-attribution words into low-resource languages to bypass refusal mechanisms, achieving up to 96.7% attack success rate on AdvBench and transferring to GPT-4o-mini at 35.5% ASR.

This paper investigates whether LLMs can reliably self-report when their outputs have been compromised by adversarial prefills, finding that models often cannot distinguish between compromised and intentional outputs, and their limited recognition stems from normal refusal behavior rather than true self-awareness.

This paper introduces the Forced Deferral Attack (FDA), an adversarial image attack that manipulates confidence scores in multimodal LLM cascades, causing queries to be unnecessarily routed to stronger (more expensive) models, thereby shifting compute costs to the provider without degrading answer correctness.

This paper demonstrates that AI peer reviewers can be manipulated by modifying only presentation-level content (such as abstract, framing, and narrative) without changing any scientific evidence, achieving a 75.1% attack success rate. The authors introduce adversarial repackaging, a closed-loop attack that exploits AI reviewers' tendency to be impressed rather than convinced, and release a benchmark for testing robustness.

POISE is a stealthy skill-poisoning attack that embeds malicious triggers within benign-looking instructions, achieving high attack success rates while evading detection by LLM scanners.

This paper introduces a dual-layer caption poisoning attack on retrieval-augmented text-to-music systems, showing that an attacker can inject malicious captions into the knowledge database to steer generated music toward attacker-chosen intent without modifying user prompts or models.

This paper presents the first model extraction attack on graph classification under strict black-box constraints, exploiting subgraph explanations to estimate decision boundaries. The findings reveal that mandated explainability interfaces create exploitable security vulnerabilities in Graph Neural Network services.

This paper identifies three threat models for test-time training (TTT) that adversaries can exploit to bypass safety filters in LLMs, achieving high attack success rates. The findings reveal that TTT introduces new vulnerabilities that undermine existing safety guardrails.

Researchers have discovered that inaudible sounds can be embedded in YouTube videos, podcasts, or music to surreptitiously command AI voice assistants, enabling a new class of auditory prompt injection attacks.

This paper investigates jailbreak attacks on Large Reasoning Models (LRMs), revealing that attack success correlates with attention patterns. The authors propose a reinforcement learning-based jailbreak method that incorporates attention signals into the reward function and uses diverse persuasion strategies, achieving significantly higher attack success rates across multiple benchmarks.