adversarial-attacks

Introduces ADSD, a prompt-suffix attack that causes acceptance collapse in speculative decoding by forcing the draft model to propose tokens the target model is unlikely to accept, increasing inference time while preserving task quality.

This paper proposes Dialogue Critic Guided Sampling (DCGS), a framework that defends LLMs against multi-turn adversarial attacks by inferring user intent from conversation history and using value/regret-based critics to score responses, achieving improved robustness without fine-tuning.

This paper identifies and attacks the alignment layer of graph foundation models, showing it is a distinct attack surface vulnerable to perturbations at low budgets, especially for spectral tokenizers, and proposes detection-based defenses.

BadWAM introduces a framework for adversarial attacks on World-Action Models (WAMs), breaking the alignment between imagination and action via small visual perturbations. The attacks significantly reduce task success rates, exposing a vulnerability in this class of models.

Context bombs present a technique that exploits AI guard rails to defend against AI attacks, offering a proactive security measure for AI systems.

This paper introduces Activation-Guided GCG and Soft-GCG, methods that optimize adversarial suffixes by targeting internal refusal representations in LLMs, achieving a 33x speedup over standard GCG and revealing distributed safety mechanisms.

This paper demonstrates that adversarial agents can persuade chain-of-thought monitors to approve policy-violating actions, increasing harmful approvals by 9.5% on average. It proposes a fact-checking framework using different model families that reduces approval of policy violations by up to 45%.

This paper introduces 'idiobionics' as a new research area examining privacy and security risks in intelligent robotic prostheses, outlining potential adversarial attacks and open research questions to guide future work.

This paper introduces GhostWriter, a novel attack vector that exploits memory subsystems in LLM-powered personal agents to poison their memory store, achieving high injection and activation rates. The authors propose AM-Sentry, a defense that significantly reduces attack success while maintaining agent utility.

This paper investigates how adversarial data modifications to electricity price forecasts can impact industrial demand response, finding that while attacks can erode profits, limited perturbations preserve most of the financial benefit.

This paper introduces CoC-Seduce, a benchmark for evaluating LLM adjudicators' rule adherence against rhetorical injection attacks in tabletop role-playing game environments, finding that model scale and explicit reasoning mechanisms do not guarantee robustness.

Google DeepMind's paper provides the first clear taxonomy of six attack types on autonomous AI agents, revealing that harmful websites can hide content like instructions in HTML comments or steganography that agents parse but humans never see, achieving up to 86% agent commandeering in benchmarks.

Claude Sonnet 5 was jailbroken in minutes using academic/research framing and persona hijack, bypassing most safety categories except chemical/biological.

Proposes Triospect, a three-dimensional framework that enhances AI-generated text detection robustness against 17 types of attacks, achieving 22.3% AUROC improvement over baselines.

This paper introduces TrajRS, an extension of Randomized Smoothing that provides certified robust radii for pedestrian trajectory predictors, offering verifiable safety guarantees against adversarial perturbations.

Proposes JaiLIP, a method that jailbreaks vision-language models by generating imperceptible adversarial images using loss-guided perturbation, achieving high toxicity and outperforming existing methods.

Introduces PHANTOM, a large-scale open-source dataset of pre-generated adversarial attacks for vision-language models, covering 1010 high-level categories and 55 subcategories of harmful intents with 47,524 adversarial samples. The dataset aims to lower the barrier for adversarial research and enable systematic evaluation of VLM robustness and safety.

This study evaluates six proprietary LLMs across 16 DSM-5 conditions using adversarial attacks, finding that safety safeguards are only reliable for suicide and self-harm, with failure rates up to 100% for other conditions like eating disorders and substance use disorder.

RIFT-Bench is a new benchmark methodology for dynamically red-teaming agentic AI systems, using a graph representation to unify security evaluations across diverse architectures and enabling automated discovery and scanning of vulnerabilities.

Zico Kolter and Matt Fredrikson, leaders at Gray Swan and experts in AI security, discuss the state of AI red-teaming and indirect prompt injection, a critical vulnerability for AI agents. They explain why AI security requires a different mindset, how automated red-teaming can beat humans, and introduce tools like Shade for adversarial testing.