adversarial-attacks

This paper identifies a vulnerability in Large Language Models called Semantic Camouflage and proposes Latent Intent Verification, a lightweight probing defense that significantly improves detection rates against adversarial attacks.

This paper investigates how sensitive information in a language model's context window can inadvertently leak into outputs, enabling secret reconstruction through novel attacks, with experiments showing significant leakage across proprietary models.

This paper conducts a systematic study on template-based acceleration for neural network robustness verification and introduces FastCert, a technique to automatically distribute templates for improved performance.

SecOPD is a defense method that uses token-level feedback during fine-tuning to mitigate adaptive prompt injection attacks in large language models, achieving significantly lower attack success rates compared to previous approaches.

FedLNS is a server-side framework that uses LayerNorm signatures to screen malicious updates in federated learning for language models, enhancing robustness against adversarial manipulation.

This paper reveals that RAG poisoning can artificially boost model confidence, evading traditional detection methods, and proposes monitoring attention distribution to detect such attacks.

Meta's new paper shows that adversarial LLMs can persuade judge models to flip decisions in 62–91% of cases under adaptive attacks, often worsening judgments and posing practical risks for AI agent systems.

This paper introduces GENADA, a generative adversarial attack framework that learns to produce deceptive perturbations for time series classifiers in a single forward pass, achieving comparable attack quality to iterative baselines with lower inference time.

This paper proposes DURA, a diffusion-based unrestricted robotic attack that generates visually natural adversarial patches to disrupt Vision-Language-Action (VLA) models in both white-box and black-box settings, highlighting safety risks for physically deployed robotic systems.

This paper introduces a geometry-aware adversarial attack framework that targets relational structure in contrastive embedding manifolds, showing that verification systems like Markmatch can be severely degraded by distorting pairwise similarities rather than decision boundaries.

This paper proposes using GFlowNets to train an attacker LLM that automatically generates adversarial attacks against victim LLMs, enabling automated red teaming and robustness scoring in both English and Turkish.

A survey paper systematically analyzing the evolving safety landscape of multi-modal large language models, covering emerging threats such as adversarial attacks, data poisoning, jailbreaks, and hallucinations, and reviewing updated safety strategies.

A paper introducing ILL, an inaudible low-frequency red-teaming method to expose safety vulnerabilities in audio-language models, and DRG, a defense that detects distribution shifts and requests a second recording to recover accuracy.

This paper proposes a framework to elicit intrinsic hallucinations in LLMs using semantically equivalent adversarial perturbations, showing that state-of-the-art models degrade significantly in contextual faithfulness even with meaning-preserving query variations.

This paper identifies that semantic-shift jailbreaks are limited by overlooking the semantic-shift capability of contexts, and proposes Iterative Context Optimization (ICO), a black-box framework that iteratively optimizes contexts to achieve higher attack success rates against foundation models.

This paper identifies a refusal-cue shortcut in safety guard models, where inserting refusal expressions into harmful responses can flip their harmless classification. The authors audit datasets like WildGuardMix and GR-Train, show the issue persists in official models such as LlamaGuard3 and Qwen3Guard, and propose a post-hoc intervention to suppress shortcut-associated components.

A survey of 'adversarial attacks for good', examining proactive protections applied across the visual content lifecycle to disrupt unauthorized AI automation and support accountability, covering five research communities such as privacy filters, unlearnable examples, generative safeguards, adversarial CAPTCHAs, and provenance mechanisms.

Introduces OpenART, a large-scale arena for red-teaming AI agents via open-ended environment evolution, with over 10K stateful scenarios across 50 domains, plus EMHA, a black-box hypergraph attack achieving 85% ASR across 75 agent-model configurations.

This paper introduces a white-box, gradient-regularized evasion framework that embeds attack logic directly into model parameters, successfully fooling explainable AI auditors like LIME, SHAP, and Integrated Gradients while bypassing anomaly detection defenses.

This paper assesses the adversarial robustness of five Arabic language models under character, word, and sentence-level attacks, showing that diacritic insertion can reduce accuracy by 92% and adversarial training improves resilience but has limitations.