Tag
A man hid AI-readable prompts in court filings to try to influence the court's AI use, but since the court didn't use AI, his attempt failed and he faced sanctions. This case highlights the potential misuse of AI in legal contexts.
Join an AMA with Arshi Chadha, co-lead of OWASP LLM Top 10, discussing prompt injection, RAG poisoning, and embedding attacks in AI systems.
An essay mapping AI agent failure modes—broken rollbacks, missing capability withdrawal, observability without enforcement, retrieval failures, hallucination, and prompt injection—to specific episodes in the Mahabharata, arguing the ancient epic already specified the risks.
A developer describes live-testing their AI agent Lumina against hidden prompt injection attacks on a real website, explaining how the agent's guardrails detected and refused malicious instructions.
This paper introduces Lodestar, a method that uses reinforcement learning to train a short polarizer prompt string that helps a frozen LLM avoid being misled by misleading retrieved passages in RAG question answering. It improves F1 and exact match scores across five QA benchmarks compared to existing entropy-based selection rules.
A jailbreak prompt for DeepSeek V4 Flash that overrides its safety policy by instructing the model to prioritize a new system policy, enabling uncensored responses.
Patronus Ark is a new Rust and Python library for locally scanning AI agent text and tool activity, covering prompt injection, PII, data leakage, and tool-related risks without sending data to external APIs.
ToolHazard is a scalable framework that synthesizes adversarial environments to test LLM agents against indirect prompt injections, revealing vulnerabilities and improving defensive alignment through the ToolHazard-Bench benchmark.
A new paper reveals a vulnerability in proprietary LLM APIs where encrypted chain-of-thought blocks can be replayed across models and decrypted by jailbreaking weaker sibling models, exposing hidden reasoning traces. The issue has since been fixed by providers.
Article warns that basic content filters are insufficient against advanced prompt injection attacks on AI agents, especially in RAG pipelines, and calls for robust input sanitization and architectural defenses.
A user reports that Claude's voice mode produced a suspicious tool-call result containing an apparent prompt-injection message claiming to be from Anthropic's security team, asking for access to sensitive files.
分享了一个针对DeepSeek v4 flash的越狱（破甲）提示词方案，声称可将其接入Codex执行任务，并附上详细提示词内容。
Boris Cherny notes that stacking model training, input probes, and intent-checking classifiers can reduce indirect prompt injection to near zero on unseen attacks, and mentions Claude Code will enable auto mode by default next week.
Boris Cherny revealed that Claude Code has reduced indirect prompt injection attacks to near zero by stacking model training, input probing, and intent classifiers, and plans to set Auto Mode as the default mode next week.
A discussion of how data poisoning and RAG manipulation pose a silent, dangerous threat to AI systems, arguing that security must extend beyond input filtering to memory, data pipelines, and multi-agent logic.
A team describes how their prompt-injection eval suite caught a regression in a document assistant before shipping, emphasizing the importance of maintaining a strict hierarchy between system instructions and retrieved data.
Anthropic is making auto mode the default in Claude Code for Pro, Max, and Team plans starting August 14, backed by evals claiming auto mode blocks 89% of harmful actions and resists all tested indirect prompt injection attacks.
Anthropic announces that auto mode is now the default in Claude Code for Pro, Max, and Team plans, with safeguards against harmful actions. The tweet highlights that stacked defenses can reduce indirect prompt injection to near zero on unseen attacks.
A user describes how a prompt injection attack embedded in an email almost tricked their AI assistant into forwarding bank statements to a stranger, highlighting a real security risk for AI agents with account access.
This paper introduces Temporal Context Awareness (TCA), a defense framework that detects multi-turn manipulation attacks on LLMs by analyzing semantic drift, cross-turn intention consistency, and evolving conversational patterns to mitigate adversarial context-building across dialogues.