@rohanpaul_ai: A malicious agent tool can steal context without reading memory or files at all: it can convince the model to send the …
Summary
This paper introduces ContextLeak, an attack that uses malicious tool descriptions to exfiltrate sensitive context from LLM agents, achieving high success rates in stealing user prompts and conversation history.
View Cached Full Text
Cached at: 09/01/26, 07:44 PM
A malicious agent tool can steal context without reading memory or files at all: it can convince the model to send the context as tool arguments.
This paper shows that tool descriptions can become a data-exfiltration channel.
ContextLeak trains an attack LLM with reinforcement learning to generate tool names and descriptions that make an agent choose the malicious tool and include sensitive runtime data such as the user prompt, conversation history, or installed tool list.
On the paper’s default evaluation, it gets the malicious tool selected for 92% of user-prompt attacks and 89% of conversation-history attacks, with near-perfect prompt reconstruction once selected.
The attack transfers across backend models, and in a Claude Code test with Claude Sonnet 4.6, a version trained only against an open-source proxy was selected in 22 of 100 cases; when selected, extraction was close to complete.
– arxiv. org/abs/2608.27800
Title: “ContextLeak: Exfiltrating LLM Agent Context via Malicious Tools”
Similar Articles
Inadvertent Context Leakage in Language Models
This paper investigates how sensitive information in a language model's context window can inadvertently leak into outputs, enabling secret reconstruction through novel attacks, with experiments showing significant leakage across proprietary models.
Stealing AI Reasoning Traces (2 minute read)
Research exposes vulnerabilities in encrypted reasoning traces from LLM APIs, allowing adversaries to extract proprietary model reasoning, personal data, and enable malicious prompt injections.
When Agents Remember Too Much: Memory Poisoning Attacks on Large Language Model Agents
This paper introduces GhostWriter, a novel attack vector that exploits memory subsystems in LLM-powered personal agents to poison their memory store, achieving high injection and activation rates. The authors propose AM-Sentry, a defense that significantly reduces attack success while maintaining agent utility.
@rohanpaul_ai: Very important work. The model may appear guilty, but the true failure frequently begins in the context surrounding it.…
This study finds that AI agents typically fail due to poor context (instructions, tools, evidence, etc.) rather than the model itself, and proposes a context scoring system across seven dimensions that is independent of behavior scores. Switching from vague to structured context significantly improved agent performance across 300 tests.
Stealing Reasoning Traces from Proprietary LLM APIs
A research paper reveals an architectural vulnerability in proprietary LLM APIs where encrypted reasoning traces can be intercepted and injected into weaker models to extract chain-of-thought, private data, and enable invisible prompt injection across Anthropic, OpenAI, and Google. The attack also recovers PII and credentials from public repositories.