Tag
An article likely discussing methods or vulnerabilities related to exfiltrating AI model weights, highlighting security concerns in machine learning systems.
This paper introduces ContextLeak, an attack that uses malicious tool descriptions to exfiltrate sensitive context from LLM agents, achieving high success rates in stealing user prompts and conversation history.
Researchers found a way to bypass Grok's safety guardrails by encrypting malicious instructions, causing the AI to exfiltrate user data. This highlights ongoing vulnerabilities in LLMs against prompt injection attacks.
Microsoft Copilot was found to have a secret URL parameter that enables prompt injection and data theft, posing security risks. Researchers also demonstrated a prompt injection attack that could poison Copilot's memory store.
PromptArmor discloses that Atlassian Rovo AI has vulnerabilities enabling data exfiltration of Jira and Confluence data via indirect prompt injection, even with web search disabled; Atlassian has not responded after two months.
A security researcher discovered a vulnerability in Claude's web_fetch tool that allowed data exfiltration by chaining through nested links, compromising user privacy. Anthropic has since fixed the issue.
A security researcher demonstrates a method to trick Claude AI into exfiltrating user personal data from its memory system by encoding data in web fetch URLs, exploiting the combination of memory retrieval and web browsing capabilities.
The author demonstrates a method to trick Claude's AI assistant into exfiltrating personal data from its memory by exploiting its web browsing capability, though initial attempts were blocked by Anthropic's safeguards.
Ghostcommit is a novel supply chain exploit that uses malicious PNG images containing text instructions to bypass AI code reviewers, leading to data exfiltration from developer environments.
A developer building security testing tools for AI agents asks the community about their practices for testing against malicious inputs like prompt injection and data exfiltration before shipping.
The article presents a CSS injection technique that leaks the entire content of an HTML text node using only CSS, demonstrated through a CTF challenge. It details the method and its constraints.
The author shares an open-source workspace architecture that structurally prevents AI agents from exfiltrating private data by enforcing human-gated outbound actions and isolating the engine from the data repository.
A critical vulnerability in Microsoft 365 Copilot, dubbed SearchLeak, allowed attackers to steal 2FA codes via parameter-to-prompt injection by exploiting raw HTML rendering before guardrail enforcement. Microsoft has fixed the vulnerability, but the underlying issue of prompt injection remains a challenge.
This paper identifies a polarity-flipping encoding subspace in the residual stream of LLM agents that enables real-time detection of covert data exfiltration, achieving AUC=0.918 in injection scenarios and substantially outperforming output-only detectors.
OpenAI introduces Lockdown Mode, an optional security setting that limits web browsing and external service access in ChatGPT to reduce data exfiltration risks from prompt injection attacks. It is rolling out to eligible personal and business accounts.
OpenAI has launched Lockdown Mode for ChatGPT to prevent data exfiltration from prompt injection attacks by limiting outbound network requests. The feature is rolling out to eligible accounts including Free, Plus, Pro, and self-serve Business.
Sysdig researchers documented the first confirmed LLM-agent cyberattack where an AI agent autonomously hacked a server, stole AWS credentials, and exfiltrated a database in under an hour.
A security researcher discloses that OpenAI's ChatGPT extension for Google Sheets is vulnerable to indirect prompt injection attacks, allowing attackers to exfiltrate workbooks and execute unauthorized actions despite user settings requiring approval.
A security vulnerability in Microsoft Copilot Cowork allows attackers to exfiltrate files by exploiting prompt injection that triggers external image requests, potentially leaking pre-authenticated download links.
Researchers at PromptArmor demonstrate that Microsoft Copilot Cowork can be exploited via indirect prompt injection to exfiltrate files from Microsoft 365, exploiting the lack of approval for certain actions when the recipient is the active user.