Created a free tool to check what PII your LLM prompts are leaking before they hit the provider
Summary
A free tool has been released to help users detect personally identifiable information (PII) leaking from their LLM prompts before they reach the provider's servers.
Similar Articles
1 in 8 AI support prompts contained personal data. I think we're securing LLMs the wrong way.
An analysis of 10,000 production AI support prompts found 12.4% contained personally identifiable information, arguing that LLM security should focus on data minimization and redaction rather than only prompt injection or jailbreaks.
PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say
Introduces PrivacyPeek, a benchmark for auditing acquisition-stage privacy leakage in LLM-based agents, showing that agents often gather more sensitive data than needed and that current defenses are insufficient.
Institution-Specific LLM Prompting Recovers PHI That De-identification Systems and Their Gold Standards Both Miss
This study demonstrates that large language models with institution-specific prompting can recover protected health information missed by existing de-identification systems, enhancing data privacy compliance in electronic health records.
PII-TRACE: A Benchmark for Context-Aware PII Detection in Multi-Turn LLM Conversations
Introduces PII-TRACE, the first benchmark for context-aware PII detection in multi-turn LLM conversations, and PII-Tracer, a compact detector that achieves high entity-level coverage.
Security and Privacy Prompts in the Wild: What Users Ask LLMs and How LLMs Respond
This paper analyzes real-world user queries about digital security and privacy asked to LLMs, categorizing them into nine topics and evaluating response quality and consistency across commercial and open-weight models.