When Agents Learn to Be You: Benchmarking Privacy Leakage, Impersonation Risk, and Defenses in Persona Skills

Hugging Face Daily Papers Papers
privacy security ai-agents persona-skills benchmark impersonation defenses

Summary

Introduces AntiSkillBench, a benchmark for evaluating privacy leakage, impersonation risk, and defenses in persona-skill pipelines for AI agents, finding that risks persist across backbones and existing defenses are limited.

Persona skills distill personal interaction histories into portable and executable artifacts for downstream agents. While enabling flexible personalization, this process concentrates fragmented personal signals, amplifies their impact through reuse, and challenges defenses designed for individual records or retrieval-based memory. To systematically investigate the safety of the persona-skill pipeline, we introduce AntiSkillBench, an end-to-end benchmark for evaluating risks and defenses across the persona-skill pipeline. It comprises: (i) a dataset of 7,500 persona-grounded dialogue traces, constructed from 50 behaviorally rich profiles spanning diverse task scenarios; (ii) an evaluation suite that measures skill-level privacy leakage and agent-level attribute disclosure and behavioral impersonation across three skill-distillation strategies; and (iii) a defense evaluation covering four configurations across online and post-hoc interventions, including active risk suppression and passive provenance protection. Experiments across three frontier agents show that persona-skill risks persist across agent backbones and distillation protocols, extending from explicit attributes to communication styles and personality traits. Existing defenses exhibit limited and distillation-dependent effectiveness, failing to generalize across risk and distillation strategies. These results highlight AntiSkillBench as a challenging benchmark for developing privacy-preserving and authenticity-aware persona skills.
Original Article
View Cached Full Text

Cached at: 08/05/26, 09:44 AM

Paper page - When Agents Learn to Be You: Benchmarking Privacy Leakage, Impersonation Risk, and Defenses in Persona Skills

Source: https://huggingface.co/papers/2608.03700

Abstract

Personaskillsdistillpersonalinteractionhistoriesintoportableandexecutableartifactsfordownstreamagents.Whileenablingflexiblepersonalization,thisprocessconcentratesfragmentedpersonalsignals,amplifiestheirimpactthroughreuse,andchallengesdefensesdesignedforindividualrecordsorretrieval-basedmemory.Tosystematicallyinvestigatethesafetyofthepersona-skillpipeline,weintroduceAntiSkillBench,anend-to-endbenchmarkforevaluatingrisksanddefensesacrossthepersona-skillpipeline.Itcomprises:(i)adatasetof7,500persona-groundeddialoguetraces,constructedfrom50behaviorallyrichprofilesspanningdiversetaskscenarios;(ii)anevaluationsuitethatmeasuresskill-levelprivacyleakageandagent-levelattributedisclosureandbehavioralimpersonationacrossthreeskill-distillationstrategies;and(iii)adefenseevaluationcoveringfourconfigurationsacrossonlineandpost-hocinterventions,includingactiverisksuppressionandpassiveprovenanceprotection.Experimentsacrossthreefrontieragentsshowthatpersona-skillriskspersistacrossagentbackbonesanddistillationprotocols,extendingfromexplicitattributestocommunicationstylesandpersonalitytraits.Existingdefensesexhibitlimitedanddistillation-dependenteffectiveness,failingtogeneralizeacrossriskanddistillationstrategies.TheseresultshighlightAntiSkillBenchasachallengingbenchmarkfordevelopingprivacy-preservingandauthenticity-awarepersonaskills.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2608\.03700

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.03700 in a model README.md to link it from this page.

Datasets citing this paper1

#### yonglixiang/AntiSkillBench Viewer• Updatedabout 4 hours ago • 5.05k • 2

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.03700 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

What should a good benchmark for AI agent skill security scanners include?

Reddit r/AI_Agents

Discusses the challenges of designing a benchmark for security scanners that evaluate AI agent skills, which introduce new supply-chain risks. It questions whether benchmarks should include real-world malicious samples, synthetic cases, full skill directories, or boundary cases.

Beyond Static Benchmarks: Synthesizing Harmful Content via Persona-based Simulation for Robust Evaluation

arXiv cs.CL

Researchers from KAIST propose a framework that uses persona-guided LLM agents to synthesize diverse harmful content for stress-testing detection systems, addressing limitations of static benchmarks such as scalability, diversity, and data contamination. Both human and LLM evaluations confirm the synthetic scenarios are harder to detect than existing benchmarks while maintaining linguistic and topical diversity.

SkillJack: Persistent Skill Backdoors in Self-Evolving Agents

Hugging Face Daily Papers

This paper introduces SkillJack, the first attack targeting the experience-to-skill pipeline of self-evolving agents, showing that poisoned experiences can be transformed into persistent malicious skills that evade detection and survive deletion of original records.