MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applications via Environment Simulation
Summary
MCP-Persona is a benchmark evaluating LLM agents on personalized tools interacting with individual accounts and local databases. Experiments reveal significant challenges for state-of-the-art agents in personalized tool use.
View Cached Full Text
Cached at: 06/02/26, 03:37 PM
Paper page - MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applications via Environment Simulation
Source: https://huggingface.co/papers/2606.02470 Authors:
,
,
,
,
,
,
,
,
,
,
Abstract
MCP-Persona benchmark evaluates agent performance on personalized tools interacting with individual accounts and local databases, revealing significant challenges in current SOTA agents.
TheModel Context Protocol(MCP) has emerged as a transformative standard for connectinglarge language models(LLMs) withexternal data sourcesandtools, and has been rapidly adopted across personal applications and development platforms. However, existingbenchmarks predominantly focus on generic information-seekingtoolsand fail to capture the practical challenges posed by personal social applications, wheretoolsinteract with individual accounts or local databases. To bridge this critical gap, we introduce MCP-Persona, the firstbenchmarkspecifically designed for evaluatingagent performanceon real-world, personalized MCPtools. MCP-Persona encompasses a diverse set of widely-used applications, ranging fromsocial media platformslike Reddit and Xiaohongshu (Rednote) toenterprise collaboration suitessuch as Lark (Feishu) and Slack. Our extensive experiments on various state-of-the-art (SOTA) agents demonstrate their significant struggles withpersonalized tool use, thereby highlighting thebenchmark’s crucial role in identifying and addressing these limitations. MCP-Persona is publicly available at https://github.com/wwh0411/MCP-Persona}{https://github.com/wwh0411/MCP-Persona.
View arXiv pageView PDFGitHub4Add to collection
Get this paper in your agent:
hf papers read 2606\.02470
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.02470 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.02470 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.02470 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Beyond Cooperative Simulators: Generating Realistic User Personas for Robust Evaluation of LLM Agents
Introduces Persona Policies (PPol), a plug-and-play control layer that uses LLM-driven evolutionary program search to generate diverse, human-like user personas for evaluating LLM agents. Achieves 33–62% fitness gains over baseline, with human-likeness rated at 80.4%, and improves agent robustness with +17% task success.
Persona-Guided LLM Agents for Task-Oriented Dialogue
This paper explores how LLMs can act as persona-guided agents in task-oriented dialogue, analyzing the effects of personality adaptation on task performance and user satisfaction through evaluations with models like GPT-4o and Gemini 2.0 Flash.
APeB: Benchmarking Personalization Ability of Large Language Model Agents
Introduces APeB, a benchmark for evaluating personalization in LLM agents, focusing on inferring user intent and preferences from raw queries and interaction histories. Finds that current models struggle with early-stage queries and that history-aware refinement can help.
Toward User-Conditioned Evaluation of Personal LLM Agents under Temporal Interventions
This paper argues that evaluating personal LLM agents requires replaying temporal interventions across different user-conditioned states and identifies a gap in current benchmarks. It proposes a minimal benchmark design and reporting metrics for user-conditioned adaptation.
FinPerMA: A Theory-Informed, Event-Grounded Personalized-Memory Benchmark for LLM Agents
Introduces FinPerMA, an event-grounded benchmark for evaluating personalized memory in LLM agents for financial advising, showing that current models and memory systems remain far from saturated.