Ψ-Bench: Evaluating Persona-Sensitive Influencing in Persuasive Dialogues

Hugging Face Daily Papers Papers
benchmark personalization persuasive-dialogue llm-evaluation persona-sensitive

Summary

Introduces Ψ-Bench, a benchmark for evaluating LLMs' ability to influence users through persuasive dialogues with personalized profiles. Tests 10 frontier LLMs and finds significant room for improvement, with profile access boosting performance by 18.24%.

Personalization is a crucial capability of modern language agents. However, current research primarily positions personalized agents as passive responders to user preferences, limiting their ability to interact with users and provide suggestions or guidance proactively. To systematically evaluate such proactive personalization in realistic interactions, we propose Ψ-Bench, a benchmark for assessing LLMs' ability to influence realistic users through conversation. We design three real-world interaction scenarios that involve persuasion in Ψ-Bench, and endow simulated clients with personal characteristics through explicit user profiles derived from dialogue histories. We evaluate 10 frontier LLMs on Ψ-Bench and find that while most models can produce coherent and reasonable arguments, even state-of-the-art models still leave considerable room for improvement in persuasion. We also find that providing access to client profiles yields an average performance gain of 18.24\%, highlighting the importance of user-specific information for effective persuasion. Overall, our work highlights persona-sensitive influencing as a challenging yet practical direction for evaluating and developing more proactive personalized LLM agents. Codes are available at: https://github.com/Hanpx20/Psi-Bench.
Original Article
View Cached Full Text

Cached at: 06/03/26, 03:35 AM

Paper page - Ψ-Bench: Evaluating Persona-Sensitive Influencing in Persuasive Dialogues

Source: https://huggingface.co/papers/2606.02754

Abstract

LLMs demonstrate limited effectiveness in persuasive conversation despite generating coherent arguments, with user-specific profiles significantly improving performance.

Personalization is a crucial capability of modernlanguage agents. However, current research primarily positionspersonalized agentsas passive responders to user preferences, limiting their ability to interact with users and provide suggestions or guidance proactively. To systematically evaluate such proactive personalization in realistic interactions, we propose Ψ-Bench, a benchmark for assessingLLMs’ ability to influence realistic users through conversation. We design three real-world interaction scenarios that involvepersuasionin Ψ-Bench, and endow simulated clients with personal characteristics through explicituser profilesderived fromdialogue histories. We evaluate 10 frontierLLMson Ψ-Bench and find that while most models can produce coherent and reasonable arguments, even state-of-the-art models still leave considerable room for improvement inpersuasion. We also find that providing access to client profiles yields an average performance gain of 18.24\%, highlighting the importance of user-specific information for effectivepersuasion. Overall, our work highlightspersona-sensitive influencingas a challenging yet practical direction for evaluating and developing more proactive personalized LLM agents. Codes are available at: https://github.com/Hanpx20/Psi-Bench.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2606\.02754

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2606.02754 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2606.02754 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.02754 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

$\Psi$-Bench: Evaluating Persona-Sensitive Influencing in Persuasive Dialogues

arXiv cs.LG

Ψ-Bench is a benchmark for evaluating LLMs' ability to influence users through persuasive dialogues, incorporating user profiles for personalized persuasion. Experiments show that even state-of-the-art models have room for improvement, and access to client profiles significantly boosts performance.

PersonalBench: Measuring the Authorship Gap in LLM Personalization

arXiv cs.CL

PersonalBench is a new benchmark that evaluates inference-time personalization methods in LLMs through authorship verification, LLM-as-judge, and stylometrics, finding that while methods produce author-differentiated output, they do not bridge the gap to human authorship.

Benchmarking the Personalization Capabilities of Large Language Models

arXiv cs.AI

This paper introduces SDR-Bench, a benchmark for evaluating the personalization capabilities of large language models in a two-party Bayesian Persuasion framework, finding a consistent plateau across frontier LLMs and validating the framework with a field deployment.