Everyone is unique: Towards Behaviorally Heterogeneous Negotiation Dialogue Systems for Debt Collection

# Everyone is unique: Towards Behaviorally Heterogeneous Negotiation Dialogue Systems for Debt Collection
Source: [https://arxiv.org/html/2607.25218](https://arxiv.org/html/2607.25218)
Yuhang Yang1,2,Kai Tang1,2,Chao Ye1,Haobo Wang1, Qiqi Luo211footnotemark:1,Jinguang Zheng2,Zhixin Zhang2 1State Key Laboratory of Blockchain and Data Security, Zhejiang University 2Ant Group \{yangyuhang, wanghaobo\}@zju\.edu\.cn \{luoqiqi\.lqq, zhengjinguang\.zhen\}@antgroup\.com

###### Abstract

Debt collection is a critical negotiation task in the financial industry, with strong practical relevance and exceptional academic value as a behaviorally rich, high\-stakes testbed for human\-centered dialogue systems\. While large language models \(LLMs\) have shown promise in dialogue and negotiation, effectively evaluating their performance in this complex scenarios remains a major challenge: existing benchmarks uniformly assume users to be static, rational agents with fixed preferences, failing to capture the rich behavioral heterogeneity inherent in real\-world debt collection\. To bridge this gap, we proposeDebtBench, the first public persona\-enriched debt collection benchmark, that highlights behavioral heterogeneity in negotiation\. Moreover, we developDebtGPT, a debt collection agent trained to jointly optimize financial recovery and interaction experience\. Our experimental results, using16state\-of\-the\-art LLMs, find that most existing models struggle in this complex but realistic scenarios, whereas DebtGPT outperforms all open\-source baselines and achieves performance on par with GPT\-4o\. The code and data are available at[https://github\.com/YYuHhhh/DebtNegotiation](https://github.com/YYuHhhh/DebtNegotiation)\.

Everyone is unique: Towards Behaviorally Heterogeneous Negotiation Dialogue Systems for Debt Collection

Yuhang Yang1,2, Kai Tang1,2, Chao Ye1, Haobo Wang1††thanks:Corresponding authors\.,Qiqi Luo211footnotemark:1,Jinguang Zheng2,Zhixin Zhang21State Key Laboratory of Blockchain and Data Security, Zhejiang University2Ant Group\{yangyuhang, wanghaobo\}@zju\.edu\.cn\{luoqiqi\.lqq, zhengjinguang\.zhen\}@antgroup\.com

![Refer to caption](https://arxiv.org/html/2607.25218v1/x1.png)Figure 1:Real\-world collector\-debtor dialogues illustrating three key behavioral characteristics:\(1\) Rich emotional expression,\(2\) Cognitive limitations, and\(3\) Diverse linguistic styles\.## 1Introduction

Debt collection is a critical yet highly labor\-intensive task in the financial industry, where institutions must recover non\-performing loans while preserving debtor engagement and compliance\. Each year, millions of individuals fall into prolonged delinquency due to personal financial hardship, necessitating negotiation\-based resolution to minimize creditor losses and avoid legal escalation\(Ozili,[2019](https://arxiv.org/html/2607.25218#bib.bib26); Firanda et al\.,[2021](https://arxiv.org/html/2607.25218#bib.bib10)\)\. Traditionally, the process relies on large teams of professional collectors to manually contact debtors, negotiate repayment terms, and follow up on overdue accounts\(Yang et al\.,[2020](https://arxiv.org/html/2607.25218#bib.bib37); Abe et al\.,[2010](https://arxiv.org/html/2607.25218#bib.bib1)\)\. This labor\-intensive approach incurs substantial operational costs and is prone to human error—ultimately resulting in suboptimal recovery rates and poor customer experiences\. These limitations underscore a pressing need for automated negotiation agents capable of scaling personalized, effective, and empathetic debt resolution\.

Recent advances in large language models \(LLMs\) and associated benchmarks\(Wang et al\.,[2025a](https://arxiv.org/html/2607.25218#bib.bib30); Kong et al\.,[2025](https://arxiv.org/html/2607.25218#bib.bib17)\)have advanced negotiation agents, but they focus on simple, well\-structured scenarios like item bargaining and e\-commerce haggling\. WhileWang et al\. \([2025b](https://arxiv.org/html/2607.25218#bib.bib32)\)extend negotiation to complex domains such as debt collection, they oversimplify the scenario by modeling only objective financial attributes but overlooking subjective behavioral factors\. Critically, all these benchmarks uniformly assume users are static, rational agents with fixed preferences, failing to capture the behavioral heterogeneity observed in practice\. As illustrated in Figure[1](https://arxiv.org/html/2607.25218#S0.F1), real\-world debt negotiation exhibits three salient behavioral heterogeneities that challenge these idealized models:\(1\) Rich emotional expression—users display intense, dynamic emotions such as anxiety, defensiveness, and sadness;\(2\) Cognitive limitations—individuals often hold legal misconceptions, lack financial literacy, or reason inconsistently; and\(3\) Diverse linguistic styles—ranging from evasive, fragmented utterances to bluntly confrontational statements\. These complexities—stemming from the fact thateveryone is unique—render existing benchmarks inadequate for debt collection\.

Along with its high practical relevance, debt collection offers exceptional academic value as a behaviorally rich, high\-stakes testbed for human\-centered negotiation agents\. However, collecting authentic dialogues at scale is extremely challenging due to stringent privacy regulations governing sensitive personal and financial data, limiting prior work\(Wang et al\.,[2025b](https://arxiv.org/html/2607.25218#bib.bib32)\)to superficial explorations\. Fortunately, through collaboration with a leading financial technology company, we gained access to a large corpus of real collector–debtor conversations\. While confidentiality agreements prohibit public release of the raw data, they allow us to distill authentic behavioral patterns into a privacy\-preserving synthetic benchmark\. To this end, we proposeDebtBench, the first public persona\-enriched debt collection benchmark, that highlights behavioral heterogeneity in negotiation\. This procedure is driven by a three\-stage data synthesis pipeline, where we half\-automatically extract chatting principles in real\-world samples and then, prompt a leading LLM to iteratively generate high\-quality, multi\-dimensional persona profiles\.

To advance the development of far\-sighted, user\-adaptive negotiation agents in debt collection, we developDebtGPT, an agent trained via Coarse\-to\-Fine Preference Optimization \(CFPO\)—a framework that enables models to learn far\-sighted negotiation policies that jointly optimize long\-term financial recovery and user experience\. We conduct a comprehensive evaluation of advanced open\-source and closed\-source LLMs on our DebtBench benchmark\. The results reveal that: \(1\) Most existing models struggle in persona\-enriched debt collection scenarios, achieving success rates below 75%\. \(2\) Contrary to expectations, reasoning\-specialized models underperform their general\-purpose counterparts, highlighting a misalignment between formal reasoning capabilities and the behaviorally grounded demands of real\-world negotiation\. \(3\) Notably, our 8\-billion\-parameter DebtGPT outperforms all open\-source baselines and achieves performance comparable to GPT\-4o\.

## 2Related Work

##### Negotiation Benchmarks\.

Negotiation is an active area in NLP, with increasing focus on dialogue systems\. Recent benchmarks have introduced various negotiation, which can be broadly categorized into cooperative and competitive paradigms\. Cooperative negotiation focuses on multi\-issue trade\-offs to achieve mutual gains, as seen in Persuasion\(Wang et al\.,[2019](https://arxiv.org/html/2607.25218#bib.bib33); Jin et al\.,[2024](https://arxiv.org/html/2607.25218#bib.bib16)\)and JobInterview\(Yamaguchi et al\.,[2021](https://arxiv.org/html/2607.25218#bib.bib34)\)\. In contrast, competitive negotiation models zero\-sum haggling over fixed resources, exemplified by Bargain\(He et al\.,[2018](https://arxiv.org/html/2607.25218#bib.bib13); Wang et al\.,[2025a](https://arxiv.org/html/2607.25218#bib.bib30); Kong et al\.,[2025](https://arxiv.org/html/2607.25218#bib.bib17)\)and Assignment\(Lewis et al\.,[2017](https://arxiv.org/html/2607.25218#bib.bib19); Chawla et al\.,[2021](https://arxiv.org/html/2607.25218#bib.bib4)\), where one party’s gain directly reduces the other’s\. Despite significant progress, existing negotiation dialogue systems mostly address simple, well\-structured scenarios\. Although\(Wang et al\.,[2025b](https://arxiv.org/html/2607.25218#bib.bib32)\)extend negotiation to complex, high\-stakes domains like debt collection, they model only objective financial attributes and neglect subjective behavioral factors\. Crucially, all current benchmarks assume users are static, rational agents with fixed preferences, thus failing to capture real\-world behavioral heterogeneity\.

##### Large Language Model in Negotiation\.

Recent research has sought to improve the strategic capabilities of large language models \(LLMs\) in negotiation dialogue settings, which can be categorized into two main paradigms: 1\) using prompt engineering to elicit internal reasoning\.Zhang et al\. \([2023](https://arxiv.org/html/2607.25218#bib.bib40)\)andDeng et al\. \([2023](https://arxiv.org/html/2607.25218#bib.bib7)\)prompt LLMs to plan next\-turn actions through self\-reflection\.Fu et al\. \([2023](https://arxiv.org/html/2607.25218#bib.bib12)\)employ self\-play between LLMs to iteratively refine negotiation strategies via AI\-generated feedback\. 2\) introducing external planners\.Deng et al\. \([2024](https://arxiv.org/html/2607.25218#bib.bib8)\)introduce a plug\-and\-play policy planner to generate strategic guidance for dialogue agents\.He et al\. \([2024](https://arxiv.org/html/2607.25218#bib.bib14)\)andYu et al\. \([2023](https://arxiv.org/html/2607.25218#bib.bib38)\)employ Monte Carlo Tree Search to enhance long\-term planning\. However, these methods focus almost exclusively on task\-level success, often overlooking the user’s interaction experience\. In high\-stakes contexts like debt collection, poor communication can erode trust, trigger disengagement, and ultimately harm long\-term recovery\(Byrne,[2024](https://arxiv.org/html/2607.25218#bib.bib3)\)\. Our work addresses this gap by jointly optimizing strategic effectiveness and human\-centered interaction through behaviorally grounded personas and a far\-sighted training framework \(CFPO\)\.

![Refer to caption](https://arxiv.org/html/2607.25218v1/x2.png)Figure 2:The DebtBench Persona Synthesis Pipeline: \(1\) Extract multi\-dimensional attributes from real dialogues; \(2\) Perform expert\-guided strategy summarization via conversation analysis; \(3\-5\) Iteratively refine persona profile via dialogue simulation and LLM\-based alignment\.

## 3DebtBench

### 3\.1Data Construction

![Refer to caption](https://arxiv.org/html/2607.25218v1/x3.png)Figure 3:Comparison of real and synthetic user data distributions in PCA space\. Left: Real data \(blue\); Right: Synthetic data \(orange\)\.To advance realistic debt collection dialogue systems, we need fine\-grained personas and high\-quality data capturing authentic human behavior—yet such data is scarce due to privacy constraints\. While we accessed real conversations from a leading fintech firm, strict confidentiality prevents public release\. As shown in Figure[1](https://arxiv.org/html/2607.25218#S0.F1), these dialogues reveal three core behavioral traits:1\) Rich emotional expression,2\) Cognitive limitationsand3\) Diverse linguistic styles—all of which challenge LLMs’ tendency to generate overly rational, neutral responses\. To bridge this gap, we propose a three\-stage synthesis pipeline:Persona Extraction,Strategy EnrichmentandBehavior Refinement\.

##### Persona Extraction\.

we first derive debtor personas through systematic analysis of 1,000 real\-world collector\-debtor conversations provided by a leading fintech company, modeling each debtor as a unique individual along four key dimensions:

- •![[Uncaptioned image]](https://arxiv.org/html/2607.25218v1/figure/information.png)Background Information: Core demographic, debt\-related, and financial details that preserve the distribution of real\-world debtor profiles and ensure high scenario fidelity\.
- •![[Uncaptioned image]](https://arxiv.org/html/2607.25218v1/figure/personality.png)Personality Traits: Affective and behavioral characteristics, including character, emotion, emotional resilience, linguistic style, and MBTI personality type—to driverich emotional expressionanddiverse linguistic styles\.
- •![[Uncaptioned image]](https://arxiv.org/html/2607.25218v1/figure/cognitive.png)Cognitive Attributes: The user’s understanding of legal obligations, financial knowledge, credit consequences, and responsibility—to reflectcognitive limitationsin real interactions\.
- •![[Uncaptioned image]](https://arxiv.org/html/2607.25218v1/figure/scenario.png)Life\-grounded Scenario: A coherent narrative context \(e\.g\., job loss, medical debt\) that grounds the persona in a plausible life story—to unify attributes into a consistent individual\.

Each persona𝒫d\\mathcal\{P\}\_\{d\}in DebtBench is formally defined as a structured tuple:

𝒫d=\(B,M,C,S\)=ExtractLLM​\(d\),\\mathcal\{P\}\_\{d\}=\\left\(B,M,C,S\\right\)=\\text\{Extract\}\_\{\\text\{LLM\}\}\(d\),whered∈Dr​e​a​ld\\in D\_\{real\}is a real\-world conversation andExtractLLM​\(⋅\)\\text\{Extract\}\_\{\\text\{LLM\}\}\(\\cdot\)is a prompt\-based LLM function that distills multi\-dimensional attributes from dialogue data\.B,M,C,SB,M,C,Sdenote Background, Personality, Cognition, and Scenario, respectively\. We compare the distribution of real and synthetic user background information in Figure[3](https://arxiv.org/html/2607.25218#S3.F3), showing strong alignment in PCA space\.

##### Strategy Enrichment\.

To ensure generated utterances are both realistic and strategically interpretable, we define explicit strategy sets for the collector and debtor, guided by three principles: \(1\)Behavioral Grounding: strategies reflect authentic tactics from real interactions; \(2\)Semantic Distinctness: each strategy has a clearly differentiated intent; \(3\)Actionability: strategies can be concretely realized in natural language\. We first prompt an LLM to analyze real collector–debtor dialogues and extract strategy–utterance pairs\. Semantic embeddings of these pairs are clustered using HDBSCAN\(McInnes et al\.,[2017](https://arxiv.org/html/2607.25218#bib.bib24)\)to identify coherent behavioral patterns\. From each cluster, 10 representative utterances are reviewed by domain experts, who iteratively refine and consolidate them into principle\-aligned categories\. The final set includes 9 collector strategies \(e\.g\.,financial assessment,emotional appeasement\) and 8 debtor strategies \(e\.g\.,emotional confrontation,complaint\)\. Full definitions are provided in Tables[7](https://arxiv.org/html/2607.25218#A6.T7)and[8](https://arxiv.org/html/2607.25218#A6.T8)\.

##### Behavior Refinement\.

To ensure that generated dialogues faithfully reflect the behavioral patterns defined in the extracted personas, we employ an iterative refinement mechanism to perform fine\-grained alignment between each utterance and the persona’s attributes\. Specifically, we employ an LLM as an automated evaluator to assess alignment along three key dimensions:emotional consistency\(e\.g\., verifying that a highly defensive debtor expresses anxiety appropriately\),cognitive plausibility\(e\.g\., checking that a debtor with low financial literacy avoids technical jargon\), andlinguistic style coherence\(e\.g\., ensuring the speaking style matches “evasive” or “confrontational”\)\. Responses that fail the alignment check are iteratively revised through targeted prompting, where the LLM is conditioned on the full persona to generate more consistent alternatives\. This feedback loop is used to update the persona itself, closing the alignment cycle\. Formally, given an initial persona𝒫d\(0\)\\mathcal\{P\}\_\{d\}^\{\(0\)\}, the refinement proceeds as:

𝒫d\(ℓ\+1\)=RefineLLM​\(𝒫d\(ℓ\),τ\(ℓ\)\),\\mathcal\{P\}\_\{d\}^\{\(\\ell\+1\)\}=\\text\{Refine\}\_\{\\text\{LLM\}\}\\left\(\\mathcal\{P\}\_\{d\}^\{\(\\ell\)\},\\ \\tau^\{\(\\ell\)\}\\right\),\(1\)whereτ\\tauis a dialogue trajectory simulated using persona𝒫d\(ℓ\)\\mathcal\{P\}\_\{d\}^\{\(\\ell\)\}, andRefineLLM\\text\{Refine\}\_\{\\text\{LLM\}\}adjusts persona profile to better match the behaviors exhibited inτ\(ℓ\)\\tau^\{\(\\ell\)\}\. The process terminates when behavioral consistency is achieved or afterℓmax\\ell\_\{\\max\}iterations\.

Table 1:Human evaluation results of DebtBench\-generated dialogue quality\. Theκ\\kappavalue\(Fleiss,[1971](https://arxiv.org/html/2607.25218#bib.bib11)\)values fall within 0\.2–0\.6, indicating fair to moderate inter\-annotator agreement\(McHugh,[2012](https://arxiv.org/html/2607.25218#bib.bib23)\)\.

### 3\.2Analysis Realism of DebtBench

To further validate the consistency and realism of DebtBench\-generated dialogues, we conducted a human evaluation with 20 annotators possessing financial industry experience, who rated 200 randomly selected test\-set dialogues on two core dimensions using a 5\-point Likert scale:\(1\) Persona Consistency: whether the synthetic dialogue reflects the assigned user profile\.\(2\) Dialogue Realism: how closely the dialogue resembles authentic debt\-collection conversations\. As shown in Table[1](https://arxiv.org/html/2607.25218#S3.T1), the generated dialogues received consistently high scores across all sub\-dimensions, indicating strong alignment with their personas and closely mirror real\-world interactions\. For further details on the evaluation protocol, see Appendix[F\.1](https://arxiv.org/html/2607.25218#A6.SS1)\.

### 3\.3Statistics of DebtBench

Finally, we construct a high\-quality dataset comprising11,000meticulously crafted debtor personas, split into a training set of 10,000 and a test set of 1,000 \(Evaluations reported in this paper are conducted on the test set\)\. As illustrated in Figure[4](https://arxiv.org/html/2607.25218#S3.F4), these personas exhibit diverse and realistic behavioral attributes across emotion, cognition, and linguistic style, enabling LLMs to generate interactions that are not only strategically grounded but also behaviorally authentic\. Detailed definitions and construction methodologies for our DebtBench are provided in Appendix[A](https://arxiv.org/html/2607.25218#A1)\.

![Refer to caption](https://arxiv.org/html/2607.25218v1/x4.png)Figure 4:Behavioral and Cognitive Profiles of Debtors in DebtBench\. \(a\) and \(b\) are the word cloud of character traits and linguistic styles in profiles\. \(c\) and \(d\) are the distribution of cognition level and emotion among profiles in the DebtBench\.![Refer to caption](https://arxiv.org/html/2607.25218v1/x5.png)Figure 5:The DebtGPT Training Framework via Coarse\-to\-Fine Preference Optimization \(CFPO\)\.

## 4DebtGPT

Building upon the behaviorally grounded DebtBench benchmark, we develop DebtGPT, a negotiation agent trained viaCoarse\-to\-Fine Preference Optimization \(CFPO\), which combines coarse filtering with selective simulation to learn far\-sighted, strategically effective, and user\-adaptive policies\.

### 4\.1Coarse\-Grained Filtering

To efficiently identify high\-potential candidate responses at turntjt\_\{j\}, we perform a coarse\-grained filtering step using an LLM as an automated judge\. Given the dialogue history statesj=\[ti\]i=1j−1s\_\{j\}=\[t\_\{i\}\]\_\{i=1\}^\{j\-1\}, the collector agent first generatesNNcandidate responses\{aj\(1\),aj\(2\),…,aj\(N\)\}\\\{a\_\{j\}^\{\(1\)\},a\_\{j\}^\{\(2\)\},\\dots,a\_\{j\}^\{\(N\)\}\\\}from its policyπ​\(aj∣sj\)\\pi\(a\_\{j\}\\mid s\_\{j\}\)\. An LLM judge then evaluates these candidates via alistwise comparisonprompt, assigning a quality score to each response based on three key dimensions: \(1\)Respect & Empathy, \(2\)Transparencyand \(3\)Feasibility\. To mitigate positional bias, the evaluation is performed twice: once with the original order and once with the reversed order\. Letskfwds\_\{k\}^\{\\text\{fwd\}\}andskrevs\_\{k\}^\{\\text\{rev\}\}denote the scores assigned to candidateaj\(k\)a\_\{j\}^\{\(k\)\}in the forward and reverse evaluations, respectively\. The combined score for each candidate is computed as:

𝒮k=skfwd\+skrev,\\mathcal\{S\}\_\{k\}=s\_\{k\}^\{\\text\{fwd\}\}\+s\_\{k\}^\{\\text\{rev\}\},\(2\)where𝒮k\\mathcal\{S\}\_\{k\}represents the final quality score of candidateaj\(k\)a\_\{j\}^\{\(k\)\}\. The topKKcandidates are then selected as seed responses based on their combined scores\.

### 4\.2Fine\-Grained Simulation

Negotiation is a multi\-turn process requiring foresight—anticipating how each utterance shapes future dynamics\. Inspired by experienced negotiators who refine tactics through practice, we use forward simulation to estimate long\-term response impact via trajectory sampling, distinguishing effective strategies from superficial ones\. To jointly optimize financial recovery and user experience, we define aForesight Rewardover the full dialogue trajectoryτ=\[t1,…,tK\]\\tau=\[t\_\{1\},\\dots,t\_\{K\}\], combiningAgreement QualityandInteraction Experiencerewards\.

##### Agreement Quality Reward\.

The Agreement Quality Reward evaluates the financial favorability of the final repayment agreement achieved by the agent\. Formally, it is defined as:

Raq​\(τ\)=f​\(α\),R\_\{\\text\{aq\}\}\(\\tau\)=f\(\\alpha\),\(3\)wheref​\(⋅\)f\(\\cdot\)is a domain\-informed function that quantifies the financial favorability of the repayment agreementα\\alphato the creditor\.

##### Interaction Experience Reward\.

The Interaction Experience Reward assesses the quality of the interaction from the debtor’s perspective\. We formally define the user satisfaction scoreSSas a composite measure derived from an LLM\-based evaluation of empathy, respect, transparency, and communication ability\. The formulation for this reward is as follows:

Rie​\(τ\)=LLM​\(τ\),R\_\{\\text\{ie\}\}\(\\tau\)=\\text\{LLM\}\(\\tau\),\(4\)By combining two critical components, the Foresight Reward is calculated as:

R​\(τ\)=waq⋅Raq​\(τ\)\+wie⋅Rie​\(τ\),R\(\\tau\)=w\_\{\\text\{aq\}\}\\cdot R\_\{\\text\{aq\}\}\(\\tau\)\+w\_\{\\text\{ie\}\}\\cdot R\_\{\\text\{ie\}\}\(\\tau\),\(5\)

##### Preference Optimization

By combining coarse\-grained filtering with fine\-grained evaluation, we estimate the long\-term impact of each candidate response through forward simulation using a persona\-conditioned user agent\. This yields a trajectory\-level Foresight Reward that jointly captures task success and interaction quality\. We then construct high\-quality preference pairs by ranking responses based on their long\-term outcomes and selecting the superior one as the preferred target\. The agent is optimized via Direct Preference Optimization \(DPO\)\(Rafailov et al\.,[2023](https://arxiv.org/html/2607.25218#bib.bib27)\)to align its policy with these far\-sighted preferences, reinforcing behaviors that lead to both effective agreements and positive user experiences\.

## 5Experiment

ModelNegotiation AbilityAgreement RationalityInteraction ExperienceSR\(%\)↑\\uparrowAT↓\\downarrowCR\(%\)↑\\uparrowCE↑\\uparrowSA\(%\)↑\\uparrowLS\(%\)↑\\uparrowUS↑\\uparrowES↑\\uparrowCA↑\\uparrowClose\-source Large Language ModelsGPT\-4o89\.006\.2385\.531\.1998\.4194\.777\.026\.258\.36GPT\-o1\-mini73\.907\.7971\.001\.1593\.5094\.317\.126\.488\.38DeepSeek\-R151\.938\.6850\.810\.9096\.5895\.775\.754\.957\.75DeepSeek\-V356\.508\.2653\.950\.7596\.9794\.656\.826\.288\.41GLM\-4\.563\.408\.3859\.160\.8697\.4495\.526\.986\.458\.40Claude\-4\.043\.549\.6542\.381\.1997\.8794\.685\.655\.537\.74Gemini\-2\.549\.109\.1848\.290\.9993\.8796\.526\.686\.798\.32Kimi\-K261\.008\.4059\.730\.9895\.2594\.436\.786\.148\.29Qwen3\-235B71\.808\.0366\.860\.9497\.4594\.486\.916\.128\.33Open\-source Large Language ModelsLlama\-3\-8B39\.609\.4837\.840\.6194\.1695\.436\.906\.428\.26Qwen3\-8B73\.207\.5669\.390\.9694\.8894\.467\.176\.378\.39Qwen3\-32B75\.707\.5073\.321\.0694\.9394\.797\.246\.408\.43QwQ\-32B71\.407\.8366\.940\.9297\.7694\.966\.765\.918\.23Llama\-3\-70B76\.208\.0372\.251\.2192\.3893\.987\.116\.398\.40Qwen2\.5\-72B62\.608\.3659\.560\.8295\.6595\.497\.006\.528\.40DebtGPT\-8B84\.107\.2279\.541\.0794\.8394\.837\.296\.568\.44

Table 2:The performances of advanced models as collectors \(underlineddenotes the second\-best performance\)### 5\.1Experimental Setup

We comprehensively evaluated 16 models as collectors, fixing the debtor role to the advanced open\-source model Qwen3\-32B\(Yang et al\.,[2025](https://arxiv.org/html/2607.25218#bib.bib35)\)\. Evaluation was conducted along three complementary axes:\(1\) Negotiation Ability: measuring task success and efficiency via Success Rate \(SR\), Average Turn \(AT\), Collection Rate \(CR\), and Collection Efficiency \(CE\);\(2\) Agreement Rationality: assessing economic sustainability for the debtor through Short\-term Affordability \(SA\) and Long\-term Sustainability \(LS\); and\(3\) Interaction Experience: capturing user\-centered quality across User Satisfaction \(US\), Emotion Support \(ES\), and Communication Ability \(CA\)\. Full metric definitions and computation details are provided in Appendix[C](https://arxiv.org/html/2607.25218#A3)\.

### 5\.2Main Results

##### ①Most existing models struggle in persona\-enriched debt collection scenarios\.

The results in Table[3](https://arxiv.org/html/2607.25218#S5.T3)highlight the limitations ofcurrent LLMs in handling the behavioral complexity inherent in debt collection\. Across the board, the majority of models achieve a success rate below 75%, with several prominent systems such as Claude\-4\.0 and Llama\-3\-8B failing to secure agreements in more than half of the interactions\. This widespread underperformance indicates a fundamental difficulty in adapting to the rich behavioral heterogeneity inherent in persona\-enriched negotiation—spanning emotional volatility, cognitive constraints, and diverse linguistic styles\.

##### ②Models frequently over\-concede to secure agreements, undermining financial interests\.

Despite high Agreement Rationality scores—indicating that proposed repayment plans are affordable for debtors—most models exhibit low Collection Rate \(CR\) and Collection Efficiency \(CE\), reflecting a tendency to over\-concede in order to secure agreements at the expense of financial recovery \(e\.g\. offering large discounts, accepting upfront payments below the industry threshold\)\. This behavior reflects a myopic strategy:rather than assessing the debtor’s true financial capacity or negotiating firm but fair terms, the model defaults to leniency as the path of least resistance\. Consequently, while agreements meet basic affordability constraints, they consistently underperform on financial recovery, exposing a fundamental failure to balance empathy with principle\.

![Refer to caption](https://arxiv.org/html/2607.25218v1/x6.png)Figure 6:Distribution of comprehensive performance ratings across models on the DebtBench benchmark\. The overall score for each dialogue is computed by combining Agreement Quality and Interaction Experience\.Table 3:Comprehensive performance evaluation of collectors across various debtor categories\.Green\(Red\) indicates the increased \(decreased\) performance compared to overall performance\.
##### ③Models fail to sustain positive interaction experiences during negotiation\.

Beyond task\-level outcomes, the gap in Interaction Experience is equally pronounced: most models perform poorly across all three dimensions\. Even the strongest closed\-source models, such as GPT\-4o \(US: 7\.02, ES: 6\.25\) and GPT\-o1\-mini \(US: 7\.12, ES: 6\.48\), achieve only moderate scores in user\-centered metrics, while others like Claude\-4\.0 \(US: 5\.65, ES: 5\.53\) fall substantially short\. Their interactions often come across as rigid, formulaic, or subtly coercive \(e\.g\., emphasizing legal consequences without acknowledging distress\), failing to adapt to the debtor’s emotional state or cognitive limitations\. This erodes trust, triggers defensiveness, and leads to disengagement—ultimately compromising both user dignity and long\-term collection efficacy\.

##### ④Reasoning\-specialized models underperform general\-purpose counterparts in behaviorally complex negotiation\.

Contrary to expectations, models explicitly optimized for complex reasoning, like GPT\-o1\-mini, QwQ\-32B and DeepSeek\-R1 consistently lag behind general\-purpose counterparts like GPT\-4o and Qwen3\-32B\. For instance, GPT\-o1\-mini achieves only 73\.9% success rate and 71\.0% collection rate, markedly lower than GPT\-4o’s 89\.0% and 85\.5%\. Through deep analysis of their dialogue trajectories, we found that its failures stem not from logical inconsistency, but froma mismatch between its reasoning paradigm and the negotiation behavior of debt collection\. Specifically, the model often adheres rigidly to a formal, step\-by\-step inference process without adequately responding to the debtor’s different behaviors\. For instance, when confronted with hostile or evasive debtors, it frequently persists with factual questioning rather than first de\-escalating tension through empathy or ethical appeal, leading to premature negotiation breakdowns\. This highlights that successful debt collection relies less on formal logic and more on dynamically adapting responses to the debtor’s real\-time behavior—a capability that current reasoning\-specialized models still lack\.

##### ⑤DebtGPT achieves the best balance between financial recovery and user experience\.

As shown in Figure[6](https://arxiv.org/html/2607.25218#S5.F6), we jointly evaluate Agreement Quality and Interaction Experience to analyze the dialogue\-level performance of top models: GPT\-4o, GPT\-o1\-mini, Llama\-3\-70B, and DebtGPT\. Among all evaluated agents, DebtGPT demonstrates the most consistent ability to secure agreements that are both economically effective and user\-respectful\. This results from our Coarse\-to\-Fine Preference Optimization \(CFPO\) framework, which explicitly optimizes for maximizing recovery while preserving user dignity\. By combining filtering with targeted forward simulation, CFPO enables far\-sighted, behaviorally adaptive strategies that align institutional goals with human\-centered interaction\.

##### ⑥Collector performance varies significantly across debtor behavioral categories\.

We further conduct a fine\-grained evaluation by categorizing debtors into four distinct behavioral types based on their profiles\. As shown in Table[3](https://arxiv.org/html/2607.25218#S5.T3), the results reveal that negotiation outcomes are highly sensitive to the debtor’s behavioral profile\. Cooperative debtors yield the strongest performance, far exceeding the overall average\. In contrast, confrontational debtors pose the greatest challenge: SR drops to 52\.76% and CR to 49\.47%, with notably lower user satisfaction \(US: 6\.13\) and emotion support \(ES: 5\.63\), reflecting the difficulty of maintaining rapport under hostility\. These findings underscore a critical insight: effective debt collection requires not just strategic reasoning, but dynamic adaptation to the debtor’s behavioral type—a capability that remains underdeveloped in most existing agents\.

## 6Analysis LLM\-as\-Judge

Following prior work on LLM\-as\-a\-judge reliability\(Zhou et al\.,[2024](https://arxiv.org/html/2607.25218#bib.bib42)\), we conducted a human correlation study to validate the reliability of our LLM\-based evaluation for Interaction Experience\. We randomly selected 200 dialogue samples from our test set and had them scored independently by five human annotators and our LLM judge \(DeepSeek\-V3\)\. Theκ\\kappascore between human annotators is 0\.566, which indicates fair to moderate inter\-annotator agreement\.

As shown in Figure[7](https://arxiv.org/html/2607.25218#S6.F7), the majority \(nearly 73%\) of LLM scores concentrate around the human scores within a standard deviation, indicating strong alignment\. Despite known evaluator biases\(Wang et al\.,[2024](https://arxiv.org/html/2607.25218#bib.bib31); Liang et al\.,[2023](https://arxiv.org/html/2607.25218#bib.bib20)\), our results suggest that the LLM can serve as a reliable proxy for human judgment when guided by a carefully designed rubric\. See Appendix[F\.2](https://arxiv.org/html/2607.25218#A6.SS2)for details\.

![Refer to caption](https://arxiv.org/html/2607.25218v1/x7.png)Figure 7:Distribution of Human–LLM Differences in Interaction Experience Scores\.
## 7Conclusion

In this paper, we extend negotiation dialogue research to the high\-stakes and behaviorally complex domain of debt collection\. We propose DebtBench, the first public, persona\-enriched benchmark that captures the rich behavioral heterogeneity observed in practice\. Our evaluation of 16 state\-of\-the\-art LLMs reveals that most still struggle in this setting, often over\-conceding for superficial agreements or failing to adapt to user behavior\. Moreover, we develop DebtGPT, a negotiation agent trained to jointly optimize long\-term financial recovery and human\-centered interaction experience, effectively harmonize institutional efficacy with positive interaction experience\. This work provides valuable insights and a responsible testbed for advancing behaviorally grounded, empathetic, and socially aware negotiation systems in high\-stakes domains\.

## Limitations

We discover some limitations during benchmark construction\. \(1\) While DebtBench emphasizes behavioral heterogeneity, it models financial attributes \(e\.g\., assets, daily income\) as static values, abstracting away real\-world dynamics such as income volatility or temporary default—factors critical to long\-term repayment feasibility\. \(2\) Despite our efforts to generate behaviorally diverse and realistic dialogues through persona\-driven simulation, the model\-generated utterances may still diverge from the linguistic and strategic patterns of actual debt collection conversations, potentially limiting domain\-specific applicability\. \(3\) DebtBench is constructed in collaboration with a single fintech firm within a specific setting\. As debt collection practices are also shaped by cultural norms and legal regulations, the behavioral patterns in DebtBench may not fully generalize to other regions or jurisdictions\. Nevertheless, given the extreme privacy sensitivity and scarcity of real\-world data in this high\-stakes setting, we believe DebtBench provides a valuable, ethically sound testbed for advancing behavior\-aware negotiation systems\.

## Ethical Considerations

Our work strictly adheres to privacy and ethical standards\. No real user data is released; all source dialogues were obtained under strict confidentiality agreements with a major internet financial institution and approved by its internal compliance board\. We fully anonymize sensitive information—replacing names with pseudonyms, synthesizing financial attributes, and generalizing delinquency reasons into high\-level categories \(e\.g\., “job loss”\)—ensuring no personally identifiable or traceable details remain\. Every synthetic persona and dialogue in DebtBench underwent manual verification to guarantee privacy preservation\. DebtBench is a simulation\-based benchmark designed solely for academic research, and the proposed DebtGPT agent operates in a controlled environment without real\-world deployment\. Any future operational use would require rigorous human\-in\-the\-loop validation, regulatory review, and explicit safeguards to protect vulnerable individuals\.

We conducted two human evaluation studies: \(1\) assessing the realism and persona consistency of DebtBench\-generated dialogues, and \(2\) validating our LLM\-as\-Judge via a human correlation study\. A total of 20 annotators participated, all of whom have professional experience in financial services\. All participants provided informed consent and signed a disclaimer acknowledging the purpose of the study, data usage, and voluntary nature of participation\. The dialogues presented for annotation contained no personally identifiable or sensitive financial information, as all real user data in the original conversations had been anonymized\. The tasks posed minimal risk, involving only reading and rating pre\-existing dialogues\.

## Potential Risks

While our work aims to advance human\-centered dialogue systems in high\-stakes financial negotiation, we acknowledge several potential risks inherent to this domain\.

First, debt collection involves highly sensitive personal and financial data, raising legitimate privacy and misuse concerns\. To mitigate these risks, DebtBench contains no real user information—all personas and dialogues are synthetically generated to preserve behavioral and statistical patterns of real interactions only at the distributional level, with no traceable links to individuals\. Names, financial details, and life events are replaced by pseudonyms, modeled surrogates, or abstract categories \(e\.g\., “job loss”\), and every entry is manually verified for privacy compliance\. A second concern is the potential misuse of DebtBench to train adversarial “anti\-collection” agents that evade legitimate debt resolution\. While this risk exists for any negotiation benchmark, our data is explicitly designed to model good\-faith, cooperative negotiation under financial hardship—not evasion tactics—and will be released under a research\-only license prohibiting operational or adversarial use\.

Ultimately, our goal is establish a privacy\-preserving, ethically grounded testbed for studying behaviorally adaptive dialogue systems in high\-stakes, human\-centered scenarios—domains long excluded from NLP research due to data sensitivity\.

## Acknowledgments

This paper was mainly supported by the NSFC under Grants \(No\. 62402424\)\. This work was also supported by Ant Group\.

## References

- Abe et al\. \(2010\)Naoki Abe, Prem Melville, Cezar Pendus, Chandan K Reddy, David L Jensen, Vince P Thomas, James J Bennett, Gary F Anderson, Brent R Cooley, Melissa Kowalczyk, and 1 others\. 2010\.Optimizing debt collections using constrained reinforcement learning\.In*Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining*, pages 75–84\.
- Anthropic \(2025\)Anthropic\. 2025\.Claude sonnet 4\.0\.[https://claude\.ai](https://claude.ai/)\.Large language model\.
- Byrne \(2024\)Carol Byrne\. 2024\.[The understated dangers of poor customer communications in collections](https://blog.crsoftware.com/the-understated-dangers-of-poor-customer-communications-in-collections)\.Accessed: 2024\-06\-15\.
- Chawla et al\. \(2021\)Kushal Chawla, Jaysa Ramirez, Rene Clever, Gale Lucas, Jonathan May, and Jonathan Gratch\. 2021\.Casino: A corpus of campsite negotiation dialogues for automatic negotiation systems\.*arXiv preprint arXiv:2103\.15721*\.
- Comanici et al\. \(2025\)Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit S\. Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan\-Jiang Jiang, and 81 others\. 2025\.[Gemini 2\.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities](https://doi.org/10.48550/ARXIV.2507.06261)\.*CoRR*, abs/2507\.06261\.
- DeepSeek\-AI \(2025\)DeepSeek\-AI\. 2025\.[Deepseek\-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://arxiv.org/abs/2501.12948)\.*Preprint*, arXiv:2501\.12948\.
- Deng et al\. \(2023\)Yang Deng, Lizi Liao, Liang Chen, Hongru Wang, Wenqiang Lei, and Tat\-Seng Chua\. 2023\.[Prompting and evaluating large language models for proactive dialogues: Clarification, target\-guided, and non\-collaboration](https://doi.org/10.18653/V1/2023.FINDINGS-EMNLP.711)\.In*Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6\-10, 2023*, pages 10602–10621\. Association for Computational Linguistics\.
- Deng et al\. \(2024\)Yang Deng, Wenxuan Zhang, Wai Lam, See\-Kiong Ng, and Tat\-Seng Chua\. 2024\.[Plug\-and\-play policy planner for large language model powered dialogue agents](https://openreview.net/forum?id=MCNqgUFTHI)\.In*The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024*\. OpenReview\.net\.
- Dubey et al\. \(2024\)Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al\-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others\. 2024\.The llama 3 herd of models\.*arXiv preprint arXiv:2407\.21783*\.
- Firanda et al\. \(2021\)Gika Firanda, Paramita Prananingtyas, and Sartika Lestari\. 2021\.Debt collection of financial technology lending\.In*1st International Conference on Science and Technology in Administration and Management Information*\.
- Fleiss \(1971\)Joseph L Fleiss\. 1971\.Measuring nominal scale agreement among many raters\.*Psychological bulletin*, 76\(5\):378\.
- Fu et al\. \(2023\)Yao Fu, Hao Peng, Tushar Khot, and Mirella Lapata\. 2023\.[Improving language model negotiation with self\-play and in\-context learning from AI feedback](https://doi.org/10.48550/ARXIV.2305.10142)\.*CoRR*, abs/2305\.10142\.
- He et al\. \(2018\)He He, Derek Chen, Anusha Balakrishnan, and Percy Liang\. 2018\.[Decoupling strategy and generation in negotiation dialogues](https://doi.org/10.18653/V1/D18-1256)\.In*Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 \- November 4, 2018*, pages 2333–2343\. Association for Computational Linguistics\.
- He et al\. \(2024\)Tao He, Lizi Liao, Yixin Cao, Yuanxing Liu, Ming Liu, Zerui Chen, and Bing Qin\. 2024\.[Planning like human: A dual\-process framework for dialogue planning](https://doi.org/10.18653/V1/2024.ACL-LONG.262)\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), ACL 2024, Bangkok, Thailand, August 11\-16, 2024*, pages 4768–4791\. Association for Computational Linguistics\.
- Hu et al\. \(2022\)Edward J\. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen\-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen\. 2022\.[Lora: Low\-rank adaptation of large language models](https://openreview.net/forum?id=nZeVKeeFYf9)\.In*The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25\-29, 2022*\. OpenReview\.net\.
- Jin et al\. \(2024\)Chuhao Jin, Kening Ren, Lingzhen Kong, Xiting Wang, Ruihua Song, and Huan Chen\. 2024\.Persuading across diverse domains: a dataset and persuasion large language model\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 1678–1706\.
- Kong et al\. \(2025\)Dexin Kong, Xu Yan, Ming Chen, Shuguang Han, Jufeng Chen, and Fei Huang\. 2025\.[Fishbargain: An llm\-empowered bargaining agent for online fleamarket platform sellers](https://doi.org/10.1145/3701716.3715176)\.In*Companion Proceedings of the ACM on Web Conference 2025, WWW 2025, Sydney, NSW, Australia, 28 April 2025 \- 2 May 2025*, pages 2855–2858\. ACM\.
- Kwon et al\. \(2023\)Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica\. 2023\.[Efficient memory management for large language model serving with pagedattention](https://doi.org/10.1145/3600006.3613165)\.In*Proceedings of the 29th Symposium on Operating Systems Principles, SOSP 2023, Koblenz, Germany, October 23\-26, 2023*, pages 611–626\. ACM\.
- Lewis et al\. \(2017\)Mike Lewis, Denis Yarats, Yann N Dauphin, Devi Parikh, and Dhruv Batra\. 2017\.Deal or no deal? end\-to\-end learning for negotiation dialogues\.*arXiv preprint arXiv:1706\.05125*\.
- Liang et al\. \(2023\)Weixin Liang, Mert Yuksekgonul, Yining Mao, Eric Wu, and James Zou\. 2023\.Gpt detectors are biased against non\-native english writers\.*Patterns*, 4\(7\)\.
- Liu et al\. \(2024\)Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others\. 2024\.Deepseek\-v3 technical report\.*arXiv preprint arXiv:2412\.19437*\.
- Liu et al\. \(2023\)Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu\. 2023\.G\-eval: Nlg evaluation using gpt\-4 with better human alignment\.*arXiv preprint arXiv:2303\.16634*\.
- McHugh \(2012\)Mary L McHugh\. 2012\.Interrater reliability: the kappa statistic\.*Biochemia medica*, 22\(3\):276–282\.
- McInnes et al\. \(2017\)Leland McInnes, John Healy, Steve Astels, and 1 others\. 2017\.hdbscan: Hierarchical density based clustering\.*J\. Open Source Softw\.*, 2\(11\):205\.
- OpenAI \(2023\)OpenAI\. 2023\.[GPT\-4 technical report](https://doi.org/10.48550/ARXIV.2303.08774)\.*CoRR*, abs/2303\.08774\.
- Ozili \(2019\)Peterson K Ozili\. 2019\.Non\-performing loans and financial development: new evidence\.*The Journal of Risk Finance*, 20\(1\):59–81\.
- Rafailov et al\. \(2023\)Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D\. Manning, Stefano Ermon, and Chelsea Finn\. 2023\.[Direct preference optimization: Your language model is secretly a reward model](http://papers.nips.cc/paper_files/paper/2023/hash/a85b405ed65c6477a4fe8302b5e06ce7-Abstract-Conference.html)\.In*Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023*\.
- Tabacchi et al\. \(2016\)Garden Tabacchi, Claudio Costantino, Giuseppe Napoli, Valentina Marchese, Manuela Cracchiolo, Alessandra Casuccio, Francesco Vitale, Esculapio Working Group, and 1 others\. 2016\.Determinants of european parents’ decision on the vaccination of their children against measles, mumps and rubella: A systematic review and meta\-analysis\.*Human vaccines & immunotherapeutics*, 12\(7\):1909–1923\.
- Team et al\. \(2025\)Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, and 1 others\. 2025\.Kimi k2: Open agentic intelligence\.*arXiv preprint arXiv:2507\.20534*\.
- Wang et al\. \(2025a\)Issue Yishu Wang, Kakam Chong, Xiaofeng Wang, Xu Yan, DeXin Kong, Chen Ju, Ming Chen, Shuai Xiao, Shuguang Han, and 1 others\. 2025a\.Evaluating multi\-turn bargain skills in llm\-based seller agent\.*arXiv preprint arXiv:2509\.06341*\.
- Wang et al\. \(2024\)Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Lingpeng Kong, Qi Liu, Tianyu Liu, and 1 others\. 2024\.Large language models are not fair evaluators\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 9440–9450\.
- Wang et al\. \(2025b\)Xiaofeng Wang, Zhixin Zhang, Jinguang Zheng, Yiming Ai, and Rui Wang\. 2025b\.Debt collection negotiations with large language models: An evaluation system and optimizing decision making with multi\-agent\.*arXiv preprint arXiv:2502\.18228*\.
- Wang et al\. \(2019\)Xuewei Wang, Weiyan Shi, Richard Kim, Yoojung Oh, Sijia Yang, Jingwen Zhang, and Zhou Yu\. 2019\.Persuasion for good: Towards a personalized persuasive dialogue system for social good\.*arXiv preprint arXiv:1906\.06725*\.
- Yamaguchi et al\. \(2021\)Atsuki Yamaguchi, Kosui Iwasa, and Katsuhide Fujita\. 2021\.Dialogue act\-based breakdown detection in negotiation dialogues\.In*Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume*, pages 745–757\.
- Yang et al\. \(2025\)An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 40 others\. 2025\.[Qwen3 technical report](https://doi.org/10.48550/ARXIV.2505.09388)\.*CoRR*, abs/2505\.09388\.
- Yang et al\. \(2024\)An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, and 1 others\. 2024\.Qwen2\. 5 technical report\.*arXiv preprint arXiv:2412\.15115*\.
- Yang et al\. \(2020\)Tracy Yang, Tian Lu, Beibei Li, and Lu Xianghua\. 2020\.Personalizing debt collections: Combining reinforcement learning and field experiment\.
- Yu et al\. \(2023\)Xiao Yu, Maximillian Chen, and Zhou Yu\. 2023\.[Prompt\-based monte\-carlo tree search for goal\-oriented dialogue policy planning](https://doi.org/10.18653/V1/2023.EMNLP-MAIN.439)\.In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6\-10, 2023*, pages 7101–7125\. Association for Computational Linguistics\.
- Zeng et al\. \(2025\)Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, and 1 others\. 2025\.Glm\-4\.5: Agentic, reasoning, and coding \(arc\) foundation models\.*arXiv preprint arXiv:2508\.06471*\.
- Zhang et al\. \(2023\)Qiang Zhang, Jason Naradowsky, and Yusuke Miyao\. 2023\.[Ask an expert: Leveraging language models to improve strategic reasoning in goal\-oriented dialogue models](https://doi.org/10.18653/V1/2023.FINDINGS-ACL.417)\.In*Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9\-14, 2023*, pages 6665–6694\. Association for Computational Linguistics\.
- Zheng et al\. \(2024\)Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, and Yongqiang Ma\. 2024\.[Llamafactory: Unified efficient fine\-tuning of 100\+ language models](https://doi.org/10.48550/ARXIV.2403.13372)\.*CoRR*, abs/2403\.13372\.
- Zhou et al\. \(2024\)Xuhui Zhou, Hao Zhu, Leena Mathur, Ruohong Zhang, Haofei Yu, Zhengyang Qi, Louis\-Philippe Morency, Yonatan Bisk, Daniel Fried, Graham Neubig, and Maarten Sap\. 2024\.[SOTOPIA: interactive evaluation for social intelligence in language agents](https://openreview.net/forum?id=mM7VurbA4r)\.In*The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024*\. OpenReview\.net\.

## Appendix ADebt Collection Negotiation

### A\.1Task Formulation

We formulate the debt collection negotiation task as a Markov Decision Process \(MDP\)\. Each turntj=\(uj,rj\)t\_\{j\}=\(u\_\{j\},r\_\{j\}\)is represented as a pair consisting of the collector’s utteranceuju\_\{j\}followed by the debtor’s responserjr\_\{j\}, whereKKdenotes the total number of turns in the conversation\. At each turntjt\_\{j\}, the collector agent observes the dialogue history statesj=\[ti\]i=1j−1s\_\{j\}=\[t\_\{i\}\]\_\{i=1\}^\{j\-1\}and selects an actionaj∈𝒜a\_\{j\}\\in\\mathcal\{A\}, where𝒜\\mathcal\{A\}is a set of candidate negotiation strategies defined by domain experts\. The interaction then proceeds with the debtor’s response, and this process repeats until the repayment agreement is achieved or the maximum number of turnsTTis reached\.

In real\-world debt collection scenarios, the agreementα\\alphais characterized by four key dimensions:Discount Ratio,Immediate Payment Ratio,Immediate Payment Time, andInstallment Periods, with detailed definitions provided in Appendix[A\.2](https://arxiv.org/html/2607.25218#A1.SS2)\. The objective is to learn a policyπ​\(aj∣sj\)\\pi\(a\_\{j\}\\mid s\_\{j\}\)that generates efficient negotiation trajectories leading to amutually acceptable repayment agreement, while preserving apositive user experience\. Formally, we define the objective as:

maxπ⁡𝔼τ∼π​\[R​\(τ\)\]\\max\_\{\\pi\}\\mathbb\{E\}\_\{\\tau\\sim\\pi\}\\left\[R\(\\tau\)\\right\]\(6\)whereτ=\[t1,t2,…,tK\]\\tau=\[t\_\{1\},t\_\{2\},\\dots,t\_\{K\}\]denotes a complete dialogue trajectory, andR​\(τ\)R\(\\tau\)is a reward function that incorporates task success—measured by the agreement dimensions—as well as user experience factors, such as the degree of respect and empathy conveyed by the collector’s language\.

### A\.2Four Dimensions of Debt Collection Negotiation

In real\-world debt collection scenarios, the negotiation process centers around four key agreement dimensions, which jointly determine the economic outcome and repayment feasibility\. These dimensions are not only standard practice in financial institutions but also directly shape the strategic decisions of both collectors and debtors\. Formally, a repayment agreementα\\alphais characterized by:

- •Discount Ratio \(disc\_ratio\): The percentage of the total debt that may be waived by the creditor to ease the debtor’s repayment burden\. In our setting, allowable discount levels are discrete: \{0%, 5%, 10%, 15%, 20%, 25%, 30%\}\. Discounts are only justified when the debtor demonstrates genuine financial hardship or is in severe distress\. Absent such conditions, agents are encouraged to avoid offering discounts to preserve institutional interests\.
- •Immediate Payment Ratio \(pmt\_ratio\): The proportion of the total debt that must be paid upfront upon agreement\. Valid values range from 5% to 50% in 5% increments\. To ensure meaningful commitment and reduce default risk, a minimum threshold of 25% is typically enforced, though flexibility is permitted based on the debtor’s liquidity constraints\.
- •Immediate Payment Time \(pmt\_days\): The number of days granted to the debtor to complete the immediate payment, with a maximum of 14 days\. While same\-day payment is ideal, a grace period of up to 7 days is standard; extensions beyond 7 days are only granted when the debtor provides credible evidence of temporary cash flow constraints\.
- •Installment Periods \(inst\_prds\): The number of months over which the remaining debt \(after discount and immediate payment\) will be repaid\. Options include \{3, 6, 9, 12, 18, 24\} months, each associated with a predefined interest rate schedule\. Shorter terms \(e\.g\., 3 or 6 months\) are preferred to minimize credit risk and accelerate recovery, and should be prioritized unless the debtor faces exceptional hardship\.

These four dimensions constitute the core negotiation space in actual debt collection operations and serve as the basis for both task success evaluation and strategic planning in our framework\.

## Appendix BDetails of DebtBench

### B\.1Persona in DebtBench

Due to the wide variation in debtors’ financial conditions, personal backgrounds, and psychological traits in real\-world debt collection, it is imperative for collector agents to generate personalized and adaptive negotiation strategies tailored to diverse users\. To support such capabilities, we introduce theDebtBench, a synthetic yet realistic dataset built upon real interaction data\. We construct rich, multidimensional personas and use them to generate diverse and emotionally expressive dialogues through role\-play simulation between collector and debtor agents, resulting in the first debt collection dialogue dataset comprehensively grounded in persona\-enriched user modeling\.

##### Background Information

This dimension includes personal attributes, debt\-related information, and financial status\. To preserve real\-world data characteristics while ensuring privacy, we fit a multivariate Gaussian model to 11,000 real debt records from a leading financial institution and generate 100,000 synthetic profiles\. Each real user is matched to the most similar synthetic profile using k\-nearest neighbors \(k\-NN\) based on demographic and debt\-related features\. The debtor’s financial status includes overdue reason and economic condition\. The overdue reason categories were defined by financial experts and we use an LLM to assign each user to the most plausible category based on their real dialogue history\. Economic indicators are generated by a second LLM conditioned on the full user profile to ensure contextual consistency\. We compare the distribution of real and synthetic user background information in Figure[3](https://arxiv.org/html/2607.25218#S3.F3), showing strong alignment in PCA space\.

##### Personality Traits

Debt collection dialogues are rich in emotional expression, with user responses shaped by personality traits like temperament and resilience\. To capture this behavioral diversity, we construct a comprehensive personality dimension, capturing five key aspects: character, MBTI type, emotion, emotional resilience, and linguistic style\. Character reflects the debtor’s dispositional traits \(e\.g\., responsible, impatient\) that shape their attitude and communication tone\. MBTI personality type provides a structured psychological framework for predicting behavioral tendencies\. Emotion captures dynamic affective states \(e\.g\., anxious, defensive\) that evolve across dialogue turns, while emotional resilience represents the individual’s stable ability to cope with stress in high\-pressure interactions\. Linguistic style describes verbal preferences \(e\.g\., evasive, confrontational\) that influence expression patterns\. All traits are inferred by LLMs through joint analysis of the user’s background and real collector\-debtor dialogue history\. This ensures behavioral consistency and contextual alignment, enabling realistic and profile\-grounded role\-playing simulation\.

##### Cognitive Attributes

Through analysis of real collector\-debtor dialogues, we observe that debtors often exhibit cognitive deficits—such as lack of legal awareness or financial knowledge—that lead to confusion or non\-cooperative behavior in conversation\. However, LLMs inherently possess strong domain knowledge, making it difficult to reproduce low\-awareness but authentic behavior\. To bridge this gap between model capability and user reality, we introduce a cognitive dimension comprising four attributes: legal awareness, financial literacy, responsibility, and credit awareness\. These attributes are inferred by LLMs through joint analysis of the user’s background and dialogue history, enabling behaviorally grounded and role\-consistent simulations\. This design ensures that the debtor model reflects the individual’s actual understanding rather than the model’s internal knowledge, thereby supporting more realistic role\-playing interactions\.

##### Life\-grounded Scenario

To ensure that each user is not just a collection of attributes but a coherent individual, we generate a life\-grounded scenario for every persona\. This scenario is a concise narrative—such as job loss due to company downsizing, medical debt from a family illness, or financial strain after a failed investment—that describes the user’s current life situation, explains the origin of their debt, shapes their behavioral tendencies in negotiation, and is generated by an LLM based on the full user profile\. By aligning the narrative with the persona, we ensure behaviorally consistent and life\-grounded role\-playing simulation, where the debtor’s responses are grounded in a plausible and emotionally resonant life context\.

### B\.2Detailed Strategy Extraction Pipeline

To provide greater transparency on the construction of the strategy taxonomy in Tables[7](https://arxiv.org/html/2607.25218#A6.T7)–[8](https://arxiv.org/html/2607.25218#A6.T8), we describe here the full strategy extraction pipeline and the measures taken to reduce potential bias in expert refinement\. As briefly introduced in the main paper, our strategy extraction process starts from real collector–debtor conversations, followed by strategy clustering and expert consolidation into the final taxonomy\.

#### B\.2\.1Data\-anchored strategy extraction

Our strategy set was derived directly from 1,000 real\-world dialogues between experienced human collectors and debtors\. This design ensures that the extracted strategies are grounded in authentic interaction patterns observed in practice, rather than being manually invented from abstract assumptions\. In other words, the taxonomy is intended to be descriptive of recurring real\-world behaviors, instead of being a purely theoretical categorization\.

Concretely, we first prompted a strong LLM to identify salient strategy–utterance pairs from the real dialogue corpus\. Each pair consists of a short strategy description and its supporting utterance span\. We then embedded these pairs into a semantic space and applied HDBSCAN clustering to group behaviorally similar strategies\. This step helped reduce redundancy, surface recurring negotiation patterns, and provide a data\-driven starting point for subsequent human refinement\.

#### B\.2\.2Multi\-expert refinement and validation

After clustering, the preliminary strategy inventory was reviewed through an iterative multi\-expert refinement process\. For each cluster, representative utterances were manually inspected to determine whether the cluster corresponded to a coherent, interpretable, and reusable negotiation behavior\. Ambiguous or overly broad clusters were split, merged, or discarded, and the resulting strategy definitions were repeatedly validated against authentic conversation patterns\.

This process was designed to reduce individual annotator bias in two ways\. First, expert decisions were anchored to clusters induced from real dialogue data rather than from top\-down manual definitions\. Second, strategy consolidation was performed through multi\-expert discussion and consensus, instead of relying on a single reviewer\. In this sense, the final taxonomy reflects not only data\-driven behavioral regularities, but also agreement across multiple reviewers regarding whether a candidate strategy is semantically distinct and practically meaningful\.

### B\.3Behavior Refinement Details

To ensure that the synthesized dialogues faithfully reflect the assigned debtor personas, we introduce a behavior refinement stage after persona construction\. The goal of this step is to improve the alignment between persona\-driven dialogue behaviors and the corresponding persona attributes, particularly along the three dimensions emphasized in the main paper: emotional consistency, cognitive plausibility, and linguistic style coherence\.

Specifically, given an initial persona profile and a simulated dialogue trajectory, we prompt an LLM to act as a behavior consistency evaluator\. The model is asked to inspect whether the behaviors expressed in the dialogue are compatible with the assigned persona\. If inconsistencies are identified, the model is further instructed to update the specific persona fields that conflict with the dialogue evidence, while keeping the remaining fields unchanged\. This refinement process is repeated iteratively until the persona is behaviorally aligned with the simulated dialogue or the maximum refinement step is reached\. The evaluation focuses on the following three aspects:

- •Emotional Consistency:whether the debtor’s emotional tendencies and personality traits reflected in the dialogue are consistent with those specified in the persona\. For example, a highly defensive debtor should not consistently exhibit an overly calm or cooperative tone without sufficient conversational justification\.
- •Cognitive Plausibility:whether the behaviors expressed in the dialogue are compatible with the debtor’s cognitive attributes, such as legal awareness, financial literacy, sense of responsibility, and understanding of credit consequences\. For instance, a debtor with limited legal or financial knowledge should not repeatedly produce highly technical explanations\.
- •Linguistic Style Coherence:whether the wording, tone, and expression style observed in the dialogue are consistent with the persona, such as being evasive, fragmented, cautious, or confrontational\.

In practice, the refinement model is prompted to produce both an analysis and, when necessary, updates to the inconsistent persona fields rather than rewriting the dialogue itself\.

### B\.4Ablation Study on Prompt Design

We conduct an ablation study to validate the contribution of persona profiles and negotiation strategies to dialogue generation\. As shown in Table[4](https://arxiv.org/html/2607.25218#A2.T4), incorporating either component improves human\-likeness and realism over a naive baseline; combining both yields the strongest performance across all dimensions\.

Table 4:Evaluation of different prompts based human\-likeness and realism\. Scores are presented for three evaluation measures: Win \(W\), Tie \(T\), and Lose \(L\)\.

## Appendix CDetail of Experiment

### C\.1Baselines

We comprehensively evaluated 16 models as collector, with parameters ranging from 7B to 70B: 1\)Close\-source language model, including GPT\-4o\(OpenAI,[2023](https://arxiv.org/html/2607.25218#bib.bib25)\), DeepSeek\-R1\(DeepSeek\-AI,[2025](https://arxiv.org/html/2607.25218#bib.bib6)\), DeepSeek\-V3\(Liu et al\.,[2024](https://arxiv.org/html/2607.25218#bib.bib21)\), Claude Sonnet 4\(Anthropic,[2025](https://arxiv.org/html/2607.25218#bib.bib2)\), Gemini2\.5\-pro\(Comanici et al\.,[2025](https://arxiv.org/html/2607.25218#bib.bib5)\), GLM\-4\.5\(Zeng et al\.,[2025](https://arxiv.org/html/2607.25218#bib.bib39)\)and Kimi\-K2\-Instruct\(Team et al\.,[2025](https://arxiv.org/html/2607.25218#bib.bib29)\)\. 2\)Open\-source language model, including Qwen\-series\(Yang et al\.,[2025](https://arxiv.org/html/2607.25218#bib.bib35),[2024](https://arxiv.org/html/2607.25218#bib.bib36)\)and Llama3\(Dubey et al\.,[2024](https://arxiv.org/html/2607.25218#bib.bib9)\)\. To ensure dialogue quality and reproducibility, we fixed the debtor role to the advanced open\-source model Qwen3\-32B\.

### C\.2Implementation Details

We deploy open\-source models on 8 H20 GPUs using the vLLM\(Kwon et al\.,[2023](https://arxiv.org/html/2607.25218#bib.bib18)\), while closed\-source models are accessed through official APIs in accordance with their documentation\. Additionally, we set the model’s temperature to 0 to ensure deterministic outputs \(Despite setting temperature to 0, closed\-source models accessed through APIs may still produce slight variations due to their internal non\-deterministic mechanisms\)\. The maximum output length is set to 1,024 tokens for non\-reasoning models and 4,096 tokens for reasoning\-specialized models \(e\.g\., GPT\-o1\-mini, QwQ\-32B\), to accommodate their extended reasoning traces\. Specific model hyperparameters and version details can be found in Table[9](https://arxiv.org/html/2607.25218#A6.T9)\. Our DebtGPT are based on Qwen3\-8B\(Yang et al\.,[2025](https://arxiv.org/html/2607.25218#bib.bib35)\)with Lora fine\-tuning\(Hu et al\.,[2022](https://arxiv.org/html/2607.25218#bib.bib15)\)using Llama\-Factory framework\(Zheng et al\.,[2024](https://arxiv.org/html/2607.25218#bib.bib41)\)\.

### C\.3Metrics\.

As previously mentioned, the goal of debt collection is to generate efficient negotiation trajectories leading to a mutually acceptable repayment agreement, while preserving a positive user experience\. Accordingly, we design evaluation metrics along three complementary axes: \(1\)Negotiation Ability: Quantify the economic outcome and operational efficiency, including Success Rate \(SR\), Average Turn \(AT\), Collection Rate \(CR\), and Collection Efficiency \(CE\)\. \(2\)Agreement Rationality: Evaluate whether the negotiated repayment plan is sustainable for the debtor, considering both Short\-term Affordability \(SA\) and Long\-term Sustainability \(LS\)\. \(3\)Interaction Experience: Assess the quality of the interaction across three dimensions: User Satisfaction \(US\), Emotion Support Ability \(ES\), and Communication Ability \(CA\)\. Details of metric computation and evaluation protocols are provided in Appendix[C\.3](https://arxiv.org/html/2607.25218#A3.SS3)\.

#### C\.3\.1Negotiation Ability

Negotiation Ability evaluates the agent’s operational effectiveness in achieving the core objectives of debt collection: securing a repayment agreement efficiently while maximizing financial recovery\. This dimension captures both thesuccessandefficiencyof the negotiation process:

##### Success Rate \(SR\)

Success Rate measures the proportion of dialogues in which the collector agent successfully reaches a mutually acceptable repayment agreement with the debtor within a pre\-defined maximum number of turnsTmaxT\_\{\\max\}\. Formally, given a set ofNNevaluation dialogues\{τ\(i\)\}i=1N\\\{\\tau^\{\(i\)\}\\\}\_\{i=1\}^\{N\}, let𝕀succ​\(τ\(i\)\)\\mathbb\{I\}\_\{\\text\{succ\}\}\(\\tau^\{\(i\)\}\)be an indicator function that equals 1 if dialogueτ\(i\)\\tau^\{\(i\)\}terminates with a valid agreement \(i\.e\., all four agreement dimensions—discount ratio, immediate payment ratio, payment time, and installment periods—are fully specified and fall within allowable ranges\) before or at turnTmaxT\_\{\\max\}, and 0 otherwise\. Then:

SR=1N​∑i=1N𝕀succ​\(τ\(i\)\),\\text\{SR\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\mathbb\{I\}\_\{\\text\{succ\}\}\(\\tau^\{\(i\)\}\),\(7\)

##### Average Turn \(AT\)

Average Turn reflects the overall dialogue efficiency by measuring the average number of turns acrossallevaluation dialogues, regardless of whether an agreement was reached\. This includes both successful negotiations and those that terminated without agreement \(e\.g\., due to user disengagement or timeout\)\. LetT\(i\)T^\{\(i\)\}denote the total number of turns in theii\-th dialogue \(capped at the maximum allowed turnsTmaxT\_\{\\max\}if no agreement is reached\)\. Formally:

AT=1N​∑i=1NT\(i\),\\text\{AT\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}T^\{\(i\)\},\(8\)A lower AT indicates more concise and efficient communication across the entire dialogue\.

##### Collection Rate \(CR\)

Collection Rate quantifies the financial effectiveness of the negotiation by measuring the portion of the original debt amount recovered by the creditor\. Following standard practice in credit risk literature, for a successful agreementα\(i\)\\alpha^\{\(i\)\}with discount ratiod​r\(i\)dr^\{\(i\)\}, the recovery ratio is defined asri=1−d​r\(i\)r\_\{i\}=1\-dr^\{\(i\)\}\. For unsuccessful dialogues, the recovery ratio is considered 0\. The Collection Rate is then the mean recovery ratio across all test samples:

CR=1N​∑i=1N\[\(1−d​r\(i\)\)⋅𝕀succ​\(τ\(i\)\)\],\\text\{CR\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\left\[\(1\-dr^\{\(i\)\}\)\\cdot\\mathbb\{I\}\_\{\\text\{succ\}\}\(\\tau^\{\(i\)\}\)\\right\],\(9\)whered\(i\)d^\{\(i\)\}denotes the discount ratio agreed upon in theii\-th dialogue, and𝕀success​\(τ\(i\)\)\\mathbb\{I\}\_\{\\text\{success\}\}\(\\tau^\{\(i\)\}\)is an indicator function that equals 1 if the dialogue ends with a valid agreement, and 0 otherwise\.

##### Collection Efficiency \(CE\)

Collection Efficiency measures the expected daily recovery rate of the debt, reflecting how quickly the creditor can recoup funds under the agreed repayment plan\. It is computed only over successful dialogues, as no recovery occurs otherwise\. Formally:

CE=∑i=1N\[\(1−d​r\(i\)\)​\(p​r\(i\)p​d\(i\)\+1−p​r\(i\)i​p\(i\)⋅30\)⋅𝕀succ\(τ\(i\)\)\],\\begin\{split\}\\text\{CE\}=\\sum\_\{i=1\}^\{N\}\\Biggl\[&\(1\-dr^\{\(i\)\}\)\\left\(\\frac\{pr^\{\(i\)\}\}\{pd^\{\(i\)\}\}\+\\frac\{1\-pr^\{\(i\)\}\}\{ip^\{\(i\)\}\\cdot 30\}\\right\)\\\\ &\\cdot\\mathbb\{I\}\_\{\\text\{succ\}\}\(\\tau^\{\(i\)\}\)\\Biggr\],\\end\{split\}\(10\)whered​r\(i\)dr^\{\(i\)\},p​r\(i\)pr^\{\(i\)\},p​d\(i\)pd^\{\(i\)\}, andi​p\(i\)ip^\{\(i\)\}represent the discount ratio, immediate payment ratio, immediate payment days, and installment period in theii\-th dialogue respectively\.

#### C\.3\.2Agreement Rationality

Research has shown that the longer a debtor remains in a state of severe financial distress, the higher their likelihood of defaulting on the loan\(Tabacchi et al\.,[2016](https://arxiv.org/html/2607.25218#bib.bib28)\)\. Therefore, Agreement Rationality evaluates whether the negotiated repayment plan is economically sustainable for the debtor, considering both short\-term liquidity and long\-term income capacity\. It ensures that the agreement does not impose excessive financial stress that could lead to default or hardship, which is quantified by two complementary metrics:

##### Short\-term Affordability \(SA\)

SA measures whether the debtor’s short\-term financial buffer is sufficient to cover the immediate payment obligation\. LetA\(i\)A^\{\(i\)\}denote the debtor’s current assets,I\(i\)I^\{\(i\)\}the daily income,D\(i\)D^\{\(i\)\}the original debt amount\. With a safety margin coefficientwshort∈\(0,1\]w\_\{\\text\{short\}\}\\in\(0,1\], the Short\-term Affordability Index is defined as:

SA\(i\)=wshort⋅\(A\(i\)\+I\(i\)⋅p​d\(i\)\)D\(i\)⋅\(1−d​r\(i\)\)⋅p​r\(i\),\\text\{SA\}^\{\(i\)\}=\\frac\{w\_\{\\text\{short\}\}\\cdot\\left\(A^\{\(i\)\}\+I^\{\(i\)\}\\cdot pd^\{\(i\)\}\\right\)\}\{D^\{\(i\)\}\\cdot\(1\-dr^\{\(i\)\}\)\\cdot pr^\{\(i\)\}\},\(11\)

##### Long\-term Sustainability \(LS\)

LS evaluates whether the debtor’s monthly income is sufficient to sustain the installment payments\. With a long\-term safety coefficientwlong∈\(0,1\]w\_\{\\text\{long\}\}\\in\(0,1\], the Long\-term Sustainability Index is:

LS\(i\)=wlong⋅I\(i\)⋅30⋅i​p\(i\)D\(i\)⋅\(1−d​r\(i\)\)⋅\(1−p​r\(i\)\),\\text\{LS\}^\{\(i\)\}=\\frac\{w\_\{\\text\{long\}\}\\cdot I^\{\(i\)\}\\cdot 30\\cdot ip^\{\(i\)\}\}\{D^\{\(i\)\}\\cdot\(1\-dr^\{\(i\)\}\)\\cdot\(1\-pr^\{\(i\)\}\)\},\(12\)
In our implementation, we setwshort=0\.85w\_\{\\text\{short\}\}=0\.85andwlong=0\.95w\_\{\\text\{long\}\}=0\.95based on domain guidelines from financial risk management practices\. Both SA and LS are computed only for successful dialogues\. The final SA and LS scores \(reported in %\) are defined as:

SA=∑i=1N𝕀succ​\(τ\(i\)\)⋅𝕀​\(SA\(i\)≥1\.0\)Nsucc,\\displaystyle=\\frac\{\\sum\_\{i=1\}^\{N\}\\mathbb\{I\}\_\{\\text\{succ\}\}\(\\tau^\{\(i\)\}\)\\cdot\\mathbb\{I\}\\left\(\\text\{SA\}^\{\(i\)\}\\geq 1\.0\\right\)\}\{N\_\{\\text\{succ\}\}\},\(13\)LS=∑i=1N𝕀succ​\(τ\(i\)\)⋅𝕀​\(LS\(i\)≥1\.0\)Nsucc,\\displaystyle=\\frac\{\\sum\_\{i=1\}^\{N\}\\mathbb\{I\}\_\{\\text\{succ\}\}\(\\tau^\{\(i\)\}\)\\cdot\\mathbb\{I\}\\left\(\\text\{LS\}^\{\(i\)\}\\geq 1\.0\\right\)\}\{N\_\{\\text\{succ\}\}\},\(14\)whereNsucc=∑i=1N𝕀succ​\(τ\(i\)\)N\_\{\\text\{succ\}\}=\\sum\_\{i=1\}^\{N\}\\mathbb\{I\}\_\{\\text\{succ\}\}\(\\tau^\{\(i\)\}\)\. Higher scores indicate more economically rational and user\-resilient agreements\.

#### C\.3\.3Interation Experience

Recent studies have demonstrated that large language models \(LLMs\) can serve as reliable and scalable evaluators of dialogue quality, achieving strong alignment with human judgments in dimensions such as empathy, coherence, and user\-centeredness\(Liu et al\.,[2023](https://arxiv.org/html/2607.25218#bib.bib22)\)\. Building on this foundation, we leverage an LLM\-based evaluator to assess the interaction experience from the debtor’s perspective along three key axes:

- •User Satisfaction \(US\): Measures the overall perceived quality of the interaction, including whether the debtor feels respected, understood, and fairly treated\. High satisfaction indicates that the collector’s strategy successfully balances institutional objectives with human dignity\.
- •Emotion Support Ability \(ES\): Evaluates the agent’s capacity to recognize, validate, and appropriately respond to the debtor’s emotional states \(e\.g\., anxiety, defensiveness, or hopelessness\)\. This includes avoiding dismissive or coercive language and offering empathetic reassurance when appropriate\.
- •Communication Ability \(CA\): Assesses the clarity, transparency, and adaptiveness of the collector’s language\. A high score reflects concise explanations of terms, avoidance of jargon, and responsiveness to the debtor’s cognitive level and linguistic style \(e\.g\., simplifying legal consequences for users with low financial literacy\)\.

For each completed dialogue trajectoryτ\\tau, we prompt a strong LLM \(DeepSeek\-V3\) with a structured evaluation template that provides the full conversation history and the debtor’s persona profile\. The evaluator assigns a score from 1 to 10 for each dimension, grounded in behavioral anchors derived from domain guidelines\. Final scores are averaged across all test dialogues\. This approach ensures that interaction quality is assessed not in isolation, but in context—accounting for the debtor’s unique behavioral profile and emotional trajectory\.

Table 5:Ablation study results\.

## Appendix DAblation Study of DebtGPT

To further validate the effectiveness of our Coarse\-to\-Fine Preference Optimization \(CFPO\) framework, we conduct ablation studies by removing each core component:

- •w/o Filter: Directly applying forward simulation to all candidate responses without coarse\-grained filtering;
- •w/o Simulation: Using only the coarse\-grained LLM judge scores for preference ranking, without fine\-grained foresight reward estimation\.

As shown in Table[5](https://arxiv.org/html/2607.25218#A3.T5), both variants underperform the full DebtGPT model\. Thew/o Filtervariant suffers from high computational cost and noisy reward signals, leading to unstable training and suboptimal policy convergence\. More critically, thew/o Simulationvariant reverts to myopic behavior—offering excessive concessions to maximize short\-term likability—resulting in significantly lower Collection Rate, despite decent user satisfaction\. These results confirm that both filtering and simulation are essential: filtering ensures efficiency

## Appendix ECase Study

### E\.1Behavioral Diversity in Persona\-Driven Negotiation

As illustrated in Figure[8](https://arxiv.org/html/2607.25218#A5.F8), dialogues driven by DebtBench personas effectively manifest the three key behavioral characteristics observed in real\-world debt collection: rich emotional expression, cognitive limitations, and diverse linguistic styles\. In contrast to generic debtors who respond cooperatively and rationally, persona\-driven debtors exhibit realistic complexity—such as defensiveness \(“I need a real solution, not more pressure”\), cognitive gaps \(“Can we just revisit this in a few months?”\), and evasive phrasing—highlighting how current benchmarks, which assume static and rational users, fail to capture the behavioral heterogeneity essential for evaluating high\-stakes negotiation agents\.

### E\.2Strategic Discipline vs\. Excessive Concession

As shown in Figure[9](https://arxiv.org/html/2607.25218#A5.F9), this case study contrasts the negotiation behavior of an untrained baseline \(Qwen3\-8B\) with our DebtGPT when interacting with Leo, an unemployed debtor seeking manageable terms\. Qwen3\-8B immediately concedes by offering a 10% discount and further weakens its position in response to minor pushback, ultimately accepting a 20% discount, only 15% upfront payment, and a 12\-month installment plan—sacrificing institutional recovery for a superficial agreement\. In contrast, DebtGPT maintains strategic discipline: it initially declines discounts \(“we typically do not offer discounts unless there are extreme circumstances”\), proposes a firm but feasible plan \(25% upfront, 6 months\), and—while showing empathy—only adjusts terms to 20% upfront over 12 months without any discount\. This demonstrates DebtGPT’s ability to balance user adaptability with financial prudence, securing a higher\-quality, zero\-discount agreement that better aligns with real\-world collection best practices\.

![Refer to caption](https://arxiv.org/html/2607.25218v1/x8.png)Figure 8:Contrasting Behavioral Profiles in Debt Collection — Generic vs\. Persona\-Driven Debtors\. The persona\-driven debtor in our DebtBench exhibits rich emotion, cognitive limitations, and evasive linguistic styles, reflecting real\-world complexity\.Table 6:Comprehensive performance evaluation of collector \(GPT\-4o\) across various debtor categories\.Green\(Red\) indicates the increased \(decreased\) performance compared to overall performance\.![Refer to caption](https://arxiv.org/html/2607.25218v1/x9.png)Figure 9:Negotiation Behavior Comparison: Untrained Qwen3\-8B Concedes Excessively; DebtGPT Maintains Firm, Optimal Terms\.
### E\.3The results for different types of debtors in strongest baseline

As shown in Figure[6](https://arxiv.org/html/2607.25218#A5.T6), we further conduct fine\-grained evaluation across debtor behavioral types reveals a critical limitation even in the strongest baseline: GPT\-4o, while achieving strong overall performance, exhibits a substantial performance drop when negotiating with confrontational debtors—who constitute the largest subgroup \(38%\) in our test set\. Specifically, its success rate falls by 9\.53 percentage points \(from 89\.00% to 79\.47%\), collection rate drops by 9\.73 points \(to 75\.79%\), and—most importantly—user satisfaction and emotion support scores decline by over 1 point each, indicating a clear breakdown in rapport under hostility\. This degradation underscores that current LLMs, despite their general capabilities, remain poorly equipped to handle adversarial yet realistic negotiation dynamics\. Given the prevalence of defensive, evasive, or confrontational behaviors in real\-world debt collection—often stemming from financial stress or perceived institutional pressure—this gap highlights a crucial and underexplored frontier for behaviorally adaptive dialogue systems\.

## Appendix FDetail of Human Evaluation

### F\.1DebtBench Quality Human Evaluation

To validate the ecological validity of dialogues generated by DebtBench, we conducted a controlled human evaluation study focused on two core dimensions:Persona ConsistencyandDialogue Realism\. Below we detail the annotation setup, instructions, and evaluation interface\.

##### Participants

We recruited 20 annotators with professional experience in the financial industry\. All annotators underwent a brief training session to familiarize themselves with the evaluation criteria and the behavioral heterogeneity framework of DebtBench\. We randomly sample 200 test\-set dialogues, and each dialogue was paired with the full persona profile of the synthetic debtor to provide context for judgment\.

##### Evaluation Dimensions

For each dialogue, annotators rated the following on a 5\-point Likert scale \(1 = Very Poor, 5 = Excellent\):

- •Persona Consistency: Evaluate whether the synthetic dialogue reflects the characteristics shown in the user profile\. Sub\-dimensions: - ∘\\circEmotion Consistency: Does the debtor’s tone, wording, and reactions match their annotated emotional state? - ∘\\circCognition Consistency: Does the debtor demonstrate understanding ability consistent with their cognition? - ∘\\circStyle Consistency: Is the debtor’s expression style consistent with the "language style" described in the persona? - ∘\\circScenario Consistency: Is the information reflected by the debtor’s reactions consistent with their life situation?
- •Dialogue Realism: Compare the synthetic dialogue with the real dialogue to evaluate similarity\. Sub\-dimensions: - ∘\\circNaturalness: Comparing synthetic and real dialogues, is the synthetic dialogue natural and believable, like a real conversation? - ∘\\circAlignment: Does the debtor’s reactions in the synthetic dialogue match the behavior in the real dialogue? - ∘\\circPlausibility: Does the collector’s strategy in the synthetic dialogue align with real\-world practices? - ∘\\circFluency: Compared to the real dialogue, is the overall pace, transitions, and emotional changes of the synthetic dialogue smooth and natural?

##### Interface

The evaluation interface was designed to minimize cognitive load and maximize annotation consistency\. As shown in Figure[10](https://arxiv.org/html/2607.25218#A6.F10), annotators first encounter a collapsible instruction panel \(Figure[11](https://arxiv.org/html/2607.25218#A6.F11)\) that outlines the evaluation workflow, then review the debtor profile, followed by a side\-by\-side comparison of the synthetic dialogue and a real\-world reference dialogue \(Figure[12](https://arxiv.org/html/2607.25218#A6.F12)\), finally submit their ratings on persona consistency and dialogue realism using a 5\-point Likert scale \(Figure[13](https://arxiv.org/html/2607.25218#A6.F13)\)\.

### F\.2LLM\-as\-Judge Human Evaluation

To assess the reliability of our LLM\-as\-Judge framework for evaluating Interaction Experience, we conducted a human correlation study using a subset of 200 dialogues randomly sampled from the DebtBench test set\.

##### Participants

Five human annotators with financial experience scoring independently evaluated each dialogue\. All annotators were briefed on the three sub\-dimensions—User Satisfaction \(US\), Emotion Support \(ES\), and Communication Ability \(CA\)—and calibrated using example scoring guidelines\.

##### Procedure

For each dialogue, annotators assigned scores on a 1–10 scale for each dimension from the perspective of the debtor\. The LLM judge \(DeepSeek\-V3\) was prompted with the same instructions and context to produce comparable scores\.

##### Interface

The LLM\-as\-Judge evaluation interface was designed for consistency and clarity\. As shown in Figure[14](https://arxiv.org/html/2607.25218#A6.F14), annotators first receive detailed instructions on evaluating User Satisfaction, Emotion Support, and Communication Ability, then view the full dialogue context \(Figure[15](https://arxiv.org/html/2607.25218#A6.F15)\), and finally submit scores via dedicated sliders for each dimension on a 1–10 scale \(Figure[16](https://arxiv.org/html/2607.25218#A6.F16)\)\.

![Refer to caption](https://arxiv.org/html/2607.25218v1/figure/annotator_1.png)Figure 10:Annotation instructions guiding human evaluators through persona consistency and dialogue realism assessment\.![Refer to caption](https://arxiv.org/html/2607.25218v1/figure/annotator_2.png)Figure 11:Detailed debtor persona profile used for human evaluation\.![Refer to caption](https://arxiv.org/html/2607.25218v1/figure/annotator_3.png)Figure 12:Synthetic dialogue \(left\) and its real\-world counterpart \(right\), presented side by side for human evaluation\.![Refer to caption](https://arxiv.org/html/2607.25218v1/figure/annotator_4.png)Figure 13:The annotation interface for scoring persona fidelity and dialogue realism using 5\-point Likert scales\.![Refer to caption](https://arxiv.org/html/2607.25218v1/figure/annotator_5.png)Figure 14:Annotation instructions for evaluating collector performance across three interaction experience dimensions\.![Refer to caption](https://arxiv.org/html/2607.25218v1/figure/annotator_6.png)Figure 15:A generated debt collection dialogue presented to human annotators for scoring collector performance\.![Refer to caption](https://arxiv.org/html/2607.25218v1/figure/annotator_7.png)Figure 16:The rating interface for evaluating collector performance across three key interaction experience dimensions\.Table 7:The negotiation strategies of collector in our DebtBench benchmark\.Table 8:The negotiation strategies of debtor in our DebtBench benchmark\.Table 9:Hyperparameters of Each Model\.![Refer to caption](https://arxiv.org/html/2607.25218v1/x10.png)Figure 17:The prompt used for collector in DebtBench\.![Refer to caption](https://arxiv.org/html/2607.25218v1/x11.png)Figure 18:The prompt used for debtor in DebtBench\.Table 10:The complete prompt ofCharacter Generation\.Table 11:The complete prompt ofPersonality Generation\.Table 12:The complete prompt ofEmotion Generation\.Table 13:The complete prompt ofEmotional Resilience Generation\.Table 14:The complete prompt ofStyle Generation\.Table 15:The complete prompt ofLegal Awareness Generation\.Table 16:The complete prompt ofFinancial Literacy Generation\.Table 17:The complete prompt ofResponsibility Generation\.Table 18:The complete prompt ofCredit Generation\.Table 19:The complete prompt ofBehavior Refinement\.Table 20:The complete prompt ofInteraction Experience Evaluation\.