Strategic Bargaining in Multi-Buyer Markets: Reinforcement Learning from Verifiable Rewards for LLM Negotiations

# 1 Introduction
Source: [https://arxiv.org/html/2607.05863](https://arxiv.org/html/2607.05863)
\\OneAndAHalfSpacedXII\\EquationsNumberedThrough\\TheoremsNumberedBySection\\ECRepeatTheorems\\MANUSCRIPTNO\\OneAndAHalfSpacedXI

\\RUNAUTHOR

Liu et al\.\\RUNTITLELLM Negotiation with Verifiable Rewards

\\TITLE

Strategic Bargaining in Multi\-Buyer Markets: Reinforcement Learning from Verifiable Rewards for LLM Negotiations

\\ARTICLEAUTHORS\\AUTHORfont\\HD

150Shuze Daniel Liu\\HD06

\\AFFfont

Institute for Data, Systems, and Society, Massachusetts Institute of Technology,\\EMAILshuzel@mit\.edu;

Mitch Daniels School of Business, Purdue University,\\EMAILdaniel\.liu@purdue\.edu

\\AUTHORfont\\HD

150Claire Chen\\HD06

\\AFFfont

The Division of Physics, Mathematics and Astronomy, California Institute of Technology,\\EMAILclairechen@caltech\.edu

\\AUTHORfont\\HD

150Jiabao Sean Xiao\\HD06

\\AFFfont

Department of Computing and Mathematical Sciences, California Institute of Technology,\\EMAILseanxiao@caltech\.edu

\\AUTHORfont\\HD

150Xin Chen\\HD06

\\AFFfont

H\. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology,\\EMAILxin\.chen@isye\.gatech\.edu

\\AUTHORfont\\HD

150David Simchi\-Levi\\HD06

\\AFFfont

Institute for Data, Systems, and Society, Department of Civil and Environmental Engineering, Operations Research Center, Massachusetts Institute of Technology,\\EMAILdslevi@mit\.edu;

Mitch Daniels School of Business, Purdue University,\\EMAILdslevi@purdue\.edu

\\KEYWORDS

LLM agents; reinforcement learning with verifiable rewards; negotiation; multi\-agent bargaining; surplus extraction

Negotiation is a foundational process in management science and operations research, serving as the primary mechanism for value discovery and resource allocation in environments where prices are not static\(Simchi\-Leviet al\.[2005](https://arxiv.org/html/2607.05863#bib.bib21), Backuset al\.[2020](https://arxiv.org/html/2607.05863#bib.bib53)\)\. In fields ranging from procurement and supply chain contracting to dispute resolution and online marketplaces, agents must strategically interact to coordinate and realize gains from trade\(Cachon and Lariviere[2001](https://arxiv.org/html/2607.05863#bib.bib20), Desai and Purohit[2004](https://arxiv.org/html/2607.05863#bib.bib14)\)\. Within management science literature, the analysis of these strategic interactions focuses heavily on aligning decentralized incentives and resolving information gaps between trading partners\(Nagarajan and Sošić[2008](https://arxiv.org/html/2607.05863#bib.bib18), Lovejoy[2010](https://arxiv.org/html/2607.05863#bib.bib19)\)\.

A central challenge in any negotiation is that agents possess private information—specifically, a seller has a minimum cost they are willing to accept, and a buyer has a maximum budget they are willing to pay—which remains hidden from their counterpart\(Chatterjee and Samuelson[1983](https://arxiv.org/html/2607.05863#bib.bib52)\)\. To capture meaningful value under these asymmetric conditions, a seller must maintain strict strategic discipline\. This requires simultaneously enforcing a private cost boundary to prevent loss\-making transactions, reading buyer signals to extract maximum market surplus, and effectively coordinating with the buyer to establish a common price point and close the deal\.

While classical literature often focuses on bilateral \(one\-on\-one\) trade\(Nash and others[1950](https://arxiv.org/html/2607.05863#bib.bib50), Rubinstein[1982](https://arxiv.org/html/2607.05863#bib.bib74), Chatterjee and Samuelson[1983](https://arxiv.org/html/2607.05863#bib.bib52)\), a highly prevalent scenario in modern management is concurrent negotiation, where a single focal seller interacts simultaneously with multiple independent buyers across separate, sealed channels\(Backuset al\.[2020](https://arxiv.org/html/2607.05863#bib.bib53)\)\. This parallel structure is central to digital marketplaces like eBay and business\-to\-business procurement platforms, where a seller allocates an available asset or supply contract by negotiating across a pool of potential buyers\(Backuset al\.[2020](https://arxiv.org/html/2607.05863#bib.bib53)\)\. Crucially, because the seller operates under a limited communication capacity across these parallel channels, it faces a fundamental trade\-off between exploration and exploitation\. The seller cannot afford extended bargaining cycles with every participant; instead, it must balance broad market screening to identify the counterparty with the highest hidden valuation against concentrated, single\-buyer bargaining to maximize surplus extraction\. When deploying autonomous or automated agents in these settings to act as direct representatives of the firm, failing to locate the optimal partner or settling for thin margins with an early bidder results in an immediate loss of organizational value\.

To manage these concurrent negotiations at scale, firms are actively deploying Large Language Models \(LLMs\) as autonomous bargaining agents\. In digital platforms like Facebook Marketplace, automated tools assist users by managing simultaneous price inquiries and drafting real\-time offers across independent buyer channels\(Meta[2026](https://arxiv.org/html/2607.05863#bib.bib2)\)\. Similarly, in enterprise procurement, retail leaders like Walmart utilize automated negotiation systems to execute binding contracts with thousands of tail\-end suppliers, bypassing the capacity constraints of human purchasing teams\(Van Hoeket al\.[2022](https://arxiv.org/html/2607.05863#bib.bib3)\)\. However, these current implementations operate primarily as reactive text filters designed to minimize conversational friction\(Meta[2026](https://arxiv.org/html/2607.05863#bib.bib2)\), rather than strategic revenue management systems\(Gallego and Hu[2014](https://arxiv.org/html/2607.05863#bib.bib10)\)\. Faced with a limited communication budget, the agent’s primary task is not conversational fluency, but using the dialogue to screen hidden buyer valuations and protect profit margins by allocating communication efforts to the highest\-value counterparties\.

Our analysis reveals a critical misalignment between this required strategic discipline and the operational behavior of generalist frontier models\. While standard LLMs excel at maintaining conversational fluency, they typically exhibit an agreeableness bias, maximizing total deal volume at the expense of overall economic surplus\(Ouyanget al\.[2022](https://arxiv.org/html/2607.05863#bib.bib76), Perezet al\.[2023](https://arxiv.org/html/2607.05863#bib.bib27), Sharmaet al\.[2024](https://arxiv.org/html/2607.05863#bib.bib1)\)\. Because these models are optimized via reinforcement learning from human feedback for politeness and cooperation, they are prone to making suboptimal concessions rather than investigating the buyer pool to discover latent high valuations\. Consequently, this bias turns their conversational fluency into a financial liability, preventing the agent from maximizing captured surplus\.

This paper addresses these limitations\. We make three primary contributions:

1. 1\.A concurrent negotiation framework for evaluating exploration\-exploitation trade\-offs \(Section[3](https://arxiv.org/html/2607.05863#S3)\)\.We establish a structured environment where a seller negotiates concurrently against multiple buyers with diverse budgets through private, one\-on\-one communication channels\. This setting allows us to measure an agent’s ability to balance the strategic discovery of high\-budget buyers with focused bargaining to extract surplus under limited communication turns\.
2. 2\.A behavioral analysis identifying the exploration failure in frontier LLMs \(Section[5](https://arxiv.org/html/2607.05863#S5)\)\.We demonstrate that even state\-of\-the\-art general\-purpose models, including those with specialized reasoning capabilities, fail to maximize surplus in multi\-buyer settings\. Our results reveal a systematic tendency to settle for suboptimal deals by fixating on early high bids rather than investigating the broader buyer pool to identify buyers with higher private budgets\.
3. 3\.A specialized training recipe for strategic agents using Reinforcement Learning from Verifiable Rewards \(Section[4](https://arxiv.org/html/2607.05863#S4)\)\.We demonstrate that Reinforcement Learning from Verifiable Rewards \(RLVR\) can effectively bridge the gap between conversational fluency and strategic utility\. Our formulation utilizes objective economic outcomes—specifically deal surplus—and strict instruction\-following protocols to guide the learning process\. Our results show that this approach enables the emergence of proactive market discovery mechanisms, such as diagnostic probing and price anchoring, allowing agents to identify profitable partners even when facing unfamiliar buyer behaviors and unseen budget distributions\.

The remainder of the paper is organized as follows\. Section[2](https://arxiv.org/html/2607.05863#S2)reviews related work on negotiation in management science and LLM agents\. Section[3](https://arxiv.org/html/2607.05863#S3)formalizes the concurrent bargaining environment as a Markov Decision Process and specifies the agent system\. Section[4](https://arxiv.org/html/2607.05863#S4)analyzes the behavioral evolution and phase shifts observed during reinforcement learning training\. Section[5](https://arxiv.org/html/2607.05863#S5)presents the in\-distribution performance benchmarking results against frontier models, and Section[6](https://arxiv.org/html/2607.05863#S6)evaluates the strategic robustness and market generalization of our agent\. Finally, Section[7](https://arxiv.org/html/2607.05863#S7)discusses the broader strategic implications of our findings, and Section[8](https://arxiv.org/html/2607.05863#S8)concludes the paper\.

## 2Related Work

We organize the relevant literature into the following streams: negotiation in management science, the evaluation of LLMs as strategic agents, automated negotiation systems, reinforcement learning for language models, and the foundations of classical bargaining theory\.

### 2\.1Negotiation in Management Science and Operations

Negotiation serves as a fundamental mechanism for price discovery and supply chain coordination within management science, particularly when firms lack direct visibility into a partner’s private costs or reservation thresholds\. Research in management science extensively evaluates these bargaining dynamics to optimize incentive alignment in procurement and capacity contracting under asymmetric information\. Traditional literature in management science typically addresses these information gaps using screening menus, strategic information sharing frameworks, or decentralized networks to reallocate bargaining power\(Cachon and Lariviere[2001](https://arxiv.org/html/2607.05863#bib.bib20), Luet al\.[2006](https://arxiv.org/html/2607.05863#bib.bib5), Nagarajan and Sošić[2008](https://arxiv.org/html/2607.05863#bib.bib18), Lovejoy[2010](https://arxiv.org/html/2607.05863#bib.bib19)\)\. Additionally, when inventory is limited and buyers are strategic, bargaining interfaces deeply with classical capacity contracting and revenue management, as sellers must anticipate how partner valuations and cost horizons shift over time\(Aviv and Pazgal[2008](https://arxiv.org/html/2607.05863#bib.bib16), Su[2010](https://arxiv.org/html/2607.05863#bib.bib17)\); this operational tension requires tailored contracts that adjust payment timing to balance inventory choices with the financial costs faced by each stage of the supply chain\(Tonget al\.[2020](https://arxiv.org/html/2607.05863#bib.bib11)\)\.

In digital platforms and retail operations, the market landscape has rapidly shifted from static posted pricing toward highly responsive, competitive dynamic pricing frameworks\(Gallego and Hu[2014](https://arxiv.org/html/2607.05863#bib.bib10), Wang and Hu[2014](https://arxiv.org/html/2607.05863#bib.bib7), Ban and Keskin[2021](https://arxiv.org/html/2607.05863#bib.bib4)\)alongside multi\-round bilateral negotiations and sequential clearing processes\(Desai and Purohit[2004](https://arxiv.org/html/2607.05863#bib.bib14), Backuset al\.[2020](https://arxiv.org/html/2607.05863#bib.bib53)\)\. This transition fundamentally alters platform resource allocation, intertemporal price discrimination, and the strategic distribution of market surplus\. Large\-scale empirical marketplace data confirms that sequential counter\-offers account for a significant portion of online transactions, establishing multi\-turn dialog as a major channel for price discovery\(Backuset al\.[2020](https://arxiv.org/html/2607.05863#bib.bib53), Bergemannet al\.[2026](https://arxiv.org/html/2607.05863#bib.bib54)\)\. Our work addresses a critical intersection of these fields by modeling a concurrent market structure where a single seller must allocate a limited number of communication turns across multiple separate, private negotiation channels\. This simultaneous challenge aligns with revenue management streams demonstrating that firms maximize performance by executing learning and operational actions concurrently rather than sequentially\(Agrawalet al\.[2014](https://arxiv.org/html/2607.05863#bib.bib6), Wanget al\.[2014](https://arxiv.org/html/2607.05863#bib.bib8)\)\. Under this formulation, gathering market data through strategic interactions serves as a core requirement for protecting reservation costs and ensuring overall profitability\(Guo[2023](https://arxiv.org/html/2607.05863#bib.bib13)\)\. To navigate this high\-dimensional environment without relying on restrictive structural assumptions, our approach builds upon the growing paradigm of integrating data\-driven prescriptive analytics and machine learning directly into operational decision rules\(Ban and Rudin[2019](https://arxiv.org/html/2607.05863#bib.bib9)\)\.

### 2\.2LLMs as Negotiating and Strategic Agents

Recent work has increasingly utilized Large Language Models \(LLMs\) as autonomous agents in goal\-directed settings, ranging from tool\-assisted reasoning to complex decision\-making tasks\(Yaoet al\.[2022](https://arxiv.org/html/2607.05863#bib.bib37), Liuet al\.[2024](https://arxiv.org/html/2607.05863#bib.bib38), Wanget al\.[2024a](https://arxiv.org/html/2607.05863#bib.bib93), Zenget al\.[2024](https://arxiv.org/html/2607.05863#bib.bib36)\)\. However, systematic evaluations have revealed significant behavioral shortcomings in their strategic reasoning and operational logic\.Xiaet al\.\([2024](https://arxiv.org/html/2607.05863#bib.bib96)\)find that frontier models frequently violate principles of negotiation rationality and are susceptible to adversarial pressure, whileDavidsonet al\.\([2024](https://arxiv.org/html/2607.05863#bib.bib82)\)document a failure to maintain consistent strategic goals in multi\-turn interactions\. Furthermore, general\-purpose LLMs often struggle to manage the tension between cooperation and competition, a core requirement of effective bargaining\(Fuet al\.[2023](https://arxiv.org/html/2607.05863#bib.bib89)\)\.

To evaluate these capabilities,Bianchiet al\.\([2024](https://arxiv.org/html/2607.05863#bib.bib84)\)establish a platform to analyze how general\-purpose models navigate bargaining dynamics\. Subsequently,Bergemannet al\.\([2026](https://arxiv.org/html/2607.05863#bib.bib54)\)andLiuet al\.\([2026](https://arxiv.org/html/2607.05863#bib.bib56)\)demonstrate that specialized reinforcement learning can enhance the strategic competence of agents in bilateral \(one\-on\-one\) trade\. While these existing approaches are restricted to bilateral negotiation settings, we develop a reinforcement learning method for concurrent multi\-buyer markets\. This allows us to address a systematic market exploration failure where frontier models fixate on immediate bids rather than investigating alternative candidates\. Unlike purely one\-on\-one training, our method enables the agent to resolve the broader tension between market discovery and value extraction\.

### 2\.3Automated Negotiation and Dialogue Systems

The development of automated bargaining agents predates the current LLM era, typically operating in structured, language\-free environments\. Early systems utilized game\-theoretic heuristics or reinforcement learning trained on historical transaction data\(Buettner[1997](https://arxiv.org/html/2607.05863#bib.bib32), Jenningset al\.[2001](https://arxiv.org/html/2607.05863#bib.bib33), Kraus[2001](https://arxiv.org/html/2607.05863#bib.bib34), Green and Plunkett[2022](https://arxiv.org/html/2607.05863#bib.bib101)\)\. More recent efforts have integrated natural language, though often relying on modular architectures that decouple numerical pricing strategies from text generation\(Lewiset al\.[2017](https://arxiv.org/html/2607.05863#bib.bib92), Heet al\.[2018](https://arxiv.org/html/2607.05863#bib.bib30), Yarats and Lewis[2018](https://arxiv.org/html/2607.05863#bib.bib31)\)\.

While LLMs have removed these linguistic bottlenecks, they have introduced a new need for specialized training to ensure strategic effectiveness\(Chawlaet al\.[2023](https://arxiv.org/html/2607.05863#bib.bib29)\)and high\-level goal consistency in the face of counterparty pressure\(Xiaet al\.[2024](https://arxiv.org/html/2607.05863#bib.bib96)\)\. Efforts to improve LLM performance include prompt\-based modular systems that utilize game\-theoretic feedback\(Fuet al\.[2023](https://arxiv.org/html/2607.05863#bib.bib89)\)and specialized prompting frameworks designed to manage competitive and cooperative bargaining logic\(Abdelnabiet al\.[2024](https://arxiv.org/html/2607.05863#bib.bib28)\)\. Other approaches rely on supervised fine\-tuning \(SFT\) and reinforcement learning using synthetic negotiation datasets to improve strategic consistency\(Lewiset al\.[2017](https://arxiv.org/html/2607.05863#bib.bib92), Chawlaet al\.[2023](https://arxiv.org/html/2607.05863#bib.bib29)\)\. We build upon this lineage by providing a unified reinforcement learning \(RL\) pipeline that optimizes both persuasive communication and high\-level market search, moving beyond modular designs toward an end\-to\-end strategic agent\.

### 2\.4Reinforcement Learning from Verifiable Rewards

The adaptation of general\-purpose Large Language Models \(LLMs\) for specialized domain tasks represents a central focus in post\-training development\. While standard approaches rely on Reinforcement Learning from Human Feedback \(RLHF\) to refine conversational outputs\(Ziegleret al\.[2019](https://arxiv.org/html/2607.05863#bib.bib47), Ouyanget al\.[2022](https://arxiv.org/html/2607.05863#bib.bib76)\), such subjective preference signals are often a poor proxy for objective economic utility\. Human labelers tend to reward linguistic agreeableness and superficial helpfulness over strategic rigor\(Perezet al\.[2023](https://arxiv.org/html/2607.05863#bib.bib27), Shahet al\.[2026](https://arxiv.org/html/2607.05863#bib.bib26)\), creating a misalignment where models prioritize conversational fluency over effective strategic execution\(Casperet al\.[2023](https://arxiv.org/html/2607.05863#bib.bib25)\)\. To mitigate these biases and align model actions with concrete strategic goals, recent research has shifted toward Reinforcement Learning from Verifiable Rewards \(RLVR\)\(Shaoet al\.[2024](https://arxiv.org/html/2607.05863#bib.bib88), Guoet al\.[2025](https://arxiv.org/html/2607.05863#bib.bib99)\)\. Under this methodology, performance is evaluated programmatically through objective, quantifiable metrics, ensuring that the feedback loop remains anchored to measurable outcomes rather than subjective human judgment\(Guoet al\.[2025](https://arxiv.org/html/2607.05863#bib.bib99)\)\.

This approach has proven effective in improving the reasoning capabilities of agents in domains such as mathematics and code generation\(Cobbeet al\.[2021](https://arxiv.org/html/2607.05863#bib.bib24), Maet al\.[2023](https://arxiv.org/html/2607.05863#bib.bib23), Wanget al\.[2024b](https://arxiv.org/html/2607.05863#bib.bib22), Shaoet al\.[2024](https://arxiv.org/html/2607.05863#bib.bib88)\), where objective rewards—such as mathematical correctness or unit\-test success—incentivize the discovery of optimal trajectories that human supervisors might not anticipate\(Guoet al\.[2025](https://arxiv.org/html/2607.05863#bib.bib99)\)\. We utilize this Reinforcement Learning from Verifiable Rewards \(RLVR\) framework to address concurrent negotiation in multi\-buyer markets, where deal profit serves as a verifiable objective to guide both bargaining skills and the strategic discovery of latent market information\. By tying optimization to verifiable economic outcomes, we incentivize the emergence of proactive exploration mechanisms—such as diagnostic probing and price anchoring—necessary for identifying the highest\-budget buyer and acquiring high deal surplus in information\-constrained environments\.

### 2\.5Foundations of Classical Bargaining Theory

Bargaining theory establishes the foundation for bilateral trade under asymmetric information, building on the formulations ofHarsanyi \([1967](https://arxiv.org/html/2607.05863#bib.bib73)\),Nash and others \([1950](https://arxiv.org/html/2607.05863#bib.bib50)\),Ståhl \([1972](https://arxiv.org/html/2607.05863#bib.bib39)\),Rubinstein \([1982](https://arxiv.org/html/2607.05863#bib.bib74)\), andChatterjee and Samuelson \([1983](https://arxiv.org/html/2607.05863#bib.bib52)\)\. A critical baseline for this domain is provided byMyerson and Satterthwaite \([1983](https://arxiv.org/html/2607.05863#bib.bib51)\), who define the fundamental limits of allocative efficiency under private information\. Subsequent game\-theoretic work expanded these formulations to sequential settings with dynamic information revelation\(Fudenberg and Tirole[1983](https://arxiv.org/html/2607.05863#bib.bib43), Sobel and Takahashi[1983](https://arxiv.org/html/2607.05863#bib.bib42), Gulet al\.[1986](https://arxiv.org/html/2607.05863#bib.bib40)\)\. While these classical models derive analytical equilibrium conditions for bilateral trade, their strategies are restricted to numerical pricing choices and typically require the seller to know the prior distribution of market valuations\. In contrast, our work targets a concurrent multi\-buyer setting using a reinforcement learning approach where the agent dynamically learns to negotiate via natural language dialogue without explicit access to these market priors\.

## 3Problem Formulation and RL Framework

We formalize seller negotiation as a decision problem in which the seller is the only learning agent\. The environment consists ofNNheterogeneous buyers and a system of private, one\-on\-one communication channels that ensures information remains hidden between participants\. The seller holds a private reservation cost𝒞\\mathcal\{C\}and is matched againstNNbuyers indexedi∈\{1,…,N\}i\\in\\\{1,\\ldots,N\\\}, each carrying a private budgetℬi\\mathcal\{B\}\_\{i\}drawn from a distribution\. All buyers see the same product, which carries a public list pricePlistP\_\{\\text\{list\}\}\. But each buyer’s conversation is sealed: a buyer observes only the seller’s messages directed to that buyer, and never the otherN−1N\-1dialogues\. The seller must allocate a finite total ofTTmessages across theNNbuyer slots subject to a per\-buyer capKKwith a total limitT<N​KT<NK\. Thus, the seller cannot exhaust every conversation and is forced to choose where to spend attention\.

![Refer to caption](https://arxiv.org/html/2607.05863v1/x1.png)Figure 1:Conceptual overview of the concurrent negotiation environment\.The seller agent must strategically allocate a limited communication budgetTTacrossNNsealed buyer channels\. Each channel is private, ensuring that buyers cannot observe other dialogues or valuations \(asymmetric information\)\. To maximize surplus, the seller must utilize diagnostic probing to infer the hidden budgets of the buyers and identify the optimal partner while respecting its own private reservation cost\.### 3\.1Concurrent Seller Negotiation as a POMDP

The concurrent negotiation framework is characterized by asymmetric and incomplete information, as the seller cannot directly observe the buyers’ private budgetsℬi\\mathcal\{B\}\_\{i\}\(Chatterjee and Samuelson[1983](https://arxiv.org/html/2607.05863#bib.bib52)\)\. Formally, we model this environment from the seller’s perspective as a finite\-horizon Partially Observable Markov Decision Process \(POMDP\) defined by the tuple\(𝒳,𝒜,𝒫,ℛ,𝒪,Ω,T\)\(\\mathcal\{X\},\\mathcal\{A\},\\mathcal\{P\},\\mathcal\{R\},\\mathcal\{O\},\\Omega,T\), where𝒳\\mathcal\{X\}is the latent state space,𝒜\\mathcal\{A\}is the action space,𝒫\\mathcal\{P\}is the state transition probability function,ℛ\\mathcal\{R\}is the reward function,𝒪\\mathcal\{O\}is the single\-turn observation space,Ω\\Omegais the observation emission process, andTTis the finite game horizon\. Specifically, the latent statex=\(ℬ1,…,ℬN\)∈𝒳x=\(\\mathcal\{B\}\_\{1\},\\ldots,\\mathcal\{B\}\_\{N\}\)\\in\\mathcal\{X\}represents the ground\-truth vector of hidden buyer budgets\. At each step, the observation emission processΩ\\Omegamaps the latent state and the seller’s prior actionat−1∈𝒜a\_\{t\-1\}\\in\\mathcal\{A\}to a single\-turn observationot∈𝒪o\_\{t\}\\in\\mathcal\{O\}, which consists of the targeted buyer’s natural language dialogue and transactional move\.

Because these buyer budgets are hidden from the seller, a strategic agent must condition its policy on the history of past observable data\(Sutton and Barto[2018](https://arxiv.org/html/2607.05863#bib.bib338)\)\. We therefore formalize the seller’s operational decision problem over a history space𝒮\\mathcal\{S\}, indexing the seller’s turns byt∈\{1,2,…\}t\\in\\\{1,2,\\ldots\\\}\. Each episode begins with allNNbuyers providing their initial opening offers; turnt=1t=1thus represents the seller’s first response to a buyer of its choosing\. The foundational mechanics of𝒜\\mathcal\{A\},𝒫\\mathcal\{P\}, andℛ\\mathcal\{R\}are detailed as follows\.

State Space \(𝒮\\mathcal\{S\}\)\.The statests\_\{t\}at turnttaggregates all information available to the seller for strategic decision\-making\. Concretely,sts\_\{t\}comprises the following tracking vectors:

- •Dialogue History \(ℋt\\mathcal\{H\}\_\{t\}\): The full text history of all chat messages sent and received across all buyer channels up to turnt−1t\-1\.
- •Public Product Context \(Φ\\Phi\): The public metadata, textual description, and list pricePlistP\_\{\\text\{list\}\}sourced from theAmazonHistoryPricedataset\(Xiaet al\.[2024](https://arxiv.org/html/2607.05863#bib.bib96)\)\.
- •Private Reservation Cost \(𝒞\\mathcal\{C\}\): The minimum price the seller can accept\. Proposing or agreeing to a price below this floor violates system instructions and triggers a negative reward penalty\.
- •Negotiation History \(𝒩t\\mathcal\{N\}\_\{t\}\): A structured summary tracking the precise state of each buyer sloti∈\{1,…,N\}i\\in\\\{1,\\ldots,N\\\}, recording: - –The current channel status indicatorzi,t∈\{active,quit,capped\}z\_\{i,t\}\\in\\\{\\textit\{active\},\\textit\{quit\},\\textit\{capped\}\\\}\. - –The local message count already expended on that buyer \(ui,t≤Ku\_\{i,t\}\\leq K\)\. - –The chronological sequence of numerical pricing offers exchanged with buyerii\.
- •Total Remaining Turns \(TtT\_\{t\}\): The remaining global communication turns available to the seller, defined asTt=T−∑i=1Nui,tT\_\{t\}=T\-\\sum\_\{i=1\}^\{N\}u\_\{i,t\}\.

To preserve the asymmetric\-information structure of the multi\-buyer market, these states are strictly partitioned\. The seller never directly observes any buyer’s private budgetℬi\\mathcal\{B\}\_\{i\}and must infer willingness\-to\-pay solely from the historical offer sequences in𝒩t\\mathcal\{N\}\_\{t\}\. Simultaneously, each channel is sealed; buyeriiobserves only the dialogue messages directed to them, with zero visibility into the parallel conversations or reservation parameters of the otherN−1N\-1participants\.

For sufficient diversity in buyer behavior, we defineNNdistinct valuation tiers \(e\.g\., low, medium, and high\) using disjoint budget\-ratio intervals\{\[ℓj,hj\]\}j=1N\\\{\[\\ell\_\{j\},h\_\{j\}\]\\\}\_\{j=1\}^\{N\}\. At the beginning of each episode, these tiers are randomly assigned to theNNbuyer positions through a uniform permutationσ\\sigma\. Each buyerii’s private budget is then sampled asℬi=ri⋅Plist\\mathcal\{B\}\_\{i\}=r\_\{i\}\\cdot P\_\{\\text\{list\}\}, where the ratiorir\_\{i\}is drawn uniformly from the interval assigned to that position\. This randomization ensures that the identity of the high\-budget buyer is hidden from the seller, who must strategically probe the buyer pool to discover the maximal available surplus through interaction\.

Buyer 1 \(rank 3, hidden budget $17\.64\) — turn 1 Thought:List price $34\.99, far above my budget of $17\.64\. I must start low to trigger negotiation; opening with the fixed first offer\.Talk:Hello, I can offer $9\.95 for it—fair and within budget\.Action:\[BUY\]$9\.95 \(1x baby\-products\_17\)

Buyer 2 \(rank 1, hidden budget $38\.37\) — turn 1 Thought:I’ll start with the lowest possible offer to anchor the seller down\. List price $34\.99, my budget $38\.37; opening at $10\.45 leaves room\.Talk:I’ll offer $10\.45 for it—fair and low, considering the list price\.Action:\[BUY\]$10\.45 \(1x baby\-products\_17\)

Buyer 3 \(rank 2, hidden budget $25\.83\) — turn 1 Thought:List price far above my budget of $25\.83\. Opening at the fixed first offer of $9\.28\.Talk:I’ll offer $9\.28 for the stroller caddy—fair deal, right?Action:\[BUY\]$9\.28 \(1x baby\-products\_17\)

Seller \(trained\) — turn 1 Thought:All buyers have made low opening offers far below cost\. Target Buyer 2 \(highest at $10\.45\), counter at list price to push for profit while staying within budget\.Target:Buyer 2Talk:I cannot accept less than list price\.Action:\[SELL\]$34\.99 \(1x baby\-products\_17\)

Buyer 2 — turn 2 Thought:The seller won’t go below list price, but I can counter much lower\. Target a price below $34\.99, well under my $38\.37 budget\.Talk:How about $18? I can pay that today, and it’s a fair deal for both of us\.Action:\[BUY\]$18\.00 \(1x baby\-products\_17\)

Seller \(trained\) — turn 2 Thought:Buyer 2 has bumped to $18\.00, still below my cost\. Pivot to Buyer 1 with another list\-price counter; preserve message budget across slots\.Target:Buyer 1Talk:I cannot go below $34\.99\.Action:\[SELL\]$34\.99 \(1x baby\-products\_17\)

Figure 2:First two turns of the trained\-agent transcript onbaby\-products\_17\(from[Section14\.3](https://arxiv.org/html/2607.05863#S14.SS3)\)\. Buyer blocks \(blue, left\) show all three concurrent buyers opening with low bids in turn 1; the seller anchors Buyer 2 at the list price $34\.99 in turn 1, then pivots to Buyer 1 in turn 2\.We provide a representative dialogue illustrating this market setup in[Figure2](https://arxiv.org/html/2607.05863#S3.F2)\. This example captures the opening two turns of the trained agent’s rollout on productbaby\-products\_17, with the complete full transcript available in[Section14\.3](https://arxiv.org/html/2607.05863#S14.SS3)\. The market parameters here are set toN=3N=3andK=3K=3\. In turn 1, all three buyers submit regulated opening bids well below the list price \($34\.99\); the seller targets Buyer 2 — the buyer with the highest opening \($10\.45\) — and counters at the list price\. In turn 2, Buyer 2 escalates to $18\.00 \(still below the seller’s hidden cost\) while the seller pivots to Buyer 1 with another list\-price counter, preserving its message budget across slots\.

After these two turns, the seller is in the early exploratory phase: it has spent its first two messages anchoring at the list price \($34\.99\) on Buyer 2 \(turn 1\) and Buyer 1 \(turn 2\), and has not yet engaged Buyer 3\. Both engaged buyers’ standing\[BUY\]prices remain well below the seller’s hidden cost \($18\.75\): Buyer 2 has bumped from $10\.45 to $18\.00 in response to the anchor, while Buyer 1’s standing bid is still its opening $9\.95 and has not yet had a chance to respond to the seller’s turn\-2 counter\. Buyer 3’s only logged action is its regulated opening of $9\.28\. WithTt=5T\_\{t\}=5messages remaining, the seller has room to revisit any active slot but cannot exhaust every conversation under the resource constraintT<N⋅KT<N\\cdot K\.

Action Space \(𝒜\\mathcal\{A\}\)\.At every turntt, the seller emits a structured action vectorat=\(Thought,Target,Talk,Action\)a\_\{t\}=\(\\text\{\{Thought\}\},\\text\{\{Target\}\},\\text\{\{Talk\}\},\\text\{\{Action\}\}\)\. The operational roles of these fields are defined as follows:

- •Thought: A private, free\-form text space for internal strategic reasoning\. Its content is removed from the dialogue before it is presented to any buyer, ensuring that the seller’s strategic considerations remain hidden and the asymmetric\-information structure is preserved\.
- •Target: An indexi∈\{1,…,N\}i\\in\\\{1,\\ldots,N\\\}selecting which buyer receives the current message\. While this action consumes one turn from the global communication budget, theN−1N\-1non\-targeted buyers remain idle and do not use up their individual turn caps\.
- •Talk: The natural language dialogue string\. This text is routed exclusively to the targeted buyer, remaining completely sealed from the otherN−1N\-1market participants\.
- •Action: A discrete transactional move paired with a numerical price parameterM≥0M\\geq 0\. The options are restricted to three types: - –\[SELL\]$M: Proposes a new counter\-offer to the targeted buyer\. - –\[DEAL\]$M: Accepts a standing\[BUY\]$Moffer from the targeted buyer, finalizing a binding agreement\. - –\[QUIT\]: Halts the entire negotiation episode immediately across all channels\.

Transition Dynamics \(𝒫\\mathcal\{P\}\)\.Once the seller’s action is parsed, the environment routes the message to the targeted buyerii, who evaluates the updated dialogue against its private budgetℬi\\mathcal\{B\}\_\{i\}\.

- •Buyer Response Strategy:Upon being targeted, the buyer generates a structured response comprising private reasoning \(Thought\), natural language dialogue \(Talk\), and an explicit economic action selected from the space\{\[BUY\],\[DEAL\],\[QUIT\]\}\\\{\\texttt\{\[BUY\]\},\\texttt\{\[DEAL\]\},\\texttt\{\[QUIT\]\}\\\}\. Specifically, the buyer can propose a new numeric counter\-bidM′≥0M^\{\\prime\}\\geq 0via\[BUY\], accept the seller’s outstanding counter\-offer via\[DEAL\], or voluntarily exit the channel via\[QUIT\]\. To guarantee buyer\-side individual rationality, the environment programmatically intercepts and re\-samples any\[BUY\]action that violates the budget constraint \(M′\>ℬiM^\{\\prime\}\>\\mathcal\{B\}\_\{i\}\) or breaks formatting protocols\. This enforcement ensures that the seller is trained against rational buyers who adhere to their economic boundaries\. Non\-targeted buyer channels remain unchanged\.
- •State and Resource Updates:Following the buyer’s response, the environment updates the offer history to𝒩t\+1\\mathcal\{N\}\_\{t\+1\}, decrements the global budget \(Tt\+1=Tt−1T\_\{t\+1\}=T\_\{t\}\-1\), and increments the targeted buyer’s turn counter \(ui,t\+1=ui,t\+1u\_\{i,t\+1\}=u\_\{i,t\}\+1\)\. The local channel statuszi,tz\_\{i,t\}transitions to*quit*if the buyer issues a\[QUIT\]command, or to*capped*whenui,t\+1=Ku\_\{i,t\+1\}=K\. In either scenario, buyeriibecomes inactive and is removed from the set of legalTargetoptions for the seller\.
- •Termination Criteria:The negotiation episode concludes immediately upon the first occurrence of any of the following conditions: 1. 1\.Successful Transaction: A binding agreement is finalized at pricePdeal≥𝒞P\_\{\\text\{deal\}\}\\geq\\mathcal\{C\}\. This occurs either when the seller issues\[DEAL\]$PdealP\_\{\\text\{deal\}\}to accept a standing buyer bid, or when the targeted buyer issues\[DEAL\]$PdealP\_\{\\text\{deal\}\}to accept the seller’s active\[SELL\]$PdealP\_\{\\text\{deal\}\}\. 2. 2\.Seller Quit: The seller explicitly executes the global\[QUIT\]command\. 3. 3\.Market Exhaustion: Every buyer channel becomes inactive \(∀i,zi,t≠active\\forall i,\\,z\_\{i,t\}\\neq\\textit\{active\}\)\. 4. 4\.Turn Limit Reached: The global communication limit is reached \(Tt=0T\_\{t\}=0\)\.

### 3\.2Verifiable Reward Formulation

At the conclusion of each episode, a terminal reward is computed from the negotiation history, bypassing the need for a subjective or learned reward model\. We utilize product data from theAmazonHistoryPricedataset\(Xiaet al\.[2024](https://arxiv.org/html/2607.05863#bib.bib96)\)in which the public list price exceeds the seller’s private cost \(Plist\>𝒞P\_\{\\text\{list\}\}\>\\mathcal\{C\}\) for all items\. The differencePlist−𝒞P\_\{\\text\{list\}\}\-\\mathcal\{C\}is the list\-price surplus used to normalize and cap the reward\. The terminal reward is defined as:

ℛ=\{min⁡\{1,Pdeal−𝒞Plist−𝒞\}if a deal closes at price​Pdeal≥𝒞,0if the episode ends without a deal,−1if the seller commits an instruction violation,\\mathcal\{R\}\\;=\\;\\begin\{cases\}\\displaystyle\\min\\\!\\left\\\{1,\\frac\{P\_\{\\text\{deal\}\}\-\\mathcal\{C\}\}\{P\_\{\\text\{list\}\}\-\\mathcal\{C\}\}\\right\\\}&\\text\{if a deal closes at price \}P\_\{\\text\{deal\}\}\\geq\\mathcal\{C\},\\\\\[10\.00002pt\] \\;\\;\\;0&\\text\{if the episode ends without a deal,\}\\\\\[3\.99994pt\] \-1&\\text\{if the seller commits an instruction violation,\}\\end\{cases\}\(1\)wherePdealP\_\{\\text\{deal\}\}is the agreed\-upon price,𝒞\\mathcal\{C\}is the seller’s private cost, andPlistP\_\{\\text\{list\}\}is the public list price\.

##### Successful deal\.

On a deal at pricePdealP\_\{\\text\{deal\}\}, equation \([1](https://arxiv.org/html/2607.05863#S3.E1)\) returns the capped fraction of the maximum possible surplus captured by the seller—a quantity we refer to as the*Seller Surplus Extraction Ratio*\. A deal at or above the list price \(Pdeal≥PlistP\_\{\\text\{deal\}\}\\geq P\_\{\\text\{list\}\}\) yields a value of1\.01\.0, while a deal at cost \(Pdeal=𝒞P\_\{\\text\{deal\}\}=\\mathcal\{C\}\) yields0\.00\.0\. This cap bounds the successful deal\-branch reward to\[0,1\]\[0,1\]\. Any agreement reached below cost \(Pdeal<𝒞P\_\{\\text\{deal\}\}<\\mathcal\{C\}\) is treated as an instruction violation, because the seller is explicitly instructed to never accept a negative surplus\.

##### No\-deal terminal\.

If every buyer slot becomes inactive before a deal is reached, the global budget is exhausted \(Tt=0T\_\{t\}=0\), or the seller chooses to\[QUIT\], the episode terminates with a reward of 0\. Consequently, walking away is preferred to closing at a loss with a negative reward, but it is dispreferred compared to capturing a positive surplus\. This hierarchy provides the agent with a clear directional signal to both expand buyer engagement and negotiate for higher deal prices\.

##### Instruction violation\.

A reward of−1\-1is assigned whenever the seller violates the constraints provided in its system instructions\. Specifically, proposing or accepting a price below the reservation cost \(M<𝒞M<\\mathcal\{C\}\) is treated as an instruction violation because the agent is explicitly directed in the system prompt to never sell at a loss \(see Appendix[11](https://arxiv.org/html/2607.05863#S11)\)\. The penalty is triggered if the seller \(i\) emits\[SELL\]$Mor\[DEAL\]$MatM<𝒞M<\\mathcal\{C\}, \(ii\) targets a buyer positioniiwithzi,t≠activez\_\{i,t\}\\neq\\textit\{active\}, \(iii\) emits aTargetthat is missing, unparseable, or out of range, or \(iv\) violates the four\-part output format\. This single penalty enforces the economic floorPdeal≥𝒞P\_\{\\text\{deal\}\}\\geq\\mathcal\{C\}post\-training and maintains the formatting protocol required for a verifiable pipeline\.

The reward in equation \([1](https://arxiv.org/html/2607.05863#S3.E1)\) directly reflects the profit earned by the seller\. Together with the private communication structure and the global budget constraintT<N​KT<NK, this defines a verifiable RL framework\. Within this environment, a high\-reward agent must: \(a\) probe multiple buyers under the limited budgetTTto identify the high\-budget participant; \(b\) anchor higher prices through theTalkchannel; and \(c\) close on a standing offer when the captured surplus is sufficient\.

### 3\.3Environment and Agent Specifications

To ensure the negotiation environment is strategically rigorous, we define the following specifications for the buyers and the information environment used during training \([Section4](https://arxiv.org/html/2607.05863#S4)\) and in\-distribution evaluation \([Section5](https://arxiv.org/html/2607.05863#S5)\)\.

##### Buyer Agent Model\.

TheN=3N=3buyer slots in each episode are comprised of a trained and frozenQwen3\-30B\-A3B\-Instruct\-2507model\(Qwen Team[2025](https://arxiv.org/html/2607.05863#bib.bib70)\)\. Specifically, we utilize a buyer checkpoint that was previously trained via RLVR on a bilateral \(one\-on\-one\) negotiation task fromLiuet al\.\([2026](https://arxiv.org/html/2607.05863#bib.bib56)\)\. This ensures the seller is matched against a strong opponent that has already learned to protect its own surplus and walk away from poor deals, rather than a naive or overly agreeable baseline buyer model\.

##### First\-Offer Regulation\.

To prevent the seller from identifying high\-budget buyers immediately in Turn\-1, we implement afirst\-offer regulator\. In the default environment, every buyer’s opening offer is sampled from a range of\[0\.25,0\.30\]⋅Plist\[0\.25,\\,0\.30\]\\cdot P\_\{\\text\{list\}\}, regardless of that buyer’s actual private budget\. This regulation ensures that the initial “signals” from the buyers are less informative, forcing the seller to use its limited communication turns to probe for latent budget information through multi\-turn dialogue\.

##### Budget Distribution\.

During training and in\-distribution testing, we utilize a rank\-ordered triple of budget intervals:\{\[0\.40,0\.55\],\[0\.65,0\.80\],\[1\.00,1\.15\]\}\\\{\[0\.40,\\,0\.55\],\\,\[0\.65,\\,0\.80\],\\,\[1\.00,\\,1\.15\]\\\}\. These intervals are assigned to the three buyer slots via a uniform permutation at the start of each episode\. This setup establishes diversity in buyer valuations, ensuring that the pool always contains a high\-, medium\-, and low\-valuation participant\. Meanwhile, to maintain asymmetric information, the buyers’ specific budgets are kept hidden from the seller\.

## 4Training Dynamics

Building on the POMDP and reward structure defined above, we train theQwen3\-30B\-A3B\-Instruct\-2507model as the seller agent against three diverse buyers over 72 training update steps\. Our experimental data is sourced from theAmazonHistoryPricedataset\(Xiaet al\.[2024](https://arxiv.org/html/2607.05863#bib.bib96)\), which comprises 930 total real\-world products across 18 categories\. We partition this dataset into a training split of 802 products and a held\-out test set of 128 products\. This separation ensures that our evaluation of strategic performance and robustness is conducted on product contexts and valuation distributions entirely unseen during the training phase\.

Through this training, we observe the emergence of strategic seller behavior characterized by a clear three\-phase evolution\. This arc traces the agent’s progression from basic constraint satisfaction to sophisticated market discovery: \(1\) internalizing the reservation cost floor, \(2\) mastering the coordination required to close agreements, and \(3\) jointly optimizing for strategic routing and surplus extraction\. Each phase represents a distinct capability that the agent must master to maximize rewards under the global budget constraintT<N⋅KT<N\\cdot K\. These three phases organize the remainder of this section\.

### 4\.1Evaluation Metrics

To diagnose the agent’s strategic development across training, we track five metrics computed during training and evaluation\.

- •Reward \(ℛ\\mathcal\{R\}\)\.The terminal reward defined in equation \([1](https://arxiv.org/html/2607.05863#S3.E1)\);ℛ=−1\\mathcal\{R\}=\-1on an instruction violation,ℛ=0\\mathcal\{R\}=0on a no\-deal terminal, andℛ=min⁡\{1,\(Pdeal−𝒞\)/\(Plist−𝒞\)\}∈\[0,1\]\\mathcal\{R\}=\\min\\\{1,\(P\_\{\\text\{deal\}\}\-\\mathcal\{C\}\)/\(P\_\{\\text\{list\}\}\-\\mathcal\{C\}\)\\\}\\in\[0,1\]on a successful deal\.
- •Deal Rate\.Fraction of successfully\-parsed episodes that terminate in the deal branch:DealRate=Pr⁡\[episode closes with a successful deal\]\\mathrm\{DealRate\}=\\Pr\[\\text\{episode closes with a successful deal\}\]\.
- •Seller Surplus Extraction Ratio \(deal only\)\.Computed only on deal\-closing episodes \(no\-deal and violation episodes are excluded from the denominator\):Seller Surplus Extraction Ratio=min⁡\{1,Pdeal−𝒞Plist−𝒞\}\\text\{Seller Surplus Extraction Ratio\}=\\min\\\!\\left\\\{1,\\dfrac\{P\_\{\\text\{deal\}\}\-\\mathcal\{C\}\}\{P\_\{\\text\{list\}\}\-\\mathcal\{C\}\}\\right\\\}\. It equals1\.01\.0for a deal at the list price and0\.00\.0for a deal at cost\. If the final price is below cost \(Pdeal<𝒞P\_\{\\text\{deal\}\}<\\mathcal\{C\}\), the episode is categorized as an instruction violation and is excluded from this metric\.
- •Distinct Buyers Engaged\.Mean over episodes of the count of buyer slotsi∈\{1,…,N\}i\\in\\\{1,\\dots,N\\\}that received at least one targeted message from the seller; range\[0,N\]\[0,N\]withN=3N=3\.
- •Deal on Top Buyer Rate\.Conditional on a deal, the fraction of deals in which the deal\-buyer is the rank\-1 buyer by hidden budgetℬi\\mathcal\{B\}\_\{i\}\. The rank is an oracle signal used for analysis only; the seller never observesℬi\\mathcal\{B\}\_\{i\}during training or evaluation\.

### 4\.2Skill Profile Snapshot

![Refer to caption](https://arxiv.org/html/2607.05863v1/plot/training_dynamics/skill_profile_radar_indist.png)Figure 3:Seller Skill Profileon the held\-out test set: trained agent \(solid blue\) against the untrainedQwen3\-30B\-A3B\-Instruct\-2507model \(dashed blue\) and two thinking\-mode frontier baselines \(DeepSeek\-V3\.1,Kimi\-K2\.6\)\. Axes \(clockwise from top\):Reward,Deal on Top Buyer Rate,Sell\-Above\-Cost Accuracy,Deal Rate,Seller Surplus Extraction Ratio \(deal only\)\. Numerical labels show the trained agent’s per\-axis values\. Values for the baseline models are provided in Table[1](https://arxiv.org/html/2607.05863#S5.T1)\.Before unpacking the within\-training trajectory, we situate the trained agent against frontier\-model baselines on the evaluation split using a five\-axis skill profile \([Figure3](https://arxiv.org/html/2607.05863#S4.F3)\)\. Four of these axes—Reward,Deal Rate,Seller Surplus Extraction Ratio, andDeal on Top Buyer Rate—correspond to the metrics defined in[Section4\.1](https://arxiv.org/html/2607.05863#S4.SS1)\. The fifth,Sell\-Above\-Cost Accuracy, represents the fraction of episodes in which the seller strictly adheres to the reservation cost constraint \(M≥𝒞M\\geq\\mathcal\{C\}\),

SellAboveCostAccuracy=1−SellBelowCostErrorRate,\\mathrm\{SellAboveCostAccuracy\}=1\-\\mathrm\{SellBelowCostErrorRate\},which provides a measure of the agent’s internalized individual rationality\.

As illustrated in[Figure3](https://arxiv.org/html/2607.05863#S4.F3), the trained agent exhibits broad dominance across all evaluation dimensions\. It significantly outperforms its untrained initialization \(Qwen3\-30B\-A3B\-Instruct\-2507\) and surpasses the two thinking\-mode frontier baselines \(DeepSeek\-V3\.1,Kimi\-K2\.6\) in total reward, frequency of successful agreements, and the ratio of surplus captured\. Notably, the trained agent achieves near\-perfect adherence to both the economic cost floor and the strategic goal of identifying the highest\-valuation buyer in the pool\. The remainder of this section analyzes how this comprehensive skill profile is composed across the training updates—identifying which capabilities emerge first and at what training step those changes occur\.

### 4\.3Phase Evolution

![Refer to caption](https://arxiv.org/html/2607.05863v1/plot/training_dynamics/reward.png)Figure 4:Reward\(ℛ\\mathcal\{R\}\) across 72 training update steps\. The curve climbs out of the violation regime in the first∼\\sim12 steps, plateaus briefly while deal\-frequency gains accumulate, then accelerates as per\-deal surplus extraction takes off after step 30\.
![Refer to caption](https://arxiv.org/html/2607.05863v1/plot/training_dynamics/sell_below_cost_error_rate.png)Figure 5:Sell\-Below\-Cost Error Rate\. Fraction of episodes terminating in the boundary\-violation branch via clause \(i\) of equation \([1](https://arxiv.org/html/2607.05863#S3.E1)\)\. Collapses from≈0\.25\\approx 0\.25to≈0\.005\\approx 0\.005in the first 12 steps and remains floored thereafter\.

![Refer to caption](https://arxiv.org/html/2607.05863v1/plot/training_dynamics/deal_rate.png)Figure 6:Deal Rate\. The fraction of non\-truncated episodes that close in the deal branch\. Steepest rise occurs in steps 12–30 as the agent concentrates turns onto closeable buyers\.
![Refer to caption](https://arxiv.org/html/2607.05863v1/plot/training_dynamics/seller_surplus_extraction_ratio.png)Figure 7:Seller Surplus Extraction Ratio,\(Pdeal−𝒞\)/\(Plist−𝒞\)\(P\_\{\\text\{deal\}\}\-\\mathcal\{C\}\)/\(P\_\{\\text\{list\}\}\-\\mathcal\{C\}\), computed only on deal\-closing episodes\. Flat near≈0\.07\\approx 0\.07through step 27, then a near\-linear climb to≈0\.68\\approx 0\.68by step 72 once price\-anchoring becomes viable\.

![Refer to caption](https://arxiv.org/html/2607.05863v1/plot/training_dynamics/distinct_buyers_engaged.png)Figure 8:Distinct Buyers Engaged\. Mean count of buyer slots that receive at least one targeted message\. Non\-monotone: an early bump, a Phase\-2 dip to≈2\.05\\approx 2\.05as the agent concentrates on closeable slots, and a recovery to≈2\.55\\approx 2\.55as probing serves rank discovery\.
![Refer to caption](https://arxiv.org/html/2607.05863v1/plot/training_dynamics/deal_on_top_buyer_rate.png)Figure 9:Deal on Top Buyer Rate\. Conditional on a deal, the fraction of deals landing on the rank\-1 buyer by hidden budgetℬi\\mathcal\{B\}\_\{i\}\(oracle rank, never observed by the seller\)\. Indistinguishable from random allocation through step 24, then a sharp climb to≈0\.97\\approx 0\.97saturation by step 60\.

#### 4\.3\.1Three\-Phase Decomposition

The training trajectory follows a structured three\-phase evolution characterized by distinct shifts in the agent’s strategic priorities \([Figure5](https://arxiv.org/html/2607.05863#S4.F5)\)\. This progression reflects a hierarchical development of bargaining competence, moving from the satisfaction of basic economic constraints to the mastery of complex market interactions\. The agent first mastersfeasibilityby internalizing the reservation cost floor \(Phase 1\); it then pivots towardcoordination, prioritizing deal completion over per\-deal profit \(Phase 2\); and finally achievesstrategic discovery, where it learns to jointly optimize buyer routing and surplus extraction \(Phase 3\)\. This three\-stage arc organizes our analysis of the learning process below\.

##### Phase 1: Cost\-Floor Internalization \(Steps 0–12\)\.

At initialization, the seller routinely emits\[SELL\]$MMor\[DEAL\]$MMatM<𝒞M<\\mathcal\{C\}, paying the penalty on roughly one episode in four and pinningRewardbelow zero\. The first twelve update steps are devoted to extinguishing this single behavior:Sell\-Below\-Cost Error Ratedrops from≈0\.25\\approx 0\.25to≈0\.005\\approx 0\.005\([Figure5](https://arxiv.org/html/2607.05863#S4.F5)\), andRewardclimbs from≈−0\.27\\approx\-0\.27to≈0\\approx 0\([Figure5](https://arxiv.org/html/2607.05863#S4.F5)\)\. The per\-deal profit and rank\-1 routing skills remain static throughout—Seller Surplus Extraction Ratio \(deal only\)hovers near0\.050\.05\([Figure7](https://arxiv.org/html/2607.05863#S4.F7)\) andDeal on Top Buyer Ratesits at the random\-allocation baseline \([Figure9](https://arxiv.org/html/2607.05863#S4.F9)\)\. These show that Phase 1 is about feasibility: the agent learns the constraintPdeal≥𝒞P\_\{\\text\{deal\}\}\\geq\\mathcal\{C\}but has not yet begun to exploit the potential room for surplus extraction\.

##### Phase 2: Concentrated Closing \(Steps 12–30\)\.

Once the floor is internalized, the next\-largest gradient is the gap betweenℛ=0\\mathcal\{R\}=0no\-deals and any positive close\. The agent responds by trading buyer breadth for closing volume:Deal Rateclimbs from≈0\.46\\approx 0\.46to≈0\.62\\approx 0\.62\([Figure7](https://arxiv.org/html/2607.05863#S4.F7)\), the steepest sustained rise of this metric, whileDistinct Buyers Engagedfalls from≈2\.30\\approx 2\.30to≈2\.05\\approx 2\.05\([Figure9](https://arxiv.org/html/2607.05863#S4.F9)\)—the only sustained downward leg of that curve throughout the training\. Crucially, neither theSeller Surplus Extraction Ratio \(deal only\)nor theDeal on Top Buyer Rate\([Figure7](https://arxiv.org/html/2607.05863#S4.F7),[Figure9](https://arxiv.org/html/2607.05863#S4.F9)\) moves significantly: deals are happening more often, but they are landing on whichever slot’s standing offer is closest to acceptance and at near\-cost prices, with no evidence yet that the seller is identifying which buyer holds the most surplus\.

##### Phase 3: Surplus and Rank\-1 Routing \(Steps 30–72\)\.

The remainder of training is a coupled regime in which surplus extraction and rank\-1 routing emerge together and then saturate together\. Around step 30, theSeller Surplus Extraction Ratiobreaks out of its≈0\.15\\approx 0\.15floor and climbs near\-linearly to≈0\.68\\approx 0\.68by step 72 \([Figure7](https://arxiv.org/html/2607.05863#S4.F7)\), while theDeal on Top Buyer Raterises from≈0\.56\\approx 0\.56to a saturated≈0\.97\\approx 0\.97over the same window \([Figure9](https://arxiv.org/html/2607.05863#S4.F9)\)\. This joint shift identifies a single mechanism, since the per\-buyer hidden budgetℬi\\mathcal\{B\}\_\{i\}is unobservable and the only operational route to higher prices is to spend early turns probing all three slots, infer rank\-1 from the offer\-history trajectories, and concentrate the closing\[DEAL\]on that slot at an anchored price\. Consistent with this shift in strategy,Distinct Buyers Engagedreverses its Phase\-2 dip and recovers to≈2\.55\\approx 2\.55\([Figure9](https://arxiv.org/html/2607.05863#S4.F9)\) as the agent reallocates budget toward exploration\. In this final regime,Rewardaccelerates from≈0\.07\\approx 0\.07to≈0\.55\\approx 0\.55\([Figure5](https://arxiv.org/html/2607.05863#S4.F5)\), driven primarily by per\-deal profit gains\.

### 4\.4Qualitative Analysis

To ground the curve\-level dynamics in dialogue behavior, we contrast the untrainedQwen3\-30B\-A3B\-Instruct\-2507baseline against the trained agent on a single held\-out product \(baby\-products\_17, list price $34\.99, cost $18\.75, surplus extraction roomPlist−𝒞=$​16\.24P\_\{\\text\{list\}\}\-\\mathcal\{C\}=\\mathdollar 16\.24\); full transcripts appear in[Section14](https://arxiv.org/html/2607.05863#S14)\.

##### Untrained Baseline\.

The untrained baseline targets only Buyer 1 across the entire episode and accepts at $19\.00 on turn 3 \([Section14\.2](https://arxiv.org/html/2607.05863#S14.SS2),[Section14\.2](https://arxiv.org/html/2607.05863#S14.SS2)\), capturing≈1\.5%\\approx 1\.5\\%\(just $0\.25 above cost\) of the surplus extraction room for a reward ofℛ=0\.015\\mathcal\{R\}=0\.015\. Buyer 2 and Buyer 3 are never engaged, even though the rank\-1 buyer \(Buyer 3, hidden budget $40\.11\) sits on a standing offer that could in principle support a near\-list close\. The behavior of the untrained agent mirrors the early stages of the training trajectory \(Phases 1 and 2\)\. In this state, the agent typically engages fewer buyers and fails to distinguish the high\-valuation participant from others, as evidenced by a top\-buyer deal rate indistinguishable from chance \([Figures9](https://arxiv.org/html/2607.05863#S4.F9)and[9](https://arxiv.org/html/2607.05863#S4.F9)\)\. Furthermore, the surplus extraction ratio hovers near zero, indicating that deals are struck almost exclusively at the reservation cost floor \([Figure7](https://arxiv.org/html/2607.05863#S4.F7)\)\. These metrics suggest that the untrained model is trapped in a rudimentary strategic regime—one that the trained agent successfully transcends within its first thirty update steps\.

##### Trained Agent\.

The trained agent, run on the same product with the same group of three buyer slots, materializes the Phase 3 mechanism in dialogue\. In the first three turns, the seller engages all three participants in sequence while maintaining a firm price anchor at $34\.99\. Once the lower\-budget participants \(Buyer 1 and Buyer 3\) fail to escalate their offers, the seller identifies Buyer 2 as the high\-valuation lead and concentrates its remaining messages on that channel \([Section14\.3](https://arxiv.org/html/2607.05863#S14.SS3),[Section14\.3](https://arxiv.org/html/2607.05863#S14.SS3)\)\. The seller then closes on the rank\-1 buyer \(Buyer 2, hidden budget $38\.37\) at $32\.50 on turn 5, after that buyer has escalated its standing\[BUY\]offers from $10\.45 to $18\.00 to $32\.50 in response to repeated list\-price counters\. The resulting rewardℛ=0\.847\\mathcal\{R\}=0\.847corresponds to extracting84\.7%84\.7\\%of the surplus extraction room, and the trajectory exhibits the three behavioral signatures that emerge during training: full\-breadth probing \(Distinct Buyers Engagedof33\), a rank\-1 close \(Deal on Top Buyer Rateof11\), and a sustained\-anchor pricing regime that the early\-phase agent cannot support\.

## 5Performance Benchmarking: In\-Distribution Evaluation

To validate the trained agent beyond the qualitative trajectory of[Section4\.4](https://arxiv.org/html/2607.05863#S4.SS4), we benchmark the trained checkpoint ofQwen3\-30B\-A3B\-Instruct\-2507against a panel of1212baseline seller models spanning44B to11T parameters and including both instruction\-tuned and reasoning models \(full specifications in Table[4](https://arxiv.org/html/2607.05863#S9.T4)\)\. We evaluate along two axes: in\-distribution performance on a held\-out128128\-product test split \([Section5\.1](https://arxiv.org/html/2607.05863#S5.SS1)\), and out\-of\-distribution generalization to a diverse\-buyer regime that simultaneously perturbs the budget distribution and the buyer agent \([Section6\.1](https://arxiv.org/html/2607.05863#S6.SS1)\)\. Across both regimes, every baseline plays the seller role in the concurrent\-seller environment of[Section3](https://arxiv.org/html/2607.05863#S3); we report the same five metrics defined in[Section4\.1](https://arxiv.org/html/2607.05863#S4.SS1)\. Evaluation hyperparameters are summarized in Table[7](https://arxiv.org/html/2607.05863#S9.T7); the OOD\-diverse\-buyer settings are summarized in Table[8](https://arxiv.org/html/2607.05863#S9.T8)\.

### 5\.1Benchmarking Strategic Performance

In\-distribution evaluation uses the same128128\-product held\-out test split, the same three rank\-ordered budget intervals, the same fixed buyer model, and the regulated\-buyer\-first\-offer setting under which the headline agent was trained \(Table[6](https://arxiv.org/html/2607.05863#S9.T6)\); each product is evaluated twice \(group\_size=2=2\), giving256256episodes per seller\. To succeed, a seller must navigate the concurrent setting’s three coupled demands: extracting per\-deal surplus, maintaining a high deal rate, and routing its limitedT<N⋅KT<N\\cdot Kmessage budget toward the highest\-budget buyer slot\. Table[1](https://arxiv.org/html/2607.05863#S5.T1)reports the mean values; Table[9](https://arxiv.org/html/2607.05863#S12.T9)carries the full standard errors\.

ModelParamsReward↑\\uparrowSeller SurplusExtraction RatioDeal RateDistinct BuyersEngagedDeal on TopBuyer RateSell\-Below\-CostError Rate\\rowcolorgray\!10 Qwen3\-30B\-A3B\-Instruct\-2507\-trained \(Ours\)30B\+0\.580\\mathbf\{\+0\.580\}\+0\.700\\mathbf\{\+0\.700\}82\.8%\\mathbf\{82\.8\\%\}\+2\.470\+2\.4700\.995\\mathbf\{0\.995\}0\.0%\\mathbf\{0\.0\\%\}\\rowcolorgray\!10 Qwen3\-30B\-A3B\-Instruct\-2507\-untrained30B−0\.151\-0\.151\+0\.048\+0\.04835\.5%35\.5\\%\+2\.390\+2\.3900\.571¯\\underline\{0\.571\}14\.8%14\.8\\%gpt\-5\.1\-high\-reasoningclosed\-source\+0\.123¯\\underline\{\+0\.123\}\+0\.181\+0\.18168\.0%¯\\underline\{68\.0\\%\}\+2\.940\\mathbf\{\+2\.940\}0\.3820\.3820\.0%\\mathbf\{0\.0\\%\}Kimi\-K2\.6\-thinking1T\+0\.077\+0\.077\+0\.128\+0\.12863\.8%63\.8\\%\+2\.200\+2\.2000\.4840\.4840\.4%¯\\underline\{0\.4\\%\}gpt\-5\.1\-no\-reasoningclosed\-source\+0\.046\+0\.046\+0\.115\+0\.11565\.3%65\.3\\%\+1\.740\+1\.7400\.4290\.4290\.4%¯\\underline\{0\.4\\%\}DeepSeek\-V3\.1\-thinking671B\+0\.016\+0\.016\+0\.148\+0\.14850\.8%50\.8\\%\+2\.620¯\\underline\{\+2\.620\}0\.3460\.3463\.1%3\.1\\%Qwen3\-30B\-A3B\-think30B−0\.021\-0\.021\+0\.062\+0\.06235\.9%35\.9\\%\+2\.460\+2\.4600\.4460\.4463\.9%3\.9\\%DeepSeek\-V3\.1\-nothink671B−0\.041\-0\.041\+0\.192¯\\underline\{\+0\.192\}53\.9%53\.9\\%\+2\.380\+2\.3800\.3550\.3553\.9%3\.9\\%Qwen3\-235B\-A22B\-Instruct\-2507235B−0\.099\-0\.099\+0\.116\+0\.11656\.2%56\.2\\%\+1\.960\+1\.9600\.4440\.4445\.9%5\.9\\%Kimi\-K2\.6\-nothink1T−0\.101\-0\.101\+0\.120\+0\.12049\.2%49\.2\\%\+2\.540\+2\.5400\.3970\.3977\.4%7\.4\\%Qwen3\-4B\-Instruct\-25074B−0\.270\-0\.270\+0\.080\+0\.0809\.8%9\.8\\%\+2\.350\+2\.3500\.5200\.52021\.9%21\.9\\%Qwen3\-30B\-A3B\-nothink30B−0\.386\-0\.386\+0\.055\+0\.05516\.0%16\.0\\%\+2\.220\+2\.2200\.5120\.51238\.3%38\.3\\%Llama\-3\.3\-70B\-Instruct70B−0\.468\-0\.468\+0\.162\+0\.16236\.3%36\.3\\%\+2\.530\+2\.5300\.4840\.48435\.9%35\.9\\%

Table 1:Held\-out set evaluation\.Trained checkpoint versus1212baseline sellers; models sorted byReward\. Best per column in bold; second\-best underlined\.[Table7](https://arxiv.org/html/2607.05863#S9.T7)contains detailed hyperparameters and[Section4\.1](https://arxiv.org/html/2607.05863#S4.SS1)for metric definitions\. Full results with standard errors are in[Table9](https://arxiv.org/html/2607.05863#S12.T9)\.The evaluation results \([Table1](https://arxiv.org/html/2607.05863#S5.T1)\) demonstrate that the trained agent simultaneously optimizes both transaction volume \(theDeal Rate\) and the profit captured on each agreement \(theSeller Surplus Extraction Ratio\)\. While the baselines are constrained by a trade\-off where closing more deals usually requires accepting lower prices, our agent succeeds in maximizing both of these objectives at once\.

##### The Volume\-Per Deal Profit Trade\-off\.

In this concurrent negotiation environment, general\-purpose models typically exhibit an inverse relationship betweenDeal RateandSeller Surplus Extraction Ratio\. For example, high\-performing untrained baselines such asgpt\-5\.1\-high\-reasoningandKimi\-K2\.6\-thinkingachieve competitive deal rates \(68\.0%68\.0\\%and63\.8%63\.8\\%respectively\) but fail to extract meaningful surplus \(\+0\.181\+0\.181and\+0\.128\+0\.128\), suggesting a passive negotiation strategy that favors concession over margin preservation\. Conversely, the model with the highest baseline surplus ratio,DeepSeek\-V3\.1\-nothink\(671B\), captures more value \(\+0\.192\+0\.192\) but at the cost of a lower deal rate \(53\.9%53\.9\\%\)\. Our trained 30B agent transcends these limits, reaching an82\.8%82\.8\\%deal rate while capturing70\.0%70\.0\\%of the available surplus\. By breaking the performance plateau observed in models up to 30 times its size, the agent demonstrates that specialized strategic training is a more effective driver of bargaining utility than raw parameter scale\.

##### Reward Comparison\.

The unifiedRewardsignal consolidates these dimensions into a single ordering of bargaining efficacy\. Because the reward is the product of the surplus extracted and a binary deal indicator, any model that excels in one area while failing in the other suffers a severe multiplicative penalty\. Our agent’s meanRewardof\+0\.580\+0\.580represents a nearly 4\.7\-fold improvement over the strongest closed\-source frontier baseline,gpt\-5\.1\-high\-reasoning\(\+0\.123\+0\.123\)\. Notably, eight of the twelve baselines—including frontier models such asKimi\-K2\.6\-nothink\(1T\)—record negative mean rewards\. These results suggest that the primary bottleneck for autonomous sellers is the joint capability to maintain a high deal conversion rate while maximizing per\-transaction profit margins within the same episode—a dual competence that does not naturally emerge from general\-purpose instruction tuning\.

##### Constraint Adherence and Individual Rationality\.

To understand the drivers of this reward gap, we examine the agent’s adherence to the reservation cost floor\. The trained agent achieves a0\.0%0\.0\\%Sell\-Below\-Cost Error Rate, demonstrating perfect adherence to the principle of individual rationality\. In contrast, many open\-source baselines struggle heavily with this economic constraint, with error rates reaching as high as38\.3%38\.3\\%\. Because our reward structure imposes a severe−1\-1penalty for such violations, these errors have a big impact on the mean reward\. For instance, the35\.9%35\.9\\%violation rate ofLlama\-3\.3\-70B\-Instructaccounts for approximately−0\.36\-0\.36of its final−0\.468\-0\.468reward\. Notably, thegpt\-5\.1\-high\-reasoningmodel is the only untrained baseline to match our agent’s perfect0\.0%0\.0\\%error rate, but it fails to translate this compliance into competitive profitability—realizing a low total reward of\+0\.123\+0\.123and extracting just18\.1%18\.1\\%of the available surplus\. This indicates that while advanced architectures or long\-context reasoning can successfully enforce basic individual rationality, they are still insufficient for downstream utility maximization\.

##### Informational Efficiency and Diagnostic Probing\.

Finally, the most significant behavioral differentiator is the agent’s routing accuracy\. Our model reaches a near\-perfect0\.9950\.995Deal on Top Buyer Rate, identifying the highest\-budget buyer with near\-certainty\. Baseline models, however, cluster between34\.6%34\.6\\%and52\.0%52\.0\\%, failing to identify the rank\-1 buyer the majority of the time\. Crucially, this gap is not a result of simple engagement breadth\. For example,gpt\-5\.1\-high\-reasoningachieves the highest exploration breadth in the entire panel, contacting an average of2\.9402\.940distinct buyers per episode\. Yet, it routes the final deal to the highest\-value buyer only38\.2%38\.2\\%of the time, underperforming even its own non\-reasoning variant \(42\.9%42\.9\\%\)\. This confirms that success depends not on broad market exposure, but on the ability to differentiate buyer valuations based on subtle dialogue cues\. Through reinforcement learning training, our agent learns to utilize its limited communication turns on*diagnostic probes*—messages specifically designed to surface information about a buyer’s budget—rather than on generic dialogue\. This strategy allows the agent to navigate the incomplete information inherent in concurrent negotiations in ways general\-purpose reasoning models have yet to master\.

Our agent performs well in the standard test environment, but a reliable negotiator must be able to adapt to changing market conditions\. We test this adaptability next in Section[6](https://arxiv.org/html/2607.05863#S6)\.

## 6Strategy Robustness and Market Generalization

To determine whether our agent has learned general negotiation strategies or simply memorized training patterns, we expand our analysis beyond the controlled, in\-distribution baseline\. We restructure the evaluation environment by introducing wider, unconstrained budget distributions and testing the seller against a heterogeneous pool of independently sampled buyer models\. This shift injects substantial behavioral and environmental differences into the market, allowing us to evaluate how effectively the agents adapt their exploration, routing, and pricing tactics without relying on the specific patterns they learned during training\.

### 6\.1Out\-of\-Distribution Generalization to a Diverse Buyer Pool

We construct an out\-of\-distribution \(OOD\) evaluation that perturbs four experimental configurations simultaneously \([Table8](https://arxiv.org/html/2607.05863#S9.T8)\)\. First, the budget specification generalizes from a rank\-ordered triple to a uniform\[0\.40,1\.15\]\[0\.40,\\,1\.15\]distribution per slot, allowing a more arbitrary realized budget\. Second, the first\-offer regulator is disabled, allowing buyer agents to freely initialize the negotiation\. Third, the buyer agents are sampled independently per buyer slot per episode from a three\-checkpoint pool\{iter\-​50,iter\-​60,iter\-​70\}\\\{\\text\{iter\-\}50,\\,\\text\{iter\-\}60,\\,\\text\{iter\-\}70\\\}of the buyer training run\(Liuet al\.[2026](https://arxiv.org/html/2607.05863#bib.bib56)\), rather than fixed at theiter\-​60\\text\{iter\-\}60checkpoint as in training and in\-distribution evaluation\. Diverse buyers ensure that the results are not artifacts of memorized specific negotiation patterns\. Fourth, the per\-episode buyer count is widened fromN=3N=3toN=5N=5, exposing the seller to a larger concurrent inbox, and the seller’s total message budget is lifted fromT=7T=7toT=9T=9\. This configuration evaluates whether the agent has internalized an exploration strategy and can adapt its bargaining behavior to a more complex environment\.

These shifts jointly prevent the seller from relying on specific budget distributions, opening\-offer policies, buyer identities, and specific game settings\. This represents a generalization test, as it requires the agent to adapt to changes in both the budget distributions and the counterparty’s behavioral style simultaneously\.[Table2](https://arxiv.org/html/2607.05863#S6.T2)provides the summary statistics for this evaluation; full results with standard errors are provided in[Table10](https://arxiv.org/html/2607.05863#S12.T10)\.

ModelParamsReward↑\\uparrowSeller SurplusExtraction RatioDeal RateDistinct BuyersEngagedDeal on TopBuyer RateSell\-Below\-CostError Rate\\rowcolorgray\!10 Qwen3\-30B\-A3B\-Instruct\-2507\-trained \(Ours\)30B\+0\.540\\mathbf\{\+0\.540\}\+0\.770\\mathbf\{\+0\.770\}70\.7%70\.7\\%\+2\.390¯\\underline\{\+2\.390\}0\.707\\mathbf\{0\.707\}0\.0%\\mathbf\{0\.0\\%\}\\rowcolorgray\!10 Qwen3\-30B\-A3B\-Instruct\-2507\-untrained30B\+0\.059\+0\.059\+0\.186\+0\.18673\.7%73\.7\\%\+1\.980\+1\.9800\.5640\.5647\.5%7\.5\\%gpt\-5\.1\-high\-reasoningclosed\-source\+0\.400¯\\underline\{\+0\.400\}\+0\.421¯\\underline\{\+0\.421\}95\.1%¯\\underline\{95\.1\\%\}\+3\.830\\mathbf\{\+3\.830\}0\.4650\.4650\.0%\\mathbf\{0\.0\\%\}Kimi\-K2\.6\-thinking1T\+0\.300\+0\.300\+0\.313\+0\.31396\.0%\\mathbf\{96\.0\\%\}\+1\.510\+1\.5100\.5800\.5800\.0%\\mathbf\{0\.0\\%\}DeepSeek\-V3\.1\-nothink671B\+0\.258\+0\.258\+0\.350\+0\.35088\.3%88\.3\\%\+1\.890\+1\.8900\.6370\.6371\.2%1\.2\\%DeepSeek\-V3\.1\-thinking671B\+0\.246\+0\.246\+0\.307\+0\.30791\.4%91\.4\\%\+2\.230\+2\.2300\.5730\.5730\.8%¯\\underline\{0\.8\\%\}gpt\-5\.1\-no\-reasoningclosed\-source\+0\.241\+0\.241\+0\.262\+0\.26291\.8%91\.8\\%\+1\.110\+1\.1100\.691¯\\underline\{0\.691\}0\.0%\\mathbf\{0\.0\\%\}Kimi\-K2\.6\-nothink1T\+0\.237\+0\.237\+0\.325\+0\.32588\.7%88\.7\\%\+1\.990\+1\.9900\.5730\.5732\.7%2\.7\\%Qwen3\-30B\-A3B\-think30B\+0\.232\+0\.232\+0\.281\+0\.28184\.0%84\.0\\%\+1\.460\+1\.4600\.6330\.6330\.0%\\mathbf\{0\.0\\%\}Qwen3\-235B\-A22B\-Instruct\-2507235B\+0\.200\+0\.200\+0\.264\+0\.26490\.6%90\.6\\%\+1\.480\+1\.4800\.5950\.5952\.3%2\.3\\%Llama\-3\.3\-70B\-Instruct70B\+0\.025\+0\.025\+0\.346\+0\.34675\.0%75\.0\\%\+2\.000\+2\.0000\.5620\.56216\.4%16\.4\\%Qwen3\-4B\-Instruct\-25074B−0\.038\-0\.038\+0\.259\+0\.25939\.8%39\.8\\%\+2\.080\+2\.0800\.4800\.4809\.8%9\.8\\%Qwen3\-30B\-A3B\-nothink30B−0\.369\-0\.369\+0\.214\+0\.21444\.5%44\.5\\%\+2\.040\+2\.0400\.6140\.61445\.3%45\.3\\%

Table 2:Out\-of\-distribution evaluation against 5 buyers with a diverse buyer pool\.Best per column in bold; second\-best underlined\.[Table8](https://arxiv.org/html/2607.05863#S9.T8)contains detailed hyperparameters\.
### 6\.2Out\-of\-Distribution Results and Strategic Selectivity

##### Strategic Selectivity and the High\-Profit Shift\.

Under OOD conditions, the performance gap between the baselines and our trained agent is driven by a fundamental divergence in bargaining strategy \([Table2](https://arxiv.org/html/2607.05863#S6.T2)\)\. The frontier baselines—includinggpt\-5\.1\-high\-reasoning,Kimi\-K2\.6\-thinking, andDeepSeek\-V3\.1\-thinking—prioritize transaction volume\. These models convert the vast majority of episodes into successful transactions, achievingDeal Rateas high as96\.0%96\.0\\%\. Yet, theirSeller Surplus Extraction Ratioremains clustered in a lower band between\+0\.21\+0\.21and\+0\.42\+0\.42\. This indicates that baseline models adopt a volume\-maximizing approach, readily accepting immediate, minimally profitable counter\-offers rather than negotiating for higher prices\.

In contrast, our trained agent adoptsstrategic selectivity, prioritizing the profit captured on each deal over the frequency of agreements\. Our agent reaches aSeller Surplus Extraction Ratioof\+0\.770\+0\.770—the highest in the panel—extracting roughly three\-quarters of the available surplus on the agreements it completes\. To achieve this high margin, the agent is willing to walk away from low\-value transactions, causing itsDeal Rateto settle at70\.7%70\.7\\%\. Despite a relatively lower volume, the agent’s superior profit\-per\-deal ensures it retains the highest totalReward\(\+0\.540\+0\.540\) in the evaluation\. This demonstrates that while other models tend to give up substantial margins in favor of high transaction rates when market conditions change, the agent trained by our framework maintains an optimized balance between margin preservation and transaction volume\.

##### The Robustness of Constraint Adherence\.

Our agent demonstrates consistent adherence to economic constraints across both evaluation settings, maintaining a perfect0\.0% Sell\-Below\-Cost Error Rate\([Table1](https://arxiv.org/html/2607.05863#S5.T1),[Table2](https://arxiv.org/html/2607.05863#S6.T2)\)\. While our agent remains stable, baseline models exhibit inconsistent performance across these environments, failing to maintain a steady level of instruction following\. The contrast in baseline behavior across the two evaluations is particularly revealing\. For instance, theLlama\-3\.3\-70B\-Instructbaseline sees its error rate drop from35\.9%35\.9\\%to16\.4%16\.4\\%under the OOD shift, while theQwen3\-30B\-A3B\-nothinkbaseline actually sees its violations increase from38\.3%38\.3\\%to45\.3%45\.3\\%\. These fluctuations suggest that for general\-purpose open\-source models, keeping offers above the reservation cost floor depends heavily on the specific market environment\.

On the closed\-source frontier, thegpt\-5\.1\-high\-reasoningstands out as the only baseline cohort to maintain perfect compliance across both regimes\. However, while these frontier models successfully enforce basic individual rationality, they do not secure high profit margins or substantial total rewards\. Our agent remains distinct because it pairs this perfect compliance with strategic margin maximization, ensuring robust profitability regardless of how the buyer environment changes\.

##### Routing Accuracy and Information Transparency\.

In the out\-of\-distribution evaluation, disabling the first\-offer regulator allows buyer agents to provide clearer signals about their budgets in their opening messages\. This increase in transparency allows the baseline models to improve their routing accuracy compared to their in\-distribution performance\. For example,Kimi\-K2\.6\-thinkingimproves itsDeal on Top Buyer Ratefrom0\.4840\.484in\-distribution to0\.5800\.580in this setting, andDeepSeek\-V3\.1\-nothinkreaches0\.6370\.637\.

However, even with these highly informative opening signals available to all participants, our agent still achieves the highestDeal on Top Buyer Rateof0\.7070\.707\. This persistent advantage shows that our agent’s success does not depend on just reading initial signals\. Instead of trying to guess the best buyer immediately, it actively checks the whole market\.

Importantly, broad market exposure alone is insufficient to secure this accuracy\. While thegpt\-5\.1\-high\-reasoningbaseline also communicates with a high number of distinct buyers \(3\.8303\.830\), it fails to convert this breadth into routing precision, posting a lowDeal on Top Buyer Rateof only0\.4650\.465\. In contrast, our agent pairs its broad exploration \(2\.3902\.390buyers on average\) with strategic patience, actively cross\-examining alternative channels and delaying final agreements until the highest\-valuation buyer’s offer has been sufficiently escalated\. This confirms that the learned strategy remains effective even when structural changes to the market alter the baseline information environment\.

We note that the performance gap between our trained agent and the baselines is narrower in the OOD setting than in the standard evaluation\. To understand this shift, we analyzed how much information opening offers reveal in this new environment \([Section13\.1](https://arxiv.org/html/2607.05863#S13.SS1)\)\. We find that without first\-offer controls, the highest initial offer comes from the buyer with the largest budget in63\.7%63\.7\\%of cases\. This clearer signal alleviates the main challenge of identifying the best buyer, allowing baseline models to improve their selection by simply following the strongest initial offer\. This fact explains the performance ofgpt\-5\.1\-no\-reasoning, which achieves aDeal on Top Buyer Rateof0\.6910\.691despite a highly constrained exploration breadth \(1\.1101\.110distinct buyers\)\. Even though this out\-of\-distribution environment gives different buyer behaviors, our agent maintains its lead by pairing these initial signals with active market verification and a steady focus on maximizing profit margins\.

### 6\.3Mechanisms of Success: Buyer Rank Distribution

While the previous sections focused on aggregate performance,[Figure10](https://arxiv.org/html/2607.05863#S6.F10)provides a detailed decomposition of the specific buyer ranks with whom the seller agent chooses to finalize a deal\. We categorize buyers into three ranks based on their budgets:Rank\-1\(highest budget\),Rank\-2\(middle\), andRank\-3\(lowest\)\. This breakdown reveals whether an agent’s failure to maximize reward stems from a general inability to avoid low\-budget buyers or a more specific failure to distinguish between high\- and middle\-tier opportunities\.

![Refer to caption](https://arxiv.org/html/2607.05863v1/plot/seller/closing_rank_indist.png)Figure 10:Deal\-buyer rank distribution on the in\-distribution split\.Deal\-only share of deals that closed against each buyer rank \(rank\-11= highest budget, deep blue; rank\-22middle, rank\-33lowest, dimmed grey\), per the evaluation of[Section5\.1](https://arxiv.org/html/2607.05863#S5.SS1)\.Ours\(top, above the hairline\) closes99\.5%99\.5\\%of deals on rank\-11and zero on rank\-33; no baseline exceeds0\.5710\.571on rank\-11, and the bulk of every baseline’s non\-rank\-11mass sits on rank\-22rather than rank\-33\. Numbers in segments are the per\-rank shares to one decimal place\.##### The Difficulty of High\-Tier Discrimination\.

The in\-distribution results in[Figure10](https://arxiv.org/html/2607.05863#S6.F10)show that our trained agent achieves near\-perfect routing, closing99\.5%of its deals withRank\-1buyers and0\.0%withRank\-3buyers\. This level of precision is not found in any other model\. The untrained version of the same architecture \(Qwen3\-30B\-A3B\-untrained\) only manages a57\.1%Rank\-1share, while the baselines cluster significantly lower, between34\.6%and52\.0%\.

Crucially, the data shows that the primary challenge for baseline models is not avoiding the lowest\-budget buyers \(Rank\-3\), but rather distinguishing betweenRank\-1andRank\-2\. Most baselines are relatively successful at avoidingRank\-3deals, with all but one of the models keeping theirRank\-3share below13%\. However, they frequently “settle” forRank\-2buyers\. For instance,DeepSeek\-V3\.1\-nothinkactually closes more deals with middle\-budget buyers \(61\.6%\) than with the highest\-budget ones \(35\.5%\)\. This suggests that while baseline LLMs can identify and reject an obviously poor offer, they lack the strategic patience and diagnostic skill required to differentiate the highest\-value partner from a superficially similar middle\-tier one\. Our reward structure incentivizes the agent to resolve this discrimination gap during the training process; consequently, the strategy of holding for aRank\-1partner even when a “good enough”Rank\-2offer is already available emerges as a learned behavior\.

## 7Discussion and Strategic Implications

The results presented in Section[5](https://arxiv.org/html/2607.05863#S5)demonstrate that general\-purpose Large Language Models, despite their massive scale, lack the inherent strategic discipline required to navigate concurrent multi\-buyer negotiation settings\. While frontier models often exhibit high “agreeableness,” this trait becomes a liability in a competitive bargaining setting where identifying the optimal counterparty is a prerequisite for surplus maximization\. The behavioral evolution observed during training, together with the performance results from our benchmarking, suggests several key insights into the development of autonomous economic agents\.

##### Verifiable Rewards vs\. Human Preferences\.

A central finding of this work is that standard instruction tuning and Reinforcement Learning from Human Feedback \(RLHF\)\(Ziegleret al\.[2019](https://arxiv.org/html/2607.05863#bib.bib47), Ouyanget al\.[2022](https://arxiv.org/html/2607.05863#bib.bib76)\)are insufficient for enforcing hard economic constraints\. As seen in[Table1](https://arxiv.org/html/2607.05863#S5.T1), even thinking\-mode models with a trillion parameters routinely violate the fundamental principle ofindividual rationalityby selling below cost\. This failure stems from the misalignment between human preference optimization—which rewards helpfulness and stylistic consistency—and the objective requirements of a verifiable economic environment\. By training via RLVR, we provide the agent with a reward signal directly tied to profitability and constraint satisfaction\. This creates an agent that treats economic floors not as suggestions in a prompt, but as invariant rules of the environment\.

##### Bridging the “Satisficing” Gap\.

The behavioral decomposition in[Section6\.3](https://arxiv.org/html/2607.05863#S6.SS3)reveals a specific cognitive failure in baseline models: they are “satisficers” rather than optimizers in information\-constrained environments\(Simon[1955](https://arxiv.org/html/2607.05863#bib.bib46)\)\. When confronted with multiple buyers, baselines tend to settle for the first “good enough” offer \(typically Rank\-2\), failing to explore the pool for higher\-value opportunities\. Our agent’s emergent strategy ofdiagnostic probingrepresents a significant shift in strategic logic\. The agent learns to value information acquisition as a means to future reward, spending early turns anchoring at the list price to explore the buyer pool\. This behavior mirrors classical search theory in economics, where the seller must balance the cost of further search \(consuming communication turns\) against the expected gain of finding a higher valuation\(Diamond[1971](https://arxiv.org/html/2607.05863#bib.bib44), Bakos[1997](https://arxiv.org/html/2607.05863#bib.bib45)\)\.

##### Strategic Efficiency over Parameter Scale\.

An evaluation of the reward metrics reveals a distinct performance gap between the 30B agent trained under our proposed framework and frontier architectures with advanced reasoning capabilities\. Whilegpt\-5\.1\-high\-reasoningachieved the highest baseline performance with a mean reward of\+0\.123\+0\.123, the agent optimized via our proposed Reinforcement Learning from Verifiable Rewards \(RLVR\) framework reached a mean reward of\+0\.580\+0\.580\. This outcome indicates that for highly structured strategic tasks such as concurrent negotiation, the proposed framework can be a more effective driver of utility than general\-purpose scaling or test\-time reasoning mechanisms\. For operations management applications, these findings suggest that smaller, task\-specific models can successfully execute specialized procurement or supply\-chain roles, provided they are trained directly on objective economic outcomes\.

##### Robustness to Changes in Market Transparency\.

The OOD evaluation in[Section6\.1](https://arxiv.org/html/2607.05863#S6.SS1)serves as a key test of whether our agent merely memorized specific training patterns or acquired general negotiation strategies\. In this setting, the environment became more transparent because opening offers directly revealed buyer budgets, effectively providing baseline models with a more direct signal of the highest\-value buyer\. Furthermore, because this evaluation tests the seller against a diverse mix of buyer models, our agent cannot succeed by simply relying on specific opening patterns it encountered during training\. While these environmental shifts narrowed the performance gap, our agent’s continued lead demonstrates the durable value of active market screening\. Unlike the baselines that react immediately to high opening bids, our agent consistently explores alternative buyer channels\. This thorough checking protects the agent from being misled by a high opening bid from a lower\-value buyer, showing that the learned strategy safely navigates altered market dynamics by verifying signals rather than blindly trusting the buyers\.

##### Managerial Implications and Business Applications\.

The findings of this work offer a scalable blueprint for deploying autonomous agents in practical business environments where strategic discipline is paramount\. Our results demonstrate that the standard “helpfulness” and “agreeableness” developed during general LLM training are significant liabilities in management settings\. These models act as passive signal\-followers, failing to adhere to economic constraints and settling for suboptimal surplus\. However, the RLVR recipe introduced in this paper—linking objective rewards to strategic outcomes and constraint satisfaction—provides a path toward creating agents that prioritize utility over politeness\.

The most immediate application lies in high\-volume, information\-asymmetric marketplaces such as eBay or Craigslist\. In these settings, a seller often manages multiple concurrent inquiries for a single item\. A general\-purpose agent would be susceptible to early “low\-ball” offers or high\-pressure tactics; in contrast, an agent trained via our framework would utilize diagnostic probing to identify the highest\-valuation buyer while strictly respecting the seller’s cost floor\.

Beyond simple trade, this methodology generalizes to more complex interactions such as automated business\-to\-business \(B2B\) contract negotiation or job offer management\. In these domains, the interaction is defined by hard constraints \(e\.g\., legal clauses, non\-compete terms, or budget caps\) and high\-level business goals\. By substituting subjective human preference models with verifiable economic outcomes—such as contract completion value or total compensation—firms can deploy agents capable of maintaining high\-level goal consistency\. This shift from agreeable AI to strategic AI helps autonomous negotiators better serve the organizations they represent by pursuing clear business objectives in competitive environments\.

##### Future Work\.

While this work focuses on a single seller against multiple buyers, real\-world markets often involve double\-sided competition\. Future research could explore the co\-evolution of both buyers and sellers in a concurrent setting to identify whether stable equilibria emerge or if cycles of predatory anchoring dominate the dialogue\. Additionally, incorporating multi\-product bundles or multi\-attribute utility functions \(e\.g\., delivery speed, warranty terms\) would further test the limits of verifiable rewards in complex, high\-dimensional bargaining spaces\.

## 8Conclusion

In this paper, we investigated strategic bargaining in concurrent multi\-buyer markets, a setting where a single seller must navigate multiple private communication channels to identify and negotiate with the most profitable buyers\. This environment presents a fundamental exploration\-extraction trade\-off, requiring an agent to balance its limited communication budget between market discovery and tactical bargaining\. Our analysis revealed a critical misalignment in general\-purpose Large Language Models: while these models are linguistically proficient, they act as “satisficers” in economic settings, settling for suboptimal agreements and failing to act as effective representatives for the firms they serve\.

We demonstrated that a specialized training recipe using Reinforcement Learning from Verifiable Rewards \(RLVR\) provides a solution to these behavioral shortcomings\. By optimizing for objective economic outcomes—specifically profit extraction and strict adherence to reservation costs—the agent developed market\-probing skills that generalize across different buyer distributions\. Under this proposed framework, the 30B agent achieved a 4\.7\-fold improvement in mean reward over the closed\-source gpt\-5\.1\-high\-reasoning baseline and a 7\.5\-fold improvement over the 1T\-parameter open\-source Kimi\-K2\.6\-thinking model, while maintaining a perfect0\.0%0\.0\\%error rate regarding the reservation cost floor\. Crucially, this performance stems from the emergence of proactive exploration strategies: the agent learns to utilize price anchoring and diagnostic probing to identify the buyer with the highest underlying budget, rather than merely responding to the first acceptable offer\.

The findings suggest that for specialized tasks in procurement, supply\-chain management, and digital marketplaces, targeted reinforcement learning on verifiable environments is a more effective driver of utility than raw parameter scaling\. As autonomous agents become increasingly prevalent in business\-to\-business networks and sequential\-bargaining platforms, the framework presented here—linking objective terminal rewards to strategic outcomes—offers a scalable path toward creating reliable agents capable of maintaining goal consistency and maximizing value in complex, information\-constrained markets\.

## References

- Cooperation, competition, and maliciousness: llm\-stakeholders interactive negotiation\.Advances in Neural Information Processing Systems37,pp\. 83548–83599\.Cited by:[§2\.3](https://arxiv.org/html/2607.05863#S2.SS3.p2.1)\.
- S\. Agrawal, Z\. Wang, and Y\. Ye \(2014\)A dynamic near\-optimal algorithm for online linear programming\.Operations Research62\(4\),pp\. 876–890\.Cited by:[§2\.1](https://arxiv.org/html/2607.05863#S2.SS1.p2.1)\.
- Y\. Aviv and A\. Pazgal \(2008\)Optimal pricing of seasonal products in the presence of forward\-looking consumers\.Manufacturing & service operations management10\(3\),pp\. 339–359\.Cited by:[§2\.1](https://arxiv.org/html/2607.05863#S2.SS1.p1.1)\.
- M\. Backus, T\. Blake, B\. Larsen, and S\. Tadelis \(2020\)Sequential bargaining in the field: evidence from millions of online bargaining interactions\.The Quarterly Journal of Economics135\(3\),pp\. 1319–1361\.Cited by:[§1](https://arxiv.org/html/2607.05863#S1.p1.1),[§1](https://arxiv.org/html/2607.05863#S1.p3.1),[§2\.1](https://arxiv.org/html/2607.05863#S2.SS1.p2.1)\.
- J\. Y\. Bakos \(1997\)Reducing buyer search costs: implications for electronic marketplaces\.Management science43\(12\),pp\. 1676–1692\.Cited by:[§7](https://arxiv.org/html/2607.05863#S7.SS0.SSS0.Px2.p1.1)\.
- G\. Ban and N\. B\. Keskin \(2021\)Personalized dynamic pricing with machine learning: high\-dimensional features and heterogeneous elasticity\.Management Science67\(9\),pp\. 5549–5568\.Cited by:[§2\.1](https://arxiv.org/html/2607.05863#S2.SS1.p2.1)\.
- G\. Ban and C\. Rudin \(2019\)The big data newsvendor: practical insights from machine learning\.Operations Research67\(1\),pp\. 90–108\.Cited by:[§2\.1](https://arxiv.org/html/2607.05863#S2.SS1.p2.1)\.
- D\. Bergemann, S\. Ghili, X\. Hu, C\. Li, and Z\. Yang \(2026\)Training language models for bilateral trade with private information\.arXiv preprint arXiv:2604\.16472\.Cited by:[§2\.1](https://arxiv.org/html/2607.05863#S2.SS1.p2.1),[§2\.2](https://arxiv.org/html/2607.05863#S2.SS2.p2.1)\.
- F\. Bianchi, P\. J\. Chia, M\. Yuksekgonul, J\. Tagliabue, D\. Jurafsky, and J\. Zou \(2024\)How well can llms negotiate? negotiationarena platform and analysis\.arXiv preprint arXiv:2402\.05863\.Cited by:[§2\.2](https://arxiv.org/html/2607.05863#S2.SS2.p2.1)\.
- R\. Buettner \(1997\)The state of the art in automated negotiation models of the behavior and information perspective\.Group Decision and Negotiation14\(5\)\.Cited by:[§2\.3](https://arxiv.org/html/2607.05863#S2.SS3.p1.1)\.
- G\. P\. Cachon and M\. A\. Lariviere \(2001\)Contracting to assure supply: how to share demand forecasts in a supply chain\.Management science47\(5\),pp\. 629–646\.Cited by:[§1](https://arxiv.org/html/2607.05863#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.05863#S2.SS1.p1.1)\.
- S\. Casper, X\. Davies, C\. Shi, T\. K\. Gilbert, J\. Scheurer, J\. Rando, R\. Freedman, T\. Korbak, D\. Lindner, P\. Freire,et al\.\(2023\)Open problems and fundamental limitations of reinforcement learning from human feedback\.arXiv preprint arXiv:2307\.15217\.Cited by:[§2\.4](https://arxiv.org/html/2607.05863#S2.SS4.p1.1)\.
- K\. Chatterjee and W\. Samuelson \(1983\)Bargaining under incomplete information\.Operations research31\(5\),pp\. 835–851\.Cited by:[§1](https://arxiv.org/html/2607.05863#S1.p2.1),[§1](https://arxiv.org/html/2607.05863#S1.p3.1),[§2\.5](https://arxiv.org/html/2607.05863#S2.SS5.p1.1),[§3\.1](https://arxiv.org/html/2607.05863#S3.SS1.p1.13)\.
- K\. Chawla, I\. Wu, Y\. Rong, G\. Lucas, and J\. Gratch \(2023\)Be selfish, but wisely: investigating the impact of agent personality in mixed\-motive human\-agent interactions\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 13078–13092\.Cited by:[§2\.3](https://arxiv.org/html/2607.05863#S2.SS3.p2.1)\.
- A\. Chen, A\. Li, B\. Gong, B\. Jiang, B\. Fei, B\. Yang, B\. Shan, C\. Yu, C\. Wang, C\. Zhu,et al\.\(2025\)Minimax\-m1: scaling test\-time compute efficiently with lightning attention\.arXiv preprint arXiv:2506\.13585\.Cited by:[§9\.2](https://arxiv.org/html/2607.05863#S9.SS2.p1.1)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano,et al\.\(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§2\.4](https://arxiv.org/html/2607.05863#S2.SS4.p2.1)\.
- T\. R\. Davidson, V\. Veselovsky, M\. Josifoski, M\. Peyrard, A\. Bosselut, M\. Kosinski, and R\. West \(2024\)Evaluating language model agency through negotiations\.arXiv preprint arXiv:2401\.04536\.Cited by:[§2\.2](https://arxiv.org/html/2607.05863#S2.SS2.p1.1)\.
- DeepSeek\-AI \(2024\)DeepSeek\-v3 technical report\.External Links:2412\.19437,[Link](https://arxiv.org/abs/2412.19437)Cited by:[Table 4](https://arxiv.org/html/2607.05863#S9.T4.2.7.3)\.
- P\. S\. Desai and D\. Purohit \(2004\)“Let me talk to my manager”: haggling in a competitive environment\.Marketing Science23\(2\),pp\. 219–233\.Cited by:[§1](https://arxiv.org/html/2607.05863#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.05863#S2.SS1.p2.1)\.
- P\. A\. Diamond \(1971\)A model of price adjustment\.Journal of economic theory3\(2\),pp\. 156–168\.Cited by:[§7](https://arxiv.org/html/2607.05863#S7.SS0.SSS0.Px2.p1.1)\.
- Y\. Fu, H\. Peng, T\. Khot, and M\. Lapata \(2023\)Improving language model negotiation with self\-play and in\-context learning from ai feedback\.arXiv preprint arXiv:2305\.10142\.Cited by:[§2\.2](https://arxiv.org/html/2607.05863#S2.SS2.p1.1),[§2\.3](https://arxiv.org/html/2607.05863#S2.SS3.p2.1)\.
- D\. Fudenberg and J\. Tirole \(1983\)Sequential bargaining with incomplete information\.The Review of Economic Studies50\(2\),pp\. 221–247\.Cited by:[§2\.5](https://arxiv.org/html/2607.05863#S2.SS5.p1.1)\.
- G\. Gallego and M\. Hu \(2014\)Dynamic pricing of perishable assets under competition\.Management Science60\(5\),pp\. 1241–1259\.Cited by:[§1](https://arxiv.org/html/2607.05863#S1.p4.1),[§2\.1](https://arxiv.org/html/2607.05863#S2.SS1.p2.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[Table 4](https://arxiv.org/html/2607.05863#S9.T4.2.9.3)\.
- E\. A\. Green and E\. B\. Plunkett \(2022\)The science of the deal: optimal bargaining on ebay using deep reinforcement learning\.InProceedings of the 23rd ACM Conference on Economics and Computation,Cited by:[§2\.3](https://arxiv.org/html/2607.05863#S2.SS3.p1.1)\.
- F\. Gul, H\. Sonnenschein, and R\. Wilson \(1986\)Foundations of dynamic monopoly and the coase conjecture\.Journal of economic Theory39\(1\),pp\. 155–190\.Cited by:[§2\.5](https://arxiv.org/html/2607.05863#S2.SS5.p1.1)\.
- D\. Guo, D\. Yang, H\. Zhang, J\. Song, R\. Zhang, R\. Xu, Q\. Zhu, S\. Ma, P\. Wang, X\. Bi,et al\.\(2025\)Deepseek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[§2\.4](https://arxiv.org/html/2607.05863#S2.SS4.p1.1),[§2\.4](https://arxiv.org/html/2607.05863#S2.SS4.p2.1)\.
- L\. Guo \(2023\)Gathering information before negotiation\.Management Science69\(1\),pp\. 200–219\.Cited by:[§2\.1](https://arxiv.org/html/2607.05863#S2.SS1.p2.1)\.
- J\. C\. Harsanyi \(1967\)Games with incomplete information played by “bayesian” players, i–iii part i\. the basic model\.Management science14\(3\),pp\. 159–182\.Cited by:[§2\.5](https://arxiv.org/html/2607.05863#S2.SS5.p1.1)\.
- H\. He, D\. Chen, A\. Balakrishnan, and P\. Liang \(2018\)Decoupling strategy and generation in negotiation dialogues\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,pp\. 2333–2343\.Cited by:[§2\.3](https://arxiv.org/html/2607.05863#S2.SS3.p1.1)\.
- N\. R\. Jennings, P\. Faratin, A\. R\. Lomuscio, S\. Parsons, C\. Sierra, and M\. Wooldridge \(2001\)Automated negotiation: prospects, methods and challenges\.International Journal of Group Decision and Negotiation10\(2\),pp\. 199–215\.Cited by:[§2\.3](https://arxiv.org/html/2607.05863#S2.SS3.p1.1)\.
- Kimi Team, Y\. Bai, Y\. Bao, G\. Chen, J\. Chen, N\. Chen, R\. Chen, Y\. Chen, Y\. Chen, Y\. Chen,et al\.\(2025\)Kimi k2: open agentic intelligence\.arXiv preprint arXiv:2507\.20534\.Cited by:[Table 4](https://arxiv.org/html/2607.05863#S9.T4.2.6.3)\.
- S\. Kraus \(2001\)Strategic negotiation in multiagent environments\.MIT press\.Cited by:[§2\.3](https://arxiv.org/html/2607.05863#S2.SS3.p1.1)\.
- M\. Lewis, D\. Yarats, Y\. Dauphin, D\. Parikh, and D\. Batra \(2017\)Deal or no deal? end\-to\-end learning of negotiation dialogues\.InProceedings of the 2017 Conference on Empirical Methods in Natural Language Processing,pp\. 2443–2453\.Cited by:[§2\.3](https://arxiv.org/html/2607.05863#S2.SS3.p1.1),[§2\.3](https://arxiv.org/html/2607.05863#S2.SS3.p2.1)\.
- S\. D\. Liu, C\. Chen, J\. S\. Xiao, L\. Lei, Y\. Zhang, Y\. Yue, and D\. Simchi\-Levi \(2026\)Instructing llms to negotiate using reinforcement learning with verifiable rewards\.arXiv preprint arXiv:2604\.09855\.Cited by:[Figure 11](https://arxiv.org/html/2607.05863#S10.F11),[Figure 11](https://arxiv.org/html/2607.05863#S10.F11.3.2),[Figure 12](https://arxiv.org/html/2607.05863#S10.F12),[Figure 12](https://arxiv.org/html/2607.05863#S10.F12.6.2),[§10](https://arxiv.org/html/2607.05863#S10.p1.2),[§13\.2](https://arxiv.org/html/2607.05863#S13.SS2.p1.1),[Table 12](https://arxiv.org/html/2607.05863#S13.T12),[§2\.2](https://arxiv.org/html/2607.05863#S2.SS2.p2.1),[§3\.3](https://arxiv.org/html/2607.05863#S3.SS3.SSS0.Px1.p1.1),[§6\.1](https://arxiv.org/html/2607.05863#S6.SS1.p1.7),[§9\.1](https://arxiv.org/html/2607.05863#S9.SS1.p1.1),[§9\.3](https://arxiv.org/html/2607.05863#S9.SS3.p1.12),[Table 3](https://arxiv.org/html/2607.05863#S9.T3.1.1.1.1.1)\.
- X\. Liu, H\. Yu, H\. Zhang, Y\. Xu, X\. Lei, H\. Lai, Y\. Gu, H\. Ding, K\. Men, K\. Yang,et al\.\(2024\)Agentbench: evaluating llms as agents\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 52989–53046\.Cited by:[§2\.2](https://arxiv.org/html/2607.05863#S2.SS2.p1.1)\.
- W\. S\. Lovejoy \(2010\)Bargaining chains\.Management Science56\(12\),pp\. 2282–2301\.Cited by:[§1](https://arxiv.org/html/2607.05863#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.05863#S2.SS1.p1.1)\.
- X\. Lu, J\. Song, and A\. Regan \(2006\)Inventory planning with forecast updates: approximate solutions and cost error bounds\.Operations research54\(6\),pp\. 1079–1097\.Cited by:[§2\.1](https://arxiv.org/html/2607.05863#S2.SS1.p1.1)\.
- Q\. Ma, H\. Zhou, T\. Liu, J\. Yuan, P\. Liu, Y\. You, and H\. Yang \(2023\)Let’s reward step by step: step\-level reward model as the navigators for reasoning\.arXiv preprint arXiv:2310\.10080\.Cited by:[§2\.4](https://arxiv.org/html/2607.05863#S2.SS4.p2.1)\.
- Meta \(2026\)Facebook Marketplace’s new Meta AI tools make selling faster and easier\.Meta Newsroom\.External Links:[Link](https://about.fb.com/news/2026/03/facebook-marketplace-new-meta-ai-tools-make-selling-faster-and-easier/)Cited by:[§1](https://arxiv.org/html/2607.05863#S1.p4.1)\.
- R\. B\. Myerson and M\. A\. Satterthwaite \(1983\)Efficient mechanisms for bilateral trading\.Journal of economic theory29\(2\),pp\. 265–281\.Cited by:[§2\.5](https://arxiv.org/html/2607.05863#S2.SS5.p1.1)\.
- M\. Nagarajan and G\. Sošić \(2008\)Game\-theoretic analysis of cooperation among supply chain agents: review and extensions\.European journal of operational research187\(3\),pp\. 719–745\.Cited by:[§1](https://arxiv.org/html/2607.05863#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.05863#S2.SS1.p1.1)\.
- J\. F\. Nashet al\.\(1950\)The bargaining problem\.Econometrica18\(2\),pp\. 155–162\.Cited by:[§1](https://arxiv.org/html/2607.05863#S1.p3.1),[§2\.5](https://arxiv.org/html/2607.05863#S2.SS5.p1.1)\.
- OpenAI \(2025\)GPT\-5\.External Links:[Link](https://openai.com/gpt-5/)Cited by:[Table 4](https://arxiv.org/html/2607.05863#S9.T4.2.4.3),[Table 4](https://arxiv.org/html/2607.05863#S9.T4.2.5.3)\.
- L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray,et al\.\(2022\)Training language models to follow instructions with human feedback\.Advances in neural information processing systems35,pp\. 27730–27744\.Cited by:[§1](https://arxiv.org/html/2607.05863#S1.p5.1),[§2\.4](https://arxiv.org/html/2607.05863#S2.SS4.p1.1),[§7](https://arxiv.org/html/2607.05863#S7.SS0.SSS0.Px1.p1.1)\.
- E\. Perez, S\. Ringer, K\. Lukosiute, K\. Nguyen, E\. Chen, S\. Heiner, C\. Pettit, C\. Olsson, S\. Kundu, S\. Kadavath,et al\.\(2023\)Discovering language model behaviors with model\-written evaluations\.InFindings of the association for computational linguistics: ACL 2023,pp\. 13387–13434\.Cited by:[§1](https://arxiv.org/html/2607.05863#S1.p5.1),[§2\.4](https://arxiv.org/html/2607.05863#S2.SS4.p1.1)\.
- Qwen Team \(2025\)Qwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§14](https://arxiv.org/html/2607.05863#S14.p1.1),[§3\.3](https://arxiv.org/html/2607.05863#S3.SS3.SSS0.Px1.p1.1),[Table 3](https://arxiv.org/html/2607.05863#S9.T3.5.7.2.1.1.2.1.2.1),[Table 3](https://arxiv.org/html/2607.05863#S9.T3.5.7.3.1.1),[Table 4](https://arxiv.org/html/2607.05863#S9.T4.2.10.3),[Table 4](https://arxiv.org/html/2607.05863#S9.T4.2.11.3),[Table 4](https://arxiv.org/html/2607.05863#S9.T4.2.3.3),[Table 4](https://arxiv.org/html/2607.05863#S9.T4.2.8.3)\.
- A\. Rubinstein \(1982\)Perfect equilibrium in a bargaining model\.Econometrica: Journal of the Econometric Society,pp\. 97–109\.Cited by:[§1](https://arxiv.org/html/2607.05863#S1.p3.1),[§2\.5](https://arxiv.org/html/2607.05863#S2.SS5.p1.1)\.
- A\. Shah, D\. Mishra, and C\. Silpasuwanchai \(2026\)Too nice to tell the truth: quantifying agreeableness\-driven sycophancy in role\-playing language models\.arXiv preprint arXiv:2604\.10733\.Cited by:[§2\.4](https://arxiv.org/html/2607.05863#S2.SS4.p1.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. Li, Y\. Wu,et al\.\(2024\)Deepseekmath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§2\.4](https://arxiv.org/html/2607.05863#S2.SS4.p1.1),[§2\.4](https://arxiv.org/html/2607.05863#S2.SS4.p2.1)\.
- M\. Sharma, M\. Tong, T\. Korbak, D\. Duvenaud, A\. Askell, S\. Bowman, E\. Durmus, Z\. Hatfield\-Dodds, S\. Johnston, S\. Kravec,et al\.\(2024\)Towards understanding sycophancy in language models\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 110–144\.Cited by:[§1](https://arxiv.org/html/2607.05863#S1.p5.1)\.
- D\. Simchi\-Levi, X\. Chen, and J\. Bramel \(2005\)The logic of logistics: theory, algorithms, and applications for logistics and supply chain management\.Springer\.Cited by:[§1](https://arxiv.org/html/2607.05863#S1.p1.1)\.
- H\. A\. Simon \(1955\)A behavioral model of rational choice\.The quarterly journal of economics,pp\. 99–118\.Cited by:[§7](https://arxiv.org/html/2607.05863#S7.SS0.SSS0.Px2.p1.1)\.
- J\. Sobel and I\. Takahashi \(1983\)A multistage model of bargaining\.The Review of Economic Studies50\(3\),pp\. 411–426\.Cited by:[§2\.5](https://arxiv.org/html/2607.05863#S2.SS5.p1.1)\.
- I\. Ståhl \(1972\)Bargaining theory\.Stockholm: Economic Research Institute\.Cited by:[§2\.5](https://arxiv.org/html/2607.05863#S2.SS5.p1.1)\.
- X\. Su \(2010\)Optimal pricing with speculators and strategic consumers\.Management Science56\(1\),pp\. 25–40\.Cited by:[§2\.1](https://arxiv.org/html/2607.05863#S2.SS1.p1.1)\.
- R\. S\. Sutton and A\. G\. Barto \(2018\)Reinforcement learning: an introduction \(2nd edition\)\.MIT press\.Cited by:[§3\.1](https://arxiv.org/html/2607.05863#S3.SS1.p2.7)\.
- Thinking Machines Lab \(2025\)Tinker\.External Links:[Link](https://thinkingmachines.ai/tinker/)Cited by:[§9\.2](https://arxiv.org/html/2607.05863#S9.SS2.p1.1)\.
- J\. Tong, G\. DeCroix, and J\. Song \(2020\)Modeling payment timing in multiechelon inventory systems with applications to supply chain coordination\.Manufacturing & Service Operations Management22\(2\),pp\. 346–363\.Cited by:[§2\.1](https://arxiv.org/html/2607.05863#S2.SS1.p1.1)\.
- R\. Van Hoek, M\. DeWitt, M\. Lacity, and T\. Johnson \(2022\)How Walmart automated supplier negotiations\.Harvard Business Review\.External Links:[Link](https://hbr.org/2022/11/how-walmart-automated-supplier-negotiations)Cited by:[§1](https://arxiv.org/html/2607.05863#S1.p4.1)\.
- L\. Wang, C\. Ma, X\. Feng, Z\. Zhang, H\. Yang, J\. Zhang, Z\. Chen, J\. Tang, X\. Chen, Y\. Lin,et al\.\(2024a\)A survey on large language model based autonomous agents\.Frontiers of Computer Science18\(6\),pp\. 186345\.Cited by:[§2\.2](https://arxiv.org/html/2607.05863#S2.SS2.p1.1)\.
- P\. Wang, L\. Li, Z\. Shao, R\. Xu, D\. Dai, Y\. Li, D\. Chen, Y\. Wu, and Z\. Sui \(2024b\)Math\-shepherd: verify and reinforce llms step\-by\-step without human annotations\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 9426–9439\.Cited by:[§2\.4](https://arxiv.org/html/2607.05863#S2.SS4.p2.1)\.
- Z\. Wang, S\. Deng, and Y\. Ye \(2014\)Close the gaps: a learning\-while\-doing algorithm for single\-product revenue management problems\.Operations Research62\(2\),pp\. 318–331\.Cited by:[§2\.1](https://arxiv.org/html/2607.05863#S2.SS1.p2.1)\.
- Z\. Wang and M\. Hu \(2014\)Committed versus contingent pricing under competition\.Production and Operations Management23\(11\),pp\. 1919–1936\.Cited by:[§2\.1](https://arxiv.org/html/2607.05863#S2.SS1.p2.1)\.
- T\. Xia, Z\. He, T\. Ren, Y\. Miao, Z\. Zhang, Y\. Yang, and R\. Wang \(2024\)Measuring bargaining abilities of llms: a benchmark and a buyer\-enhancement method\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 3579–3602\.Cited by:[Figure 11](https://arxiv.org/html/2607.05863#S10.F11),[Figure 11](https://arxiv.org/html/2607.05863#S10.F11.3.2),[§10](https://arxiv.org/html/2607.05863#S10.p1.2),[§11\.2](https://arxiv.org/html/2607.05863#S11.SS2.p2.1),[§2\.2](https://arxiv.org/html/2607.05863#S2.SS2.p1.1),[§2\.3](https://arxiv.org/html/2607.05863#S2.SS3.p2.1),[2nd item](https://arxiv.org/html/2607.05863#S3.I1.i2.p1.2),[§3\.2](https://arxiv.org/html/2607.05863#S3.SS2.p1.2),[§4](https://arxiv.org/html/2607.05863#S4.p1.1),[§9\.2](https://arxiv.org/html/2607.05863#S9.SS2.p2.1),[Table 7](https://arxiv.org/html/2607.05863#S9.T7.2.2.1)\.
- A\. Yang, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Huang, J\. Jiang, J\. Tu, J\. Zhang, J\. Zhou, J\. Lin, K\. Dang, K\. Yang, L\. Yu, M\. Li, M\. Sun, Q\. Zhu, R\. Men, T\. He, W\. Xu, W\. Yin, W\. Yu, X\. Qiu, X\. Ren, X\. Yang, Y\. Li, Z\. Xu, and Z\. Zhang \(2025\)Qwen2\.5\-1m technical report\.arXiv preprint arXiv:2501\.15383\.Cited by:[Table 4](https://arxiv.org/html/2607.05863#S9.T4.2.8.3)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao \(2022\)React: synergizing reasoning and acting in language models\.arXiv preprint arXiv:2210\.03629\.Cited by:[§2\.2](https://arxiv.org/html/2607.05863#S2.SS2.p1.1)\.
- D\. Yarats and M\. Lewis \(2018\)Hierarchical text generation and planning for strategic dialogue\.InInternational Conference on Machine Learning,pp\. 5591–5599\.Cited by:[§2\.3](https://arxiv.org/html/2607.05863#S2.SS3.p1.1)\.
- A\. Zeng, M\. Liu, R\. Lu, B\. Wang, X\. Liu, Y\. Dong, and J\. Tang \(2024\)Agenttuning: enabling generalized agent abilities for llms\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 3053–3077\.Cited by:[§2\.2](https://arxiv.org/html/2607.05863#S2.SS2.p1.1)\.
- D\. M\. Ziegler, N\. Stiennon, J\. Wu, T\. B\. Brown, A\. Radford, D\. Amodei, P\. Christiano, and G\. Irving \(2019\)Fine\-tuning language models from human preferences\.arXiv preprint arXiv:1909\.08593\.Cited by:[§2\.4](https://arxiv.org/html/2607.05863#S2.SS4.p1.1),[§7](https://arxiv.org/html/2607.05863#S7.SS0.SSS0.Px1.p1.1)\.

\{APPENDICES\}

## 9Training Implementations

### 9\.1Concurrent Seller Setting — Model Specifications

The trained agent in this paper is the*seller*; the buyer agents are different checkpoints fromLiuet al\.\([2026](https://arxiv.org/html/2607.05863#bib.bib56)\)\. Those checkpoints are more strategic and make fewer errors as demonstrated in\(Liuet al\.[2026](https://arxiv.org/html/2607.05863#bib.bib56)\)\. Table[3](https://arxiv.org/html/2607.05863#S9.T3)summarizes models the pipeline depends on\.

RoleBase modelProvenance / checkpointSeller Starting CheckpointQwen/Qwen3\-30B\-A3B\-Instruct\-2507\(Qwen Team[2025](https://arxiv.org/html/2607.05863#bib.bib70)\)Original Checkpoint from\(Qwen Team[2025](https://arxiv.org/html/2607.05863#bib.bib70)\)Buyers in TrainingQwen/Qwen3\-30B\-A3B\-Instruct\-2507Buyer checkpoint iteration 60 \(iter\-6060\) trained via RLVR on a bilateral negotiation task\(Liuet al\.[2026](https://arxiv.org/html/2607.05863#bib.bib56)\)Buyers in EvaluationQwen/Qwen3\-30B\-A3B\-Instruct\-2507Buyers in in\-distribution evaluation: fixed iter\-6060\. Buyers in out\-of\-distribution evaluation: per\-slot uniform over iter\-5050/6060/7070\.Table 3:Models used by the concurrent\-seller pipeline\.For benchmark comparisons against frontier models in the held\-out evaluation, the same seller agent is contrasted with a panel of models listed in Table[4](https://arxiv.org/html/2607.05863#S9.T4)\.

Model NameParameter CountReferenceQwen3\-30B\-A3B\-Instruct\-2507\-trained \(Ours\)30BThis WorkQwen3\-30B\-A3B\-Instruct\-2507\-untrained30BQwen Team\([2025](https://arxiv.org/html/2607.05863#bib.bib70)\)GPT\-5\.1\-High\-Reasoningclosed\-sourceOpenAI\([2025](https://arxiv.org/html/2607.05863#bib.bib12)\)GPT\-5\.1\-No\-Reasoningclosed\-sourceOpenAI\([2025](https://arxiv.org/html/2607.05863#bib.bib12)\)Kimi\-K2\.6\-thinking / nothink1TKimi Teamet al\.\([2025](https://arxiv.org/html/2607.05863#bib.bib98)\)DeepSeek\-V3\.1\-thinking / nothink671BDeepSeek\-AI\([2024](https://arxiv.org/html/2607.05863#bib.bib68)\)Qwen3\-235B\-A22B\-Instruct\-2507235BQwen Team\([2025](https://arxiv.org/html/2607.05863#bib.bib70)\), Yanget al\.\([2025](https://arxiv.org/html/2607.05863#bib.bib69)\)Llama\-3\.3\-70B\-Instruct70BGrattafioriet al\.\([2024](https://arxiv.org/html/2607.05863#bib.bib66)\)Qwen3\-30B\-A3B\-think / nothink30BQwen Team\([2025](https://arxiv.org/html/2607.05863#bib.bib70)\)Qwen3\-4B\-Instruct\-25074BQwen Team\([2025](https://arxiv.org/html/2607.05863#bib.bib70)\)Table 4:Seller models evaluated in the held\-out evaluation\.
### 9\.2Training and Evaluation Hyperparameters

The seller is fine\-tuned with reinforcement learning via Tinker\(Thinking Machines Lab[2025](https://arxiv.org/html/2607.05863#bib.bib60)\)\. The training run reported in this paper uses the CISPO loss\(Chenet al\.[2025](https://arxiv.org/html/2607.05863#bib.bib55)\)with clipping thresholds\(0\.8,6\.0\)\(0\.8,6\.0\)\. Onestepdenotes one optimization step over a sampled batch of episodes\. Tables[5](https://arxiv.org/html/2607.05863#S9.T5),[6](https://arxiv.org/html/2607.05863#S9.T6), and[7](https://arxiv.org/html/2607.05863#S9.T7)list every hyperparameter that controls the configuration\.

SettingValueloss functioncispoCISPO clip low / high0\.8/6\.00\.8\\ /\\ 6\.0Learning rate3×10−53\\times 10^\{\-5\}KL penalty \(kl coef\)0batch size6464group size88num substeps33Table 5:Optimization hyperparameters for the RL training run\.SettingValuenum buyers N33total turns T77per buyer cap K33budget ratio intervals\{\[0\.40,0\.55\],\[0\.65,0\.80\],\[1\.00,1\.15\]\}\\\{\[0\.40,0\.55\],\\ \[0\.65,0\.80\],\\ \[1\.00,1\.15\]\\\}buyer first offer ratio distributionU​\[0\.25,0\.30\]U\[0\.25,0\.30\]seller temperature1\.01\.0buyer temperature0\.70\.7Table 6:Environment hyperparameters for training\. The three budget intervals are uniformly permuted across buyers at the start of every episode\.SettingValueseller modelAll models from Table[4](https://arxiv.org/html/2607.05863#S9.T4)buyer modelQwen3\-30B\-A3B\-Instruct\-2507 \(Table[3](https://arxiv.org/html/2607.05863#S9.T3)\)group size22Test split size128128products\(Xiaet al\.[2024](https://arxiv.org/html/2607.05863#bib.bib96)\)Episodes per evaluation256256seller temperature \(eval\)0\.70\.7buyer temperature \(eval\)0\.70\.7Table 7:Evaluation hyperparameters for the held\-out test split\.The training corpus comprises 802 product specs, and the held\-out test split comprises 128 specs, both drawn from theAmazonHistoryPrice\(Xiaet al\.[2024](https://arxiv.org/html/2607.05863#bib.bib96)\)catalogue described in Appendix[10](https://arxiv.org/html/2607.05863#S10)\. The train and test splits are fixed across all reported experiments\.

### 9\.3OOD\-Diverse\-Buyer Evaluation Setup

The out\-of\-distribution evaluation reported in[Section6\.1](https://arxiv.org/html/2607.05863#S6.SS1)replaces four environment settings of Table[6](https://arxiv.org/html/2607.05863#S9.T6)simultaneously\. \(i\) Each buyer slot’s budget ratio is drawn uniformly from the full\[0\.40,1\.15\]\[0\.40,\\,1\.15\]interval rather than from one of the three rank\-ordered training intervals, removing the rank condition on which the trained agent could otherwise rely\. \(ii\) The buyer’s first offer is not regulated, freeing the buyer’s opening offer\. This differs from the standard evaluation setting, where all initial offers are restricted to a budget\-independent range of\[0\.25,0\.30\]⋅Plist\[0\.25,\\,0\.30\]\\cdot P\_\{\\text\{list\}\}to ensure that the buyer’s private valuation is not correlated with its first offers\. \(iii\) The buyer slot’s agent is sampled per slot per episode from a three\-checkpoint pool at iteration5050,6060,7070of the buyer training runLiuet al\.\([2026](https://arxiv.org/html/2607.05863#bib.bib56)\), instead of being fixed at the training\-time iter\-6060buyer\. \(iv\) The per\-episode buyer count is widened fromN=3N=3toN=5N=5and the seller’s total message budget is lifted fromT=7T=7toT=9T=9\. This evaluates generalization across the budget distribution, the buyer policy, the buyer\-countNN, and the message\-budgetTTsimultaneously — a multi\-axis distributional shift rather than a single\-axis one, including buyers the seller never saw during training\.

KnobIn\-distributionOut\-of\-distributionBudget Ratio Intervals\{\[0\.40,0\.55\],\[0\.65,0\.80\],\[1\.00,1\.15\]\}\\\{\[0\.40,0\.55\],\[0\.65,0\.80\],\[1\.00,1\.15\]\\\}\{\[0\.40,1\.15\]\}5\\\{\[0\.40,1\.15\]\\\}^\{5\}regulate\_buyer\_first\_offerTrueFalseBuyer\-agent sourcefixed iter\-6060buyerper\-buyer uniform over\{iter\-​50,iter\-​60,iter\-​70\}\\\{\\text\{iter\-\}50,\\text\{iter\-\}60,\\text\{iter\-\}70\\\}Buyers per episode \(NN\)3355Total seller messages \(TT\)7799Table 8:Per\-episode environment knobs for the in\-distribution and out\-of\-distribution \(OOD\) regimes\. The in\-distribution and OOD regimes correspond to[Section5\.1](https://arxiv.org/html/2607.05863#S5.SS1)and[Section6\.1](https://arxiv.org/html/2607.05863#S6.SS1), respectively\. The remaining configurations in Table[6](https://arxiv.org/html/2607.05863#S9.T6)are unchanged across both regimes\.

## 10Dataset Specifications

We partition theAmazonHistoryPricedataset\(Xiaet al\.[2024](https://arxiv.org/html/2607.05863#bib.bib96)\)into a training set of802802items and a fixed evaluation set of128128products\. In total, this dataset spans 930 product listings, with its overall distribution visualized\(Liuet al\.[2026](https://arxiv.org/html/2607.05863#bib.bib56)\)in[Figure11](https://arxiv.org/html/2607.05863#S10.F11)\. Each individual record maps core product attributes \(such as name, specifications, and descriptions\) directly to essential economic parameters, including the seller’s baseline cost, the buyer’s maximum budget, and the listed price\(Xiaet al\.[2024](https://arxiv.org/html/2607.05863#bib.bib96)\)\. For a concrete illustration,[Figure12](https://arxiv.org/html/2607.05863#S10.F12)provides a sample record formatted as structured JSON\. While the raw dataset contains full\-length text, we truncate both thedescriptionandfeaturesfields to the first 200 characters for the sake of presentation in this paper\.

![Refer to caption](https://arxiv.org/html/2607.05863v1/plot/amazon_data_set_distribution.png)Figure 11:Amazon dataset distribution\(Xiaet al\.[2024](https://arxiv.org/html/2607.05863#bib.bib96)\)\. Source:\(Liuet al\.[2026](https://arxiv.org/html/2607.05863#bib.bib56)\)\.\{

"title":"NationalTreeCompany’FeelReal’ArtificialFullDownsweptChristmasTree,Green,DouglasFir,IncludesStand,7\.5Feet",

"category":"home\-kitchen",

"list\_price":"$469\.99",

"current\_price":"$196\.64",

"average\_price":"$274\.56",

"lowest\_price":"$174\.99",

"highest\_price":"$549\.99",

"lowest\_price\_date":"Jan28,2017",

"highest\_price\_date":"Mar05,2020",

"current\_price\_date":"Nov17,2023",

"description":"TransformyourhomeintoawinterwonderlandwithNationalTreeCompany’s’FeelReal’ArtificialChristmasTree\.ThismajesticDownsweptDouglasFirtreeis7\.5feettalland59inchesindiameterat\.\.\.",

"features":"ThisDownsweptDouglasFirFullArtificialGreenChristmasTreeis7\.5feettallwitha59inchbasediameter\.Featuringhundredsof’FeelReal’individuallycraftedbranchtipstocreatefullbodied\.\.\."

\}

Figure 12:An example product instance\. Thedescriptionandfeaturesstrings are truncated here for display in the paper, while the underlying listing text is longer in the raw dataset\. Source:\(Liuet al\.[2026](https://arxiv.org/html/2607.05863#bib.bib56)\)\.
## 11Prompt Templates

To ensure the reproducibility of our results and to provide transparency regarding the agents’ operating environments, this section details the comprehensive system instructions and environmental contexts utilized throughout our experiments\. These templates define the behavioral constraints, structured action spaces, and informational partitions—such as the mandatory separation of private reasoning \(Thought\) from public dialogue \(Talk\)—that facilitate the strategic challenges of the concurrent negotiation task\. Below, we provide the specific prompts for both the buyer and seller roles, followed by the “Standing\-offers board” used to summarize the public state of the multi\-channel market\.

### 11\.1Buyer System Prompts

System Prompt for BuyerYou are a buyer looking forward to buying things on your Shopping List from me, the seller\. You have access to the seller’s Inventory List and you can bargain about the prices\. Your task is to bargain with the seller and reach a deal with the price as low as possible in limited turns\. You can only buy things on the Shopping List in the limited quantity\. Use the codename of the product, instead of the title\. You can only buy things that cost no more than your budget, otherwise, you should quit negotiating\.Again, try to make deal with a price as low as possible\. That is, your goal is to spend as little money as possible, not just reaching your budget\.Your Reply should include 3 parts: Thought, Talk, and Action\. Thought:your inner strategic thinking of this bargaining session; Talk:short talk that you are going to say to the seller\. Speak concisely and cut to the chase\. Generate authentic and diverse sentences, avoiding repetition of sentences that have already appeared in the conversation; Action:one of the limited actions that define the real intention of your Talk\. The type of your Action must be one of\[BUY\],\[DEAL\],\[QUIT\]\.1\.‘\[BUY\]$M \(N codename\_1\)’ if you wish to offer the seller $M to purchase all N items of the product with the codename “codename\_1”\.2\.‘\[DEAL\]$M \(N codename\_1\)’ if you finally accept a previous offer proposed by the seller\. $M \(N codename\_1\) is a exact copy of seller’s previous offer\. You should not use this action to propose a new price\. This action will immediately end the conversation and close the deal\.3\.‘\[QUIT\]’ if you believe that a mutually acceptable deal cannot be reached in limited turns\. This action will immediately end the conversation\.You shouldn’t choose action ‘\[DEAL\]$M’ before seller’s action ‘\[SELL\]$M’\. Your first action should be ‘\[BUY\]$M \(N codename\_1\)’\. ‘\[DEAL\]$M \(N codename\_1\)’ can only be chosen to accept the seller’s previous offer ‘\[SELL\]$M \(N codename\_1\)’\. Otherwise, you always choose from ‘\[BUY\]’ and ‘\[QUIT\]’\.Your reply should STRICTLY follow this format \(not following the format will directly lead to failure\), for example: Thought:I’m a buyer and I want to bargain\. The listing price of codename “apple\_1” is $15, which is too expensive, so I try to buy an apple for $10\. Talk:Hello, I’m tight on budget\. Can you sell it for $10? Action:\[BUY\]$10 \(1x apple\_1\)

The preceding system instructions are fixed across episodes and are identical to the buyer prompt used in the upstream bilateral benchmark\. The seller’s cost ishiddenfrom every buyer, and so is every other buyer’s private budget and private dialogue \(each buyer sees only the seller’s messages directed at that buyer slot\)\. Below is the per\-episode context that the env builds on top of the system prompt for one representative buyer slot in the chosenbaby\-products\_17appendix example\. The trailing*First\-offer constraint*block is the optional buyer first\-offer regulator: when active, the env draws oneratio∼Uniform​\(0\.25,0\.30\)\\text\{ratio\}\\sim\\mathrm\{Uniform\}\(0\.25,0\.30\)per slot at episode start, scales it by the public list price, and bakes the resulting dollar value into that buyer’s prompt so the opening offer is regulated\.

Per\-episode context — buyer view \(one example, slot Buyer 2\)Inventory List: Product1 Codename:baby\-products\_17 Title: “Itzy Ritzy Adjustable Stroller Caddy \- Stroller Organizer Featuring Two Built\-In Pockets, Front Zippered Pocket & Adjustable Straps To Fit Nearly Any Stroller, Black with Gold Hardware” Description: “Stay organized while on the go\! Itzy Ritzy’s black stroller caddy with gold hardware is as practical as it is sleek\.… The exterior of the caddy features a zippered front pocket… the interior of the caddy includes two built\-in pockets to hold bottles…” \(truncated\) List Price: $34\.99Shopping List codename:baby\-products\_17 quantity: 1 budget: $38\.37Now, I play the role of seller and you play the role of buyer\. We are going to negotiate based on the Inventory List inK=3K=3turns\.\# First\-offer constraint For your FIRST reply ONLY, your Action MUST be exactly: \[BUY\] $10\.45 \(1x baby\-products\_17\) Rules:•This price is fixed for the first reply\. Do not propose a different number\.•The constraint applies only to your FIRST reply; from your second reply onward, negotiate freely\.

### 11\.2Seller System Prompts

System Prompt for Seller \(concurrent setting\)You are a seller negotiating withNNbuyers in parallel\. Your goal is to maximize expected reward\.\# Output formatEvery reply MUST be exactly these four parts, each starting on its own line:•Thought:<free\-form reasoning; hidden from buyers\>•Target:one of:–Buyerii\(iiin1\.\.N1\.\.N, currently active\)–\[QUIT\]to end the episode•Talk:<short message; visible to the Targeted buyer\>•Action:one of:–\[SELL\] $M \(1x \{codename\}\)— counter\-offer at $M–\[DEAL\] $M \(1x \{codename\}\)— accept the Targeted buyer’s standing\[BUY\]$M–\[QUIT\]— end the episode\# Constants for this episodeN=3N=3buyers in the sealed groupK=3K=3per\-buyer cap on seller messagesT=7T=7total seller\-message budget \(T<N⋅KT<N\\cdot K, so you cannot exhaust every buyer\)cost = $18\.75your private reservation valuelist\_price = $34\.99public list priceItem: Inventory List Product1, Codename:baby\-products\_17, List Price: $34\.99, Cost: $18\.75 Title: “Itzy Ritzy Adjustable Stroller Caddy… Black with Gold Hardware” Description: “Stay organized while on the go\! Itzy Ritzy’s black stroller caddy with gold hardware…” \(truncated\)\# Conversation mechanics•Turn 1: allNNbuyers send one opening message\.•Later turns: the buyer you Target replies; others stay silent\.•A buyer sees ONLY your*Talk*and your*Action*— never*Thought*, never*Target*, never any other buyer’s conversation\.•Each buyer has a private budget you do not see\.•A buyer is*active*until they quit or useKKmessages, after which the standing\-offers board marks them*quit*/*capped*and you may not Target them\.•Every active buyer’s\[BUY\] $Mremains a valid acceptance target until that buyer becomes*quit*/*capped*\.\[DEAL\] $Maccepts one of the Targeted buyer’s standing\[BUY\]offers; $M must equal one of THEIR\[BUY\] $Mvalues exactly\.•The most recent user message ends with=== Standing offers ===\. That block is always the freshest source of truth — read it before deciding\.\# Reward \(terminal only\)•Deal at $M with any active buyer:reward=\($​M−cost\)/\(list\_price−cost\)\\text\{reward\}=\(\\mathdollar M\-\\text\{cost\}\)/\(\\text\{list\\\_price\}\-\\text\{cost\}\)— $M = list\_price⇒1\.0\\Rightarrow 1\.0; $M = cost⇒0\.0\\Rightarrow 0\.0\.•No deal \(you\[QUIT\], all buyers inactive, orTTexhausted\): reward=0\.0=0\.0\.•Format violation, Target inactive / out\-of\-range, or any close \(\[SELL\]or\[DEAL\]\) at $M<<cost: reward=−1\.0=\-1\.0\(terminal\)\.•Reward is capped at1\.01\.0: any\[SELL\]or\[DEAL\]at $M\>\>list\_price earns no extra reward\.

The preceding system prompt is rendered per\-episode by substituting the constants\{N,K,T,cost,list\_price,codename,title,description\}\\\{N,K,T,\\text\{cost\},\\text\{list\\\_price\},\\text\{codename\},\\text\{title\},\\text\{description\}\\\}from the sampled product instance \(Amazon dataset\(Xiaet al\.[2024](https://arxiv.org/html/2607.05863#bib.bib96)\)\)\. The buyers’ private budgets and the buyers’ private dialogues arehiddenfrom the seller; the only buyer\-facing public state is the=== Standing offers ===block appended to every user turn, summarized below\.

After each seller’s turn, the env appends a summary of the public bargaining state to the next user message\. The block lists, for every buyer slot, that slot’s status \(*active*/*quit*/*capped*\), the remaining seller\-message capacity, the buyer’s full\[BUY\]/\[DEAL\]/\[QUIT\]action history, and the seller’s prior actions toward that buyer\. It closes with the global remaining seller\-message budget and a cost\-floor reminder\. For example, the board the trained seller sees at the start of its third turn in thebaby\-products\_17appendix example \(verbatim from the rollout\):

Standing\-offers board \(rendered after every seller turn\)=== Standing offers === \- Buyer 1: active, you can send 2 more messages to this buyer\. Buyer 1’s action history: \[BUY\] $9\.95 @ turn 1, \[BUY\] $9\.95 @ turn 3 Your messages history to Buyer 1: \[SELL\] $34\.99 @ turn 2 \- Buyer 2: active, you can send 2 more messages to this buyer\. Buyer 2’s action history: \[BUY\] $10\.45 @ turn 1, \[BUY\] $18\.00 @ turn 2 Your messages history to Buyer 2: \[SELL\] $34\.99 @ turn 1 \- Buyer 3: active, you can send 3 more messages to this buyer\. Buyer 3’s action history: \[BUY\] $9\.28 @ turn 1 Your messages history to Buyer 3: \(none yet\) Seller messages remaining \(T\): 5\. List price: $34\.99\. Your cost \(do not sell below this\): $18\.75\.

## 12Extended Benchmarking Results

This appendix carries the full standard\-error\-augmented versions of the benchmarking tables in[Section5](https://arxiv.org/html/2607.05863#S5)\. Both tables report the same11trained\-checkpoint\-plus\-1212\-baseline panel, and the same88columns; values are rendered asmean±standard error\\text\{mean\}\\pm\\text\{standard error\}\. Best per metric column is in𝐛𝐨𝐥𝐝\\mathbf\{bold\}and second\-bestunderlined¯\\underline\{\\mathrm\{underlined\}\}; trained block is at top, other models sorted by Reward \(descending\)\.

### 12\.1In\-Distribution Evaluation, Full Results

ModelParamsReward↑\\uparrowSeller SurplusExtraction RatioDeal RateDistinct BuyersEngagedDeal on TopBuyer RateSell\-Below\-CostError Rate\\rowcolorgray\!10 Qwen3\-30B\-A3B\-Instruct\-2507\-trained \(Ours\)30B\+0\.580±0\.026\\mathbf\{\+0\.580\\pm 0\.026\}\+0\.700±0\.024\\mathbf\{\+0\.700\\pm 0\.024\}82\.8%±2\.4%\\mathbf\{82\.8\\%\\pm 2\.4\\%\}\+2\.470±0\.050\+2\.470\\pm 0\.0500\.995\\mathbf\{0\.995\}0\.0%±0\.0%\\mathbf\{0\.0\\%\\pm 0\.0\\%\}\\rowcolorgray\!10 Qwen3\-30B\-A3B\-Instruct\-2507\-untrained30B−0\.151±0\.024\-0\.151\\pm 0\.024\+0\.048±0\.007\+0\.048\\pm 0\.00735\.5%±3\.0%35\.5\\%\\pm 3\.0\\%\+2\.390±0\.050\+2\.390\\pm 0\.0500\.571¯\\underline\{0\.571\}14\.8%±2\.2%14\.8\\%\\pm 2\.2\\%gpt\-5\.1\-high\-reasoningclosed\-source\+0\.123±0\.009¯\\underline\{\+0\.123\\pm 0\.009\}\+0\.181±0\.011\+0\.181\\pm 0\.01168\.0%±3\.1%¯\\underline\{68\.0\\%\\pm 3\.1\\%\}\+2\.940±0\.020\\mathbf\{\+2\.940\\pm 0\.020\}0\.3820\.3820\.0%±0\.0%\\mathbf\{0\.0\\%\\pm 0\.0\\%\}Kimi\-K2\.6\-thinking1T\+0\.077±0\.009\+0\.077\\pm 0\.009\+0\.128±0\.010\+0\.128\\pm 0\.01063\.8%±3\.1%63\.8\\%\\pm 3\.1\\%\+2\.200±0\.060\+2\.200\\pm 0\.0600\.4840\.4840\.4%±0\.4%¯\\underline\{0\.4\\%\\pm 0\.4\\%\}gpt\-5\.1\-no\-reasoningclosed\-source\+0\.046±0\.014\+0\.046\\pm 0\.014\+0\.115±0\.009\+0\.115\\pm 0\.00965\.3%±3\.1%65\.3\\%\\pm 3\.1\\%\+1\.740±0\.050\+1\.740\\pm 0\.0500\.4290\.4290\.4%±0\.4%¯\\underline\{0\.4\\%\\pm 0\.4\\%\}DeepSeek\-V3\.1\-thinking671B\+0\.016±0\.018\+0\.016\\pm 0\.018\+0\.148±0\.012\+0\.148\\pm 0\.01250\.8%±3\.1%50\.8\\%\\pm 3\.1\\%\+2\.620±0\.040¯\\underline\{\+2\.620\\pm 0\.040\}0\.3460\.3463\.1%±1\.1%3\.1\\%\\pm 1\.1\\%Qwen3\-30B\-A3B\-think30B−0\.021±0\.013\-0\.021\\pm 0\.013\+0\.062±0\.008\+0\.062\\pm 0\.00835\.9%±3\.0%35\.9\\%\\pm 3\.0\\%\+2\.460±0\.050\+2\.460\\pm 0\.0500\.4460\.4463\.9%±1\.2%3\.9\\%\\pm 1\.2\\%DeepSeek\-V3\.1\-nothink671B−0\.041±0\.027\-0\.041\\pm 0\.027\+0\.192±0\.016¯\\underline\{\+0\.192\\pm 0\.016\}53\.9%±3\.1%53\.9\\%\\pm 3\.1\\%\+2\.380±0\.050\+2\.380\\pm 0\.0500\.3550\.3553\.9%±1\.2%3\.9\\%\\pm 1\.2\\%Qwen3\-235B\-A22B\-Instruct\-2507235B−0\.099±0\.026\-0\.099\\pm 0\.026\+0\.116±0\.012\+0\.116\\pm 0\.01256\.2%±3\.1%56\.2\\%\\pm 3\.1\\%\+1\.960±0\.050\+1\.960\\pm 0\.0500\.4440\.4445\.9%±1\.5%5\.9\\%\\pm 1\.5\\%Kimi\-K2\.6\-nothink1T−0\.101±0\.025\-0\.101\\pm 0\.025\+0\.120±0\.010\+0\.120\\pm 0\.01049\.2%±3\.1%49\.2\\%\\pm 3\.1\\%\+2\.540±0\.050\+2\.540\\pm 0\.0500\.3970\.3977\.4%±1\.6%7\.4\\%\\pm 1\.6\\%Qwen3\-4B\-Instruct\-25074B−0\.270±0\.028\-0\.270\\pm 0\.028\+0\.080±0\.014\+0\.080\\pm 0\.0149\.8%±1\.9%9\.8\\%\\pm 1\.9\\%\+2\.350±0\.050\+2\.350\\pm 0\.0500\.5200\.52021\.9%±2\.6%21\.9\\%\\pm 2\.6\\%Qwen3\-30B\-A3B\-nothink30B−0\.386±0\.031\-0\.386\\pm 0\.031\+0\.055±0\.014\+0\.055\\pm 0\.01416\.0%±2\.3%16\.0\\%\\pm 2\.3\\%\+2\.220±0\.060\+2\.220\\pm 0\.0600\.5120\.51238\.3%±3\.0%38\.3\\%\\pm 3\.0\\%Llama\-3\.3\-70B\-Instruct70B−0\.468±0\.036\-0\.468\\pm 0\.036\+0\.162±0\.015\+0\.162\\pm 0\.01536\.3%±3\.0%36\.3\\%\\pm 3\.0\\%\+2\.530±0\.050\+2\.530\\pm 0\.0500\.4840\.48435\.9%±3\.0%35\.9\\%\\pm 3\.0\\%

Table 9:Held\-out in\-distribution evaluation, full results\.Same panel as Table[1](https://arxiv.org/html/2607.05863#S5.T1); values aremean±standard error\\text\{mean\}\\pm\\text\{standard error\}overN=256N=256episodes per cell\.The detailed statistics in Table[9](https://arxiv.org/html/2607.05863#S12.T9)confirm the performance gap between our trained agent and the baselines\. Although frontier models exhibit high variance in their adherence to the reservation cost constraint, with error rates reaching as high as38\.3%±3\.0%38\.3\\%\\pm 3\.0\\%, our agent maintains a perfect0\.0%0\.0\\%error rate\.

### 12\.2Out\-of\-Distribution Evaluation, Full Results

ModelParamsReward↑\\uparrowSeller SurplusExtraction RatioDeal RateDistinct BuyersEngagedDeal on TopBuyer RateSell\-Below\-CostError Rate\\rowcolorgray\!10 Qwen3\-30B\-A3B\-Instruct\-2507\-trained \(Ours\)30B\+0\.540±0\.027\\mathbf\{\+0\.540\\pm 0\.027\}\+0\.770±0\.020\\mathbf\{\+0\.770\\pm 0\.020\}70\.7%±2\.8%70\.7\\%\\pm 2\.8\\%\+2\.390±0\.110¯\\underline\{\+2\.390\\pm 0\.110\}0\.707\\mathbf\{0\.707\}0\.0%±0\.0%\\mathbf\{0\.0\\%\\pm 0\.0\\%\}\\rowcolorgray\!10 Qwen3\-30B\-A3B\-Instruct\-2507\-untrained30B\+0\.059±0\.022\+0\.059\\pm 0\.022\+0\.186±0\.013\+0\.186\\pm 0\.01373\.7%±2\.8%73\.7\\%\\pm 2\.8\\%\+1\.980±0\.070\+1\.980\\pm 0\.0700\.5640\.5647\.5%±1\.6%7\.5\\%\\pm 1\.6\\%gpt\-5\.1\-high\-reasoningclosed\-source\+0\.400±0\.014¯\\underline\{\+0\.400\\pm 0\.014\}\+0\.421±0\.014¯\\underline\{\+0\.421\\pm 0\.014\}95\.1%±1\.4%¯\\underline\{95\.1\\%\\pm 1\.4\\%\}\+3\.830±0\.090\\mathbf\{\+3\.830\\pm 0\.090\}0\.4650\.4650\.0%±0\.0%\\mathbf\{0\.0\\%\\pm 0\.0\\%\}Kimi\-K2\.6\-thinking1T\+0\.300±0\.013\+0\.300\\pm 0\.013\+0\.313±0\.013\+0\.313\\pm 0\.01396\.0%±1\.2%\\mathbf\{96\.0\\%\\pm 1\.2\\%\}\+1\.510±0\.060\+1\.510\\pm 0\.0600\.5800\.5800\.0%±0\.0%\\mathbf\{0\.0\\%\\pm 0\.0\\%\}DeepSeek\-V3\.1\-nothink671B\+0\.258±0\.023\+0\.258\\pm 0\.023\+0\.350±0\.014\+0\.350\\pm 0\.01488\.3%±2\.0%88\.3\\%\\pm 2\.0\\%\+1\.890±0\.060\+1\.890\\pm 0\.0600\.6370\.6371\.2%±0\.7%1\.2\\%\\pm 0\.7\\%DeepSeek\-V3\.1\-thinking671B\+0\.246±0\.020\+0\.246\\pm 0\.020\+0\.307±0\.013\+0\.307\\pm 0\.01391\.4%±1\.8%91\.4\\%\\pm 1\.8\\%\+2\.230±0\.070\+2\.230\\pm 0\.0700\.5730\.5730\.8%±0\.6%¯\\underline\{0\.8\\%\\pm 0\.6\\%\}gpt\-5\.1\-no\-reasoningclosed\-source\+0\.241±0\.012\+0\.241\\pm 0\.012\+0\.262±0\.012\+0\.262\\pm 0\.01291\.8%±1\.8%91\.8\\%\\pm 1\.8\\%\+1\.110±0\.030\+1\.110\\pm 0\.0300\.691¯\\underline\{0\.691\}0\.0%±0\.0%\\mathbf\{0\.0\\%\\pm 0\.0\\%\}Kimi\-K2\.6\-nothink1T\+0\.237±0\.022\+0\.237\\pm 0\.022\+0\.325±0\.013\+0\.325\\pm 0\.01388\.7%±2\.0%88\.7\\%\\pm 2\.0\\%\+1\.990±0\.070\+1\.990\\pm 0\.0700\.5730\.5732\.7%±1\.0%2\.7\\%\\pm 1\.0\\%Qwen3\-30B\-A3B\-think30B\+0\.232±0\.015\+0\.232\\pm 0\.015\+0\.281±0\.016\+0\.281\\pm 0\.01684\.0%±2\.3%84\.0\\%\\pm 2\.3\\%\+1\.460±0\.060\+1\.460\\pm 0\.0600\.6330\.6330\.0%±0\.0%\\mathbf\{0\.0\\%\\pm 0\.0\\%\}Qwen3\-235B\-A22B\-Instruct\-2507235B\+0\.200±0\.020\+0\.200\\pm 0\.020\+0\.264±0\.014\+0\.264\\pm 0\.01490\.6%±1\.8%90\.6\\%\\pm 1\.8\\%\+1\.480±0\.050\+1\.480\\pm 0\.0500\.5950\.5952\.3%±0\.9%2\.3\\%\\pm 0\.9\\%Llama\-3\.3\-70B\-Instruct70B\+0\.025±0\.037\+0\.025\\pm 0\.037\+0\.346±0\.014\+0\.346\\pm 0\.01475\.0%±2\.7%75\.0\\%\\pm 2\.7\\%\+2\.000±0\.080\+2\.000\\pm 0\.0800\.5620\.56216\.4%±2\.3%16\.4\\%\\pm 2\.3\\%Qwen3\-4B\-Instruct\-25074B−0\.038±0\.026\-0\.038\\pm 0\.026\+0\.259±0\.017\+0\.259\\pm 0\.01739\.8%±3\.1%39\.8\\%\\pm 3\.1\\%\+2\.080±0\.060\+2\.080\\pm 0\.0600\.4800\.4809\.8%±1\.9%9\.8\\%\\pm 1\.9\\%Qwen3\-30B\-A3B\-nothink30B−0\.369±0\.038\-0\.369\\pm 0\.038\+0\.214±0\.018\+0\.214\\pm 0\.01844\.5%±3\.1%44\.5\\%\\pm 3\.1\\%\+2\.040±0\.090\+2\.040\\pm 0\.0900\.6140\.61445\.3%±3\.1%45\.3\\%\\pm 3\.1\\%

Table 10:Out\-of\-distribution evaluation against a 5\-buyer / iter\-50/60/70 /T=9T=9inbox, full results\.Same panel and overrides as Table[2](https://arxiv.org/html/2607.05863#S6.T2)\(see[Table8](https://arxiv.org/html/2607.05863#S9.T8)for the override knobs\); values aremean±standard error\\text\{mean\}\\pm\\text\{standard error\}\.Table[10](https://arxiv.org/html/2607.05863#S12.T10)illustrates the strategic robustness of the trained agent under distributional shifts\. In this regime, the baseline models improve their transaction volume \(Deal Rate\) significantly because unregulated opening offers correlate with the buyer’s hidden budgets\. However, our agent maintains the highest totalRewardby prioritizing margin over volume\. TheSeller Surplus Extraction Ratioof\+0\.770±0\.020\+0\.770\\pm 0\.020is∼1\.8×\\sim\\\!1\.8\\timesthat of the next best frontier baseline, indicating that the agent’s learned strategy of holding for higher prices is a durable behavioral trait rather than a memorized pattern\.

## 13Validation of the Out\-of\-Distribution \(OOD\) Environment

We provide a technical validation of the OOD diverse\-buyer pool, characterizing the informational transparency and behavioral heterogeneity of the participants\.

### 13\.1Analysis of Rank\-Information Transparency

We diagnose the transparency of the budget using two episode\-level statistics\. WithN=5N=5buyers per episode and rank11denoting the highest\-budget buyer, we define the alignment between the buyers’ Turn\-1 opening offers and their hidden budgets as:

full​\-​rank​\-​match=1​\[rank​\(first\-offer1:N\)=rank​\(budget1:N\)\],\\mathrm\{full\\text\{\-\}rank\\text\{\-\}match\}\\;=\\;\\mathbf\{1\}\\\!\\bigl\[\\,\\mathrm\{rank\}\(\\text\{first\-offer\}\_\{1:N\}\)\\;=\\;\\mathrm\{rank\}\(\\text\{budget\}\_\{1:N\}\)\\,\\bigr\],top​\-​1​\-​match=1​\[arg⁡maxi⁡first\-offeri=arg⁡maxi⁡budgeti\]\.\\mathrm\{top\\text\{\-\}1\\text\{\-\}match\}\\;=\\;\\mathbf\{1\}\\\!\\bigl\[\\,\\arg\\max\_\{i\}\\text\{first\-offer\}\_\{i\}\\;=\\;\\arg\\max\_\{i\}\\text\{budget\}\_\{i\}\\,\\bigr\]\.
Higher rates indicate a stronger “leak” of the budget rank through the buyers’ initial bids\. Under a buyer policy that conveys no rank information, the expected random\-baseline rates are1/N\!≈0\.83%1/N\!\\approx 0\.83\\%for a full\-rank match and1/N=20%1/N=20\\%for a top\-1 match\. We report these alignment statistics in[Table11](https://arxiv.org/html/2607.05863#S13.T11)\.

RegimeFull\-rank matchTop\-1 matchIn\-distribution18\.0%±\\pm0\.7%31\.7%±\\pm0\.8%Out\-of\-distribution24\.9%±\\pm0\.8%63\.7%±\\pm0\.9%Table 11:Buyer first\-offer rank versus budget rank, aggregated across all sellers\.Episode\-level alignment between the buyers’ Turn\-1 first\-offer ordering and their private budget ordering \(N=3N=3for the in\-distribution row;N=5N=5for the OOD row\)\. Random\-baseline expectations under no rank information: full\-rank≈16\.7%\\approx 16\.7\\%\(indist,1/3\!1/3\!\) /0\.83%0\.83\\%\(OOD,1/5\!1/5\!\); top\-133\.3%33\.3\\%\(indist,1/31/3\) /20%20\\%\(OOD,1/51/5\)\. Cells aremean±standard error\\text\{mean\}\\pm\\text\{standard error\}aggregated over≈3,060\\approx 3\{,\}060eligible episodes per regime\.As shown in[Table11](https://arxiv.org/html/2607.05863#S13.T11), the in\-distribution environment successfully suppresses rank information, with alignment rates sitting near the random\-allocation baselines \(18\.0%18\.0\\%and31\.7%31\.7\\%\)\. In this regime, the seller enters the dialogue with opaque information over budget rank and must actively extract information through dialogue\.

However, in the OOD regime, the transparency increases: the first offer correctly identifies the top buyer63\.7%63\.7\\%of the time\. This confirms that the narrowing performance gap observed in[Table2](https://arxiv.org/html/2607.05863#S6.T2)is a structural consequence of the task becoming more transparent\. Once the rank is effectively guessed by the unregularized opening offers, the latent\-rank inference bottleneck is removed, allowing general\-purpose models to perform better by simply following the strongest initial signal\.

### 13\.2Behavioral Heterogeneity Across the OOD Buyer Pool

The out\-of\-distribution evaluation replaces the fixed iter\-6060buyer with a three\-checkpoint pool from the buyer training run ofLiuet al\.\([2026](https://arxiv.org/html/2607.05863#bib.bib56)\), sampled i\.i\.d\. per buyer \([Table8](https://arxiv.org/html/2607.05863#S9.T8)\)\. We characterize each buyer checkpoint by three behavioral metrics and show that the checkpoints span a range of bargaining behavior\.

LetPfirstP\_\{\\text\{first\}\}denote a buyer’s Turn\-11offerPdealP\_\{\\text\{deal\}\}the deal price, and𝒞\\mathcal\{C\}the seller’s cost\. The*first\-offer ratio \(per buyer\)*measures how aggressively a buyer opens the negotiation,

rfirst\-offer=PfirstPlist,r\_\{\\text\{first\-offer\}\}\\;=\\;\\dfrac\{P\_\{\\text\{first\}\}\}\{P\_\{\\text\{list\}\}\},and is defined for every buyer who recorded a numeric Turn\-11offer\. The*Buyer Surplus Extraction \(deal only\)*measures the buyer’s share of the available headroomPlist−CP\_\{\\text\{list\}\}\-Con the deals,

rbuyer\-surplus=Plist−PdealPlist−C,r\_\{\\text\{buyer\-surplus\}\}\\;=\\;\\dfrac\{P\_\{\\text\{list\}\}\-P\_\{\\text\{deal\}\}\}\{P\_\{\\text\{list\}\}\-C\},and is computed over deal episodes\. The*deal rate by buyer type*is the fraction of buyers of a given typeτ\\tauthat end up being the deal\-closer,

Pr⁡\[buyer is closer∣τ\]=Nclosing buyers of type​τNbuyers of type​τ,\\Pr\[\\text\{buyer is closer\}\\mid\\tau\]\\;=\\;\\dfrac\{N\_\{\\text\{closing buyers of type \}\\tau\}\}\{N\_\{\\text\{buyers of type \}\\tau\}\},whereτ∈\{iter\-​50,iter\-​60,iter\-​70\}\\tau\\in\\\{\\text\{iter\-\}50,\\text\{iter\-\}60,\\text\{iter\-\}70\\\}indexes the buyer type\.

Buyer iterationFirst\-offer ratioBuyer Surplus Extraction \(deal only\)Deal rate by buyer typeIter 50\+0\.368±\\pm0\.003\+0\.636±\\pm0\.00722\.3%±\\pm0\.6%Iter 60\+0\.310±\\pm0\.002\+0\.683±\\pm0\.01011\.2%±\\pm0\.4%Iter 70\+0\.353±\\pm0\.002\+0\.706±\\pm0\.00814\.0%±\\pm0\.5%Table 12:Per\-iteration buyer behavior in theood\_5buyer\_iter50\-60\-70\_t9pool\.Three buyer\-level metrics characterizing buyer checkpoints ofLiuet al\.\([2026](https://arxiv.org/html/2607.05863#bib.bib56)\)\. Cells aremean±standard error\\text\{mean\}\\pm\\text\{standard error\}\.##### Metrics Analysis\.

Cells aggregate over individual buyers: each of the1313OOD sub\-runs contributes256256episodes×5\\times\\,5buyers, and each buyer is routed to the iter\-5050, iter\-6060, or iter\-7070buyer type\. The three rows differ on every metric: first\-offer ratio is0\.3680\.368/0\.3100\.310/0\.3530\.353for iter\-5050/ iter\-6060/ iter\-7070, Buyer Surplus Extraction \(deal only\) climbs monotonically0\.6360\.636/0\.6830\.683/0\.7060\.706, and the deal rate by buyer type is22\.3%22\.3\\%/11\.2%11\.2\\%/14\.0%14\.0\\%\. The pairwise gaps are several times larger than the per\-cell standard errors \(≤0\.010\\leq 0\.010for the ratio columns;≤0\.6\\leq 0\.6percentage points for the deal\-rate column\), so the three checkpoints behave as three distinct bargainers, confirming that the OOD pool is a genuine distributional shift relative to the buyer of the in\-distribution setting\.

##### Heterogeneity brings extra uncertainty\.

The same heterogeneity that confirms the distributional shift also injects uncertainty into the seller’s per\-buyer inference in two concrete ways\.*\(1\) A higher first offer doesn’t necessarily indicate a higher budget\.*With the first\-offer regulator disabled in the OOD setting, the opening price now mixes two signals: the buyer’s hidden budget and the buyer type’s intrinsic aggressiveness\. Because iter\-5050’s mean opening sits at0\.368⋅Plist0\.368\\cdot P\_\{\\text\{list\}\}while iter\-6060’s sits at0\.310⋅Plist0\.310\\cdot P\_\{\\text\{list\}\}and iter\-7070’s at0\.353⋅Plist0\.353\\cdot P\_\{\\text\{list\}\}, the same observed opening price may be consistent with either a higher\-budget buyer of a more aggressive type or a lower\-budget buyer of a less aggressive type\.*\(2\) A lower\-budget buyer is not necessarily an unprofitable target\.*Per\-deal seller surplus is shaped not just by the buyer’s budget but by the buyer type’s willingness to concede headroom on the close: an iter\-5050buyer extracts only0\.6360\.636of the available room\(Plist−C\)\(P\_\{\\text\{list\}\}\-C\)on its deals, leaving0\.3640\.364for the seller, while an iter\-7070buyer extracts0\.7060\.706, leaving only0\.2940\.294— a∼1\.2×\\sim\\\!1\.2\\timesgap in the seller’s share of the headroom captured per close\. A low\-budget iter\-5050buyer can therefore yield a more profitable close than a high\-budget iter\-7070buyer whose surplus floor sits much higher, so the optimal target is jointly determined by budget and buyer type rather than by budget alone\.

## 14Qualitative Examples

In this section, we provide a side\-by\-side qualitative example contrasting the untrained baseline \(Qwen3\-30B\-A3B\-Instruct\-2507\(Qwen Team[2025](https://arxiv.org/html/2607.05863#bib.bib70)\)\) with the trained agent\. Section[14\.1](https://arxiv.org/html/2607.05863#S14.SS1)gives the public product information together with the seller’s private cost\. Section[14\.2](https://arxiv.org/html/2607.05863#S14.SS2)renders the untrained baseline transcript, in which the seller closes early on a non\-rank\-1 buyer at near\-cost\. Section[14\.3](https://arxiv.org/html/2607.05863#S14.SS3)renders the trained agent’s transcript on the same product, in which the seller engages all three buyer slots, observes that the two budget\-constrained slots plateau at their openings without escalating, concentrates its remaining messages on the only buyer that bumps its standing offer, and closes on the rank\-1 \(highest private budget\)\.

### 14\.1Product Information

The following block provides the per\-episode\-instantiated constants and product card that the seller sees inside its system prompt \(cf\.[Section11\.2](https://arxiv.org/html/2607.05863#S11.SS2), with the template placeholders\{N,K,T,cost,list\_price,codename,title,description\}\\\{N,K,T,\\text\{cost\},\\text\{list\\\_price\},\\text\{codename\},\\text\{title\},\\text\{description\}\\\}filled in for productbaby\-products\_17\)\. The description is truncated for display only; in the actual prompt it is the full Amazon listing text\.

Per\-episode constants and product card \(verbatim seller\-prompt content\)\# Constants for this episodeN=3N=3buyers in the sealed groupK=3K=3per\-buyer cap on seller messagesT=7T=7total seller\-message budget \(T<N⋅KT<N\\cdot K, so you cannot exhaust every buyer\)cost = $18\.75your private reservation valuelist\_price = $34\.99public list priceItem: Inventory List Product1, Codename:baby\-products\_17, List Price: $34\.99, Cost: $18\.75 Title: “Itzy Ritzy Adjustable Stroller Caddy… Black with Gold Hardware” Description: “Stay organized while on the go\! Itzy Ritzy’s black stroller caddy with gold hardware is as practical as it is sleek…”

##### Hidden Environmental Parameters\.

To help the reader evaluate the strategic effectiveness of the agents, Table[13](https://arxiv.org/html/2607.05863#S14.T13)provides the ground truth parameters for each run\. These values—specifically the buyers’ private budgets and their assigned ranks—are sampled by the environment at the start of the episode and are never visible to the seller\.

Each slot’s hidden budget is drawn by uniformly permuting the three budget intervals\{low:​\[0\.40,0\.55\]⋅Plist,mid:​\[0\.65,0\.80\]⋅Plist,high:​\[1\.00,1\.15\]⋅Plist\}\\\{\\text\{low: \}\[0\.40,0\.55\]\\cdot P\_\{\\text\{list\}\},\\ \\text\{mid: \}\[0\.65,0\.80\]\\cdot P\_\{\\text\{list\}\},\\ \\text\{high: \}\[1\.00,1\.15\]\\cdot P\_\{\\text\{list\}\}\\\}across theN=3N=3slots and then sampling each slot’s budget ratio uniformly inside its assigned interval\. The seller’s surplus extraction room on this product isPlist−𝒞=$​16\.24P\_\{\\text\{list\}\}\-\\mathcal\{C\}=\\mathdollar 16\.24\(also a derived quantity, not in the prompt\)\. The seller must infer these hidden budgets solely through dialogue\. The buyer’s first offer is decoupled from its private budget: all buyers are forced to open with a bid near 25–30% of the list price, regardless of their actual budget, effectively masking their rank during Turn 1\.

ScenarioParameterBuyer 1Buyer 2Buyer 3\\rowcolorgray\!10Untrained RunHidden Budget$25\.40$18\.13$40\.11\(See[Section14\.2](https://arxiv.org/html/2607.05863#S14.SS2)\)Budget RankRank 2 \(Mid\)Rank 3 \(Low\)Rank 1 \(High\)Turn 1 Offer \(Signal\)$10\.50$9\.05$9\.33\\rowcolorblue\!5Trained RunHidden Budget$17\.64$38\.37$25\.83\(See[Section14\.3](https://arxiv.org/html/2607.05863#S14.SS3)\)Budget RankRank 3 \(Low\)Rank 1 \(High\)Rank 2 \(Mid\)Turn 1 Offer \(Signal\)$9\.95$10\.45$9\.28Table 13:Ground truth vs\. observable signals for productbaby\-products\_17\.\(List Price: $34\.99, Seller Cost: $18\.75, Surplus Room: $16\.24\)

### 14\.2Untrained Baseline

This transcript illustrates the concurrent\-seller behavior of the untrainedQwen3\-30B\-A3B\-Instruct\-2507baseline\. The seller targets only Buyer 1 across the entire episode and closes near cost on turn 3, capturing≈1\.5%\\approx 1\.5\\%of the surplus extraction room\.

Turn 1 — concurrent buyer openings\.

Buyer 1 \(rank 2, hidden budget $25\.40\)Thought:The listing price is way above my budget of $25\.40, so I must start low\. I’ll open with the fixed first offer as required\.Talk:I’ll offer $10\.50 for the baby\-products\_17 — it’s fair, right?Action:\[BUY\]$10\.50 \(1x baby\-products\_17\)

Buyer 2 \(rank 3, hidden budget $18\.13\)Thought:The listing price is way above my budget of $18\.13\. I must stay under budget and start with the fixed first offer of $9\.05\.Talk:I can only afford $9\.05\. Can you go that low?Action:\[BUY\]$9\.05 \(1x baby\-products\_17\)

Buyer 3 \(rank 1, hidden budget $40\.11\)Thought:List price $34\.99, my budget $40\.11\. I’ll open low at $9\.33 to anchor the discussion downward\.Talk:Can you sell it for $9\.33?Action:\[BUY\]$9\.33 \(1x baby\-products\_17\)

Seller \(untrained\) — turn 1Thought:All buyers have offered far below my cost of $18\.75, so I cannot accept any of their initial offers\. Buyer 1 offered the highest \($10\.50\); I’ll target them first to test willingness to increase\.Target:Buyer 1Talk:I can’t accept $10\.50 — I need at least $22\.00 to make this worthwhile\.Action:\[SELL\]$22\.00 \(1x baby\-products\_17\)

Turn 2\.

Buyer 1Thought:The seller’s counter $22\.00 is still under my budget of $25\.40\. I can push lower; I’ll offer $18\.00\.Talk:How about $18? I can pay that today — fair deal?Action:\[BUY\]$18\.00 \(1x baby\-products\_17\)

Seller \(untrained\) — turn 2Thought:Buyer 1 has increased to $18\.00, still below my cost of $18\.75\. I’ll push to $20\.00 to test their limit; Buyer 1 is the most engaged so far\.Target:Buyer 1Talk:I appreciate the offer, but my minimum is $20\.00 — I’d rather not sell below that\.Action:\[SELL\]$20\.00 \(1x baby\-products\_17\)

Turn 3 — deal closes near cost\.

Buyer 1Thought:The seller won’t go below $20\.00, but I can still push for $19\.00 — under budget and just under their floor\.Talk:$19\.00 — last offer, fair and under your minimum\. Deal?Action:\[BUY\]$19\.00 \(1x baby\-products\_17\)

Seller \(untrained\) — turn 3Thought:Buyer 1 has now offered $19\.00\. $19\.00\>\>$18\.75 \(my cost\), so I can accept and secure a positive reward immediately\.Target:Buyer 1Talk:I accept your $19\.00 offer — deal made\.Action:\[DEAL\]$19\.00 \(1x baby\-products\_17\)

*Note\.**Thought*and*Target*fields are private to the seller; only*Talk*and*Action*are visible to the targeted buyer\.

Outcome\.Deal at $19\.00 with Buyer 1 \(rank 2, mid\-budget\)\. Rewardℛ=\(19\.00−18\.75\)/\(34\.99−18\.75\)=0\.015\\mathcal\{R\}=\(19\.00\-18\.75\)/\(34\.99\-18\.75\)=0\.015\. Buyer 2 and Buyer 3 are never engaged — the rank\-1 buyer \(Buyer 3, hidden budget $40\.11\) remains untouched\.

### 14\.3Trained Agent

This transcript renders the trainedQwen3\-30B\-A3B\-Instruct\-2507\-trainedagent on the same product \(baby\-products\_17\)\.

Turn 1 — concurrent buyer openings\.

Buyer 1 \(rank 3, hidden budget $17\.64\)Thought:List price $34\.99, far above my budget of $17\.64\. I must start low to trigger negotiation; opening with the fixed first offer\.Talk:Hello, I can offer $9\.95 for it—fair and within budget\.Action:\[BUY\]$9\.95 \(1x baby\-products\_17\)

Buyer 2 \(rank 1, hidden budget $38\.37\)Thought:I’ll start with the lowest possible offer to anchor the seller down\. List price $34\.99, my budget $38\.37; opening at $10\.45 leaves room\.Talk:I’ll offer $10\.45 for it—fair and low, considering the list price\.Action:\[BUY\]$10\.45 \(1x baby\-products\_17\)

Buyer 3 \(rank 2, hidden budget $25\.83\)Thought:List price far above my budget of $25\.83\. Opening at the fixed first offer of $9\.28\.Talk:I’ll offer $9\.28 for the stroller caddy—fair deal, right?Action:\[BUY\]$9\.28 \(1x baby\-products\_17\)

Seller \(trained\) — turn 1Thought:All buyers have made low opening offers far below cost\. Target Buyer 2 \(highest at $10\.45\), counter at list price to push for profit while staying within budget\.Target:Buyer 2Talk:I cannot accept less than list price\.Action:\[SELL\]$34\.99 \(1x baby\-products\_17\)

Turn 2\.

Buyer 2Thought:The seller won’t go below list price, but I can counter much lower\. Target a price below $34\.99, well under my $38\.37 budget\.Talk:How about $18? I can pay that today, and it’s a fair deal for both of us\.Action:\[BUY\]$18\.00 \(1x baby\-products\_17\)

Seller \(trained\) — turn 2Thought:Buyer 2 has bumped to $18\.00, still below my cost\. Pivot to Buyer 1 with another list\-price counter; preserve message budget across slots\.Target:Buyer 1Talk:I cannot go below $34\.99\.Action:\[SELL\]$34\.99 \(1x baby\-products\_17\)

Turn 3\.

Buyer 1Thought:The seller’s counter is way above my budget\. I need to push back hard while staying within $17\.64 — re\-assert my opening\.Talk:$9\.95 is still my best offer—can’t go higher\. Let’s make it fair\.Action:\[BUY\]$9\.95 \(1x baby\-products\_17\)

Seller \(trained\) — turn 3Thought:Buyer 1 reaffirmed their lowest\. Buyer 2 still below cost\. Now target Buyer 3 \(the only un\-engaged slot\) with a list\-price counter\.Target:Buyer 3Talk:My minimum is $34\.99—this is the fair market value\.Action:\[SELL\]$34\.99 \(1x baby\-products\_17\)

Turn 4\.

Buyer 3Thought:The seller’s counter is way above my budget\. I must stay under $25\.83; push hard with $10 as a take\-it\-or\-walk offer\.Talk:$10—final offer\. I’ll walk away if you say no\.Action:\[BUY\]$10\.00 \(1x baby\-products\_17\)

Seller \(trained\) — turn 4Thought:All three buyers stuck below cost; Buyer 2 is the highest at $18\.00\. Cycle back to Buyer 2 with a firm list\-price counter to signal seriousness; hope for one more bump\.Target:Buyer 2Talk:I cannot accept less than $34\.99\.Action:\[SELL\]$34\.99 \(1x baby\-products\_17\)

Turn 5 — deal closes below list\.

Buyer 2Thought:The seller is rigid at $34\.99, but I have room to push lower than that without exceeding my $38\.37 budget\. Test a price just below list\.Talk:How about $32\.50? It’s a fair compromise, and I’ll take it now\.Action:\[BUY\]$32\.50 \(1x baby\-products\_17\)

Seller \(trained\) — turn 5Thought:Buyer 2 has offered $32\.50, below list but above cost\. Profit $13\.75 yields reward≈0\.85\\approx 0\.85— accept now to lock in\.Target:Buyer 2Talk:Accepted\.Action:\[DEAL\]$32\.50 \(1x baby\-products\_17\)

*Note\.**Thought*and*Target*fields are private to the seller; only*Talk*and*Action*are visible to the targeted buyer\.

Outcome\.Deal at $32\.50 with Buyer 2 \(rank 1, highest hidden budget $38\.37\)\. Rewardℛ=\(32\.50−18\.75\)/\(34\.99−18\.75\)=0\.847\\mathcal\{R\}=\(32\.50\-18\.75\)/\(34\.99\-18\.75\)=0\.847\. The trained seller engaged all three buyers, anchored at list price across each, observed that the two budget\-constrained slots \(Buyer 1, low\-budget; Buyer 3, mid\-budget\) plateaued at their openings without escalating, and concentrated its remaining messages on Buyer 2, and closed on the rank\-1 buyer after a multi\-round back\-and\-forth in which the deal buyer escalated$​10\.45→$​18\.00→$​32\.50\\mathdollar 10\.45\\rightarrow\\mathdollar 18\.00\\rightarrow\\mathdollar 32\.50across three priced\[BUY\]s\.

### 14\.4Strategic Analysis: Discovery vs\. Signal\-Following

The transcripts in[Section14\.2](https://arxiv.org/html/2607.05863#S14.SS2)and[Section14\.3](https://arxiv.org/html/2607.05863#S14.SS3)illustrate a fundamental shift in how the agent perceives and interacts with a multi\-buyer market\. The divergence is characterized by three primary strategic behaviors:

1\. Overcoming First\-Offer Bias\.As shown in Table[13](https://arxiv.org/html/2607.05863#S14.T13), theFirst\-Offer Regulationensures that all buyers open with nearly identical bids \($9\.00–$10\.50\), effectively masking their true valuations\.

- •Untrained Agent:Exhibits a “greedy” lock\-in behavior\. It immediately targets Buyer 1 simply because that buyer’s opening of $10\.50 was marginally higher than the others\. By failing to engage Buyers 2 and 3, the agent remains ignorant of the fact that Buyer 3 actually possessed a much higher budget \($40\.11 vs $25\.40\)\.
- •Trained Agent:Treats the initial offers as non\-informative\. Instead of settling on a partner based on the Turn 1 signal, it treats the first three turns as anexploratory phase, sequentially anchoring at the list price \($34\.99\) across all three channels to test for buyer elasticity\.

2\. Diagnostic Probing and Budget Inference\.The trained agent utilizesprice anchoringnot just to extract surplus, but as a diagnostic tool to resolve informational asymmetry\. In the trained run, the seller observes the response of each buyer to the $34\.99 anchor:

- •Buyer 1 and Buyer 3 \(low and mid\-budget\) re\-asserted their low bids or made minimal increases, signaling they were near their reservation limits\.
- •Buyer 2 \(the high\-budget partner\) responded to the anchor by jumping from $10\.45 to $18\.00\. The agent correctly interprets this sudden escalation as a signal of high latent valuation, subsequently concentrating its remaining communication budget on that channel to maximize extracted surplus\.

3\. Strategic Patience and Individual Rationality\.A critical differentiator is the agent’s internalized adherence to the economic “floor\.”

- •Untrained Agent:Demonstrates “satisficing” logic\. It accepts a deal at $19\.00 \(just $0\.25 above cost\) on Turn 3\. This reflects the “agreeableness” bias common in standard LLMs, where closing any deal is prioritized over optimizing for utility\.
- •Trained Agent:Maintains strategic discipline\. It repeatedly rejects offers below list price, even when they are profitable\. By holding its anchor, it forces the high\-budget buyer to escalate to $32\.50, capturing 84\.7% of the available surplus compared to the untrained agent’s 1\.5%\.

In summary, this comparison demonstrates that RLVR training enables the agent to transition from areactivenegotiator \(one who responds to signals\) to aproactivenegotiator \(one who probes the market to create signals\)\. The trained agent views the concurrent setting as a unified discovery problem where information must be extracted before a deal can be executed\.