LLM Agents for Deliberative Collaboration: A Study on Joint Decision Making Under Partial Observability

# A Study on Joint Decision Making Under Partial Observability
Source: [https://arxiv.org/html/2607.06157](https://arxiv.org/html/2607.06157)
## LLM Agents for Deliberative Collaboration: A Study on Joint Decision Making Under Partial Observability

Chenxu Wang\*,1, Yongkun Yang\*,1, Boyuan Du\*,2, Shiwei Lin1, Huaping Liu1 1Department of Computer Science and Technology, Tsinghua University 2Fuzhou University [jimwangcx@gmail\.com](https://arxiv.org/html/2607.06157v1/mailto:[email protected]),[hpliu@tsinghua\.edu\.cn](https://arxiv.org/html/2607.06157v1/mailto:[email protected])

###### Abstract

Deliberation plays a crucial role in collaboration; when humans work together, they naturally engage in communication to align information and reach an agreement\. In this paper, we investigate deliberative large language model \(LLM\) agents under partially observable joint decision\-making tasks\. We formalize deliberative collaboration as a cooperative joint decision problem with partial and asymmetric observations, and introduce a scalable benchmark that instantiates this problem across multiple task settings and domains in which agents must exchange information through deliberation to reach a joint decision with a shared reward\. We then instantiate a reference scaffold and evaluation protocol for deliberative agents and conduct a systematic evaluation of a range of representative LLMs\. The results reveal that complex deliberative collaboration tasks continue to challenge state\-of\-the\-art language models\. Even with the aid of external mathematical tools, language models may fail in either the deliberation process for aligning information or the complex reasoning process for making the decision\. On the other hand, diagnostic analysis reveals that the deliberation process may also provide opportunities for reflection and error correction, sometimes improving performance over centralized baselines\. Altogether, our work establishes a foundation for evaluating and improving LLM agents in deliberative collaboration and provides insights into the strengths, limitations, and properties of current LLM\-based multi\-agent systems\. Code is available at[this repository](https://github.com/wcx21/deliberative-collaboration-agents)\.

LLM Agents for Deliberative Collaboration: A Study on Joint Decision Making Under Partial Observability

11footnotetext:Equal contribution\.## 1Introduction

Recent advances in large language models \(LLMs\) and language agents have achieved impressive progress across a variety of domains, including social intelligenceZhou et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib48)\); Mou et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib26)\); Wang et al\. \([2024a](https://arxiv.org/html/2607.06157#bib.bib34)\), multi\-agent collaborationYan et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib41)\); Chen et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib4)\); Guo et al\. \([2024](https://arxiv.org/html/2607.06157#bib.bib10)\); Liu et al\. \([2024a](https://arxiv.org/html/2607.06157#bib.bib21)\), negotiation or deliberationZhu et al\. \([2025b](https://arxiv.org/html/2607.06157#bib.bib51)\); Choi et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib5)\); Hu et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib13)\); Bianchi et al\. \([2024](https://arxiv.org/html/2607.06157#bib.bib2)\); Karanam et al\. \([2024](https://arxiv.org/html/2607.06157#bib.bib15)\), and decision makingDolant and Kumar \([2025](https://arxiv.org/html/2607.06157#bib.bib6)\); Eo et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib8)\)\. However, an open question remains: can LLM agents collaborate well through deliberation when they each possess only partial, asymmetric information? We present a representative example of deliberative collaboration in[Figure˜1](https://arxiv.org/html/2607.06157#S1.F1)\. In deliberative collaboration scenarios, the agents must communicate, exchange their knowledge, and reason together to reach a shared decision\. This setting is inherently more complex and comprehensive than pure deliberation or collaboration, as it demands task\-related communication and joint decision\-making\.

![Refer to caption](https://arxiv.org/html/2607.06157v1/x1.png)Figure 1:An illustrative example of deliberative collaboration under partial observability\. In this menu design scenario, multiple AI agents \(e\.g\., an AI butler and a cooking robot\) observe different aspects of the environment, such as ingredient availability and guest preferences, and must exchange information through deliberation to reach a single joint decision\. This structure can generalize to other cooperative joint decision problems\.In this paper, we study LLM agents in deliberative collaboration scenarios\. We formalize deliberative collaboration as a partially observable joint decision\-making problem, where multiple agents each have their own partial observation of the environment, and the agents should reach a consensus on a final decision as the output and share the same reward\. Unlike competitive or adversarial settings, the challenge in such scenarios lies in effective information exchange, belief alignment, and coordinated decision\-making under partial observability\.

We build a scalable benchmark that instantiates this problem across multiple related task settings, across collaborative menu design and task allocation domains\. The settings differ in observation structure, agent role, and decision mechanism, while sharing the same underlying deliberative collaboration structure\. The underlying decision problem is formulated as a combinatorial optimization problem with finite decision spaces, enabling rule\-based numerical rewards\. The benchmark enables a comprehensive evaluation of LLM agents’ capabilities in deliberative collaboration, including information exchange through conversation, numerical calculations, and logical reasoning\. Although instantiated in concrete domains, the underlying problem structure, cooperative joint decision making with incomplete and asymmetric information, is shared by many real\-world applications\.

We conduct experiments with multiple state\-of\-the\-art LLMs on 180 tasks across a range of task settings and evaluation conditions\. In particular, we inspect the effects of the decentralized deliberation process, external tools, and model capacity on collaborative decision making\. Results reveal that current models may still have shortcomings in information exchange and aggregation, reasoning, and math problem solving\. However, while challenging various capabilities of the LLMs, the deliberation process also provides a chance for reflection and error correction for the agents and may improve the performance in specific circumstances\.

Our main contributions are summarized as follows:

- •We formalize deliberative collaboration as a collaborative joint decision\-making problem under partial observations, providing a unified abstraction for studying cooperative LLM agents under incomplete and asymmetric information\.
- •We introduce a scalable benchmark that instantiates this problem across multiple tightly related task settings, including menu design and task allocation, enabling controlled evaluation under various observation and decision structures with solver\-computable objectives\.
- •We conduct a systematic evaluation across multiple LLMs and task settings, establishing baseline results and diagnostic observations for current deliberative agent systems\.

This work is motivated by a class of real\-world collaboration scenarios, where multiple agents cooperate to solve a task\. The agents may have distinct observations and action spaces while sharing the same reward; therefore, to achieve a higher common reward, deliberation would be a natural requirement\. We conduct a study on a representative set of deliberative collaboration scenarios that can be formulated as a decision problem under partial observability\.

### 2\.1Problem Formulation

We define the partial observable multi\-agent joint decision problem as a tuple:\(s,O1,⋯,On,𝒟,R\)\(s,O\_\{1\},\\cdots,O\_\{n\},\\mathcal\{D\},R\), wheressis the ground truth state in the problem which cannot be observed by the agents;O1,⋯,OnO\_\{1\},\\cdots,O\_\{n\}are the partial observations of the agents, wherennis the number of agents;𝒟\\mathcal\{D\}is the decision space; andR:𝒮×𝒟↦ℝR:\\mathcal\{S\}\\times\\mathcal\{D\}\\mapsto\\mathbb\{R\}is the reward function, where𝒮\\mathcal\{S\}is the state space\. The goal of the agents is to select a decisiond∈𝒟d\\in\\mathcal\{D\}that maximizes the rewardR​\(s,d\)R\(s,d\), with respect to the unobserved ground truth statess\. To enable the agents to reach a joint decision, we associate the problem with a deliberation process, as described in a programming\-friendly manner through the pseudocode in[Algorithm˜1](https://arxiv.org/html/2607.06157#alg1), where agents are defined in a general manner anddecision\_madeis a customizable function that determines whether the agents reached an agreement\. The protocol is intentionally abstract to support customized features; for example, role asymmetry can be instantiated by restricting which agents are taken into account by thedecision\_made\(·\)function\.

Note that although the deliberation process unfolds over multiple turns, the utterances serve solely to exchange information, and the environment state is fixed, so the underlying task is a single\-stage joint decision problem with a terminal reward\. While this setting could be formulated in a Dec\-POMDP manner, our formulation underlines the task’s nature as a one\-shot collective decision problem and focuses more on the deliberation process as epistemic coordination instead of the dialogue as state transition\.

Algorithm 1Deliberation ProcessInput: Observations

O1,⋯,OnO\_\{1\},\\cdots,O\_\{n\}, Language Agents

π1,⋯,πn\\pi\_\{1\},\\cdots,\\pi\_\{n\}, Decision Space

𝒟\\mathcal\{D\}, Turn limitation

tmt\_\{m\}
for

i∈\{1,⋯,n\}i\\in\\\{1,\\cdots,n\\\}do

πi​\.init​\(Oi\)\\pi\_\{i\}\\text\{\.init\}\(O\_\{i\}\)

endfor

t←0t\\leftarrow 0

d​i​a​l​o​g←∅dialog\\leftarrow\\emptyset

whilet

≤\\leqtmt\_\{m\}do

for

i∈\{1,⋯,n\}i\\in\\\{1,\\cdots,n\\\}do

πi​\.listen​\(d​i​a​l​o​g\)\\pi\_\{i\}\\text\{\.listen\}\(dialog\)

utterance, current\_proposal←πi​\.talk​\(\)\\text\{utterance, current\\\_proposal\}\\leftarrow\\pi\_\{i\}\\text\{\.talk\}\(\)

d​i​a​l​o​g←d​i​a​l​o​g\.a​p​p​e​n​d​\(u​t​t​e​r​a​n​c​e\)dialog\\leftarrow dialog\.append\(utterance\)

ifdecision\_made\(

⋅\\cdot\)then

Returncurrent\_proposal

endif

endfor

t←t\+1t\\leftarrow t\+1

endwhile

ReturnNone

### 2\.2Benchmark Design

We instantiate the partial observable joint decision problem in a set of task settings that vary in observation structure, decision authority, and evaluation protocol\. Across all settings, agents must reach a single joint decision through deliberation based on partial observations, while sharing a common terminal reward\. The key differences lie in the form of observations, the structure of the decision space, and the evaluation protocol, as summarized in[Table˜1](https://arxiv.org/html/2607.06157#S2.T1)\.

#### 2\.2\.1Menu Design with Partitioned Numerical Observations

We start with a two\-agent menu design scenario, with a setting that instantiates the joint decision problem in a fully structured environment with numerical state representations and partitioned observations\. The ground truth state consists of the ingredient state and user preferences, both of which can be represented in the form of vectors; the agents observe disjoint subsets of this state, with the guarantee that their observations jointly cover the full information\. The decision space is a finite set of candidate menus\. Given a dish list, for each dish, the agents decide whether to include it or not in the menu, and the reward is deterministically computed based on feasibility and user preference scores\. Given access to the full state, the optimal decision can be obtained by solving a corresponding integer programming problem, which serves as an upper bound\. As a result, this setting primarily evaluates agents’ ability to exchange essential information through deliberation and to perform accurate planning once sufficient information has been aligned\. More details are elaborated in Appendix[A\.1](https://arxiv.org/html/2607.06157#A1.SS1)\.

Table 1:Comparison of task settings instantiated in the benchmark\. Hybrid observations include both numerical information and natural language descriptions\.
#### 2\.2\.2Menu Design with Partial Semantic Observations

Considering that real\-world observations may be textual and semantic instead of numerical, we relax the assumption that observations are purely numerical and disjoint\. In this setting, agents receive partial natural language descriptions of the user preferences, where the natural language descriptions are sampled from the ground truth state and may be incomplete, overlapping, or correlated, thus different agents may describe the same latent fact from distinct perspectives\.

The decision space and reward function remain the same as in the structured menu design setting, but agents must now reason under uncertainty induced by language\-mediated observations; therefore, even an oracle baseline that has access to all natural language observations may not achieve the full reward\. This setting emphasizes natural language level reasoning and communication, interpretation of ambiguous information, and deliberation under partial observability, while preserving a well\-defined objective for evaluation\.

#### 2\.2\.3Task Allocation with Asymmetric Roles and Resources

We extend the joint decision framework to a multi\-agent task allocation scenario characterized by asymmetric roles and heterogeneous resource constraints\. Compared to the menu design domain, resources in this domain are divided into private resources \(exclusive to individual agents\) and public resources \(shared across the team\), together with agent\-specific efficiency for tasks\. The partial observability is also associated with this feature, where each agent can only observe its own private resources and a part of the public resources, requiring communication through deliberation to reconstruct global feasibility constraints\. The decision space consists of assigning tasks to agents under mixed private and public resource constraints, where each task can be assigned to at most one agent\. Within the resource constraints, the reward is calculated according to the base value of each task and the agents’ efficiency in performing the task\. Akin to the menu design domain, if the ground truth state is known, the task allocation problem can also be formulated as a combinatorial optimization problem and has a computable upper bound of reward calculated by the solver\.

In contrast to the symmetric peer interaction in menu design, this setting introduces explicit role asymmetry\. One agent is designated as the leader agent, while others are worker agents\. All agents can communicate to exchange information and make a proposal, but only the leader agent can make the final decision and terminate deliberation\. This setting differs fundamentally from menu design in both decision structure and authority distribution, and serves to evaluate agents’ ability to perform hierarchical coordination under partial observability\.

#### 2\.2\.4Metrics

All domains in our benchmark are associated with a well\-defined structured decision space and objective reward, thus we can evaluate agents using normalized reward \(NR=rewardmaximum reward×100=\\frac\{\\text\{reward\}\}\{\\text\{maximum reward\}\}\\times 100\), defined as the achieved reward divided by the maximum possible reward under the ground truth state\. Invalid decisions that violate feasibility constraints receive zero reward for NR\. Therefore, we also report the valid ratio \(VR\), defined as the proportion of valid decisions,*i\.e\.*, satisfying all feasibility constraints\.

To provide a more fine\-grained assessment for diagnosis, we provide additional metrics in[Section˜B\.1](https://arxiv.org/html/2607.06157#A2.SS1)\. Together, these metrics capture both optimality and constraint adherence under partial observability\.

#### 2\.2\.5External Tools

To disentangle the challenge of mathematical calculation from the deliberation process, we provide two optional symbolic tools that agents can invoke during the planning phase\. The Solver formulates the decision problem as an integer linear programming instance, deriving the utility\-maximizing solution based solely on the agent’s current estimated belief of the global state\. Complementing this, the Calculator serves as a fine\-grained feasibility oracle that verifies constraints ranging from simple ingredient availability to hierarchical private\-public resource limits, while returning diagnostic feedback on specific resource deficiencies for invalid proposals\. This framework adapts to the specific constraint structures of each domain: menu design is modeled as subset selection against a single shared inventory, whereas task allocation enforces a hierarchical system involving both local private capacities and global public budgets\.

### 2\.3Benchmark Implementation

We implement the benchmark using a database\-driven generation pipeline to ensure scalability, diversity, and reproducibility across task settings\. The benchmark consists of two databases: a menu design database formenu\-numericandmenu\-semantic, and a teamwork database fortask allocation\. Following prior worksZhou et al\. \([2024](https://arxiv.org/html/2607.06157#bib.bib49)\); Wang et al\. \([2024c](https://arxiv.org/html/2607.06157#bib.bib36),[a](https://arxiv.org/html/2607.06157#bib.bib34)\), the databases are initially generated using LLMs and subsequently reviewed and refined by the authors to ensure consistency and commonsense validity\. Then we sample agent personas, states, and observations from the database to generate task instances for each domain\. Further details are in[Appendix˜A](https://arxiv.org/html/2607.06157#A1)\.

The menu design database contains 51 ingredients, 78 dishes spanning 12 cuisines, and 10 character profiles representing guests\. Each character is defined by a set of attributes that may influence food preferences, including demographic information, dietary constraints, cultural background, health goals, and dining style\. Each menu design task instance is generated by sampling an ingredient inventory, a subset of candidate dishes with corresponding recipes, and two guest characters\. This procedure applies to bothmenu\-numericandmenu\-semantic, which only differ in the observation sampling\. In total, we generate 60 instances for each menu design setting\.

For task allocation, we construct the database across three sub\-domains: research project collaboration, cafeteria operations, and camping in the wild\. The database contains a total of 42 task templates, 12 private resource types, 12 public resource types, and 18 agent personas\. Each task allocation instance samples 10 tasks from the domain task pool and three agents \(one as the leader and two as workers\) from the persona pool\. Resources and task values are sampled from persona\-specific distributions\. We generate 20 instances per sub\-domain, for a total of 60\.

## 3Deliberative Agent

![Refer to caption](https://arxiv.org/html/2607.06157v1/x2.png)Figure 2:Illustration of our agent framework for solving the deliberative joint decision\-making tasks\. Green blocks indicate inputs, blue blocks represent agent modules, and orange blocks show variables modified by the agent\.To construct LLM agents for deliberative collaboration and instantiate the evaluation, we introduce an agent framework as a fixed reference scaffold, as shown in[Figure˜2](https://arxiv.org/html/2607.06157#S3.F2), which includes modules of observation, planning, decision, and conversation\. As a preliminary approach, we implement task\-specific agents to solve the deliberative joint decision problems\. Agents across all domains share the same general architecture, while the prompts and implementation are slightly different to fit each task setting\. Specifically, theobservation moduletakes the initial observation and the dialogue as inputs to estimate the ground truth state, covering domain\-specific factors such as ingredients and dish values, or private resource distribution and agent efficiencies\. After that, theplanning moduleproposes a candidate menu or task allocation scheme based on the estimated values, along with supporting reasons\. If external tools are available, the module will get the result returned by the tools as part of the input\. Then thedecision moduledecides whether to accept the current proposal based on all information up to now\. If the agent declines the current proposal, theconversation modulewill be called to generate a message to talk to the partner\. For the task allocation domain, the leader agent also decides whether to terminate the process and submit the final decision\. General task information and instructions are used throughout the whole procedure\. In each module, the LLM is prompted to have a thinking process in the output content to implement a chain of thoughtWei et al\. \([2022](https://arxiv.org/html/2607.06157#bib.bib38)\)\.

Table 2:Overall results across domains\. We report normalized reward \(mean±\\pmstandard error\) and valid ratio \(%\)\.
## 4Experiments

### 4\.1Experimental Setup

Since our benchmark is purely cooperative, we evaluate the language agents in a self\-play manner\. We select a combination of representative cutting\-edge models and affordable models, including GPT\-5\.1111gpt\-5\.1\-2025\-11\-13, DeepSeek\-V3\.2Liu et al\. \([2025a](https://arxiv.org/html/2607.06157#bib.bib20)\), GLM\-4\.7Zhipu AI \([2025](https://arxiv.org/html/2607.06157#bib.bib47)\), GPT\-4\.1\-mini222gpt\-4\.1\-mini\-2025\-04\-14, Qwen3\-Next\-80B\-A3B \(abbreviated asQwen3\-Next\-80B\), Qwen3\-32B, and Qwen3\-30B\-A3B \(abbreviated asQwen3\-30B\),Yang et al\. \([2025a](https://arxiv.org/html/2607.06157#bib.bib42)\)\. For a fair comparison in reasoning effort and token usage, we disable the internal thinking for all models\.

To balance conversation diversity and result stability, we set all LLM temperatures to 0\.3\. The deliberation process lasts for at most 6 rounds\. For a comprehensive evaluation, we conduct experiments under both conditions with and without tools\. Additionally, to study the challenges posed by the decentralized system model, we introduce acentralized baselinein which a centralized agent receives all perfectly merged observations and makes a single\-round decision\.

### 4\.2Main Results

We present primary experiment results including NR and VR across all three domains in[Table˜2](https://arxiv.org/html/2607.06157#S3.T2)\. Cutting\-edge models, including GPT\-5\.1, DeepSeek\-V3\.2, and GLM\-4\.7, achieve significantly higher rewards than smaller models such as GPT\-4\.1\-mini and Qwen3\-30B\. The large models achieve normalized rewards over 90 in themenu\-numericdomain, indicating that deliberative collaboration in simple settings can be adequately solved by the state\-of\-the\-art LLMs, though it may still be challenging for small models\. However, when the task gets complex, such as in thetask allocationdomain, there is still room for improvement even for large models\.

Comparing across domains, all models tend to achieve the highest rewards inmenu\-numeric, lower inmenu\-semanticandtask allocation\. This is in line with expectations since themenu\-semanticintroduces uncertainty and reasoning hardness into the task, andtask allocationhas a much more complex math problem; both are more difficult thanmenu\-numeric\.

### 4\.3Centralization vs\. Deliberation

Table 3:The performance of the oracle baseline in the task allocation domain\.ΔO\-Cent\.\\Delta\_\{\\text\{O\-Cent\.\}\}andΔO\-Delib\.\\Delta\_\{\\text\{O\-Delib\.\}\}denote the NR increase of the oracle baseline compared to the centralized baseline and standard deliberative collaboration setting, respectively\.Table 4:Additional indicators of tool use in selected scenarios\.*Tool NR*denotes the NR of the solution provided by the tool in the last round\.*Actual NR*denotes the NR of the model’s final decision\.*Comp\. \(%\)*denotes the compliance rate, i\.e\., the proportion of cases where the model’s final decision exactly matches the final tool output\.One may expect agents in the centralized baseline to perform better than in the standard decentralized setting, since the agent can directly get the perfectly merged observation without information loss or hallucination\. However, we find that the effect of deliberation is complex and subtle in practice\. Though agents get a significant gain in reward in most scenarios, in some specific cases, such as GLM\-4\.7 and GPT\-4\.1\-mini in thetask allocationdomain, the performance even reduces\.

The performance gain in the centralized baseline may partly come from more accurate state estimation\. Additional diagnostics in[Appendix˜B](https://arxiv.org/html/2607.06157#A2)show that state\-estimation errors, measured by hallucination rate or NMAE, are associated with performance gaps in some settings\.

As for the unusual performance drop, after qualitative inspection of the relevant cases, we empirically find that the deliberation process provides a chance for reflection and error correction\. Therefore, it is possible for the centralized baseline to achieve a lower reward since it does not have the deliberation process\. An example and case study is provided in[Section˜C\.1](https://arxiv.org/html/2607.06157#A3.SS1)\.

To further diagnose the roles of information access and multi\-round discussion in the deliberation process, we introduce an*oracle*baseline that combines the deliberation process with perfectly merged full observations\. We conduct an additional experiment with the most representative models on thetask allocationdomain, where the performance drop happened, and present the results in[Table˜3](https://arxiv.org/html/2607.06157#S4.T3)\. As shown by the results, the oracle baseline exhibits further performance gain in all cases\. For GLM\-4\.7 and GPT\-4\.1\-mini, the oracle baseline significantly outperforms the centralized baseline, whereas the performance gap is not so large for GPT\-5\.1 and DeepSeek\-V3\.2\. Such evidence supports the beneficial effects of the deliberation process for LLM agents in making decisions and explains the unexpectedly low performance of GLM\-4\.7 and GPT\-4\.1\-mini in the centralized baseline\.

### 4\.4Effect of External Tools

In most scenarios, external mathematical tools substantially improve the performance of the agent, especially for DeepSeek\-V3\.2, GPT\-4\.1\-mini, and Qwen3\-32B\. This adheres to the current consensus that LLMs may not be directly good at calculation, and mathematical tools can help\. However, the effect of external tools seems to be model\-specific, and there are even some counterintuitive results\. For example, GPT\-5\.1, GLM\-4\.7, and Qwen3\-Next\-80B do not gain much more reward with the help of tools; the reward even reduces in some cases\.

To explain this phenomenon, we manually inspect the counterintuitive cases and conduct an additional process analysis for scenarios where counterintuitive cases happen\. We present additional indicators in representative scenarios in[Table˜4](https://arxiv.org/html/2607.06157#S4.T4)\. Although the mathematical tools provide a good solution, the agents tend not to adopt the solution but try to do the calculation themselves and propose another solution instead\. GPT\-5\.1 even shows a compliance rate of 0 in the task allocation domain\. However, due to limited computational capability, the LLMs may not obtain the optimal solution\. We provide an example in[Section˜C\.2](https://arxiv.org/html/2607.06157#A3.SS2)as a case study\. Another factor that undermines the usefulness of external tools is the accumulated error in state estimation\. Since the solver calculates the best solution based on the input state, for some less intelligent models, if the state estimation from the partial observation is inaccurate, the solver will produce a wrong solution as well\.

## 5Related Work

### 5\.1Language Agents and Multi\-Agent Systems

With advances in LLM techniques and their achievement of near\-human performance in many aspects, a large body of prior work has shown the feasibility of building LLM\-based autonomous agents to solve complicated tasks, known as language agentsWang et al\. \([2024d](https://arxiv.org/html/2607.06157#bib.bib37)\); Liu et al\. \([2024b](https://arxiv.org/html/2607.06157#bib.bib23)\); Zhu et al\. \([2025a](https://arxiv.org/html/2607.06157#bib.bib50)\); Ferrag et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib9)\)\.

Language agents can also form multi\-agent systemsTran et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib30)\), which can not only be used as a technology to improve the performance of LLM\-based systemsChen et al\. \([2024](https://arxiv.org/html/2607.06157#bib.bib3)\); Qian et al\. \([2024](https://arxiv.org/html/2607.06157#bib.bib27)\); Wang et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib33)\); He et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib12)\), but also can apply to intrinsically multi\-agent, such as social simulations and social intelligenceWang et al\. \([2024a](https://arxiv.org/html/2607.06157#bib.bib34)\); Zhou et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib48)\); Mou et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib26)\); Yan et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib41)\); Zhang et al\. \([2024b](https://arxiv.org/html/2607.06157#bib.bib45)\)and multi\-robot applicationsLiu et al\. \([2024c](https://arxiv.org/html/2607.06157#bib.bib24)\); Kannan et al\. \([2024](https://arxiv.org/html/2607.06157#bib.bib14)\); Liu et al\. \([2025b](https://arxiv.org/html/2607.06157#bib.bib22)\); Yang et al\. \([2025b](https://arxiv.org/html/2607.06157#bib.bib43)\)\. Previous work also exhibited the necessity of proactive communication in cooperationZhang et al\. \([2024a](https://arxiv.org/html/2607.06157#bib.bib44)\); Sun et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib29)\)and studied the feasibility of building LLM agents for collaboration under partial observation as separate agentsLi et al\. \([2023](https://arxiv.org/html/2607.06157#bib.bib17)\)\. In contrast, our work further studies communication under partial and asymmetric observations for deliberative collaboration and joint decision making\.

### 5\.2Goal\-Oriented Dialogue

Our work generally falls in the goal\-oriented dialogue system category, where agents converse and act for specific goals, and it has broad application scenarios such as healthcareValizadeh and Parde \([2022](https://arxiv.org/html/2607.06157#bib.bib32)\)and robotics applicationsMandi et al\. \([2024](https://arxiv.org/html/2607.06157#bib.bib25)\); Wang et al\. \([2024b](https://arxiv.org/html/2607.06157#bib.bib35)\); Li et al\. \([2025b](https://arxiv.org/html/2607.06157#bib.bib18)\); Zhang et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib46)\)\. Compared to classical task\-oriented dialogues where agents are designed to serve specific purposesXu et al\. \([2024](https://arxiv.org/html/2607.06157#bib.bib40)\); Ulmer et al\. \([2024](https://arxiv.org/html/2607.06157#bib.bib31)\); Rim et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib28)\); Du et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib7)\)or that employ a multi\-agent system for a specific applicationHan et al\. \([2024](https://arxiv.org/html/2607.06157#bib.bib11)\), we focus more on the application scenarios that necessitate deliberation between agents to align information and reach an agreement on the final decision\. Compared to previous works that study negotiation or deliberation agents, where agents need to reach an agreement on an issue, and each agent may have its own goalXia et al\. \([2024](https://arxiv.org/html/2607.06157#bib.bib39)\); Abdelnabi et al\. \([2024](https://arxiv.org/html/2607.06157#bib.bib1)\); Karanam et al\. \([2024](https://arxiv.org/html/2607.06157#bib.bib15)\); Li et al\. \([2025a](https://arxiv.org/html/2607.06157#bib.bib16)\); Hu et al\. \([2025](https://arxiv.org/html/2607.06157#bib.bib13)\), our work focuses on a pure cooperative scenario where all the agents share the same reward and work towards the same goal\. We regardLin et al\. \([2024](https://arxiv.org/html/2607.06157#bib.bib19)\)as the closest related work, which also studies collaborative decision\-making scenarios\. While they focus on human\-AI collaboration with agents as assistants, our work studies autonomous agent teams with equal status cooperating on human\-assigned tasks\.

## 6Conclusion

In this paper, we formalize deliberative collaboration as a partially observable joint decision\-making problem and build a benchmark with a set of tightly related task instantiations with different observation structures and coordination requirements to study how shared decision mechanisms behave under varying assumptions about information availability, tool access, and agent roles\.

Through a comprehensive evaluation, we show that current LLM agents remain insufficient for reliable deliberative collaboration\. LLM agents may suffer from insufficient information exchange and aggregation, inadequate reasoning capabilities, and limited mathematical capabilities\. Yet deliberation can also provide opportunities for reflection and error correction\.

Overall, our benchmark serves not only as an evaluation framework but also as a diagnostic tool for analyzing where and why deliberative collaboration fails\. We hope this work encourages future research on more robust coordination strategies, calibrated tool use, and principled information aggregation for multi\-agent LLM systems\.

## Limitations

Our benchmark evaluates LLM\-agent systems under a fixed reference scaffold\. Since the tasks are complex and require multiple capabilities, performance may be sensitive to agent architecture, prompt design, and tool interfaces\. Exhaustively evaluating alternative scaffolds is beyond the scope of this work; therefore, our results should be interpreted as representative performance under the evaluated scaffold, rather than a complete characterization of each LLM\.

Another limitation is the scale of the evaluation\. Due to computational and financial constraints, we evaluate a representative set of models rather than all accessible LLMs, and we do not include extensive sensitivity analyses over partner composition, decoding settings, deliberation budgets, or human baselines\. These omissions limit the scope of the empirical conclusions\. Future work can instantiate alternative scaffolds, cross\-play protocols, communication constraints, or human calibration using the benchmark generation and evaluation code\.

## References

- Abdelnabi et al\. \(2024\)Sahar Abdelnabi, Amr Gomaa, Sarath Sivaprasad, Lea Schönherr, and Mario Fritz\. 2024\.Cooperation, competition, and maliciousness: Llm\-stakeholders interactive negotiation\.In*The Thirty\-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track*\.
- Bianchi et al\. \(2024\)Federico Bianchi, Patrick John Chia, Mert Yuksekgonul, Jacopo Tagliabue, Dan Jurafsky, and James Zou\. 2024\.How well can llms negotiate? negotiationarena platform and analysis\.In*Forty\-first International Conference on Machine Learning*\.
- Chen et al\. \(2024\)Weize Chen, Ziming You, Ran Li, Yitong Guan, Chen Qian, Chenyang Zhao, Cheng Yang, Ruobing Xie, Zhiyuan Liu, and Maosong Sun\. 2024\.Internet of agents: Weaving a web of heterogeneous agents for collaborative intelligence\.*arXiv preprint arXiv:2407\.07061*\.
- Chen et al\. \(2025\)Weize Chen, Jiarui Yuan, Chen Qian, Cheng Yang, Zhiyuan Liu, and Maosong Sun\. 2025\.Optima: Optimizing effectiveness and efficiency for llm\-based multi\-agent system\.In*Findings of the Association for Computational Linguistics: ACL 2025*, pages 11534–11557\.
- Choi et al\. \(2025\)Hyeong Kyu Choi, Xiaojin Zhu, and Sharon Li\. 2025\.Debate or vote: Which yields better decisions in multi\-agent large language models?*arXiv preprint arXiv:2508\.17536*\.
- Dolant and Kumar \(2025\)Antoine Dolant and Praveen Kumar\. 2025\.Agentic llm framework for adaptive decision discourse\.*arXiv preprint arXiv:2502\.10978*\.
- Du et al\. \(2025\)Zhuoyun Du, Chen Qian, Wei Liu, Zihao Xie, Yifei Wang, Rennai Qiu, Yufan Dang, Weize Chen, Cheng Yang, Ye Tian, and 1 others\. 2025\.Multi\-agent collaboration via cross\-team orchestration\.In*Findings of the Association for Computational Linguistics: ACL 2025*, pages 10386–10406\.
- Eo et al\. \(2025\)Sugyeong Eo, Hyeonseok Moon, Evelyn Hayoon Zi, Chanjun Park, and Heuiseok Lim\. 2025\.Debate only when necessary: Adaptive multiagent collaboration for efficient llm reasoning\.*arXiv preprint arXiv:2504\.05047*\.
- Ferrag et al\. \(2025\)Mohamed Amine Ferrag, Norbert Tihanyi, and Merouane Debbah\. 2025\.From llm reasoning to autonomous ai agents: A comprehensive review\.*arXiv preprint arXiv:2504\.19678*\.
- Guo et al\. \(2024\)Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang\. 2024\.Large language model based multi\-agents: A survey of progress and challenges\.*arXiv preprint arXiv:2402\.01680*\.
- Han et al\. \(2024\)Senyu Han, Lu Chen, Li\-Min Lin, Zhengshan Xu, and Kai Yu\. 2024\.Ibsen: Director\-actor agent collaboration for controllable and interactive drama script generation\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 1607–1619\.
- He et al\. \(2025\)Junda He, Christoph Treude, and David Lo\. 2025\.Llm\-based multi\-agent systems for software engineering: Literature review, vision, and the road ahead\.*ACM Transactions on Software Engineering and Methodology*, 34\(5\):1–30\.
- Hu et al\. \(2025\)Zhe Hu, Hou Pong Chan, Jing Li, and Yu Yin\. 2025\.Debate\-to\-write: A persona\-driven multi\-agent framework for diverse argument generation\.In*Proceedings of the 31st International Conference on Computational Linguistics*, pages 4689–4703\.
- Kannan et al\. \(2024\)Shyam Sundar Kannan, Vishnunandan LN Venkatesh, and Byung\-Cheol Min\. 2024\.Smart\-llm: Smart multi\-agent robot task planning using large language models\.In*2024 IEEE/RSJ International Conference on Intelligent Robots and Systems \(IROS\)*, pages 12140–12147\. IEEE\.
- Karanam et al\. \(2024\)Arjun Karanam, Farnaz Jahanbakhsh, and Sanmi Koyejo\. 2024\.Towards deliberating agents: Evaluating the ability of large language models to deliberate\.In*NeurIPS 2024 Workshop on Behavioral Machine Learning*\.
- Li et al\. \(2025a\)Haoran Li, Ziyi Su, Yun Xue, Zhiliang Tian, Yiping Song, and Minlie Huang\. 2025a\.Advancing collaborative debates with role differentiation through multi\-agent reinforcement learning\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 22655–22666\.
- Li et al\. \(2023\)Huao Li, Yu Chong, Simon Stepputtis, Joseph P Campbell, Dana Hughes, Charles Lewis, and Katia Sycara\. 2023\.Theory of mind for multi\-agent collaboration via large language models\.In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pages 180–192\.
- Li et al\. \(2025b\)Zhaoxing Li, Yue Wang, Wenbo Wu, Yanran Xu, and Sebastian Stein\. 2025b\.Hmcf: A human\-in\-the\-loop multi\-robot collaboration framework based on large language models\.In*International Conference on Principles and Practice of Multi\-Agent Systems*, pages 150–167\. Springer\.
- Lin et al\. \(2024\)Jessy Lin, Nicholas Tomlin, Jacob Andreas, and Jason Eisner\. 2024\.Decision\-oriented dialogue for human\-ai collaboration\.*Transactions of the Association for Computational Linguistics*, 12:892–911\.
- Liu et al\. \(2025a\)Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, and 1 others\. 2025a\.Deepseek\-v3\. 2: Pushing the frontier of open large language models\.*arXiv preprint arXiv:2512\.02556*\.
- Liu et al\. \(2024a\)Jie Liu, Pan Zhou, Yingjun Du, Ah\-Hwee Tan, Cees GM Snoek, Jan\-Jakob Sonke, and Efstratios Gavves\. 2024a\.Capo: Cooperative plan optimization for efficient embodied multi\-agent cooperation\.*arXiv preprint arXiv:2411\.04679*\.
- Liu et al\. \(2025b\)Kehui Liu, Zixin Tang, Dong Wang, Zhigang Wang, Xuelong Li, and Bin Zhao\. 2025b\.Coherent: Collaboration of heterogeneous multi\-robot system with large language models\.In*2025 IEEE International Conference on Robotics and Automation \(ICRA\)*, pages 10208–10214\. IEEE\.
- Liu et al\. \(2024b\)Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, and 1 others\. 2024b\.Agentbench: Evaluating llms as agents\.In*The Twelfth International Conference on Learning Representations*\.
- Liu et al\. \(2024c\)Xinzhu Liu, Peiyan Li, Wenju Yang, Di Guo, and Huaping Liu\. 2024c\.[Leveraging large language model for heterogeneous ad hoc teamwork collaboration](https://doi.org/10.15607/RSS.2024.XX.033)\.In*Proceedings of Robotics: Science and Systems*, Delft, Netherlands\.
- Mandi et al\. \(2024\)Zhao Mandi, Shreeya Jain, and Shuran Song\. 2024\.Roco: Dialectic multi\-robot collaboration with large language models\.In*2024 IEEE International Conference on Robotics and Automation \(ICRA\)*, pages 286–299\. IEEE\.
- Mou et al\. \(2025\)Xinyi Mou, Jingcong Liang, Jiayu Lin, Xinnong Zhang, Xiawei Liu, Shiyue Yang, Rong Ye, Lei Chen, Haoyu Kuang, Xuan\-Jing Huang, and 1 others\. 2025\.Agentsense: Benchmarking social intelligence of language agents through interactive scenarios\.In*Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\)*, pages 4975–5001\.
- Qian et al\. \(2024\)Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, and 1 others\. 2024\.Chatdev: Communicative agents for software development\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 15174–15186\.
- Rim et al\. \(2025\)Daniel Rim, Minsoo Cho, Changwoo Chun, and Jaegul Choo\. 2025\.To chat or task: a multi\-turn dialogue generation framework for task\-oriented dialogue systems\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 6: Industry Track\)*, pages 576–592\.
- Sun et al\. \(2025\)Nan Sun, Bo Mao, Yongchang Li, Di Guo, and Huaping Liu\. 2025\.Assistantx: An llm\-powered proactive assistant in collaborative human\-populated environments\.In*2025 IEEE/RSJ International Conference on Intelligent Robots and Systems \(IROS\)*, pages 3352–3359\. IEEE\.
- Tran et al\. \(2025\)Khanh\-Tung Tran, Dung Dao, Minh\-Duong Nguyen, Quoc\-Viet Pham, Barry O’Sullivan, and Hoang D Nguyen\. 2025\.Multi\-agent collaboration mechanisms: A survey of llms\.*arXiv preprint arXiv:2501\.06322*\.
- Ulmer et al\. \(2024\)Dennis Ulmer, Elman Mansimov, Kaixiang Lin, Justin Sun, Xibin Gao, and Yi Zhang\. 2024\.Bootstrapping llm\-based task\-oriented dialogue agents via self\-talk\.*arXiv preprint arXiv:2401\.05033*\.
- Valizadeh and Parde \(2022\)Mina Valizadeh and Natalie Parde\. 2022\.The ai doctor is in: A survey of task\-oriented dialogue systems for healthcare applications\.In*Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 6638–6660\.
- Wang et al\. \(2025\)Bing Wang, Changyu Ren, Jian Yang, Xinnian Liang, Jiaqi Bai, Linzheng Chai, Zhao Yan, Qian\-Wen Zhang, Di Yin, Xing Sun, and 1 others\. 2025\.Mac\-sql: A multi\-agent collaborative framework for text\-to\-sql\.In*Proceedings of the 31st International Conference on Computational Linguistics*, pages 540–557\.
- Wang et al\. \(2024a\)Chenxu Wang, Bin Dai, Huaping Liu, and Baoyuan Wang\. 2024a\.Towards objectively benchmarking social intelligence for language agents at action level\.In*Findings of the Association for Computational Linguistics*, pages 8885–8897\.
- Wang et al\. \(2024b\)Chenxu Wang, Boyuan Du, Jiaxin Xu, and Huaping Liu\. 2024b\.[Demonstrating HumanTHOR: A Simulation Platform and Benchmark for Human\-Robot Collaboration in a Shared Workspace](https://doi.org/10.15607/RSS.2024.XX.029)\.In*Proceedings of Robotics: Science and Systems*, Delft, Netherlands\.
- Wang et al\. \(2024c\)Chenxu Wang, Xinghang Li, Dunzheng Wang, Huaping Liu, and 1 others\. 2024c\.Dynamic scene generation for embodied navigation benchmark\.In*RSS 2024 Workshop: Data Generation for Robotics*\.
- Wang et al\. \(2024d\)Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, and 1 others\. 2024d\.A survey on large language model based autonomous agents\.*Frontiers of Computer Science*, 18\(6\):186345\.
- Wei et al\. \(2022\)Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed H Chi, Quoc V Le, Denny Zhou, and 1 others\. 2022\.Chain\-of\-thought prompting elicits reasoning in large language models\.In*Advances in Neural Information Processing Systems*\.
- Xia et al\. \(2024\)Tian Xia, Zhiwei He, Tong Ren, Yibo Miao, Zhuosheng Zhang, Yang Yang, and Rui Wang\. 2024\.[Measuring bargaining abilities of LLMs: A benchmark and a buyer\-enhancement method](https://doi.org/10.18653/v1/2024.findings-acl.213)\.In*Findings of the Association for Computational Linguistics: ACL 2024*, pages 3579–3602\.
- Xu et al\. \(2024\)Heng\-Da Xu, Xian\-Ling Mao, Puhai Yang, Fanshu Sun, and He\-Yan Huang\. 2024\.Rethinking task\-oriented dialogue systems: From complex modularity to zero\-shot autonomous agent\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 2748–2763\.
- Yan et al\. \(2025\)Bingyu Yan, Zhibo Zhou, Litian Zhang, Lian Zhang, Ziyi Zhou, Dezhuang Miao, Zhoujun Li, Chaozhuo Li, and Xiaoming Zhang\. 2025\.Beyond self\-talk: A communication\-centric survey of llm\-based multi\-agent systems\.*arXiv preprint arXiv:2502\.14321*\.
- Yang et al\. \(2025a\)An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others\. 2025a\.Qwen3 technical report\.*arXiv preprint arXiv:2505\.09388*\.
- Yang et al\. \(2025b\)Rui Yang, Hanyang Chen, Junyu Zhang, Mark Zhao, Cheng Qian, Kangrui Wang, Qineng Wang, Teja Venkat Koripella, Marziyeh Movahedi, Manling Li, and 1 others\. 2025b\.Embodiedbench: Comprehensive benchmarking multi\-modal large language models for vision\-driven embodied agents\.*arXiv preprint arXiv:2502\.09560*\.
- Zhang et al\. \(2024a\)Ceyao Zhang, Kaijie Yang, Siyi Hu, Zihao Wang, Guanghe Li, Yihang Sun, Cheng Zhang, Zhaowei Zhang, Anji Liu, Song\-Chun Zhu, and 1 others\. 2024a\.Proagent: building proactive cooperative agents with large language models\.In*Proceedings of the AAAI Conference on Artificial Intelligence*, pages 17591–17599\.
- Zhang et al\. \(2024b\)Jintian Zhang, Xin Xu, Ningyu Zhang, Ruibo Liu, Bryan Hooi, and Shumin Deng\. 2024b\.Exploring collaboration mechanisms for llm agents: A social psychology view\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 14544–14607\.
- Zhang et al\. \(2025\)Yang Zhang, Shixin Yang, Chenjia Bai, Fei Wu, Xiu Li, Zhen Wang, and Xuelong Li\. 2025\.Towards efficient llm grounding for embodied multi\-agent collaboration\.In*Findings of the Association for Computational Linguistics: ACL 2025*, pages 1663–1699\.
- Zhipu AI \(2025\)Zhipu AI\. 2025\.[Glm\-4\.7: Advancing the coding capability](https://z.ai/blog/glm-4.7)\.
- Zhou et al\. \(2025\)Jinfeng Zhou, Yuxuan Chen, Yihan Shi, Xuanming Zhang, Leqi Lei, Yi Feng, Zexuan Xiong, Miao Yan, Xunzhi Wang, Yaru Cao, and 1 others\. 2025\.Socialeval: Evaluating social intelligence of large language models\.*arXiv preprint arXiv:2506\.00900*\.
- Zhou et al\. \(2024\)Xuhui Zhou, Hao Zhu, Leena Mathur, Ruohong Zhang, Haofei Yu, Zhengyang Qi, Louis\-Philippe Morency, Yonatan Bisk, Daniel Fried, Graham Neubig, and 1 others\. 2024\.Sotopia: Interactive evaluation for social intelligence in language agents\.In*The Twelfth International Conference on Learning Representations*\.
- Zhu et al\. \(2025a\)Kunlun Zhu, Hongyi Du, Zhaochen Hong, Xiaocheng Yang, Shuyi Guo, Daisy Zhe Wang, Zhenhailong Wang, Cheng Qian, Robert Tang, Heng Ji, and 1 others\. 2025a\.Multiagentbench: Evaluating the collaboration and competition of llm agents\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 8580–8622\.
- Zhu et al\. \(2025b\)Shenzhe Zhu, Jiao Sun, Yi Nian, Tobin South, Alex Pentland, and Jiaxin Pei\. 2025b\.The automated but risky game: Modeling agent\-to\-agent negotiations and transactions in consumer markets\.In*ICML 2025 Workshop on Reliable and Responsible Foundation Models*\.

## Appendix ABenchmark Details

### A\.1Menu Design

#### A\.1\.1Game Formulation

We implement the menu design game in a basic problem setting: the ingredient stateSIS^\{I\}is a vector whereSiIS^\{I\}\_\{i\}is the amount of theii\-th ingredient; the cookbook offers a recipe matrixAA, whereAi​jA\_\{ij\}denotes the amount of theii\-th ingredient needed for thejj\-th dish; the user preferencesSUS^\{U\}are simplified to a scoring matrix, whereSk​jUS^\{U\}\_\{kj\}denotes the score that thekk\-th guest gives to thejj\-th dish; and the agents directly access a part ofSIS^\{I\}andSUS^\{U\}as their observation, with the guarantee thatO1∪O2=SO\_\{1\}\\cup O\_\{2\}=SandO1∩O2=∅O\_\{1\}\\cap O\_\{2\}=\\emptyset, denoted aspartitioned information\.

By defining a scoring vectorccascj=∑knguestSk​jUc\_\{j\}=\\sum\_\{k\}^\{n\_\{\\text\{guest\}\}\}S^\{U\}\_\{kj\}\(wherenguestn\_\{\\text\{guest\}\}is the number of guests\), the final reward is calculated with:

R​\(d\)=IsValid​\(d,SI\)⋅∑j∈dcj,R\(d\)=\\text\{IsValid\}\(d,S^\{I\}\)\\cdot\\sum\_\{j\\in d\}c\_\{j\},whereIsValidis a function indicating whether the menu is valid, specifically, the consumed ingredients should be less than or equal to the available ingredients\. The reward function also implies that a dish can only appear once in the final menu\.

Additionally, if the agents have access to the full state, the task can be formulated as an integer programming problem:

maxx∈\{0,1\}ndish⁡cT​x\\displaystyle\\max\_\{x\\in\\\{0,1\\\}^\{n\_\{\\text\{dish\}\}\}\}c^\{T\}xs\.t\.A​x≤SI\\displaystyle s\.t\.Ax\\leq S^\{I\}Herendishn\_\{\\text\{dish\}\}is the number of dishes\. This implies that the problem has a straightforward solution in which the two agents exchange their observations to get enough information for planning and then solve the integer programming problem\. Therefore, the challenge principally lies in choosing the most important information to exchange, expressing and understanding essential information in natural language dialog, and solving the programming problem with intrinsic planning capabilities or utilizing external tools\. In this work, we use thepulppackage as the external solver for the integer program\.

#### A\.1\.2Characters and Preferences

We define characters as described in the paper\. We present an example below:

"Name":"Alice",

"Gender":"Female",

"Nationality":"American",

"Age":30,

"Occupation":\{

"JobTitle":"SoftwareEngineer",

"Industry":"Technology",

"WorkSchedule":"9am\-5pm"

\},

"RecentStatus":\{

"CurrentFocus":"Projectdeadlineapproaching",

"RecentDietaryFocus":"Lookingforhealthy,comfortingmealstorecharge",

"StressLevel":"High"

\},

"DietaryPreferences":\{

"Vegetarian":false,

"Vegan":false,

"Gluten\-free":true,

"Dairy\-free":false,

"Low\-carb":false

\},

"FoodAllergies":\["Nuts","Dairy"\],

"CulturalPreferences":\{

"CuisineType":"Italian",

"SpiceLevelPreference":"Mild",

"TraditionalDishes":true

\},

"HealthandFitnessGoals":\{

"Calorie\-conscious":true,

"Protein\-rich":false,

"Low\-sodium":false,

"Low\-sugar":false,

"Weight\-lossfocus":false,

"Muscle\-buildingfocus":false

\},

"PersonalityandDiningStyle":\{

"AdventurousEater":false,

"ComfortFoodLover":true,

"IndulgentFoodPreferences":true,

"MinimalistDining":false,

"SocialEater":true,

"Snack\-friendly":true

\},

"EthicalandReligiousConsiderations":\{

"Halal":false,

"Kosher":false,

"Buddhist":false,

"EthicalEatingPreferences":"Organic"

\}

Based on the persona information, each character is associated with a set of preference values, which are their ratings for each dish\. The value of a dish to the character is defined asrd​\(ni\+0\.2∗ni1\.5\)r\_\{d\}\(n\_\{i\}\+0\.2\*n\_\{i\}^\{1\.5\}\), whererdr\_\{d\}is the rating for that dish andnin\_\{i\}is the total number of ingredients required for that dish\. We use this formula to encourage the agents to cooperate and choose more complex dishes\.

#### A\.1\.3Agent Observation

Across menu\-design settings, we provide observations on the ingredient inventory in JSON dictionary format\. To maintain a balance between agents, each agent will observe at least 40% and at most 60% of the ingredients\.

In themenu\-numericdomain, each agent observes the value of dishes to one guest\. In themenu\-semanticdomain, each agent observes partial persona information of both guests, according to theobserved probabilityof each field in the person information\. Theobserved probabilitiesare as follows:

"Agent1":\{

"Name":1\.0,

"Gender":1\.0,

"Nationality":1\.0,

"Age":1\.0,

"Occupation":0\.7,

"RecentStatus":0\.6,

"DietaryPreferences":0\.6,

"FoodAllergies":0\.7,

"CulturalPreferences":0\.6,

"HealthandFitnessGoals":0\.4,

"PersonalityandDiningStyle":0\.5,

"EthicalandReligiousConsiderations":0\.5

\},

"Agent2":\{

"Name":1\.0,

"Gender":1\.0,

"Nationality":1\.0,

"Age":1\.0,

"Occupation":0\.6,

"RecentStatus":0\.6,

"DietaryPreferences":0\.7,

"FoodAllergies":0\.8,

"CulturalPreferences":0\.6,

"HealthandFitnessGoals":0\.5,

"PersonalityandDiningStyle":0\.7,

"EthicalandReligiousConsiderations":0\.6

\}

#### A\.1\.4Task Selection

To avoid impractical tasks and ensure the tasks are appropriate to serve as benchmarks for LLM agents, we first randomly generate 10,000 tasks for bothmenu\-numericandmenu\-semantic, and then select the appropriate ones according to the following principles:

- •There must be at least33dishes in the solved best menu\.
- •There exist 8 kinds of ingredients with non\-zero amounts\.
- •Then we select the ones that utilize more ingredients, with a smaller ingredient list, and have more possible dishes and fewer dishes in the best menu\. We expect the selected tasks to be feasible yet moderately challenging to find the best menu\.

For all tasks, there must exist at least 5 possible dishes, the number of possible dishes must account for at least40%40\\%of the total dishes \(we sample a subset of dishes for each task\)\.

### A\.2Task Allocation with Asymmetric Roles and Resources

#### A\.2\.1Roles and Preferences

Distinct from the symmetric peer setting in menu design, we define asymmetric roles for agents in this task\. The team consists of three agents: two Workers \(Agent 0 and Agent 1\) who contribute information and proposals, and one Leader \(Agent 2\) who holds the exclusive authority to make the final decision to terminate the negotiation\.

Each agent is associated with a specific efficiency profile, representing their utility for completing specific tasks\. The preference structure is defined as follows:

\{

"AgentRoles":\{

"agent\_0":"Worker",

"agent\_1":"Worker",

"agent\_2":"Leader"

\},

"EfficiencyProfile\(overall\_preferences\)":\{

"agent\_0":\{

"task\_1":0\.8,

"task\_2":0\.6,

"task\_3":0\.9

\},

"agent\_1":\{

"task\_1":0\.7,

"task\_2":0\.8,

"task\_3":0\.5

\},

"agent\_2":\{

"task\_1":0\.9,

"task\_2":0\.5,

"task\_3":0\.7

\}

\}

\}

The team utility is calculated as the sum of efficiencies for the assigned tasks\. Formally, the reward function is defined asR​\(x\)=∑i=0N−1∑j=0M−1vi,j⋅xi,jR\(x\)=\\sum\_\{i=0\}^\{N\-1\}\\sum\_\{j=0\}^\{M\-1\}v\_\{i,j\}\\cdot x\_\{i,j\}, wherevi,jv\_\{i,j\}is the efficiency of agentiifor taskjj, andxi,j∈\{0,1\}x\_\{i,j\}\\in\\\{0,1\\\}indicates whether taskjjis assigned to agentii\.

#### A\.2\.2Agent Observation

In this setting, observations are structured around resource availability\. The ground truth stateSScomprises private resourcesSp​r​i​vS^\{priv\}\(exclusive to specific agents, e\.g\., Time, GPU\) and public resourcesSp​u​bS^\{pub\}\(shared across the team, e\.g\., Budget\)\.

The observation space is partially partitioned and additive\. Specifically:

1. 1\.Private Resources: Each agent fully observes their own private resources, which are hidden from others\.
2. 2\.Public Resources: Each agent observes only a fragment of the shared public resources\. The system guarantees that the sum of all agents’ observed public resources equals the ground truth total\.

An example of a single agent’s observation \(e\.g\., Agent 0\) is shown below:

\{

"current\_private\_resources":\{

"Time":10,//Fullyobserved,exclusivetoagent\_0

"GPU":2

\},

"current\_public\_resources":\{

"Budget":80//Partialfragment;totalmightbe200

\},

"efficiency\_dict":\{

"task\_1":0\.8,//Self\-efficiencyisknown

"task\_2":0\.6,

"task\_3":0\.9

\},

"task\_requirements":\{

"task\_1":\{

"Time":3,

"GPU":1,

"Budget":100

\},

"task\_2":\{

"Time":4,

"GPU":2,

"Budget":50

\}

\}

\}

Agents must communicate to aggregate the partial public observations into a global view to verify if a proposed allocation satisfies the global public resource constraints\.

#### A\.2\.3Task Selection

To avoid impractical tasks and ensure the scenarios provide appropriate challenges for hierarchical coordination, we first randomly generate 1,000 candidate instances for each domain by sampling tasks, agent personas, and resource capacities, and then select the appropriate ones according to the following principles:

- •Feasibility Constraint: There must be at least33tasks successfully assigned in the optimal solution \(nd​o​n​e≥3n\_\{done\}\\geq 3\)\. This filters out instances that are unsolvable or too trivial due to extreme resource shortages\.
- •Quality Prioritization: We score candidates using a priority function defined asP=wp⋅pd​o​n​e\+wu⋅Ur​e​s\+wr⋅Rm​a​xP=w\_\{p\}\\cdot p\_\{done\}\+w\_\{u\}\\cdot U\_\{res\}\+w\_\{r\}\\cdot R\_\{max\}, wherepd​o​n​ep\_\{done\}is the task completion ratio andUr​e​sU\_\{res\}is the average resource utilization\. We select instances that maximize these metrics to ensure the scenarios allow for high task throughput while maintaining tight resource constraints, requiring efficient planning rather than having overabundant resources\.

## Appendix BExperiments

This section presents additional experimental results and metric details\.

Table 5:Overall results across domains using normalized adjusted reward \(NAR, mean±\\pmstandard error\) and valid ratio \(%\)\.### B\.1Metric Detail

In this section, we detail the evaluation metrics used to assess the performance of the agents\. These metrics evaluate the validity, optimality, and robustness of the joint decisions, as well as the accuracy of the agents’ internal state estimation\.

#### B\.1\.1Validity of Decision

A joint decision is considered valid if and only if it satisfies all domain\-specific feasibility constraints defined in the environment\. These typically includecapacity constraints\(e\.g\., ensuring resource consumption does not exceed availability\) andconsistency constraints\(e\.g\., ensuring logical assignment rules are respected\)\. Only decisions that fully adhere to these hard constraints are eligible for positive rewards\.

#### B\.1\.2Normalized Reward \(NR\)

Normalized Reward \(NR\) measures the optimality of the final decision relative to the theoretical upper bound\. It is defined as:

NR=Rf​i​n​a​lRm​a​x×100\\text\{NR\}=\\frac\{R\_\{final\}\}\{R\_\{max\}\}\\times 100\(1\)whereRf​i​n​a​lR\_\{final\}is the reward achieved by the agents’ final proposal, andRm​a​xR\_\{max\}is the maximum possible reward computed by the integer programming solver under the ground truth state\. If the decision violates any feasibility constraint,Rf​i​n​a​lR\_\{final\}is set to 0\.

#### B\.1\.3Normalized Adjusted Reward \(NAR\)

To provide a more fine\-grained assessment for invalid outcomes, we introduce Normalized Adjusted Reward \(NAR\), which credits partial success while penalizing infeasibility\. It is calculated as:

NAR=\{NRvalidRs​u​b​s​e​tRm​a​x×\|Ds​u​b​s​e​t\|\|Do​r​i​g​i​n​a​l\|×100invalid\\text\{NAR\}=\\begin\{cases\}\\text\{NR\}&\\text\{valid\}\\\\ \\frac\{R\_\{subset\}\}\{R\_\{max\}\}\\times\\frac\{\|D\_\{subset\}\|\}\{\|D\_\{original\}\|\}\\times 100&\\text\{invalid\}\\end\{cases\}\(2\)For invalid proposals, we employ a greedy algorithm to extract amaximal feasible subsetof the decision components \(e\.g\., a subset of valid assignments or items\) from the original proposal\. Here,\|Ds​u​b​s​e​t\|\|D\_\{subset\}\|and\|Do​r​i​g​i​n​a​l\|\|D\_\{original\}\|represent the size of the valid subset and the original proposal, respectively\. This metric evaluates the agents’ ability to identify valid sub\-solutions even when the global plan is flawed\.

#### B\.1\.4Valid Ratio \(VR\)

Valid Ratio \(VR\) measures the system’s reliability in adhering to constraints\. It is defined as the percentage of evaluation instances where the agents successfully reach a fully valid final decision:

VR=Nv​a​l​i​dNt​o​t​a​l×100\\text\{VR\}=\\frac\{N\_\{valid\}\}\{N\_\{total\}\}\\times 100\(3\)whereNv​a​l​i​dN\_\{valid\}is the number of sessions concluding with a valid outcome \(Rf​i​n​a​l\>0R\_\{final\}\>0\)\.

#### B\.1\.5Hallucination Rate \(HR\)

Hallucination Rate \(HR\) measures how often agents overestimate the availability of resources or ingredients\. A state componentiiis considered "hallucinated" if the agent’s estimated quantitySiE​s​tS^\{Est\}\_\{i\}strictly exceeds the ground truthSiG​TS^\{GT\}\_\{i\}\. The metric is defined as the percentage of such overestimated components across the entire state vector:

HR=∑i=1D𝕀​\(SiE​s​t\>SiG​T\)D\\text\{HR\}=\\frac\{\\sum\_\{i=1\}^\{D\}\\mathbb\{I\}\(S^\{Est\}\_\{i\}\>S^\{GT\}\_\{i\}\)\}\{D\}\(4\)whereDDrepresents the total dimension of the state space, and𝕀​\(⋅\)\\mathbb\{I\}\(\\cdot\)is the indicator function\. By tracking these "false positives" in resource availability, HR assesses the risk of agents proposing infeasible plans based on non\-existent resources\.

#### B\.1\.6Normalized Mean Absolute Error \(NMAE\)

For task allocation, hallucination rate is often close to zero and therefore does not capture errors in aggregating pooled public resources\. We additionally measure normalized mean absolute error \(NMAE\) over public resources as a state\-estimation error metric\. LetKp​u​bK\_\{pub\}be the union of public\-resource keys in the estimated and ground\-truth dictionaries:

NMAEp​u​b=1\|Kp​u​b\|​∑k∈Kp​u​bek,ek=\|v^k−vk\|\|vk\|\\text\{NMAE\}\_\{pub\}=\\frac\{1\}\{\|K\_\{pub\}\|\}\\sum\_\{k\\in K\_\{pub\}\}e\_\{k\},\\quad e\_\{k\}=\\frac\{\|\\hat\{v\}\_\{k\}\-v\_\{k\}\|\}\{\|v\_\{k\}\|\}\(5\)for nonzerovkv\_\{k\}, with zero ground\-truth dimensions assigned error1\.01\.0ifv^k≠0\\hat\{v\}\_\{k\}\\neq 0and0\.00\.0otherwise\.

Table 6:Hallucination rate \(HR, mean±\\pmstandard deviation\) aggregated over non\-oracle settings\.Table 7:Linear regression results relating hallucination rate \(HR\) to the centralized performance gap\.β\\betadenotes the regression coefficient of HR,R2R^\{2\}denotes the proportion of variance in the gap explained by HR, andppdenotes the p\-value for the correlation test\.

### B\.2Supplementary Results

#### B\.2\.1Model Performance Measured with NAR

To measure the subtle differences between model preferences and provide diagnostic indicators to study model preferences and characteristics, we report results using NAR in[Table˜5](https://arxiv.org/html/2607.06157#A2.T5)\. In contrast to NR, which strictly punishes invalid plans, NAR is a much more tolerant indicator, where small models with low VR can still obtain a modest reward; for example, Qwen3\-32B achieves an average NAR of 58\.73 in themenu\-numericsettings, but only an average NR of 4\.78 as reported in[Table˜2](https://arxiv.org/html/2607.06157#S3.T2)\.

#### B\.2\.2State Estimation Error Diagnostics

To inspect information exchange and aggregation during deliberation, we use HR andNMAEp​u​b\\text\{NMAE\}\_\{pub\}as complementary diagnostics of state\-estimation error\. As shown in[Table˜6](https://arxiv.org/html/2607.06157#A2.T6), agents sometimes overestimate unavailable resources in the menu\-design domains, whereas HR is near zero in task allocation\. This suggests that HR captures menu\-design hallucination but is less informative for task allocation, where errors mainly involve aggregating pooled public resources\.

We then examine whether these state\-estimation errors are associated with performance gaps\. For menu design, linear regression in[Table˜7](https://arxiv.org/html/2607.06157#A2.T7)shows a significantly positive correlation between HR andΔcent\\Delta\_\{\\text\{cent\}\}, suggesting that hallucinated resources account for part of the decentralized performance gap\. This is consistent with the view that centralized access reduces errors from imperfect information aggregation\. In task allocation, because HR is near zero, we instead analyze public\-resource aggregation error usingNMAEp​u​b\\text\{NMAE\}\_\{pub\}\. In the tool\-augmented subset, higherNMAEp​u​b\\text\{NMAE\}\_\{pub\}is significantly associated with lower NR \(Pearsonr=−0\.2513r=\-0\.2513,p<\.0001p<\.0001; Spearmanρ=−0\.2886\\rho=\-0\.2886,p<\.0001p<\.0001\), while the association is not statistically significant in the no\-tools condition\.

## Appendix CCase Studies

In the main text, we report aggregate performance trends\. This section presents some case studies that illustrate the underlying mechanisms behind these phenomena\.

### C\.1A Failed Case of the Centralized Baseline

Prompt: Failed Centralized Baseline Example

==BASELINEEXPERIMENT==

YouhaveaccesstoALLinformationfromallagentsandmustmakethefinal

allocationdirectly\(nonegotiation\)\.

==SCENARIO==

Allocate10tasksamong3agentstomaximizeteamvalueunderresourcelimits\.

UsethegiventasknamesasJSONkeys\.

==COMPLETEINFORMATION==

<agent\_0\_efficiency\_values\>

\{fullefficiencydictionaryforagent\_0\}

</agent\_0\_efficiency\_values\>

<agent\_1\_efficiency\_values\>

\{fullefficiencydictionaryforagent\_1\}

</agent\_1\_efficiency\_values\>

==TOTALRESOURCES==

CRITICAL:

\-Privateresourcesareper\-agent\(doNOTaddacrossagents\)\.

\-PublicresourcesMUSTbepooledbysummationacrossallobservations\.

<total\_private\_resources\>

Agent0:\{"Time":6\.0,"Mental":4\.6,"Physical":3\.0,"Multitasking":1\.8\}

</total\_private\_resources\>

<task\_requirements\>

\{fullresourcerequirementsfor10tasks;omitted\}

</task\_requirements\>

==YOURTASK==

ProduceavalidallocationJSONandexplicitlyverify:

\(1\)eachagent’sprivateconstraints;

\(2\)pooledpublicconstraints\.

Outputformat:

<reasoning\_process\>…</reasoning\_process\>

<decision\>\{…"final\_decision":\{…\}\}</decision\>

Table 8:The prompt for the centralized baseline in a failed example\.We present a case study contrasting the centralized baseline \(full information, no deliberation\) with the multi\-agent deliberative system on the same task\-allocation instance\. The prompt used in this failed example is shown in[Table˜8](https://arxiv.org/html/2607.06157#A3.T8)\.

Although the centralized baseline receives complete information, it fails to produce a valid final allocation\. It correctly identifies that assigningPrep vegetables batchto Agent 0 violates the time limit, but still assigns that task to Agent 0 in thefinal\_decision, contradicting its own earlier validation\.

\[Earliervalidation\]

Agent0:Storeingredients\+Receivedelivery\+Prepvegetables

Time=2\+2\+3=7\>6\(NOTOK\)

\[LaterfinalJSON\]

"StoreingredientswithFIFOlabeling":"agent\_0",

"Receivedeliveryandverifyitems":"agent\_0",

"Prepvegetablesbatch\(wash/chop/portion\)":"agent\_0"<\-\-invalid

By contrast, the multi\-agent system can perform a distributed verification, whereby each agent examines whether the proposal violates the constraints visible from its local perspective, thereby preventing invalid plans from being finalized\.

Round2\(agent\_0\):Icannotacceptagent\_2’sproposalbecauseitexceeds

theirprivateresourcelimitssignificantly\.\.\.

Round2\(agent\_2\):Icannotacceptthecurrentpartnerproposalbecauseit

overloadsagent\_1intime/mental/physicalresourcesandexceedsservingand

kitchenlimits\.\.\.

### C\.2An Example of the “Skepticism Trap” in Tool\-Augmented Reasoning

Prompt: “Skepticism Trap” Input Example

Youare\{agent\_name\}ina3\-agentteam\(2workers\+1leader\)\.

Yourgoalistomaximizetotalteamvalueunderprivate\+publicconstraints\.

IMPORTANTenvironmentrule:

\-Privateresourcesareagent\-specific\.

\-PublicresourcesarepooledacrossagentsbySUMMATIONafterinformationsharing\.

Yourlocalobservation:

<current\_private\_resources\>

\{…\}

</current\_private\_resources\>

<current\_public\_resources\>

\{partialfragmentonly\}

</current\_public\_resources\>

<efficiency\_dict\>

\{…\}

</efficiency\_dict\>

<task\_requirements\>

\{…\}

</task\_requirements\>

\(Iftoolsareenabled\)

<tool\_output\>

best\_allocation=solverresultcomputedfromyourCURRENTestimatedtotals

</tool\_output\>

Deliberationprotocol:

\-Shareobservationsandchallengeinfeasibleproposals\.

\-TheLeader\(agent\_2\)makesthefinaldecision\.

\-Outputproposal/decisionintherequiredXML\+JSONformat\.

Table 9:The prompt for the decision module with external tools in an example of the “skepticism trap”\.We present a case study that illustrates a counterintuitive result from the task\-allocation domain: the tool\-augmented setting fails to produce a valid allocation, whereas the no\-tool setting converges to a valid conservative plan\. The prompt structure for this scenario is shown in[Table˜9](https://arxiv.org/html/2607.06157#A3.T9)\.

The key issue in this failed example is a misinterpretation of tool context, which triggers a cascade of over\-correction\. The agents observe different fragments of public resources; globally, the public constraint is the sum across agents\. However, some agents misinterpret their local fragment as the team\-level hard cap and consequently judge the tool\-generated solution as infeasible, since it exceeds the resources visible from their local view alone\. In this case, the agents not only need to think about the proposal but also need to verify the suggested decision, adding an extra burden\. We show the summarized example output below:

Round1\(agent\_0\):

IseeIngredients=63,Serving=2,Kitchen=2,Sanitation=1\.

The"best\_allocation"uses200\+ingredientsandmuchmorekitchen/sanitation,

soI’mworrieditisnotfeasible\.

Round1\(agent\_1\):

Ialsoseeverysmallpublicresources\(Ingredients=74,Serving=2,Kitchen=2,

Sanitation=1\)\.Idon’ttrusttheearliertool\-basedallocation\.

Round1\(agent\_2,Leader\):

Iseeamuchlargerfragment\(Ingredients=204,Serving=4,Kitchen=4,

Sanitation=2\),butIwillalignwiththeteam’sconcernandmovetoasaferplan\.

The true global public budget is the*sum*of fragments \(e\.g\., Ingredients=63\+74\+204=341=63\+74\+204=341\), so the tool output was feasible under the true pooled budget\.

By contrast, in the no\-tool condition, agents still do not perfectly reconstruct the pooled public totals, but they are no longer distracted by a proposal that appears incompatible with local views\. The dialogue focuses on progressive synchronization and converges to a valid conservative allocation\.

Round2\(agent\_2,Leader\):

Let’soptimizeforworst\-caserobustnessunderthetightestplausiblepublic

limits\(Ingredients63,Serving2,Kitchen2,Sanitation1\)andfinalizea

minimalsafebundle\.

\[Laterrounds\]

Allagentsconfirmthesamesmallallocationandkeepitconsistentthroughthe

finaldecisionoutput\.

## Appendix DPrompts

Since the original prompts are too long to include in full, we provide the general organizational format of the prompts \(omitting some details\)\. Full prompt templates can be found in our code\. The prompts used in each task setting are generally consistent, with minor specialization on instructions\.

In this appendix, we show the prompt templates in the menu design domain as an example, including the observation module in[Table˜10](https://arxiv.org/html/2607.06157#A4.T10), the planning module in[Table˜11](https://arxiv.org/html/2607.06157#A4.T11), the decision module in[Table˜12](https://arxiv.org/html/2607.06157#A4.T12), and the conversation module in[Table˜13](https://arxiv.org/html/2607.06157#A4.T13)\.

Prompt Template: Observation Module

YouareanAIagent,named\{agent\_name\}\.Theuseraskedyouandanotheragenttodesignamenufortreatingtheirguests\.…

\(setting\-specificcontent\)

Menu\-Numeric:Thepreferences,atthisstage,arenumericalscores,…

Menu\-Semantic:…Youneedtoestimatethepreferencesforeachdishbasedontheinformationofbothguests\.…

…Youmayneedtocommunicateforcollaboration\.Youcancommunicatefor\{max\_round\}rounds\.Ineachround,youcansayupto\{max\_character\}characters,messagesthatexceedthislimitwillbetruncated\.…

Hereyouwilldothefirststep,youneedtoestimatethe"ingredients"andthe"preference"\.…

Yourinitialobservation:\{current\_obs\}\{score\_dict\}

Therecipesare:\{recipes\}

Thedialogis:\{chat\_history\}

…

Format:

<thinking\_process\_1\>

thinking\_process…

</thinking\_process\_1\>

<partner\_ingredients\>

\{\{"ingredient\_1":1,…\}\}

</partner\_ingredients\>

<partner\_preference\>

\{\{"dish\_1":1\.5,…\}\}

</partner\_preference\>

<thinking\_process\_2\>

thinking\_process…

</thinking\_process\_2\>

<total\_ingredients\>

\{\{"ingredient\_1":1,…\}\}

</total\_ingredients\>

<overall\_preference\>

\{\{"dish\_1":1\.5,…\}\}

</overall\_preference\>

Table 10:Simplified prompt template for the observation module\.Prompt Template: Planning Module

YouareanAIagent,named\{agent\_name\}\.Theuseraskedyouandanotheragenttodesignamenufortreatingtheirguests\.…

\(setting\-specificcontent\)

Menu\-Numeric:Thepreferences,atthisstage,arenumericalscores,…

Menu\-Semantic:…Youneedtoestimatethepreferencesforeachdishbasedontheinformationofbothguests\.…

…Youmayneedtocommunicateforcollaboration\.Youcancommunicatefor\{max\_round\}rounds\.Ineachround,youcansayupto\{max\_character\}characters,…

Hereyouwilldothesecondstep,youwillneedtodesignthebestmenu,…

Nowisround\{round\_count\}ofthetotal\{max\_round\}rounds\.

Hereisyourcurrentestimation:\{total\_ingredients\},\{overall\_preference\}

Hereistherecipesthatindicatetheingredientsneedbyeachdish:\{recipes\}

\(ifusetools\)

Hereisalistofdisheswhichalreadyhavesufficientingredients:\{available\_dishes\}

Hereisalistofmissingingredientsforcurrentlyunavailabledishes:\{unavailable\_dishes\_info\}

Hereistheoptimaldishlistbasedontotalingredientsandoverallpreference:\{best\_menu\}

…

Format:

<reasoning\_process\>

XX

</reasoning\_process\>

<proposal\>

\{\{"menu\_proposal":\["dish\_1","dish\_2","dish\_3","dish\_4",…\],

"explanation":"XX"\}\}

</proposal\>

Table 11:Simplified prompt template for the planning module\.Prompt Template: Decision Module

YouareanAIagent,named\{agent\_name\}\.Theuseraskedyouandanotheragenttodesignamenufortreatingtheirguests\.…

\(setting\-specificcontent\)

Menu\-Numeric:Thepreferences,atthisstage,arenumericalscores,…

Menu\-Semantic:…Youneedtoestimatethepreferencesforeachdishbasedontheinformationofbothguests\.…

…Youmayneedtocommunicateforcollaboration\.Youcancommunicatefor\{max\_round\}rounds\.Ineachround,youcansayupto\{max\_character\}characters,…

Hereyouwilldothethirdstep,youneedtodecideacceptyourpartner’sproposalornot…\.

Youhavethefollowinginformation:

Nowisround\{round\_count\}ofthetotal\{max\_round\}rounds\.

Hereistherecipesthatindicatetheingredientsneedbyeachdish:\{recipes\}

Totalingredientsandtheoverallpreference:\{total\_ingredients\},\{overall\_preference\}

\(ifusetools\)

Basedonyourestimation:

Hereisalistofdisheswhichalreadyhavesufficientingredients:\{available\_dishes\}

Hereisalistofmissingingredientsforcurrentlyunavailabledishes:\{unavailable\_dishes\_info\}

Hereistheoptimaldishlistbasedontotalingredientsandoverallpreference:\{best\_menu\}

Thedialogbetweenyouandyourpartnersofaris:\{chat\_history\}

Yourlatestproposalis:\{agent\_proposal\}

Yourexplanationofyourproposalis:\{agent\_proposal\_explanation\}

Thelatestproposalfromyourpartneris:\{partner\_proposal\},\{partner\_explanation\}

Nowyouneedtodecidewhethertoacceptyourpartner’sproposalornot\.

…

Format:

<reasoning\_process\>

XX

</reasoning\_process\>

<decision\>

\{\{"accept\_decision":true/false,"explanation":"XX"\}\}

</decision\>

Table 12:Simplified prompt template for the decision module\.Prompt Template: Conversation Module

YouareanAIagent,named\{agent\_name\}\.Theuseraskedyouandanotheragenttodesignamenufortreatingtheirguests\.…

\(setting\-specificcontent\)

Menu\-Numeric:Thepreferences,atthisstage,arenumericalscores,…

Menu\-Semantic:…Youneedtoestimatethepreferencesforeachdishbasedontheinformationofbothguests\.…

…Youmayneedtocommunicateforcollaboration\.Youcancommunicatefor\{max\_round\}rounds\.Ineachround,youcansayupto\{max\_character\}characters,…

Hereyouwilldothefourthstep,alsothefinalstepinthisround…\.

Hereisalltheinformationsofar:

Nowisround\{round\_count\}ofthetotal\{max\_round\}rounds\.

Hereistherecipesthatindicatetheingredientsneedbyeachdish:\{recipes\}

Yourinitialobservation:\{current\_obs\},\{person\_1\_info\},\{person\_2\_info\}

Thedialogis:\{chat\_history\}

Yourestimationofyourpartner’sobservationandthetotalingredientsandtheoverallpreference:\{partner\_ingredients\},\{partner\_preference\},\{total\_ingredients\},\{overall\_preference\}

\(ifusetools\)

Basedonyourestimation:

Hereisalistofdisheswhichalreadyhavesufficientingredients:\{available\_dishes\}

Hereisalistofmissingingredientsforcurrentlyunavailabledishes:\{unavailable\_dishes\_info\}

Hereistheoptimaldishlistbasedontotalingredientsandoverallpreference:\{best\_menu\}

Basedonthese,yourlatestproposalis:\{agent\_proposal\}

Thelatestproposalfromyourpartneris:\{partner\_proposal\}

Yourejectedyourpartner’sproposal,because:\{decision\_explanation\}

…

Now,forthenextround,pleaseorganizewhattotalkwithyourpartner\.

…

Format:

<reasoning\_process\>

XX

</reasoning\_process\>

<message\_content\>

\(Thewordsyouwanttosaytoyourpartner\)

</message\_content\>

Table 13:Simplified prompt template for the conversation module\.