Paper page - Training LLMs for Multi-Agent Collaboration with Verifiable Information Rewards

Source: https://huggingface.co/papers/2610.17893 🚀 Can we train LLMs to collaborate, rather than only design better multi-agent workflows?

I’m sharing our new preprint, Training LLMs for Multi-Agent Collaboration with Verifiable Information Rewards (CoSIR).

Most LLM post-training focuses on chatbots and individual agents: instruction following, reasoning, and task completion. In multi-agent systems (MAS), however, connecting several LLMs through prompts or workflows does not guarantee effective teamwork.

Agents need to learn when to share useful evidence, when to ask teammates for missing information, and how to incorporate their contributions into decisions.

So we ask: can collaboration itself be an explicit target of LLM post-training?

💡 The challenge is reward design. In natural-language tasks, final success tells us little about which messages helped. A statement’s value depends on what teammates already know and whether it helps them identify the correct answer.

We introduce CoSIR with three components:

🔹 Scalable, verifiable environment. Agents see different parts of a rating table and must exchange information to select the best option. We can generate many tasks while tracking each agent’s information and the correct answer.

🔹 Verifiable information rewards. A Bayesian evaluator scores shared evidence by its reduction in team surprisal. Requests are scored by the potential value of truthful answers, with additional rewards for correctly using received information.

🔹 Self-play RL. These fine-grained rewards directly post-train the underlying LLM, rather than only modifying an inference-time workflow.

📊 The key test is transfer to more complex tasks—not performance on the toy task alone.

We post-train Qwen3-4B, Qwen3-8B, and Gemma-3-4B-it, then evaluate them on 7 unseen natural-language collaboration benchmarks spanning distributed evidence, mathematical reasoning, and collaborative decision-making. CoSIR achieves the highest seven-task score sum among compared methods on all 3 backbones (Qwen3-8B: +31.85 points versus base).

We also examine behavior. For Qwen3-8B on HiddenBench, private-fact disclosure rises from 68.5% to 74.3%. In MAST-based analysis pooled across the three models, failures to use teammates’ input fall from 26.3% to 22.8%, and failures to seek clarification from 36.4% to 32.0%. Performance on GSM8K, MMLU, and IFEval remains largely preserved.

🎯 Our takeaway: Collaboration is a capability that should be explicitly trained. Better workflows matter—but so does teaching the underlying LLM how to work with other agents.

📄https://arxiv.org/abs/2610.17893