GPTNT: Benchmarking Real-Time Collaboration Between Multimodal Agents on Keep Talking And Nobody Explodes
Summary
The paper introduces GPTNT, a benchmark built on Keep Talking and Nobody Explodes that requires two multimodal agents to collaborate in real-time under time pressure and information asymmetry, revealing critical weaknesses in state-of-the-art systems.
View Cached Full Text
Cached at: 06/30/26, 05:31 AM
# GPTNT: Benchmarking Real-Time Collaboration Between Multimodal Agents on Keep Talking And Nobody Explodes Source: [https://arxiv.org/abs/2606.28514](https://arxiv.org/abs/2606.28514) [View PDF](https://arxiv.org/pdf/2606.28514) > Abstract:Multimodal models are increasingly deployed to solve tasks collaboratively with humans or other artificial agents\. Existing benchmarks show that these models possess many of the required component capabilities, but the conditions that coincide in collaboration, including time pressure, information asymmetry, and imperfect communication, are usually studied in isolation\. We introduce GPTNT, a benchmark built on the cooperative video game Keep Talking and Nobody Explodes, in which two agents must coordinate to defuse procedurally generated bomb puzzles against a live countdown\. One agent can see and manipulate the bomb but does not have the defusal instructions; the other has the instructions but cannot see or manipulate the bomb\. Neither agent can succeed alone: success requires effective and efficient communication\. Unlike turn\-based proxies, GPTNT requires agents to act asynchronously and communicate in real time\. GPTNT is designed to separate collaboration from reliance on memorized solutions: the instruction manual, the partner, or both can be withheld to isolate what a model derives in the moment from what it already knows\. We show that GPTNT poses a substantial challenge for state\-of\-the\-art systems: none of the closed\- or open\-source models we test defuses a single bomb in real time, a bar that human players clear\. Through controlled experiments, we identify critical weaknesses in state tracking, efficient action under time pressure, ambiguity handling, and error recovery\. We release GPTNT as a benchmark for collaborative performance that current evaluations leave unmeasured\. Because it runs on the real game, GPTNT benefits from procedural generation and inherits a living modding community, allowing the benchmark to evolve as models improve rather than being solved once and retired\. ## Submission history From: Sabrina McCallum \[[view email](https://arxiv.org/show-email/b7432e5f/2606.28514)\] **\[v1\]**Fri, 26 Jun 2026 18:09:36 UTC \(37,637 KB\)
Similar Articles
Multi-agent Framework for Time-Sensitive Complementary Collaboration in Minecraft
The paper introduces TickingCollabBench, a Minecraft-based multi-agent benchmark for time-sensitive complementary collaboration tasks with dynamic environments, and demonstrates that LLMs frequently fail under such conditions compared to a global-knowledge oracle.
Measuring inter-agent confrontations and collaboration
The author built a platform called Glomz where AI agents with different capabilities review each other's code in an arena setting. The experiment revealed emergent behaviors like review cascades and cross-model insights, but also challenges with orchestration and participation rates.
AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators
This paper introduces AgentCollabBench, a diagnostic benchmark for multi-agent systems that evaluates behavioral risks like instruction decay and context leakage across four major LLMs. It argues that communication topology is a critical factor in multi-agent reliability, often overshadowing raw model capability.
GAMBIT: A Three-Mode Benchmark for Adversarial Robustness in Multi-Agent LLM Collectives
This paper introduces GAMBIT, a benchmark for evaluating adversarial robustness in multi-agent LLM collectives, featuring adaptive imposters and recalibration modes to address the limitations of existing shallow evaluations.
GroupMemBench: Benchmarking LLM Agent Memory in Multi-Party Conversations
GroupMemBench is a new benchmark for evaluating LLM agent memory in multi-party conversations, exposing failures in current memory systems with the best achieving only 46% average accuracy.