Do Latent Channels Actually Communicate? A Causal Audit of Latent Multi-Agent LLM

# Do Latent Channels Actually Communicate? A Causal Audit of Latent Multi-Agent LLM Communication
Source: [https://arxiv.org/html/2607.26773](https://arxiv.org/html/2607.26773)
###### Abstract

Latent communication in large language model \(LLM\)\-based multi\-agent systems \(MAS\) transmits continuous internal representations instead of text, but greater representational capacity does not establish that the receiver uses task\-relevant information\. End\-task performance alone also cannot reveal whether an observed effect depends on message presence, content generated for the evaluated example, or information supplied by a separate agent\. We introduce a causal audit that applies controlled message replacements at the boundary where the sender\-produced representation enters the receiver\. Four message settings support five measurements of encoded sender information, receiver sensitivity to message presence and identity, the task value of example\-specific content, and the additional value supplied by a separate agent\. We apply the audit to latent relay with Qwen3\-4B and Qwen3\-8B on GSM8K, ARC\-C, and MATH\-500\. On GSM8K, the Qwen3\-4B overall performance effect of−1\.00\-1\.00percentage point decomposes into a−6\.17\-6\.17\-point effect retained by an other\-example message and a\+5\.17\+5\.17\-point effect attributable to example\-specific content; both component directions reverse at 8B\. On MATH\-500, the Qwen3\-4B gain of15\.0015\.00points comprises8\.338\.33points retained by an other\-example message and6\.676\.67points attributable to example\-specific content, while the 8B gain is dominated by the former component\. Self\-substitution comparisons further show that example\-specific content and other\-agent value are distinct\. These results show that aggregate accuracy does not identify how a latent message affects the receiver and motivate controlled message comparisons as a standard evaluation for latent communication\.

## Introduction

Recent large language model \(LLM\)\-based multi\-agent systems \(MAS\) have begun to explore communication channels beyond natural language\. Conventional MAS typically coordinate agents through text messages, but text communication requires an agent to project its internal computation into discrete token sequences\. This discretization exposes only the sampled symbols while discarding alternative information represented during generation\(Phamet al\.[2023](https://arxiv.org/html/2607.26773#bib.bib20); Duet al\.[2026](https://arxiv.org/html/2607.26773#bib.bib5)\)\. To overcome this limitation, recent studies have explored richer communication carriers, including probability\-weighted embeddings, hidden states, and key\-value \(KV\) caches\(Phamet al\.[2023](https://arxiv.org/html/2607.26773#bib.bib20); Zouet al\.[2026](https://arxiv.org/html/2607.26773#bib.bib28); Duet al\.[2026](https://arxiv.org/html/2607.26773#bib.bib5)\)\.

We refer to these approaches collectively as latent communication: communication protocols that transmit continuous internal representations, such as embeddings, hidden states, or KV caches, without requiring intermediate decoding into natural language\. These approaches share a common intuition: preserving more information at the communication boundary may provide the receiver with a richer signal than text exchange\. They report improved task performance over text\-based communication or single\-agent baselines, often together with reduced communication cost or inference overhead\(Phamet al\.[2023](https://arxiv.org/html/2607.26773#bib.bib20); Zouet al\.[2026](https://arxiv.org/html/2607.26773#bib.bib28); Duet al\.[2026](https://arxiv.org/html/2607.26773#bib.bib5)\)\.

A performance gain after communication can arise from multiple sources: sender\-specific content transferred across the boundary, additional computation introduced by the communication process, context reuse, or redundant reasoning trajectories\(Cemriet al\.[2025](https://arxiv.org/html/2607.26773#bib.bib2)\)\. Therefore, an end\-task accuracy difference alone cannot identify whether latent communication achieves genuine information transfer\. The distinction is analogous to causal analysis in neural networks, where observing a representation’s correlation with an output does not establish that the representation mediates the computation\(Viget al\.[2020](https://arxiv.org/html/2607.26773#bib.bib24); Menget al\.[2022](https://arxiv.org/html/2607.26773#bib.bib19)\)\. A latent channel may influence receiver behavior without the receiver using the sender\-specific information contained in the message\(Loweet al\.[2019](https://arxiv.org/html/2607.26773#bib.bib16)\)\. The research question is whether transmitted information causally contributes to receiver behavior\.

![Refer to caption](https://arxiv.org/html/2607.26773v1/Figures/fig_1_overview.png)Figure 1:A causal audit of latent communication\. \(a\) The sender produces a latent messageMMas an embedding sequence, hidden states, or a KV cache; each intervention removes or replacesMMbefore it is injected into the receiver, while the receiver context and downstream computation are held fixed\. \(b\) An example is one test case from a benchmark test set\. The four message settings are no message; an other\-example message, generated by the sender through the same communication interface from a different example in the same test set and matched approximately in length; a compute\-matched self\-generated message; and the current\-example message, generated from the evaluated example\. Their comparisons define PL and CIC at the prediction\-distribution level, and the overall performance effect, CAG and SSG at the task\-performance level\. PS instead tests whether the correctness of the sender’s answer is decodable fromMM\. \(c\) On GSM8K with LatentMASλH\\lambda\_\{H\}, the overall performance effect decomposes into the other\-example message effect plus CAG:−1\.00=−6\.17\+5\.17\-1\.00=\-6\.17\+5\.17percentage points \(pp\) for Qwen3\-4B \(n=100n=100\) and\+1\.67=\+3\.96−2\.29\+1\.67=\+3\.96\-2\.29pp for Qwen3\-8B \(n=60n=60\)\. Bars show point estimates and whiskers show 95% confidence intervals\. The near\-zero 4B overall effect combines two opposing effects whose intervals exclude zero; both component estimates reverse sign at 8B; an interval crossing zero establishes neither a sign nor negligibility\.Existing analyses of LLM\-based MAS do not yet provide such an identification test\. Failure taxonomies characterize recurrent coordination problems, while recent counterfactual studies intervene on individual agents, communication edges, or no\-communication topologies to study error propagation and correlated agreement\(Cemriet al\.[2025](https://arxiv.org/html/2607.26773#bib.bib2); Shenet al\.[2025](https://arxiv.org/html/2607.26773#bib.bib22); Huanget al\.[2026](https://arxiv.org/html/2607.26773#bib.bib8); Liet al\.[2026b](https://arxiv.org/html/2607.26773#bib.bib11)\)\. These analyses establish whether communication helps or harms a system, but they do not determine which information inside a latent message causes the effect\. Existing taxonomies classify latent\-communication methods by the transmitted representation, sender–receiver alignment, and receiver\-side fusion mechanism, but do not test whether the receiver uses content generated for the evaluated example\(Liu[2026](https://arxiv.org/html/2607.26773#bib.bib14)\)\.

We introduce a causal audit that intervenes on the message before it enters the receiver while holding the receiver context and downstream computation fixed\. We use example to mean one test case from a benchmark test set\. The current\-example message is the original message generated by the sender from the evaluated example\. The first question is whether the message contains information about the sender\. Positive signaling \(PS\) asks whether a declared sender variable is decodable from the message\. In this work, the primary variable is whether the sender’s own answer is correct\. PS measures information encoded in the message, not whether the receiver uses that information\. Encoded information establishes receiver use only when changing the message changes receiver behavior\. Positive listening \(PL\) therefore compares the receiver’s prediction distributions under the current\-example message and no message\. This comparison asks whether message presence affects the receiver at all, without yet identifying which part of the message produces the change\.

To test whether content from the evaluated example matters, we replace the current\-example message with an other\-example message\. An other\-example message is generated by the sender from a different example in the same test set and is approximately matched in length\. Because it is model\-generated through the same communication interface, it preserves the original message structure, but its content comes from another example\. Causal influence of communication \(CIC\) measures how the receiver’s prediction distribution changes between the current\-example and other\-example messages\. Content\-attributable gain \(CAG\) measures the corresponding signed difference in task performance\. CIC tests whether message identity affects receiver predictions, whereas CAG isolates the task value of content generated for the evaluated example\. Content generated for the evaluated example may be useful without requiring a separate agent\. The self\-substitution gap \(SSG\) therefore compares the sender’s current\-example message with a self\-generated message produced by the receiver through the same communication interface under a matched computation budget\. This comparison asks whether a separate agent contributes task value beyond what the receiver can generate for itself\.

Figure[1](https://arxiv.org/html/2607.26773#Sx1.F1)summarizes the audit\. Panel \(a\) places embedding sequences, hidden states, and KV caches behind a shared message boundary, so every intervention removes or replaces the message before receiver injection\. Panel \(b\) presents four message settings: no message, an other\-example message, a self\-generated message, and the current\-example message\. Their comparisons separate message presence, message identity, example\-specific content, and other\-agent value, while PS is measured directly on the message\. Panel \(c\) shows that the overall performance effect decomposes exactly into the other\-example message effect and CAG\. The results illustrate why the decomposition is necessary\. On GSM8K, the near\-zero overall performance effect for Qwen3\-4B combines two opposing components, while both component directions reverse for Qwen3\-8B\. Across models and tasks, similar overall performance can therefore arise from different communication mechanisms\. We make the following contributions:

- •We formulate latent communication as a capacity\-versus\-usage problem and provide a common causal audit for probability\-weighted embeddings, latent\-thought hidden states, KV caches\.
- •We introduce a five\-part measurement suite comprising PS, PL, CIC, CAG, and SSG\. The suite separates information encoded in a message, receiver sensitivity to message presence and identity, the task value of example\-specific content, and the value contributed by a separate agent\.
- •We develop a standardized intervention design centered on four model\-generated message settings, complemented by a preservation\-ordered diagnostic ladder, component restoration, message\-realism measurements, receiver\-instability checks, and paired inference with positive, negligible\-effect, and inconclusive outcomes\.
- •We provide evidence that latent\-channel behavior is heterogeneous: faithful relay can cause degradation in one operating regime, while another regime exhibits both a substantial content\-attributable effect and a comparably content\-independent effect\. This decomposition changes the interpretation of the same end\-task accuracy gain\.

## Related Work

This section positions our audit against four lines of work: latent\-communication methods, measurement of emergent communication, causal interventions on neural representations, and system\-level analyses of MAS communication\. Table[1](https://arxiv.org/html/2607.26773#Sx2.T1)summarizes the identification properties that distinguish these lines\.

Table 1:Identification properties of related work\. Columns indicate whether each work studies a latent inter\-agent message, intervenes on the message itself, separates the effect of content generated for the evaluated example from the effect retained under an other\-example message, compares against a compute\-matched self\-generated alternative \(self\-subst\.\), and supports negligible\-effect conclusions through equivalence\-based inference\. \(∙\\bullet\) satisfied; \(△\\triangle\) partial; \(∘\\circ\) absent\.Latent inter\-agent communication builds on work in single\-agent latent reasoning, where Coconut feeds a model its own last hidden state back as the next input instead of a sampled token\(Haoet al\.[2025](https://arxiv.org/html/2607.26773#bib.bib7)\)\. Multi\-agent methods replace text messages with different continuous representations\. CIPHER transmits the expectation of vocabulary embeddings under the sender’s token belief\(Phamet al\.[2023](https://arxiv.org/html/2607.26773#bib.bib20)\)\. LatentMAS generates latent thoughts as last\-layer states and relays the sender’s layer\-wise KV cache as working memory\(Zouet al\.[2026](https://arxiv.org/html/2607.26773#bib.bib28)\)\. DroidSpeak reuses prompt KV across same\-architecture models to cut serving latency\(Liuet al\.[2026](https://arxiv.org/html/2607.26773#bib.bib15)\)\. C2C learns a projector and fusion module that injects a sender’s KV cache into a target model\(Fuet al\.[2025](https://arxiv.org/html/2607.26773#bib.bib6)\)\. Interlat trains a receiver\-side adapter over compressed hidden states\(Duet al\.[2026](https://arxiv.org/html/2607.26773#bib.bib5)\), and ThoughtComm fuses inferred latent thoughts through a shared workspace\(Zhenget al\.[2025](https://arxiv.org/html/2607.26773#bib.bib27)\)\. A recent framework organizes these designs by carrier and injection mechanism\(Liu[2026](https://arxiv.org/html/2607.26773#bib.bib14)\)\. These methods are evaluated primarily through end\-task performance and, in several cases, communication or inference efficiency\. Compression results further challenge a simple bandwidth explanation: compressed KV relay can match or exceed full relay on several benchmarks, indicating that the size of the transmitted representation does not map monotonically to downstream utility\(Liet al\.[2026a](https://arxiv.org/html/2607.26773#bib.bib12)\)\. Table[1](https://arxiv.org/html/2607.26773#Sx2.T1), block A, locates these proposals by their identification properties\.

Emergent\-communication research has long distinguished information encoded by a sender from information used by a receiver\. Referential\-game studies induced protocols between neural agents\(Lazaridouet al\.[2017](https://arxiv.org/html/2607.26773#bib.bib10)\), and measurement pitfalls followed quickly\.Loweet al\.\([2019](https://arxiv.org/html/2607.26773#bib.bib16)\)distinguish positive signaling, where the message depends on the sender’s state, from positive listening, where the receiver’s behavior depends on the message, and show that widely used metrics can certify the former while the latter fails entirely\.Jaqueset al\.\([2019](https://arxiv.org/html/2607.26773#bib.bib9)\)make listening causal, scoring a message by the counterfactual shift it induces in the receiver’s policy\. We adapt these constructs as PS, PL, and CIC for continuous latent messages between LLM agents\. PS introduces an additional estimation problem because the message is continuous and high\-dimensional\. Distribution\-free mutual\-information lower bounds are limited by sample size, while variational estimators trade bias against variance\(McAllester and Stratos[2020](https://arxiv.org/html/2607.26773#bib.bib18); Pooleet al\.[2019](https://arxiv.org/html/2607.26773#bib.bib21); Song and Ermon[2020](https://arxiv.org/html/2607.26773#bib.bib23)\)\. We therefore interpret signaling estimates as finite\-sample lower bounds rather than direct measurements of the channel’s full information content\. Section 3 adapts these measurements to continuous pretrained\-model representations transmitted without a token interface\.

Our intervention design also builds on causal analyses of internal model representations\. Causal mediation analysis introduced indirect\-effect estimands for neural components\(Viget al\.[2020](https://arxiv.org/html/2607.26773#bib.bib24)\)\. ROME operationalized causal tracing through clean, corrupted, and restored forward passes\(Menget al\.[2022](https://arxiv.org/html/2607.26773#bib.bib19)\)\. Later work cataloged the design choices that make patching results trustworthy, including corruption type, metric choice, and a preference for in\-distribution replacement over Gaussian noising\(Zhang and Nanda[2024](https://arxiv.org/html/2607.26773#bib.bib26)\), and documented off\-distribution hazards when patching real LLMs\(Yeoet al\.[2025](https://arxiv.org/html/2607.26773#bib.bib25)\)\. We inherit the protocol and its cautions: restoration sweeps are primary, and we accompany each intervention with checks of message\-distribution similarity and receiver instability\. The difference is the locus and the target\. Patching work localizes circuits within one forward pass; we patch across agents, at the pre\-injection boundary where one model’s tensor enters another model’s computation\. The mediator is the relayed message itself, and the quantity of interest is a channel\-level verdict about transmitted content, not a component map\.

A parallel line analyzes communication failures and dependencies at the system level\. MAST derives a fourteen\-mode failure taxonomy from more than sixteen hundred annotated traces and reports that measured gains over single agents are often minimal, with inter\-agent misalignment as one of its three top\-level categories\(Cemriet al\.[2025](https://arxiv.org/html/2607.26773#bib.bib2)\)\. CAPE and TCTE intervene on an agent’s text output with a do\-operation and measure how errors and insights propagate through communication topologies\(Shenet al\.[2025](https://arxiv.org/html/2607.26773#bib.bib22)\)\. CAGE\-CAL contrasts a post\-communication agent graph with a matched no\-communication counterfactual and shows that communication can produce correlated consensus, which vote\-share confidence mistakes for evidence\(Huanget al\.[2026](https://arxiv.org/html/2607.26773#bib.bib8)\)\.

Together, these studies show that end\-task performance and agent agreement are insufficient to identify content\-based communication\. Yet they analyze or intervene on text outputs or agent\-level structures rather than the latent message crossing the sender–receiver boundary\. Prior work therefore leaves three methodological strands separate: latent\-channel design, causal listening measures for discrete protocols, and system\-level counterfactual analysis of text communication\. The missing test is a latent\-boundary audit that intervenes on the transmitted message itself and separates example\-specific content from the other\-example message effect\. It must also compare the sender’s message with a compute\-matched self\-generated alternative and support equivalence\-based conclusions when content or other\-agent value is practically negligible\. Our audit provides this identification test\.

## The Audit Framework

This section defines a common intervention boundary for latent communication and the measurements it supports\. We first formalize the message pathway, then introduce four message settings, five audit metrics, component attribution, and the associated inference checks\.

### Setup and Audited Boundary

We use*example*to mean one test case from a benchmark test set\. In the formal notation, an example and its associated contexts form an episode

e=\(q,a⋆,cS,cR\)∼𝒟,e=\(q,a^\{\\star\},c\_\{S\},c\_\{R\}\)\\sim\\mathcal\{D\},\(1\)
whereqqis the question,a⋆a^\{\\star\}is the gold answer, andcSc\_\{S\}andcRc\_\{R\}are the sender and receiver contexts\. The sender produces a latent messageM=ϕ​\(e\)M=\\phi\(e\), and the receiver combines it with its context through the injection mapψ\\psi\. For a replacement messageM~\\widetilde\{M\}, the receiver output is

yM~​\(e\)=R​\(ψ​\(cR,M~\)\)\.y\_\{\\widetilde\{M\}\}\(e\)=R\\\!\\left\(\\psi\(c\_\{R\},\\widetilde\{M\}\)\\right\)\.\(2\)
A message intervention replacesMMwithM~\\widetilde\{M\}while holding the receiver context, model weights, prompt template, decoding procedure, and downstream computation fixed\. The declared message must be the only inter\-agent information path that changes\. The same boundary applies to probability\-weighted embeddings, hidden states, and KV caches used by existing latent\-communication methods\(Phamet al\.[2023](https://arxiv.org/html/2607.26773#bib.bib20); Zouet al\.[2026](https://arxiv.org/html/2607.26773#bib.bib28); Duet al\.[2026](https://arxiv.org/html/2607.26773#bib.bib5)\): intervention always occurs after sender\-side construction and before receiver\-side injection\.

Operationally, each method exposes its native message representation at this boundary\. For CIPHER, we replace the probability\-weighted embedding sequence before it enters the receiver input\. For hidden\-state methods, we replace the transmitted hidden\-state sequence before receiver\-side adaptation or re\-encoding\. For KV\-cache relay, we replace only sender\-produced rows in the receiver’s initial cache, leaving receiver\-generated states unchanged\. Current\-example and other\-example messages are generated through the same native interface and are approximately length\-matched\. The comparison therefore changes example identity while preserving the communication carrier, message structure, and receiver\-side computation\.

Table[2](https://arxiv.org/html/2607.26773#Sx3.T2)previews the audit questions and their identifying comparisons\.

Table 2:Audit metrics and their identifying comparisons\. OPE, OME, BME, and DSC denote the overall performance, other\-example message, benchmark\-match, and derived self\-generated contrasts;OPE=OME\+CAG\\mathrm\{OPE\}=\\mathrm\{OME\}\+\\mathrm\{CAG\}andCAG=DSC\+SSG\\mathrm\{CAG\}=\\mathrm\{DSC\}\+\\mathrm\{SSG\}\.
### Message Settings and Audit Metrics

The audit uses four message settings\. The current\-example message is generated by the sender from the evaluated example\. The other\-example message is generated by the same sender from a different, approximately length\-matched example in the same test set; it preserves the original message structure, but its content comes from another example\. The self\-generated message is produced by the receiver through the same interface under matched computation\. The fourth setting supplies no message:

M0\\displaystyle M\_\{0\}=∅,\\displaystyle=\\varnothing,Mcur\\displaystyle\\qquad M\_\{\\mathrm\{cur\}\}=ϕS​\(e\),\\displaystyle=\\phi\_\{S\}\(e\),\(3\)Moth\\displaystyle M\_\{\\mathrm\{oth\}\}=ϕS​\(e′\),\\displaystyle=\\phi\_\{S\}\(e^\{\\prime\}\),Mself\\displaystyle\\qquad M\_\{\\mathrm\{self\}\}=ϕR\(c⋆\)​\(e\)\.\\displaystyle=\\phi\_\{R\}^\{\(c^\{\\star\}\)\}\(e\)\.
wheree′≠ee^\{\\prime\}\\neq eandc⋆c^\{\\star\}is the matched message\-generation budget\. A length\-matched other\-benchmark message is used only for BME\.

PS asks whether a finite sender variableXX, here primarily sender answer correctness, is decodable fromMcurM\_\{\\mathrm\{cur\}\}\. We estimatePSX=I​\(Mcur;X\)\\mathrm\{PS\}\_\{X\}=I\(M\_\{\\mathrm\{cur\}\};X\)with a cross\-fitted lower bound and compare it with a permutation reference\. PS establishes encoded information, not receiver use\.

For settingaa, letPaP\_\{a\}be the receiver prediction distribution andYaY\_\{a\}its parsed answer\. Define

D¯a,b=𝔼e​\[D​\(Pa,Pb\)\],U¯a=𝔼e​\[U​\(Ya\)\]\.\\bar\{D\}\_\{a,b\}=\\mathbb\{E\}\_\{e\}\[D\(P\_\{a\},P\_\{b\}\)\],\\qquad\\bar\{U\}\_\{a\}=\\mathbb\{E\}\_\{e\}\[U\(Y\_\{a\}\)\]\.\(4\)
We use Jensen–Shannon divergence at the first completion token and along a teacher\-forced reference continuation for prediction\-level readouts\. Task\-level readouts use exact\-match accuracy or the declared task\-specific score\. The receiver\-use metrics are

PL\\displaystyle\\mathrm\{PL\}=D¯cur,0,\\displaystyle=\\bar\{D\}\_\{\\mathrm\{cur\},0\},CIC\\displaystyle\\qquad\\mathrm\{CIC\}=D¯cur,oth,\\displaystyle=\\bar\{D\}\_\{\\mathrm\{cur\},\\mathrm\{oth\}\},\(5\)CAG\\displaystyle\\mathrm\{CAG\}=U¯cur−U¯oth,\\displaystyle=\\bar\{U\}\_\{\\mathrm\{cur\}\}\-\\bar\{U\}\_\{\\mathrm\{oth\}\},SSG\\displaystyle\\qquad\\mathrm\{SSG\}=U¯cur−U¯self\.\\displaystyle=\\bar\{U\}\_\{\\mathrm\{cur\}\}\-\\bar\{U\}\_\{\\mathrm\{self\}\}\.
PL measures the message\-presence effect and CIC the message\-identity effect\. CAG isolates the task value of example\-specific content, whereas SSG measures the additional task value supplied by a separate agent\. The task\-level contrasts satisfy

U¯cur−U¯0\\displaystyle\\bar\{U\}\_\{\\mathrm\{cur\}\}\-\\bar\{U\}\_\{0\}=\(U¯oth−U¯0\)\+CAG,\\displaystyle=\(\\bar\{U\}\_\{\\mathrm\{oth\}\}\-\\bar\{U\}\_\{0\}\)\+\\mathrm\{CAG\},\(6\)CAG\\displaystyle\\mathrm\{CAG\}=\(U¯self−U¯oth\)\+SSG\.\\displaystyle=\(\\bar\{U\}\_\{\\mathrm\{self\}\}\-\\bar\{U\}\_\{\\mathrm\{oth\}\}\)\+\\mathrm\{SSG\}\.

### Component Attribution

CIC shows whether message identity changes receiver predictions, but not which components cause the change\. Starting from an other\-example message, we restore selected current\-example componentsCCwhile leaving the rest fixed\. Their contribution is summarized by

NLD​\(C\)=LDrest​\(C\)−LDothLDcur−LDoth,\\mathrm\{NLD\}\(C\)=\\frac\{\\mathrm\{LD\}\_\{\\mathrm\{rest\}\(C\)\}\-\\mathrm\{LD\}\_\{\\mathrm\{oth\}\}\}\{\\mathrm\{LD\}\_\{\\mathrm\{cur\}\}\-\\mathrm\{LD\}\_\{\\mathrm\{oth\}\}\},\(7\)
whereLD\\mathrm\{LD\}is the teacher\-forced answer log\-probability difference\. NLD near11indicates substantial restoration and NLD near0little restoration; it is evaluated only when the denominator exceeds the numerical reference floor\. Components follow the message structure, such as sequence windows or layer by region by key\-or\-value cells\.

### Inference and Validity Checks

All task\-performance contrasts are paired at the example level\. Multiple other\-example messages are averaged within each example before aggregation\. We report paired bootstrap confidence intervals together with sign\-flip or exact paired tests\. PS uses label permutations, PL and CIC use the numerical reference floor from repeated identical runs, and signed task\-level effects are compared with zero\. Practical negligibility is assessed with an equivalence test under a declared margin\.

Synthetic messages, including noise, scrambling, and shuffling, are used only as diagnostics\. Each intervention is accompanied by message\-distribution similarity and receiver\-instability measurements, and outputs with no valid parsed answer remain in the analysis with task score zero\. Three hard checks validate the implementation: masking the message must reduce identity effects to the numerical floor, replacing an other\-example message with the current\-example message must yield zero CIC and CAG, and full restoration must giveNLD=1\\mathrm\{NLD\}=1\.

## Experiments and Results

### Experimental Setup

We evaluate Qwen3\-4B and Qwen3\-8B on GSM8K\(Cobbeet al\.[2021](https://arxiv.org/html/2607.26773#bib.bib4)\), ARC\-Challenge \(ARC\-C\)\(Clarket al\.[2018](https://arxiv.org/html/2607.26773#bib.bib3)\), and MATH\-500\(Lightmanet al\.[2024](https://arxiv.org/html/2607.26773#bib.bib13)\)\. GSM8K and ARC\-C retain two benchmarks from the LatentMAS evaluation\(Zouet al\.[2026](https://arxiv.org/html/2607.26773#bib.bib28)\)and cover open\-form arithmetic reasoning and multiple\-choice science question answering, respectively\. MATH\-500 adds a lower\-accuracy setting of competition\-level mathematics, reducing the risk that ceiling performance obscures communication effects\. The paired analyses contain 100 and 60 GSM8K examples, 80 and 40 ARC\-C examples, and 60 and 40 MATH\-500 examples for the 4B and 8B models, respectively\. All runs use one NVIDIA A40 GPU with 48GB of memory, 16 Intel Xeon Gold 6338 CPU cores, and 128GB of RAM under Ubuntu 24\.04, Python 3\.12, CUDA 12\.6, PyTorch 2\.11\.0, and Transformers 5\.13\.0\.

We evaluated latent\-step countsm∈\{0,2,5,10,20,40\}m\\in\\\{0,2,5,10,20,40\\\}and usem=40m=40in the reported experiments\. This value follows the original method’s reported4040–8080high\-performance range and selects its lower\-compute endpoint; it was not selected using performance on the evaluated examples\. Across the repository, decoding was evaluated with greedy sampling andT=0\.6T=0\.6, generation caps of 256, 2048, and 3072 tokens, thinking mode enabled and disabled, and the default and aligned KV\-relay variants\. The reported experiments use nucleus sampling withT=0\.6T=0\.6and top\-p=0\.95p=0\.95, a 2048\-token generation cap, thinking mode enabled, and the default KV\-relay implementation to match the original method’s main configuration\. The number of other\-example messages was fixed atK=4K=4rather than selected through tuning\. Unless otherwise specified, the sender denotes one or more upstream agents other than the final\-answer agent, the receiver denotes the agent that produces the final answer, and the audited communication boundary is the point at which the sender\-produced message is handed to the receiver\.

![Refer to caption](https://arxiv.org/html/2607.26773v1/Figures/fig_2_mechanism_math500.png)Figure 2:MATH\-500 decomposition of the overall performance effect into the other\-example message effect and CAG\. Bars show accuracy differences and whiskers show paired 95% confidence intervals for Qwen3\-4B \(n=60n=60\) and Qwen3\-8B \(n=40n=40\)\.Each model–benchmark configuration is evaluated on a fixed example set using independently seeded decoding runs\. The Qwen3\-4B and Qwen3\-8B GSM8K results aggregate three and two seeds, respectively; the corresponding ARC\-C results aggregate two and two seeds, and both MATH\-500 configurations use one seed\. These settings yield 300, 120, 160, 80, 60, and 40 example\-draws for 4B GSM8K, 8B GSM8K, 4B ARC\-C, 8B ARC\-C, 4B MATH\-500, and 8B MATH\-500, respectively\. Within each example\-draw, the no\-message, current\-example, and self\-generated settings are each executed once, while the other\-example setting uses four independently assigned, length\-matched messages\. The four other\-example outcomes are averaged within the evaluated example before paired aggregation\. Runs across decode seeds are independent stochastic draws, whereas duplicate replays of an identical configuration reproduce the output bitwise\. Accuracy differences are reported in percentage points \(pp\) with paired 95% confidence intervals\.

### Aggregate Performance Conceals Distinct Effects

Figure[1](https://arxiv.org/html/2607.26773#Sx1.F1)\(c\) shows the GSM8K decomposition\. For Qwen3\-4B, the overall performance effect is only−1\.00\-1\.00pp, but it combines a−6\.17\-6\.17pp other\-example message effect with a\+5\.17\+5\.17pp CAG\. Both component intervals exclude zero\. For Qwen3\-8B, the overall effect remains small at\+1\.67\+1\.67pp, while the component point estimates reverse direction: the other\-example message effect is\+3\.96\+3\.96pp and CAG is−2\.29\-2\.29pp\. A near\-zero aggregate effect can therefore conceal substantial and opposing communication effects\.

MATH\-500 produces a different decomposition\. As shown in Figure[2](https://arxiv.org/html/2607.26773#Sx4.F2), the Qwen3\-4B overall performance effect is\+15\.00\+15\.00pp, comprising a\+8\.33\+8\.33pp other\-example message effect and a\+6\.67\+6\.67pp CAG\. The CAG confidence interval is\[0\.42,12\.50\]\[0\.42,12\.50\]pp\. For Qwen3\-8B, the\+10\.00\+10\.00pp overall effect is dominated by the\+8\.13\+8\.13pp other\-example message effect, while CAG is\+1\.88\+1\.88pp with an interval crossing zero\. Thus, similar overall improvements can differ in how much they depend on content generated for the evaluated example\.

ARC\-C provides a lower\-effect comparison\. For Qwen3\-4B, the−0\.63\-0\.63pp overall effect combines a−2\.97\-2\.97pp other\-example message effect with a\+2\.34\+2\.34pp CAG\. The corresponding Qwen3\-8B point estimates are0\.000\.00,−1\.56\-1\.56, and\+1\.56\+1\.56pp\. Their intervals include zero, but the decomposition again separates effects hidden by the aggregate comparison\.

### Example\-Specific Content Does Not Imply Other\-Agent Value

CAG asks whether content from the evaluated example adds task value, whereas SSG asks whether that value requires a separate sender\. Figure[3](https://arxiv.org/html/2607.26773#Sx4.F3)shows that these questions can receive different answers\. On GSM8K with Qwen3\-4B, CAG is\+5\.17\+5\.17pp, while SSG is−2\.00\-2\.00pp with an interval crossing zero\. The sender’s message therefore outperforms an other\-example message, but not the receiver’s compute\-matched self\-generated message\. On MATH\-500 with Qwen3\-4B, CAG and SSG have the same point estimate of\+6\.67\+6\.67pp, although the SSG interval crosses zero\. For Qwen3\-8B, CAG is only\+1\.88\+1\.88pp, whereas SSG is\+10\.00\+10\.00pp with a 95% interval of\[2\.50,20\.00\]\[2\.50,20\.00\]pp\. A separate agent can therefore add value even when the current\-example versus other\-example performance difference is small\. None of the displayed intervals lies entirely within the±1\\pm 1pp margin, so the data do not establish practical equivalence\.

![Refer to caption](https://arxiv.org/html/2607.26773v1/Figures/fig_3_forest_cag_ssg.png)Figure 3:CAG and SSG across GSM8K and MATH\-500\. Points show signed accuracy differences, whiskers show paired 95% confidence intervals, and the shaded region marks the±1\\pm 1pp equivalence margin\.
### Prediction Sensitivity and Task Value Are Distinct

Figure[4](https://arxiv.org/html/2607.26773#Sx4.F4)compares first\-token PL and CIC\. The measured Jensen–Shannon divergences span several orders of magnitude\. Several Qwen3\-4B runs show large first\-token responses to both message presence and message identity, whereas the plotted Qwen3\-8B CIC values are consistently much smaller than their corresponding PL values\.

This distributional pattern does not determine task performance\. MATH\-500 exhibits positive task\-level effects even in runs with small first\-token divergence, while large first\-token sensitivity on GSM8K or ARC\-C does not imply a positive overall performance effect\. PL, CIC, CAG, and SSG provide complementary rather than interchangeable evidence\.

![Refer to caption](https://arxiv.org/html/2607.26773v1/Figures/fig_4_first_token_sensitivity.png)Figure 4:First\-token prediction sensitivity to message presence \(PL\) and message identity \(CIC\)\. Each pair reports Jensen–Shannon divergence for the same model, benchmark, and decode seed; the logarithmic axis emphasizes differences across scales rather than task\-level utility\.

## Conclusion

We presented a causal audit for testing whether latent messages in LLM\-based multi\-agent systems carry information that the receiver actually uses\. By intervening at a common sender–receiver boundary, the audit separates encoded sender information, sensitivity to message presence and identity, the task value of example\-specific content, and the additional value supplied by a separate agent\. Our results show that aggregate performance alone does not identify the mechanism of latent communication\. Similar overall effects can arise from opposing components, while example\-specific content and other\-agent value can differ substantially across models and tasks\. These findings motivate evaluating latent\-communication methods through controlled message comparisons rather than a single end\-task score\.

## References

- M\. Cemri, M\. Z\. Pan, S\. Yang, L\. A\. Agrawal, B\. Chopra, R\. Tiwari, K\. Keutzer, A\. G\. Parameswaran, D\. Klein, K\. Ramchandran, M\. A\. Zaharia, J\. E\. Gonzalez, and I\. Stoica \(2025\)Why do multi\-agent LLM systems fail?\.InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diego, CA, USA, December 2\-7, 2025 / Mexico City, Mexico, November 30 \- December 5, 2025,D\. Belgrave, C\. Zhang, L\. N\. Montoya, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, N\. Chen, I\. V\. M\. Ruíz, and A\. Loaiza\-Bonilla \(Eds\.\),Cited by:[Introduction](https://arxiv.org/html/2607.26773#Sx1.p3.1),[Introduction](https://arxiv.org/html/2607.26773#Sx1.p4.1),[Table 1](https://arxiv.org/html/2607.26773#Sx2.T1.50.50.6),[Related Work](https://arxiv.org/html/2607.26773#Sx2.p5.1)\.
- P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. Tafjord \(2018\)Think you have solved question answering? Try ARC, the AI2 reasoning challenge\.CoRRabs/1803\.05457\.External Links:1803\.05457Cited by:[Experimental Setup](https://arxiv.org/html/2607.26773#Sx4.SSx1.p1.1)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. Schulman \(2021\)Training verifiers to solve math word problems\.CoRRabs/2110\.14168\.External Links:2110\.14168Cited by:[Experimental Setup](https://arxiv.org/html/2607.26773#Sx4.SSx1.p1.1)\.
- Z\. Du, R\. Wang, H\. Bai, Z\. Cao, X\. Zhu, Y\. Cheng, B\. Zheng, W\. Chen, and H\. Ying \(2026\)Enabling agents to communicate entirely in latent space\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 27106–27129\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.1248),ISBN 979\-8\-89176\-390\-6Cited by:[Introduction](https://arxiv.org/html/2607.26773#Sx1.p1.1),[Introduction](https://arxiv.org/html/2607.26773#Sx1.p2.1),[Table 1](https://arxiv.org/html/2607.26773#Sx2.T1.30.30.6),[Related Work](https://arxiv.org/html/2607.26773#Sx2.p2.1),[Setup and Audited Boundary](https://arxiv.org/html/2607.26773#Sx3.SSx1.p5.2)\.
- T\. Fu, Z\. Min, H\. Zhang, J\. Yan, G\. Dai, W\. Ouyang, and Y\. Wang \(2025\)Cache\-to\-Cache: Direct semantic communication between large language models\.CoRRabs/2510\.03215\.External Links:2510\.03215,[Document](https://dx.doi.org/10.48550/ARXIV.2510.03215)Cited by:[Table 1](https://arxiv.org/html/2607.26773#Sx2.T1.25.25.6),[Related Work](https://arxiv.org/html/2607.26773#Sx2.p2.1)\.
- S\. Hao, S\. Sukhbaatar, D\. Su, X\. Li, Z\. Hu, J\. E\. Weston, and Y\. Tian \(2025\)Training Large Language Models to Reason in a Continuous Latent Space\.InSecond Conference on Language Modeling,Cited by:[Related Work](https://arxiv.org/html/2607.26773#Sx2.p2.1)\.
- J\. Huang, M\. Li, Z\. Li, S\. Kwon, H\. Yu, and C\. Zhang \(2026\)Counterfactual graph for multi\-agent LLM calibration\.arXiv\.External Links:2605\.30653,[Document](https://dx.doi.org/10.48550/arXiv.2605.30653)Cited by:[Introduction](https://arxiv.org/html/2607.26773#Sx1.p4.1),[Table 1](https://arxiv.org/html/2607.26773#Sx2.T1.60.60.6),[Related Work](https://arxiv.org/html/2607.26773#Sx2.p5.1)\.
- N\. Jaques, A\. Lazaridou, E\. Hughes, Ç\. Gülçehre, P\. A\. Ortega, D\. Strouse, J\. Z\. Leibo, and N\. de Freitas \(2019\)Social influence as intrinsic motivation for multi\-agent deep reinforcement learning\.InProceedings of the 36th International Conference on Machine Learning, ICML 2019, 9\-15 June 2019, Long Beach, California, USA,K\. Chaudhuri and R\. Salakhutdinov \(Eds\.\),Proceedings of Machine Learning Research, Vol\.97,pp\. 3040–3049\.Cited by:[Table 1](https://arxiv.org/html/2607.26773#Sx2.T1.45.45.6),[Related Work](https://arxiv.org/html/2607.26773#Sx2.p3.1)\.
- A\. Lazaridou, A\. Peysakhovich, and M\. Baroni \(2017\)Multi\-Agent Cooperation and the Emergence of \(Natural\) Language\.InInternational Conference on Learning Representations,Cited by:[Related Work](https://arxiv.org/html/2607.26773#Sx2.p3.1)\.
- Y\. Li, Z\. An, and W\. Du \(2026a\)When less latent leads to better relay: information\-preserving compression for latent multi\-agent LLM collaboration\.arXiv\.External Links:2604\.13349,[Document](https://dx.doi.org/10.48550/arXiv.2604.13349)Cited by:[Table 1](https://arxiv.org/html/2607.26773#Sx2.T1.15.15.6),[Related Work](https://arxiv.org/html/2607.26773#Sx2.p2.1)\.
- Z\. Li, W\. Tian, J\. Chen, H\. Zhang, Y\. Liu, Y\. Ban, and F\. Zhuang \(2026b\)Counterfactual credit policy optimization for multi\-agent collaboration\.arXiv\.External Links:2603\.21563,[Document](https://dx.doi.org/10.48550/ARXIV.2603.21563)Cited by:[Introduction](https://arxiv.org/html/2607.26773#Sx1.p4.1)\.
- H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. Cobbe \(2024\)Let’s verify step by step\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024,Cited by:[Experimental Setup](https://arxiv.org/html/2607.26773#Sx4.SSx1.p1.1)\.
- Y\. Liu \(2026\)Beyond Tokens: a unified framework for latent communication in LLM\-based multi\-agent systems\.arXiv\.External Links:2606\.05711,[Document](https://dx.doi.org/10.48550/ARXIV.2606.05711)Cited by:[Introduction](https://arxiv.org/html/2607.26773#Sx1.p4.1),[Related Work](https://arxiv.org/html/2607.26773#Sx2.p2.1)\.
- Y\. Liu, Y\. Huang, J\. Yao, S\. Feng, Z\. Gu, K\. Du, H\. Li, Y\. Cheng, J\. Jiang, S\. Lu, M\. Musuvathi, and E\. Choukse \(2026\)DroidSpeak: KV Cache Sharing Across Fine\-tuned Model Variants\.InNSDI’26,Cited by:[Table 1](https://arxiv.org/html/2607.26773#Sx2.T1.20.20.6),[Related Work](https://arxiv.org/html/2607.26773#Sx2.p2.1)\.
- R\. Lowe, J\. Foerster, Y\. Boureau, J\. Pineau, and Y\. Dauphin \(2019\)On the Pitfalls of Measuring Emergent Communication\.InProceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems \- Volume 1,New York, New York,pp\. 693–701\.External Links:[Document](https://dx.doi.org/10.65109/KNVJ7743)Cited by:[Introduction](https://arxiv.org/html/2607.26773#Sx1.p3.1),[Table 1](https://arxiv.org/html/2607.26773#Sx2.T1.40.40.6),[Related Work](https://arxiv.org/html/2607.26773#Sx2.p3.1)\.
- D\. McAllester and K\. Stratos \(2020\)Formal limitations on the measurement of mutual information\.InProceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics,pp\. 875–884\.External Links:ISSN 2640\-3498Cited by:[Related Work](https://arxiv.org/html/2607.26773#Sx2.p3.1)\.
- K\. Meng, D\. Bau, A\. Andonian, and Y\. Belinkov \(2022\)Locating and Editing Factual Associations in GPT\.InAdvances in Neural Information Processing Systems 35,New Orleans, Louisiana, USA,pp\. 17359–17372\.External Links:[Document](https://dx.doi.org/10.52202/068431-1262),ISBN 978\-1\-7138\-7108\-8Cited by:[Introduction](https://arxiv.org/html/2607.26773#Sx1.p3.1),[Related Work](https://arxiv.org/html/2607.26773#Sx2.p4.1)\.
- C\. Pham, B\. Liu, Y\. Yang, Z\. Chen, T\. Liu, J\. Yuan, B\. A\. Plummer, Z\. Wang, and H\. Yang \(2023\)Let Models Speak Ciphers: Multiagent Debate through Embeddings\.InThe Twelfth International Conference on Learning Representations,Cited by:[Introduction](https://arxiv.org/html/2607.26773#Sx1.p1.1),[Introduction](https://arxiv.org/html/2607.26773#Sx1.p2.1),[Table 1](https://arxiv.org/html/2607.26773#Sx2.T1.5.5.6),[Related Work](https://arxiv.org/html/2607.26773#Sx2.p2.1),[Setup and Audited Boundary](https://arxiv.org/html/2607.26773#Sx3.SSx1.p5.2)\.
- B\. Poole, S\. Ozair, A\. V\. D\. Oord, A\. Alemi, and G\. Tucker \(2019\)On Variational Bounds of Mutual Information\.InProceedings of the 36th International Conference on Machine Learning,pp\. 5171–5180\.External Links:ISSN 2640\-3498Cited by:[Related Work](https://arxiv.org/html/2607.26773#Sx2.p3.1)\.
- X\. Shen, Y\. Liu, Y\. Dai, Y\. Wang, R\. Miao, Y\. Tan, S\. Pan, and X\. Wang \(2025\)Understanding the Information Propagation Effects of Communication Topologies in LLM\-based Multi\-Agent Systems\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,Suzhou, China,pp\. 12358–12372\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.623)Cited by:[Introduction](https://arxiv.org/html/2607.26773#Sx1.p4.1),[Table 1](https://arxiv.org/html/2607.26773#Sx2.T1.55.55.6),[Related Work](https://arxiv.org/html/2607.26773#Sx2.p5.1)\.
- J\. Song and S\. Ermon \(2020\)Understanding the limitations of variational mutual information estimators\.In8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26\-30, 2020,Cited by:[Related Work](https://arxiv.org/html/2607.26773#Sx2.p3.1)\.
- J\. Vig, S\. Gehrmann, Y\. Belinkov, S\. Qian, D\. Nevo, Y\. Singer, and S\. Shieber \(2020\)Investigating Gender Bias in Language Models Using Causal Mediation Analysis\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 12388–12401\.Cited by:[Introduction](https://arxiv.org/html/2607.26773#Sx1.p3.1),[Related Work](https://arxiv.org/html/2607.26773#Sx2.p4.1)\.
- W\. J\. Yeo, R\. Satapathy, and E\. Cambria \(2025\)Towards faithful natural language explanations: A study using activation patching in large language models\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, EMNLP 2025, Suzhou, China, November 4\-9, 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),pp\. 10425–10447\.External Links:[Document](https://dx.doi.org/10.18653/V1/2025.EMNLP-MAIN.529)Cited by:[Related Work](https://arxiv.org/html/2607.26773#Sx2.p4.1)\.
- F\. Zhang and N\. Nanda \(2024\)Towards best practices of activation patching in language models: Metrics and methods\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024,Cited by:[Related Work](https://arxiv.org/html/2607.26773#Sx2.p4.1)\.
- Y\. Zheng, Z\. Zhao, Z\. Li, Y\. Xie, M\. Gao, L\. Zhang, and K\. Zhang \(2025\)Thought communication in multiagent collaboration\.InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diego, CA, USA, December 2\-7, 2025 / Mexico City, Mexico, November 30 \- December 5, 2025,D\. Belgrave, C\. Zhang, L\. N\. Montoya, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, N\. Chen, I\. V\. M\. Ruíz, and A\. Loaiza\-Bonilla \(Eds\.\),Cited by:[Table 1](https://arxiv.org/html/2607.26773#Sx2.T1.35.35.6),[Related Work](https://arxiv.org/html/2607.26773#Sx2.p2.1)\.
- J\. Zou, R\. Qiu, G\. Li, X\. Yang, K\. Tieu, P\. Lu, K\. Shen, H\. Tong, Y\. Choi, J\. He, J\. Zou, M\. Wang, and L\. Yang \(2026\)Latent Collaboration in Multi\-Agent Systems\.InForty\-Third International Conference on Machine Learning,Cited by:[Introduction](https://arxiv.org/html/2607.26773#Sx1.p1.1),[Introduction](https://arxiv.org/html/2607.26773#Sx1.p2.1),[Table 1](https://arxiv.org/html/2607.26773#Sx2.T1.10.10.6),[Related Work](https://arxiv.org/html/2607.26773#Sx2.p2.1),[Setup and Audited Boundary](https://arxiv.org/html/2607.26773#Sx3.SSx1.p5.2),[Experimental Setup](https://arxiv.org/html/2607.26773#Sx4.SSx1.p1.1)\.