Latent Cache Flow: Model-to-Model Communication Without Text

# Latent Cache Flow: Model-to-Model Communication Without Text
Source: [https://arxiv.org/html/2605.22863](https://arxiv.org/html/2605.22863)
###### Abstract

LLM agents today communicate via text, which incurs considerable latency and information loss due to the need to autoregressively decode the sharer model’s state and encode at the receiver model\. Recent work such as Cache\-to\-Cache\(C2C; Fuet al\.,[2026](https://arxiv.org/html/2605.22863#bib.bib6)\)seeks to exchange KV caches by learning adapters that translate sharer KV matrices to the receiver model\. However, the adapters are large and expensive to train, and translate individual tokens, which requires the target context to be identical\. This is unsuitable for agent communication, where the LLMs have differing context\.

We introduce Latent Cache Flow \(LCF\)\. To address efficiency, we observe that keys and values can be jointly translated and compressed, reducing the adapter to about 4% of C2C’s size\. To address differing context, we design the adapter to transmit a summary of new information that the target model does not have\. Our early experiments show that a 13MBLCF adapter can be more accurate than a 956MBC2C adapter in shared\-context settings; for different contexts, LCF is 23% more accurate and8\.5×8\.5\\timesfaster than text\-based communication\.

Latent Cache Flow, LCF, Cache\-to\-Cache, C2C, KV Cache, Cache Communication, Cross\-Model Communication, Cross\-Context Communication, Multi\-Model Systems, LLM Communication, LLM Inference, Parameter Efficiency, Adapter Efficiency, Bottleneck Compression, Key\-Value Compression, Multi\-Head Latent Attention, Text\-to\-Text Communication

## 1Introduction

Language\-model workflows involve multiple models with different roles, tools, and contexts\(Yaoet al\.,[2023](https://arxiv.org/html/2605.22863#bib.bib1); Duet al\.,[2024](https://arxiv.org/html/2605.22863#bib.bib2); Wuet al\.,[2024](https://arxiv.org/html/2605.22863#bib.bib3); Chenet al\.,[2024](https://arxiv.org/html/2605.22863#bib.bib4)\)\. These models must communicate to combine their work, and they usually do so through generated text\. For example, one model may summarize a document chunk, and another may use that summary to answer a question over a different chunk\. This text\-mediated channel is slow because the receiver must wait for the sender to decode, and lossy because much of the sender’s internal state is compressed into discrete tokens\.

Cache\-to\-Cache \(C2C\)\(Fuet al\.,[2026](https://arxiv.org/html/2605.22863#bib.bib6); Deryet al\.,[2026](https://arxiv.org/html/2605.22863#bib.bib11)\)addresses the text bottleneck by exchanging KV caches directly\. In C2C, the sharer and receiver process the same input, and a learned adapter fuses their caches at matching token positions\. This bypasses intermediate text generation, but it makes C2C position\-wise by design\. The assumption is restrictive for deployed multi\-model systems, where models are used because they handle different sub\-tasks, tools, or context\. When their contexts differ, their token positions no longer align, and position\-wise cache exchange does not apply\. C2C also introduces substantial adapter overhead: for a 1\.1B\-parameter sharer–receiver pair, its adapter is 956MB\.

We introduce Latent Cache Flow \(LCF\) to address both limitations\. LCF treats cache transfer as compressed communication rather than token\-by\-token translation\. It replaces separate key and value fusers with a shared low\-dimensional cache channel, and uses layer pruning to remove receiver layers that do not benefit from cache updates\. We extend LCF to cross\-context communication with LCF\-X\. Instead of translating each sharer token into a receiver\-aligned cache update, LCF\-X summarizes the sharer’s full KV cache into a fixed\-size tensor that the receiver can condition on\.

Text\-to\-text \(T2T\) and C2C represent opposite extremes\. T2T is flexible because the sharer can send free\-form text, but it pays decoding latency and token\-projection loss\. C2C avoids decoding by exchanging caches at matching positions, but requires aligned inputs\. Figure[1](https://arxiv.org/html/2605.22863#S1.F1)illustrates the middle ground targeted by LCF and LCF\-X: compressed cache\-level communication that improves efficiency while relaxing the shared\-context constraint\.

![Refer to caption](https://arxiv.org/html/2605.22863v1/Efficiency-Flexibility.png)Figure 1:Conceptual efficiency–flexibility trade\-off\. LCF improves prior cache\-level transfer efficiency while relaxing C2C’s alignment constraint toward T2T\-like flexibility\.
## 2Background: Cache\-to\-Cache

The natural target for cross\-model transfer is the KV cache\. This cache is the residue of the sharer’s forward pass and holds the per\-layer key and value tensors that summarize its reading of the prompt\. However, direct cache reuse is not feasible across different architectures\. The sharer and receiver may differ in the number of layers, attention heads, head dimensions, or the geometry of the learned latent space\. As a result, the receiver cannot consume the sharer’s KV tensors verbatim\. Cache\-to\-Cache \(C2C\) addresses this challenge by treating the sharer’s cache as a learned semantic conditioning signal\. C2C trains a fuser module that takes both models’ caches as input and generates a residual update, which is added to the receiver cache before decoding\.

Figure[2](https://arxiv.org/html/2605.22863#S2.F2)illustrates the C2C fuser, which contains two independent duplicate modules for the key and value tensors\. We describe the key side; values are handled similarly\.

After the sharer and receiver process the prompt, they emit layer\-aligned multi\-head keysSKS\_\{K\}andRKR\_\{K\}\. The fuser concatenates these keys and flattens them along the head dimension to form a joint per\-token feature vector\. A linear layer then projects this vector down to the receiver’s hidden dimension\. An MLP performs feature fusion, mixing the sharer and receiver projections into a joint encoding\.

The fused features split into two parallel paths\. The projection path produces a per\-head cache updateΔK\\Delta\_\{K\}in the receiver’s\(Ht,Dh\)\(H\_\{t\},D\_\{h\}\)shape\. The dynamic weighting path produces a per\-head scalarαK\\alpha\_\{K\}that re\-weights the update\. Their element\-wise product is masked by a learned Gumbel\-sigmoid gate, a differentiable relaxation of an on/off decision that is annealed toward a binary layer\-selection at inference\. Finally, the result is added toRKR\_\{K\}as a residual, yielding the enriched cacheRK′R^\{\\prime\}\_\{K\}\.

![Refer to caption](https://arxiv.org/html/2605.22863v1/x1.png)Figure 2:C2C pipeline\. Yellow denotes projection and dimension\-changing operations; blue denotes sharer outputs, and green denotes receiver states\.
## 3Latent Cache Flow

In this section, we first describe how the design of Latent Cache Flow \(LCF\) improves the efficiency of KV transfer when the contexts match\. We then describe an extension called LCF\-X that pre\-processes the sharer model’s KV cache to support communication between differing contexts\.

### 3\.1Latent Cache Flow for Exact Context Sharing

We address two main sources of inefficiencies in C2C\. First, C2C processes keys and values independently by duplicating the fuser pipeline for each\. However, keys and values are projections of the same hidden state\. Recent architectures, such as Gemma 4\(Google,[2026](https://arxiv.org/html/2605.22863#bib.bib9)\), showcase this redundancy through unified key\-value tensors for global attention layers\. LCF replaces C2C’s dual pipelines with a single shared key\-value pipeline\.

Second, C2C keeps communication in the receiver’s high\-dimensional cache space\. It concatenates the sharer and receiver caches, fuses them at full width, and outputs receiver\-shaped residuals\. This makes transfer expensive: each cache type needs its own full\-width fuser\.

LCF instead transfers information through a low\-dimensional latent bottleneck\. This bottleneck is motivated by two observations: KV states can be compressed within a model\(DeepSeek\-AI,[2024](https://arxiv.org/html/2605.22863#bib.bib7)\), and cross\-model representations can be approximately linearly aligned\(Chenet al\.,[2025](https://arxiv.org/html/2605.22863#bib.bib8); Deryet al\.,[2026](https://arxiv.org/html/2605.22863#bib.bib11)\)\. LCF combines these observations by projecting the joint key–value representation into a compact latent space, mixing it, and up\-projecting the result back into receiver\-cache residuals\.

![Refer to caption](https://arxiv.org/html/2605.22863v1/x2.png)Figure 3:LCF/LCF\-X architecture\. Blue denotes sharer\-only states, green denotes receiver\-only states, and yellow denotes projection/compression operations\. Sharer\-side pooling enables cross\-context summaries; the shared LCF projector compresses KV inputs and up\-projects receiver\-cache residuals\.Figure[3](https://arxiv.org/html/2605.22863#S3.F3)shows the full architecture in two parts: the LCF\-X pooling module at the top and the core LCF projector at the bottom\. This section focuses on the core LCF projector, which assumes token\-aligned sharer and receiver caches\.

1\. Input\.Each receiver transformer layer receives a pair of sharer cache\(𝐒K,𝐒V\)\(\\mathbf\{S\}\_\{K\},\\mathbf\{S\}\_\{V\}\)and receiver cache\(𝐑K,𝐑V\)\(\\mathbf\{R\}\_\{K\},\\mathbf\{R\}\_\{V\}\)\.

2\. Concat & FlattenThe KV cache tensors from the Sharer and Receiver are concatenated and flattened along the head dimension into a single tensor\.

3\. Shared Latent Pipeline\.The concatenated tensor𝐱\\mathbf\{x\}is linearly down\-projected todddimensions:𝐳=Wdown​𝐱\\mathbf\{z\}=W\_\{\\mathrm\{down\}\}\\mathbf\{x\}, followed by a two\-layer MLP\. We selectd∈\{64,128,256,512\}d\\in\\\{64,128,256,512\\\}as a hyperparameter based on downstream accuracy \(Figure[4](https://arxiv.org/html/2605.22863#S4.F4)\)\.

4\. Latent split\.The latent tensor is then split into separate key and value tensors:\[𝐳K,𝐳V\]=𝐳\[\\mathbf\{z\}\_\{K\},\\mathbf\{z\}\_\{V\}\]=\\mathbf\{z\}where𝐳i∈ℝB×N×d/2\\mathbf\{z\}\_\{i\}\\in\\mathbb\{R\}^\{B\\times N\\times d/2\}\. At this point, the keys and values will be reconstructed independently\.

5\. Gates\.Per\-head gatesαi∈ℝhr\\alpha\_\{i\}\\in\\mathbb\{R\}^\{h\_\{r\}\}are produced from the full latent𝐳\\mathbf\{z\}, allowing K and V updates to be independently scaled using shared latent features\.

6\. Upward project\.The Key and Value tensors are projected back to receiver\-cache size:

Δi=Wupi​𝐳i,i∈\{K,V\}\.\\Delta\_\{i\}=W\_\{\\mathrm\{up\}\}^\{i\}\\mathbf\{z\}\_\{i\},\\quad i\\in\\\{K,V\\\}\.We adopt C2C’s per\-layer Gumbel\-sigmoid gate, producinggi∈\{0,1\}g\_\{i\}\\in\\\{0,1\\\}at inference\. The receiver cache is updated as:

𝐑i′=𝐑i\+gi⋅\(αi⊙Δi\),i∈\{K,V\}\.\\mathbf\{R\}^\{\\prime\}\_\{i\}=\\mathbf\{R\}\_\{i\}\+g\_\{i\}\\cdot\(\\alpha\_\{i\}\\odot\\Delta\_\{i\}\),\\quad i\\in\\\{K,V\\\}\.
7\. Enriched receiver KV\.The gated residuals update the receiver’s K and V caches\.

Adapter Parameters\.For a pair of 1\.1B parameter models, C2C requires a 477\.8M\-parameter adapter model\. Atd=128d=128, LCF only requires 19\.4M parameters across 28 layers \(693K per layer\)—24\.6×24\.6\\timesfewer than C2C\.

### 3\.2LCF\-X: Cross\-Context Communication

LCF reduces C2C’s adapter overhead, but it still assumes shared context: the sharer and receiver must process the same prompt so that each receiver token has a matching sharer token\. This position\-wise requirement prevents cross\-context communication\. LCF\-X removes it by replacing matched sharer tokens with a position\-free summary of what the sharer learned from its own context\. As shown in Figure[3](https://arxiv.org/html/2605.22863#S3.F3), LCF\-X adds a sharer\-side pooling module before the standard LCF projector; the receiver therefore conditions on the sharer’s learned context, not on a token\-aligned variation of its own prompt\.

We divide the sharer’s context intoPPspansS1,…,SPS\_\{1\},\\ldots,S\_\{P\}\. A span is a contiguous block of source tokens, such as a paragraph, passage, or fixed\-size chunk\. The first part of Figure[3](https://arxiv.org/html/2605.22863#S3.F3)shows this pooling step\. LCF\-X uses attention as a pooling operation: within each span, a learned query attends only to the tokens in that span and produces one key–value summary\. Across all spans, this gives

𝐊~,𝐕~∈ℝB×H×P×D\.\\tilde\{\\mathbf\{K\}\},\\tilde\{\\mathbf\{V\}\}\\in\\mathbb\{R\}^\{B\\times H\\times P\\times D\}\.ThePPsummaries keep span identity before global pooling\. LargerPPgives cross\-span attention more local summaries to weight; smallerPPgives it fewer, coarser summaries\. Because the same pooling query is reused for every span,PPcan vary at inference; one checkpoint can summarize one passage, five passages, or a sliding set of chunks\. Appendix[E\.4](https://arxiv.org/html/2605.22863#A5.SS4)gives the span\-invariance analysis\.

LCF\-X then pools across spans\. A second attention step aggregates thePPspan summaries into one per\-head sharer summary,

𝐊∗,𝐕∗∈ℝB×H×1×D\.\\mathbf\{K\}^\{\*\},\\mathbf\{V\}^\{\*\}\\in\\mathbb\{R\}^\{B\\times H\\times 1\\times D\}\.This summary is broadcast across the receiver sequence and replaces the sharer’s matched\-position cache in LCF\. The lower part of Figure[3](https://arxiv.org/html/2605.22863#S3.F3)shows that, after this replacement, the standard LCF projector and receiver cache are unchanged\.

LCF and LCF\-X address different limits of C2C\. LCF replaces C2C’s high\-dimensional, cache\-specific fusers with a compressed shared key–value pipeline\. LCF\-X removes the shared\-context assumption by replacing matched\-position sharer caches with pooled, position\-free summaries\. The next section tests whether these changes preserve cache\-transfer accuracy while reducing adapter overhead\.

## 4Experiments

We first evaluate LCF as compared to C2C in a shared\-context setting, and then evaluate LCF\-X in a cross\-context setting as compared to T2T communication, where models observe different contexts and must share novel information\.

### 4\.1Shared\-Context

#### 4\.1\.1Experimental Setup

The shared\-context setting assumes aligned token positions between models\. It tests LCF’s compression claim: whether a smaller latent fuser can match or improve C2C’s full\-width cache fusion\.

The sharer is Qwen2\.5\-0\.5B\-Instruct, and the receiver is Qwen3\-0\.6B\. We use OpenHermes 2\.5\(Teknium,[2023](https://arxiv.org/html/2605.22863#bib.bib12)\), matching C2C’s training data, and keep both base models frozen\. For C2C, we use the released full\-schedulenics\-efc/C2C\_Fusercheckpoint\. We train LCF under the same public C2C recipe, changing only the fuser architecture\. Full hyperparameter, filtering, and compute details are provided in Appendix[B](https://arxiv.org/html/2605.22863#A2)\.

We evaluate LCF’s sensitivity to bottleneck dimension withd=\{64,128,256,512\}d=\\\{64,128,256,512\\\}\. We also evaluate layer pruning, which removes receiver layers whose learned gates are near zero\. This tests whether LCF can further reduce adapter size without losing downstream accuracy\.

We evaluate on four logit\-based zero\-shot multiple\-choice benchmarks\. We use MMLU\-Redux \(MMLU\-R\), ARC\-Challenge \(ARC\-C\), and OpenBookQA \(OBQA\) from C2C’s benchmark suite, and replace C\-Eval with MMLU\-Pro for stronger English\-language coverage\. Logit\-based scoring avoids free\-form parsing noise by reading the predicted answer directly from the model’s logits over the answer choices\. We also compare with an optimal\-routing frontier \(ORF\), a per\-question oracle that selects the correct answer when either base model is correct\.

#### 4\.1\.2Training Cost

Under the matched pipeline, training time is dominated by frozen model prefills, not adapter computation\. Each optimizer step runs a sharer prefill, a receiver prefill, cache fusion, and a backward pass through the adapter\. This approach holds the communication workload fixed for both adapters, rather than optimizing around LCF\.

With this setup, LCF\-128 trains for 300 optimizer steps in about 4\.5 hours on a single Colab A100\. Each optimizer step takes 52 seconds, with an effective batch size of 260\. For comparison, our C2C run on a single A100 takes 66 seconds per optimizer step after warmup, measured with an effective batch size of 256\. We use 300 LCF steps because both the published C2C curve and our C2C reproduction achieve their best MCQ accuracy at step 250\. Later checkpoints do not improve downstream accuracy\.

The modest per\-step wall\-clock difference reflects where training time is spent\. Most computation comes from the frozen sharer and receiver forward passes, which both methods share\. The smaller LCF adapter reduces trainable parameters, adapter memory, and adapter\-side backward cost\. However, these savings are partly hidden by the frozen\-model prefill cost\. When KV caches are precomputed or reused, adapter\-side costs become more visible\. In those settings, smaller adapters may provide larger practical savings\.

#### 4\.1\.3Shared\-Context Results

Figure[4](https://arxiv.org/html/2605.22863#S4.F4)summarizes the shared\-context results, including both bottleneck scaling and layer\-pruned variants\. LCF establishes an accuracy–efficiency frontier for inter\-model communication, outperforming C2C at substantially lower adapter overhead\. The frontier is traced by the progression from LCF\-128\-9L to LCF\-128, and finally to LCF\-256\. Increasing latent capacity improves accuracy while keeping adapter size much smaller than C2C\.

![Refer to caption](https://arxiv.org/html/2605.22863v1/pareto_overview.png)Figure 4:Accuracy–efficiency frontier for shared\-context communication\. Weighted accuracy versus bf16 projector footprint\. 9L denotes the 9\-layer pruned variant\.LCF\-256 achieves the strongest overall performance, reaching a weighted average of 32\.88 across the evaluated benchmarks while using only107107MB\. LCF\-128 provides a particularly efficient operating point near the Pareto knee, achieving 31\.94 weighted accuracy with only 39 MB of overhead\. Aggressively pruned variants remain competitive, the 9\-layer configuration reaches 31\.26 weighted accuracy using only 13MB\.

The frontier also reveals both under\- and over\-capacity regimes\. LCF\-64 underperforms, suggesting insufficient communication bandwidth\. In contrast, LCF\-512 increases adapter size substantially but provides weaker accuracy than LCF\-256\. This non\-monotonic behavior suggests that the bottleneck acts as a compression mechanism and as a regularizer for communication\.

Notably, LCF\-256 exceeds the optimal routing frontier \(ORF\) on ARC\-C and MMLU\-Pro, suggesting that communication can recover information unavailable to either frozen model alone\.

Table 1:Zero\-shot logits accuracy \(%\)\. Wtd avg uses test sizes 5632/1150/500/12032\.∗p<0\.05p<0\.05vs\. C2C\.
#### 4\.1\.4Layer Pruning

Layer pruning reveals that LCF communication concentrates in a small subset of receiver layers\. We prune LCF\-128 by removing layers whose learned Gumbel\-sigmoid gates converge toward zero during training\.

Table 2:Layer pruning of LCF\-128 with C2C as reference\. Parentheses denote retained layers\. Weighted average is computed across MMLU\-R, ARC\-C, OBQA, and MMLU\-Pro\. Size reports bf16 projector footprint only; “vs\. C2C” shows the reduction in projector size relative to C2C\.As shown in Table[2](https://arxiv.org/html/2605.22863#S4.T2), performance degrades gradually as layers are removed, indicating that LCF communication is highly concentrated rather than uniformly distributed across the network\. The 19\-layer pruned model reduces projector size from 39 MB to 26 MB with almost no loss in weighted accuracy\. The 9\-layer critical subset continues to outperform C2C while requiring only 13 MB of overhead\.

The aggressive 6\-layer variant approximately matches C2C performance \(29\.41 vs\. 29\.13\) using only 8 MB of projector parameters, corresponding to a115×115\\timesreduction in overhead\. This suggests that a small subset of receiver layers accounts for most inter\-model communication gains\. Layer indices and pruning details are provided in Appendix[D](https://arxiv.org/html/2605.22863#A4)\.

### 4\.2Cross\-Context Information Transfer

#### 4\.2\.1Experimental Setup

The cross\-context setting tests whether LCF\-X can perform cache\-level transfer without being restricted to token\-aligned sharer and receiver inputs\.

We create the cross\-context setting by partitioning HotpotQA’s distractor context\(Yanget al\.,[2018](https://arxiv.org/html/2605.22863#bib.bib10)\)\. In the standard distractor setting, each question is paired with ten paragraphs: two supporting paragraphs and eight distractors\. We split these ten paragraphs into two disjoint sets of five, with each set containing one supporting paragraph and four distractors\. One set is assigned to the sharer and the other to the receiver\. This partition gives the models different contexts while preserving the original two\-hop evidence structure\. Since the receiver sees only one supporting paragraph, exact\-match answering generally requires new context from the sharer’s partition\.

Both models are Qwen3\-0\.6B instances, isolating communication effects from model capacity\. We first establish three receiver\-only baselines to measure how parametric knowledge and evidence availability affect performance: \(i\) question\-only, where the receiver answers using internal knowledge without supporting paragraphs, \(ii\) half\-context, where the receiver sees only its partition, and \(iii\) full\-context, where the receiver sees all ten paragraphs\.

We then compare T2T with LCF\-X\. In the T2T baseline, the sharer reads its partition and generates a text message, which is appended to the receiver’s input\. We evaluate T2T with communication budgets of 50, 100, 150, and 200 generated tokens\. In LCF\-X, the sharer’s KV cache is pooled into a fixed\-size cache summary and passed through the LCF projector, enabling cross\-context cache transfer without token alignment\.

We train LCF\-X directly on the partitioned HotpotQA training set\. Both the sharer and receiver are frozen; only the LCF projector and hierarchical pooling queries are trainable\. Training runs for one epoch, 276 optimizer steps, with an effective batch size of 256\. The process takes roughly 75 minutes on a single NVIDIA RTX PRO 6000 Blackwell GPU\. Full training details are provided in Appendix[E](https://arxiv.org/html/2605.22863#A5)\.

We evaluate answer quality on the HotpotQA\-bridge \(n=5,899n=5\{,\}899\) evaluation split\. We report Exact Match \(EM\), which measures whether the predicted answer matches the ground truth string exactly\. We also report token\-level F1, which measures partial overlap between predicted and reference answers\. We additionally report latency in terms of time\-to\-first\-token \(TTFT\) and time\-to\-end\-of\-answer \(TTEoA\)\.

#### 4\.2\.2Cross\-Context Results

LCF\-X improves both quality and latency in the cross\-context setting\. On HotpotQA\-bridge, LCF\-X reaches 35\.13 F1 and 25\.28 EM, outperforming the strongest T2T baseline, T2T Max 200, by \+2\.47 F1 and \+4\.75 EM\. These gains are statistically robust under both paired F1 and exact\-match tests \(Wilcoxonp=5\.42×10−5p=5\.42\\times 10^\{\-5\}; McNemarp=4\.23×10−12p=4\.23\\times 10^\{\-12\}\)\.

ProtocolF1EMTTFTTTEoA\(ms\)\(ms\)ReceiverQuestion\-only7\.481\.3711129Half context24\.6214\.4814104Full context29\.9816\.7718117T2TMax 5030\.8118\.83250342Max 10031\.6819\.75290381Max 15032\.5820\.46327419Max 20032\.6620\.53410502LCFLCF\-X35\.1325\.284894Table 3:Cross\-context results on HotpotQA\-bridge \(n=5,899n\{=\}5\{,\}899\)\. TTFT is time to first token; TTEoA is time to end of answer\.The receiver\-only baselines show that evidence availability matters\. Moving from question\-only to half\-context raises F1 from 7\.48 to 24\.62\. Providing full context raises it further to 29\.98\. Both T2T and LCF\-X exceed the full\-context receiver baseline\. This suggests that communication recovers missing paragraphs and changes how evidence reaches the receiver\. Information now arrives through the sharer’s forward pass, either as generated text or as a pooled cache summary\.

The key comparison is therefore between communication methods\. Increasing the T2T budget from 50 to 200 tokens improves F1 from 30\.81 to 32\.66, but increases TTFT from 250 ms to 410 ms\. T2T buys modest quality gains with additional decoding\. LCF\-X avoids this trade\-off: it reaches higher quality than every T2T budget while reducing TTFT to 48 ms and TTEoA to 94 ms\. Because TTEoA also depends on the generated answer length, we treat TTFT as a cleaner measure of communication overhead and report TTEoA as the end\-to\-end response time\.

![Refer to caption](https://arxiv.org/html/2605.22863v1/LCF_Pareto.png)Figure 5:Quality–latency trade\-off on HotpotQA\-bridge\.Figure[5](https://arxiv.org/html/2605.22863#S4.F5)summarizes the quality–latency frontier\. LCF\-X lies above and to the left of the T2T sweep, indicating higher answer quality at lower communication latency\.

## 5Limitations and Future Work

While these results are promising, they are an initial validation of LCF\. Our shared\-context experiments use one model pair, Qwen2\.5\-0\.5B\-Instruct to Qwen3\-0\.6B, and our cross\-context experiments use two Qwen3\-0\.6B instances\. Although C2C works across model families and larger scales\(Fuet al\.,[2026](https://arxiv.org/html/2605.22863#bib.bib6)\), we have not yet tested whether LCF preserves this behavior\.

Our asymmetric communication setting is also deliberately controlled\. Partitioned HotpotQA isolates whether information can move through the latent channel, but richer agentic workflows may involve longer contexts, tool outputs, heterogeneous roles, and multi\-turn communication\.

## 6Conclusion

Multi\-model language workflows need communication channels that are fast, lightweight, and compatible with distinct model contexts\. Text\-to\-text communication is flexible but sequential and lossy\. Prior cache\-based methods avoid text generation, but they require shared, token\-aligned inputs and large adapters for each model pair\.

Latent Cache Flow \(LCF\) shows that cache communication does not need full\-width cache transfer\. In the shared\-context setting, LCF matches or outperforms C2C across all four benchmarks with a13​MB13~\\mathrm\{MB\}adapter, compared with C2C’s956​MB956~\\mathrm\{MB\}adapter\. This result shows that cache information can flow from one model to another through a latent channel\.

LCF\-X then moves cache communication toward greater context flexibility\. On partitioned HotpotQA, LCF\-X transfers information across distinct contexts, improves exact\-match accuracy by23%23\\%over text communication, and reduces time\-to\-first\-token by8\.5×8\.5\\times\. Together, these results move inter\-model communication toward a different interface: cache\-level rather than textual, compressed rather than full\-width, and flexible beyond token\-aligned contexts\.

## References

- A\. Chen, J\. Merullo, A\. Stolfo, and E\. Pavlick \(2025\)Transferring linear features across language models with model stitching\.InAdvances in Neural Information Processing Systems,Cited by:[§3\.1](https://arxiv.org/html/2605.22863#S3.SS1.p3.1)\.
- W\. Chen, Y\. Su, J\. Zuo, C\. Yang, C\. Yuan, C\. Chan, H\. Yu, Y\. Lu, Y\. Hung, C\. Qian, Y\. Qin, X\. Cong, R\. Xie, Z\. Liu, M\. Sun, and J\. Zhou \(2024\)AgentVerse: facilitating multi\-agent collaboration and exploring emergent behaviors\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2605.22863#S1.p1.1)\.
- DeepSeek\-AI \(2024\)DeepSeek\-V2: a strong, economical, and efficient mixture\-of\-experts language model\.arXiv preprint arXiv:2405\.04434\.Cited by:[§3\.1](https://arxiv.org/html/2605.22863#S3.SS1.p3.1)\.
- L\. M\. Dery, Z\. Yahav, H\. Prior, Q\. Feng, J\. Shen, and A\. Szlam \(2026\)Latent space communication via k\-v cache alignment\.arXiv preprint arXiv:2601\.06123\.External Links:[Link](https://arxiv.org/abs/2601.06123),2601\.06123Cited by:[§1](https://arxiv.org/html/2605.22863#S1.p2.1),[§3\.1](https://arxiv.org/html/2605.22863#S3.SS1.p3.1)\.
- Y\. Du, S\. Li, A\. Torralba, J\. B\. Tenenbaum, and I\. Mordatch \(2024\)Improving factuality and reasoning in language models through multiagent debate\.InInternational Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2605.22863#S1.p1.1)\.
- T\. Fu, Z\. Min, H\. Zhang, J\. Yan, G\. Dai, W\. Ouyang, and Y\. Wang \(2026\)Cache\-to\-cache: direct semantic communication between large language models\.InInternational Conference on Learning Representations,Cited by:[Appendix A](https://arxiv.org/html/2605.22863#A1.p2.1),[§1](https://arxiv.org/html/2605.22863#S1.p2.1),[§5](https://arxiv.org/html/2605.22863#S5.p1.1)\.
- Google \(2026\)Gemma 4 model card\.Note:[https://ai\.google\.dev/gemma/docs/core/model\_card\_4](https://ai.google.dev/gemma/docs/core/model_card_4)Accessed: 2026\-05\-02Cited by:[§3\.1](https://arxiv.org/html/2605.22863#S3.SS1.p1.1)\.
- Teknium \(2023\)OpenHermes 2\.5: an open dataset of synthetic data for generalist llm assistants\.HuggingFace\.External Links:[Link](https://huggingface.co/datasets/teknium/OpenHermes-2.5)Cited by:[§4\.1\.1](https://arxiv.org/html/2605.22863#S4.SS1.SSS1.p2.1)\.
- Q\. Wu, G\. Bansal, J\. Zhang, Y\. Wu, B\. Li, E\. Zhu, L\. Jiang, X\. Zhang, S\. Zhang, J\. Liu, A\. H\. Awadallah, R\. W\. White, D\. Burger, and C\. Wang \(2024\)AutoGen: enabling next\-gen LLM applications via multi\-agent conversation framework\.InFirst Conference on Language Modeling \(COLM\),Cited by:[§1](https://arxiv.org/html/2605.22863#S1.p1.1)\.
- Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. W\. Cohen, R\. Salakhutdinov, and C\. D\. Manning \(2018\)HotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,pp\. 2369–2380\.External Links:[Document](https://dx.doi.org/10.18653/v1/D18-1259)Cited by:[§4\.2\.1](https://arxiv.org/html/2605.22863#S4.SS2.SSS1.p2.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao \(2023\)ReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2605.22863#S1.p1.1)\.

## Appendix AFull Results Overview

Table[4](https://arxiv.org/html/2605.22863#A1.T4)lists all methods evaluated in this paper\. T2T is a cited baseline from C2C\. We do not re\-evaluate it\. ORF is the optimal\-routing frontier\. This is a per\-question oracle that picks whichever base model gives the correct answer\. ORF can therefore be seen as a retrospective optimal routing system\.

Table 4:Full results overview\. Accuracy is reported in percent\. Weighted average uses test set sizes: MMLU\-R \(n=5632n=5632\), ARC\-C \(n=1150n=1150\), OBQA \(n=500n=500\), and MMLU\-Pro \(n=12032n=12032\)\. Size reports bf16 deployment footprint of the projector only\. Significance markers are vs\. C2C under the one\-sided exact paired McNemar test:p∗<0\.05\{\}^\{\*\}p<0\.05,p∗∗<0\.01\{\}^\{\*\*\}p<0\.01,p∗⁣∗∗<0\.001\{\}^\{\*\*\*\}p<0\.001\. Bold denotes the best fuser result per column\.†T2T numbers are fromFuet al\.\([2026](https://arxiv.org/html/2605.22863#bib.bib6)\)\. We do not re\-evaluate them\. C2C does not report T2T on MMLU\-Pro\. No weighted average is computed\.

‡LCF\-128\-9L is the 9\-layer critical pruned variant\. It keeps 9 of 28 adapter layers\. Total trainable parameters: 6\.24M\. This is a 76\.7×\\timesreduction versus C2C’s 477\.8M fuser\.

All rows except T2T use our logits\-based evaluation pipeline\. The same prompt template and alignment strategy are used\. Significance testing is by per\-question join\. Receiver and sharer are bare frozen models\. C2C uses the publishednics\-efc/C2C\_Fusercheckpoint\. LCF rows use our best checkpoints from bottleneck and pruning sweeps\.

## Appendix BTraining Details and C2C Comparison Protocol

To isolate the effect of the fuser architecture, we match C2C on the frozen model pair, training objective, OpenHermes data, and MCQ evaluation\. C2C uses the published full\-schedulenics\-efc/C2C\_Fusercheckpoint\. LCF is trained under the matched protocol below\. The main text reports the corresponding wall\-clock and per\-step training costs\.

Across LCF variants, the only architectural variable is latent width; this width also sets the 4×\\timesintermediate MLP dimension shown in Table[5](https://arxiv.org/html/2605.22863#A2.T5)\. All variants use the same data, optimizer, schedule, hardware, and seed\. We adjust per\-device batch size and gradient accumulation to keep VRAM under 80GB on a single A100, while keeping effective batch size at260±4260\\pm 4\.

### B\.1Matched Training Protocol

SettingLCF\-64LCF\-128LCF\-256LCF\-512Latent dim64128256512Intermediate dim25651210242048MLP layers2222Params / layer282K693K1\.9M5\.9MTotal trainable7\.9M19\.4M53\.4M165\.5MCompression vs\. joint 230436×\\times18×\\times9×\\times4\.5×\\timesPer\-device batch101098Grad\-accum steps26262933Effective batch260260261264Table 5:LCF training settings by bottleneck width\. The architectural variable is latent width and its corresponding4×4\\timesintermediate MLP width\. “Compression vs\. joint” is the ratio of the pre\-bottleneck joint KV dimension \(4​H​D=23044HD=2304\) to the latent dimdd\.Table 6:Settings held constant across all LCF runs\.##### Training cost\.

Under this matched pipeline, LCF\-128 takes roughly 52 seconds per optimizer step on a single Colab A100, with an effective batch size of 260\. C2C takes roughly 66 seconds per step at batch size 256\. The modest wall\-clock difference reflects that both methods spend most training time in frozen sharer and receiver forward passes\. LCF’s size reduction appears primarily as lower trainable\-parameter and adapter\-memory cost, not as a proportional wall\-clock speedup\.

##### Why 300 steps\.

C2C trains for one full OpenHermes epoch \(1929 optimizer steps\)\. In both the published C2C curve and our re\-run, best MCQ accuracy is reached by step 300\. Later checkpoints do not help and can degrade results\. LCF shows the same pattern: all bottleneck widths peak between steps 200 and 300\. We therefore train LCF ablations for 300 steps\. The C2C baseline in our tables is not a 300\-step re\-run, but the published full\-schedule checkpoint\. This makes the comparison conservative: we compare 300\-step LCF against full\-schedule C2C\.

### B\.2Eval\-Loss Convergence

Held\-out OpenHermes loss converges quickly for all bottleneck widths\. Table[7](https://arxiv.org/html/2605.22863#A2.T7)reports next\-token prediction loss on the held\-out 1% split every 50 steps\.

Table 7:Held\-out OpenHermes NTP loss across LCF bottleneck widths\.LCF\-64 through LCF\-256 converge to within 0\.006 nats of each other\. This holds despite a 7×\\timesdifference in trainable parameters\. LCF\-512 reaches the lowest NTP loss, but this does not give the best downstream accuracy\. It underperforms LCF\-256 on three of four benchmarks\. Validation NTP loss is not a reliable proxy for fuser quality here\. Bottleneck selection requires direct downstream evaluation\.

### B\.3Training\-Curve Summary

For all bottleneck widths, train and eval loss track closely\. There is no persistent train/eval gap\. Larger bottlenecks converge faster at first, but downstream accuracy does not monotonically follow NTP loss\. This supports the main ablation result: for this model pair and protocol, useful latent communication peaks atd=256d=256\. More projector capacity does not always help\.

## Appendix CStatistical Significance

This appendix reports every significance test in the paper\. We use a one\-sided exact paired McNemar test on per\-question correctness, with alternativeH1:LCF\>C2CH\_\{1\}:\\mathrm\{LCF\}\>\\mathrm\{C2C\}\. Outputs are paired by joining method CSVs on\(subject, question\_id, true\_answer\)\.

For each LCF variant and benchmark, letbbcount questions where LCF is correct and C2C is wrong, andcccount the reverse\. Concordant pairs are uninformative\. Conditional onn=b\+cn=b\+cunderH0H\_\{0\},b∼Binomial​\(n,0\.5\)b\\sim\\mathrm\{Binomial\}\(n,0\.5\), giving the one\-sided exact p\-value

p=Pr⁡\[Binomial​\(n,0\.5\)≥b\]\.p=\\Pr\[\\mathrm\{Binomial\}\(n,0\.5\)\\geq b\]\.We use the exact binomial test rather than theχ2\\chi^\{2\}approximation because some discordant counts are small\. Significance markers are uncorrected:p∗<0\.05\{\}^\{\*\}p<0\.05,p∗∗<0\.01\{\}^\{\*\*\}p<0\.01,p∗⁣∗∗<0\.001\{\}^\{\*\*\*\}p<0\.001\.

### C\.1Main LCF Variants

Table 8:Accuracy \(%\) and one\-sided exact paired McNemar significance vs\. C2C\. Test sizes: MMLU\-Rn=5632n=5632, ARC\-Cn=1150n=1150, OBQAn=500n=500, MMLU\-Pron=12032n=12032\. Markers are uncorrected\.The strongest pattern appears on the two largest benchmarks\. All LCF variants withd≥128d\\geq 128beat C2C on MMLU\-Redux and MMLU\-Pro atp<0\.001p<0\.001\. On ARC\-C and OBQA, test sets are smaller\. Gains of 1 or 2 points remain positive but are often not significant\. LCF\-256’s ARC\-C gain clearsp<0\.01p<0\.01\. LCF\-512’s OBQA gain clearsp<0\.05p<0\.05\.

### C\.2Pruned LCF\-128 Variants

Table 9:One\-sided exact paired McNemar p\-values for pruned LCF\-128 variants vs\. C2C\.The 19\-layer and 9\-layer pruned variants remain significant on MMLU\-Redux and MMLU\-Pro, while ARC\-C and OBQA do not clearp<0\.05p<0\.05\. The 6\-layer variant does not significantly outperform C2C\.

All p\-values are computed from paired per\-question CSV outputs joined on\(subject, question\_id, true\_answer\)usingscipy\.stats\.binomtestwithalternative="greater"\.

## Appendix DLayer Pruning

LCF uses one independent projector per receiver layer\. For Qwen3\-0\.6B, this means 28 projectors\. Each contributes a residual KV edit at a specific receiver layer\. This design makes LCF amenable to post\-hoc layer pruning\. A trained subset of projectors can be retained at evaluation time without retraining\. Dropped projectors simply produce no edit at their layer\.

Unlike conventional weight pruning, LCF pruning is structural\. Entire layer\-projectors are removed, not individual weights\. Because projectors are independent across layers, dropping one is equivalent to setting its gate\-times\-residual contribution to zero\. The remaining projectors operate exactly as they did during training\.

##### Parameter accounting\.

LCF\-128 trains 28 projectors of about 693K parameters each, for a total of 19\.4M trainable parameters\. C2C trains 28 fuser layers totaling 477\.8M parameters for the same model pair\. KeepingKKLCF\-128 projectors gives roughlyK⋅693​KK\\cdot 693\\mathrm\{K\}trainable parameters\. AtK=9K=9, this is 6\.24M, or76\.7×76\.7\\timesfewer than C2C\.

##### Pruning method\.

Layers are selected in three passes\. First, a gate audit removes layers with non\-positive trained Gumbel\-sigmoid gate logits for both K and V, as these contribute zero signal under the hard evaluation gate\. Second, layers that are net harmful by single\-layer ablation are removed\. Third, the remaining layers are ranked by ablation importance, and the top\-KKlayers are kept\.

On the trained LCF\-128 checkpoint, six layers are dead by the gate audit:

\{0,1,2,3,5,12\}\.\\\{0,1,2,3,5,12\\\}\.Three more are harmful by ablation:

The top\-9 retained layers are:

\{6,8,9,10,15,16,17,19,21\}\.\\\{6,8,9,10,15,16,17,19,21\\\}\.No additional fine\-tuning is done after pruning\.

Table 10:LCF\-128 layer pruning\.KKis the number of retained projectors\. Parameter counts are projector\-only\.The top\-9 configuration is the main pruned result\. With only 6\.24M trainable parameters, it remains above C2C on all four benchmarks:\+3\.11\+3\.11on MMLU\-Redux,\+0\.70\+0\.70on ARC\-C,\+0\.60\+0\.60on OBQA, and\+1\.88\+1\.88on MMLU\-Pro\. The gains on MMLU\-Redux and MMLU\-Pro are statistically significant under the one\-sided exact McNemar test in Appendix[C](https://arxiv.org/html/2605.22863#A3)\. The top\-9 configuration is the main pruned result\. With only 6\.24M trainable parameters, it remains above C2C on all four benchmarks:\+3\.11\+3\.11on MMLU\-Redux,\+0\.70\+0\.70on ARC\-C,\+0\.60\+0\.60on OBQA, and\+1\.88\+1\.88on MMLU\-Pro\. The gains on MMLU\-Redux and MMLU\-Pro are statistically significant under the one\-sided exact McNemar test in Appendix[C](https://arxiv.org/html/2605.22863#A3)\.

##### Why pruning works\.

The trained gates already disable several layers, and ablations show some active layers are net harmful\. Edits at one receiver layer propagate through later layers via the residual stream, so adjacent edits can have correlated effects\. The pruning sweep suggests most useful LCF\-128 signal is concentrated in a mid\-to\-deep subset of receiver layers\.

##### Limitations\.

The retained layer indices are specific to the trained Qwen2\.5\-0\.5B\-Instruct→\\toQwen3\-0\.6B checkpoint\. The pruning method generalizes, but the selected layers may differ for other model pairs, training runs, or tasks\.

## Appendix ELCF\-X Training Details

LCF\-X is trained on HotpotQA\-bridge with the same joint\-KV bottleneck projector as LCF\-128, but includes two changes: a hierarchical attention pool \(see Section[3\.2](https://arxiv.org/html/2605.22863#S3.SS2)\) and a gate\-logit initialization of\+1\.0\+1\.0\. Both Qwen3\-0\.6B models remain frozen\. Only the LCF projector and the two pool queries per layer,qbase\(ℓ\)q\_\{\\text\{base\}\}^\{\(\\ell\)\}andqlayer\(ℓ\)q\_\{\\text\{layer\}\}^\{\(\\ell\)\}, are updated\.

The agent setting differs from the base LCF in two ways\. First, we use a symmetric Qwen3\-0\.6B/Qwen3\-0\.6B pair because both models must process partial context independently\. Second, training is task\-specific: LCF\-X is trained directly on HotpotQA\-bridge, so the latent channel learns to route partitioned multi\-hop evidence\.

### E\.1Hyperparameters

Table 11:LCF\-X training hyperparameters\.Training partitions the sharer’s input using HotpotQA’s five\-paragraph structure, soP=5P=5during training\. The architecture is partition\-agnostic:PPis not a parameter shape and need not remain fixed at inference; see Section[E\.4](https://arxiv.org/html/2605.22863#A5.SS4)\.

We initialize the gate logit to\+1\.0\+1\.0instead of0\.00\.0\. In early LCF\-X runs, zero\-initialized gates stayed near sigmoid\(0\)=0\.5\(0\)=0\.5because the residual branch was small, producing weak gate gradients\. Setting it to\+1\.0\+1\.0gives the residual branch a meaningful gain at step 0, sigmoid\(1\.0\)=0\.73\(1\.0\)=0\.73, while preserving trainability\.

### E\.2Training Trajectory

We log dev F1/EM every 50 steps during training\. Validation metrics are reported on the held\-out HotpotQA\-bridge split \(n=5,899n\{=\}5\{,\}899\) at the final checkpoint\.

Table 12:LCF\-X training trajectory on the 2,000\-item development holdout\. Final evaluation is run once on HotpotQA\-bridge val using the final checkpoint\.After warmup, performance improves steadily, with the best checkpoint at the end of the single\-epoch run\. The span\-invariance check in Section[E\.4](https://arxiv.org/html/2605.22863#A5.SS4)rules out dependence on the specific five\-paragraph partition used during training\.

We use the final checkpoint for evaluation\. On HotpotQA\-bridge val \(n=5,899n\{=\}5\{,\}899\), the final checkpoint achieves 35\.13 F1 and 25\.28 EM\.

### E\.3Architectural Cost

The LCF\-X pool adds two query vectors per receiver layer: a base queryqbase\(ℓ\)∈ℝH×dq\_\{\\text\{base\}\}^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{H\\times d\}for the first pooling stage and a layer queryqlayer\(ℓ\)∈ℝH×dq\_\{\\text\{layer\}\}^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{H\\times d\}for the second pooling stage:

\|qbase\(ℓ\)\|\+\|qlayer\(ℓ\)\|=2​H​d\.\|q\_\{\\text\{base\}\}^\{\(\\ell\)\}\|\+\|q\_\{\\text\{layer\}\}^\{\(\\ell\)\}\|=2Hd\.For Qwen3\-0\.6B, withH=8H\{=\}8KV heads andd=128d\{=\}128,

2⋅8⋅128=2,0482\\cdot 8\\cdot 128=2\{,\}048parameters per layer, totaling57,34457\{,\}344across the 28\-layer receiver\. This is0\.22%0\.22\\%of the LCF\-X projector’s25\.8725\.87M trainable parameters and about0\.005%0\.005\\%of the frozen two\-model system\.

Critically, the pool size is independent ofPP: the same two queries are reused for every source span, regardless of how many spans the deployment chooses\. The pool, therefore, supports variable span counts without retraining or changing parameter shapes\.

### E\.4Span\-Scheme Invariance

LCF\-X is trained with the natural HotpotQA five\-paragraph partition\. Because the pool has noPP\-dependent parameters, the same trained checkpoint admits different partition schemes at inference\. We test this directly by holding the trained weights fixed and evaluating on HotpotQA\-bridge validation under paragraph\-, token\-, and sentence\-level span schemes\.

Table 13:Inference\-time span\-scheme ablations on a single trained LCF\-X checkpoint\. The span scheme varies across rows; trained weights are identical\.F1 stays within a0\.320\.32\-point band of the natural\-paragraph reference, even across a4\.7×4\.7\\timesspan\-count range and three granularities\. The best result comes from 200\-token windows with no overlap, improving F1 by\+0\.32\+0\.32and EM by\+0\.22\+0\.22\. All schemes use the same 55\.1 GB VRAM footprint, indicating that changing the span partition changes the source summary structure without changing the deployed checkpoint\.

At inference time,PPcan be chosen to match the input structure, using retrieved passages, sentence chunks, sliding windows, or a single span\. No retraining or deployment\-specific fine\-tuning is needed\.