Memory Is Communication: The Frontier Between Remembering and Signaling

# Memory Is Communication:The Frontier Between Remembering and Signaling
Source: [https://arxiv.org/html/2608.17053](https://arxiv.org/html/2608.17053)
Yashar TalebiradEden RedmanAffiliation:Network for Applied Technology, Edmonton, Canadatalebira@ualberta\.ca, eden@nat\.ltdAli ParsaeeAffiliation:Alberta Machine Intelligence Institute, University of Alberta, Edmonton, CanadaOsmar R\. ZaïaneAffiliation:Alberta Machine Intelligence Institute, University of Alberta, Edmonton, Canada

###### Abstract

A bounded agent may obtain information for a decision from its own past, from peers, or from both sources\. Retaining task\-relevant history can reduce later communication, while a peer message can supply what memory lacks\. Under limits on both resources, how should an agent allocate its information budget? Given a fixed task and decision rule, the memory and message rate pairs attaining a performance threshold form an achievable region under specified rules for using history and peer observations\. We call its efficient boundary the remembering–signaling frontier\. Across conditions where history permits the same maximum reduction in task loss, we hypothesize that a bounded agent will need less peer communication when it obtains a larger loss reduction from history\. In preliminary referential games, target repetition coincided with shorter successful messages, while predictability from a hidden cyclic rule did not shorten them\. Experiments varying memory and message rates can estimate the frontier and test this prediction across cooperative tasks\.

## Introduction

Cooperative agents often act from partial observations under limits on memory and communication bandwidth\. Existing protocols optimize which agents communicate, what they send, and to whom\([9](https://arxiv.org/html/2608.17053#bib.bib4)\), while local history can contain information a peer would otherwise transmit\. Memory has been described as communication between temporally separated selves\([2](https://arxiv.org/html/2608.17053#bib.bib5)\), and we use*memory is communication*for cases where retained history can replace peer input at fixed task performance\. When accumulated history exceeds what an agent can process for a single decision, hierarchical memory groups and compresses past information for retrieval under a finite input budget\([7](https://arxiv.org/html/2608.17053#bib.bib10)\)\. Continual learning faces the related problem of preserving usable experience as new data arrive\([4](https://arxiv.org/html/2608.17053#bib.bib9)\)\. Rate\-distortion theory relates stored or transmitted bits to permitted error\([6](https://arxiv.org/html/2608.17053#bib.bib2)\), and Wyner–Ziv coding shows how information already at a receiver can reduce the bits needed to meet an error target\([10](https://arxiv.org/html/2608.17053#bib.bib3)\)\. We formulate this allocation for a fixed task and decision rule, specifying which historical records and peer observations may be used\.

## Framework and Hypotheses

Fix a taskTT, including its instance distribution and loss, and let the decoderAAbe the decision rule mapping the decision maker’s current observation and supplied representations to the task output\. The source rules𝗌𝗋𝖼\\mathsf\{src\}identify the decision maker and its peers, and state which records from the decision maker’s past and which peer observations may be encoded\. They also specify when these sources are available and exclude future records or other information unavailable to the decision maker\. Before evaluation, specify a familyΠA​\(𝗌𝗋𝖼\)\\Pi\_\{A\}\(\\mathsf\{src\}\)of allowed encoding schemes\. Eachπ∈ΠA​\(𝗌𝗋𝖼\)\\pi\\in\\Pi\_\{A\}\(\\mathsf\{src\}\)describes how allowed history is retained and encoded asMπM\_\{\\pi\}, and how allowed peer observations are encoded as a messageCπC\_\{\\pi\}\. StoringMπM\_\{\\pi\}remotely still counts toward the memory rate, and the earlier transmission of a cachedCπC\_\{\\pi\}still counts toward communication\. Either encoded output may be empty, allowing a scheme to use one source, both sources, or neither\. LetDA​\(π,T\)D\_\{A\}\(\\pi;T\)be the expected task loss whenAAreceives the decision maker’s current observation,MπM\_\{\\pi\}, andCπC\_\{\\pi\}\. LetRm​\(π\)R\_\{m\}\(\\pi\)andRc​\(π\)R\_\{c\}\(\\pi\)be the expected encoded lengths ofMπM\_\{\\pi\}andCπC\_\{\\pi\}, measured in bits per decision\. The achievable memory–communication rate region is

ℛAMC,ε\\displaystyle\\mathcal\{R\}\_\{A\}^\{\\mathrm\{MC\},\\varepsilon\}=\{\(bm,bc\)∈ℝ\+2:\\displaystyle=\\\{\(b\_\{m\},b\_\{c\}\)\\in\\mathbb\{R\}\_\{\+\}^\{2\}:\(1\)∃π∈ΠA\(𝗌𝗋𝖼\),DA\(π;T\)≤ε,\\displaystyle\\exists\\pi\\in\\Pi\_\{A\}\(\\mathsf\{src\}\),\\quad D\_\{A\}\(\\pi;T\)\\leq\\varepsilon,Rm\(π\)≤bm,Rc\(π\)≤bc\}\.\\displaystyle R\_\{m\}\(\\pi\)\\leq b\_\{m\},\\quad R\_\{c\}\(\\pi\)\\leq b\_\{c\}\\\}\.The region depends on the decoder, task, source rules, and scheme family, and contains each budget pair under which an allowed scheme reaches expected loss at mostε\\varepsilon\. The Pareto\-efficient lower boundary contains allocations where reducing either rate would require increasing the other or exceeding the loss threshold\. We call this boundary the*remembering–signaling frontier*\. Letcm,cc\>0c\_\{m\},c\_\{c\}\>0be the respective per\-bit costs of memory and communication\. Any region point minimizingcm​bm\+cc​bcc\_\{m\}b\_\{m\}\+c\_\{c\}b\_\{c\}lies on the frontier\.

Our formulation counts the bits used by the historical representation as well as the peer message, and asks whether the loss reduction a bounded decoder obtains from history predicts the decrease in message rate at the same task loss\. On held\-out task instances, the decoder is evaluated under three input conditions at rates fixed in advance\. The first includes only the decision maker’s current observation, the second adds the history representation, and the third adds the peer message\. The loss reduction after adding history is the*usable\-history gain*, and the additional reduction after adding peer input is the*usable\-peer gain*\. The addition order affects both gains, so we keep the source order, rates, and loss measure fixed across comparisons\. A Bayes\-optimal decoder has the lowest possible expected loss from the supplied inputs under the task distribution\. Among conditions where it gains equally from history, we hypothesize that the learned decoder will need fewer message bits to reach the target loss when its usable\-history gain is larger\.

The second hypothesis asks whether measurements that vary one rate at a time can predict the least\-cost allocation when memory and message rates vary together\. Before mapping the full region, loss is measured as memory rate increases with no peer message, and then as message rate increases while memory rate is held at a value fixed in advance\. We predict that these two curves identify the memory–message split minimizingcm​bm\+cc​bcc\_\{m\}b\_\{m\}\+c\_\{c\}b\_\{c\}when both rates vary\. As targets repeat more often or states remain unchanged for longer, the least\-cost split may shift from using more message bits to using more memory bits\. We call the point at which this shift occurs the*crossover*and predict it from the two curves before varying both rates together\.

## Preliminary Results

To probe the relation between target predictability and message length, we varied the target process in a Lewis signaling game\([3](https://arxiv.org/html/2608.17053#bib.bib1)\)adapted from[8](https://arxiv.org/html/2608.17053#bib.bib8), using sender–receiver pairs powered by[instruction\-tuned Gemma 4 31B](https://huggingface.co/google/gemma-4-31B-it)\. On each round, a sender saw the designated target among four rendered shapes drawn from eight color–shape–size combinations and sentL∈\{1,2,3\}L\\in\\\{1,2,3\\\}symbols over\{A,B,C\}\\\{A,B,C\\\}to a receiver, allowing3L3^\{L\}messages, equivalent toL​log2​3L\\log\_\{2\}3bits per round\. The receiver saw the same shapes and guessed the target\. The symbols had no assigned meanings, and feedback revealed the target and outcome after each guess\. Each agent retained a private notebook and the previous twenty interactions\. For each condition and seed, separate pairs played forty rounds forL=1,2,3L=1,2,3\. We recorded the smallestLLwhose pair achieved accuracy≥0\.85\\geq 0\.85over the final ten rounds asLminL\_\{\\min\}\(chance=0\.25=0\.25\)\.

In the repeat condition, the next target copied the previous target with probabilityppand was otherwise sampled uniformly\. Across two batches of three seeds, meanLminL\_\{\\min\}fell monotonically from2\.672\.67atp=0p=0to1\.001\.00atp=0\.95p=0\.95\. Because episode history was never removed while learned symbol mappings were preserved, these runs do not isolate history’s contribution\.

Table 1:MeanLminL\_\{\\min\}over three seeds\. Repeat usesp=\(0,0\.5,0\.8,0\.95\)p=\(0,0\.5,0\.8,0\.95\)and rotation usesp=\(0,0\.5,0\.85,1\)p=\(0,0\.5,0\.85,1\)\.Under shuffled rotation, the eight targets formed a fixed, randomly ordered cycle hidden from the agents\. On each round, the sequence advanced one step with probabilityppand otherwise jumped to a random target\. Whenp=1p=1, the sequence completed five cycles in forty rounds, yet meanLminL\_\{\\min\}increased from2\.332\.33atp=0p=0to3\.003\.00atp=1p=1\.

## Discussion

The contrasting trends motivate our hypothesis, which remains to be tested with usable\-history measurements and message\-rate predictions registered in advance\. Interpretation is further limited because the target processes differed in rule complexity and in the number of distinct targets encountered, while the evidence comes from one model family, three seeds per batch, and forty interaction rounds\. These limitations motivate three tests of increasing generality\.

The first uses a small state\-estimation task in which the hidden state remains the same from one round to the next with a fixed probability and otherwise changes\. An agent estimates its value from a compressed record of its own noisy observations and a rate\-limited message about a peer’s noisy observation\. Because states and codes are finite, the Bayes\-optimal frontier can be calculated exactly for the allowed schemes and compared with the learned decoder’s frontier\.

The second maps the tested part of the region by varying memory rate, message rate, and target predictability in the referential game\. Within each condition, all rate settings use the same target stream, and comparisons across processes match target frequencies and the Bayes\-optimal gain from history\. A memory ablation removes episode history while preserving the learned symbol mappings\. Longer runs and another model family can distinguish insufficient learning time from an inability to use the rotation reliably\.

The third test repeats these rate experiments on distributed tasks with local views\. AgentsNet provides coloring, maximal matching, vertex cover, leader election, and consensus\([1](https://arxiv.org/html/2608.17053#bib.bib6)\)\. LoopBench provides repeated graph coloring in which observable actions may convey information\([5](https://arxiv.org/html/2608.17053#bib.bib7)\)\. We preregister a subset containing local and global coordination tasks, then vary how long private inputs or local states remain unchanged while holding the communication graph and task objective fixed\. To compare tasks, we charge memory, explicit messages, and information conveyed by observable actions against the same bit budget\. For each task and model family, we use measurements that vary one rate at a time to predict the crossover before varying both rates together\.

## Acknowledgments

This research was supported by the Alberta Machine Intelligence Institute \(Amii\) and the Canada CIFAR AI Chairs Program\. We also thank the Network for Applied Technology \(NAT\) for its support\. The authors used Claude \(Anthropic\) and Codex \(OpenAI\) to assist with code development and manuscript editing\. All AI\-assisted outputs were reviewed and verified by the authors, who take full responsibility for the work\.

## References

- Grötschlaet al\.\(2025\)F\. Grötschla, L\. Müller, J\. Tönshoff, M\. Galkin, and B\. PerozziAgentsNet: coordination and collaborative reasoning in multi\-agent LLMs\.arXiv preprint arXiv:2507\.08616\.External Links:[Link](https://arxiv.org/abs/2507.08616)Cited by:[Discussion](https://arxiv.org/html/2608.17053#Sx4.p4.1)\.
- Levin \(2024\)M\. LevinSelf\-improvising memory: a perspective on memories as agential, dynamically reinterpreting cognitive glue\.Entropy26\(6\),pp\. 481\.External Links:[Document](https://dx.doi.org/10.3390/e26060481)Cited by:[Introduction](https://arxiv.org/html/2608.17053#Sx1.p1.1)\.
- Lewis \(1969\)D\. LewisConvention: a philosophical study\.Harvard University Press\.Cited by:[Preliminary Results](https://arxiv.org/html/2608.17053#Sx3.p1.1)\.
- Parisiet al\.\(2019\)G\. I\. Parisi, R\. Kemker, J\. L\. Part, C\. Kanan, and S\. WermterContinual lifelong learning with neural networks: a review\.Neural Networks113,pp\. 54–71\.External Links:[Document](https://dx.doi.org/10.1016/j.neunet.2019.01.012),[Link](https://doi.org/10.1016/j.neunet.2019.01.012)Cited by:[Introduction](https://arxiv.org/html/2608.17053#Sx1.p1.1)\.
- Parsaeeet al\.\(2025\)A\. Parsaee, Y\. Talebirad, C\. Szepesvári, V\. Ohal, and E\. RedmanLoopBench: discovering emergent symmetry breaking strategies with LLM swarms\.arXiv preprint arXiv:2512\.13713\.External Links:[Link](https://arxiv.org/abs/2512.13713)Cited by:[Discussion](https://arxiv.org/html/2608.17053#Sx4.p4.1)\.
- Shannon \(1959\)C\. E\. ShannonCoding theorems for a discrete source with a fidelity criterion\.InIRE National Convention Record,Vol\.4,pp\. 142–163\.Cited by:[Introduction](https://arxiv.org/html/2608.17053#Sx1.p1.1)\.
- Talebiradet al\.\(2026a\)Y\. Talebirad, A\. Parsaee, C\. Y\. Szepesvári, A\. Nadiri, and O\. R\. ZaïaneToward a theory of hierarchical memory for language agents\.Note:ICLR 2026 Workshop on Memory for LLM\-Based Agentic SystemsExternal Links:[Link](https://arxiv.org/abs/2603.21564)Cited by:[Introduction](https://arxiv.org/html/2608.17053#Sx1.p1.1)\.
- Talebiradet al\.\(2026b\)Y\. Talebirad, E\. Redman, A\. Parsaee, and O\. R\. ZaïaneFrom signals to structure: how memory architecture drives language emergence in LLM agents\.arXiv preprint arXiv:2607\.00233\.Note:Accepted at the 2026 Conference on Artificial LifeExternal Links:[Document](https://dx.doi.org/10.48550/arXiv.2607.00233)Cited by:[Preliminary Results](https://arxiv.org/html/2608.17053#Sx3.p1.1)\.
- Wanget al\.\(2020\)R\. Wang, X\. He, R\. Yu, W\. Qiu, B\. An, and Z\. RabinovichLearning efficient multi\-agent communication: an information bottleneck approach\.InProceedings of the 37th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.119,pp\. 9908–9918\.External Links:[Link](https://proceedings.mlr.press/v119/wang20i.html)Cited by:[Introduction](https://arxiv.org/html/2608.17053#Sx1.p1.1)\.
- Wyner and Ziv \(1976\)A\. Wyner and J\. ZivThe rate\-distortion function for source coding with side information at the decoder\.IEEE Transactions on Information Theory22\(1\),pp\. 1–10\.External Links:[Document](https://dx.doi.org/10.1109/TIT.1976.1055508)Cited by:[Introduction](https://arxiv.org/html/2608.17053#Sx1.p1.1)\.