ConvMem: Convolutional Memory for Long-Context Reasoning

# ConvMem: Convolutional Memory for Long-Context Reasoning
Source: [https://arxiv.org/html/2609.10441](https://arxiv.org/html/2609.10441)
Hongming Zhang, Zhaozhen Gu, Fengshuo Bai, Ming Hao, Qingyang Zhang, Yuanyuan Wang, Shiyang Tang, Yanna Wang, Bo XuInstitute of Automation, Chinese Academy of Sciences

###### Abstract

While Large Language Models \(LLMs\) have demonstrated impressive capabilities, they often struggle with extremely long contexts due to fixed context limits\. To address this, sequential approaches like MemAgent extend the effective context by reading text in segments and iteratively updating a fixed\-size memory\. However, this sequential paradigm suffers from high latency and requires costly reinforcement learning \(RL\) training, which can lead to overfitting on specific datasets\. To overcome these limitations, we propose ConvMem, a training\-free, highly parallelizable framework that reformulates long\-context reasoning as a hierarchical convolution\. Inspired by CNNs, ConvMem treats an LLM prompted with a specific query as a convolutional kernel\. This kernel summarizes text segments hierarchically, shortening the reasoning path from a linear chain into a logarithmic tree\. Specifically, ConvMem integratesConfigurable StridesandSkip Connectionsto ensure robust evidence capture and propagation, while employingMulti\-Kernel Convolutionto decompose complex queries into disentangled semantic channels\. This design not only mitigates error accumulation but also enables massive parallelization across both text segments and reasoning threads\. Experiments on RULER\-HotpotQA and RULER\-2WikiMultiHopQA demonstrate that ConvMem outperforms training\-free baselines and avoids the risk of overfitting to parametric priors often observed in RL\-trained models on out\-of\-distribution tasks\.

## 1Introduction

The capability to process extremely long contexts, ranging from digesting entire books to executing complex multi\-step reasoning, has become a central objective for large language models \(LLMs\)\([Achiam et al\., 2023](https://arxiv.org/html/2609.10441#bib.bib29);[Liu et al\., 2024a](https://arxiv.org/html/2609.10441#bib.bib28);[Team et al\., 2025](https://arxiv.org/html/2609.10441#bib.bib30)\)\. While advancements in positional encoding and attention mechanisms\([Su et al\., 2024](https://arxiv.org/html/2609.10441#bib.bib4);[Beltagy et al\., 2020](https://arxiv.org/html/2609.10441#bib.bib2);[Liu et al\., 2023](https://arxiv.org/html/2609.10441#bib.bib7)\)have theoretically extended context windows to millions of tokens, practical deployment faces bottlenecks regarding inference latency and performance degradation\. Specifically, standard self\-attention suffers from quadratic complexity \(O⁡\(N2\)O\(N^\{2\}\)\), making it prohibitive for massive inputs\. Furthermore, even models capable of processing long texts often exhibit the lost\-in\-the\-middle phenomenon\([Liu et al\., 2024b](https://arxiv.org/html/2609.10441#bib.bib1)\), where reasoning performance degrades as context length increases\.

To address these scalability issues, recent research has pivoted towards memory\-based agents, such as MemAgent\([Yu et al\., 2025](https://arxiv.org/html/2609.10441#bib.bib8)\), Mem\-α\\alpha\([Wang et al\., 2025](https://arxiv.org/html/2609.10441#bib.bib9)\), and grounded\-memory agents\([Yang et al\., 2025](https://arxiv.org/html/2609.10441#bib.bib33);[Cui et al\., 2025](https://arxiv.org/html/2609.10441#bib.bib34)\)\. By treating text as a sequential stream with iterative memory updates, these methods achieve linear complexity \(O⁡\(N\)O\(N\)\)\. However, this paradigm introduces two critical limitations\. The first is thesequential bottleneck, where the strict temporal dependency of memory states prohibits parallelization and results in high inference latency\. The second is therisk of overfitting\. To optimize memory policies, these systems rely heavily on reinforcement learning \(RL\)\([Sutton and Barto, 2018](https://arxiv.org/html/2609.10441#bib.bib27);[Dong et al\., 2020](https://arxiv.org/html/2609.10441#bib.bib39);[Zhang and Yu, 2020](https://arxiv.org/html/2609.10441#bib.bib38)\)\. Our analysis reveals that RL\-trained agents tend to overfit to specific datasets, often hallucinating superficially correct answers based on pre\-trained knowledge rather than faithfully reasoning over the provided context\.

To overcome these challenges, we propose Convolutional Memory \(ConvMem\), a training\-free framework designed for high\-fidelity, parallelizable long\-context reasoning\. Drawing inspiration from convolutional neural networks \(CNNs\), ConvMem reformulates long\-text processing by treating a frozen LLM prompted with a specific query as a semantic convolutional kernel\. Unlike sequential approaches, ConvMem scans text segments concurrently at each layer\. This hierarchical architecture transforms the information flow from a linear chain into a logarithmic tree structure \(O⁡\(log⁡N\)O\(\\log N\)depth\)\. This topological shift not only accelerates inference via parallelization but also mitigates cumulative error propagation by shortening the reasoning path\.

ConvMem incorporates three architectural mechanisms to enhance reasoning fidelity and precision:

- •Configurable Strides for Robust Evidence Capture: Relying on a single pass renders reasoning vulnerable to boundary truncation and model stochasticity\. ConvMem introduces overlapping windows to perform multi\-view scanning\. This approach enables the kernel to cross\-verify information across different receptive fields, enhancing the stability and recall of key evidence\.
- •Skip Connections for Detail Preservation: Hierarchical summarization inevitably compresses information, risking the loss of fine\-grained details\. We introduce a semantic residual mechanism where high\-confidence raw evidence bypasses intermediate layers and propagates directly to the final reasoning stage, ensuring the answer is grounded in precise details\.
- •Multi\-Kernel Convolution for Semantic Disentanglement: Complex tasks often involve intertwined logical threads\. ConvMem decomposes queries into sub\-questions and deploys distinct kernels to extract relevant information into separate semantic channels, preventing interference between orthogonal reasoning paths\.

We evaluate ConvMem on RULER\-HotpotQA and RULER\-2WikiMultiHopQA\([Hsieh et al\., 2024](https://arxiv.org/html/2609.10441#bib.bib14);[Yang et al\., 2018](https://arxiv.org/html/2609.10441#bib.bib15);[Ho et al\., 2020](https://arxiv.org/html/2609.10441#bib.bib16)\)\. Results demonstrate that ConvMem outperforms training\-free baselines and exhibits superior generalization compared to RL\-trained models in out\-of\-distribution tasks\. Our contributions are threefold:

- •We propose ConvMem, the first training\-free, CNN\-inspired hierarchical framework that enables parallelizable long\-context reasoning by breaking the sequential bottleneck\.
- •We empirically expose the overfitting risks of RL\-based memory agents, demonstrating that their performance relies heavily on parametric memorization rather than in\-context reasoning\.
- •We demonstrate that through the principled integration of configurable strides, semantic skip connections, and multi\-kernel convolution, ConvMem establishes a new state\-of\-the\-art for training\-free methods, while offering superior generalization compared to RL\-trained models\.

## 2ConvMem for Long\-Context Reasoning

In this section, we present ConvMem, a training\-free framework that adapts the architectural principles of convolutional neural networks \(CNNs\) to long\-context reasoning\. Unlike sequential memory agents that process text as a linear stream, ConvMem reformulates the reasoning process as a hierarchical, multi\-channel convolutional operation\. We first formulate the problem and define the semantic convolutional kernel in[Section2\.2](https://arxiv.org/html/2609.10441#S2.SS2)\. We then detail the core architectural mechanisms in[Section2\.3](https://arxiv.org/html/2609.10441#S2.SS3), including configurable strides, skip connections and multi\-kernel convolution, that adapt visual convolution to textual reasoning\. Finally, we describe the complete hierarchical parallel inference workflow in[Section2\.4](https://arxiv.org/html/2609.10441#S2.SS4)\.

### 2\.1Problem Formulation

We consider a long\-context question answering task\. Let𝒟=\{t1,t2,…,tN\}\\mathcal\{D\}=\\\{t\_\{1\},t\_\{2\},\\dots,t\_\{N\}\\\}denote a massive input document consisting ofNNtokens, andQQdenote a user query\. Our objective is to generate an answerAAthat accurately addressesQQbased on the evidence contained in𝒟\\mathcal\{D\}\. Drawing an analogy to CNN’s spatial convolution, we define key notations adapted for textual reasoning:

- •WW\(Kernel Size\): The maximum context length, in tokens, scanned by a single LLM kernel\.
- •SS\(Stride\): The step size for the scanning kernel\. SettingS<WS<Wcreates overlapping windows, allowing each token to be scanned multiple times \(W/SW/S\) to enhance information recall\.
- •LL\(Depth\): The total number of layers in the hierarchical architecture\.
- •CC\(Channels\): The number of sub\-questions decomposed fromQQ\.

### 2\.2The Semantic Convolutional Kernel

The fundamental building block of ConvMem is the semantic convolutional kernel\. Unlike standard CNN kernels that perform linear algebraic operations \(dot products\), our semantic kernel performs a non\-linear, text\-to\-text transformation involving information extraction and compression\.

Formally, we define the semantic kernel𝒦\\mathcal\{K\}as a function parameterized by a frozen LLM and an instruction promptpp\. For a given layer, the kernel takes a queryqqand a local text segment𝐱⊂𝒟\\mathbf\{x\}\\subset\\mathcal\{D\}\(analogous to the receptive field\) as input\. The output is a tuple consisting of a summarized text and a relevance score:

\(h,r\)=𝒦LLM​\(𝐱,q,p\),\(h,r\)=\\mathcal\{K\}\_\{\\text\{LLM\}\}\(\\mathbf\{x\},q;p\),\(1\)wherehhis the condensed textual summary of𝐱\\mathbf\{x\}, distilling information relevant toqq\.r∈\{0,1,2\}r\\in\\\{0,1,2\\\}is a discrete relevance score indicating the importance of the segment:r=0r=0\(Irrelevant\) denotes that the segment contains no useful information;r=1r=1\(Relevant\) denotes the presence of potentially useful background or context; andr=2r=2\(Critical\) identifies segments containing direct evidence or the answer itself\. This semantic convolution performs two functions simultaneously: information extraction \(filtering noise\) and dimensionality reduction \(compressing token length\)\.

To extend this local operation to the full document, analogous to CNNs scanning an entire image, we define the semantic convolution operationConv​\(⋅\)\\text\{Conv\}\(\\cdot\)as follows:

𝐇,𝐑=Conv​\(𝒟,𝒦\)=\(⨁i=1Mhi,\{ri\}i=1M\),\\mathbf\{H\},\\mathbf\{R\}=\\text\{Conv\}\(\\mathcal\{D\},\\mathcal\{K\}\)=\\left\(\\bigoplus\_\{i=1\}^\{M\}h\_\{i\},\\\{r\_\{i\}\\\}\_\{i=1\}^\{M\}\\right\),\(2\)where\(hi,ri\)=𝒦LLM​\(𝐱i,q,p\)\\text\{where\}\\quad\(h\_\{i\},r\_\{i\}\)=\\mathcal\{K\}\_\{\\text\{LLM\}\}\(\\mathbf\{x\}\_\{i\},q;p\)\. This operation involves sliding the kernel𝒦\\mathcal\{K\}across all segments𝐱i⊂𝒟\\mathbf\{x\}\_\{i\}\\subset\\mathcal\{D\}\. Here,⨁i=1Mhi\\bigoplus\_\{i=1\}^\{M\}h\_\{i\}denotes the sequential concatenation of local summariesh1,…,hMh\_\{1\},\.\.\.,h\_\{M\}, forming a global summary sequence𝐇\\mathbf\{H\}\.𝐑=\[r1,…,rM\]\\mathbf\{R\}=\[r\_\{1\},\\dots,r\_\{M\}\]is the sequence of relevance scores corresponding to eachhih\_\{i\}, used to identify critical segments for the semantic skip connection mechanism\.

Figure 1:Illustration of the hierarchical convolutional mechanism within a single channel\. At Layer 0, the semantic kernel𝒦\\mathcal\{K\}scans raw text segments in parallel to generate condensed summaries\. Subsequent layers recursively process the concatenated summaries from the previous layer\. AfterLLlayers, the global summary is combined with critical raw segments \(skip connections\) to generate the final answerAA\.![Refer to caption](https://arxiv.org/html/2609.10441v1/overview1-v3.png)
### 2\.3CNN\-Inspired Architectural Mechanisms

To effectively adapt the convolutional paradigm to language reasoning, ConvMem incorporates three key mechanisms: configurable strides, skip connections and multi\-kernel convolution\.

#### 2\.3\.1Configurable Strides

Standard segmentation often leads to boundary truncation, where semantic dependencies are disrupted by the rigid partitioning\. We address this limitation by implementing overlapping sliding windows with strideS<WS<W\. The input sequence is sliced into segments where the overlap region of lengthW−SW\-Sensures contextual continuity\. Specifically, tokens at the edge of segmentiiare re\-processed near the center of segmenti\+1i\+1, preserving their contextual integrity\. This mechanism provides two benefits:

- •Contextual Continuity: Overlapping windows preserve the semantic integrity that span segmentation boundaries, effectively eliminate information loss caused by arbitrary truncation\.
- •Multi\-View Robustness: Each token is scanned multiple times \(with an over\-scanning factor ofW/SW/S\) from varying window positions\. This mechanism cross\-verifies information across different receptive fields, mitigating the stochasticity of the LLM kernel and resulting in a robust, ensembled understanding of the content\.

#### 2\.3\.2Skip Connections

Hierarchical summarization entails the risk of losing fine\-grained details, such as exact dates or entity names\. Inspired by residual networks\([He et al\., 2016](https://arxiv.org/html/2609.10441#bib.bib10)\), we introduce a semantic skip connection mechanism based on the relevance scorerr\.

During convolution at the input layer, if the kernel assigns a critical scorer=2r=2to a segment𝐱i\\mathbf\{x\}\_\{i\}, this raw segment will add a skip connection to the final layer, bypassing the summarization and is directly appended to a residual bufferℬ\\mathcal\{B\}:

ℬ←ℬ∪\{𝐱i∣Score​\(𝐱i\)=2\}\\mathcal\{B\}\\leftarrow\\mathcal\{B\}\\cup\\\{\\mathbf\{x\}\_\{i\}\\mid\\text\{Score\}\(\\mathbf\{x\}\_\{i\}\)=2\\\}\(3\)This allows the final generation step to reason over high\-level abstract summaries while accessing unmodified, high\-fidelity evidence from the raw text\.

#### 2\.3\.3Multi\-Kernel Convolution

Complex reasoning tasks often involve tracking multiple, intertwined logical threads \(e\.g\., “What is the relationship between Person A and Person B?”\)\. A single summary stream may conflate distinct entities or events\. To handle this, we employ multi\-kernel convolution to treat the text as a multi\-channel signal\.

Given the user queryQQ, we first decomposes it intoCCdistinct sub\-questions\{q\(1\),q\(2\),…,q\(C\)\}\\\{q^\{\(1\)\},q^\{\(2\)\},\\dots,q^\{\(C\)\}\\\}\. Each sub\-question initializes a unique kernel𝒦\(c\)=𝒦LLM​\(⋅,q\(c\),p\)\\mathcal\{K\}^\{\(c\)\}=\\mathcal\{K\}\_\{\\text\{LLM\}\}\(\\cdot,q^\{\(c\)\};p\)\. These kernels operate in parallel on the same document but focus on different semantic aspects, preventing interference between orthogonal reasoning paths\. For example,𝒦\(1\)\\mathcal\{K\}^\{\(1\)\}might focus on “Person A”, while𝒦\(2\)\\mathcal\{K\}^\{\(2\)\}tracks “Person B”\. This process generatesCCdistinct reasoning channels:

𝐇\(c\)=Conv​\(𝒟,𝒦\(c\)\)\\mathbf\{H\}^\{\(c\)\}=\\text\{Conv\}\(\\mathcal\{D\},\\mathcal\{K\}^\{\(c\)\}\)\(4\)Accordingly, the global notations defined previously are extended to channel\-specific versions \(e\.g\.,ℬ→ℬ\(c\)\\mathcal\{B\}\\to\\mathcal\{B\}^\{\(c\)\}\), ensuring isolation between different reasoning threads\. This mechanism ensures that the path for each sub\-question remains disentangled and preserved until the final aggregation\.

### 2\.4Hierarchical Parallel Inference Workflow

ConvMem integrates these components into a hierarchical workflow, transforming the raw document𝒟\\mathcal\{D\}into a global answerAAthroughLLlayers\.

##### Layer 0 \(Input Processing\)\.

The raw document𝒟\\mathcal\{D\}is sliced intoM0M\_\{0\}overlapping segments\[𝐱1\(0\),𝐱2\(0\),…,𝐱M0\(0\)\]\[\\mathbf\{x\}\_\{1\}^\{\(0\)\},\\mathbf\{x\}\_\{2\}^\{\(0\)\},\\dots,\\mathbf\{x\}\_\{M\_\{0\}\}^\{\(0\)\}\]using window sizeWWand strideSS\. Each kernel𝒦\(c\)\\mathcal\{K\}^\{\(c\)\}scans these segments in parallel to produce initial summaries and update the residual bufferℬ\(c\)\\mathcal\{B\}^\{\(c\)\}\.

##### Layerll\(Recursive Summarization\)\.

For hidden layersl\>0l\>0, we perform recursive summarization\. For each channelcc, the summaries from the previous layer are first concatenated to form an intermediate text stream𝐇\(l−1,c\)\\mathbf\{H\}^\{\(l\-1,c\)\}\. This stream is sliced into new overlapping segments, and the semantic kernel𝒦\(c\)\\mathcal\{K\}^\{\(c\)\}is applied recursively using the semantic convolution operation:

𝐇\(l,c\)=Conv​\(𝐇\(l−1,c\),𝒦\(c\)\)\\mathbf\{H\}^\{\(l,c\)\}=\\text\{Conv\}\(\\mathbf\{H\}^\{\(l\-1,c\)\},\\mathcal\{K\}^\{\(c\)\}\)\(5\)ConvMem achieves dimensionality reduction through semantic compression\. Assuming an average compression factorα\\alpha\(the ratio of summary length to input length\), the total sequence length decays exponentially:Nl≈N0⋅αlN\_\{l\}\\approx N\_\{0\}\\cdot\\alpha^\{l\}\. As the hierarchy deepens, the effective receptive field expands exponentially, condensing the massive document into a manageable set of global representations𝐇\(L,c\)\\mathbf\{H\}^\{\(L,c\)\}\.

Figure 2:The overall workflow of ConvMem\. \(a\)Query Decomposition:The complex queryQQis disentangled intoCCsub\-questions to isolate reasoning threads\. \(b\)Multi\-Kernel Convolution:Each sub\-question drives a distinct kernel to scan the document in parallel\. Through hierarchical layers, local evidence is recursively summarized and aggregated to synthesize the final global answerAA\.![Refer to caption](https://arxiv.org/html/2609.10441v1/overview2-v3.png)
##### Final Aggregation\.

At the final layerLL, the inference proceeds in two stages to generate the global answer\. First, each channel independently generates a sub\-answera\(c\)a^\{\(c\)\}for its sub\-questionq\(c\)q^\{\(c\)\}\. Crucially, this step utilizes both the high\-level summaries𝐇\(L,c\)\\mathbf\{H\}^\{\(L,c\)\}and the high\-fidelity raw evidence in the residual bufferℬ\(c\)\\mathcal\{B\}^\{\(c\)\}to ensure precision:

a\(c\)=LLM​\(q\(c\)⊕𝐇\(L,c\)⊕ℬ\(c\)\)a^\{\(c\)\}=\\text\{LLM\}\(q^\{\(c\)\}\\oplus\\mathbf\{H\}^\{\(L,c\)\}\\oplus\\mathcal\{B\}^\{\(c\)\}\)\(6\)
Second, the model performs global reasoning to derive the final answerAA\. It aggregates the original queryQQwith the sequence of resolved sub\-question/sub\-answer pairs:

A=LLM​\(Q⊕⨁c=1C\(q\(c\)⊕a\(c\)\)\)A=\\text\{LLM\}\\left\(Q\\oplus\\bigoplus\_\{c=1\}^\{C\}\(q^\{\(c\)\}\\oplus a^\{\(c\)\}\)\\right\)\(7\)This design ensures that fine\-grained details are resolved at the sub\-problem level, while the final aggregation focuses on logical synthesis\.

##### Parallelization\.

A critical advantage of ConvMem over sequential memory agents \(e\.g\., MemAgent\) is the decoupling of temporal dependencies\. In MemAgent, stepttstrictly depends on the memory state of stept−1t\-1, forcing anO⁡\(N\)O\(N\)sequential execution\. In contrast, within any layerllof ConvMem, the operations on all segments are independent\. Furthermore, computations across different kernelsccare also orthogonal\.

This independence enables massive parallelization\. Theoretically, the inference latency is determined solely by the tree depth rather than the sequence length:

Tlatency∝O⁡\(log⁡N\)T\_\{\\text\{latency\}\}\\propto O\(\\log N\)\(8\)This logarithmic scaling enables ConvMem to process million\-token contexts with latency comparable to processing short contexts\.

## 3Experiments

In this section, we conduct extensive experiments to investigate three key research questions:Q1: Performance Efficacy\.How does ConvMem compare to standard language models and memory\-enhanced agents on established benchmarks?Q2: Robustness and Scalability\.RL\-based approaches often suffer from performance degradation when shifting to unseen data distributions\. In contrast, does ConvMem demonstrate superior cross\-domain consistency compared to these specialists? Furthermore, is ConvMem model\-agnostic, enabling immediate performance gains when integrated different backbone LLMs without adaptation?Q3: Mechanism Analysis\.How do the proposed architectural components \(configurable strides, skip connections, multi\-kernel convolution\) contribute to the system’s effectiveness?

### 3\.1Experimental Setup

##### Baselines\.

To ensure a comprehensive evaluation, we compare ConvMem against baselines categorized into three distinct paradigms:

- •Standard LLMs:We employQwen2\.5\-32B\-Instruct\([Qwen et al\., 2025](https://arxiv.org/html/2609.10441#bib.bib11)\)as the primary baseline\. To investigate performance scaling across model sizes, we also includeQwen2\.5\-7B\-InstructandQwen2\.5\-72B\-Instruct\([Team, 2024](https://arxiv.org/html/2609.10441#bib.bib12)\)as comparative baselines\([Team, 2024](https://arxiv.org/html/2609.10441#bib.bib12)\)\.
- •Training\-Free Memory Methods:We evaluate retrieval\-based and memory\-based approaches, includingRAG\-BM25\([Mem\-Lab, 2024](https://arxiv.org/html/2609.10441#bib.bib13)\),MemAgent\-W/O\-RL\([Yu et al\., 2025](https://arxiv.org/html/2609.10441#bib.bib8)\), andMem\-α\\alpha\-W/O\-RL\([Wang et al\., 2025](https://arxiv.org/html/2609.10441#bib.bib9)\)\. All these methods utilize Qwen2\.5\-32B\-Instruct as the backbone to ensure fair comparison\.
- •RL\-Trained Specialists:We benchmark againstMemAgent\([Yu et al\., 2025](https://arxiv.org/html/2609.10441#bib.bib8)\)andMem\-α\\alpha\([Wang et al\., 2025](https://arxiv.org/html/2609.10441#bib.bib9)\)\. These models represent the RL paradigm, which optimizes sequential memory updates through extensive training\.

##### Datasets\.

We conduct evaluations on two benchmarks designed to assess both in\-distribution and out\-of\-distribution performance:

- •RULER\-HotpotQA \(In\-Distribution\):Following the MemAgent protocol, we use the RULER\-HotpotQA benchmark\([Hsieh et al\., 2024](https://arxiv.org/html/2609.10441#bib.bib14);[Yang et al\., 2018](https://arxiv.org/html/2609.10441#bib.bib15)\)\. This dataset adapts multi\-hop questions from HotpotQA into a long\-context Needle\-in\-a\-Haystack \(NIAH\) paradigm\. Crucially, as MemAgent utilizes this specific data distribution for policy optimization, it serves as the in\-distribution testbed\.
- •RULER\-2WikiMultiHopQA \(Out\-of\-Distribution\):To rigorously distinguish reasoning from memorization, we synthesize a novel long\-context dataset based on 2WikiMultiHopQA\([Ho et al\., 2020](https://arxiv.org/html/2609.10441#bib.bib16)\)using the same construction protocol as RULER\-HotpotQA\. This dataset remains unseen during the training of MemAgent, providing a strict OOD setting to evaluate generalization\.

Table 1:Main results on RULER\-HotpotQA \(In\-Distribution\) and RULER\-2WikiMultiHopQA \(Out\-of\-Distribution\)\. We report F1 and Sub\-EM scores across context lengths ranging from 28k to 896k\. The best results are highlighted in bold, and the second best are underlined\.CategoryModelMetricRULER\-HotpotQA2WikiMultiHopQA28k56k112k224k448k896k28k56k112k224k448k896kBase ModelQwen2\.5\-32B\-InstructF161\.957\.7748\.8644\.0418\.5916\.7856\.5257\.250\.4541\.7329\.4520\.93Sub\-EM69\.5363\.2853\.9144\.5317\.1917\.9767\.1965\.6262\.550\.034\.3828\.12Training\-FreeMethodsRAG\-BM25F145\.9851\.1147\.2135\.8629\.2730\.6734\.5431\.1724\.5523\.1921\.2517\.34Sub\-EM46\.0950\.0048\.4435\.9430\.4732\.8157\.0358\.5947\.6646\.8839\.0643\.75MemAgent\-W/O\-RLF163\.9565\.2862\.5257\.9855\.4258\.7360\.5157\.4245\.3150\.647\.6649\.65Sub\-EM67\.9771\.0965\.6259\.3856\.2560\.1671\.0967\.9758\.5963\.2860\.1662\.5Mem\-α\\alpha\-W/O\-RLF16\.137\.786\.57\.037\.197\.356\.566\.475\.946\.195\.196\.0Sub\-EM56\.2553\.1243\.7546\.8846\.8849\.2757\.3859\.3862\.553\.1253\.1250\.0RL\-TrainedMethodsMemAgentF175\.5575\.2075\.2673\.5473\.1068\.8060\.9260\.1163\.1958\.8258\.558\.41Sub\-EM79\.6979\.6980\.4777\.3478\.9174\.2270\.3171\.0972\.6669\.5367\.9769\.31Mem\-α\\alphaF15\.846\.817\.06\.091\.310\.691\.251\.130\.941\.030\.971\.09Sub\-EM34\.3840\.6240\.6243\.7543\.7525\.059\.3856\.2550\.062\.550\.053\.12OursConvMemF167\.4467\.8657\.8163\.2756\.1463\.0972\.371\.2567\.2161\.9661\.3359\.06Sub\-EM73\.4472\.6667\.1969\.5362\.5069\.5382\.8182\.4777\.3471\.8873\.3170\.62

##### Metric\.

We report four metrics: F1 score, Exact Match \(EM\), Sub\-EM \(Sub Exact Match\) and LLM\-as\-a\-Judge \(ACCL\\text\{ACC\}\_\{L\}\)\([Rajpurkar et al\., 2016](https://arxiv.org/html/2609.10441#bib.bib32);[Zheng et al\., 2023](https://arxiv.org/html/2609.10441#bib.bib17)\)\. F1 represents the harmonic mean of precision and recall\. EM indicates whether the predicted value exactly matches the ground truth\. Sub\-EM measures whether the predicted value is a subset of the reference value, or vice versa\.ACCL\\text\{ACC\}\_\{L\}using LLMs as judges to evaluate the performance\. Due to space constraints, our main analysis focuses on F1 and Sub\-EM, as they offer a balanced view of retrieval recall and generation flexibility\. Full results across all metrics are provided in Appendix[C](https://arxiv.org/html/2609.10441#A3)\.

##### Implementation Details\.

We set the kernel sizeW=8000W=8000tokens and strideS=1600S=1600, implying scanning 5 times for each token\. The number of channelsCCis dynamically determined by the LLM during query decomposition\. ConvMem requires no parameter updates\. Prompt templates are detailed in[AppendixF](https://arxiv.org/html/2609.10441#A6)\.

### 3\.2Main Results and Analysis

Table[1](https://arxiv.org/html/2609.10441#S3.T1)presents the performance comparison across varying context lengths\. As observed, ConvMem establishes a new state\-of\-the\-art among training\-free methods and demonstrates superior robustness compared to RL\-based specialists\.

##### Superiority Over Training\-Free Baselines\.

On both datasets, ConvMem significantly outperforms the vanilla Qwen2\.5\-32B\-Instruct and all training\-free baselines\. Retrieval\-based methods \(RAG\) struggle with multi\-hop reasoning as they retrieve isolated chunks, disrupting logical dependencies\. Sequential methods \(MemAgent\-W/O\-RL\) suffer from cumulative forgetting and the interference of intertwined logical threads, rendering early evidence unrecoverable\.

Case 1illustrates this failure mode: sequential agents MemAgent\-W/O\-RL encounter the entity "Shirley Temple Black" early in the stream but fail to link it to the film “Kiss and Tell” appearing much later, as the initial memory is overwritten\. In contrast, ConvMem’s hierarchical aggregation preserves both facts in parallel channels, successfully retrieving the answer\.

Case 1: Mitigating Sequential Forgetting\.Query:What government position was held by the woman who portrayed Corliss Archer in the film Kiss and Tell?Failure Analysis\(MemAgent\-W/O\-RL\):The agent identifies “Shirley Temple Black” early but forgets her government position by the time the film “Kiss and Tell” appears later in the stream, due to the interference of intertwined logical threads\.ConvMem Success:Thanks to its multi\-kernel convolution and hierarchical aggregation, ConvMem retains both the entity attribute and the film relation, enabling successful multi\-hop reasoning\.

##### Robustness Against RL\-Induced Instability\.

We observe that RL\-based baselines suffer from distinct instability issues\. Mem\-α\\alphaachieves a significantly lower F1 score compared to ConvMem\. This is due to its tendency to output excessively verbose answers, which degrades precision\.

While MemAgent peaks on its training domain \(RULER\-HotpotQA\), its performance collapses on the unseen RULER\-2WikiMultiHopQA dataset\. This discrepancy suggests that RL models may prioritize parametric memorization over in\-context reasoning\.Case 2provides empirical evidence of this phenomenon\. The query asks to choose between “LevYilmaz” or “Pamela B\. Green”\. However, the dataset ground truth contains a typo “LevniYilmaz”\. MemAgent outputs “LevniYilmaz”, ignoring the provided context to match the memorized label\. ConvMem faithfully extracts “LevYilmaz” from the context\. Although penalized by the metric, ConvMem demonstrates superior faithfulness to the input, whereas the RL agent exhibits hallucination from priors\.

Case 2: Correct Answer via Parametric Hallucination\.Query:Which filmmaker was known for animation,LevYilmaz or Pamela B\. Green?Dataset Label \(Typo\):LevniYilmaz\.MemAgent Output:LevniYilmaz \(Matches label, contradicts context\)\.ConvMem Output:LevYilmaz \(Matches context, penalized as wrong\)\.

This case highlights a critical limitation of RL\-based agents: they risk regressing into parametric retrieval systems that prioritize memorized priors over input evidence\. In contrast, ConvMem, by design, functions as a faithful reasoning engine grounded in the provided context\.

##### Model Agnosticism and Scalability\.

A key advantage of ConvMem is its model\-agnostic nature\. As shown in[Fig\.3](https://arxiv.org/html/2609.10441#S3.F3), applying ConvMem to backbones of varying sizes \(from 7B to 72B\) yields immediate performance gains\. This confirms that our method effectively scales the long\-context reasoning of diverse LLMs without adaptation costs\.

Figure 3:ConvMem yields consistent performance improvements across diverse backbone sizes \(7B\-72B\), validating its effective scaling capabilities\.

### 3\.3Ablation Study

We conduct component\-wise ablations on RULER\-2WikiMultiHopQA to validate our design choices\.

##### Impact of Strides \(Over\-Scanning Factor\)\.

We investigate the effect of theover\-scanning factor\(W/SW/S\), which determines how many times each token is processed\. As shown in[Fig\.4](https://arxiv.org/html/2609.10441#S3.F4)\(Left\), single\-pass scanning \(S=WS=W\) leads to a noticeable performance drop due to boundary truncation\. Increasing this factor consistently improves performance, with5×5\\timesover\-scanning achieving the optimal balance between coverage and noise\.

##### Effect of Skip Connections\.

Removing skip connections \([Fig\.4](https://arxiv.org/html/2609.10441#S3.F4), Right\) results in a degradation of fine\-grained details, particularly for exact entity retrieval\. This confirms that skip connections function as a semantic highway, allowing high\-confidence evidence to bypass compression loss\.

Figure 4:\(Left\)Higher over\-scanning factors \(W/SW/S\) improve recall by mitigating segmentation boundaries, optimal at5×5\\times\.\(Right\)The absence of the residual path results in noticeable degradation, confirming the necessity of propagating raw high\-fidelity evidence\.
##### Significance of Multi\-Kernel Convolution\.

We compare the single\-kernel against our multi\-kernel convolution\. Results indicate that query decomposition significantly boosts accuracy on multi\-hop tasks \([Fig\.5](https://arxiv.org/html/2609.10441#S3.F5), Left\)\. Single\-kernel models often conflate distinct reasoning threads, whereas multi\-kernel convolution successfully disentangles semantic dependencies, reducing interference\.

##### Kernel Size Sensitivity\.

We test kernel sizesW∈\{500,5000,8000,10000\}W\\in\\\{500,5000,8000,10000\\\}\. Extremely small kernels \(500500\) fragment semantic context, while overly large kernels \(1000010000\) dilute local signal density\.W=8000W=8000provides an optimal balance between context coherence and signal extraction\.

Figure 5:\(Left\)Decomposing queries into parallel channels \(C\>1C\>1\) significantly boosts performance by disentangling interference between reasoning threads\.\(Right\)W=8000W=8000achieves the optimal trade\-off between context coherence and signal density\.

## 4Related Work

Existing long\-context approaches generally fall into three categories: architectural extrapolation, memory\-augmented systems, and reinforcement learning optimizations\.

### 4\.1Long\-Context Extrapolation and Efficient Architectures

To process inputs exceeding pre\-training limits, research has focused on positional extrapolation \(e\.g\., PI, YaRN, Ring Attention\)\([Chen et al\., 2023](https://arxiv.org/html/2609.10441#bib.bib18);[Peng et al\., 2023](https://arxiv.org/html/2609.10441#bib.bib19);[Liu et al\., 2023](https://arxiv.org/html/2609.10441#bib.bib7)\)and efficiency optimizations via Linear/Sparse Attention\([Xiao et al\., 2023](https://arxiv.org/html/2609.10441#bib.bib3);[Katharopoulos et al\., 2020](https://arxiv.org/html/2609.10441#bib.bib5);[Child et al\., 2019](https://arxiv.org/html/2609.10441#bib.bib6)\)or KV Cache compression\([Li et al\., 2024](https://arxiv.org/html/2609.10441#bib.bib20);[Zhang et al\., 2023b](https://arxiv.org/html/2609.10441#bib.bib21)\)\. Despite these advancements, native long\-context models often suffer from the lost\-in\-the\-middle phenomenon\([Liu et al\., 2024b](https://arxiv.org/html/2609.10441#bib.bib1)\), where reasoning performance degrades as effective context length increases\. Furthermore, processing massive documents in a single pass imposes prohibitive hardware demands and latency\. In contrast, ConvMem bypasses the quadratic bottleneck by decomposing the context into manageable hierarchical receptive fields, ensuring robust reasoning without modifying the underlying attention\.

### 4\.2Retrieval\-/Memory\-Augmented Agents

To decouple context length from computational cost, recent works employ external memory systems\. Retrieval\-Augmented Generation \(RAG\)\([Lewis et al\., 2020](https://arxiv.org/html/2609.10441#bib.bib22)\)reduces context by fetching top\-kkchunks, but it often fails on multi\-hop reasoning tasks due to the retrieval of isolated fragments lacking global connectivity\. Alternatively, Sequential Memory Agents, such as MemGPT\([Packer et al\., 2023](https://arxiv.org/html/2609.10441#bib.bib23)\), Mem0\([Chhikara et al\., 2025](https://arxiv.org/html/2609.10441#bib.bib24)\), and MIRIX\([Wang and Chen, 2025](https://arxiv.org/html/2609.10441#bib.bib25)\), MemAgent\([Yu et al\., 2025](https://arxiv.org/html/2609.10441#bib.bib8)\)treat text as a continuous stream, maintaining a persistent memory state that evolves step\-by\-step\. While effective, these sequential methods introduce a temporal dependency \(O⁡\(N\)O\(N\)\) that prohibits parallelization, leading to high inference latency\. ConvMem addresses these limitations by adopting a tree\-structured, CNN\-inspired topology \(O⁡\(log⁡N\)O\(\\log N\)\), enabling massive parallelization and shortening the information propagation path to minimize error accumulation\.

### 4\.3Reinforcement Learning for Memory Optimization

Acknowledging that heuristic memory updates may be suboptimal, recent works employ RL to learn memory policies, following the long tradition of exploiting memory in RL\([Zhang et al\., 2023a](https://arxiv.org/html/2609.10441#bib.bib36);[Zhang et al\., 2024b](https://arxiv.org/html/2609.10441#bib.bib35);[Zhang et al\., 2024a](https://arxiv.org/html/2609.10441#bib.bib37)\)\. For instance, MemAgent\([Yu et al\., 2025](https://arxiv.org/html/2609.10441#bib.bib8)\)utilizes Multi\-Conv DAPO to optimize memory overwrite decisions, while Memory\-R1\([Yan et al\., 2025](https://arxiv.org/html/2609.10441#bib.bib26)\)and Mem\-α\\alpha\([Wang et al\., 2025](https://arxiv.org/html/2609.10441#bib.bib9)\)train agents to manage complex memory structures via reward signals derived from downstream task accuracy\. While RL yields performance gains on in\-domain benchmarks, it introduces significant training costs and are susceptible to overfitting dataset\. In contrast, ConvMem proposes a training\-free paradigm\. By leveraging the intrinsic instruction\-following capabilities of pre\-trained LLMs rather than biased reward optimization, ConvMem achieves superior robustness and true in\-context reasoning\.

## 5Conclusion

In this work, we proposeConvMem, a training\-free framework that reformulates long\-context reasoning as a hierarchical, multi\-channel convolutional process\. By reconceptualizing LLM\-based reasoning through the lens of Convolutional Neural Networks, ConvMem successfully transforms the linear dependency chain \(O⁡\(N\)O\(N\)\) into a logarithmic tree structure \(O⁡\(log⁡N\)O\(\\log N\)\), effectively mitigating cumulative error propagation\. Our empirical evaluation reveals that this principled adaptation, leveraging multi\-kernel convolution, configurable strides, and skip connections, achieves competitive performance on standard benchmarks while demonstrating superior generalization on out\-of\-distribution tasks compared to RL\-trained specialists\. Furthermore, our analysis exposes the tendency of RL\-based agents to overfit to dataset artifacts, suggesting that ConvMem’s training\-free, context\-anchored approach provides a more robust path for genuine in\-context reasoning\. We hope this work encourages the community to further explore architectural innovations that balance computational efficiency with reasoning fidelity\.

## 6Limitations

While ConvMem offers significant advantages in latency and robustness, we acknowledge two primary limitations that point towards future research directions\. First, regarding token consumption vs\. latency trade\-off, although ConvMem achieves logarithmic latency through parallelization, the total computational cost is higher than that of linear scanning methods\. Second, the system exhibits a dependency on query decomposition\. The efficacy of our parallel convolution relies on the backbone model’s ability to correctly disentangle complex queries into orthogonal sub\-questions\. If the initial decomposition is flawed, subsequent kernels may operate on incomplete premises\.

## 7Ethical Considerations

This work proposes a training\-free framework for long\-context reasoning\. While our approach requires multiple passes over the text, increasing inference computation, it eliminates the energy consumption associated with model training\. Limitations regarding potential biases stem from the underlying frozen LLMs used as kernels\. All datasets used in this study are publicly available, and no private data was involved in the experiments\.

## References

- Achiamet al\.\(2023\)J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman, S\. Anadkat,et al\.Gpt\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[§1](https://arxiv.org/html/2609.10441#S1.p1.1)\.
- Beltagyet al\.\(2020\)I\. Beltagy, M\. E\. Peters, and A\. CohanLongformer: the long\-document transformer\.arXiv preprint arXiv:2004\.05150\.Cited by:[§1](https://arxiv.org/html/2609.10441#S1.p1.1)\.
- Chenet al\.\(2023\)S\. Chen, S\. Wong, L\. Chen, and Y\. TianExtending context window of large language models via positional interpolation\.arXiv preprint arXiv:2306\.15595\.Cited by:[§4\.1](https://arxiv.org/html/2609.10441#S4.SS1.p1.1)\.
- Chhikaraet al\.\(2025\)P\. Chhikara, D\. Khant, S\. Aryan, T\. Singh, and D\. YadavMem0: building production\-ready ai agents with scalable long\-term memory\.arXiv preprint arXiv:2504\.19413\.Cited by:[§4\.2](https://arxiv.org/html/2609.10441#S4.SS2.p1.1)\.
- Childet al\.\(2019\)R\. Child, S\. Gray, A\. Radford, and I\. SutskeverGenerating long sequences with sparse transformers\.arXiv preprint arXiv:1904\.10509\.Cited by:[§4\.1](https://arxiv.org/html/2609.10441#S4.SS1.p1.1)\.
- Cuiet al\.\(2025\)S\. Cui, A\. He, S\. Xu, H\. Zhang, Y\. Wang, Q\. Zhang, Y\. Wang, and B\. XuSelf\-guided function calling in large language models via stepwise experience recall\.arXiv preprint arXiv:2508\.15214\.Cited by:[§1](https://arxiv.org/html/2609.10441#S1.p2.1)\.
- Donget al\.\(2020\)H\. Dong, Z\. Ding, and S\. ZhangDeep reinforcement learning: fundamentals, research and applications\.Springer Nature\.Cited by:[§1](https://arxiv.org/html/2609.10441#S1.p2.1)\.
- Heet al\.\(2016\)K\. He, X\. Zhang, S\. Ren, and J\. SunDeep residual learning for image recognition\.InProceedings of the IEEE conference on computer vision and pattern recognition,pp\. 770–778\.Cited by:[§2\.3\.2](https://arxiv.org/html/2609.10441#S2.SS3.SSS2.p1.1)\.
- Hoet al\.\(2020\)X\. Ho, A\. D\. Nguyen, S\. Sugawara, and A\. AizawaConstructing a multi\-hop qa dataset for comprehensive evaluation of reasoning steps\.arXiv preprint arXiv:2011\.01060\.Cited by:[Appendix A](https://arxiv.org/html/2609.10441#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.10441#S1.p5.1),[2nd item](https://arxiv.org/html/2609.10441#S3.I2.i2.p1.1)\.
- Hsiehet al\.\(2024\)C\. Hsieh, S\. Sun, S\. Kriman, S\. Acharya, D\. Rekesh, F\. Jia, Y\. Zhang, and B\. GinsburgRULER: what’s the real context size of your long\-context language models?\.arXiv preprint arXiv:2404\.06654\.Cited by:[Appendix A](https://arxiv.org/html/2609.10441#A1.SS0.SSS0.Px3.p1.1),[§1](https://arxiv.org/html/2609.10441#S1.p5.1),[1st item](https://arxiv.org/html/2609.10441#S3.I2.i1.p1.1)\.
- Katharopouloset al\.\(2020\)A\. Katharopoulos, A\. Vyas, N\. Pappas, and F\. FleuretTransformers are rnns: fast autoregressive transformers with linear attention\.InInternational conference on machine learning,pp\. 5156–5165\.Cited by:[§4\.1](https://arxiv.org/html/2609.10441#S4.SS1.p1.1)\.
- Kwonet al\.\(2023\)W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. Gonzalez, H\. Zhang, and I\. StoicaEfficient memory management for large language model serving with pagedattention\.InProceedings of the 29th symposium on operating systems principles,pp\. 611–626\.Cited by:[§B\.1](https://arxiv.org/html/2609.10441#A2.SS1.p1.1)\.
- Lewiset al\.\(2020\)P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel,et al\.Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.Advances in neural information processing systems33,pp\. 9459–9474\.Cited by:[§4\.2](https://arxiv.org/html/2609.10441#S4.SS2.p1.1)\.
- Liet al\.\(2024\)Y\. Li, Y\. Huang, B\. Yang, B\. Venkitesh, A\. Locatelli, H\. Ye, T\. Cai, P\. Lewis, and D\. ChenSnapkv: llm knows what you are looking for before generation\.Advances in Neural Information Processing Systems37,pp\. 22947–22970\.Cited by:[§4\.1](https://arxiv.org/html/2609.10441#S4.SS1.p1.1)\.
- Liuet al\.\(2024a\)A\. Liu, B\. Feng, B\. Xue, B\. Wang, B\. Wu, C\. Lu, C\. Zhao, C\. Deng, C\. Zhang, C\. Ruan,et al\.Deepseek\-v3 technical report\.arXiv preprint arXiv:2412\.19437\.Cited by:[§1](https://arxiv.org/html/2609.10441#S1.p1.1)\.
- Liuet al\.\(2023\)H\. Liu, M\. Zaharia, and P\. AbbeelRing attention with blockwise transformers for near\-infinite context\.arXiv preprint arXiv:2310\.01889\.Cited by:[§1](https://arxiv.org/html/2609.10441#S1.p1.1),[§4\.1](https://arxiv.org/html/2609.10441#S4.SS1.p1.1)\.
- Liuet al\.\(2024b\)N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. LiangLost in the middle: how language models use long contexts\.Transactions of the Association for Computational Linguistics12,pp\. 157–173\.Cited by:[§1](https://arxiv.org/html/2609.10441#S1.p1.1),[§4\.1](https://arxiv.org/html/2609.10441#S4.SS1.p1.1)\.
- Mem\-Lab \(2024\)Mem\-LabQwen2\.5\-7b\-rl\-rag\-q2\-em\-release commit history\.Note:[https://huggingface\.co/Mem\-Lab/Qwen2\.5\-7B\-RL\-RAG\-Q2\-EM\-Release/commits/main](https://huggingface.co/Mem-Lab/Qwen2.5-7B-RL-RAG-Q2-EM-Release/commits/main)Cited by:[2nd item](https://arxiv.org/html/2609.10441#S3.I1.i2.p1.1)\.
- Packeret al\.\(2023\)C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. GonzalezMemGPT: towards llms as operating systems\.arXiv preprint arXiv:2310\.08560\.Cited by:[§4\.2](https://arxiv.org/html/2609.10441#S4.SS2.p1.1)\.
- Penget al\.\(2023\)B\. Peng, J\. Quesnelle, H\. Fan, and E\. ShippoleYarn: efficient context window extension of large language models\.arXiv preprint arXiv:2309\.00071\.Cited by:[§4\.1](https://arxiv.org/html/2609.10441#S4.SS1.p1.1)\.
- Qwenet al\.\(2025\)Qwen, :, A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Tang, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. QiuQwen2\.5 technical report\.External Links:2412\.15115,[Link](https://arxiv.org/abs/2412.15115)Cited by:[1st item](https://arxiv.org/html/2609.10441#S3.I1.i1.p1.1)\.
- Rajpurkaret al\.\(2016\)P\. Rajpurkar, J\. Zhang, K\. Lopyrev, and P\. LiangSquad: 100,000\+ questions for machine comprehension of text\.arXiv preprint arXiv:1606\.05250\.Cited by:[§3\.1](https://arxiv.org/html/2609.10441#S3.SS1.SSS0.Px3.p1.1)\.
- Suet al\.\(2024\)J\. Su, M\. Ahmed, Y\. Lu, S\. Pan, W\. Bo, and Y\. LiuRoformer: enhanced transformer with rotary position embedding\.Neurocomputing568,pp\. 127063\.Cited by:[§1](https://arxiv.org/html/2609.10441#S1.p1.1)\.
- Sutton and Barto \(2018\)R\. S\. Sutton and A\. G\. BartoReinforcement learning: an introduction\.MIT press\.Cited by:[§1](https://arxiv.org/html/2609.10441#S1.p2.1)\.
- Teamet al\.\(2025\)K\. Team, Y\. Bai, Y\. Bao, G\. Chen, J\. Chen, N\. Chen, R\. Chen, Y\. Chen, Y\. Chen, Y\. Chen,et al\.Kimi k2: open agentic intelligence\.arXiv preprint arXiv:2507\.20534\.Cited by:[§1](https://arxiv.org/html/2609.10441#S1.p1.1)\.
- Team \(2024\)Q\. TeamQwen2\.5: a party of foundation models\.External Links:[Link](https://qwenlm.github.io/blog/qwen2.5/)Cited by:[1st item](https://arxiv.org/html/2609.10441#S3.I1.i1.p1.1)\.
- Wang and Chen \(2025\)Y\. Wang and X\. ChenMirix: multi\-agent memory system for llm\-based agents\.arXiv preprint arXiv:2507\.07957\.Cited by:[§4\.2](https://arxiv.org/html/2609.10441#S4.SS2.p1.1)\.
- Wanget al\.\(2025\)Y\. Wang, R\. Takanobu, Z\. Liang, Y\. Mao, Y\. Hu, J\. McAuley, and X\. WuMem\-\{\\\{\\\\backslashalpha\}\\\}: learning memory construction via reinforcement learning\.arXiv preprint arXiv:2509\.25911\.Cited by:[§1](https://arxiv.org/html/2609.10441#S1.p2.1),[2nd item](https://arxiv.org/html/2609.10441#S3.I1.i2.p1.1),[3rd item](https://arxiv.org/html/2609.10441#S3.I1.i3.p1.1),[§4\.3](https://arxiv.org/html/2609.10441#S4.SS3.p1.1)\.
- Xiaoet al\.\(2023\)G\. Xiao, Y\. Tian, B\. Chen, S\. Han, and M\. LewisEfficient streaming language models with attention sinks\.arXiv preprint arXiv:2309\.17453\.Cited by:[§4\.1](https://arxiv.org/html/2609.10441#S4.SS1.p1.1)\.
- Yanet al\.\(2025\)S\. Yan, X\. Yang, Z\. Huang, E\. Nie, Z\. Ding, Z\. Li, X\. Ma, K\. Kersting, J\. Z\. Pan, H\. Schütze,et al\.Memory\-r1: enhancing large language model agents to manage and utilize memories via reinforcement learning\.arXiv preprint arXiv:2508\.19828\.Cited by:[§4\.3](https://arxiv.org/html/2609.10441#S4.SS3.p1.1)\.
- Yanget al\.\(2025\)W\. Yang, J\. Xiao, H\. Zhang, Q\. Zhang, Y\. Wang, and B\. XuCoarse\-to\-fine grounded memory for llm agent planning\.arXiv preprint arXiv:2508\.15305\.Cited by:[§1](https://arxiv.org/html/2609.10441#S1.p2.1)\.
- Yanget al\.\(2018\)Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. Cohen, R\. Salakhutdinov, and C\. D\. ManningHotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InProceedings of the 2018 conference on empirical methods in natural language processing,pp\. 2369–2380\.Cited by:[Appendix A](https://arxiv.org/html/2609.10441#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.10441#S1.p5.1),[1st item](https://arxiv.org/html/2609.10441#S3.I2.i1.p1.1)\.
- Yuet al\.\(2025\)H\. Yu, T\. Chen, J\. Feng, J\. Chen, W\. Dai, Q\. Yu, Y\. Zhang, W\. Ma, J\. Liu, M\. Wang,et al\.MemAgent: reshaping long\-context llm with multi\-conv rl\-based memory agent\.arXiv preprint arXiv:2507\.02259\.Cited by:[Appendix A](https://arxiv.org/html/2609.10441#A1.SS0.SSS0.Px3.p1.1),[§B\.3](https://arxiv.org/html/2609.10441#A2.SS3.p1.1),[§1](https://arxiv.org/html/2609.10441#S1.p2.1),[2nd item](https://arxiv.org/html/2609.10441#S3.I1.i2.p1.1),[3rd item](https://arxiv.org/html/2609.10441#S3.I1.i3.p1.1),[§4\.2](https://arxiv.org/html/2609.10441#S4.SS2.p1.1),[§4\.3](https://arxiv.org/html/2609.10441#S4.SS3.p1.1)\.
- Zhanget al\.\(2024a\)H\. Zhang, T\. Ren, C\. Xiao, D\. Schuurmans, and B\. DaiProvable representation with efficient planning for partially observable reinforcement learning\.InProceedings of the 41st International Conference on Machine Learning,pp\. 59759–59782\.Cited by:[§4\.3](https://arxiv.org/html/2609.10441#S4.SS3.p1.1)\.
- Zhanget al\.\(2024b\)H\. Zhang, C\. Xiao, C\. Gao, H\. Wang, M\. Müller,et al\.Exploiting the replay memory before exploring the environment: enhancing reinforcement learning through empirical mdp iteration\.Advances in Neural Information Processing Systems37,pp\. 85658–85692\.Cited by:[§4\.3](https://arxiv.org/html/2609.10441#S4.SS3.p1.1)\.
- Zhanget al\.\(2023a\)H\. Zhang, C\. Xiao, H\. Wang, J\. Jin, B\. Xu, and M\. MüllerReplay memory as an empirical MDP: combining conservative estimation with experience replay\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=SjzFVSJUt8S)Cited by:[§4\.3](https://arxiv.org/html/2609.10441#S4.SS3.p1.1)\.
- Zhang and Yu \(2020\)H\. Zhang and T\. YuTaxonomy of reinforcement learning algorithms\.Deep reinforcement learning: Fundamentals, research and applications,pp\. 125–133\.Cited by:[§1](https://arxiv.org/html/2609.10441#S1.p2.1)\.
- Zhanget al\.\(2023b\)Z\. Zhang, Y\. Sheng, T\. Zhou, T\. Chen, L\. Zheng, R\. Cai, Z\. Song, Y\. Tian, C\. Ré, C\. Barrett,et al\.H2o: heavy\-hitter oracle for efficient generative inference of large language models\.Advances in Neural Information Processing Systems36,pp\. 34661–34710\.Cited by:[§4\.1](https://arxiv.org/html/2609.10441#S4.SS1.p1.1)\.
- Zhenget al\.\(2023\)L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. Xing,et al\.Judging llm\-as\-a\-judge with mt\-bench and chatbot arena\.Advances in neural information processing systems36,pp\. 46595–46623\.Cited by:[§3\.1](https://arxiv.org/html/2609.10441#S3.SS1.SSS0.Px3.p1.1)\.

## Appendix

## Appendix ADataset Details

In this section, we provide detailed descriptions of the source datasets and the construction methodology for the long\-context benchmarks used in our experiments\.

##### HotpotQA\.

HotpotQA\([Yang et al\., 2018](https://arxiv.org/html/2609.10441#bib.bib15)\)is a large\-scale dataset designed for multi\-hop question answering, collected from Wikipedia\. Unlike standard QA datasets that require only a single document, HotpotQA necessitates reasoning across multiple documents \(typically two or more\) to derive the correct answer\. Crucially, it provides supporting sentences to ensure the reasoning process is explainable\. In the context of long\-context evaluation, these supporting sentences serve as the ground truth for identifying golden paragraphs containing the necessary evidence\.

##### 2WikiMultiHopQA\.

2WikiMultiHopQA\([Ho et al\., 2020](https://arxiv.org/html/2609.10441#bib.bib16)\)builds upon the structure of HotpotQA but introduces more rigorous evidence chains\. It utilizes Wikipedia articles and structured Wikidata to generate questions\. A key advantage of 2WikiMultiHopQA is its comprehensive evidence paths \(including entity\-relation triples\), which reduce the likelihood of shortcut reasoning often observed in HotpotQA\.

##### RULER\-HotpotQA\.

We adopt the RULER\-HotpotQA\([Hsieh et al\., 2024](https://arxiv.org/html/2609.10441#bib.bib14)\)benchmark introduced by[Yu et al\. \(2025\)](https://arxiv.org/html/2609.10441#bib.bib8)\. This dataset synthesizes samples from HotpotQA by iteratively injecting distractor paragraphs to reach specific target context lengths \(ranging from 28k to 896k tokens\)\. Notably,[Yu et al\. \(2025\)](https://arxiv.org/html/2609.10441#bib.bib8)applied a Best\-Of\-2 filtering mechanism during construction to ensure task difficulty, discarding questions that base models could answer with 100% accuracy using only internal parametric knowledge\. We utilize this established benchmark to rigorously evaluate contextual reasoning capabilities in an in\-distribution setting\.

##### RULER\-2WikiMultiHopQA \(Ours\)\.

Building upon the synthesis framework of RULER\-HotpotQA used in MemAgent, we construct a novel OOD benchmark using the 2WikiMultiHopQA corpus\. We employ a similar Needle\-in\-a\-Haystack approach, inserting relevant paragraphs into a vast pool of noise documents to extend context lengths while preserving the original multi\-hop logic\. By transitioning to 2WikiMultiHopQA, we mitigate the inherent ambiguity issues present in HotpotQA and ensure that performance metrics reflect genuine retrieval and reasoning capabilities rather than dataset\-specific biases\. This dataset serves to validate the cross\-domain generalizability of long\-context agents\.

## Appendix BImplementation Details

In this section, we provide a comprehensive breakdown of the experimental environment, inference infrastructure, and algorithmic configurations to facilitate reproducibility\.

### B\.1Inference Infrastructure

All experiments were conducted on a high\-performance computing cluster equipped with NVIDIA A800 \(80GB\) GPUs\. To maximize inference throughput, we leveraged thevLLMlibrary\([Kwon et al\., 2023](https://arxiv.org/html/2609.10441#bib.bib31)\), utilizing its PagedAttention mechanism and continuous batching to manage the massive KV cache requirements of long\-context processing\. For the backbone model Qwen2\.5\-32B\-Instruct, we employedbfloat16precision to maintain numerical stability while optimizing memory usage\. To handle the concurrent kernel executions in ConvMem, we implemented an asynchronous job scheduler that dynamically batches independent segment inputs into the vLLM engine, ensuring maximum GPU utilization\.

### B\.2Algorithmic Configurations

##### Hierarchical Recursion\.

The hierarchical summarization process in ConvMem continues recursively until the total token count of the concatenated summaries fits within the kernel sizeWW\. Specifically, if the length of the intermediate contexth\(l\)h^\{\(l\)\}exceedsWW, a new layerl\+1l\+1is instantiated\. This typically results in a tree depth ofL∈\[2,3\]L\\in\[2,3\]for context lengths up to 128k, andL∈\[3,4\]L\\in\[3,4\]for extreme lengths up to 1M tokens, depending on the compression rateα\\alpha\.

##### Query Decomposition and Parsing\.

For multi\-kernel convolution, we enforce a flexible upper limit ofCmax=5C\_\{\\text\{max\}\}=5sub\-questions to prevent channel explosion, though in practice, the model typically generates 2\-4 sub\-questions\. We employ robust regular expression parsing to extract structured JSON outputs \(keywords, sub\-questions, and relevance scores\) from the LLM responses\. In rare cases of parsing failure, the system falls back to a default relevant state \(r=1r=1\) to prevent information loss\.

### B\.3Baselines Setup

To ensure a fair comparison, all baselines \(including RAG and MemAgent\) utilize the same backbone \(Qwen2\.5\-32B\-Instruct\) and identical hardware environment\. For RAG\-BM25, we retrieve the top\-kkchunks wherekkis dynamically calculated to fill the model’s effective context window\. For MemAgent, we adhere strictly to the hyperparameter settings reported in the original paper\([Yu et al\., 2025](https://arxiv.org/html/2609.10441#bib.bib8)\)\.

### B\.4Hyperparameters

The specific hyperparameters used for ConvMem across all experiments are detailed in[Table2](https://arxiv.org/html/2609.10441#A2.T2)\.

Table 2:Detailed Hyperparameters for ConvMem\.ConfigurationValueModel SettingsBackbone ModelQwen2\.5\-32B\-InstructPrecisionbfloat16Inference EnginevLLM \(v0\.6\.0\)ConvMem ArchitectureKernel Size \(WW\)8,000 tokensStride \(SS\)1,600 tokensOver\-scanning Factor \(W/SW/S\)5Max Channels \(CmaxC\_\{\\text\{max\}\}\)10Recursion Stop ConditionSequence Length<W<WGeneration ParametersTemperature0\.7Top\-P0\.95Skip Connection FlagTrue

## Appendix CFull Experimental Results

We present the comprehensive performance comparison across all context lengths \(28k\-896k\) with four metrics in[Table3](https://arxiv.org/html/2609.10441#A3.T3)\.

Table 3:Full performance comparison on RULER\-HotpotQA and 2WikiMultiHopQA datasets with four metrics: : F1, Exact Match \(EM\), Substring Exact Match \(Sub\-EM\), and LLM\-as\-a\-Judge \(ACCL\\text\{ACC\}\_\{L\}\)\.CategoryModelMetricRULER\-HotpotQA2WikiMultiHopQA28k56k112k224k448k896k28k56k112k224k448k896kBase ModelQwen2\.5\-32B\-InstructF161\.957\.7748\.8644\.0418\.5916\.7856\.5257\.250\.4541\.7329\.4520\.93EM47\.6644\.5334\.3832\.818\.597\.0347\.6647\.6642\.9734\.3824\.2217\.19Sub\-EM69\.5363\.2853\.9144\.5317\.1917\.9767\.1965\.6262\.550\.034\.3828\.12ACCL\\text\{ACC\}\_\{L\}80\.6275\.7866\.1360\.3133\.0930\.1672\.8171\.7668\.0555\.2738\.7931\.76Training\-FreeMethodsRAG\-BM25F145\.9851\.1147\.2135\.8629\.2730\.6734\.5431\.1724\.5523\.1921\.2517\.34EM32\.0336\.7234\.3821\.8819\.5320\.3125\.021\.8816\.5117\.1915\.6210\.94Sub\-EM46\.0950\.0048\.4435\.9430\.4732\.8157\.0358\.5947\.6646\.8839\.0643\.75ACCL\\text\{ACC\}\_\{L\}56\.9162\.1959\.8049\.4146\.6846\.8059\.2656\.9148\.2445\.5139\.5338\.55MemAgent\-W/O\-RLF163\.9565\.2862\.5257\.9855\.4258\.7360\.5157\.4245\.3150\.647\.6649\.65EM45\.3148\.4444\.5340\.6239\.0642\.9746\.8849\.2245\.3139\.8436\.7239\.06Sub\-EM67\.9771\.0965\.6259\.3856\.2560\.1671\.0967\.9758\.5963\.2860\.1662\.5ACCL\\text\{ACC\}\_\{L\}80\.6281\.3778\.6373\.3272\.1176\.1376\.9171\.9966\.2167\.0765\.1264\.54Mem\-α\\alpha\-W/O\-RLF16\.137\.786\.57\.037\.197\.356\.566\.475\.946\.195\.196\.0EM0\.00\.00\.00\.00\.00\.00\.00\.00\.00\.00\.00\.0Sub\-EM56\.2553\.1243\.7546\.8846\.8848\.3657\.3859\.3862\.553\.1253\.1250\.0ACCL\\text\{ACC\}\_\{L\}73\.7572\.0365\.1667\.1967\.6649\.2762\.9762\.9155\.9458\.5960\.3154\.0RL\-TrainedMethodsMemAgentF175\.5575\.2075\.2673\.5473\.1068\.8060\.9260\.1163\.1958\.8258\.558\.41EM60\.1659\.3855\.4756\.2555\.4752\.3446\.8848\.4450\.046\.0946\.8846\.09Sub\-EM79\.6979\.6980\.4777\.3478\.9174\.2270\.3171\.0972\.6669\.5367\.9769\.31ACCL\\text\{ACC\}\_\{L\}87\.9987\.0387\.2986\.2783\.1080\.0373\.5573\.4172\.6672\.1170\.1270\.95Mem\-α\\alphaF15\.846\.817\.06\.091\.310\.691\.251\.130\.941\.030\.971\.09EM0\.00\.00\.00\.00\.00\.00\.00\.00\.00\.00\.00\.0Sub\-EM34\.3840\.6240\.6243\.7543\.7525\.059\.3856\.2550\.062\.550\.053\.12ACCL\\text\{ACC\}\_\{L\}55\.1658\.7568\.5955\.3132\.0336\.7244\.6939\.2239\.3840\.030\.7853\.12OursConvMemF167\.4467\.8657\.8163\.2756\.1463\.0972\.371\.2567\.2161\.9661\.3359\.06EM47\.6647\.6639\.0643\.7539\.0644\.5360\.9457\.0352\.3650\.7851\.5646\.88Sub\-EM73\.4472\.6667\.1969\.5362\.5069\.5382\.8182\.4777\.3471\.8873\.3170\.62ACCL\\text\{ACC\}\_\{L\}83\.7981\.7274\.9678\.9872\.5477\.6685\.3180\.9475\.9472\.5873\.5971\.36

## Appendix DQualitative Analysis of Model Behaviors

In this section, we analyze the distinct failure modes of baseline models, providing empirical evidence for the limitations of sequential architectures and the overfitting tendencies of RL\-trained agents\.

### D\.1Sequential Information Loss in Recurrent Architectures

For multi\-hop questions where evidence is distributed across the text stream, sequential memory agents \(e\.g\., MemAgent\) suffer from a forgetting bottleneck\. Since they update a fixed\-size memory step\-by\-step, early information that seems irrelevant at the time of reading is often discarded, making it unrecoverable when its relevance is later revealed by subsequent context\.

Case 1: Disconnected EvidenceQuery:What government position was held by the woman who portrayed Corliss Archer in the film Kiss and Tell? Context:•Document 1053: Shirley Temple Shirley Temple Black \(April 23, 1928 – February 10, 2014\) was an American actress, singer, dancer, businesswoman, and diplomat who was Hollywood’s number one box\-office draw as a child actress from 1935 to 1938\. As an adult, she was named United States ambassador to Ghana and to Czechoslovakia and also served as Chief of Protocol of the United States\.•Document 1348: Kiss and Tell \(1945 film\) Kiss and Tell is a 1945 American comedy film starring then 17\-year\-old Shirley Temple as Corliss Archer\. In the film, two teenage girls cause their respective parents much concern when they start to become interested in boys\. The parents’ bickering about which girl is the worse influence causes more problems than it solves\.Analysis:The document mentioning “Shirley Temple Black” \(Document 1053\) appears early in the stream\. The document linking her to the film “Kiss and Tell” \(Document 1348\) appears much later\. By the time the agent reads Document 1348 and realizes the relevance of Shirley Temple, the memory of her specific government position \(Ambassador/Chief of Protocol\) has already been forgotten\. ConvMem avoids this by processing both segments in parallel channels\.

Case 2: Retrospective ReasoningQuery:Who is the younger brother of the episode guest stars of The Hard Easy? Context:•Document 591: Brian Doyle\-Murray Brian Doyle\-Murray \(born Brian Murray, October 31, 1945\) is an American actor, voice artist, comedian and screenwriter\. He is the older brother of actor/comedian Bill Murray, and the two have acted together in several films, including “Caddyshack”, “Scrooged”, “Ghostbusters II”, “The Razor’s Edge”, and “Groundhog Day”\. He co\-starred on the TBS sitcom on “Sullivan & Son”, where he played the foul\-mouthed Hank Murphy\. he also appeared in the Cartoon Network original animated series “The Marvelous Misadventures of Flapjack” as the surly Captain K’Nuckles and a pirate ghost, The Flying Dutchman from the Nickelodeon animated series, “SpongeBob SquarePants”, he appears in a recurring role as Don Ehlert on the ABC sitcom “The Middle”\.•Document 1348: The Hard Easy \(Adventure Time\) “The Hard Easy” is the twenty\-third episode of the fourth season of the American animated television series “Adventure Time”\. The episode was written and storyboarded by Tom Herpich and Skyler Page, from a story by Patrick McHale, Kent Osborne, and Pendleton Ward\. It originally aired on Cartoon Network on October 1, 2012\. The episode guest stars Brian Doyle\-Murray as Prince Huge and Jonathan Katz as the Mudscamp elder\.Analysis:Similar to Case 1, the relationship between Brian Doyle\-Murray and Bill Murray is established in Document 591\. The connection to the episode “The Hard Easy” is only revealed in Document 1348\. A sequential agent reading Document 591 has no incentive to retain the sibling relationship in memory, leading to failure when the query eventually demands this specific fact\. Sequential agents lack the “global view” required to link these distant facts\.

### D\.2RL\-Induced Overfitting and Parametric Hallucination

Our experiments reveal that RL\-trained agents \(e\.g\., MemAgent\) tend to overfit to the training data distribution, prioritizing memorized parametric priors over the provided context\.

Case 3: Parametric Bias vs\. ContextQuery:Which filmmaker was known for animation, Lev Yilmaz or Pamela B\. Green? Context Evidence:The document explicitly states “LevYilmaz”\. Dataset Label \(Typo\):“LevniYilmaz”\. MemAgent Output:“Levni Yilmaz”\. Analysis:The RL agent ignores the provided context and outputs the memorized label \(Levni\), proving it relies on parametric priors rather than reasoning\. ConvMem faithfully extracts “Lev Yilmaz”, demonstrating adherence to the input\.

Case 4: Hallucinating CollaboratorsQuery:Ellie Goulding worked with what other writers on her third studio album, Delirium?Context:•Document 37:On My Mind \(Ellie Goulding song\) "On My Mind" is a song by English singer Ellie Goulding from her third studio album "Delirium" \(2015\)\. It was released as the album’s lead single on 17 September 2015\. It was written by Goulding, Max Martin, Savan Kotecha and Ilya Salmanzadeh\. "On My Mind" is an electropop and R&B song whose instrumentation consists of scratchy guitars, trap drums, slapped beats and sharp, syncopated electronica\. Lyrically, "On My Mind" talks about a one\-night stand with someone the protagonist shouldn’t be with, having a dichotomy between heart and head\. Though firmly denied by Goulding, many critics considered it an answer song to Ed Sheeran’s "Don’t"•Document 41:Love Me like You Do "Love Me like You Do " is a song recorded by English singer Ellie Goulding for the "Fifty Shades of Grey" \(2015\)\. The song was written by Savan Kotecha, Ilya Salmanzadeh, Tove Lo, Max Martin and Ali Payami; the latter two also produced it\. Goulding was selected to sing the track\. It was released on 7 January 2015 as the second single from the soundtrack\. The song was also included on Goulding’s third studio album, "Delirium" \(2015\)\.Ground Truth:Max Martin, Savan Kotecha and Ilya SalmanzadehAnalysis:In this example, both “On My Mind” and “Love Me Like You Do” are featured on the album Delirium\. The former’s collaborators include Max Martin, Savan Kotecha, and Ilya Salmanzadeh, while the latter’s collaborators include Savan Kotecha, Ilya Salmanzadeh, Tove Lo, Max Martin, and Ali Payam\. Large models that have undergone enhanced training consistently provide the correct answer—the former\. Observing untrained large models reveals a tendency to list more collaborators for the latter, as it includes all collaborators from the former\.

## Appendix EData Quality and Metric Justification

Standard Exact Match \(EM\) is often too rigid for long\-context generation\. In this section, we justify our use of robust metrics \(Sub\-EM andACCL\\text\{ACC\}\_\{L\}\) using concrete examples of dataset ambiguity and valid variations\.

### E\.1Dataset Ambiguity and Missing Evidence

Some queries in the synthetic benchmarks are inherently ambiguous or unanswerable based strictly on the provided context, highlighting flaws in the dataset generation process\.

Case 5: Missing EvidenceQuery:Brown State Fishing Lake is in a country that has a population of how many inhabitants? Context:•Document 44:Brown County, Kansas Brown County \(county code BR\) is a county located in the northeast portion of the U\.S\. state of Kansas\. As of the 2010 census, the county population was 9,984\. Its county seat and most populous city is Hiawatha\. Brown County is the location of the Kickapoo Indian Reservation of Kansas, the majority of the Sac and Fox Reservation and the majority of the Iowa Reservation of Kansas and Nebraska\.LLM answer:The problem is asking for the population of the country where Brown State Fishing Lake is located, which is the United States\. However, the provided memory only gives the population of Brown County, Kansas, which is 9,984 as of the 2010 census\. It does not provide the population of the United States\. Therefore, the answer is that the population of the United States cannot be determined from the provided information\. Analysis:The term “country” in the question is ambiguous, as it can refer to either a “nation” or a “rural area/region”\. Observing the large model’s reasoning process, the task typically involves information about a “nation”, yet there is no associated data regarding the population of that “nation”, making it impossible to provide a correct response\. The query explicitly asks for the population of thecountry\(i\.e\., the United States\)\. However, the provided context only contains the population ofBrown County\(9,984\)\. The system is forced to either hallucinate the U\.S\. population using internal parametric knowledge or incorrectly answer with the county population\. This highlights a flaw in the dataset generation process where essential evidence \(U\.S\. population data\) was excluded from the context window\.

### E\.2The Need for Sub\-EM \(Substring Exact Match\)

We observed that strict Exact Match \(EM\) often fails when the model generates correct but verbose answers, valid aliases, or redundant confirmations\. This necessitates the use of more robust metrics such as Sub\-EM\.

Case 6: Name VariationQuery:What British singer\-songwriter hosted the 16th Annual Young Hollywood Awards? Context:•Document 48:Kelly Osbourne, Kelly Lee Osbourne \(born 27 October 1984\) is a British singer\-songwriter, actress, television presenter and fashion designer\. The daughter of Ozzy and Sharon Osbourne, she is known for her appearances on "The Osbournes" with her family, for which they won a 2002 Emmy Award for Outstanding Reality Program, as well as on E\!’s "Fashion Police", where she was a presenter from 2010 to 2015\. She has also appeared on "Dancing with the Stars", in which she and her professional dance partner Louis van Amstel took third place\. She is the voice of Hildy Gloom in the Disney XD animated series "The 7D"\. She is also a judge on both "Australia’s Got Talent" and "Project Runway Junior"Ground Truth:Kelly Lee Osbourne Model Prediction:Kelly Osbourne Evaluation:EM = 0, Sub\-EM = 1\. The model is correct but penalized by EM for omitting the middle name\.

Case 7: Verbose GenerationQuery:Who was known by his stage name Aladin and helped organizations improve their performance as a consultant? Ground Truth:Eenasul Fateh LLM answer:Eenasul Fateh, known by his stage name Aladin, who has a background as a former international management consultant and has helped organizations improve their performance\. Scores:\{ F1: 0\.16, EM=0,Sub\-EM=1\.0\}\. The answer contains the ground truth but includes extra context copied from the document\.

Case 8: Redundant Confirmationinput:Are Local H and For Against both from the United States? Ground Truth:yes LLM answer:yes, both Local H and For Against are from the United States\. eval:\{‘F1’: 0\.0, ‘EM’: 0\.0, ‘Sub\-EM’: 1\.0\} The model provides a complete sentence instead of a single word\.

### E\.3The Need for LLM\-as\-a\-Judge

In cases of acronyms or semantic rephrasing, even Sub\-EM fails\.ACCL\\text\{ACC\}\_\{L\}is essential to capture these equivalences\.

Case 9: Corporate Entity NamingQuery:Maxeda has been owned since 2004 by which American multinational equity firm? Context:•Document 18:Kohlberg Kravis Roberts, KKR & Co\. L\.P\. \(formerly known as Kohlberg Kravis Roberts & Co\.\) is an American multinational private equity firm, specializing in leveraged buyouts, headquartered in New York City\. The firm sponsors and manages private equity investment funds\. The firm has completed over $400 billion of private equity transactions since its inception\.Ground Truth:KKR & Co Model Prediction:Kohlberg Kravis Roberts Evaluation:EM = 0, Sub\-EM = 0,ACCL\\text\{ACC\}\_\{L\}=1\.0\. KKR is the abbreviation for Kohlberg Kravis Roberts; both are correct\. The model provided the full name instead of the abbreviation in the label\.

Case 10: Acronym MatchingQuery:What station broadcast the episode “Marry Me a Little, Marry Me a Little More”, of the series created by Max Mutchnick and David Kohan? Ground Truth:National Broadcasting Company Model Prediction:NBC Evaluation:F1 = 0, EM = 0, Sub\-EM = 0\. However, NBC is the acronym for the ground truth\.ACCL\\text\{ACC\}\_\{L\}correctly assigns a score of 1\.0\.

Case 11: Semantic RephrasingQuery:What was the 58th quadrennial American presidential election held after the 2016 Michigan Democratic primary? Ground Truth:United States presidential election of 2016 LLM answer:the 2016 U\.S presidential election Evaluation:\{F1: 0\.6, EM: 0\.0, Sub\-EM: 0\.0,ACCL\\text\{ACC\}\_\{L\}=1\.0\}\. The phrasing differs, but the semantic meaning is identical\.

## Appendix FPrompt Templates

We provide the prompt templates used in each stage of the ConvMem framework\.

### F\.1Query Decomposition

The following prompt is used to decompose the complex user query into independent sub\-questions \(channels\)\.

Multi\-Kernel Convolution PromptYou will encounter a complex reasoning problem requiring you to extract keywords or key phrases from the question that aid in solving it\. <Question\> \{question\} </Question\><Output Format\> \{\{ "subproblem": \["Subproblem 1", "Subproblem 2", …\.\], "keyword": \["Keyword1", "Keyword2",…\] \}\} </Output Format\> <Example\> Original Question: “Where was the director of the movie Inception born?” Sub\-questions: \[“Who is the director of ‘Inception’?”, “What information is there about the director’s birthplace?”\] Original Question: “What is the height of the male lead in The Revenant?” Keywords: \[“The Revenant protagonist”, “Human height”\] <Example\> <Note\> 1\. The decomposed subproblems must fully resolve the original problem\. 2\. The number of subproblems should be the minimum decomposition required to solve the original problem\. 3\. Keywords must be highly relevant to solving the original query, not irrelevant terms, and must not create ambiguity with the original query\. </Note\> Please complete the extraction of keywords from the question\.

### F\.2Semantic Kernel Operation

These prompts define the behavior of the convolutional kernel, including the kernel prompt, skip connection and summarization\.

Skip Connection PromptYou will see a question and a paragraph that may contain information related to the question\. Please read the paragraph carefully and make the relevant judgment\.<Question\> \{question\} </Question\> <Paragraph\> \{paragraph\} </Paragraph\> <Output Format\> \{\{"0": "Completely unrelated to the question"\}\} or \{\{"1": "Potentially related to the question"\}\} or \{\{"2": "Definitely contains the answer to the question"\}\} </Output Format\> <Notes\> 1\. To ensure no useful information is omitted, only select "0": "Completely unrelated to the question" if you are absolutely certain the paragraph content is irrelevant to the question\. 2\. Do not output any redundant information beyond the <Output Format\>\. </Notes\> Your Answer:

Kernel Summarization PromptYou will be presented with a question and a passage that may contain information relevant to answering it\. Read the passage carefully and update your memory with new information that helps solve the problem\. Be sure to retain all relevant details in your memory that may aid in solving the problem\.<Question\> \{question\} </Question\> <passage\> \{passage\} </passage\> <format\> Updated memory: \-………\. \-………\. \-………\. … </format\> <notes\> 1\. Retain all details within the paragraph that are relevant to the question, ensuring the integrity of the original content\. 2\. If none of the paragraph’s information relates to the question, output “Updated Memory: No relevant information\.” </notes\> Updated Memory:

### F\.3Hierarchical Aggregation

These prompts are used for recursive summarization in hidden layers and the final answer generation\.

Hidden Layer Aggregation PromptYou will see a question and \{num\} memory fragments\. Carefully review the provided memory fragments and combine information from each fragment that helps solve the problem\.<Question\> \{question\} </Question\> <Memory Fragment\> \{memory\_content\} </Memory Fragment\> <format\> Updated memory: \-………\. \-………\. \-………\. … </format\> <notes\> 1\. Retain all details within the paragraph that are relevant to the question, ensuring the integrity of the original content\. 2\. Avoid redundancy; if a fact already exists in memory, it need not be repeated unless that section provides additional clarification or correction\. </notes\> Updated Memory:

Sub\-Question Answer PromptYou will see a question and its preceding memory\. Answer the question based on the preceding memory\. Your response must include all details relevant to the question; omitting any information is prohibited\. Subsequent questions will require you to answer based on these details\. Note: If the memory does not contain information relevant to the question, simply answer “None\.”<question\> \{question\} </question\> <memory\> \{memory\} </memory\> Your answer:

Final Answer Generation PromptYou will see a question along with its associated dependency information\. Please read the question carefully, identify the core objective to be addressed, and answer based on the dependency information\. Organize your response using the following format: “"Hence, the answer is \(insert answer here\)\.”<Question\> \{question\} </Question\> <Related Dependency Information\> \{related\_dependency\_information\} </Related Dependency Information\> <notes\> 1\. Carefully read the <Question\> to precisely identify the target objective to be a ddressed, the focus of your response and avoid being misled by lengthy questions\. 2\. Carefully read <Relevant Dependency Information\> to capture details relevant to the question, avoiding distraction from extraneous information\. 3\. Whenever possible, derive answers directly from <Related Dependency Information\> without unnecessary rephrasing\. 4\. Avoid redundant explanations or unnecessary elaboration on the question’s answer\. For example: If the question asks for place names/person names/other information, etc\.you only need to provide the place names/person names/other information, etc\. themselves\. No additional background information is required\. </notes\> Your Answer:

### F\.4Evaluation

The prompt used for LLM\-as\-a\-Judge evaluation\.

LLM\-as\-a\-Judge PromptYou must evaluate the quality of the AI assistant’s responses to user questions as an impartial judge\. Scores should comprehensively consider accuracy \(high priority\) and completeness \(whether the response covers all key points\)\.Please rate on a scale of 0 to 10, where 0 indicates the response is completely incorrect and 10 indicates the response is completely correct\. <Submit your feedback in the following format\> \{\{"Overall Score": "\(Your rating, a floating\-point number between 0 and 10\)"\}\} </Submit your feedback in the following format\><Below are the question and answers\> Question: \{question\} Ground Truth: \{ground\_truth\} User Response: \{answer\} </Below are the question and answers\> Please complete the rating\.