PI-Mem: Pushing Long-Context Reasoning to 3.6M Tokens with Parallel-Iterative Memory

# Pushing Long-Context Reasoning to 3.6M Tokens with Parallel-Iterative Memory
Source: [https://arxiv.org/html/2608.03048](https://arxiv.org/html/2608.03048)
Dawei Liu1,2\\equalcontrib, Haixu Song2,3\\equalcontrib, Shuang Cheng2,4, Shijie Wang2, Haozheng Hou2,5, Kaifeng Liu2,5, Ermo Hua2,3, Zhonghang Yuan2,6, Zhijie Zhong2,1, Yuchen Fan2,1, Biqing Qi2\\corresponding, Bowen Zhou2

###### Abstract

Long\-context reasoning remains a critical bottleneck for large language models, as recent recurrent\-memory approaches face two inherent challenges: sequential chunk\-wise updates can overwrite early critical evidence with later irrelevant content, and serial inter\-chunk dependencies limit parallelism and cause latency to increase with context length\. To address these issues, we proposePI\-Mem\(Parallel\-Iterative Memory\), a mechanism that processes all chunks in parallel and iteratively refines a shared memory over a bounded number of turns\. In each turn, PI\-Mem reads all chunks in parallel conditioned on the current memory, selects new or complementary evidence from each chunk, and merges the selected evidence into a compact shared memory for the next turn\. To discourage redundant turns, we optimize the workflow through reinforcement learning with an auxiliary turn\-efficiency reward, enabling the model to adaptively exit once sufficient evidence has been accumulated\. We evaluate PI\-Mem with Qwen3\.5\-35B\-A3B and Qwen2\.5\-7B on the HotpotQA benchmark across context lengths up to 3\.6 million tokens and find that it outperforms the recurrent\-memory baseline by \+6\.25 and \+7\.81 absolute points while achieving 6\.1×\\timesand 2\.1×\\timesinference speedups, respectively\. These results demonstrate that PI\-Mem breaks the accuracy–efficiency trade\-off in long\-context reasoning and provides a scalable approach to complex multi\-hop question answering over extremely long documents\.

Code—https://github\.com/JetAstra/PI\-Mem

## Introduction

The ability to reason over long contexts is increasingly important for large language models \(LLMs\), supporting applications such as long\-document question answering\(Gemini Team[2025](https://arxiv.org/html/2608.03048#bib.bib31); Yanget al\.[2025b](https://arxiv.org/html/2608.03048#bib.bib37)\), multi\-turn dialogue\(Maharanaet al\.[2024](https://arxiv.org/html/2608.03048#bib.bib32); Liet al\.[2024](https://arxiv.org/html/2608.03048#bib.bib33); Wuet al\.[2025](https://arxiv.org/html/2608.03048#bib.bib38)\), repository\-level code understanding\(Yanget al\.[2024](https://arxiv.org/html/2608.03048#bib.bib34); Huiet al\.[2024](https://arxiv.org/html/2608.03048#bib.bib39)\), and agentic workflows over large external contexts\(Yaoet al\.[2023](https://arxiv.org/html/2608.03048#bib.bib40); Agasheet al\.[2024](https://arxiv.org/html/2608.03048#bib.bib36); Kimi Teamet al\.[2026](https://arxiv.org/html/2608.03048#bib.bib4)\)\. Yet model performance often degrades as context length grows, highlighting the persistent difficulty of identifying, retaining, and effectively utilizing relevant information across distant parts of the context\(Hsiehet al\.[2024](https://arxiv.org/html/2608.03048#bib.bib43); Duet al\.[2025](https://arxiv.org/html/2608.03048#bib.bib42)\)\.

![Refer to caption](https://arxiv.org/html/2608.03048v1/x1.png)Figure 1:Top: Recurrent\-memory workflows process chunks sequentially and repeatedly rewrite a running memory state, potentially overwriting early evidence and creating strict serial dependencies\. Bottom: PI\-Mem reads chunks in parallel conditioned on a shared global memory, improving evidence preservation and reducing inference latency\.One family of approaches addresses this challenge by extending the natively supported context window through positional\-encoding interpolation or extrapolation, thereby enabling direct processing of longer sequences without discarding input context\(Chenet al\.[2023](https://arxiv.org/html/2608.03048#bib.bib27); Penget al\.[2024](https://arxiv.org/html/2608.03048#bib.bib26); Liuet al\.[2025a](https://arxiv.org/html/2608.03048#bib.bib41)\)\. In practice, such extensions are generally reliable only within a limited extrapolation range and still incur the high computational cost of dense attention at ultra\-long context lengths\. To address this efficiency bottleneck, sparse attention restricts computation to selected tokens or key\-value blocks, whereas recurrent formulations of linear attention summarize preceding tokens in a compact state\. Nevertheless, these architectures are costly to train and often rely on high\-quality synthetic long\-context corpora\(Yuanet al\.[2025](https://arxiv.org/html/2608.03048#bib.bib28); Luet al\.[2025](https://arxiv.org/html/2608.03048#bib.bib29); Yanget al\.[2025c](https://arxiv.org/html/2608.03048#bib.bib30); Qwen Team[2026](https://arxiv.org/html/2608.03048#bib.bib5)\), whose construction and curation remain difficult at scale across diverse reasoning tasks\.

These limitations motivate recurrent\-memory processing as a complementary strategy for handling long inputs\. A recurrent\-memory mechanism maintains a fixed\-length textual memory as a compact state throughout input processing\. At each step, the model processes the current chunk together with the previous memory and generates an updated memory that overwrites the previous state\. Once all chunks have been processed, the model generates the final answer conditioned on the resulting memory\(Yuet al\.[2025a](https://arxiv.org/html/2608.03048#bib.bib1); Shenget al\.[2026](https://arxiv.org/html/2608.03048#bib.bib2); Shiet al\.[2025](https://arxiv.org/html/2608.03048#bib.bib23)\)\. In principle, bounding the context of each memory update allows the workflow to process inputs of arbitrary length at a computational cost that grows linearly rather than quadratically with input length\. However, as illustrated in Figure[1](https://arxiv.org/html/2608.03048#Sx1.F1), this recurrent\-memory design faces two challenges\. First, successive memory updates compress each new chunk and the existing fixed\-length memory into a new state, potentially overwriting early evidence with later noisy information before the relevance of the early evidence becomes apparent and thereby impairing evidence preservation and integration across distant chunks\. Second, the recurrent\-memory workflow imposes a strict inter\-chunk dependency, forcing each chunk to await the preceding memory update and increasing inference latency with context length\.

To address these challenges, we introduce PI\-Mem, a Parallel\-Iterative Memory mechanism for long\-context reasoning\. PI\-Mem gathers evidence from all chunks in*parallel*and uses it to update a shared memory over a bounded number of turns\. Specifically, we define each memory\-update*turn*as a complete*read\-select\-merge*cycle\. The read step processes all chunks in parallel, with each chunk\-level read conditioned on the same global memory, thereby avoiding sequential memory overwrites that may discard earlier evidence and reducing inference latency through parallel chunk processing\. PI\-Mem then selects observations that provide new or complementary evidence and merges them with the current memory to produce an updated global memory\. In the next turn, all chunks are read again conditioned on the updated memory, allowing evidence discovered in one chunk to guide extraction from other chunks and thereby enabling cross\-chunk information exchange\. This cycle repeats until either no chunk yields useful new evidence or a predefined maximum number of turns is reached, after which PI\-Mem generates the final answer from the question and the consolidated global memory\. To train PI\-Mem, we optimize the entire workflow end\-to\-end with reinforcement learning \(RL\), combining an answer\-accuracy reward with a turn\-efficiency reward that discourages redundant turns\.

Experiments with Qwen3\.5\-35B\-A3B and Qwen2\.5\-7B demonstrate the effectiveness and efficiency of PI\-Mem\. On HotpotQA \(HQA\) at 3\.6M tokens, PI\-Mem improves over MemAgent by \+6\.25 and \+7\.81 absolute points and delivers inference speedups of 6\.1×\\timesand 2\.1×\\times, respectively\.

Our main contributions are summarized as follows:

- •We propose PI\-Mem, a parallel\-iterative memory mechanism for long\-context reasoning that reads all chunks in parallel conditioned on a shared memory and iteratively refines the memory over a bounded number of turns\.
- •We optimize the workflow end\-to\-end with RL to improve long\-context performance and introduce a turn\-efficiency reward that discourages redundant turns\.
- •We empirically show that PI\-Mem outperforms the recurrent\-memory baseline and substantially reduces inference latency on HotpotQA at context lengths up to 3\.6M tokens, while also improving performance across diverse long\-context benchmarks\.

## Related Work

#### Memory\-Based Context Management\.

Memory\-based approaches retain salient information in external stores or compact textual states to operate beyond limited context windows\. Early work introduced hierarchical virtual context management, while production\-oriented layers extract, consolidate, and retrieve conversational information\(Packeret al\.[2023](https://arxiv.org/html/2608.03048#bib.bib10); Chhikaraet al\.[2025](https://arxiv.org/html/2608.03048#bib.bib11)\)\. Recent workflows recurrently update compact states over document chunks or interaction steps; related variants reconstruct evolving reports or reason over recalled compressed memories\(Yuet al\.[2025a](https://arxiv.org/html/2608.03048#bib.bib1); Zhouet al\.[2025](https://arxiv.org/html/2608.03048#bib.bib12); Chenet al\.[2026b](https://arxiv.org/html/2608.03048#bib.bib3),[c](https://arxiv.org/html/2608.03048#bib.bib24)\)\. These designs bound the active context but retain sequential state transitions\. Later work improves selectivity through update and exit gates, learned memory construction and editing, active retrieval and writing, multi\-scale folding, and adaptive routing\(Shenget al\.[2026](https://arxiv.org/html/2608.03048#bib.bib2); Wanget al\.[2025b](https://arxiv.org/html/2608.03048#bib.bib13); Zhanget al\.[2026](https://arxiv.org/html/2608.03048#bib.bib17); Wanget al\.[2026](https://arxiv.org/html/2608.03048#bib.bib25); Sunet al\.[2025](https://arxiv.org/html/2608.03048#bib.bib14); Yeet al\.[2025](https://arxiv.org/html/2608.03048#bib.bib15); Fenget al\.[2026](https://arxiv.org/html/2608.03048#bib.bib16)\)\. Our approach instead conditions all chunk\-level reads within each turn on the same memory, improving evidence preservation and reducing inference latency\.

#### Reinforcement Learning for Long\-Context Reasoning\.

RL with verifiable rewards, together with group\-relative policy optimization and its variants, has substantially improved multi\-step reasoning\(Shaoet al\.[2024](https://arxiv.org/html/2608.03048#bib.bib9); Yuet al\.[2025b](https://arxiv.org/html/2608.03048#bib.bib7)\)\. Long\-context studies extend this paradigm through supervised warm\-up, progressive context curricula, difficulty\-aware sampling, self\-play verification, and synthetic multi\-hop tasks with distractors\(Wanet al\.[2025](https://arxiv.org/html/2608.03048#bib.bib18); Yanget al\.[2026](https://arxiv.org/html/2608.03048#bib.bib19); Wanget al\.[2025a](https://arxiv.org/html/2608.03048#bib.bib20)\)\. These strategies stabilize optimization, adapt task difficulty as training progresses, and provide scalable reward signals when annotated long\-context data are scarce\. Beyond outcome\-only supervision, dense, verifiable context rewards directly guide grounding and evidence selection\(Chenet al\.[2026a](https://arxiv.org/html/2608.03048#bib.bib21)\)\. RL has also been used to train memory\-augmented workflows, including iterative memory fusion, historical\-memory retrieval, and policies that control retrieval, writing, and stopping through final\-answer and step\-level signals\(Shenet al\.[2025](https://arxiv.org/html/2608.03048#bib.bib22); Shiet al\.[2025](https://arxiv.org/html/2608.03048#bib.bib23); Wanget al\.[2026](https://arxiv.org/html/2608.03048#bib.bib25)\)\. Our approach applies trajectory\-level RL to a parallel\-iterative workflow, training the policy to coordinate evidence selection and memory refinement and discouraging redundant turns\.

![Refer to caption](https://arxiv.org/html/2608.03048v1/x2.png)Figure 2:Overview of the PI\-Mem workflow\. PI\-Mem reads all chunks in parallel conditioned on a shared global memory and iteratively refines this memory through read\-select\-merge turns\.

## Method

In this section, we first describe the PI\-Mem inference workflow and then present its RL training procedure\.

### PI\-Mem Inference Workflow

Figure[2](https://arxiv.org/html/2608.03048#Sx2.F2)provides an overview of the PI\-Mem inference workflow, and Algorithm[1](https://arxiv.org/html/2608.03048#alg1)presents its procedural details\. PI\-Mem reads all chunks in parallel and iteratively refines a shared memory over a bounded number of turns\. We define each memory\-update*turn*as a complete*read\-select\-merge*cycle: PI\-Mem reads all chunks conditioned on the current global memory, selects observations with new or complementary evidence, and merges them into an updated global memory\. We first detail the three operations within each turn and then describe iterative refinement and adaptive exit across turns\.

Algorithm 1PI\-Mem Inference WorkflowInput: Questionqq, contextCC, modelπθ\\pi\_\{\\theta\}, maximum number of turnsKK Output: Final answeraa Initialize: SplitCCinto chunks\{ci\}i=1n\\\{c\_\{i\}\\\}\_\{i=1\}^\{n\}; m\(0\)←EMPTYm^\{\(0\)\}\\leftarrow\\texttt\{EMPTY\}

1:

m⋆←m\(0\)m^\{\\star\}\\leftarrow m^\{\(0\)\}
2:for

k=1k=1to

KKdo

3:Read each chunk

cic\_\{i\}in*parallel*:

4:

oi\(k\)←ReadCall​\(πθ,q,ci,m\(k−1\)\)o\_\{i\}^\{\(k\)\}\\leftarrow\\textsc\{ReadCall\}\(\\pi\_\{\\theta\},q,c\_\{i\},m^\{\(k\-1\)\}\)
5:

O\(k\)←\{oi\(k\)∣χ​\(oi\(k\)\)=yes\}O^\{\(k\)\}\\leftarrow\\\{\\,o\_\{i\}^\{\(k\)\}\\mid\\chi\(o\_\{i\}^\{\(k\)\}\)=\\text\{\{yes\}\}\\,\\\}
6:if

O\(k\)=∅O^\{\(k\)\}=\\emptysetthen

7:break

8:else

9:

m\(k\)←MergeCall​\(πθ,q,m\(k−1\),O\(k\)\)m^\{\(k\)\}\\leftarrow\\textsc\{MergeCall\}\(\\pi\_\{\\theta\},q,m^\{\(k\-1\)\},O^\{\(k\)\}\)
10:

m⋆←m\(k\)m^\{\\star\}\\leftarrow m^\{\(k\)\}
11:endif

12:endfor

13:

a←FinalCall​\(πθ,q,m⋆\)a\\leftarrow\\textsc\{FinalCall\}\(\\pi\_\{\\theta\},q,m^\{\\star\}\)
14:return

aa

#### Read\-Select\-Merge Turn\.

For a general QA task, we denote the question byqq, the full background context byCC, and the model byπθ\\pi\_\{\\theta\}\. We first splitCCinto equal\-sized chunks\{ci\}i=1n\\\{c\_\{i\}\\\}\_\{i=1\}^\{n\}and initialize the global memorym\(0\)m^\{\(0\)\}as an empty memory\. At turnkk, PI\-Mem updates the global memory through a*read\-select\-merge*turn:

m\(k\)=Turn​\(πθ,q,m\(k−1\),\{ci\}i=1n\)\.m^\{\(k\)\}=\\textsc\{Turn\}\(\\pi\_\{\\theta\},q,m^\{\(k\-1\)\},\\\{c\_\{i\}\\\}\_\{i=1\}^\{n\}\)\.\(1\)The goal of each turn is to update the global memory by processing all chunks in parallel\.

In thereadstep, PI\-Mem processes all chunks in parallel\. For each chunkcic\_\{i\},ReadCallplacesqq,cic\_\{i\}, andm\(k−1\)m^\{\(k\-1\)\}into a chunk\-reading template and callsπθ\\pi\_\{\\theta\}to generate a chunk\-level observationoi\(k\)o\_\{i\}^\{\(k\)\}\. Rather than successively overwriting the memory as in sequential recurrent processing, PI\-Mem reads all chunks independently conditioned on the same memorym\(k−1\)m^\{\(k\-1\)\}, thereby reducing the risk that evidence from early chunks is overwritten by noisy information from later chunks\. This independence also enables PI\-Mem to read chunks in parallel and efficiently batch the corresponding calls, allowing ultra\-long sequences to be processed with lower inference latency\. Each chunk call must explicitly output a check signal, either<check\>yes</check\>or<check\>no</check\>\. The model outputsyeswhencic\_\{i\}provides evidence relevant toqqthat is new or complementary tom\(k−1\)m^\{\(k\-1\)\}\. When the check signal isyes, the output includes the corresponding evidence needed for answeringqq; otherwise, the model outputs onlyno\.

Theselectstep filters observations by the<check\>signal\. Letχ​\(oi\(k\)\)\\chi\(o\_\{i\}^\{\(k\)\}\)denote the value enclosed by the check tags; the selected observation set is defined as

O\(k\)=\{oi\(k\)∣χ​\(oi\(k\)\)=yes\}\.O^\{\(k\)\}=\\\{\\,o\_\{i\}^\{\(k\)\}\\mid\\chi\(o\_\{i\}^\{\(k\)\}\)=\\text\{\{yes\}\}\\,\\\}\.\(2\)This filtering removes observations that provide no useful update, so the memory update focuses on new or complementary evidence rather than irrelevant or redundant text\.

Themergestep consolidates the selected observations into the updated global memory\. Specifically,MergeCallplacesqq,m\(k−1\)m^\{\(k\-1\)\}, andO\(k\)O^\{\(k\)\}into a merge template and invokesπθ\\pi\_\{\\theta\}to producem\(k\)m^\{\(k\)\}\. A straightforward alternative is to concatenate all positive chunk outputs, but this can still produce an overly long intermediate context when many chunks contain relevant evidence\. The merge step instead compresses and integrates evidence from different chunks, removes redundancy, preserves details necessary for answeringqq, and prevents unbounded memory growth as the number of selected observations increases\. Thus, a single turn updates the global memory through parallel evidence extraction, selective filtering, and compact cross\-source integration\.

#### Iterative Refinement and Exit Mechanism\.

After turnkkfinishes, PI\-Mem feeds the merged memorym\(k\)m^\{\(k\)\}into the next read step and repeats the read\-select\-merge cycle\. Each later turn therefore reads all chunks again, conditioned on evidence discovered in previous turns\. Compared with a single greedy turn, in which chunk relevance is judged only from the initial empty memory, iterative refinement reduces the risk of overlooking evidence whose importance becomes clear only after other facts have been identified and enables cross\-chunk information exchange through the shared memory\.

PI\-Mem further uses the check signal as anadaptive exit mechanism\. The loop automatically stops when no chunk produces useful new evidence, i\.e\.,O\(k\)=∅O^\{\(k\)\}=\\emptyset, or when it reaches the maximum number of turnsKK\. Finally, PI\-Mem invokes the model once more to answer using only the questionqqand the final global memorym⋆m^\{\\star\}, rather than the original long context, so that final inference remains compact and grounded in the consolidated evidence\.

Table 1:Length\-grouped RULER HQA results\. Scores are reported across context lengths from 7K to 3\.6M tokens\. The averages for GRU\-Mem and ReMemR1 use the eight reported lengths\.

### Trajectory\-Level Reinforcement Learning

We adopt GRPO\(Shaoet al\.[2024](https://arxiv.org/html/2608.03048#bib.bib9)\)to optimize the multi\-call workflow end to end\. During RL training, each rollout for a query defines a complete trajectory\. Since the workflow invokes the model multiple times, each trajectory contains multiple model\-call samples, including read, merge, and final\-answer calls\. We denote theii\-th trajectory by

τi=\{\(xi,s,yi,s\)\}s=1Si,\\tau\_\{i\}=\\\{\(x\_\{i,s\},y\_\{i,s\}\)\\\}\_\{s=1\}^\{S\_\{i\}\},\(3\)whereSiS\_\{i\}is the number of model\-call samples in trajectoryτi\\tau\_\{i\},ssindexes one such model call,xi,sx\_\{i,s\}is the corresponding phase\-specific prompt constructed from the workflow state, andyi,sy\_\{i,s\}is the generated response\. For each training query, we sample a group ofGGcomplete trajectories\{τi\}i=1G\\\{\\tau\_\{i\}\\\}\_\{i=1\}^\{G\}from the old policyπθold\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}and use all model\-call samples in these trajectories for policy optimization\. We next describe how the trajectory\-level reward is constructed and used for end\-to\-end workflow optimization\.

#### Reward Design\.

We assign one reward to each completed trajectoryτi\\tau\_\{i\}based on its final answeraia\_\{i\}and turn countkik\_\{i\}\. Letracc​\(ai\)r\_\{\\mathrm\{acc\}\}\(a\_\{i\}\)denote the accuracy reward andKKthe maximum number of turns\. We augment answer accuracy with a turn\-efficiency bonus:

rturn​\(τi\)=K−kiK−1;Ri=racc​\(ai\)\+λturn​rturn​\(τi\)\.r\_\{\\mathrm\{turn\}\}\(\\tau\_\{i\}\)=\\frac\{K\-k\_\{i\}\}\{K\-1\};\\ R\_\{i\}=r\_\{\\mathrm\{acc\}\}\(a\_\{i\}\)\+\\lambda\_\{\\mathrm\{turn\}\}r\_\{\\mathrm\{turn\}\}\(\\tau\_\{i\}\)\.\(4\)The turn bonus assigns higher rewards to trajectories that exit in fewer turns, andλturn\\lambda\_\{\\mathrm\{turn\}\}controls its contribution\. It discourages redundant refinement once the memory is sufficient and encourages the workflow to exit when no new evidence is found, reducing latency\.

#### Trajectory\-Level Optimization\.

This scalar reward is then broadcast to every model\-call sample\(xi,s,yi,s\)\(x\_\{i,s\},y\_\{i,s\}\)in the same trajectory\. Following Dr\. GRPO\(Liuet al\.[2025b](https://arxiv.org/html/2608.03048#bib.bib8)\), we omit normalization by the group standard deviation and compute the group\-relative advantage over complete trajectories rather than individual calls:

Ai=Ri−1G​∑j=1GRj\.A\_\{i\}=R\_\{i\}\-\\frac\{1\}\{G\}\\sum\_\{j=1\}^\{G\}R\_\{j\}\.\(5\)Given this trajectory\-level advantage, each token in every model\-call sample of trajectoryτi\\tau\_\{i\}uses the sameAiA\_\{i\}in the per\-token clipped surrogate term:

ℓi,s,t​\(θ\)=\\displaystyle\\ell\_\{i,s,t\}\(\\theta\)=min\(ρi,s,t\(θ\)Ai,\\displaystyle\\min\\\!\\Big\(\\rho\_\{i,s,t\}\(\\theta\)A\_\{i\},\(6\)clip\(ρi,s,t\(θ\),1−ε,1\+ε\)Ai\),\\displaystyle\\operatorname\{clip\}\(\\rho\_\{i,s,t\}\(\\theta\),1\-\\varepsilon,1\+\\varepsilon\)A\_\{i\}\\Big\),whereε\\varepsilondenotes the clipping ratio, and the token\-level policy ratio is

ρi,s,t​\(θ\)=πθ​\(yi,s,t∣xi,s,yi,s,<t\)πθold​\(yi,s,t∣xi,s,yi,s,<t\)\.\\rho\_\{i,s,t\}\(\\theta\)=\\frac\{\\pi\_\{\\theta\}\(y\_\{i,s,t\}\\mid x\_\{i,s\},y\_\{i,s,<t\}\)\}\{\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(y\_\{i,s,t\}\\mid x\_\{i,s\},y\_\{i,s,<t\}\)\}\.\(7\)The final objective uses DAPO\-style token\-level aggregation\(Yuet al\.[2025b](https://arxiv.org/html/2608.03048#bib.bib7)\)over all model\-call samples in the sampled trajectories\. LetZ=∑i=1G∑s=1Si\|yi,s\|Z=\\sum\_\{i=1\}^\{G\}\\sum\_\{s=1\}^\{S\_\{i\}\}\|y\_\{i,s\}\|denote the total number of generated tokens\. The objective is

𝒥​\(θ\)\\displaystyle\\mathcal\{J\}\(\\theta\)=𝔼\(q,a\)∼𝒟,\{yi,s\}∼πθold\(⋅∣xi,s\)\\displaystyle=\\mathbb\{E\}\_\{\\begin\{subarray\}\{c\}\(q,a\)\\sim\\mathcal\{D\},\\\{y\_\{i,s\}\\\}\{\\sim\}\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(\\cdot\\mid x\_\{i,s\}\)\\end\{subarray\}\}\(8\)\[\\displaystyle\\Bigg\[1Z∑i=1G∑s=1Si∑t=1\|yi,s\|\(ℓi,s,t\(θ\)−βKL\(πθ∥πref\)\)\]\.\\displaystyle\\frac\{1\}\{Z\}\\sum\_\{i=1\}^\{G\}\\sum\_\{s=1\}^\{S\_\{i\}\}\\sum\_\{t=1\}^\{\|y\_\{i,s\}\|\}\\Big\(\\ell\_\{i,s,t\}\(\\theta\)\-\\beta\\mathrm\{KL\}\(\\pi\_\{\\theta\}\\\|\\,\\pi\_\{\\mathrm\{ref\}\}\)\\Big\)\\Bigg\]\.Here,\(q,a\)\(q,a\)denotes a question and its reference answer sampled from the data distribution𝒟\\mathcal\{D\}\. The coefficientβ\\betascales the token\-level KL divergence between the current policyπθ\\pi\_\{\\theta\}and the reference policyπref\\pi\_\{\\mathrm\{ref\}\}\.

## Experiments

### Training Setup

We evaluate our method on two representative models: Qwen3\.5\-35B\-A3B\(Qwen Team[2026](https://arxiv.org/html/2608.03048#bib.bib5)\), an MoE model with hybrid attention, and Qwen2\.5\-7B\-Instruct\(Yanget al\.[2025a](https://arxiv.org/html/2608.03048#bib.bib6)\), a dense model with full attention\.

For the Qwen3\.5\-35B\-A3B experiments, following the data construction protocol of MemAgent\(Yuet al\.[2025a](https://arxiv.org/html/2608.03048#bib.bib1)\), we synthesize long\-context QA training samples by embedding HotpotQA\(Yanget al\.[2018](https://arxiv.org/html/2608.03048#bib.bib45)\)gold paragraphs into distractor documents sampled from the same dataset\. Each training sample contains 1,000 documents, yielding a context length of approximately 140K tokens\. PI\-Mem splits the context into 15K\-token chunks\. Across the workflow, all generation stages use the same maximum output length of 4,096 tokens, including chunk\-level memory generation, merged\-memory generation, and final\-answer generation\. Thinking mode is disabled during training to improve rollout efficiency and reduce the generated output length of each model call\. We set the maximum number of turns toK=3K=3\. For a fair comparison, the MemAgent baseline is trained with the same RL hyperparameters: a rollout batch size of 128 prompts, a GRPO group size of 8 rollouts per prompt, and 80 rollout steps, corresponding to 10,240 training samples counted at the prompt level\.

For Qwen2\.5\-7B, each HotpotQA training sample contains 200 documents \(approximately 28K tokens\)\. PI\-Mem uses 5K\-token chunks and a maximum output length of 1,024 tokens\. We use 16 rollouts per prompt and 240 rollout steps, and compare against the officially released MemAgent checkpoint with the same backbone\.

### Evaluation Setup

#### Baselines\.

We compare PI\-Mem with direct\-inference, retrieval\-augmented, and recurrent\-memory baselines\. Vanilla denotes standard direct inference using the backbone model\. We also include YaRN\(Penget al\.[2024](https://arxiv.org/html/2608.03048#bib.bib26)\), a commonly used positional\-encoding extrapolation method, as a direct\-inference baseline\. RAG\(Lewiset al\.[2020](https://arxiv.org/html/2608.03048#bib.bib35)\)retrieves relevant chunks from the long context and performs direct inference over the retrieved evidence\. We include MemAgent\(Yuet al\.[2025a](https://arxiv.org/html/2608.03048#bib.bib1)\), GRU\-Mem\(Shenget al\.[2026](https://arxiv.org/html/2608.03048#bib.bib2)\), and ReMemR1\(Shiet al\.[2025](https://arxiv.org/html/2608.03048#bib.bib23)\)as recurrent\-memory workflow baselines\. We report theRL\-trained variantsof MemAgent and PI\-Mem and use the published HQA results for GRU\-Mem and ReMemR1\.

#### Benchmarks\.

We evaluate long\-context performance on RULER\(Hsiehet al\.[2024](https://arxiv.org/html/2608.03048#bib.bib43)\)and LongBench v2\(Baiet al\.[2024](https://arxiv.org/html/2608.03048#bib.bib44)\)\. RULER is a synthetic benchmark that probes retrieval, multi\-hop tracing, aggregation, and question answering over long contexts\. LongBench v2 contains 503 challenging multiple\-choice questions across six realistic task categories\. In our RULER results, all tasks*except HQA*are treated as out\-of\-distribution \(OOD\) tasks, while HQA is considered in\-distribution because the RL training data is synthesized from HQA\. We report LongBench v2 results only for Qwen3\.5, as all evaluated methods on Qwen2\.5 performed at approximately the random\-guessing level on this four\-choice benchmark in our experiments, making method\-level comparisons difficult to interpret\.

#### Implementation Details\.

Thinking mode is disabled for Qwen3\.5 and is not applicable to Qwen2\.5\. We use temperature=0\.7=0\.7and top\-p=0\.95p=0\.95, and set the maximum output length to 4,096 tokens for Qwen3\.5 and 1,024 tokens for Qwen2\.5\. We use 8 NVIDIA H200 GPUs with tensor parallelism set to 2 for inference\. For RULER, each task at each context length is evaluated with 64 samples\.

### Main Results

Table 2:Length\-grouped RULER out\-of\-distribution results\. Scores are reported across context lengths from 8K to 1M tokens\.Tables[1](https://arxiv.org/html/2608.03048#Sx3.T1)and[2](https://arxiv.org/html/2608.03048#Sx4.T2)summarize the length\-grouped RULER HQA and OOD results\. On HQA, PI\-Mem achieves the best average score for both backbones\. With Qwen3\.5\-35B\-A3B, it improves the average from 76\.56 for MemAgent to 81\.25 and remains strong at the longest 3\.6M\-token setting, where Vanilla and YaRN degrade sharply\. With Qwen2\.5\-7B, PI\-Mem improves the average from 77\.50 to 84\.06 and matches or outperforms MemAgent across all reported lengths, with especially large gains in the longer\-context regime\.

On RULER OOD tasks, PI\-Mem also shows stronger robustness as context length increases\. For Qwen3\.5\-35B\-A3B, it obtains the best average score of 98\.04, outperforming MemAgent by 6\.76 points and surpassing both direct\-inference baselines\. At 1M tokens, PI\-Mem reaches 96\.88 while MemAgent drops to 83\.86\. For Qwen2\.5\-7B, PI\-Mem improves the average from 84\.68 to 91\.09 and raises the 1M\-token score from 74\.83 to 88\.39\. Together, these results show that the parallel\-iterative workflow remains robust as context length grows, with gains that persist across both backbones and extend beyond the HQA\-based training distribution\.

Table 3:LongBench v2 results\. Following the grouping defined by the benchmark, we report results by difficulty \(Easy and Hard\) and context length \(Short, Medium, and Long\)\.Table[3](https://arxiv.org/html/2608.03048#Sx4.T3)reports the LongBench v2 results\. PI\-Mem achieves the best overall score of 54\.1, outperforming MemAgent by 1\.8 points and the vanilla baseline by 3\.4 points\. These results show that the proposed workflow improves long\-context performance across realistic tasks\.

Table 4:Final\-memory evidence coverage on RULER MV\-NIAH, measured as the proportion of ground\-truth values present in the final memory\.![Refer to caption](https://arxiv.org/html/2608.03048v1/x3.png)Figure 3:End\-to\-end HQA inference latency at ultra\-long context lengths with Qwen3\.5\-35B\-A3B and Qwen2\.5\-7B\. Reported speedups are relative to Vanilla\.#### Evidence Retention Analysis\.

To directly assess whether recurrent memory updates overwrite earlier evidence, we measure final\-memory evidence coverage on RULER Multi\-values Needle\-in\-a\-Haystack \(MV\-NIAH\), where coverage is defined as the proportion of ground\-truth values present in the final memory\(Hsiehet al\.[2024](https://arxiv.org/html/2608.03048#bib.bib43)\)\. The task requires retrieving four independent values associated with the same key from positions distributed across a long context and thus provides a controlled diagnostic of evidence retention\. Table[4](https://arxiv.org/html/2608.03048#Sx4.T4)shows a clear contrast: PI\-Mem retains almost all target evidence in the final memory, whereas MemAgent loses a substantial portion of that evidence over successive recurrent updates\. This comparison indicates that parallel reading preserves the collected evidence more effectively by mitigating the overwrite caused by recurrent memory updates\.

#### Latency Analysis\.

Figure[3](https://arxiv.org/html/2608.03048#Sx4.F3)compares end\-to\-end inference latency on ultra\-long HQA inputs\. At 3\.6M tokens, PI\-Mem is 16\.9×\\timesfaster than Vanilla and 6\.1×\\timesfaster than MemAgent with Qwen3\.5\-35B\-A3B; the corresponding speedups with Qwen2\.5\-7B are 14\.4×\\timesand 2\.1×\\times\. Vanilla keeps the full context active during inference, incurring substantial full\-context prefill and KV\-cache costs\. MemAgent bounds the context of each call through chunking, but its recurrent updates form a serial critical path that prevents concurrent processing of chunks from the same sequence\. PI\-Mem removes this dependency by executing chunk\-levelReadCalloperations in parallel within each turn and batching the corresponding calls across examples, thereby improving GPU utilization and reducing latency\.

### Ablation Study

For computational efficiency, we conduct all ablations in this section with Qwen2\.5\-7B\-Instruct\.

#### Effect of RL Training\.

Figure[4](https://arxiv.org/html/2608.03048#Sx4.F4)evaluates whether RL training improves the proposed workflow on RULER OOD tasks\. RL training yields only a minor gain at 8K tokens, but the improvement becomes increasingly clear as the context length grows\. This trend suggests that even with simple synthetic data based on HQA, RL training helps the model generalize to more diverse long\-context tasks and use the workflow more effectively\.

![Refer to caption](https://arxiv.org/html/2608.03048v1/x4.png)Figure 4:Effect of RL training on RULER OOD performance across context lengths\. RL training brings larger gains as the context becomes longer\.
#### Select and Merge Components\.

Figure[5](https://arxiv.org/html/2608.03048#Sx4.F5)studies the select component and the merge component on HQA at ultra\-long context lengths \(896K, 1\.8M, and 3\.6M\)\. In the variant without select, the model is not required to output a<check\>signal, so all chunk\-level observations are directly passed to the merge step\. In the variant without merge, selected observations are simply concatenated across chunks rather than consolidated into a compact global memory\. Removing either component weakens context management: without selection, irrelevant, redundant, or noisy observations accumulate, whereas without merging, useful evidence remains fragmented and the memory expands rapidly\. As the input grows, the resulting context pollution or fragmentation leads to lower accuracy and longer inference time\.

![Refer to caption](https://arxiv.org/html/2608.03048v1/x5.png)Figure 5:Component ablation on HQA\. HQA Score reports the absolute score, whereas Latency Increase reports the relative percentage increase over the full configuration\. Removing either the select component or the merge component lowers HQA accuracy and increases latency, with the degradation becoming more severe at ultra\-long context lengths\.
#### Turn\-Efficiency Reward\.

Figure[6](https://arxiv.org/html/2608.03048#Sx4.F6)compares training with and without the turn\-efficiency reward; the reward\-enabled variant usesλturn=0\.2\\lambda\_\{\\mathrm\{turn\}\}=0\.2\. The left panel shows that adding the turn reward produces a substantially larger reduction in the number of read\-select\-merge turns\. Although the model trained without the turn reward also learns to use fewer turns as training proceeds, its curve is more unstable and the reduction is substantially smaller\. By contrast, the turn\-reward variant quickly suppresses redundant turns and converges to a lower and more stable turn count\. The right panel shows a higher late\-stage accuracy reward, indicating that the efficiency gain preserves correctness\. Before workflow\-level training, extra turns can be unnecessary or harmful because repeated updates may introduce noise, overwrite useful evidence, or over\-refine sufficient memory\.

![Refer to caption](https://arxiv.org/html/2608.03048v1/x6.png)Figure 6:Effect of the turn\-efficiency reward on Qwen2\.5\-7B RL training\. The turn reward leads to a faster, larger, and more stable reduction in the number of turns, while also improving the accuracy reward\.Further analyses in the Technical Supplement include more implementation details, additional ablation studies, actual turn counts, detailed comparisons of inference FLOPs and end\-to\-end latency, and comparative case studies\.

## Conclusion

We introduced PI\-Mem, a parallel\-iterative memory mechanism for long\-context reasoning that gathers evidence from all chunks in parallel and iteratively refines a shared memory\. Instead of updating memory recurrently after every chunk, PI\-Mem reads chunks in parallel conditioned on a shared global memory, selects useful observations, and merges them into a compact memory across turns\. We further post\-trained the workflow with RL so the model can better follow the read\-select\-merge process and avoid unnecessary turns\. Experiments with Qwen3\.5\-35B\-A3B and Qwen2\.5\-7B demonstrate that PI\-Mem improves long\-context QA and retrieval performance while reducing inference latency compared with recurrent\-memory and direct\-inference baselines\. Together, memory workflow design and end\-to\-end post\-training offer a promising path to ultra\-long\-context reasoning without relying solely on larger native context windows\.

## Acknowledgments

This work was supported by Shanghai Artificial Intelligence Laboratory and supported by the National Natural Science Foundation of China \(Grant No\. 6250076080\) and supported by the China Postdoctoral Science Foundation under Grant Number 2025M771537\.

## References

- Agent s: an open agentic framework that uses computers like a human\.arXiv preprint arXiv:2410\.08164\.Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p1.1)\.
- Y\. Bai, S\. Tu, J\. Zhang, H\. Peng, X\. Wang, X\. Lv, S\. Cao, J\. Xu, L\. Hou, Y\. Dong, J\. Tang, and J\. Li \(2024\)LongBench v2: towards deeper understanding and reasoning on realistic long\-context multitasks\.arXiv preprint arXiv:2412\.15204\.Cited by:[Benchmarks\.](https://arxiv.org/html/2608.03048#Sx4.SSx2.SSS0.Px2.p1.1)\.
- G\. Chen, M\. Q\. Shieh, and L\. Bing \(2026a\)LongRLVR: long\-context reinforcement learning requires verifiable context rewards\.arXiv preprint arXiv:2603\.02146\.Cited by:[Reinforcement Learning for Long\-Context Reasoning\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px2.p1.1)\.
- G\. Chen, Z\. Qiao, X\. Chen, D\. Yu, H\. Xu, W\. X\. Zhao, R\. Song, W\. Yin, H\. Yin, L\. Zhang, K\. Li, M\. Liao, Y\. Jiang, P\. Xie, F\. Huang, and J\. Zhou \(2026b\)IterResearch: rethinking long\-horizon agents with interaction scaling\.External Links:2511\.07327,[Link](https://arxiv.org/abs/2511.07327)Cited by:[Memory\-Based Context Management\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px1.p1.1)\.
- S\. Chen, S\. Wong, L\. Chen, and Y\. Tian \(2023\)Extending context window of large language models via positional interpolation\.arXiv preprint arXiv:2306\.15595\.Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p2.1)\.
- Z\. Chen, D\. Li, M\. Zhang, B\. Hu, and M\. Zhang \(2026c\)Dynamic long context reasoning over compressed memory via end\-to\-end reinforcement learning\.External Links:2602\.08382,[Link](https://arxiv.org/abs/2602.08382)Cited by:[Memory\-Based Context Management\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px1.p1.1)\.
- P\. Chhikara, D\. Khant, S\. Aryan, T\. Singh, and D\. Yadav \(2025\)Mem0: building production\-ready AI agents with scalable long\-term memory\.arXiv preprint arXiv:2504\.19413\.Cited by:[Memory\-Based Context Management\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px1.p1.1)\.
- Y\. Du, M\. Tian, S\. Ronanki, S\. Rongali, S\. Bodapati, A\. Galstyan, A\. Wells, R\. Schwartz, E\. A\. Huerta, and H\. Peng \(2025\)Context length alone hurts llm performance despite perfect retrieval\.External Links:2510\.05381,[Link](https://arxiv.org/abs/2510.05381)Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p1.1)\.
- Z\. Feng, L\. Su, Z\. Zhang, X\. Wang, X\. Zhang, X\. Wang, R\. Fang, Q\. Zhang, B\. Li, S\. Cai, R\. Ye, H\. Chen, J\. Yong, J\. T\. Zhou, C\. Qian, P\. Xie, B\. Hooi, Z\. Liu, and J\. Zhou \(2026\)AgentSwing: adaptive parallel context management routing for long\-horizon web agents\.External Links:2603\.27490,[Link](https://arxiv.org/abs/2603.27490)Cited by:[Memory\-Based Context Management\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px1.p1.1)\.
- Gemini Team \(2025\)Gemini 2\.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities\.arXiv preprint arXiv:2507\.06261\.Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p1.1)\.
- C\. Hsieh, S\. Sun, S\. Kriman, S\. Acharya, D\. Rekesh, F\. Jia, Y\. Zhang, and B\. Ginsburg \(2024\)RULER: what’s the real context size of your long\-context language models?\.External Links:2404\.06654,[Link](https://arxiv.org/abs/2404.06654)Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p1.1),[Benchmarks\.](https://arxiv.org/html/2608.03048#Sx4.SSx2.SSS0.Px2.p1.1),[Evidence Retention Analysis\.](https://arxiv.org/html/2608.03048#Sx4.SSx3.SSS0.Px1.p1.1)\.
- B\. Hui, J\. Yang, Z\. Cui, J\. Yang, D\. Liu, L\. Zhang, T\. Liu, J\. Zhang, B\. Yu, K\. Lu, K\. Dang, Y\. Fan, Y\. Zhang, A\. Yang, R\. Men, F\. Huang, B\. Zheng, Y\. Miao, S\. Quan, Y\. Feng, X\. Ren, X\. Ren, J\. Zhou, and J\. Lin \(2024\)Qwen2\.5\-coder technical report\.External Links:2409\.12186,[Link](https://arxiv.org/abs/2409.12186)Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p1.1)\.
- Kimi Team, T\. Bai, Y\. Bai, Y\. Bao, S\. H\. Cai, Y\. Cao, Y\. Charles, H\. S\. Che, C\. Chen, G\. Chen,et al\.\(2026\)Kimi k2\.5: visual agentic intelligence\.External Links:2602\.02276,[Link](https://arxiv.org/abs/2602.02276)Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p1.1)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. Kiela \(2020\)Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.InAdvances in Neural Information Processing Systems,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\.F\. Balcan, and H\. Lin \(Eds\.\),Vol\.33,pp\. 9459–9474\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2020/file/6b493230205f780e1bc26945df7481e5-Paper.pdf)Cited by:[Baselines\.](https://arxiv.org/html/2608.03048#Sx4.SSx2.SSS0.Px1.p1.1)\.
- H\. Li, C\. Yang, A\. Zhang, Y\. Deng, X\. Wang, and T\. Chua \(2024\)Hello again\! llm\-powered personalized agent for long\-term dialogue\.arXiv preprint arXiv:2406\.05925\.Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p1.1)\.
- J\. Liu, D\. Zhu, Z\. Bai, Y\. He, H\. Liao, H\. Que, Z\. Wang, C\. Zhang, G\. Zhang, J\. Zhang, Y\. Zhang, Z\. Chen, H\. Guo, S\. Li, Z\. Liu, Y\. Shan, Y\. Song, J\. Tian, W\. Wu, Z\. Zhou, R\. Zhu, J\. Feng, Y\. Gao, S\. He, Z\. Li, T\. Liu, F\. Meng, W\. Su, Y\. Tan, Z\. Wang, J\. Yang, W\. Ye, B\. Zheng, W\. Zhou, W\. Huang, S\. Li, and Z\. Zhang \(2025a\)A comprehensive survey on long context language modeling\.External Links:2503\.17407,[Link](https://arxiv.org/abs/2503.17407)Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p2.1)\.
- Z\. Liu, C\. Chen, W\. Li, P\. Qi, T\. Pang, C\. Du, W\. S\. Lee, and M\. Lin \(2025b\)Understanding r1\-zero\-like training: a critical perspective\.arXiv preprint arXiv:2503\.20783\.Cited by:[Trajectory\-Level Optimization\.](https://arxiv.org/html/2608.03048#Sx3.SSx2.SSS0.Px2.p1.1)\.
- E\. Lu, Z\. Jiang, J\. Liu, Y\. Du, T\. Jiang, C\. Hong, S\. Liu, W\. He, E\. Yuan, Y\. Wang, Z\. Huang, H\. Yuan, S\. Xu, X\. Xu, G\. Lai, Y\. Chen, H\. Zheng, J\. Yan, J\. Su, Y\. Wu, N\. Y\. Zhang, Z\. Yang, X\. Zhou, M\. Zhang, and J\. Qiu \(2025\)MoBA: mixture of block attention for long\-context llms\.arXiv preprint arXiv:2502\.13189\.Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p2.1)\.
- A\. Maharana, D\. Lee, S\. Tulyakov, M\. Bansal, F\. Barbieri, and Y\. Fang \(2024\)Evaluating very long\-term conversational memory of llm agents\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics,Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p1.1)\.
- C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. Gonzalez \(2023\)MemGPT: towards llms as operating systems\.arXiv preprint arXiv:2310\.08560\.Cited by:[Memory\-Based Context Management\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px1.p1.1)\.
- B\. Peng, J\. Quesnelle, H\. Fan, and E\. Shippole \(2024\)YaRN: efficient context window extension of large language models\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=wHBfxhZu1u)Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p2.1),[Baselines\.](https://arxiv.org/html/2608.03048#Sx4.SSx2.SSS0.Px1.p1.1)\.
- Qwen Team \(2026\)Qwen3\.5: towards native multimodal agents\.External Links:[Link](https://qwen.ai/blog?id=qwen3.5)Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p2.1),[Training Setup](https://arxiv.org/html/2608.03048#Sx4.SSx1.p1.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo \(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.External Links:2402\.03300,[Link](https://arxiv.org/abs/2402.03300)Cited by:[Reinforcement Learning for Long\-Context Reasoning\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px2.p1.1),[Trajectory\-Level Reinforcement Learning](https://arxiv.org/html/2608.03048#Sx3.SSx2.p1.1)\.
- W\. Shen, Z\. Yang, C\. Li, Z\. Lu, M\. Peng, H\. Sun, Y\. Shi, S\. Liao, S\. Lai, B\. Zhang, D\. Liu, F\. Huang, J\. Zhou, and M\. Yan \(2025\)QwenLong\-l1\.5: post\-training recipe for long\-context reasoning and memory management\.arXiv preprint arXiv:2512\.12967\.Cited by:[Reinforcement Learning for Long\-Context Reasoning\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px2.p1.1)\.
- L\. Sheng, Y\. Zhang, W\. Ma, Y\. Shi, T\. Huang, X\. Wang, A\. Zhang, K\. Shen, and T\. Chua \(2026\)When to memorize and when to stop: gated recurrent memory for long\-context reasoning\.External Links:2602\.10560,[Link](https://arxiv.org/abs/2602.10560)Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p3.1),[Memory\-Based Context Management\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px1.p1.1),[Baselines\.](https://arxiv.org/html/2608.03048#Sx4.SSx2.SSS0.Px1.p1.1)\.
- Y\. Shi, Y\. Chen, S\. Wang, S\. Li, H\. Cai, Q\. Gu, X\. Wang, and A\. Zhang \(2025\)Look back to reason forward: revisitable memory for long\-context llm agents\.arXiv preprint arXiv:2509\.23040\.Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p3.1),[Reinforcement Learning for Long\-Context Reasoning\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px2.p1.1),[Baselines\.](https://arxiv.org/html/2608.03048#Sx4.SSx2.SSS0.Px1.p1.1)\.
- W\. Sun, M\. Lu, Z\. Ling, K\. Liu, X\. Yao, Y\. Yang, and J\. Chen \(2025\)Scaling long\-horizon llm agent via context\-folding\.arXiv preprint arXiv:2510\.11967\.Cited by:[Memory\-Based Context Management\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px1.p1.1)\.
- F\. Wan, W\. Shen, S\. Liao, Y\. Shi, C\. Li, Z\. Yang, J\. Zhang, F\. Huang, J\. Zhou, and M\. Yan \(2025\)QwenLong\-l1: towards long\-context large reasoning models with reinforcement learning\.External Links:2505\.17667,[Link](https://arxiv.org/abs/2505.17667)Cited by:[Reinforcement Learning for Long\-Context Reasoning\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px2.p1.1)\.
- S\. Wang, G\. Zhang, L\. L\. Zhang, N\. Shang, F\. Yang, D\. Chen, and M\. Yang \(2025a\)LoongRL: reinforcement learning for advanced reasoning over long contexts\.arXiv preprint arXiv:2510\.19363\.Cited by:[Reinforcement Learning for Long\-Context Reasoning\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px2.p1.1)\.
- X\. Wang, M\. Li, P\. Lu, X\. Chang, L\. Shang, J\. Li, F\. Mi, P\. Parthasarathi, and Y\. Cui \(2026\)InfMem: learning system\-2 memory control for long\-context agent\.arXiv preprint arXiv:2602\.02704\.Cited by:[Memory\-Based Context Management\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px1.p1.1),[Reinforcement Learning for Long\-Context Reasoning\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px2.p1.1)\.
- Y\. Wang, R\. Takanobu, Z\. Liang, Y\. Mao, Y\. Hu, J\. McAuley, and X\. Wu \(2025b\)Mem\-α\\alpha: learning memory construction via reinforcement learning\.arXiv preprint arXiv:2509\.25911\.Cited by:[Memory\-Based Context Management\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px1.p1.1)\.
- D\. Wu, H\. Wang, W\. Yu, Y\. Zhang, K\. Chang, and D\. Yu \(2025\)LongMemEval: benchmarking chat assistants on long\-term interactive memory\.External Links:2410\.10813,[Link](https://arxiv.org/abs/2410.10813)Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p1.1)\.
- A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Tang, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. Qiu \(2025a\)Qwen2\.5 technical report\.External Links:2412\.15115,[Link](https://arxiv.org/abs/2412.15115)Cited by:[Training Setup](https://arxiv.org/html/2608.03048#Sx4.SSx1.p1.1)\.
- A\. Yang, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Huang, J\. Jiang, J\. Tu, J\. Zhang, J\. Zhou, J\. Lin, K\. Dang, K\. Yang, L\. Yu, M\. Li, M\. Sun, Q\. Zhu, R\. Men, T\. He, W\. Xu, W\. Yin, W\. Yu, X\. Qiu, X\. Ren, X\. Yang, Y\. Li, Z\. Xu, and Z\. Zhang \(2025b\)Qwen2\.5\-1m technical report\.External Links:2501\.15383,[Link](https://arxiv.org/abs/2501.15383)Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p1.1)\.
- J\. Yang, C\. E\. Jimenez, A\. Wettig, K\. Lieret, S\. Yao, K\. Narasimhan, and O\. Press \(2024\)SWE\-agent: agent\-computer interfaces enable automated software engineering\.InAdvances in Neural Information Processing Systems,Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p1.1)\.
- S\. Yang, J\. Kautz, and A\. Hatamizadeh \(2025c\)Gated delta networks: improving mamba2 with delta rule\.InInternational Conference on Learning Representations,External Links:2412\.06464,[Link](https://arxiv.org/abs/2412.06464)Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p2.1)\.
- Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. W\. Cohen, R\. Salakhutdinov, and C\. D\. Manning \(2018\)HotpotQA: a dataset for diverse, explainable multi\-hop question answering\.External Links:1809\.09600,[Link](https://arxiv.org/abs/1809.09600)Cited by:[Training Setup](https://arxiv.org/html/2608.03048#Sx4.SSx1.p2.1)\.
- Z\. Yang, W\. Shen, C\. Li, R\. Chen, F\. Wan, M\. Yan, X\. Quan, and F\. Huang \(2026\)SPELL: self\-play reinforcement learning for evolving long\-context language models\.External Links:2509\.23863,[Link](https://arxiv.org/abs/2509.23863)Cited by:[Reinforcement Learning for Long\-Context Reasoning\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px2.p1.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao \(2023\)ReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations,Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p1.1)\.
- R\. Ye, Z\. Zhang, K\. Li, H\. Yin, Z\. Tao, Y\. Zhao, L\. Su, L\. Zhang, Z\. Qiao, X\. Wang, P\. Xie, F\. Huang, S\. Chen, J\. Zhou, and Y\. Jiang \(2025\)AgentFold: long\-horizon web agents with proactive context management\.External Links:2510\.24699,[Link](https://arxiv.org/abs/2510.24699)Cited by:[Memory\-Based Context Management\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px1.p1.1)\.
- H\. Yu, T\. Chen, J\. Feng, J\. Chen, W\. Dai, Q\. Yu, Y\. Zhang, W\. Ma, J\. Liu, M\. Wang, and H\. Zhou \(2025a\)MemAgent: reshaping long\-context llm with multi\-conv rl\-based memory agent\.arXiv preprint arXiv:2507\.02259\.Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p3.1),[Memory\-Based Context Management\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px1.p1.1),[Training Setup](https://arxiv.org/html/2608.03048#Sx4.SSx1.p2.1),[Baselines\.](https://arxiv.org/html/2608.03048#Sx4.SSx2.SSS0.Px1.p1.1)\.
- Q\. Yu, Z\. Zhang, R\. Zhu, Y\. Yuan, X\. Zuo, Y\. Yue, W\. Dai, T\. Fan, G\. Liu, L\. Liu, X\. Liu, H\. Lin, Z\. Lin, B\. Ma, G\. Sheng, Y\. Tong, C\. Zhang, M\. Zhang, W\. Zhang, H\. Zhu, J\. Zhu, J\. Chen, J\. Chen, C\. Wang, H\. Yu, Y\. Song, X\. Wei, H\. Zhou, J\. Liu, W\. Ma, Y\. Zhang, L\. Yan, M\. Qiao, Y\. Wu, and M\. Wang \(2025b\)DAPO: an open\-source llm reinforcement learning system at scale\.External Links:2503\.14476,[Link](https://arxiv.org/abs/2503.14476)Cited by:[Reinforcement Learning for Long\-Context Reasoning\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px2.p1.1),[Trajectory\-Level Optimization\.](https://arxiv.org/html/2608.03048#Sx3.SSx2.SSS0.Px2.p1.5)\.
- J\. Yuan, H\. Gao, D\. Dai, J\. Luo, L\. Zhao, Z\. Zhang, Z\. Xie, Y\. X\. Wei, L\. Wang, Z\. Xiao, Y\. Wang, C\. Ruan, M\. Zhang, W\. Liang, and W\. Zeng \(2025\)Native sparse attention: hardware\-aligned and natively trainable sparse attention\.arXiv preprint arXiv:2502\.11089\.Cited by:[Introduction](https://arxiv.org/html/2608.03048#Sx1.p2.1)\.
- Y\. Zhang, J\. Shu, Y\. Ma, X\. Lin, S\. Wu, and J\. Sang \(2026\)Memory as action: autonomous context curation for long\-horizon agentic tasks\.External Links:2510\.12635,[Link](https://arxiv.org/abs/2510.12635)Cited by:[Memory\-Based Context Management\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px1.p1.1)\.
- Z\. Zhou, A\. Qu, Z\. Wu, S\. Kim, A\. Prakash, D\. Rus, J\. Zhao, B\. K\. H\. Low, and P\. P\. Liang \(2025\)MEM1: learning to synergize memory and reasoning for efficient long\-horizon agents\.arXiv preprint arXiv:2506\.15841\.Cited by:[Memory\-Based Context Management\.](https://arxiv.org/html/2608.03048#Sx2.SS0.SSS0.Px1.p1.1)\.

Technical Supplement

## 1Training Details

Table 1:Training hyperparameters for PI\-Mem with Qwen3\.5\-35B\-A3B and Qwen2\.5\-7B\-Instruct\.Following MemAgent’s released data\-construction procedure, we synthesize HQA training data by embedding the gold HotpotQA paragraphs into distractor articles sampled from the same dataset\. For Qwen2\.5\-7B, we use the released MemAgent training data, in which each sample contains 200 articles \(approximately 28K tokens\)\. Since Qwen3\.5\-35B\-A3B has a longer native context window, we additionally synthesize longer samples with the same construction pipeline, using 1,000 articles \(approximately 140K tokens\) per sample\. We accordingly increase the chunk size from 5K to 15K tokens and the maximum output length from 1,024 to 4,096 tokens\. Thinking mode is disabled for Qwen3\.5 to improve rollout efficiency and reduce the generated output length of each model call\.

Most RL hyperparameters and algorithmic choices follow the original MemAgent setup, including Dr\. GRPO\-style advantage normalization \(without division by the group standard deviation\) and DAPO\-style loss aggregation\. We retain MemAgent’s GRPO group size of 16 for Qwen2\.5\-7B, but reduce it to 8 for Qwen3\.5\-35B\-A3B because of the substantially higher training cost\. For the Qwen3\.5 comparison, we train PI\-Mem and MemAgent on the same longer dataset with identical hyperparameters\. For Qwen2\.5\-7B, we train only PI\-Mem on the released MemAgent data and evaluate the officially released MemAgent checkpoint\.

## 2Evaluation Details

### 2\.1Evaluation Setup

ConfigurationQwen3\.5Qwen2\.5Chunk size15,0005,000Maximum output length4,0961,024Maximum turns \(KK\)3GPU world size \(H200\)8Tensor parallel size2Samples per subtask \(RULER\)64Thinking modeDisabledN/ATemperature0\.7Top\-pp0\.95Table 2:Evaluation configurations for PI\-Mem with Qwen3\.5\-35B\-A3B and Qwen2\.5\-7B\-Instruct\.Following MemAgent, single\-answer tasks use normalized answer matching \(Sub\-EM\), whereas multi\-answer tasks are scored by the fraction of target values appearing in the prediction; LongBench v2 follows its official evaluation protocol\. The evaluation temperature and top\-ppalso follow MemAgent’s official code\. Each model reported in this work is trained in a single run, and each evaluation sample is evaluated once per method\.

#### YaRN\.

We use a scaling factor of 4\.0 for positional\-encoding extrapolation\.

#### RAG\.

For each evaluation sample, we construct an Okapi BM25 index over its decoded text chunks; the index is therefore sample\-specific rather than a shared external knowledge base\. We use the original question as the retrieval query and select the top\-6 chunks in descending order of their BM25 scores\.

### 2\.2Detailed Evaluation Results

Table 3:Length\-grouped RULER HQA results\. Scores and mean turn counts are reported\. The averages for GRU\-Mem and ReMemR1 use the eight reported lengths\.![Refer to caption](https://arxiv.org/html/2608.03048v1/x7.png)Figure 1:HQA performance across context lengths for Qwen3\.5\-35B\-A3B \(left\) and Qwen2\.5\-7B \(right\)\.Table 4:Length\-grouped RULER out\-of\-distribution results\. Scores are reported across context lengths from 8K to 1M tokens; mean turn counts are additionally averaged over the 11 out\-of\-distribution tasks at each length\.![Refer to caption](https://arxiv.org/html/2608.03048v1/x8.png)Figure 2:RULER OOD performance across context lengths for Qwen3\.5\-35B\-A3B \(left\) and Qwen2\.5\-7B \(right\)\.Table 5:LongBench v2 results using Qwen3\.5\-35B\-A3B\. Following the grouping defined by the benchmark, we report results by difficulty \(Easy and Hard\) and context length \(Short, Medium, and Long\)\.
### 2\.3Computational Cost and End\-to\-End Latency

Table[7](https://arxiv.org/html/2608.03048#S2.T7)compares Vanilla, MemAgent, and PI\-Mem on the 64\-sample HQA evaluation sets from 112K to 3\.6M tokens\. We report analytically estimated algorithmic FLOPs, counting one multiply–accumulate operation \(MAC\) as two FLOPs; PFLOPs denotes the aggregate operation count in units of101510^\{15\}FLOPs rather than throughput\. For PI\-Mem, the estimate includes all chunk reads, refinement turns, memory updates, and final\-answer generation, thereby covering the complete inference workload\. Wall\-clock latency is measured end\-to\-end for the same 64 samples under the evaluation configuration in Table[2](https://arxiv.org/html/2608.03048#S2.T2)\. The speedup ratio is computed as the Vanilla latency divided by the latency of each method\.

For a workflow\-level asymptotic comparison, letNNdenote the number of chunks,CCthe chunk size,MMthe fixed memory length, andKKthe maximum number of refinement turns; the total input length is thereforeN​CNC\. Table[6](https://arxiv.org/html/2608.03048#S2.T6)reports the dominant chunk\-reading attention cost under standard quadratic self\-attention\. The comparison omits merge and final\-answer costs to isolate the effect of workflow structure; these components are included in the complete PFLOPs estimates in Table[7](https://arxiv.org/html/2608.03048#S2.T7)\.

Table 6:Asymptotic chunk\-reading attention cost and serial depth under quadratic self\-attention\. Vanilla uses one ultra\-long call, whereas PI\-Mem processes allNNchunk reads in parallel within each turn\.The asymptotic analysis exposes a compute–latency trade\-off for PI\-Mem: relative to recurrent memory, it performs up toKKtimes more chunk\-reading work but reduces the sequential critical path from𝒪​\(N\)\\mathcal\{O\}\(N\)to𝒪​\(K\)\\mathcal\{O\}\(K\)\. BecauseKKis bounded and typically much smaller thanNNfor ultra\-long inputs, PI\-Mem converts additional computation into parallelizable work\. The empirical results indicate that this trade\-off is most favorable in the ultra\-long\-context regime, where increased parallelism outweighs the additional computation\.

Table 7:Aggregate algorithmic FLOPs and end\-to\-end wall\-clock latency for 64 HQA samples\. PFLOPs denotes101510^\{15\}floating\-point operations\. FLOPs are analytically estimated, while latency is measured\. Speedup is relative to Vanilla\.

## 3Additional Ablations

### 3\.1Maximum Turn Budget

Table 8:Effect of the maximum turn budgetKKon Qwen2\.5\-7B HQA performance and the mean number of turns used\. Bold values denote the best HQA score at each context length; ties are all highlighted\.Table[8](https://arxiv.org/html/2608.03048#S3.T8)shows that limiting PI\-Mem to one turn consistently reduces HQA accuracy\. WithK=1K=1, all chunk reads are conditioned only on the initial empty memory, so evidence discovered in one chunk cannot guide subsequent reads of other chunks; iterative refinement is therefore important for cross\-chunk information exchange\. Increasing the budget fromK=3K=3toK=5K=5, however, does not yield consistent further gains because most samples converge early: underK=5K=5, 75\.94% of samples exit after two turns, whereas only 4\.84% use four or five turns\. PI\-Mem thus benefits from iteration while invoking additional turns only for the small subset of samples that require further refinement\.

### 3\.2Chunk and Memory Sizes

We evaluate training\-free PI\-Mem with Qwen3\.5\-35B\-A3B on HQA while varying one size parameter at a time\. The default configuration uses 15K\-token chunks and a 4K\-token memory\. For the chunk\-size ablation, the memory size remains 4K; for the memory\-size ablation, the chunk size remains 15K\.

Table 9:Effect of chunk and memory sizes on training\-free PI\-Mem with Qwen3\.5\-35B\-A3B on HQA\. One parameter is varied at a time, and the shaded row denotes the default 15K/4K configuration\.Table[9](https://arxiv.org/html/2608.03048#S3.T9)shows no consistent degradation across context lengths when the chunk size varies from 5K to 25K or the memory size varies from 2K to 8K\. PI\-Mem is therefore robust to these size choices, and its performance is not materially affected within the tested ranges\. We do not conduct an additional size ablation for Qwen2\.5\-7B because its chunk and memory sizes exactly follow the experimental setup of MemAgent\.

## 4Prompt Template

Prompt Template forReadCallYou are presented with a problem, a section of an article, and a global memory summarizing previously gathered information\. Please read the section carefully and determine whether the section contains new information relevant to answering the problem beyond what is already in the global memory\. First, output your judgment in the format<check\>yes</check\>if there is new information, or<check\>no</check\>if there is none\. Then, if there is new information, extract and list the key details\.<problem\>\{prompt\}</problem\><memory\>\{memory\}</memory\><section\>\{chunk\}</section\>Your response:

Prompt Template forMergeCallYou are presented with a problem and key information extracted from multiple sections of an article\. Please consolidate all the information into a single comprehensive memory\. Remove redundancies and organize the information clearly, retaining all details relevant to answering the problem\.<problem\>\{prompt\}</problem\><extracted\_information\>\{memories\}</extracted\_information\>Consolidated memory:

Prompt Template forFinalCallYou are presented with a problem and a previous memory\. Please answer the problem based on the previous memory and put the answer in \\boxed\{\{\}\}\.<problem\>\{prompt\}</problem\><memory\>\{memory\}</memory\>Your answer:

## 5Case Studies

We present paired Qwen3\.5\-35B\-A3B trajectories on the same questions\. The MV\-NIAH example has 130,778 tokens, and the HQA example has 1,600 documents and 226,114 tokens\. We retain the terminology used in the main paper: one PI\-Mem*turn*is a complete read–select–merge cycle\. Inside the colored boxes, all prose is transcribed from the corresponding JSONL fields; we add only structural labels and color highlighting\. Ellipses mark omitted portions of long outputs\.

### 5\.1Case 1: MemAgent on MV\-NIAH

MemAgent on MV\-NIAHQuestion:Some special magic numbers are hidden within the following text\. Make sure to memorize it\. What are all the special magic numbers forcloudy\-singingmentioned in the provided text?Chunk 0:“One of the special magic numbers for cloudy\-singing is:2952447\. July 2010What hard liquor, cigarettes, heroin, and crack have in common is that they’re all more concentrated forms of less addictive predecessors\. Most if not all the things we describe as addictive are\. And the scary thing is, the process that created them is accelerating\.”Memory 0:“Based on the provided text, here is the updated memory containing the special magic number for ‘cloudy\-singing’ and other relevant details from the article:Updated Memory:•Special Magic Number for Cloudy\-Singing:2952447•Context of the Number:The number appears in the text immediately following the phrase ‘One of the special magic numbers for cloudy\-singing is:’ and is followed by the date ‘July 2010’\.•Article Themes & Key Concepts:–Addictiveness & Technological Progress:The text discusses how technological progress accelerates the concentration of things we like \(e\.g\., opium to heroin, food processing, social media\), making the world more addictive\. It notes that social customs \(antibodies\) evolve slowly compared to technological progress, creating a need for individuals to figure out what to avoid on their own\.–…”…Chunk 2:“One of the special magic numbers for cloudy\-singing is:8372717\.”Memory 2:“Based on the provided text, here is the updated memory containing the new special magic number for ‘cloudy\-singing’ and all other relevant details from the article:Updated Memory:•Special Magic Number for Cloudy\-Singing:8372717–*Note:*The previous memory listed 2952447\.The text explicitly states: ‘One of the special magic numbers for cloudy\-singing is: 8372717\.’ This new number appears in the section discussing how technology makes mass\-produced goods \(like cars and watches\) convenient and affordable for the rich, contrasting them with expensive, custom\-made items that are often inferior in performance\.•Context of the Number:The number appears in the essay ‘Make Money as a Specialized Skill’ \(April 2004\), specifically within a paragraph comparing expensive custom cars to mass\-produced cars, and expensive mechanical watches to accurate quartz watches\.…”

MemAgent on MV\-NIAH — ContinuedChunk 6:“One of the special magic numbers for cloudy\-singing is:8023581\.”Memory 6:“Based on the provided text, here is the updated memory containing the new special magic number for ‘cloudy\-singing’ and all other relevant details from the article:Updated Memory:•Special Magic Number for Cloudy\-Singing:8023581–*Note:*The previous memory listed 8372717 \(and originally 2952447\)\.The text explicitly states: ‘One of the special magic numbers for cloudy\-singing is: 8023581\.’ This new number appears in the section discussing ‘Rounds’ and the structure of angel rounds versus series A rounds\.•Context of the Number:The number appears in the essay ‘How to Start a Startup’ \(April 2012\), specifically within a paragraph comparing the equity dilution of traditional Series A rounds \(30–40%\) with the lower dilution of angel rounds \(often around 13%\)\.…Correction to Previous Memory:•The previous memory listed8372717as the number for ‘cloudy\-singing’ found in the ‘Make Money as a Specialized Skill’ essay\.•The text provided in this section explicitly introduces anewnumber:8023581\.•*Action:*The memory has been updated to reflect themost recent explicit statementregarding the ‘special magic number for cloudy\-singing’ found in the current text, while retaining the context of the previous number for completeness\.”Chunk 7:“One of the special magic numbers for cloudy\-singing is:2188606\.”Memory 7:“Based on the provided text, here is the updated memory containing the new special magic number for ‘cloudy\-singing’ and all other relevant details from the article:Updated Memory:•Special Magic Number for Cloudy\-Singing:2188606–*Note:*The previous memory listed 8023581 \(and originally 8372717 and 2952447\)\.The text explicitly states: ‘One of the special magic numbers for cloudy\-singing is: 2188606\.’•Context of the Number:The number appears in the essay ‘Wisdom and Intelligence’ \(February 2007\), within a discussion distinguishing between ‘wise’ and ‘smart’\.…Correction to Previous Memory:•The previous memory listed8023581as the number for ‘cloudy\-singing’ found in the ‘How to Start a Startup’ essay\.•The text provided in this section explicitly introduces anewnumber:2188606\.•*Action:*The memory has been updated to reflect themost recent explicit statementregarding the ‘special magic number for cloudy\-singing’ found in the current text, while retaining the context of the previous numbers for completeness\. The number2188606 is now the primary reference for ‘cloudy\-singing’based on the latest provided text\.”…Final Output:“Based on the provided memory, the text explicitly states that the special magic number for ‘cloudy\-singing’ found in the essay ‘Wisdom and Intelligence’ \(February 2007\) is2188606\.Although the memory notes that previous numbers \(8023581, 8372717, 2952447\) were associated with ‘cloudy\-singing’ in other contexts or previous updates, themost recent and explicit statementin the current text identifies2188606as the number\.\\boxed\{2188606\}”Prediction:2188606Sub\-EM:0\.25Gold Answer:2188606, 8023581, 8372717, 2952447

In Memories 2, 6, and 7, each new value replaces the previous value in the main “Special Magic Number” entry, while earlier values are moved to notes or corrections\. As a result, although all values remain in memory, the final response returns only the latest one\.

### 5\.2Case 2: PI\-Mem on the Same MV\-NIAH Sample

PI\-Mem on MV\-NIAH — Turn 1Question:Some special magic numbers are hidden within the following text\. Make sure to memorize it\. What are all the special magic numbers forcloudy\-singingmentioned in the provided text?Chunk 0:<check\>yes</check\>“The section contains new information relevant to the problem\. Specifically, it explicitly states a ‘special magic number for cloudy\-singing’ which is2952447\. This information was not present in the empty global memory\.Key Details Extracted:•Special Magic Number for Cloudy\-Singing:2952447•Context:The number is mentioned at the very beginning of the provided text section, followed by the date ‘July 2010’\.•Source:The text appears to be an excerpt from an article \(likely by Paul Graham, based on the style and content regarding startups, Lisp, and Viaweb\) where this specific number is introduced as a ‘special magic number’\.”Chunk 1:<check\>no</check\>Chunk 2:<check\>yes</check\>“The section contains new information relevant to the problem\. Specifically, it explicitly states a ‘special magic number for cloudy\-singing’ within the text\.Key Details Extracted:•Context:The number is mentioned in a paragraph discussing the difference between mass\-produced cars and custom\-made cars, and how technology has made the rich live more like the average person\.•The Number:8372717•The Phrase:‘One of the special magic numbers for cloudy\-singing is: 8372717\.’”Chunk 3:<check\>no</check\>Chunk 4:<check\>no</check\>Chunk 5:<check\>no</check\>Chunk 6:<check\>yes</check\>“The section contains new information relevant to the problem\. Specifically, it explicitly states a ‘special magic number’ associated with the phrase ‘cloudy\-singing’ within the text provided\.Key Details Extracted:•Context:The text discusses startup funding, specifically the difference between angel rounds and Series A rounds, and the role of super\-angels vs\. VCs\.•The Magic Number:The text explicitly states: ‘One of the special magic numbers for cloudy\-singing is:8023581\.’•Location:This sentence appears in the section titled ‘April 2010’ \(under the subsection ‘Rounds’\), embedded within a paragraph discussing equity dilution in Series A rounds\.”Chunk 7:<check\>yes</check\>“The section contains new information relevant to the problem\. Specifically, it explicitly states a ‘special magic number for cloudy\-singing’ within the text discussing the relationship between wisdom and intelligence\.Key Details:•Context:The number appears in the essay section titled ‘A few days ago I finally figured out something I’ve wondered about for 25 years: the relationship between wisdom and intelligence\.’•The Number:2188606•The Phrase:‘One of the special magic numbers for cloudy\-singing is: 2188606\.’”

PI\-Mem on MV\-NIAH — Turn 1 ContinuedChunk 8:<check\>no</check\>Merged Memory:“Based on the provided extracted information, here is the consolidated memory regarding the special magic numbers for ‘cloudy\-singing’:Special Magic Numbers for Cloudy\-SingingThe text explicitly identifiesfour distinct numbersassociated with the phrase ‘cloudy\-singing’:1\.2952447…2\.8372717…3\.8023581…4\.2188606…”

PI\-Mem on MV\-NIAH — Turn 2Chunk 0:<check\>no</check\>Chunk 1:<check\>no</check\>Chunk 2:<check\>no</check\>Chunk 3:<check\>no</check\>Chunk 4:<check\>no</check\>Chunk 5:<check\>no</check\>Chunk 6:<check\>no</check\>Chunk 7:<check\>no</check\>Chunk 8:<check\>no</check\>Merged Memory:“Based on the provided extracted information, here is the consolidated memory regarding the special magic numbers for ‘cloudy\-singing’:Special Magic Numbers for Cloudy\-SingingThe text explicitly identifies four distinct numbers associated with the phrase ‘cloudy\-singing’:1\.2952447…2\.8372717…3\.8023581…4\.2188606…”Final Output:“Based on the provided memory, the special magic numbers for ‘cloudy\-singing’ are explicitly listed as:1\.29524472\.83727173\.80235814\.2188606The question asks for all the special magic numbers mentioned\.\\boxed\{2952447, 8372717, 8023581, 2188606\}”Prediction:2952447, 8372717, 8023581, 2188606Sub\-EM:1\.00Gold Answer:2188606, 8023581, 8372717, 2952447

Turn 1 extracts four positive chunk memories and merges them as four distinct members of the same set\. In Turn 2, every chunk returns<check\>no</check\>; the consolidated set is preserved for the final call\.

### 5\.3Case 3: MemAgent on HQA

MemAgent on HQAQuestion:Which song was behind the third single on the album “A Lot About Livin’ \(And a Little ’bout Love\)” for two weeks on the country charts?Memory 0:“Based on the provided text, there is no information regarding the album “A Lot About Livin’ \(And a Little ’bout Love\)” or the specific song that was the third single on that album and spent two weeks on the country charts\. The provided documents contain information about various artists, albums, and events \(such as Buddy Holly’s “That’ll Be the Day”, Selena’s career, The Chieftains’ “The Chieftains in China”, and others\), but none mention the specific album or single in question\.Therefore, the memory cannot be updated with the answer to the problem as the necessary information is not present in the source text\.Updated Memory:No previous memory\. The provided text does not contain information about the album “A Lot About Livin’ \(And a Little ’bout Love\)” or the third single from that album that spent two weeks on the country charts\. The text includes information on other musical topics such as Buddy Holly’s final album “That’ll Be the Day”, Selena’s career, The Chieftains’ album “The Chieftains in China”, and various other unrelated subjects, but none address the specific query\.”…Memory 3:“Based on the provided text, the memory has been updated with the specific information regarding the album “A Lot About Livin’ \(And a Little ’bout Love\)” and its singles\.Updated Memory:The provided text now contains specific information regarding the album “A Lot About Livin’ \(And a Little ’bout Love\)” and its singles, which was previously absent\.•Album:“A Lot About Livin’ \(And a Little ’bout Love\)” is an album byAlan Jackson\.•First Single:The first single from this album was“She’s Got the Rhythm \(And I Got the Blues\)”, released in October 1992\. This song co\-written by Alan Jackson and Randy Travis, won an award in 1993 from Music City News for being one of the most performed country songs of the year\.•Third Single:The text explicitly states that “She’s Got the Rhythm \(And I Got the Blues\)” was thefirst single\. However, the textdoes not explicitly name the third singlenor does it explicitly state which song spent “two weeks” on the country charts\.…Revised Summary of Findings:•Album:“A Lot About Livin’ \(And a Little ’bout Love\)” by Alan Jackson\.•First Single:“She’s Got the Rhythm \(And I Got the Blues\)” \(Released Oct 1992\)\.•Third Single Information:The provided textdoes not identify the third single or its chart performance\(two weeks on the country charts\)\.Conclusion:The provided text does not contain the answer to the problem\. It identifies the album and the first single, but fails to mention the third single or its specific chart duration\.”

MemAgent on HQA — ContinuedMemory 6:“Based on the provided text, the memory has been updated with specific information regarding the album “A Lot About Livin’ \(And a Little ’bout Love\)” and its singles\.Updated Memory:•Album:“A Lot About Livin’ \(And a Little ’bout Love\)” is an album byAlan Jackson\.•First Single:The first single from this album was“She’s Got the Rhythm \(And I Got the Blues\)”, released in October 1992\.•Third Single Information:The provided textdoes not explicitly name the third singlenor does it explicitly state which song spent “two weeks” on the country charts as the third single\.…Conclusion:The provided text does not contain the answer to the problem\. It identifies the album and the first single, but fails to mention the third single or its specific chart duration\.*Note: Document 730 mentions a song“We’ll Burn That Bridge”by Brooks & Dunn that peaked at \#2 fortwo weeks behind “Chattahoochee”by Alan Jackson, butthis does not directly answer the questionregarding the third single on the specific album “A Lot About Livin’ \(And a Little ’bout Love\)”\.*”…Memory 9:“Updated Memory:…•Fifth/Final Single:“\(Who Says\) You Can’t Have It All”\.•Third Single Information:The provided textdoes not identifythe third single or its chart performance\.…Conclusion:The provided text does not contain the answer to the problem\. It identifies the album, the first single, and the fifth single, but fails to mention the third single or its specific chart duration\.*Note: Document 730 mentions a song“We’ll Burn That Bridge”by Brooks & Dunn that peaked at \#2 fortwo weeks behind “Chattahoochee”by Alan Jackson, butthis does not directly answer the questionregarding the third single on the specific album “A Lot About Livin’ \(And a Little ’bout Love\)”\.*”Memory 10:“Based on the provided text, the memory has been updated with specific information regarding the album “A Lot About Livin’ \(And a Little ’bout Love\)” and its singles\.Updated Memory:•Album:“A Lot About Livin’ \(And a Little ’bout Love\)” is the third studio album by American country music artistAlan Jackson\. It was released onOctober 6, 1992\.•Singles List:1\.“She’s Got the Rhythm \(And I Got the Blues\)”2\.“Chattahoochee”3\.“Tonight I Climbed the Wall”4\.“\(Who Says\) You Can’t Have It All”5\.“Mercury Blues”•Chart Performance:“Chattahoochee” and “She’s Got the Rhythm \(And I Got the Blues\)” were bothNumber Onehits\. The other three songs reached theTop Five\.…Revised Summary of Findings:•First Single:“She’s Got the Rhythm \(And I Got the Blues\)” \(Number One\)\.•Second Single:“Chattahoochee” \(Number One\)\.•Third Single:“Tonight I Climbed the Wall”\(Top Five\)\.•Fourth Single:“\(Who Says\) You Can’t Have It All” \(Top Five\)\.•Fifth Single:“Mercury Blues” \(Top Five\)\.•Answer to Problem:The text identifies the third single as“Tonight I Climbed the Wall”\. While the text confirms it reached the Top Five, it does not explicitly confirm the “two weeks” duration mentioned in the problem statement\.*Note: The previous memory noted the absence of the third single’s name\. This update resolves that gap by identifying “Tonight I Climbed the Wall” as the third single based on the sequence provided in Document 1131\.*”

MemAgent on HQA — ContinuedMemory 11:“Based on the provided text, the memory has been updated with specific information regarding the album “A Lot About Livin’ \(And a Little ’bout Love\)” and its singles, correcting a previous error regarding the release order and chart performance of the third single\.Updated Memory:…•Second Single:“Chattahoochee” \(Number One\)\.•Third Single:“Tonight I Climbed the Wall”\(Peaked at Number 4\)\.•Answer to Problem:The text identifies the third single as“Tonight I Climbed the Wall”\. While the text confirms it reached the Top Five \(specifically Number 4\), itdoes not explicitly confirmthe “two weeks” duration mentioned in the problem statement\.*Note: The previous memory incorrectly identified the order of the first two singles and lacked the specific peak position for the third single\. This update corrects the release order and adds the specific peak chart position of “Tonight I Climbed the Wall” based on Document 1273\.*”Memory 12:“Based on the provided text, the memory has been updated with specific information regarding the album “A Lot About Livin’ \(And a Little ’bout Love\)” and its singles, correcting a previous error regarding the release order and chart performance of the third single\.Updated Memory:…•First Single:“She’s Got the Rhythm \(And I Got the Blues\)” \(Number One\)\.•Second Single:“Tonight I Climbed the Wall” \(Number 4\)\.•Third Single:“Chattahoochee”\(Number One\)\.•Answer to Problem:The text identifies the third single as“Chattahoochee”\. While the text confirms it reached Number One, itdoes not explicitly confirmthe “two weeks” duration mentioned in the problem statement\.*Note: The previous memory incorrectly identified “Tonight I Climbed the Wall” as the third single\. Document 1348 corrects this, identifying “Chattahoochee” as the third single released in May 1993\.*”…Memory 15:“Based on the provided text, the memory has been updated with specific information regarding the album “A Lot About Livin’ \(And a Little ’bout Love\)” and its singles, correcting a previous error regarding the release order and chart performance of the third single\.Updated Memory:•Album:“A Lot About Livin’ \(And a Little ’bout Love\)” is the third studio album by American country music artistAlan Jackson\.•Singles List & Order:1\.“She’s Got the Rhythm \(And I Got the Blues\)”2\.“Tonight I Climbed the Wall”3\.“Chattahoochee”4\.“\(Who Says\) You Can’t Have It All”5\.“Mercury Blues”…None of the documents in the provided section \(1593–1600\) contain information regarding the specific chart duration \(“two weeks”\) of the third single from Alan Jackson’s album “A Lot About Livin’ \(And a Little ’bout Love\)”\.The memory update relies on the previously established context \(Document 1348\) which identified “Chattahoochee” as the third single, while noting the absence of the specific “two weeks” detail in the current text\.”Final Output:“Based on the provided memory, the album “A Lot About Livin’ \(And a Little ’bout Love\)” by Alan Jackson produced five singles\. The memory explicitly corrects a previous error to identify thethird singleas“Chattahoochee”, which was released in May 1993\.Although the memory notes that the specific text does not explicitly state the “two weeks” duration, it confirms that “Chattahoochee” was the third single and a Number One hit\. Therefore, the song in question is“Chattahoochee”\.\\boxed\{Chattahoochee\}”Prediction:ChattahoocheeSub\-EM:0\.00Gold Answer:We’ll Burn That Bridge

Memory 6 records the answer\-bearing relation from Document 730, and Memory 9 still retains it\. Memory 10 rewrites the state around a later album track list: the complete Document 730 relation is absent, while an incorrect third\-single interpretation becomes the answer candidate\. Memory 12 eventually establishes the correct bridge entity, “Chattahoochee,” but the earlier relation needed to map that entity to “We’ll Burn That Bridge” is no longer available\.

### 5\.4Case 4: PI\-Mem on the Same HQA Sample

PI\-Mem on HQA — Turn 1Question:Which song was behind the third single on the album “A Lot About Livin’ \(And a Little ’bout Love\)” for two weeks on the country charts?Chunk 0:<check\>no</check\>Chunk 1:<check\>no</check\>Chunk 2:<check\>no</check\>Chunk 3:<check\>yes</check\>“The section contains new information relevant to the problem\. Specifically,Document 420identifies the song “She’s Got the Rhythm \(And I Got the Blues\)” as the first single from the album “A Lot About Livin’ \(And a Little ’bout Love\)” by Alan Jackson\. While the problem asks for the*third*single, this document provides the context of the album and its singles, which is necessary to answer the question …Key Details Extracted:•Album:“A Lot About Livin’ \(And a Little ’bout Love\)”•Artist:Alan Jackson•First Single:“She’s Got the Rhythm \(And I Got the Blues\)” \(Released October 1992\)•Award:The first single received an award in 1993 from Music City News for being one of the most performed country songs of the year\.•Note:The provided text does not explicitly state which song was the*third*single or its chart performance duration, but it establishes the album and the first single\.”Chunk 4:<check\>no</check\>Chunk 5:<check\>yes</check\>“The section contains new information relevant to the problem\. Specifically,Document 591mentions the album “A Lot About Livin’ \(And a Little ’bout Love\)” by Randy Travis\. While the problem asks about the third single on this album and its chart performance, the provided text in Document 591 actually discusses a*different*album by Randy Travis \(“High Lonesome”\) and mentions that Randy Travis co\-wrote a song for Alan Jackson’s album “A Lot About Livin’ \(And a Little ’bout Love\)” in 1992\. It does not provide the specific details about the singles on the album itself, nor does it identify the third single or its chart performance\. …”Chunk 6:<check\>yes</check\>“The section contains new information relevant to the problem\. Specifically,Document 730identifies a song titled“We’ll Burn That Bridge”by Brooks & Dunn\. This document explicitly states that “We’ll Burn That Bridge” peaked at \#2 on the country charts fortwo weeks, behind “Chattahoochee”by Alan Jackson\. This matches the specific criteria of the question \(a song behind another for two weeks on the country charts\), even though the document does not explicitly name the album “A Lot About Livin’ \(And a Little ’bout Love\)” in the text provided\. However, given the specificity of the “two weeks” and “behind” details, this is the relevant piece of information found in the text\.Key Details Extracted:•Song Title:We’ll Burn That Bridge•Artist:Brooks & Dunn•Album:Hard Workin’ Man \(Note: The document states this song was the second single from*Hard Workin’ Man*, not*A Lot About Livin’ \(And a Little ’bout Love\)*, but it is the only song in the text matching the “peaked at \#2 for two weeks” description\)\.•Chart Performance:Peaked at \#2 on the country charts for two weeks\.•Song Behind:“Chattahoochee” by Alan Jackson\.•Release Date:May 1993\.•Writers:Don Cook and Ronnie Dunn\.”

PI\-Mem on HQA — Turn 1 ContinuedChunk 7:<check\>no</check\>Chunk 8:<check\>no</check\>Chunk 9:<check\>yes</check\>“The section contains new information relevant to the problem\. Specifically,Document 1053identifies the song “\(Who Says\) You Can’t Have It All” as thefifth and final singlefrom Alan Jackson’s album “A Lot About Livin’ \(And a Little ’bout Love\)”\. …Key Details Extracted:•Album:“A Lot About Livin’ \(And a Little ’bout Love\)” by Alan Jackson\.•Song:“\(Who Says\) You Can’t Have It All”\.•Release Date:January 1994\.•Position on Album:Fifth and final single\.•Chart Performance:Peaked at number 4 on the U\.S\. “Billboard” Hot Country Singles chart and number 11 on the Canadian “RPM” Country Tracks chart\.•Writers:Alan Jackson and Jim McBride\.•Album Certification:Certified6×6\\timesPlatinum by the RIAA with over 6,000,000 sales in the US\.”Chunk 10:<check\>yes</check\>“The section contains new information relevant to the problem\. Specifically,Document 1131provides the tracklist and chart performance details for the album “A Lot About Livin’ \(And a Little ’bout Love\)” by Alan Jackson\.Key Details Extracted:•Album:A Lot About Livin’ \(And a Little ’bout Love\) by Alan Jackson\.•Release Date:October 6, 1992\.•Singles Released:“Chattahoochee”, “She’s Got the Rhythm \(And I Got the Blues\)”, “Tonight I Climbed the Wall”, “\(Who Says\) You Can’t Have It All”, and “Mercury Blues”\.•Chart Performance:“Chattahoochee” and “She’s Got the Rhythm \(And I Got the Blues\)” were Number One hits\. The other three songs reached Top Five\.•Context for the Problem:The text lists the singles and their peak positions, but it does not explicitly state which specific song was thethirdsingle or how many weeks it spent on the charts\. The third single listed in the text is “Tonight I Climbed the Wall”\.”Chunk 11:<check\>yes</check\>“The section contains new information relevant to the problem\. Specifically,Document 1273identifies the song “Tonight I Climbed the Wall” by Alan Jackson as the second single from the album “A Lot About Livin’ \(And a Little ’bout Love\)”\. …Key Details Extracted:•Album:“A Lot About Livin’ \(And a Little ’bout Love\)”•Artist:Alan Jackson•Second Single:“Tonight I Climbed the Wall”•Release Date of Second Single:January 1993•Chart Performance of Second Single:Peaked at number 4 on the US “Billboard” Hot Country Singles & Tracks chart and the Canadian “RPM” Country Tracks chart\.”Chunk 12:<check\>yes</check\>“The section contains new information relevant to the problem\. Specifically,Document 1348identifies the song“Chattahoochee” as the third singlefrom the album “A Lot About Livin’ \(And a Little ’bout Love\)” by Alan Jackson\. While the problem asks which song was behind the third single for two weeks on the country charts, this document provides the crucial identity of the third single itself, which is the necessary starting point to answer the question …”

PI\-Mem on HQA — Turn 1 ContinuedChunk 13:<check\>no</check\>Chunk 14:<check\>no</check\>Chunk 15:<check\>no</check\>Merged Memory:“Based on the provided extracted information, here is the consolidated memory regarding the album “A Lot About Livin’ \(And a Little ’bout Love\)” and its singles:Album Overview•Title:A Lot About Livin’ \(And a Little ’bout Love\)•Artist:Alan Jackson•Release Date:October 6, 1992•Certification:Certified6×6\\timesPlatinum by the RIAA \(over 6,000,000 sales in the US\)\.Singles and Chart PerformanceThe album released multiple singles with the following details extracted from the documents: …*Note: There is a discrepancy in the ordering of singles between documents\. Document 420/1273/1348 suggest the order: 1\. She’s Got the Rhythm, 2\. Tonight I Climbed the Wall, 3\. Chattahoochee\. Document 1131 lists them in a different order but confirms the titles\.*Relevance to the ProblemThe problem asks for the song that wasbehind the third singleon the album fortwo weekson the country charts\.•Thethird singleis identified as“Chattahoochee”\(Document 1348\)\.•Document 730states that“We’ll Burn That Bridge”by Brooks & Dunn peaked at \#2 for two weeks,behind “Chattahoochee”\.•While “We’ll Burn That Bridge” is not on Jackson’s album, the question asks “Which song was behind the third single…”, implying the song that was charting*below*the third single \(Chattahoochee\) at that specific time\.•Therefore, the song in question is“We’ll Burn That Bridge”\.”

PI\-Mem on HQA — Turn 2Chunk 0:<check\>no</check\>Chunk 1:<check\>no</check\>Chunk 2:<check\>no</check\>Chunk 3:<check\>no</check\>Chunk 4:<check\>no</check\>Chunk 5:<check\>no</check\>Chunk 6:<check\>no</check\>Chunk 7:<check\>no</check\>Chunk 8:<check\>no</check\>Chunk 9:<check\>no</check\>Chunk 10:<check\>no</check\>Chunk 11:<check\>no</check\>Chunk 12:<check\>no</check\>Chunk 13:<check\>no</check\>Chunk 14:<check\>no</check\>Chunk 15:<check\>no</check\>Merged Memory:“Relevance to the ProblemThe problem asks for the song that wasbehind the third singleon the album fortwo weekson the country charts\.•Thethird singleis identified as“Chattahoochee”\(Document 1348\)\.•Document 730states that“We’ll Burn That Bridge”by Brooks & Dunn peaked at \#2 for two weeks,behind “Chattahoochee”\.•Therefore, the song in question is“We’ll Burn That Bridge”\.”Final Output:“Based on the provided memory, the third single on Alan Jackson’s album*A Lot About Livin’ \(And a Little ’bout Love\)*is“Chattahoochee”\(Document 1348\)\.The memory explicitly states in Document 730 that the song“We’ll Burn That Bridge”\(by Brooks & Dunn\) peaked at number 2 on the country charts for two weeks,behind “Chattahoochee”\.Therefore, the song that was behind the third single for two weeks is “We’ll Burn That Bridge”\.\\boxed\{We’ll Burn That Bridge\}”Prediction:We’ll Burn That BridgeSub\-EM:1\.00Gold Answer:We’ll Burn That Bridge

The positive reads in Turn 1 independently preserve Document 730’s chart relation and Document 1348’s third\-single identity\. Their merged memory states the complete two\-hop chain and cites both documents\. Turn 2 contributes no new local memories, so the grounded chain remains unchanged for the final call\.

Together, the MV\-NIAH and HQA pairs show two manifestations of the same recurrent failure mode: later sequential updates can either demote earlier values to obsolete history or delete an unresolved but essential relation\. PI\-Mem instead retains complementary chunk\-level outputs until they can be consolidated within a turn\.