CMT-RAG: Complementary Memory Traces for Multi-turn Multi-hop RAG

# Complementary Memory Traces for Multi-turn Multi-hop RAG
Source: [https://arxiv.org/html/2607.26470](https://arxiv.org/html/2607.26470)
###### Abstract

Multi\-turn information\-seeking conversations require both multi\-hop reasoning and long\-range dependency tracking across turns\. However, existing RAG systems typically represent conversational memory as raw dialogue history, rewritten queries, or unstructured summaries, making it difficult to recover the specific prior reasoning steps and evidence required for follow\-up queries\. Our key insight is to align conversational memory with retrieval by representing dialogue context as sub\-question\-level reasoning traces\. Building on this insight, we introduceMuMu\-QA, a benchmark for multi\-turn multi\-hop RAG with explicit cross\-turn sub\-question dependency annotations, andCMT\-RAG, a complementary memory framework for this setting\. At each turn, CMT\-RAG employs a state\-space trace generator, whose recurrent state serves as runtime memory, to incorporate recent conversational context and decompose the current query into structured trace drafts containing retrieval\-oriented sub\-questions and dependencies on earlier traces\. It then grounds these drafts with retrieved evidence and stores them as persistent memory traces in a session\-level DAG, enabling future turns to efficiently recover relevant prior reasoning and evidence\. Experiments on MuMu\-QA and corpus\-level RAG benchmarks show that CMT\-RAG consistently outperforms five categories of RAG baselines in answer accuracy\.

## 1Introduction

![Refer to caption](https://arxiv.org/html/2607.26470v1/x1.png)Figure 1:A multi\-turn multi\-hop conversation with cross\-turn dependencies\. The graph illustrates two dependency types:Ext\.\(Predicate Extension\) queries a new attribute or relation of a previously resolved target, andRef\.\(Entity Reference\) directly reuses a previously introduced entity\.Retrieval\-augmented generation \(RAG\) is increasingly deployed in extended information\-seeking conversations, where users refine questions, omit repeated entities, and build new requests on earlier answers\(Yeet al\.[2026](https://arxiv.org/html/2607.26470#bib.bib36); Labanet al\.[2026](https://arxiv.org/html/2607.26470#bib.bib49); Huet al\.[2026](https://arxiv.org/html/2607.26470#bib.bib35)\)\. In such settings, a new turn often remains context\-dependent while requiring multi\-hop evidence seeking, since answering it may require decomposing the query into several sub\-questions whose dependencies span earlier turns\.[Figure˜1](https://arxiv.org/html/2607.26470#S1.F1)shows a representative case\. The system must retrieve evidence for the current turn and identify the specific prior reasoning step whose subject or entity is being extended or reused\. We refer to this setting as*multi\-turn multi\-hop conversational RAG with complementary sub\-question dependencies*\.

This setting exposes a mismatch between how conversational RAG stores memory and how retrieval actually operates\. Query rewriting converts a context\-dependent turn into a standalone query\(Ananthaet al\.[2021](https://arxiv.org/html/2607.26470#bib.bib10); Moet al\.[2023](https://arxiv.org/html/2607.26470#bib.bib13); Zhuet al\.[2025a](https://arxiv.org/html/2607.26470#bib.bib33)\), which is effective for local coreference but compresses dependency chains into a single query, obscuring intermediate retrieval targets\. Query decomposition exposes sub\-question structure for multi\-hop retrieval\(Trivediet al\.[2023](https://arxiv.org/html/2607.26470#bib.bib2); Khotet al\.[2023](https://arxiv.org/html/2607.26470#bib.bib3); Chenet al\.[2026](https://arxiv.org/html/2607.26470#bib.bib29); Yeet al\.[2025](https://arxiv.org/html/2607.26470#bib.bib31)\), yet typically assumes a self\-contained query with dependencies confined to the current turn\. Memory\-based conversational systems store histories, summaries, or embeddings\(Liuet al\.[2024b](https://arxiv.org/html/2607.26470#bib.bib14); Zhonget al\.[2024](https://arxiv.org/html/2607.26470#bib.bib15)\), while dialogue graphs model utterance\-level relations\(Liet al\.[2020](https://arxiv.org/html/2607.26470#bib.bib17); Fanet al\.[2023](https://arxiv.org/html/2607.26470#bib.bib42); Zhuet al\.[2025c](https://arxiv.org/html/2607.26470#bib.bib32)\)\. None explicitly represents retrieval\-level dependencies across turns\. As a result, retrievers require sub\-question\-level memory, whereas existing systems largely maintain only turn\-level context\.

Our key insight is to align conversational memory with retrieval by storing dialogue context as sub\-question\-level reasoning traces\. A useful memory unit for this setting should preserve the retrieval target, expose the dependency that resolves missing arguments, and retain the evidence that made the earlier answer valid\. A sub\-question\-level trace provides this unit by packaging a past reasoning step as an addressable object\. When a later turn depends on it, the system can recover the relevant trace through its dependency links and keywords, then reuse the associated evidence under the current query\. Cross\-turn recall is therefore reduced from global history interpretation to trace selection and evidence reuse\.

Accordingly, we proposeCMT\-RAG, a framework built around*complementary memory traces*\. At each turn, a state\-space trace generator consumes the current query and its recurrent state to produce structured*trace drafts*, each containing a sub\-question, trace keywords, and dependencies on earlier traces\. After answer\-reference resolution, fresh evidence is retrieved for each sub\-question, and the draft is completed by attaching the corresponding paragraph identifiers\. Dependency links access prerequisite DAG nodes, while trace keywords retrieve an additional historical trace through lexical matching\. The reader transiently combines the accessed historical evidence with the freshly retrieved evidence\. After answering, the completed trace and its sub\-answer are appended to the DAG\. The recurrent state thus maintains local discourse continuity, while the trace DAG preserves explicit long\-range dependencies and reusable evidence\. The downstream reader remains stateless, receiving only the resolved sub\-question and its assembled evidence\.

To study this problem directly, we introduceMuMu\-QA, a benchmark that reorganizes multi\-hop questions into multi\-turn dialogues with cross\-turn dependency annotations\. Existing multi\-turn RAG benchmarks evaluate conversational retrieval and generation at the turn level, without exposing which current sub\-question depends on which prior sub\-question\. MuMu\-QA fills this gap by providing dialogue\-wide sub\-question identifiers, trace keywords, dependency edges, supporting paragraph IDs, and full trace DAG supervision, with long\-dialogue splits for stress\-testing dependency recovery beyond short history replay\. Experiments on MuMu\-QA show that CMT\-RAG achieves the best answer accuracy among direct C\-RAG, query rewriting, agentic retrieval, decomposition\-based RAG, and dialogue\-structure baselines\. With a stateless Qwen3\-32B reader, it reaches 41\.73 EM and 55\.63 F1 using top\-5 retrieval while keeping cross\-turn memory outside the reader\.

## 2Preliminaries

This section fixes the notation and evaluation target used throughout the paper\. We first define multi\-turn multi\-hop conversational RAG as trace\-DAG induction, where each trace is a retrieval\-level memory unit that binds a sub\-question to dependencies, keywords, and evidence\. We then describe MuMu\-QA as the benchmark instantiation of this formulation, with supervision over sub\-question dependencies and reusable paragraph evidence\.

### 2\.1Task Formalization

We consider a multi\-turn multi\-hop conversational RAG session over an unstructured corpus𝒞\\mathcal\{C\}\. The dialogue is an ordered sequence ofTTuser turns,

𝒟=⟨q1,q2,…,qT⟩,\\mathcal\{D\}=\\langle q\_\{1\},q\_\{2\},\\ldots,q\_\{T\}\\rangle,\(1\)whereqtq\_\{t\}denotes the user query at turntt\. For each turn, the system must produce an answerata\_\{t\}grounded in evidence from𝒞\\mathcal\{C\}\. The distinctive difficulty is that a turn may contain multiple retrieval\-relevant sub\-questions, each of which may depend on information from earlier turns\. We formalize this setting as session\-level induction of a directed acyclic graph of*traces*,

𝒢=\(𝒱,ℰ𝒢\),\\mathcal\{G\}=\(\\mathcal\{V\},\\mathcal\{E\}\_\{\\mathcal\{G\}\}\),\(2\)where each node\(𝒯k,ak\)∈𝒱\(\\mathcal\{T\}\_\{k\},a\_\{k\}\)\\in\\mathcal\{V\}consists of a trace and its answer\. The trace𝒯k\\mathcal\{T\}\_\{k\}binds a sub\-question, trace keywords, dependency edges, and retrieved paragraph identifiers as

𝒯k=\(qksub,k​wk,deps​\(k\),para​\_​idsk\),\\mathcal\{T\}\_\{k\}=\\bigl\(q\_\{k\}^\{\\text\{sub\}\},\\ kw\_\{k\},\\ \\mathrm\{deps\}\(k\),\\ \\mathrm\{para\\\_ids\}\_\{k\}\\bigr\),\(3\)whereqksubq\_\{k\}^\{\\text\{sub\}\}is the natural\-language sub\-question consumed by the reader and retriever,k​wkkw\_\{k\}are lookup anchors for the trace DAG,deps​\(k\)⊆\{1,…,k−1\}\\mathrm\{deps\}\(k\)\\subseteq\\\{1,\\ldots,k\-1\\\}lists prerequisite traces, andpara​\_​idsk\\mathrm\{para\\\_ids\}\_\{k\}identifies the paragraphs directly retrieved from𝒞\\mathcal\{C\}for this sub\-question\. Each edge\(𝒯j,𝒯i\)∈ℰ𝒢\(\\mathcal\{T\}\_\{j\},\\mathcal\{T\}\_\{i\}\)\\in\\mathcal\{E\}\_\{\\mathcal\{G\}\}states that trace𝒯i\\mathcal\{T\}\_\{i\}relies on the entity or subject introduced by trace𝒯j\\mathcal\{T\}\_\{j\}\.

At turntt, the trace generator maps the current queryqtq\_\{t\}and recurrent stateht−1h\_\{t\-1\}to an ordered set of trace drafts\. CMT\-RAG then resolves their predicted dependencies against𝒢<t\\mathcal\{G\}\_\{<t\}and appends the completed traces asΔ​𝒢t\\Delta\\mathcal\{G\}\_\{t\}\. This formulation couples two structured operations\. The system must decompose the current turn into retrieval units and link those units to prior traces whose subjects or entities remain necessary\. The target memory unit is not a whole utterance or an unstructured history summary\. It is a trace whose fields are directly consumed by retrieval, DAG lookup, and answering\.

### 2\.2Benchmark Construction

MuMu\-QA instantiates this formulation as a benchmark for multi\-turn multi\-hop RAG\. Existing multi\-turn C\-RAG benchmarks supervise standalone\-query rewriting or turn\-level answers\(Aliet al\.[2026](https://arxiv.org/html/2607.26470#bib.bib55); Chenget al\.[2025](https://arxiv.org/html/2607.26470#bib.bib34); Katsiset al\.[2025](https://arxiv.org/html/2607.26470#bib.bib51)\), without annotating dependencies at sub\-question granularity\. We constructMuMu\-QAfrom MuSiQue\(Trivediet al\.[2022](https://arxiv.org/html/2607.26470#bib.bib7)\), using its sub\-question decompositions, intermediate answers, and supporting paragraphs to derive supervision for trace generation\.

![Refer to caption](https://arxiv.org/html/2607.26470v1/x2.png)Figure 2:MuMu\-QA synthesis operators\. Parent questions are decomposed into sub\-questions with in\-turn dependencies\.Sub\-question Relocationmoves a sub\-question to a later turn to create cross\-turn dependencies, whileGraph Splicingjoins two reasoning chains via a bridge answer\.![Refer to caption](https://arxiv.org/html/2607.26470v1/x3.png)Figure 3:Overview of CMT\-RAG\. The framework consists of four stages: \(i\) Trace generation, where a state\-space model \(SSM\) maintains a recurrent state to generate structured trace drafts; \(ii\) Reference resolution, where cross\-turn dependencies are resolved through the trace DAG; \(iii\) Evidence retrieval and trace update, where supporting paragraphs are retrieved and completed traces are written back to the DAG; and \(iv\) Question answering, where the stateless reader answers the resolved sub\-questions using only the retrieved evidence, without replaying the dialogue history, before producing the final response\.As illustrated in[Figure˜2](https://arxiv.org/html/2607.26470#S2.F2), dialogues are synthesized using two operators\.Sub\-question Relocationmoves a sub\-question from a multi\-hop question into a separate turn and rewrites the remaining question as a follow\-up that depends on the relocated answer\.Graph Splicinglinks two reasoning chains through a shared bridge answer: a seed turn first resolves the bridge entity, and a later turn continues reasoning from that entity with explicit dependencies on earlier trace nodes\. Together, these operators generate short dialogues with controlled cross\-turn dependencies\. We further construct long dialogues by interleaving topic\-related sessions, producing conversations of up to several dozen turns for training and evaluation\. Full construction details are provided in Appendix[A](https://arxiv.org/html/2607.26470#A1)\.

## 3Method

CMT\-RAG instantiates the trace\-DAG formulation with two complementary memory channels: a state\-space trace generator that captures local discourse to produce structured trace drafts, and a session\-level trace DAG that persistently stores traces for dependency\-aware retrieval and evidence reuse\. As shown in[Figure˜3](https://arxiv.org/html/2607.26470#S2.F3), this design externalizes conversational state from the answering model, which remains stateless while the recurrent state and trace DAG jointly maintain local and long\-range conversational memory\.

### 3\.1State\-Space Trace Generation

We instantiate the trace generator with a state space model \(SSM\) backbone based on Mamba\-2\(Dao and Gu[2024](https://arxiv.org/html/2607.26470#bib.bib24); Gu and Dao[2024](https://arxiv.org/html/2607.26470#bib.bib22)\)while preserving its selective state\-space mixer, whose recurrent state serves as a compact carrier of local discourse context\. This enables the model to avoid repeatedly encoding the full dialogue history at each turn, thereby reducing exposure to lost\-in\-the\-middle effects\(Yuet al\.[2025](https://arxiv.org/html/2607.26470#bib.bib28); Liuet al\.[2024a](https://arxiv.org/html/2607.26470#bib.bib27)\)\. We further adapt the backbone through Low\-Rank Adaptation \(LoRA\) fine\-tuning and Direct Preference Optimization \(DPO\), together with a structured output vocabulary, so that it generates trace drafts rather than free\-form plans\.

At turntt, the generator receives the current queryqtq\_\{t\}, the previous hidden stateht−1h\_\{t\-1\}, and emits a set of draft traces and an updated state,

\{𝒯kdraft\}k∈t,ht\\displaystyle\\\{\\mathcal\{T\}^\{\\mathrm\{draft\}\}\_\{k\}\\\}\_\{k\\in t\},h\_\{t\}=TraceGenθ​\(ht−1,qt\),\\displaystyle=\\textsc\{TraceGen\}\_\{\\theta\}\(h\_\{t\-1\},q\_\{t\}\),\(4\)the updated statehth\_\{t\}carrying local continuity such as topic focus, intent shifts and surface coreference, is passed to the next turn\.

#### Structured trace drafts\.

Each draft has the form

𝒯kdraft=\(qkdecom,k​wk,deps​\(k\)\),\\mathcal\{T\}^\{\\mathrm\{draft\}\}\_\{k\}=\\bigl\(q\_\{k\}^\{\\mathrm\{decom\}\},\\ kw\_\{k\},\\ \\mathrm\{deps\}\(k\)\\bigr\),\(5\)whereqkdecomq\_\{k\}^\{\\mathrm\{decom\}\}is the decomposed sub\-question,k​wkkw\_\{k\}contains trace keywords for DAG lookup, anddeps​\(k\)\\mathrm\{deps\}\(k\)lists prerequisite trace identifiers\. We separate keywords from sub\-questions into two fields serving different purposes\. The sub\-question is optimized as natural\-language input for the reader and dense retrieval after reference resolution, whereas the keyword field is optimized for efficient trace\-DAG lookup via lightweight lexical matching\.

#### Global trace namespace\.

CMT\-RAG maintains an append\-only namespace shared across the dialogue\. Each completed trace is assigned a persistent trace identifier,\[T\_1\],…,\[T\_K\]\\texttt\{\[T\\\_1\]\},\\ldots,\\texttt\{\[T\\\_K\]\}, and a corresponding answer reference token\[A\_1\],…,\[A\_K\]\\texttt\{\[A\\\_1\]\},\\ldots,\\texttt\{\[A\\\_K\]\}\. The generator can therefore explicitly reference prior traces throughdeps​\(k\)\\mathrm\{deps\}\(k\)\.

### 3\.2Trace DAG as Persistent Memory

The trace DAG maintains a durable session\-level memory\. For each draft, CMT\-RAG first resolves answer\-reference tokens inqkdecomq\_\{k\}^\{\\mathrm\{decom\}\}using the answer attached to each referenced DAG node, and produces the resolved sub\-questionqksubq\_\{k\}^\{\\mathrm\{sub\}\}\. The system then performs two evidence retrieval operations\. Dependency edges directly recover evidence from prerequisite traces, while trace keywords retrieve stored paragraph identifiers from relevant historical traces\. Together, they form𝒫kprior\\mathcal\{P\}^\{\\mathrm\{prior\}\}\_\{k\}\. Since the keywords are generated as normalized retrieval anchors, lightweight lexical lookup suffices to identify relevant traces without maintaining a separate dense index\.

Fresh retrieval is then issued fromqksubq\_\{k\}^\{\\mathrm\{sub\}\}against the external corpus𝒞\\mathcal\{C\}to obtain the paragraph identifier set𝒫klocal\\mathcal\{P\}^\{\\mathrm\{local\}\}\_\{k\}\. The assembled paragraph set𝒫k\\mathcal\{P\}\_\{k\}is used for the latter inference stage\. The completed trace stores the resolved sub\-question, keywords, dependency links, and the newly retrieved paragraph set𝒫klocal\\mathcal\{P\}^\{\\mathrm\{local\}\}\_\{k\},

𝒯k\\displaystyle\\mathcal\{T\}\_\{k\}=\(qksub,k​wk,deps​\(k\),𝒫klocal\),\\displaystyle=\\bigl\(q\_\{k\}^\{\\mathrm\{sub\}\},\\ kw\_\{k\},\\ \\mathrm\{deps\}\(k\),\\ \\mathcal\{P\}^\{\\mathrm\{local\}\}\_\{k\}\\bigr\),\(6\)𝒫k\\displaystyle\\mathcal\{P\}\_\{k\}=𝒫klocal∪𝒫kprior\.\\displaystyle=\\mathcal\{P\}^\{\\mathrm\{local\}\}\_\{k\}\\cup\\mathcal\{P\}^\{\\mathrm\{prior\}\}\_\{k\}\.After the reader produces answeraka\_\{k\}, the completed DAG node\(𝒯k,ak\)\(\\mathcal\{T\}\_\{k\},a\_\{k\}\)is appended to𝒢\\mathcal\{G\}in topological order\. The trace and answer are subsequently accessed through different mechanisms: answer references resolve missing arguments in future sub\-questions, whereas trace keywords retrieve relevant traces together with their supporting evidence\.

### 3\.3Curriculum Learning and DPO Training

The trace generator is trained with curriculum\-based supervised fine\-tuning under progressively longer conversational contexts, followed by DPO to align generated traces with downstream retrieval and question answering\.

#### Three\-stage curriculum\.

Supervised training follows a three\-stage curriculum\. Stage 1 trains on single\-turn examples to learn the trace syntax and basic decomposition structure\. Stage 2 introduces short multi\-turn dialogues, and Stage 3 further extends training to longer dialogues\. Throughout all stages, the primary objective is the autoregressive language modeling lossℒlm\\mathcal\{L\}\_\{\\mathrm\{lm\}\}over the linearized gold trace sequence\. For a training instancex1:Tx\_\{1:T\},

ℒlm=−∑t=1Tlog⁡pθ​\(xt∣x<t\)\.\\mathcal\{L\}\_\{\\mathrm\{lm\}\}=\-\\sum\_\{t=1\}^\{T\}\\log p\_\{\\theta\}\(x\_\{t\}\\mid x\_\{<t\}\)\.\(7\)Later stages progressively increase dialogue length and cross\-turn dependency density, while replaying earlier\-stage examples to preserve the basic trace representation\.

#### DPO training\.

The DPO stage samples multiple candidate traces per turn from the long\-dialogue model and executes them through the fixed retrieval–reader pipeline\. Candidate traces are ranked using a composite reward combining final\-answer F1 \(FfinalF\_\{\\mathrm\{final\}\}\) and matched sub\-question F1 \(FsubF\_\{\\mathrm\{sub\}\}\):

R​\(τ;c\)=Ffinal​\(τ\)\+γ​Fsub​\(τ\),R\(\\tau;c\)=F\_\{\\mathrm\{final\}\}\(\\tau\)\+\\gamma F\_\{\\mathrm\{sub\}\}\(\\tau\),\(8\)where,ccdenotes the current dialogue context for the policy\.FfinalF\_\{\\mathrm\{final\}\}is computed between the reader’s final answer induced by candidate traceτ\\tauand the ground truth, whileFsubF\_\{\\mathrm\{sub\}\}averages F1 over lexically matched generated and reference sub\-questions\. Invalid traces are filtered after pair construction rather than rewarded explicitly\. Preference pairs are formed from sufficiently separated high\- and low\-reward traces, and DPO trains the trace generator to assign higher probability to preferred traces than rejected ones under the frozen long\-dialogue SFT reference policy\.

### 3\.4Inference with CMT\-RAG

[Algorithm˜1](https://arxiv.org/html/2607.26470#alg1)summarizes inference for a single user turn\. Conversational state is fully externalized into the recurrent state and trace DAG, allowing the reader to remain stateless and answer each resolved sub\-question using only its assembled evidence\. An additional reader call then aggregates the user query together with the current\-turn sub\-questions and sub\-answers into the final answerata\_\{t\}, while the updated DAG is carried forward to subsequent turns\.

Algorithm 1CMT\-RAG inference at turntt\.0:State

ht−1h\_\{t\-1\}, trace DAG

𝒢<t\\mathcal\{G\}\_\{<t\}, query

qtq\_\{t\}, retriever

ℛ\\mathcal\{R\}, reader

ℳ\\mathcal\{M\}
1:

\{𝒯kdraft\}t,ht←GenSSM​\(ht−1,qt\)\\\{\\mathcal\{T\}\_\{k\}^\{\\mathrm\{draft\}\}\\\}\_\{t\},h\_\{t\}\\leftarrow\\textsc\{Gen\}\_\{\\mathrm\{SSM\}\}\(h\_\{t\-1\},q\_\{t\}\)
2:

𝒢t←𝒢<t\\mathcal\{G\}\_\{t\}\\leftarrow\\mathcal\{G\}\_\{<t\}
3:foreach

𝒯kdraft=\(qkdecom,k​wk,depsk\)\\mathcal\{T\}\_\{k\}^\{\\mathrm\{draft\}\}=\(q\_\{k\}^\{\\mathrm\{decom\}\},kw\_\{k\},\\mathrm\{deps\}\_\{k\}\)in topological orderdo

4:

qksub←RefRes​\(qkdecom,depsk,𝒢t\)q\_\{k\}^\{\\mathrm\{sub\}\}\\leftarrow\\textsc\{RefRes\}\(q\_\{k\}^\{\\mathrm\{decom\}\},\\mathrm\{deps\}\_\{k\},\\mathcal\{G\}\_\{t\}\)
5:

𝒯kprior←Index​\(𝒢t,k​wk\)∪depsk\\mathcal\{T\}\_\{k\}^\{\\mathrm\{prior\}\}\\leftarrow\\textsc\{Index\}\(\\mathcal\{G\}\_\{t\},kw\_\{k\}\)\\cup\\mathrm\{deps\}\_\{k\}
6:

pidskprior←GetPIDs​\(𝒯kprior\)\\mathrm\{pids\}\_\{k\}^\{\\mathrm\{prior\}\}\\leftarrow\\textsc\{GetPIDs\}\(\\mathcal\{T\}\_\{k\}^\{\\mathrm\{prior\}\}\)
7:

pidsklocal←ℛ​\(qksub\)\\mathrm\{pids\}\_\{k\}^\{\\mathrm\{local\}\}\\leftarrow\\mathcal\{R\}\(q\_\{k\}^\{\\mathrm\{sub\}\}\)
8:

parask←Get​\(pidskprior∪pidsklocal\)\\mathrm\{paras\}\_\{k\}\\leftarrow\\textsc\{Get\}\(\\mathrm\{pids\}\_\{k\}^\{\\mathrm\{prior\}\}\\cup\\mathrm\{pids\}\_\{k\}^\{\\mathrm\{local\}\}\)
9:

ak←ℳ​\(qksub,parask\)a\_\{k\}\\leftarrow\\mathcal\{M\}\(q\_\{k\}^\{\\mathrm\{sub\}\},\\mathrm\{paras\}\_\{k\}\)
10:

𝒯k←\(qksub,k​wk,depsk,pidsklocal\)\\mathcal\{T\}\_\{k\}\\leftarrow\(q\_\{k\}^\{\\mathrm\{sub\}\},kw\_\{k\},\\mathrm\{deps\}\_\{k\},\\mathrm\{pids\}\_\{k\}^\{\\mathrm\{local\}\}\)
11:

𝒢t←𝒢t∪\{\(𝒯k,ak\)\}\\mathcal\{G\}\_\{t\}\\leftarrow\\mathcal\{G\}\_\{t\}\\cup\\\{\(\\mathcal\{T\}\_\{k\},a\_\{k\}\)\\\}
12:endfor

13:

at←Aggregate​\(qt,\{qks​u​b\}t,\{ak\}t\)a\_\{t\}\\leftarrow\\textsc\{Aggregate\}\(q\_\{t\},\\\{q\_\{k\}^\{sub\}\\\}\_\{t\},\\\{a\_\{k\}\\\}\_\{t\}\)
14:

15:return

at,ht,𝒢ta\_\{t\},h\_\{t\},\\mathcal\{G\}\_\{t\}

## 4Experiments

ReaderMethodTop\-k⋆k^\{\\star\}Avg\. Paras\.EM↑\\uparrow\(%\)F1↑\\uparrow\(%\)GoldCtx↑\\uparrow\(%\)Qwen3\-32BDirect C\-RAG202035\.6648\.8188\.42Direct C\-RAG \(With Thinking Mode\)202038\.2350\.2088\.42ReAct\(Yaoet al\.[2023](https://arxiv.org/html/2607.26470#bib.bib43)\)202023\.7037\.8090\.70Self\-Ask\(Presset al\.[2023](https://arxiv.org/html/2607.26470#bib.bib1)\)202027\.9038\.5084\.00HippoRAG\(Gutiérrezet al\.[2024](https://arxiv.org/html/2607.26470#bib.bib21)\)202028\.1740\.4972\.15SuRe\(Kimet al\.[2024](https://arxiv.org/html/2607.26470#bib.bib48)\)202030\.3043\.0089\.70Adaptive\-RAG\(Jeonget al\.[2024](https://arxiv.org/html/2607.26470#bib.bib44)\)202030\.3043\.7088\.50IRCoT\(Trivediet al\.[2023](https://arxiv.org/html/2607.26470#bib.bib2)\)202033\.1046\.2088\.80ChatQA\(Liuet al\.[2024b](https://arxiv.org/html/2607.26470#bib.bib14)\)202034\.9347\.1986\.14ConvSearch\-R1\(Zhuet al\.[2025a](https://arxiv.org/html/2607.26470#bib.bib33)\)202033\.3947\.9693\.38RQ\-RAG\(Chanet al\.[2024](https://arxiv.org/html/2607.26470#bib.bib4)\)514\.7228\.0538\.8277\.95RQ\-RAG†514\.7231\.2642\.3477\.95ChainRAG†\(Zhuet al\.[2025b](https://arxiv.org/html/2607.26470#bib.bib50)\)2021\.7635\.5448\.7283\.88LogicRAG†\(Chenet al\.[2026](https://arxiv.org/html/2607.26470#bib.bib29)\)2026\.8337\.3251\.6982\.28RLTST\(Fanet al\.[2023](https://arxiv.org/html/2607.26470#bib.bib42)\)202032\.2044\.7089\.60StructuredDDP\(Chi and Rudnicky[2022](https://arxiv.org/html/2607.26470#bib.bib45)\)202036\.3050\.8085\.60\\rowcolorcyan\!6\!blue\!4\\cellcolorwhiteCMT\-RAG \(ours\)513\.9941\.7355\.6386\.25\\rowcolorcyan\!6\!blue\!4\\cellcolorwhiteOracle traces514\.9042\.1856\.2388\.08Llama\-3\.3\-70B\-InstructIRCoT\(Trivediet al\.[2023](https://arxiv.org/html/2607.26470#bib.bib2)\)202037\.2748\.7288\.77StructuredDDP\(Chi and Rudnicky[2022](https://arxiv.org/html/2607.26470#bib.bib45)\)202037\.8149\.3185\.56ConvSearch\-R1\(Zhuet al\.[2025a](https://arxiv.org/html/2607.26470#bib.bib33)\)202040\.5752\.5793\.88LogicRAG†\(Chenet al\.[2026](https://arxiv.org/html/2607.26470#bib.bib29)\)2032\.1339\.2953\.1580\.39Direct C\-RAG202040\.6253\.7088\.42\\rowcolorcyan\!6\!blue\!4\\cellcolorwhiteCMT\-RAG \(ours\)514\.0544\.7057\.5585\.10\\rowcolorcyan\!6\!blue\!4\\cellcolorwhiteOracle traces515\.3047\.2260\.8289\.83Table 1:Main results on the MuMu\-QA long\-dialogue split\. All non\-oracle systems use DRAGON retrieval\. Top\-k⋆k^\{\\star\}is selected fromk∈\{5,10,20\}k\\\!\\in\\\!\\\{5,10,20\\\}by F1 for each baseline\. Avg\. Paras\. denotes the mean number of unique paragraphs in the reader context per turn after deduplication, and GoldCtx the mean recall of gold supporting paragraphs\.†\\daggerdenotes replaying the accumulated question–answer history before answer generation\. Oracle traces replace only the generated trace drafts with gold traces, leaving retrieval and the reader unchanged\.We evaluate whether complementary sub\-question\-level*memory traces*improve multi\-turn multi\-hop conversational RAG\. The experiments address five questions\.RQ1Does CMT\-RAG improve answer accuracy across different stateless readers and five categories of baseline methods?RQ2What gains come from the trace\-management framework and the trained trace generator?RQ3How do recurrent SSM state and persistent DAG memory contribute across dialogue lengths?RQ4How does the trace\-generator backbone affect answer quality and latency?RQ5Does CMT\-RAG transfer to additional shared\-corpus RAG benchmarks?

### 4\.1Experimental Setup

#### Dataset\.

As detailed in Appendix[A\.3](https://arxiv.org/html/2607.26470#A1.SS3), MuMu\-QA comprises three dialogue\-length regimes: short \(3–7 turns\), long \(6–32 turns\), and ultra\-long \(33–67 turns\)\. The short and long splits are used for training, while the ultra\-long split is reserved for the evaluation of length\-extrapolation\.

#### Baselines\.

We compare CMT\-RAG with five baseline families\. Direct C\-RAG retrieves with the unresolved current\-turn query\. Iterative and agentic retrieval methods include ReAct, Self\-Ask, HippoRAG, SuRe, Adaptive\-RAG, and IRCoT\. Conversational context methods include ChatQA and ConvSearch\-R1\. Query\-decomposition methods include RQ\-RAG, ChainRAG, and LogicRAG\. Dialogue\-structure methods include RLTST and StructuredDDP\. Within each reader setting, all non\-oracle systems use the same DRAGON corpus index, while each baseline retains its native reasoning or decomposition procedure\. The†\\daggervariants additionally replay the accumulated question–answer history at each turn\.

#### Implementation details\.

We initialize the trace generator from Mamba\-2\-2\.7B and train LoRA adapters with the three\-stage SFT curriculum in[Section˜3\.3](https://arxiv.org/html/2607.26470#S3.SS3), followed by reader\-specific DPO with reward weightsγ=0\.2\\gamma=0\.2\. Unless otherwise specified, CMT\-RAG retrieves five paragraphs per resolved sub\-question using DRAGON and additional traces via keyword\-overlap DAG lookup\. Full training and hyperparameter details are provided in Appendix[B](https://arxiv.org/html/2607.26470#A2)\.

#### Evaluation metrics\.

For end\-task QA, we report Exact Match \(EM\) and token\-level F1 against turn\-level gold answers\. Avg\. Paras\. is the mean number of unique paragraphs in the final reader context per turn after merging and deduplication\. GoldCtx is the mean per\-turn recall of gold supporting paragraphs in that final context\. For efficiency analyses, end\-to\-end latency includes trace or plan generation, retrieval, intermediate reader calls, and final\-answer generation, while excluding one\-time model and index initialization\. Further details appear in Appendix[C](https://arxiv.org/html/2607.26470#A3)\.

### 4\.2Main Results across Readers \(RQ1\)

We evaluate on the MuMu\-QA long\-dialogue split with Qwen3\-32B and Llama\-3\.3\-70B\-Instruct as stateless readers\. The two CMT\-RAG variants share the same Stage 3 SFT checkpoint, while each uses a DPO adapter trained from preference pairs generated with the corresponding reader\. This protocol evaluates compatibility with two readers rather than zero\-shot reader swapping\. At inference, CMT\-RAG carries the SSM state across turns, retrieves fresh paragraphs with DRAGON\(Linet al\.[2023](https://arxiv.org/html/2607.26470#bib.bib52)\)from resolved sub\-questions, and uses trace keywords for long\-range DAG lookup\.

[Table˜1](https://arxiv.org/html/2607.26470#S4.T1)shows that the iterative baselines do not surpass Direct C\-RAG under the shared evaluation protocol\. ConvSearch\-R1 obtains the highest GoldCtx recall without attaining the highest answer accuracy\. CMT\-RAG achieves the best non\-oracle EM/F1 with Qwen3\-32B \(41\.73/55\.63\) and Llama\-3\.3\-70B\-Instruct \(44\.70/57\.55\)\. Relative to Direct C\-RAG, the gains are 6\.07 EM and 6\.82 F1 with Qwen and 4\.08 EM and 3\.85 F1 with Llama\. These gains are obtained with approximately 14 unique paragraphs per turn instead of 20\. CMT\-RAG does not attain the highest GoldCtx recall, so the evidence supports more effective use of a smaller retrieved context rather than uniformly better supporting\-paragraph retrieval\. Oracle traces add 0\.60 F1 with Qwen and 3\.27 F1 with Llama, quantifying the remaining headroom in trace generation\.

### 4\.3Component Contributions \(RQ2\)

To isolate the contributions of the CMT framework and the trained trace generator, we compare three settings\. \(1\) A control setting that performs reader\-based query decomposition without constructing memory traces or modeling explicit cross\-turn dependencies\. \(2\) A CMT\-only setting that introduces the complete trace\-management framework but uses the reader, instead of a trained SSM, to generate trace drafts\. \(3\) The full CMT\-RAG model, in which the trace drafts are generated by the SFT\+DPO\-trained trace generator\.

Table 2:Ablation of the CMT framework and learned trace generator \(L\. Gen\.\)\. CMT performs trace construction, persistence, and DAG lookup\. When L\. Gen\. is disabled, the reader generates trace drafts\. MP denotes the mean number of unique paragraphs provided to the reader per turn\.[Table˜2](https://arxiv.org/html/2607.26470#S4.T2)separates the gain from complementary memory traces and the learned generator\. With reader\-generated traces, enabling CMT raises F1 by 3\.78 points for Qwen and 2\.66 points for Llama\. Replacing reader\-generated drafts with the trained SSM trace generator yields a further 4\.74 and 3\.17 F1\-point gain, respectively\. Although CMT increases the average number of retrieved paragraphs from about 10 to 14 per turn, the consistent improvements in EM and F1 indicate that the additional evidence is effectively utilized rather than introducing distracting context\.

### 4\.4Memory across Dialogue Lengths \(RQ3\)

We conduct an ablation study to evaluate the two complementary memory components of CMT\-RAG\. All experiments follow the same settings as[Table˜1](https://arxiv.org/html/2607.26470#S4.T1)with Qwen3\-32B\. We further conduct evaluation on the ultra\-long dialogue split with 33–67 turns of MuMu\-QA to assess the robustness of the trained trace generator under dialogue lengths beyond those seen during training\.

![Refer to caption](https://arxiv.org/html/2607.26470v1/x4.png)Figure 4:Ablation of the complementary memory framework across dialogue lengths\.*w/o DAG*removes persistent trace\-DAG lookup while retaining SSM runtime memory\.*w/o SSM state*disables cross\-turn SSM state carry\-over while retaining persistent trace\-DAG memory\.[Figure˜4](https://arxiv.org/html/2607.26470#S4.F4)shows distinct length profiles for the two memory channels\. Removing the DAG changes F1 by 0\.02, 0\.04, 0\.46, and 1\.15 points across the 3–7, 6–15, 16–28, and 33–67 turn bins, respectively, concentrating the DAG benefit in longer dialogues\. Removing cross\-turn SSM state reduces F1 by 8\.49, 8\.46, 11\.14, and 7\.11 points, making recurrent state the larger contributor in every length regime\. Overall, the results confirm the complementary roles of the two memory channels across different dialogue lengths\.

### 4\.5Backbone Comparison \(RQ4\)

We further evaluate a Transformer\-based variant of CMT\-RAG by replacing the original SSM backbone with Pythia\-2\.8B\(Bidermanet al\.[2023](https://arxiv.org/html/2607.26470#bib.bib53)\), whose parameter scale is comparable to that of the original Mamba2\-2\.7B backbone\. We train the model on MuMu\-QA using the same curriculum SFT and DPO procedure as the SSM\-based counterpart\. The resulting model serves as the decomposer and produces the same structured trace format described in[Section˜3\.1](https://arxiv.org/html/2607.26470#S3.SS1)\.

Table 3:Comparison of SSM and Transformer trace generators on the MuMu\-QA long\-dialogue split using Qwen3\-32B as the reader\. The SSM and Transformer are instantiated with Mamba\-2\-2\.7B and Pythia\-2\.8B, respectively\. E2E time \(s/turn\) includes decomposition, retrieval, and reader inference\.As shown in[Table˜3](https://arxiv.org/html/2607.26470#S4.T3), the SSM\-based decomposer consistently outperforms the Transformer baseline under both training settings\. Under SFT, it achieves modest gains of 0\.18 EM and 0\.07 F1\. The advantage becomes larger after DPO, reaching 41\.73 EM and 55\.63 F1, surpassing the Transformer by 1\.64 EM and 1\.60 F1, respectively\. Beyond answer quality, the SSM is also substantially more efficient\. In our implementation, it maintains a recurrent state across turns, whereas Pythia re\-encodes the accumulated dialogue history at every turn, reducing end\-to\-end latency from 1\.61 to 0\.83 seconds per turn after DPO \(a 48\.4% reduction\)\.

Table 4:Results on RECOR, HotpotQA, and 2WikiMultiHopQA\. All methods retrieve from the same open corpus\. We report token\-level F1, BLEU\-1, and ROUGE\-L on RECOR; EM and token\-level F1 on HotpotQA and 2WikiMultiHopQA\. E2E time includes decomposition \(or reasoning\), passage retrieval, and reader inference, averaged over dialogue turns\. All methods rerank the merged retrieval candidates and retain at most 10 passages for the final reader\.
### 4\.6Transfer across Benchmarks \(RQ5\)

We further evaluate transfer on the conversational retrieval benchmark RECOR\(Aliet al\.[2026](https://arxiv.org/html/2607.26470#bib.bib55)\)and the single\-turn multi\-hop QA benchmarks HotpotQA\(Yanget al\.[2018](https://arxiv.org/html/2607.26470#bib.bib56)\)and 2WikiMultiHopQA\(Hoet al\.[2020](https://arxiv.org/html/2607.26470#bib.bib57)\), using Qwen3\-32B as the reader\. All methods retrieve from a shared benchmark corpus with the same DRAGON dense retriever\(Linet al\.[2023](https://arxiv.org/html/2607.26470#bib.bib52)\); for HotpotQA and 2WikiMultiHopQA, the released per\-example contexts are merged into benchmark\-level corpora, rather than using the official FullWiki setting\. We report EM and token\-level F1 on the two development sets, and F1, BLEU\-1, and ROUGE\-L across all RECOR dialogue turns\.

[Table˜4](https://arxiv.org/html/2607.26470#S4.T4)compares CMT\-RAG with the strongest implemented representative from each baseline family\. On the multi\-turn benchmark RECOR, CMT\-RAG achieves the best results on all reported metrics, demonstrating that complementary memory traces effectively capture and reuse conversational context beyond the setting of MuMu\-QA\. On the single\-turn benchmarks HotpotQA and 2WikiMultiHopQA, CMT\-RAG also remains competitive, achieving the best EM on HotpotQA and the second\-best EM and F1 on 2WikiMultiHopQA\. Although LogicRAG attains slightly stronger results on 2WikiMultiHopQA, it relies on multiple iterative reader calls, incurring substantially higher inference latency and requiring 8\.4×\\timesmore end\-to\-end inference time than CMT\-RAG\. By contrast, CMT\-RAG delegates conversational parsing and memory management to a lightweight SSM trace generator, achieving a substantially better accuracy–efficiency trade\-off while remaining competitive on single\-turn multi\-hop QA\.

## 5Related Work

Conversational RAG and multi\-hop retrieval address complementary requirements of context\-dependent information seeking\. CMT\-RAG connects these directions by maintaining retrieval\-oriented sub\-question traces whose dependencies and evidence persist across dialogue turns\.

Conversational RAG methods represent dialogue context through history encoding\(Yanget al\.[2025](https://arxiv.org/html/2607.26470#bib.bib60); Qianet al\.[2022](https://arxiv.org/html/2607.26470#bib.bib59)\), memory\(Yeet al\.[2026](https://arxiv.org/html/2607.26470#bib.bib36); Zhuet al\.[2025c](https://arxiv.org/html/2607.26470#bib.bib32); Zhonget al\.[2024](https://arxiv.org/html/2607.26470#bib.bib15)\), or query reformulation\(Wuet al\.[2022](https://arxiv.org/html/2607.26470#bib.bib12); Ananthaet al\.[2021](https://arxiv.org/html/2607.26470#bib.bib10)\)\. ChatQA\(Liuet al\.[2024b](https://arxiv.org/html/2607.26470#bib.bib14)\)encodes conversational context for retrieval, and ConvGQR\(Moet al\.[2023](https://arxiv.org/html/2607.26470#bib.bib13)\)and ConvSearch\-R1\(Zhuet al\.[2025a](https://arxiv.org/html/2607.26470#bib.bib33)\)reformulate context\-dependent turns into standalone queries\. These approaches effectively resolve local ambiguity and coreference, but leave retrieval dependencies among intermediate sub\-questions implicit\. Dialogue discourse parsers and graph\-based conversational models make cross\-turn structure explicit\(Liet al\.[2020](https://arxiv.org/html/2607.26470#bib.bib17); Shi and Huang[2019](https://arxiv.org/html/2607.26470#bib.bib16); Fanet al\.[2023](https://arxiv.org/html/2607.26470#bib.bib42); Chi and Rudnicky[2022](https://arxiv.org/html/2607.26470#bib.bib45)\)\. Their nodes typically represent utterances or discourse units, and their edges encode discourse relations rather than dependencies between the sub\-questions that drive evidence retrieval\. CMT\-RAG instead represents dialogue context at retrieval\-oriented granularity, allowing each current sub\-question to address the specific prior trace and evidence on which it depends\.

Reasoning\-based retrieval decomposes complex questions into simpler units\(Huanget al\.[2023](https://arxiv.org/html/2607.26470#bib.bib8); Wolfsonet al\.[2020](https://arxiv.org/html/2607.26470#bib.bib5); Perezet al\.[2020](https://arxiv.org/html/2607.26470#bib.bib9)\)or alternates retrieval with reasoning\(Asaiet al\.[2024](https://arxiv.org/html/2607.26470#bib.bib46); Vermaet al\.[2024](https://arxiv.org/html/2607.26470#bib.bib30)\)\. QDG\(Hasson and Berant[2021](https://arxiv.org/html/2607.26470#bib.bib6)\), RQ\-RAG\(Chanet al\.[2024](https://arxiv.org/html/2607.26470#bib.bib4)\), ChainRAG\(Zhuet al\.[2025b](https://arxiv.org/html/2607.26470#bib.bib50)\), and LogicRAG\(Chenet al\.[2026](https://arxiv.org/html/2607.26470#bib.bib29)\)expose sub\-question or dependency structure, whereas IRCoT\(Trivediet al\.[2023](https://arxiv.org/html/2607.26470#bib.bib2)\)and Self\-Ask\(Presset al\.[2023](https://arxiv.org/html/2607.26470#bib.bib1)\)generate intermediate reasoning steps that guide successive retrieval\. These methods generally operate on self\-contained queries and do not persist the resulting sub\-questions, dependencies, and evidence as dialogue\-level memory\. Corpus\-level graph RAG methods organize relations among documents or entities\(Gutiérrezet al\.[2024](https://arxiv.org/html/2607.26470#bib.bib21); Edgeet al\.[2024](https://arxiv.org/html/2607.26470#bib.bib20)\), which complements rather than captures conversational dependencies\. CMT\-RAG uses a session\-level trace DAG whose nodes bind resolved sub\-questions, dependency links, lookup keywords, and supporting evidence, making the retrieval unit and the memory unit the same persistent object across turns\.

## 6Conclusion

We presentCMT\-RAG, a complementary memory framework that combines recurrent SSM state for local conversational context with a persistent trace DAG for long\-range dependency resolution and evidence reuse\. Each trace binds a resolved sub\-question, dependency links, lookup keywords, and trace\-local supporting evidence while keeping the reader stateless\. We also introduceMuMu\-QA, which provides sub\-question\-level cross\-turn supervision and long\-dialogue evaluation\. Experiments demonstrate that CMT\-RAG improves answer accuracy across two reader backbones while maintaining compact retrieval contexts\. As CMT\-RAG relies on accurate sub\-question decomposition and dependency prediction, future work will focus on more robust trace generation\. In addition, MuMu\-QA is synthetic and inherits the domain bias of MuSiQue, motivating further evaluation on human\-authored conversations\.

## References

- M\. Ali, A\. Abdallah, A\. Agarwal, H\. L\. Patel, and A\. Jatowt \(2026\)RECOR: reasoning\-focused multi\-turn conversational retrieval benchmark\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 2688–2723\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.129),[Link](https://aclanthology.org/2026.findings-acl.129/)Cited by:[§C\.6](https://arxiv.org/html/2607.26470#A3.SS6.p1.1),[§2\.2](https://arxiv.org/html/2607.26470#S2.SS2.p1.1),[§4\.6](https://arxiv.org/html/2607.26470#S4.SS6.p1.1)\.
- R\. Anantha, S\. Vakulenko, Z\. Tu, S\. Longpre, S\. Pulman, and S\. Chappidi \(2021\)Open\-domain question answering goes conversational via question rewriting\.InNorth American Chapter of the Association for Computational Linguistics \(NAACL\),pp\. 520–534\.External Links:[Document](https://dx.doi.org/10.18653/v1/2021.naacl-main.44),[Link](https://aclanthology.org/2021.naacl-main.44/)Cited by:[§1](https://arxiv.org/html/2607.26470#S1.p2.1),[§5](https://arxiv.org/html/2607.26470#S5.p2.1)\.
- A\. Asai, Z\. Wu, Y\. Wang, A\. Sil, and H\. Hajishirzi \(2024\)Self\-rag: learning to retrieve, generate, and critique through self\-reflection\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=hSyW5go0v8)Cited by:[§5](https://arxiv.org/html/2607.26470#S5.p3.1)\.
- S\. Biderman, H\. Schoelkopf, Q\. G\. Anthony, H\. Bradley, K\. O’Brien, E\. Hallahan, M\. A\. Khan, S\. Purohit, U\. S\. Prashanth, E\. Raff, A\. Skowron, L\. Sutawika, and O\. van der Wal \(2023\)Pythia: a suite for analyzing large language models across training and scaling\.InInternational Conference on Machine Learning \(ICML\),pp\. 2397–2430\.External Links:[Link](https://proceedings.mlr.press/v202/biderman23a.html)Cited by:[§4\.5](https://arxiv.org/html/2607.26470#S4.SS5.p1.1)\.
- C\. Chan, C\. Xu, R\. Yuan, H\. Luo, W\. Xue, Y\. Guo, and J\. Fu \(2024\)RQ\-RAG: learning to refine queries for retrieval augmented generation\.InConference on Language Modeling \(COLM\),External Links:[Link](https://openreview.net/forum?id=tzE7VqsaJ4)Cited by:[Table 1](https://arxiv.org/html/2607.26470#S4.T1.8.19.11.1),[§5](https://arxiv.org/html/2607.26470#S5.p3.1)\.
- S\. Chen, C\. Zhou, Z\. Yuan, Q\. Zhang, Z\. Cui, H\. Chen, Y\. Xiao, J\. Cao, and X\. Huang \(2026\)You don’t need pre\-built graphs for RAG: retrieval augmented generation with adaptive reasoning structures\.InAAAI Conference on Artificial Intelligence \(AAAI\),pp\. 30270–30278\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v40i36.40278),[Link](https://ojs.aaai.org/index.php/AAAI/article/view/40278)Cited by:[§1](https://arxiv.org/html/2607.26470#S1.p2.1),[Table 1](https://arxiv.org/html/2607.26470#S4.T1.7.7.1),[Table 1](https://arxiv.org/html/2607.26470#S4.T1.8.8.1),[§5](https://arxiv.org/html/2607.26470#S5.p3.1)\.
- Y\. Cheng, K\. Mao, Z\. Zhao, G\. Dong, H\. Qian, Y\. Wu, T\. Sakai, J\. Wen, and Z\. Dou \(2025\)CORAL: benchmarking multi\-turn conversational retrieval\-augmented generation\.InFindings of the North American Chapter of the Association for Computational Linguistics \(NAACL Findings\),pp\. 1308–1330\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-naacl.72),[Link](https://aclanthology.org/2025.findings-naacl.72/)Cited by:[§2\.2](https://arxiv.org/html/2607.26470#S2.SS2.p1.1)\.
- T\. Chi and A\. Rudnicky \(2022\)Structured dialogue discourse parsing\.InAnnual Meeting of the Special Interest Group on Discourse and Dialogue \(SIGDIAL\),pp\. 325–335\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.sigdial-1.32),[Link](https://aclanthology.org/2022.sigdial-1.32/)Cited by:[Table 1](https://arxiv.org/html/2607.26470#S4.T1.8.21.13.1),[Table 1](https://arxiv.org/html/2607.26470#S4.T1.8.25.17.1),[§5](https://arxiv.org/html/2607.26470#S5.p2.1)\.
- T\. Dao and A\. Gu \(2024\)Transformers are SSMs: generalized models and efficient algorithms through structured state space duality\.InInternational Conference on Machine Learning \(ICML\),pp\. 10041–10071\.External Links:[Link](https://proceedings.mlr.press/v235/dao24a.html)Cited by:[§3\.1](https://arxiv.org/html/2607.26470#S3.SS1.p1.1)\.
- D\. Edge, H\. Trinh, N\. Cheng, J\. Bradley, A\. Chao, A\. Mody, S\. Truitt, D\. Metropolitansky, R\. O\. Ness, and J\. Larson \(2024\)From local to global: a graph RAG approach to query\-focused summarization\.arXiv preprint arXiv:2404\.16130,pp\. 1–26\.External Links:[Link](https://arxiv.org/abs/2404.16130)Cited by:[§5](https://arxiv.org/html/2607.26470#S5.p3.1)\.
- Y\. Fan, F\. Jiang, P\. Li, F\. Kong, and Q\. Zhu \(2023\)Improving dialogue discourse parsing via reply\-to structures of addressee recognition\.InConference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 8484–8495\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.526),[Link](https://aclanthology.org/2023.emnlp-main.526/)Cited by:[§1](https://arxiv.org/html/2607.26470#S1.p2.1),[Table 1](https://arxiv.org/html/2607.26470#S4.T1.8.20.12.1),[§5](https://arxiv.org/html/2607.26470#S5.p2.1)\.
- A\. Gu and T\. Dao \(2024\)Mamba: linear\-time sequence modeling with selective state spaces\.InConference on Language Modeling \(COLM\),External Links:[Link](https://openreview.net/forum?id=tEYskw1VY2)Cited by:[§3\.1](https://arxiv.org/html/2607.26470#S3.SS1.p1.1)\.
- B\. J\. Gutiérrez, Y\. Shu, Y\. Gu, M\. Yasunaga, and Y\. Su \(2024\)HippoRAG: neurobiologically inspired long\-term memory for large language models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),pp\. 59532–59569\.External Links:[Document](https://dx.doi.org/10.52202/079017-1902),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/6ddc001d07ca4f319af96a3024f6dbd1-Abstract-Conference.html)Cited by:[Table 1](https://arxiv.org/html/2607.26470#S4.T1.8.13.5.1),[§5](https://arxiv.org/html/2607.26470#S5.p3.1)\.
- M\. Hasson and J\. Berant \(2021\)Question decomposition with dependency graphs\.InAutomated Knowledge Base Construction \(AKBC\),External Links:[Link](https://openreview.net/forum?id=j920k2uqQNz)Cited by:[§5](https://arxiv.org/html/2607.26470#S5.p3.1)\.
- X\. Ho, A\. Duong Nguyen, S\. Sugawara, and A\. Aizawa \(2020\)Constructing a multi\-hop qa dataset for comprehensive evaluation of reasoning steps\.InProceedings of the 28th International Conference on Computational Linguistics \(COLING\),pp\. 6609–6625\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.coling-main.580),[Link](https://aclanthology.org/2020.coling-main.580/)Cited by:[§C\.6](https://arxiv.org/html/2607.26470#A3.SS6.p1.1),[§4\.6](https://arxiv.org/html/2607.26470#S4.SS6.p1.1)\.
- Y\. Hu, Y\. Wang, and J\. McAuley \(2026\)Evaluating memory in LLM agents via incremental multi\-turn interactions\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=DT7JyQC3MR)Cited by:[§1](https://arxiv.org/html/2607.26470#S1.p1.1)\.
- X\. Huang, S\. Cheng, Y\. Shu, Y\. Bao, and Y\. Qu \(2023\)Question decomposition tree for answering complex questions over knowledge bases\.InAAAI Conference on Artificial Intelligence \(AAAI\),pp\. 12924–12932\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v37i11.26519),[Link](https://ojs.aaai.org/index.php/AAAI/article/view/26519)Cited by:[§5](https://arxiv.org/html/2607.26470#S5.p3.1)\.
- S\. Jeong, J\. Baek, S\. Cho, S\. J\. Hwang, and J\. Park \(2024\)Adaptive\-rag: learning to adapt retrieval\-augmented large language models through question complexity\.InNorth American Chapter of the Association for Computational Linguistics \(NAACL\),pp\. 7036–7050\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.naacl-long.389),[Link](https://aclanthology.org/2024.naacl-long.389/)Cited by:[Table 1](https://arxiv.org/html/2607.26470#S4.T1.8.15.7.1)\.
- Y\. Katsis, S\. Rosenthal, K\. Fadnis, C\. Gunasekara, Y\. Lee, L\. Popa, V\. Shah, H\. Zhu, D\. Contractor, and M\. Danilevsky \(2025\)MTRAG: a multi\-turn conversational benchmark for evaluating retrieval\-augmented generation systems\.Transactions of the Association for Computational Linguistics \(TACL\)13,pp\. 784–808\.External Links:[Document](https://dx.doi.org/10.1162/TACL.a.19),[Link](https://doi.org/10.1162/TACL.a.19)Cited by:[§2\.2](https://arxiv.org/html/2607.26470#S2.SS2.p1.1)\.
- T\. Khot, H\. Trivedi, M\. Finlayson, Y\. Fu, K\. Richardson, P\. Clark, and A\. Sabharwal \(2023\)Decomposed prompting: a modular approach for solving complex tasks\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=_nGgzQjzaRy)Cited by:[§1](https://arxiv.org/html/2607.26470#S1.p2.1)\.
- J\. Kim, J\. Nam, S\. Mo, J\. Park, S\. Lee, M\. Seo, J\. Ha, and J\. Shin \(2024\)SuRe: summarizing retrievals using answer candidates for open\-domain qa of llms\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=w4DW6qkRmt)Cited by:[Table 1](https://arxiv.org/html/2607.26470#S4.T1.8.14.6.1)\.
- P\. Laban, H\. Hayashi, Y\. Zhou, and J\. Neville \(2026\)LLMs get lost in multi\-turn conversation\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Document](https://dx.doi.org/10.48550/arXiv.2505.06120),[Link](https://openreview.net/forum?id=VKGTGGcwl6)Cited by:[§1](https://arxiv.org/html/2607.26470#S1.p1.1)\.
- J\. Li, M\. Liu, M\. Kan, Z\. Zheng, Z\. Wang, W\. Lei, T\. Liu, and B\. Qin \(2020\)Molweni: a challenge multiparty dialogues\-based machine reading comprehension dataset with discourse structure\.InInternational Conference on Computational Linguistics \(COLING\),pp\. 2642–2652\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.coling-main.238),[Link](https://aclanthology.org/2020.coling-main.238/)Cited by:[§1](https://arxiv.org/html/2607.26470#S1.p2.1),[§5](https://arxiv.org/html/2607.26470#S5.p2.1)\.
- S\. Lin, A\. Asai, M\. Li, B\. Oguz, J\. Lin, Y\. Mehdad, W\. Yih, and X\. Chen \(2023\)How to train your DRAGON: diverse augmentation towards generalizable dense retrieval\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 6385–6400\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.findings-emnlp.423),[Link](https://aclanthology.org/2023.findings-emnlp.423/)Cited by:[§C\.1](https://arxiv.org/html/2607.26470#A3.SS1.p1.1),[§4\.2](https://arxiv.org/html/2607.26470#S4.SS2.p1.1),[§4\.6](https://arxiv.org/html/2607.26470#S4.SS6.p1.1)\.
- N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. Liang \(2024a\)Lost in the middle: how language models use long contexts\.Transactions of the Association for Computational Linguistics \(TACL\)12,pp\. 157–173\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00638),[Link](https://aclanthology.org/2024.tacl-1.9/)Cited by:[§3\.1](https://arxiv.org/html/2607.26470#S3.SS1.p1.1)\.
- Z\. Liu, W\. Ping, R\. Roy, P\. Xu, C\. Lee, M\. Shoeybi, and B\. Catanzaro \(2024b\)ChatQA: surpassing GPT\-4 on conversational QA and RAG\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 15416–15459\.External Links:[Document](https://dx.doi.org/10.52202/079017-0493),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/1c0d54ebd0a6e58c4eca7d591e374b9d-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2607.26470#S1.p2.1),[Table 1](https://arxiv.org/html/2607.26470#S4.T1.8.17.9.1),[§5](https://arxiv.org/html/2607.26470#S5.p2.1)\.
- F\. Mo, K\. Mao, Y\. Zhu, Y\. Wu, K\. Huang, and J\. Nie \(2023\)ConvGQR: generative query reformulation for conversational search\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 4998–5012\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.acl-long.274),[Link](https://aclanthology.org/2023.acl-long.274/)Cited by:[§1](https://arxiv.org/html/2607.26470#S1.p2.1),[§5](https://arxiv.org/html/2607.26470#S5.p2.1)\.
- OpenAI \(2025\)OpenAI: gpt\-oss\-120b & gpt\-oss\-20b model card\.arXiv preprint arXiv:2508\.10925,pp\. 1–34\.External Links:[Link](https://arxiv.org/abs/2508.10925)Cited by:[§A\.2](https://arxiv.org/html/2607.26470#A1.SS2.SSS0.Px2.p2.1)\.
- E\. Perez, P\. Lewis, W\. Yih, K\. Cho, and D\. Kiela \(2020\)Unsupervised question decomposition for question answering\.InConference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 8864–8880\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.713),[Link](https://aclanthology.org/2020.emnlp-main.713/)Cited by:[§5](https://arxiv.org/html/2607.26470#S5.p3.1)\.
- O\. Press, M\. Zhang, S\. Min, L\. Schmidt, N\. A\. Smith, and M\. Lewis \(2023\)Measuring and narrowing the compositionality gap in language models\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 5687–5711\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.findings-emnlp.378),[Link](https://aclanthology.org/2023.findings-emnlp.378/)Cited by:[Table 1](https://arxiv.org/html/2607.26470#S4.T1.8.12.4.1),[§5](https://arxiv.org/html/2607.26470#S5.p3.1)\.
- J\. Qian, B\. Zou, M\. Dong, X\. Li, A\. T\. Aw, and Y\. Hong \(2022\)Capturing conversational interaction for question answering via global history reasoning\.InFindings of the North American Chapter of the Association for Computational Linguistics \(NAACL Findings\),pp\. 2065–2075\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.findings-naacl.159),[Link](https://aclanthology.org/2022.findings-naacl.159/)Cited by:[§5](https://arxiv.org/html/2607.26470#S5.p2.1)\.
- Z\. Shi and M\. Huang \(2019\)A deep sequential model for discourse parsing on multi\-party dialogues\.InAAAI Conference on Artificial Intelligence \(AAAI\),pp\. 7007–7014\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v33i01.33017007),[Link](https://ojs.aaai.org/index.php/AAAI/article/view/4680)Cited by:[§5](https://arxiv.org/html/2607.26470#S5.p2.1)\.
- H\. Trivedi, N\. Balasubramanian, T\. Khot, and A\. Sabharwal \(2022\)MuSiQue: multihop questions via single\-hop question composition\.Transactions of the Association for Computational Linguistics \(TACL\)10,pp\. 539–554\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00475),[Link](https://aclanthology.org/2022.tacl-1.31/)Cited by:[Appendix A](https://arxiv.org/html/2607.26470#A1.p1.1),[§2\.2](https://arxiv.org/html/2607.26470#S2.SS2.p1.1)\.
- H\. Trivedi, N\. Balasubramanian, T\. Khot, and A\. Sabharwal \(2023\)Interleaving retrieval with chain\-of\-thought reasoning for knowledge\-intensive multi\-step questions\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 10014–10037\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.acl-long.557),[Link](https://aclanthology.org/2023.acl-long.557/)Cited by:[§1](https://arxiv.org/html/2607.26470#S1.p2.1),[Table 1](https://arxiv.org/html/2607.26470#S4.T1.8.16.8.1),[Table 1](https://arxiv.org/html/2607.26470#S4.T1.8.24.16.2),[§5](https://arxiv.org/html/2607.26470#S5.p3.1)\.
- P\. Verma, S\. P\. Midigeshi, G\. Sinha, A\. Solin, N\. Natarajan, and A\. Sharma \(2024\)Plan×\\timesRAG: planning\-guided retrieval augmented generation\.arXiv preprint arXiv:2410\.20753,pp\. 1–19\.External Links:[Link](https://arxiv.org/abs/2410.20753)Cited by:[§5](https://arxiv.org/html/2607.26470#S5.p3.1)\.
- T\. Wolfson, M\. Geva, A\. Gupta, M\. Gardner, Y\. Goldberg, D\. Deutch, and J\. Berant \(2020\)Break it down: a question understanding benchmark\.Transactions of the Association for Computational Linguistics \(TACL\)8,pp\. 183–198\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00309),[Link](https://aclanthology.org/2020.tacl-1.13/)Cited by:[§5](https://arxiv.org/html/2607.26470#S5.p3.1)\.
- Z\. Wu, Y\. Luan, H\. Rashkin, D\. Reitter, H\. Hajishirzi, M\. Ostendorf, and G\. S\. Tomar \(2022\)CONQRR: conversational query rewriting for retrieval with reinforcement learning\.InConference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 10000–10014\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.emnlp-main.679),[Link](https://aclanthology.org/2022.emnlp-main.679/)Cited by:[§5](https://arxiv.org/html/2607.26470#S5.p2.1)\.
- S\. Yang, J\. Lee, J\. Bang, K\. Shim, M\. Kim, and S\. Chang \(2025\)Learning contextual retrieval for robust conversational search\.InConference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 11991–12003\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.602),[Link](https://aclanthology.org/2025.emnlp-main.602/)Cited by:[§5](https://arxiv.org/html/2607.26470#S5.p2.1)\.
- Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. W\. Cohen, R\. Salakhutdinov, and C\. D\. Manning \(2018\)HotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 2369–2380\.External Links:[Document](https://dx.doi.org/10.18653/v1/D18-1259),[Link](https://aclanthology.org/D18-1259/)Cited by:[§C\.6](https://arxiv.org/html/2607.26470#A3.SS6.p1.1),[§4\.6](https://arxiv.org/html/2607.26470#S4.SS6.p1.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. R\. Narasimhan, and Y\. Cao \(2023\)ReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=WE_vluYUL-X)Cited by:[Table 1](https://arxiv.org/html/2607.26470#S4.T1.8.11.3.1)\.
- L\. Ye, L\. Yu, Z\. Lei, Q\. Chen, J\. Zhou, and L\. He \(2025\)Optimizing question semantic space for dynamic retrieval\-augmented multi\-hop question answering\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 17814–17824\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.871),[Link](https://aclanthology.org/2025.acl-long.871/)Cited by:[§1](https://arxiv.org/html/2607.26470#S1.p2.1)\.
- Z\. Ye, J\. Huang, W\. Chen, and Y\. Zhang \(2026\)H\-Mem: hybrid multi\-dimensional memory management for long\-context conversational agents\.InConference of the European Chapter of the Association for Computational Linguistics \(EACL\),pp\. 7756–7775\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.eacl-long.363),[Link](https://aclanthology.org/2026.eacl-long.363/)Cited by:[§1](https://arxiv.org/html/2607.26470#S1.p1.1),[§5](https://arxiv.org/html/2607.26470#S5.p2.1)\.
- Y\. Yu, H\. Jiang, X\. Luo, Q\. Wu, C\. Lin, D\. Li, Y\. Yang, Y\. Huang, and L\. Qiu \(2025\)Mitigate position bias in LLMs via scaling a single hidden states channel\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 6092–6111\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.316),[Link](https://aclanthology.org/2025.findings-acl.316/)Cited by:[§3\.1](https://arxiv.org/html/2607.26470#S3.SS1.p1.1)\.
- W\. Zhong, L\. Guo, Q\. Gao, H\. Ye, and Y\. Wang \(2024\)MemoryBank: enhancing large language models with long\-term memory\.InAAAI Conference on Artificial Intelligence \(AAAI\),pp\. 19724–19731\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v38i17.29946),[Link](https://ojs.aaai.org/index.php/AAAI/article/view/29946)Cited by:[§1](https://arxiv.org/html/2607.26470#S1.p2.1),[§5](https://arxiv.org/html/2607.26470#S5.p2.1)\.
- C\. Zhu, S\. Wang, R\. Feng, K\. Song, and X\. Qiu \(2025a\)ConvSearch\-R1: enhancing query reformulation for conversational search with reasoning via reinforcement learning\.InConference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 26547–26564\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1349),[Link](https://aclanthology.org/2025.emnlp-main.1349/)Cited by:[§1](https://arxiv.org/html/2607.26470#S1.p2.1),[Table 1](https://arxiv.org/html/2607.26470#S4.T1.8.18.10.1),[Table 1](https://arxiv.org/html/2607.26470#S4.T1.8.26.18.1),[§5](https://arxiv.org/html/2607.26470#S5.p2.1)\.
- R\. Zhu, X\. Liu, Z\. Sun, Y\. Wang, and W\. Hu \(2025b\)Mitigating lost\-in\-retrieval problems in retrieval augmented multi\-hop question answering\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 22362–22375\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.1089),[Link](https://aclanthology.org/2025.acl-long.1089/)Cited by:[Table 1](https://arxiv.org/html/2607.26470#S4.T1.6.6.1),[§5](https://arxiv.org/html/2607.26470#S5.p3.1)\.
- Z\. Zhu, T\. Hu, H\. Zhang, D\. Yang, H\. Chen, M\. Zhang, and X\. Chen \(2025c\)CID\-GraphRAG: enhancing multi\-turn dialogue systems through dual\-pathway retrieval of conversation flow and context semantics\.arXiv preprint arXiv:2506\.19385,pp\. 1–18\.External Links:[Link](https://arxiv.org/abs/2506.19385)Cited by:[§1](https://arxiv.org/html/2607.26470#S1.p2.1),[§5](https://arxiv.org/html/2607.26470#S5.p2.1)\.

## Appendix AMuMu\-QA Construction

MuMu\-QA is designed to evaluate multi\-turn multi\-hop RAG under sub\-question\-level cross\-turn dependencies\. Starting from the sub\-question decompositions and supporting evidence provided by MuSiQue\(Trivediet al\.[2022](https://arxiv.org/html/2607.26470#bib.bib7)\), we reorganize independent reasoning chains into conversational sessions in which later turns may depend on intermediate results established earlier\. The construction process preserves the original reasoning and evidence supervision while introducing dialogue\-level dependency structure, enabling controlled evaluation of sub\-question decomposition, cross\-turn trace linking, and evidence reuse\. This section details the source data, dialogue synthesis procedure, split construction, and annotation schema\.

### A\.1Source Data and Filtering

MuMu\-QA is constructed from the answerable split of MuSiQue, which provides multi\-hop questions together with supporting paragraphs, sub\-question decompositions, and intermediate answers\. We exclude the unanswerable portion of the full split because MuMu\-QA targets cross\-turn dependency tracking rather than answerability detection or refusal behavior\. We further remove near\-duplicate examples whose decompositions and answers are effectively identical, preventing synthesized dialogues from collapsing into paraphrased repetitions of the same reasoning chain\.

### A\.2Dialogue Synthesis

MuMu\-QA is synthesized in two stages\. First, deterministic graph operations construct sub\-question nodes, intermediate answers, dependency edges, and evidence annotations directly from the MuSiQue reasoning graphs\. Second, an LLM realizes the resulting graph fragments as natural conversational questions while preserving the underlying reasoning structure\. Long\- and ultra\-long dialogues are subsequently obtained by interleaving synthesized sessions and globally remapping trace identifiers, dependencies, and paragraph indices\. The subsequent interleaving and identifier\-remapping stages require no additional LLM calls\.

#### Graph synthesis\.

MuMu\-QA uses two complementary graph\-level synthesis operations\.Sub\-question Relocationmoves an independently answerable sub\-question from a later source graph into an earlier conversational turn\. The original parent question is rewritten so that its reasoning naturally incorporates the relocated result, while the later source question becomes a follow\-up that explicitly depends on the relocated trace\. This operation introduces cross\-turn dependencies without modifying the remaining reasoning graph or paragraph supervision\.Graph Splicingchains multiple MuSiQue reasoning graphs by making a follow\-up graph depend on an intermediate answer established in an earlier turn\. For each follow\-up, we retain the minimal subgraph required to derive its final answer and reconnect the selected entry node to the preceding trace\. A teacher LLM then rewrites the selected reasoning graph into a natural conversational question, while all intermediate answers, dependency relations, and supporting\-evidence annotations are inherited directly from the underlying MuSiQue graphs\.

#### LLM\-based question realization\.

Only the reader\-facing turn questions are generated by an LLM\. All sub\-question nodes, intermediate answers, dependency edges, and evidence annotations are deterministically inherited from the original MuSiQue graphs\. Depending on the synthesis operator and graph structure, different question\-realization prompts are applied, as summarized in Table[5](https://arxiv.org/html/2607.26470#A1.T5)\. Long\- and ultra\-long dialogue synthesis does not invoke the LLM again; these stages only interleave previously synthesized dialogues and globally remap sub\-question identifiers, answer references, dependency edges, and paragraph indices\.

Table 5:Question realization under the two Stage 2 synthesis operators\. GS and SQR denote Graph Splicing and Sub\-question Relocation, respectively\. “Original” indicates direct reuse of the original \(or conversationalized\) MuSiQue question without LLM generation\.The default realization backend uses the open\-source LLM \(currentlygpt\-oss\-120b\(OpenAI[2025](https://arxiv.org/html/2607.26470#bib.bib58)\)\) with temperature 0\.2, a maximum of 220 generated tokens, and at most two generation attempts\. Any comparable instruction\-tuned LLM can be used\. We choose GPT\-OSS solely because it is open\-source and reproducible\.

#### Prompt templates\.

Prompt A is used only when the first graph\-splice turn corresponds to a dependency closure ending at an intermediate MuSiQue node rather than a complete source question\. Prompt B realizes the carrier turn after relocating an independently answerable sub\-question from a later reasoning graph while preserving the carrier’s original final answer\. Prompt C is shared by graph\-splice follow\-up turns and relocated source turns\. It requires the generated question to refer to the previous intermediate answer through an entity\-type\-compatible expression \(e\.g\., “that person” or “that city”\) rather than explicitly mentioning the answer itself\. The complete prompt templates are listed below\.

#### Prompt A \(Partial graph realization\)\.

Fuse a partial reasoning graph into a natural parent question\.

1YouaregivenapartialreasoninggraphfromMuSiQue\.

2

3Generateonenaturalparentquestionwhoseansweristhetargetanswer\.

4

5Input:

6\-OriginalMuSiQuequestion

7\-Selectedreasoningsteps

8\-Targetanswer

9

10Requirements:

11\-Thequestionmustbeanswerableusingonlytheselectedreasoningsteps\.

12\-Donotrevealthetargetanswer\.

13\-ReturnJSON:

14\{"question":"\.\.\."\}

#### Prompt B \(Carrier question realization\)\.

Generate a carrier question that naturally preserves a relocated sub\-question\.

1Youaregivenareasoninggraphcontainingitsoriginalreasoningchainandonerelocatedauxiliarysub\-question\.

2

3Generateonenaturalparentquestionwhosefinalanswerremainsunchangedwhilenaturallyincorporatingtheauxiliaryreasoningstep\.

4

5Input:

6\-OriginalMuSiQuequestion

7\-Carrierreasoninggraph

8\-Relocatedsub\-question

9\-Targetanswer

10

11ReturnJSON:

12\{"question":"\.\.\."\}

#### Prompt C \(Dependency follow\-up realization\)\.

Generate a context\-dependent follow\-up question using implicit references\.

1Youaregivenareasoninggraphwhosefirststepdependsonapreviousconversationalanswer\.

2

3Generateonenaturalfollow\-upquestionusingthespecifiedreferencephrase\(e\.g\.,"thatcity"\)insteadofexplicitlymentioningthepreviousanswer\.

4

5Input:

6\-Previousanswer

7\-Referencephrase

8\-Selectedreasoninggraph

9\-OriginalMuSiQuequestion

10\-Finalanswer

11

12Requirements:

13\-Usethereferencephrase\.

14\-Donotrevealeitherthepreviousanswerorthefinalanswer\.

15\-ReturnJSON:

16\{"question":"\.\.\."\}

#### Generation validation\.

Generated questions are automatically validated before being included in MuMu\-QA\. We reject generations that omit required reference phrases, reveal bridge entities or final answers, violate entity\-type constraints, contain malformed or repetitive wording, or exceed the prescribed length limit\. As a result, LLM generation is restricted to the natural\-language realization of reader\-facing questions, while the trace graph, dependency structure, intermediate answers, and supporting\-evidence annotations remain identical to those inherited from the underlying MuSiQue graphs\.

### A\.3Splits and Leakage Control

We partition dialogues by grouped supporting\-document titles and bridge entities rather than synthesized dialogue identifiers, preventing train and development splits from sharing nearly identical evidence configurations or intermediate answers under different surface forms\. Supporting paragraphs inherited from MuSiQue serve as the gold evidence annotations\.

Table 6:Statistics of the MuMu\-QA splits\. Each row reports the number of dialogues together with the range and average of dialogue turns, global sub\-questions, and cross\-turn dependency edges\. Cross\-turn edges refer only to dependencies pointing to sub\-questions in earlier turns\.Table[6](https://arxiv.org/html/2607.26470#A1.T6)summarizes the resulting dataset statistics\. For each dialogue, we report the number of turns \(TdT\_\{d\}\), global sub\-questions \(SdS\_\{d\}\), and cross\-turn dependency edges \(EdE\_\{d\}\)\. Turns, SubQ, and X\-Edge denote the minimum–maximum ranges within each split, whereas Avg\. T, Avg\. SQ, and Avg\. X\-E report the corresponding unweighted per\-dialogue averages\. Here, X\-Edge counts individual dependency links from a current\-turn sub\-question to prerequisite sub\-questions introduced in earlier turns\.

### A\.4Annotation Schema

Each dialogue contains a dialogue\-wide global\_subquestions namespace, cross\-turn dependency links, and one trace record for every node in the session DAG\. Each record specifies current trace identifier, a sub\-question, trace keywords used for DAG lookup, predecessor trace identifiers, and supporting\-paragraph identifiers\.

MuMu\-QA therefore evaluates whether a system can decompose each turn into sub\-question\-level retrieval units, connect these units to prerequisite traces from earlier turns, and ground each trace in paragraph\-level evidence that may be reused later in the dialogue\.

### A\.5Breakdown by Synthesis Mode

MuMu\-QA is synthesized using two complementary operations with different structural characteristics\.*Sub\-question Relocation*introduces cross\-turn dependencies by relocating intermediate reasoning steps across conversational turns, whereas*Graph Splicing*constructs longer reasoning chains by connecting multiple source reasoning graphs\. In the Stage 3 development split, Graph Splicing accounts for 75\.39% of evaluation turns and Sub\-question Relocation for the remaining 24\.61%\. The mixed Stage 3 setting therefore reflects the natural distribution of both synthesis modes in the final benchmark\.

Table 7:Performance of CMT\-RAG across different MuMu\-QA synthesis modes\. “Avg\. Paras\.” denotes the average number of unique retrieved paragraphs per turn after deduplication\.As shown in Table[7](https://arxiv.org/html/2607.26470#A1.T7), CMT\-RAG performs consistently across the two synthesis modes\. Graph Splicing achieves higher EM \(42\.31 vs\. 39\.98\) while using fewer retrieved paragraphs \(13\.41 vs\. 15\.76\), whereas both modes obtain nearly identical F1 scores\. The mixed Stage 3 split closely matches the overall benchmark performance, indicating that CMT\-RAG generalizes well across dialogue synthesis strategies with different cross\-turn dependency structures\.

## Appendix BDetails of Trace Generator

### B\.1Architecture

Figure[5](https://arxiv.org/html/2607.26470#A2.F5)illustrates the architecture of the SSM\-based trace generator\. We instantiate the generator with a pretrained Mamba\-2 backbone, whose selective state\-space mixer maintains a recurrent hidden state throughout the dialogue\. Unlike Transformer\-based generators that must replay the entire dialogue history at every turn, the SSM processes each new query incrementally while propagating its hidden state between consecutive turns\. This recurrent state serves as a compact short\-term memory that summarizes recent conversational context and enables efficient long\-dialogue generation\.

![Refer to caption](https://arxiv.org/html/2607.26470v1/x5.png)Figure 5:Architecture of the SSM\-based trace generator\. The recurrent SSM state carries dialogue context across turns, while the decoder generates structured sub\-question, keyword, and dependency fields\.At dialogue turntt, the current user query is appended to the input stream and processed together with the recurrent stateht−1h\_\{t\-1\}inherited from the previous turn\. After passing through the stacked selective SSM blocks, the model updates its hidden state tohth\_\{t\}, which is preserved and reused when processing the next user turn\. During autoregressive decoding, the generator emits a structured trace consisting of dedicated control tokens together with multiple trace fields,

\[TRACE\],\[SubQ\],\[KW\],\[DEP\],\[/TRACE\],\[\\texttt\{TRACE\}\],\\;\[\\texttt\{SubQ\}\],\\;\[\\texttt\{KW\}\],\\;\[\\texttt\{DEP\}\],\\;\[\\texttt\{/TRACE\}\],where each trace records a decomposed sub\-question, its retrieval keyword, and dependencies on previously generated traces\. These structural tokens are added to the tokenizer vocabulary before fine\-tuning so that trace generation can be learned through standard causal language modeling\.

Following common parameter\-efficient fine\-tuning practice, LoRA adapters are inserted only into the input and output projection layers of each Mamba\-2 block, while all pretrained backbone parameters remain frozen\. Consequently, the model learns to generate structured traces while preserving the long\-context modeling capability inherited from the pretrained SSM\.

### B\.2Computing Infrastructure

All training and evaluation experiments were conducted on a Linux server equipped with four NVIDIA H100 GPUs \(80 GB HBM3 memory each; 320 GB total\), two Intel Xeon Platinum 8468V CPUs \(96 physical cores and 192 hardware threads in total\), and 2\.0 TiB system memory\. The server ran Ubuntu 22\.04\.4 LTS with Linux kernel 5\.14\.0\.

#### Training environment:

Python 3\.10\.19; PyTorch 2\.3\.1 \(CUDA 12\.1\); Transformers 4\.43\.0; PEFT 0\.11\.1; Mamba\-SSM 2\.2\.2; causal\-conv1d 1\.4\.0; Triton 2\.3\.1; FAISS 1\.8\.0; Sentence\-Transformers 5\.3\.0\.

#### Inference environment:

We used Qwen3\-32B as the primary reader and Llama\-3\.3\-70B\-Instruct for additional reader\-backbone experiments\. Both readers were served in BF16 using vLLM 0\.11\.0, PyTorch 2\.8\.0 with CUDA 12\.8, and Transformers 4\.57\.1\.

### B\.3Trace Generator Training

Following the architecture described above, we optimize the trace generator using a three\-stage supervised curriculum followed by Direct Preference Optimization \(DPO\)\. We use the GPT\-NeoX tokenizer associated with Mamba\-2\-2\.7B and extend its vocabulary with seven structural tokens:\[PLAN\],\[/PLAN\],\[TRACE\],\[/TRACE\],\[SubQ\],\[KW\], and\[DEP\]\. Each generated trace consists of a decomposed sub\-question, a retrieval keyword for DAG lookup, and a list of predecessor trace identifiers\. During fine\-tuning, we insert Low\-Rank Adaptation \(LoRA\) modules into the Mambain\_projandout\_projprojections and jointly optimize the embedding and output rows corresponding to the newly introduced structural tokens\. All remaining backbone parameters remain frozen\.

#### Curriculum Supervised Fine\-Tuning\.

We optimize the trace generator using a three\-stage supervised curriculum\. Stage 1 trains a rank\-16 LoRA adapter on 9,653 single\-turn examples with 1,220 validation examples to learn the trace syntax and basic decomposition structure\. The resulting adapter is merged into the backbone before Stage 2, which trains a rank\-8 LoRA adapter on 3,104 multi\-turn dialogues from theshort\-dialoguesplit\. Stage3 continues training the same adapter on 5,045 dialogues from thelong\-dialogueextra\-training split, supplemented with 1,009 randomly sampled dialogues from the Stage2 training set for replay\. Stage 2 and Stage 3 checkpoints are selected according to validation loss\. Table[8](https://arxiv.org/html/2607.26470#A2.T8)summarizes the training hyperparameters\.

Table 8:Training configuration of the SSM trace generator used for the reported results\. Training\-item counts denote single\-turn examples, dialogue sessions, or offline preference pairs\. Stage 3 continues training from the Stage 2 LoRA adapter\.Unless otherwise specified, all SFT stages use AdamW with gradient clipping \(maximum norm 1\.0\), a linear warmup over the first 5% of optimization steps followed by linear learning\-rate decay, and random seed 42\. The maximum numbers of generated trace nodes are 64 and 128 for Stages 2 and 3, respectively\.

#### Preference Optimization\.

Preference pairs are constructed from the selected SFT Stage 3 policy\. For each dialogue turn with contextcc, we sample four candidate traces using temperature 1\.0 and top\-kksampling \(k=40k=40\)\. Each candidate traceτ\\tauis executed through the fixed retrieval–reader pipeline and assigned the reward

R​\(τ;c\)=Ffinal​\(τ\)\+γ​Fsub​\(τ\)\.R\(\\tau;c\)=F\_\{\\mathrm\{final\}\}\(\\tau\)\+\\gamma F\_\{\\mathrm\{sub\}\}\(\\tau\)\.\(9\)
whereFfinalF\_\{\\mathrm\{final\}\}is the final\-answer F1 andFsubF\_\{\\mathrm\{sub\}\}is the average F1 over semantically matched intermediate sub\-questions\. Intermediate sub\-questions are matched to the reference decomposition using a minimum semantic\-similarity threshold of 0\.55\.

Preference pairs are formed from the highest\- and lowest\-reward candidates whenever their reward difference is at least 0\.05\. Candidate traces with invalid formats are filtered before finalizing the preference pairs, resulting in 45,420 offline preference pairs\. We merge the Stage 3 LoRA adapter into the backbone and attach a new rank\-8 LoRA adapter containing 10\.7M trainable parameters for DPO, while the merged Stage 3 policy serves as the frozen reference model\. The policy is optimized for one epoch using standard DPO withβ=0\.05\\beta=0\.05, batch size 1, and no auxiliary SFT loss\. Notably, DPO is run for a pre\-specified single epoch, without downstream\-QA\-based early stopping or checkpoint search\.

### B\.4Hyperparameter Selection

We conduct a one\-factor\-at\-a\-time study on the MuMu\-QA development set using Qwen3\-32B\. This held\-out subset was excluded from all SFT and DPO training, including preference\-pair construction\.\. Unless otherwise specified, all remaining settings are kept identical to those in Table[1](https://arxiv.org/html/2607.26470#S4.T1)\. As shown in the left panel of Figure[6](https://arxiv.org/html/2607.26470#A2.F6), retrieving one trace from the DAG achieves the best EM/F1 of 41\.73/55\.63, compared with 41\.09/55\.24 without trace lookup\. Retrieving additional traces provides no further improvement\.

The right panel shows that retrieving five paragraphs per sub\-question performs best\. Increasingkparak\_\{\\mathrm\{para\}\}from 3 to 5 improves EM/F1 from 41\.14/54\.65 to 41\.73/55\.63, whereas larger retrieval budgets reduce performance\. We therefore setkDAG=1k\_\{\\mathrm\{DAG\}\}=1andkpara=5k\_\{\\mathrm\{para\}\}=5in all experiments\.

![Refer to caption](https://arxiv.org/html/2607.26470v1/x6.png)Figure 6:Hyperparameter selection for CMT\-RAG\. Left: number of traces retrieved from the session DAG\. Right: number of paragraphs retrieved per sub\-question\. Shaded columns indicate the selected settings\.Finally, we study the matched sub\-question reward weightγ\\gammaused to rank candidates during DPO preference\-pair construction\. For each value ofγ\\gamma, we recompute the composite reward from the same candidate pool and reconstruct preference pairs using the same selection and reward\-margin criteria, while keeping all other training and evaluation settings fixed\. As shown in Figure[7](https://arxiv.org/html/2607.26470#A2.F7), removing the matched sub\-question term \(γ=0\\gamma=0\) yields an EM/F1 of 39\.90/53\.68\. Settingγ=0\.2\\gamma=0\.2improves performance to 41\.73/55\.63, the best point estimate among the evaluated settings\. Increasing the weight further provides no consistent benefit:γ=0\.4\\gamma=0\.4,0\.60\.6, and0\.80\.8obtain EM/F1 scores of 41\.31/55\.55, 41\.49/55\.53, and 40\.81/54\.93, respectively\. These results indicate that moderate intermediate\-answer supervision improves preference construction, whereas overemphasizing the sub\-question term can weaken its alignment with final\-answer quality\. We therefore setγ=0\.2\\gamma=0\.2in all DPO experiments\.

![Refer to caption](https://arxiv.org/html/2607.26470v1/x7.png)Figure 7:DPO reward\-weight selection for CMT\-RAG with Qwen3\-32B on the MuMu\-QA development split\. We vary the coefficientγ\\gammaof matched sub\-question F1 in the pair\-ranking reward while keeping the final\-answer coefficient fixed at one\. EM is shown on the left axis and F1 on the right axis; the shaded column indicates the selected setting,γ=0\.2\\gamma=0\.2\.

## Appendix CInference and Evaluation

### C\.1Retrieval Details

Cross\-turn dependencies are first resolved using the dependency identifiers stored in each generated trace\. The referenced traces are retrieved from the session DAG through their global namespace identifiers, and their intermediate answers are substituted into the corresponding placeholders to obtain resolved sub\-questions\. Each resolved sub\-question then retrieves its top\-kksupporting paragraphs using the local DRAGON dense retriever\(Linet al\.[2023](https://arxiv.org/html/2607.26470#bib.bib52)\)\.

In parallel, the keyword field of each generated trace is used to perform keyword\-overlap lookup over the session DAG, retrieving at most one additional related trace node whose similarity score is at least 0\.05\. The paragraph identifiers stored in the retrieved trace are used to fetch the associated supporting paragraphs, which are merged with the paragraphs returned by DRAGON and deduplicated by document identifier to form the reader context\.

### C\.2Reader Details

The reader receives two types of inputs: \(i\) dependency facts resolved from previously generated traces and \(ii\) the merged supporting paragraphs retrieved for the current turn\. It is instructed to answer strictly according to the retrieved evidence and to output only the shortest answer span \(oryes/nowhen appropriate\), without explanation\.

#### System prompt\.

The system prompt defines the reader’s role and output constraints\.

1Answerthequestionusingonlytheprovidedparagraphs\.Replywithonlytheanswerspan,yes,orno\.Donotexplain\.

#### Sub\-question prompt\.

Each resolved sub\-question is answered using the following prompt template\.

1Providedparagraphs:

2\[Paragraph1\|score=\{score\_1\}\]

3Title:\{title\_1\}

4\{paragraph\_text\_1\}

5

6\[Paragraph2\|score=\{score\_2\}\]

7Title:\{title\_2\}

8\{paragraph\_text\_2\}

9

10\.\.\.

11

12Question:\{resolvedsub\-question\}

13

14Answer:

The final user query is answered using an analogous prompt that additionally includes the complete reasoning trace from the current turn\. Unless otherwise stated, inference uses a batch size of 8, at most 32 concurrent requests, and a maximum generation length of 512 tokens\. For models supporting an explicit thinking mode, we disable it during both sub\-question answering and final\-answer generation\.

### C\.3Stateful SSM Inference

The trace generator performs stateful inference by preserving its recurrent state across dialogue turns, allowing each new turn to process only the current prompt instead of replaying the full dialogue history\. Inference is performed in BF16 with a maximum recurrent context length of 8,192 tokens and greedy decoding with up to 512 generated tokens per turn\. To support efficient multi\-session serving, we batch up to 16 concurrent sessions and employ packed recurrent caches, GPU\-resident decoding buffers, and CUDA Graph execution to reduce memory overhead and decoding latency\.

### C\.4Case Study

[Figure˜8](https://arxiv.org/html/2607.26470#A3.F8)illustrates an actual evaluation example from MuMu\-QA\. At Turn 13, the user asks which UK label was bought by the major broadcaster that, together with ABC and NBC, is based in New York\. The trace generator decomposes the query into two structured traces\. Trace 28 first identifies the broadcaster by resolving dependency 20 while generating retrieval keywords, and Trace 29 then asks which UK label was bought by the resolved entity through dependency 28\.

During dependency resolution, the predicted trace \(Node 28\) resolves dependency 20 and identifies the broadcaster as*CBS*\. The DAG lookup further recovers an earlier trace node \(Node 2\) from Turn 1 using the trace keywords, reusing its stored supporting paragraphs as complementary evidence\. After Trace 28 is completed, the retriever retrieves additional paragraphs for Trace 29, and the reader receives the union of reused and newly retrieved evidence\. Based on the aggregated evidence, the reader correctly identifies*Oriole Records*as the final answer\. This example demonstrates how CMT\-RAG combines complementary memory traces: the predicted trace resolves recent conversational dependencies, while the trace DAG retrieves long\-range evidence that would otherwise be unavailable from local dialogue context alone\.

![Refer to caption](https://arxiv.org/html/2607.26470v1/x8.png)Figure 8:A real evaluation example from MuMu\-QA\. The current dialogue turn is first decomposed into structured traces with explicit dependencies and keywords\. Dependency resolution combines the predicted dependency with a keyword\-recovered historical trace node from the DAG, enabling evidence reuse across distant turns\. The retrieved and reused evidence is then aggregated and passed to the stateless reader to produce the final answer\.
### C\.5Evaluation Protocol

We evaluate on the completelong\-dialoguedevelopment split, containing 548 dialogues and 5,396 turns\. Unless otherwise specified, evaluation uses greedy trace generation, stateful SSM memory, DRAGON top\-5 paragraph retrieval, and top\-1 QR\-overlap trace lookup

We report exact match, token\-level F1, supporting\-context coverage, and the average number of unique paragraphs retrieved per turn\. End\-to\-end latency includes trace generation, retrieval, intermediate reader calls, final\-answer generation, while excluding one\-time model and retriever initialization\.

### C\.6Corpus\-level RAG Evaluation

We evaluate on three corpus\-level RAG benchmarks: HotpotQA\(Yanget al\.[2018](https://arxiv.org/html/2607.26470#bib.bib56)\), 2WikiMultiHopQA\(Hoet al\.[2020](https://arxiv.org/html/2607.26470#bib.bib57)\), and RECOR\(Aliet al\.[2026](https://arxiv.org/html/2607.26470#bib.bib55)\)\. HotpotQA contains 7,405 development questions requiring multi\-hop reasoning, primarily through bridge and comparison relations, over evidence distributed across two supporting Wikipedia documents\. 2WikiMultiHopQA contains 12,576 development questions covering bridge\-comparison, comparison, compositional, and inference reasoning patterns over two to four Wikipedia documents\. RECOR is a reasoning\-focused conversational retrieval benchmark comprising 707 multi\-turn conversations and 2,971 evaluation turns from 11 domains, where each turn requires resolving its information need from the dialogue history before retrieving supporting evidence\.

For all three benchmarks, every method retrieves from a shared corpus rather than an example\-specific context\. For HotpotQA and 2WikiMultiHopQA, we construct the retrieval corpus by pooling the released contexts across all examples, resulting in corpora containing 507,494 and 398,354 title documents, respectively\. Consequently, our HotpotQA protocol differs from the official FullWiki setting\. For RECOR, we directly use the complete released corpus for each domain\. Unless otherwise specified, Qwen3\-32B serves as the answering model\. We report EM and token\-level F1 on the HotpotQA and 2WikiMultiHopQA development sets, and token\-level F1, BLEU\-1, and ROUGE\-L on all RECOR turns\.