D$^2$F-ReAG: Dynamic Decomposition and Filtering for Multi-Hop Reasoning-Augmented Generation

# D2F-ReAG: Dynamic Decomposition and Filtering for Multi-Hop Reasoning-Augmented Generation
Source: [https://arxiv.org/html/2608.04444](https://arxiv.org/html/2608.04444)
11institutetext:Northeastern University, Shenyang, China22institutetext:Kunming University of Science and Technology, Kunming, China33institutetext:NiuTrans Research, Shenyang, ChinaJunhao RuanShengwei TangSaihan ChenKaiyan ChangZhengtao YuTong XiaoJingbo ZhuJiaoyang LiJunhao RuanShengwei TangSaihan ChenKaiyan ChangZhengtao YuTong XiaoJingbo Zhu

###### Abstract

Large language models \(LLMs\) often generate inaccurate answers due to their reliance on static internal knowledge\. Retrieval\-augmented generation \(RAG\) addresses this limitation by integrating external knowledge and excelling at single\-hop queries\. However, it struggles with multi\-hop questions that require cross\-document reasoning\. Existing methods, such as graph structured RAG or question decomposition, often lack dynamic decomposition and effective filtering, which leads to lower efficiency and accuracy\. To overcome these limitations, we proposeDynamicDecomposition andFiltering for Multi\-HopReasoning\-AugmentedGeneration \(D2F\-ReAG\), a novel paradigm that adaptively controls reasoning depth by judging the reliability of the root\-level reasoning\. If the root reasoning is reliable, the model directly generates the answer\. Otherwise, the question is logically decomposed into sub\-questions, and the verified reasoning derived from these sub\-questions is used to refine the root reasoning\. Experiments on three multi\-hop benchmarks demonstrate the effectiveness of our method in handling complex multi\-hop questions\.

## 1Introduction

Large Language Models \(LLMs\) have demonstrated strong capabilities in language understanding and generation across a wide range of tasks\[[1](https://arxiv.org/html/2608.04444#bib.bib29)\]\. Despite these advances, they still suffer from inherent limitations such as outdated parametric knowledge and hallucinations, which often lead to factually incorrect or unsupported outputs and undermine their reliability in knowledge\-intensive scenarios\[[9](https://arxiv.org/html/2608.04444#bib.bib8)\]\. Retrieval\-augmented Generation \(RAG\) mitigates these issues by incorporating external knowledge sources into the generation process, thereby improving factuality and reducing the model’s reliance on its static internal knowledge\[[11](https://arxiv.org/html/2608.04444#bib.bib9)\]\.

For simple factual questions, retrievers can often locate the necessary evidence within a single retrieval step, making the process both efficient and straightforward\[[15](https://arxiv.org/html/2608.04444#bib.bib11)\]\. In contrast, multi\-hop reasoning questions require connecting multiple pieces of evidence that are typically scattered across different documents\[[3](https://arxiv.org/html/2608.04444#bib.bib12)\]\. To address this challenge, Graph\-based RAG \(Graph RAG\) organizes facts through pre\-built graph structures, enabling effective evidence traversal and multi\-hop retrieval across interconnected knowledge units\[[6](https://arxiv.org/html/2608.04444#bib.bib13)\]\. However, such graphs often suffer from inherent limitations: they may be incomplete due to the difficulty of capturing all relevant relations, are costly to construct and maintain at scale, and become less effective when the underlying knowledge is frequently updated or evolving\[[12](https://arxiv.org/html/2608.04444#bib.bib10)\]\.

Recent advances such as LogicRAG\[[2](https://arxiv.org/html/2608.04444#bib.bib34)\]reduce the reliance on pre\-constructed graphs by decomposing complex queries into sub\-questions and iteratively compressing retrieved evidence into a document\-level memory, enabling multi\-hop reasoning without explicit graph structures\. However, such compression cannot fully filter out redundant or erroneous content, allowing noise to accumulate across iterations and gradually mislead subsequent reasoning\[[21](https://arxiv.org/html/2608.04444#bib.bib15)\]\. Moreover, their query decomposition is often rigid and lacks adaptive control over granularity, so that simple questions tend to be over\-decomposed while genuinely complex ones are not decomposed deeply enough, ultimately degrading reasoning performance\[[10](https://arxiv.org/html/2608.04444#bib.bib31)\]\.

To overcome these limitations, we propose D2F\-ReAG, a multi\-hop reasoning framework that performs on\-demand question decomposition driven by reasoning reliability and augments generation with verified sub\-question reasoning paths\. Concretely, D2F\-ReAG continuously assesses the model’s confidence in its current reasoning, and triggers further decomposition only when the reasoning over the current question is judged unreliable\. Once a sub\-question is solved with sufficient confidence and deemed relevant to the query, its reasoning trace is propagated upward to update and guide the reasoning of the root question, so that accurate intermediate results are effectively integrated rather than discarded or compressed away\. To prevent over\-reasoning and unnecessary computational overhead, D2F\-ReAG also adopts an early\-stopping strategy that terminates the process as soon as the root question can be answered reliably\. In summary, our contributions are:

- •We propose D2F\-ReAG, a novel framework that dynamically decomposes complex questions into a sequence of logical sub\-questions and progressively augments the reasoning process of the root question with the correct reasoning paths derived from these sub\-questions, ensuring that accurate intermediate results are effectively propagated back to guide the final answer rather than being lost in lossy memory compression\.
- •We dynamically and adaptively balance the decomposition depth according to question complexity through a confidence\-driven control mechanism, enabling the framework to perform deeper decomposition only when necessary while avoiding redundant decomposition for simpler questions, thus achieving truly on\-demand decomposition that adapts to varying reasoning difficulty\.
- •We leverage reliable reasoning chains derived from sub\-questions to iteratively update and correct the root reasoning process, which effectively identifies and fixes intermediate errors, thereby reducing error accumulation across reasoning steps and improving the overall correctness of the final answer\.

## 2Related Work

Graph\-based methods organize external knowledge into structured graph representations to better support multi\-hop retrieval and cross\-document reasoning\. GraphRAG\[[6](https://arxiv.org/html/2608.04444#bib.bib13)\]performs hierarchical community search by jointly leveraging local and global queries, allowing retrieval to operate at varying levels of granularity\. LightRAG\[[7](https://arxiv.org/html/2608.04444#bib.bib16)\]further improves large\-scale retrieval efficiency through a two\-stage, graph\-augmented indexing pipeline that balances coverage and cost\. RAPTOR\[[14](https://arxiv.org/html/2608.04444#bib.bib17)\]constructs hierarchical summaries over the corpus to enable multi\-granular retrieval at different levels of abstraction, thereby accommodating queries of diverse specificity\. HippoRAG\[gutiérrez2024hipporag\]and its extension HippoRAG 2\[gutiérrez2025ragmemorynonparametriccontinual\]additionally enhance long\-context coherence by ranking nodes with PageRank\[[13](https://arxiv.org/html/2608.04444#bib.bib33)\]and integrating paragraph\-level memory, which helps preserve contextual continuity across retrieval steps\. Despite these advances, graph\-based methods generally rely on pre\-constructed structures that are costly to build and maintain, and may become outdated when the underlying knowledge evolves\.

Prompt\-based methods, in contrast, interleave reasoning and retrieval through carefully designed prompts without modifying model parameters, offering a more flexible and lightweight alternative\. ReAct\[[19](https://arxiv.org/html/2608.04444#bib.bib20)\]alternates between reasoning and retrieval in a step\-by\-step manner, enabling the model to dynamically decide when to retrieve based on its current reasoning state and observed evidence\. ChainRAG\[[20](https://arxiv.org/html/2608.04444#bib.bib21)\]decomposes complex questions into sub\-questions guided by prompts, and subsequently performs iterative sentence\-graph retrieval to capture both local fine\-grained details and global document\-level structures\. LogicRAG\[[2](https://arxiv.org/html/2608.04444#bib.bib34)\]similarly decomposes questions into logical components and aggregates evidence across multiple reasoning steps, leveraging logical relationships among sub\-questions to enhance retrieval coherence and reduce redundancy\. SiGIR\[[4](https://arxiv.org/html/2608.04444#bib.bib36)\]refines the reasoning process iteratively to progressively approach the final answer; however, it decomposes questions purely by atomicity and lacks explicit correctness verification during retrieval, which may allow erroneous intermediate results to propagate and accumulate across reasoning steps, ultimately degrading the quality of the final answer\.

## 3Method

We propose D2F\-ReAG, a dynamic framework for multi\-hop reasoning\. As shown in Figure[1](https://arxiv.org/html/2608.04444#S3.F1), D2F\-ReAG begins with \(I\) Retrieval & Generation, where first retrieves top\-kkdocuments to generate question’s reasoning\. In \(II\) Judge Reliability, a judge model evaluates the reliability of the reasoning\. If reliable, the question is solved; otherwise, \(III\) Decomposition & Rewriting decomposes the question into sub\-questions and rewrites them to retrieve more relevant evidence\. Finally, \(IV\) ReAG \(Reasoning\-Augmented Generation\) augments root reasoning with reliable, relevant reasoning traces of sub\-questions\. D2F\-ReAG stops once the root reasoning score exceeds a threshold\.

![Refer to caption](https://arxiv.org/html/2608.04444v1/x1.png)Figure 1:The D2F\-ReAG framework consists of four stages: \(I\) Retrieval & Generation, which retrieves relevant documents to generate a reasoning\. \(II\) Judge Reliability, where an LLM evaluates reasoning reliability against a threshold\. \(III\) Decomposition & Rewriting, which decomposes and rewrites the question when reliability is insufficient\. \(IV\) ReAG \(Reasoning\-Augments generation\), which integrates reliable and relevant sub\-question reasoning to update the root reasoning\. The process stops once the root question is answered reliably\.### 3\.1Retrieval & Generation

For the root questionqr​o​o​tq\_\{root\}and each sub\-questionqiq\_\{i\}, we retrieve the top\-kkmost relevant documentsD​\(q\)D\(q\)from the corpus𝒞\\mathcal\{C\}using a dense retriever, and then generate the corresponding reasoning processr​\(q\)r\(q\)conditioned on the retrieved evidence\. The retrieved documents serve as external knowledge that grounds the model’s reasoning, mitigating hallucinations and providing factual support for downstream judgment\. Formally, for any questionq∈\{qr​o​o​t,q1,…,qn\}q\\in\\\{q\_\{root\},q\_\{1\},\\dots,q\_\{n\}\\\},

r​\(q\)=Generator​\(q,D​\(q\)\),r\(q\)=\\mathrm\{Generator\}\\bigl\(q,D\(q\)\\bigr\),\(1\)whereGenerator\\mathrm\{Generator\}denotes the model that generates detailed reasoning based on relevant documents\. By explicitly conditioning onD​\(q\)D\(q\), the generator is encouraged to perform evidence\-aware reasoning rather than relying solely on its parametric memory, which is particularly important for knowledge\-intensive multi\-hop questions\.

### 3\.2Judge Reliability

After generating the reasoning for the root or sub\-questions, we judge its reliability using an LLM\-based scoring mechanism\. Rather than treating every generated reasoning chain as trustworthy, we introduce an explicit verification step that assesses whether the reasoning is logically coherent, factually consistent with the retrieved documents, and sufficient to answer the question\. We denote the reliability score of the reasoning process assr​\(q\)s\_\{r\}\(q\), which is calculated by LLM:

sr​\(q\)=Score​\(q,r​\(q\)\),s\_\{r\}\(q\)=\\text\{Score\}\(q,r\(q\)\),\(2\)wherer​\(q\)r\(q\)represents the reasoning process, andScore​\(q,r​\(q\)\)\\text\{Score\}\(q,r\(q\)\)is the reasoning score provided by the LLM according to our predefined rubric\. The rubric jointly considers multiple dimensions, including evidence grounding, logical consistency, and answer completeness, so that the resulting score reflects a holistic assessment of reasoning quality\.

The reasoning scoresr​\(q\)∈\[0,10\]s\_\{r\}\(q\)\\in\[0,10\]measures reasoning reliability \(higher is better\)\. We compare it to a thresholdθ\\theta\(set to 7 in our experiments\): ifsr​\(q\)\>θs\_\{r\}\(q\)\>\\theta, we consider the question solved\. Otherwise, D2F\-ReAG thinks the reasoning unreliable and further decomposes the question for continued reasoning \(Section[3\.3](https://arxiv.org/html/2608.04444#S3.SS3)\)\. This adaptive mechanism allows the system to allocate more computational effort only to questions that genuinely require deeper exploration, while terminating early on questions that are already well\-addressed\.

Judge=\{Solved,if​sr​\(q\)\>θDecompose,if​sr​\(q\)≤θ\.\\mathrm\{Judge\}=\\begin\{cases\}\\text\{Solved\},&\\text\{if \}s\_\{r\}\(q\)\>\\theta\\\\ \\text\{Decompose\},&\\text\{if \}s\_\{r\}\(q\)\\leq\\theta\.\\end\{cases\}\(3\)If the current question is the root question and is deemed solved, the LLM generates the final answer based on the corresponding reasoning process, ensuring that the answer is directly traceable to verified reasoning steps:

aroot=Answer​\(rqroot\)\.a\_\{\\text\{root\}\}=\\text\{Answer\}\\\!\\left\(r\_\{q\_\{\\text\{root\}\}\}\\right\)\.\(4\)If a sub\-question is reliably solved, we use its relevant sub\-questions to update the root reasoning in ReAG \(Section[3\.4](https://arxiv.org/html/2608.04444#S3.SS4)\), so that the verified intermediate conclusions can progressively enrich the global reasoning context\.

### 3\.3Decomposition & Rewriting

When the root question or any sub\-question remains unsolved, we use prompt engineering to logically decompose it into smaller, more tractable sub\-questions, each focusing on a single reasoning hop or a narrower information need\. This divide\-and\-conquer strategy reduces the cognitive load on the generator and makes retrieval more targeted\.

s​u​b​\(q\)=Decompose​\(q\),sub\(q\)=\\text\{Decompose\}\(q\),\(5\)wheres​u​b​\(q\)sub\(q\)is the set of sub\-questions obtained by logically decomposing the questionqq\.

D2F\-ReAG first solves each sub\-question in order and judges the reliability of the reasoning\. If reliable, we rewrite the related other sub\-questionsq∈𝒮iq\\in\\mathcal\{S\}\_\{i\}based on the correct reasoningrqr\_\{q\}, so that subsequent sub\-questions can leverage previously verified intermediate results and avoid redundant or contradictory reasoning paths\.

𝒮i′=\{Rewrite​\(q,rq\)∣q∈𝒮i\},\\mathcal\{S\}\_\{i\}^\{\\prime\}=\\\{\\,\\text\{Rewrite\}\(q,r\_\{q\}\)\\mid q\\in\\mathcal\{S\}\_\{i\}\\,\\\},\(6\)whererqr\_\{q\}is the reasoning to the current question, andqiq\_\{i\}is the related sub\-question\. Through this rewriting step, ambiguous references and missing entities in the original sub\-questions can be resolved using the newly acquired evidence, leading to higher\-quality retrieval in the next iteration\.

### 3\.4ReAG

When a sub\-question is solved reliably, we check its relevance to the root question and use its reasoning to update the root reasoning if relevant\. This relevance check prevents tangential or off\-topic sub\-question results from contaminating the global reasoning state\.

Check​\(qi,q\)=\{relevantirrelevant\\text\{Check\}\(q\_\{i\},q\)=\\begin\{cases\}\\text\{relevant\}\\\\ \\text\{irrelevant\}\\end\{cases\}\(7\)where Check\(qi,q\)\(q\_\{i\},q\)denotes the relevance ofqiq\_\{i\}to the root questionqq\.

If a sub\-question’s reasoning is reliable and relevant, it is used to update the root reasoning\. The update operation integrates newly verified evidence and intermediate conclusions into the existing root reasoning, gradually building a more complete and coherent chain of thought toward the final answer\. Formally, this can be written as:

r′​\(qr​o​o​t\)=Update​\(r​\(qr​o​o​t\),r​\(q\)\),r^\{\\prime\}\(q\_\{root\}\)=\\text\{Update\}\(r\(q\_\{root\}\),r\(q\)\),\(8\)wherer′​\(qr​o​o​t\)r^\{\\prime\}\(q\_\{root\}\)is the updated reasoning process of the root problemqq,r​\(q\)r\(q\)is the reasoning process of the sub\-questionqiq\_\{i\}, andUpdate​\(r​\(qr​o​o​t\),r​\(q\)\)\\text\{Update\}\(r\(q\_\{root\}\),r\(q\)\)updatesr​\(qr​o​o​t\)r\(q\_\{root\}\)by incorporatingr​\(q\)r\(q\)\.

Once the root reasoning is reliable, we stop processing the remaining sub\-questions and obtain the final answer to avoid over\-reasoning\. This early\-stopping strategy not only reduces unnecessary computational overhead but also prevents the introduction of noise from over\-decomposition, which could otherwise dilute the focus of the root reasoning and degrade answer accuracy\.

Table 1:Performance comparison on HotpotQA, 2WikiMultiHopQA, and MuSiQue using Str\-Acc and LLM\-Acc\.†\\daggerdenotes our reimplementation; the others are from LogicRAG\. Best results are in bold, and second\-best results are underlined\.TypeMethodHotpotQA2WikiMuSiQueStr\.LLMStr\.LLMStr\.LLMZero\-shotLlama3 \(8B\)17\.111\.122\.34\.72\.32\.0Llama3 \(13B\)23\.720\.133\.815\.46\.46\.0GPT\-3\.5\-Turbo31\.535\.424\.022\.07\.910\.9GPT\-4o\-Mini38\.736\.326\.424\.317\.614\.0GraphRAPTOR48\.157\.847\.745\.925\.229\.1GraphRAG39\.645\.246\.343\.316\.519\.4LightRAG47\.857\.743\.136\.318\.119\.4HippoRAG53\.556\.647\.247\.224\.930\.1HippoRAG256\.761\.950\.047\.127\.032\.6Prompt\-RAGVanillaRAG43\.253\.143\.042\.020\.323\.6ReAct†54\.256\.554\.950\.828\.831\.9ChainRAG†52\.156\.668\.966\.231\.033\.8LogicRAG†54\.262\.565\.362\.629\.636\.5OursD2F\-ReAG55\.863\.470\.368\.932\.237\.9

## 4Experiments

### 4\.1Dataset and Metrics

We evaluate on three standard multi\-hop reasoning benchmarks: HotpotQA\[[18](https://arxiv.org/html/2608.04444#bib.bib23)\], MuSiQue\[[16](https://arxiv.org/html/2608.04444#bib.bib22)\]and 2WikiMultiHopQA\[[8](https://arxiv.org/html/2608.04444#bib.bib24)\], covering diverse cross\-document and multi\-hop reasoning\. Following HippoRAG 2\[gutiérrez2025ragmemorynonparametriccontinual\], we use the same retrieval corpus and randomly sample 1,000 questions from each validation set for evaluation, ensuring fair comparison\.

Following LogicRAG\[[2](https://arxiv.org/html/2608.04444#bib.bib34)\], we adopt two metrics:Str\-AccandLLM\-Acc\.Str\-Accmeasures lexical correctness by checking whether the prediction exactly matches the ground\-truth answer after standard normalization\.LLM\-Accinstead employs a strong LLM as an automatic judge to assess whether the prediction is semantically equivalent to the reference, tolerating paraphrasing and surface\-form variations\. Reporting both metrics enables a balanced evaluation of strict lexical matching and flexible semantic correctness\.

### 4\.2Baselines

We compare D2F\-ReAG with three categories of baselines that cover representative paradigms in multi\-hop question answering\.Zero\-shotbaselines directly prompt LLMs without any external retrieval, including LLaMA3 \(8B\) and LLaMA3 \(13B\)\[[5](https://arxiv.org/html/2608.04444#bib.bib35)\], as well as gpt\-3\.5\-turbo and gpt\-4o\-mini\[[1](https://arxiv.org/html/2608.04444#bib.bib29)\], which serve to reflect the intrinsic reasoning ability of LLMs\.Graph RAGbaselines incorporate pre\-constructed graph structures to support multi\-hop retrieval, including RAPTOR\[[14](https://arxiv.org/html/2608.04444#bib.bib17)\], GraphRAG\[[6](https://arxiv.org/html/2608.04444#bib.bib13)\], LightRAG\[[7](https://arxiv.org/html/2608.04444#bib.bib16)\], HippoRAG\[gutiérrez2024hipporag\], and HippoRAG 2\[gutiérrez2025ragmemorynonparametriccontinual\]\.Prompt\-based RAGbaselines instead interleave reasoning and retrieval through prompt design without relying on explicit graph construction, including ReAct\[[19](https://arxiv.org/html/2608.04444#bib.bib20)\], ChainRAG\[[20](https://arxiv.org/html/2608.04444#bib.bib21)\], and LogicRAG\[[2](https://arxiv.org/html/2608.04444#bib.bib34)\]\.

### 4\.3Implementation Details\.

For fair comparison, all methods adopt the same experimental configuration\. We use sentence\-transformers/all\-MiniLM\-L6\-v2\[[17](https://arxiv.org/html/2608.04444#bib.bib28)\]as the unified embedding model for dense retrieval\. The number of retrieved passages \(top\-kk\) is fixed to 3 across all methods\. For answer generation, we employ gpt\-4o\-mini\[[1](https://arxiv.org/html/2608.04444#bib.bib29)\]as the backbone LLM, ensuring that performance differences stem from the retrieval and reasoning strategies rather than the underlying generator\. All experiments are conducted on a single NVIDIA RTX 3090 GPU\.

### 4\.4Main Results

Table[1](https://arxiv.org/html/2608.04444#S3.T1)reports results on three multi\-hop reasoning benchmarks \(HotpotQA, 2WikiMultiHopQA, and MuSiQue\)\. D2F\-ReAG achieves the best or near\-best performance on both Str\-Acc and LLM\-Acc across all three datasets, demonstrating consistent superiority over diverse baselines\.

Zero\-shot LLMs benefit from stronger backbones, with gpt\-4o\-mini clearly surpassing LLaMA3 \(8B/13B\), yet even the best zero\-shot model trails retrieval\-based methods, underscoring the need for external evidence in multi\-hop QA\. Graph\-based RAG methods \(RAPTOR, GraphRAG, LightRAG, HippoRAG, HippoRAG 2\) improve performance by structuring context into explicit graphs, with HippoRAG 2 being the strongest \(e\.g\., 56\.7 Str\-Acc on HotpotQA\), though their gains shrink on harder datasets like MuSiQue where graphs miss multi\-hop relations\. Prompt\-based RAG methods interleave retrieval and reasoning: ChainRAG performs well on 2Wiki via sub\-question decomposition with sentence\-graph retrieval, and LogicRAG attains the best baseline LLM\-Acc on HotpotQA \(62\.5\) through logical decomposition, but both suffer from fixed decomposition and noise accumulated over iterations\.

D2F\-ReAG consistently outperforms all baselines on 2Wiki \(70\.3 / 68\.9\) and MuSiQue \(32\.2 / 37\.9\), and achieves the highest LLM\-Acc on HotpotQA \(63\.4\), with the largest gains on 2WikiMultiHopQA where deep multi\-hop reasoning is most needed\. Against the strongest prompt\-based baseline LogicRAG, it yields up to 5\.4\-point Str\-Acc and 6\.4\-point LLM\-Acc improvements on 2Wiki, showing that on\-demand decomposition and reliability\-guided reasoning mitigate the over\- or under\-decomposition of fixed\-depth methods\. Joint gains on both metrics further indicate that the recovered reasoning paths are not only lexically aligned with gold answers but also semantically more faithful\. We report the best scores across runs, and provide a case study in Appendix[6](https://arxiv.org/html/2608.04444#S6)to illustrate how D2F\-ReAG corrects intermediate errors via reliable sub\-question reasoning\.

MuSiQue2WikiHotpotQA00\.20\.20\.40\.40\.60\.60\.80\.80\.590\.590\.630\.630\.80\.80\.410\.410\.370\.370\.20\.2ratew/o decompositionw/ decompositionFigure 2:Solved with vs\. without decomposition on three benchmarks\.Table 2:Accuracy and efficiency comparison on 2WikiMultiHopQA \(with vs\. without decomposition\)\.MethodStrAccLLMAccAvg\.Time \(s\)Avg\.TokensReAct54\.950\.813\.9311287LogicRAG65\.362\.615\.351998Ours \(No D\)71\.971\.09\.492267Ours \(\+D\)67\.365\.073\.5413321Ours \(Avg\.\)70\.368\.932\.46057
### 4\.5Decomposition & Efficiency Comparison

Figure[2](https://arxiv.org/html/2608.04444#S4.F2)shows that a large proportion of questions can be solved without decomposition, demonstrating the necessity of dynamic decomposition to efficiently handle both simple and complex questions\. Since ChainRAG also uses a pre\-built sentence graph, we compare latency and efficiency with ReAct and LogicRAG\. Table[2](https://arxiv.org/html/2608.04444#S4.T2)shows that always decomposing and judging reliability greatly increases time and token cost\. In contrast, on\-demand decomposition skips unnecessary steps, speeding up easy cases while still decomposing for hard ones\. While our multi\-step judgments can take longer on some difficult examples, Figure[2](https://arxiv.org/html/2608.04444#S4.F2)confirms these long cases are relatively few\. Overall, D2F\-ReAG balances between performance and efficiency and better matches human\-like reasoning\.

## 5Conclusion

Existing RAG methods still struggle with multi\-hop questions, primarily due to their fixed decomposition strategies and limited ability to filter out erroneous or irrelevant information during reasoning\. To address these issues, we propose D2F\-ReAG \(Dynamic Decomposition and Filtering for Multi\-Hop Reasoning\-Augmented Generation\), a framework that adaptively decomposes questions only when necessary and selectively filters intermediate reasoning according to its reliability\. By leveraging verified reasoning from reliable sub\-questions to progressively refine the root\-level generation, D2F\-ReAG effectively suppresses error propagation and produces more faithful reasoning chains, achieving strong performance across challenging multi\-hop benchmarks\.

## 6Case Study

We present a case study comparing the reasoning behaviors ofD2F\-ReAGandLogicRAG\.

Question:Which film has the director who died later,45 Calibre EchoorBons Baisers De Hong Kong? Gold Answer:Bons Baisers De Hong Kong

Failure Case of LogicRAG\. Model Answer:45 Calibre Echo\(Incorrect\)Rounds:1

#### Retrieved Contexts\.

LogicRAG retrieves three biography\-style passages:

- •Bruce M\. Mitchell: includes his death date \(September 26, 1952\), supporting reasoning about45 Calibre Echo\.
- •John Edward Bruce / Fred Bradley \(rower\): unrelated to either queried film\.

#### Dependency Analysis \(sorted\)\.

Death date of Bruce M\. Mitchell; Death date of Yvan Chiffre\.

#### Round 1\.

Query:Death date of Bruce M\. Mitchell\.can\_answer:true\. Understanding:Bruce M\. Mitchell \(director of45 Calibre Echo\) died on September 26, 1952; the death date of Yvan Chiffre is not retrieved\.LogicRAG nevertheless concludes45 Calibre Echodue to the missing comparison target\.

Correct Case of D2F\-ReAG\. Model Answer:Bons Baisers De Hong Kong\(Correct\) Root Rationale:The retrieved context names the directors \(Bruce M\. Mitchell and Yvan Chiffre\) but does not provide their death dates\.Judging reliability:0\.4

Decompose & Rewriting:

- •Who directed45 Calibre Echoand when did the director die?
- •Who directedBons Baisers De Hong Kongand when did the director die?
- •Which year is later?

Filtering for Multi\-Hop Reasoning\-Augmented Generation\. Subq 1:Director of45 Calibre Echois Bruce M\. Mitchell\.Reliability:0\.8 Subq 2:Director ofBons Baisers De Hong Kongis Yvan Chiffre\.Reliability:0\.8

Updating Root Reasoning and Rewriting\. Rewritten sub\-questions: When did Bruce M\. Mitchell die? When did Yvan Chiffre die?

Filtering for Multi\-Hop Reasoning\-Augmented Generation\. Yvan Chiffre died on 1990\-01\-01; Bruce M\. Mitchell died on 1988\-02\-19\.

Updating Root Reasoning and Rewriting\. Rewritten sub\-question 3:Which year is later, 1988 or 1990? Subq 3:1990\.Reliability:1\.0 Final Answer:Bons Baisers De Hong Kong

## References

- \[1\]J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman, S\. Anadkat,et al\.\(2023\)Gpt\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[§1](https://arxiv.org/html/2608.04444#S1.p1.1),[§4\.2](https://arxiv.org/html/2608.04444#S4.SS2.p1.1),[§4\.3](https://arxiv.org/html/2608.04444#S4.SS3.p1.1)\.
- \[2\]S\. Chen, C\. Zhou, Z\. Yuan, Q\. Zhang, Z\. Cui, H\. Chen, Y\. Xiao, J\. Cao, and X\. Huang\(2025\)You don’t need pre\-built graphs for rag: retrieval augmented generation with adaptive reasoning structures\.arXiv preprint arXiv:2508\.06105\.Cited by:[§1](https://arxiv.org/html/2608.04444#S1.p3.1),[§2](https://arxiv.org/html/2608.04444#S2.p2.1),[§4\.1](https://arxiv.org/html/2608.04444#S4.SS1.p2.1),[§4\.2](https://arxiv.org/html/2608.04444#S4.SS2.p1.1)\.
- \[3\]R\. Cheng, J\. Liu, Y\. Zheng, F\. Ni, J\. Du, H\. Mao, F\. Zhang, B\. Wang, and J\. Hao\(2025\)DualRAG: a dual\-process approach to integrate reasoning and retrieval for multi\-hop question answering\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Cited by:[§1](https://arxiv.org/html/2608.04444#S1.p2.1)\.
- \[4\]Z\. Chu, H\. Fan, J\. Chen, Q\. Wang, M\. Yang, J\. Liang, Z\. Wang, H\. Li, G\. Tang, M\. Liu, and B\. Qin\(2025\-07\)Self\-critique guided iterative reasoning for multi\-hop question answering\.InFindings of the Association for Computational Linguistics: ACL 2025,Cited by:[§2](https://arxiv.org/html/2608.04444#S2.p2.1)\.
- \[5\]A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Yang, A\. Fan,et al\.\(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§4\.2](https://arxiv.org/html/2608.04444#S4.SS2.p1.1)\.
- \[6\]D\. Edge, H\. Trinh, N\. Cheng, J\. Bradley, A\. Chao, A\. Mody, S\. Truitt, D\. Metropolitansky, R\. O\. Ness, and J\. Larson\(2024\)From local to global: a graph rag approach to query\-focused summarization\.arXiv preprint arXiv:2404\.16130\.Cited by:[§1](https://arxiv.org/html/2608.04444#S1.p2.1),[§2](https://arxiv.org/html/2608.04444#S2.p1.1),[§4\.2](https://arxiv.org/html/2608.04444#S4.SS2.p1.1)\.
- \[7\]Z\. Guo, L\. Xia, Y\. Yu, T\. Ao, and C\. Huang\(2025\)LightRAG: simple and fast retrieval\-augmented generation\.InFindings of the Association for Computational Linguistics: EMNLP 2025,Cited by:[§2](https://arxiv.org/html/2608.04444#S2.p1.1),[§4\.2](https://arxiv.org/html/2608.04444#S4.SS2.p1.1)\.
- \[8\]X\. Ho, A\. Duong Nguyen, S\. Sugawara, and A\. Aizawa\(2020\)Constructing a multi\-hop QA dataset for comprehensive evaluation of reasoning steps\.InProceedings of the 28th International Conference on Computational Linguistics,Cited by:[§4\.1](https://arxiv.org/html/2608.04444#S4.SS1.p1.1)\.
- \[9\]C\. Jiang, B\. Qi, X\. Hong, D\. Fu, Y\. Cheng, F\. Meng, M\. Yu, B\. Zhou, and J\. Zhou\(2024\)On large language models’ hallucination with regard to known facts\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),Cited by:[§1](https://arxiv.org/html/2608.04444#S1.p1.1)\.
- \[10\]G\. I\. Kim, J\. W\. Kim, and B\. Jang\(2025\-11\)UniRAG: a unified RAG framework for knowledge\-intensive queries with decomposition, break\-down reasoning, and iterative rewriting\.InFindings of the Association for Computational Linguistics: EMNLP 2025,Cited by:[§1](https://arxiv.org/html/2608.04444#S1.p3.1)\.
- \[11\]P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel,et al\.\(2020\)Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.Advances in neural information processing systems33,pp\. 9459–9474\.Cited by:[§1](https://arxiv.org/html/2608.04444#S1.p1.1)\.
- \[12\]J\. Li, J\. Ruan, S\. Tang, S\. Chen, K\. Chang, Y\. Ge, T\. Xiao, and J\. Zhu\(2025\)SUBQRAG: sub\-question driven dynamic graph rag\.arXiv preprint arXiv:2510\.07718\.Cited by:[§1](https://arxiv.org/html/2608.04444#S1.p2.1)\.
- \[13\]L\. Page, S\. Brin, R\. Motwani, and T\. Winograd\(1999\)The pagerank citation ranking: bringing order to the web\.\.Technical reportStanford infolab\.Cited by:[§2](https://arxiv.org/html/2608.04444#S2.p1.1)\.
- \[14\]P\. Sarthi, S\. Abdullah, A\. Tuli, S\. Khanna, A\. Goldie, and C\. D\. Manning\(2024\)RAPTOR: recursive abstractive processing for tree\-organized retrieval\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2608.04444#S2.p1.1),[§4\.2](https://arxiv.org/html/2608.04444#S4.SS2.p1.1)\.
- \[15\]Z\. Shi, S\. Zhang, W\. Sun, S\. Gao, P\. Ren, Z\. Chen, and Z\. Ren\(2024\)Generate\-then\-ground in retrieval\-augmented generation for multi\-hop question answering\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Cited by:[§1](https://arxiv.org/html/2608.04444#S1.p2.1)\.
- \[16\]H\. Trivedi, N\. Balasubramanian, T\. Khot, and A\. Sabharwal\(2022\)MuSiQue: multihop questions via single\-hop question composition\.Transactions of the Association for Computational Linguistics\.Cited by:[§4\.1](https://arxiv.org/html/2608.04444#S4.SS1.p1.1)\.
- \[17\]W\. Wang, H\. Bao, S\. Huang, L\. Dong, and F\. Wei\(2021\-08\)MiniLMv2: multi\-head self\-attention relation distillation for compressing pretrained transformers\.InFindings of the Association for Computational Linguistics: ACL\-IJCNLP 2021,Cited by:[§4\.3](https://arxiv.org/html/2608.04444#S4.SS3.p1.1)\.
- \[18\]Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. Cohen, R\. Salakhutdinov, and C\. D\. Manning\(2018\)HotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,Cited by:[§4\.1](https://arxiv.org/html/2608.04444#S4.SS1.p1.1)\.
- \[19\]S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao\(2023\)ReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2608.04444#S2.p2.1),[§4\.2](https://arxiv.org/html/2608.04444#S4.SS2.p1.1)\.
- \[20\]R\. Zhu, X\. Liu, Z\. Sun, Y\. Wang, and W\. Hu\(2025\)Mitigating lost\-in\-retrieval problems in retrieval augmented multi\-hop question answering\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Cited by:[§2](https://arxiv.org/html/2608.04444#S2.p2.1),[§4\.2](https://arxiv.org/html/2608.04444#S4.SS2.p1.1)\.
- \[21\]Z\. Zhuang, Z\. Zhang, S\. Cheng, F\. Yang, J\. Liu, S\. Huang, Q\. Lin, S\. Rajmohan, D\. Zhang, and Q\. Zhang\(2024\)EfficientRAG: efficient retriever for multi\-hop question answering\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Cited by:[§1](https://arxiv.org/html/2608.04444#S1.p3.1)\.