How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMs
Summary
FlowTracer is an RL framework that uses attention-induced graphs to trace reasoning flows and assign token-level credit, improving performance on reasoning tasks.
View Cached Full Text
Cached at: 06/10/26, 05:45 AM
Paper page - How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMs
Source: https://huggingface.co/papers/2606.10646 Authors:
,
,
,
,
,
,
,
,
,
,
Abstract
FlowTracer is an RL framework that uses attention-induced graphs to trace reasoning flows and assign token-level credit based on global information propagation structures.
Token-level credit assignmentremains a key obstacle forreinforcement learning(RL) inlarge language models(LLMs), where RL recipes typically treat all tokens equally, failing to distinguish decisive reasoning steps from routine formatting or fluent filler. Recent attempts leverage model-internal signals to assign finer-grained credit, but these are often point-wise heuristics that ignore the global structure of information propagation. We propose FlowTracer, an RL framework that traces answer-targeted reasoning flow on anattention-induced directed acyclic graphin which nodes correspond to tokens andedge capacitiescome from aggregated attention weights and derives token credit from this global structure. Theedge capacitiesare reweighted to retain only the influence that can reach the answer region, while enforcing localflow conservationso intermediate tokens neither lose nor gain effective mass due to path length or irrelevant branches. On this graph, FlowTracer extracts aninformation-flow backboneconnecting the question to the answer and scores tokens by flow throughput, revealing high-impact hubs and aggregation checkpoints that mediate long-range dependencies. These derived importances are used to shapetoken-level rewards, enabling learning signals to focus precisely on the tokens that route information toward (or away from) correct answers and delivering consistent performance gains across a range of reasoning tasks.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2606\.10646
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.10646 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.10646 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.10646 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
ReasoningFlow: Discourse Structures for Understanding LLM Reasoning Traces
Introduces ReasoningFlow, a framework to capture discourse structures of large language model reasoning traces as directed acyclic graphs, enabling fine-grained analysis of reasoning behaviors like self-reflection and backtracking. Based on manual and automatic annotation of thousands of traces, it reveals structural similarities across models and that most erroneous steps do not contribute to final answers.
AtManRL: Towards Faithful Reasoning via Differentiable Attention Saliency
AtManRL is a method that uses differentiable attention manipulation and reinforcement learning to train LLMs to generate more faithful chain-of-thought reasoning by ensuring reasoning tokens causally influence final predictions. Experiments on GSM8K and MMLU with Llama-3.2-3B demonstrate the approach can identify influential reasoning tokens and improve reasoning transparency.
FlowEdit: Information-Theoretic Control of LLM Reasoning Flows for Ill-posed Problems Involving Conflicts
FlowEdit is a novel framework that uses information-theoretic principles to regulate internal reasoning flows in LLMs, enabling them to generate multiple alternative responses in a single pass for ill-posed problems with conflicting conditions. Experiments show 68% improvement in exact-set-match accuracy and 24% boost in response informativeness over leading proprietary models.
How Do Answer Tokens Read Reasoning Traces? Self-Reading Patterns in Thinking LLMs for Quantitative Reasoning
Study reveals that answer tokens in thinking LLMs follow a structured self-reading pattern—forward drift plus focus on key anchors—during quantitative reasoning, and proposes a training-free SRQ steering method to exploit this for accuracy gains.
Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key
This paper introduces ScaleLogic, a framework demonstrating that RL training compute scales as a power law with reasoning depth in LLMs. It highlights that logical expressiveness is key to improving downstream transfer and training efficiency.