Teaching Thinking Models to Reason with Tools: A Full-Pipeline Recipe for Tool-Integrated Reasoning
Summary
This paper presents a full-pipeline recipe for teaching thinking models to reason with tools, achieving state-of-the-art performance on benchmarks like AIME 2025 when applied to Qwen3 models.
View Cached Full Text
Cached at: 05/08/26, 07:45 AM
# Teaching Thinking Models to Reason with Tools: A Full-Pipeline Recipe for Tool-Integrated Reasoning Source: [https://arxiv.org/abs/2605.06326](https://arxiv.org/abs/2605.06326) Authors:[Qianjia Cheng](https://arxiv.org/search/cs?searchtype=author&query=Cheng,+Q),[Yuchen Zhang](https://arxiv.org/search/cs?searchtype=author&query=Zhang,+Y),[Zhilin Wang](https://arxiv.org/search/cs?searchtype=author&query=Wang,+Z),[Yuxin Zuo](https://arxiv.org/search/cs?searchtype=author&query=Zuo,+Y),[Shunkai Zhang](https://arxiv.org/search/cs?searchtype=author&query=Zhang,+S),[Yuchen Fan](https://arxiv.org/search/cs?searchtype=author&query=Fan,+Y),[Yu Qiao](https://arxiv.org/search/cs?searchtype=author&query=Qiao,+Y),[Bowen Zhou](https://arxiv.org/search/cs?searchtype=author&query=Zhou,+B),[Ning Ding](https://arxiv.org/search/cs?searchtype=author&query=Ding,+N),[Yu Cheng](https://arxiv.org/search/cs?searchtype=author&query=Cheng,+Y),[Yun Luo](https://arxiv.org/search/cs?searchtype=author&query=Luo,+Y),[Ganqu Cui](https://arxiv.org/search/cs?searchtype=author&query=Cui,+G) [View PDF](https://arxiv.org/pdf/2605.06326) > Abstract:Tool\-integrated reasoning \(TIR\) offers a direct way to extend thinking models beyond the limits of text\-only reasoning\. Paradoxically, we observe that tool\-enabled evaluation can degrade reasoning performance even when the strong thinking models make almost no actual tool calls\. In this paper, we investigate how to inject natural tool\-use behavior into a strong thinking model without sacrificing its no\-tool reasoning ability, and present a comprehensive TIR recipe\. We highlight that \(i\) the effectiveness of TIR supervised fine\-tuning \(SFT\) hinges on the learnability of teacher trajectories, which should prioritize problems inherently suited for tool\-augmented solutions; \(ii\) controlling the proportion of tool\-use trajectories could mitigate the catastrophic forgetting of text\-only reasoning capacity; \(iii\) optimizing for pass@k and response length instead of training loss could maximize TIR SFT gains while preserving headroom for reinforcement learning \(RL\) exploration; \(iv\) a stable RL with verifiable rewards \(RLVR\) stage, built upon suitable SFT initialization and explicit safeguards against mode collapse, provides a simple yet remarkably effective solution\. When applied to Qwen3 thinking models at 4B and 30B scales, our recipe yields models that achieve state\-of\-the\-art performance in a wide range of benchmarks among open\-source models, such as 96\.7% and 99\.2% on AIME 2025 for 4B and 30B, respectively\. ## Submission history From: Qianjia Cheng \[[view email](https://arxiv.org/show-email/3461b66e/2605.06326)\] **\[v1\]**Thu, 7 May 2026 14:23:21 UTC \(975 KB\)
Similar Articles
OmniThoughtVis: A Scalable Distillation Pipeline for Deployable Multimodal Reasoning Models
This paper introduces OmniThoughtVis, a scalable pipeline for distilling multimodal reasoning capabilities from large teacher models to smaller, deployment-oriented MLLMs. The method uses curated chain-of-thought data to significantly improve reasoning performance on benchmarks like MathVerse and MMMU-Pro for models ranging from 2B to 8B parameters.
Teaching Language Models to Think in Code
This paper introduces ThinC (Thinking in Code), a framework where language models use code blocks exclusively for reasoning after a brief natural language planning step, outperforming existing tool-integrated reasoning baselines on math benchmarks.
Efficient Reasoning with Balanced Thinking
This paper introduces ReBalance, a training-free, plug-and-play method that dynamically balances overthinking and underthinking in large reasoning models, improving efficiency and accuracy across multiple benchmarks.
Improving Reasoning Capabilities in Small Models through Mixture-of-Layers Distillation with Stepwise Attention on Key Information
This paper proposes a novel Chain-of-Thought distillation framework that transfers teacher models' stepwise attention on key information to student models through a Mixture-of-Layers module for dynamic layer alignment. The method achieves consistent performance improvements on mathematical and commonsense reasoning benchmarks by explicitly guiding student models to progressively focus on critical information during reasoning.
I trained TIME: short context-triggered thinking on Qwen model instead of overthinking
A personal project led to an ACL 2026 paper introducing TIME, a method training Qwen3 models to engage in short, context-triggered thinking rather than excessive reasoning. The work uses QLoRA and a four-phase curriculum, with all data and code released open-source.