Knowing When Thinking Is Not Enough: Teaching Small Reasoning Models to Reason Beyond Their Parametric Knowledge

Hugging Face Daily Papers Papers
small-reasoning-models test-time-computation selective-querying flyby reinforcement-learning knowledge-bottleneck fine-tuning

Summary

This paper introduces FlyBy, a selective querying framework that trains small reasoning models to diagnose knowledge bottlenecks and query stronger models, improving reasoning performance on hard problems with lower serving cost.

Scaling test-time computation is a powerful way to improve language-model reasoning, and is particularly appealing for small reasoning models (sRMs) that are cheap to serve. However, is additional thinking always the right operation? By intervening at intermediate reasoning states across two model families and multiple scales, we find that self-refinement largely consolidates probability mass onto solutions already reachable from the current state, rather than making new ones reachable. These interventions reveal two failure regimes: execution bottlenecks, where the correct path is reachable and reflection can recover it, and knowledge bottlenecks, where relevant external information makes it reachable. Motivated by this distinction, we introduce FlyBy, a selective querying framework, and train 4B and 8B variants to reason first, diagnose what remains unresolved, and, at a knowledge bottleneck, query stronger models whose parametric knowledge extends beyond its own. Supervised fine-tuning bootstraps a multi-depth query action, and cost-aware reinforcement learning calibrates whether to query, what to ask, and how much to spend. On 1,158 hard problems across six benchmarks, FlyBy-4B achieves 45.96% pass@8, surpassing Qwen3-14B (41.64%) at 2.7 times lower serving cost, while also exceeding Qwen3-8B in pass@1 (16.85% vs. 15.31%). Scaling to FlyBy-8B further improves pass@8 to 51.81%.
Original Article
View Cached Full Text

Cached at: 09/29/26, 08:17 AM

Paper page - Knowing When Thinking Is Not Enough: Teaching Small Reasoning Models to Reason Beyond Their Parametric Knowledge

Source: https://huggingface.co/papers/2609.34327

Abstract

Scalingtest-timecomputationisapowerfulwaytoimprovelanguage-modelreasoning,andisparticularlyappealingforsmallreasoningmodels(sRMs)thatarecheaptoserve.However,isadditionalthinkingalwaystherightoperation?Byinterveningatintermediatereasoningstatesacrosstwomodelfamiliesandmultiplescales,wefindthatself-refinementlargelyconsolidatesprobabilitymassontosolutionsalreadyreachablefromthecurrentstate,ratherthanmakingnewonesreachable.Theseinterventionsrevealtwofailureregimes:executionbottlenecks,wherethecorrectpathisreachableandreflectioncanrecoverit,andknowledgebottlenecks,whererelevantexternalinformationmakesitreachable.Motivatedbythisdistinction,weintroduceFlyBy,aselectivequeryingframework,andtrain4Band8Bvariantstoreasonfirst,diagnosewhatremainsunresolved,and,ataknowledgebottleneck,querystrongermodelswhoseparametricknowledgeextendsbeyonditsown.Supervisedfine-tuningbootstrapsamulti-depthqueryaction,andcost-awarereinforcementlearningcalibrateswhethertoquery,whattoask,andhowmuchtospend.On1,158hardproblemsacrosssixbenchmarks,FlyBy-4Bachieves45.96%pass@8,surpassingQwen3-14B(41.64%)at2.7timeslowerservingcost,whilealsoexceedingQwen3-8Binpass@1(16.85%vs.15.31%)[email protected]%.

View arXiv pageView PDFGitHub0Add to collection

Get this paper in your agent:

hf papers read 2609\.34327

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.34327 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.34327 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.34327 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Enhanced and Efficient Reasoning in Large Learning Models

arXiv cs.AI

This paper proposes a method for improving reasoning in large language models by recoding data to explicitly represent relationships, enabling efficient principled reasoning with polynomial-time learnability for relational rules, which addresses hallucinations and supports sound reasoning across multiple calls.

Reasoning Can Be Restored by Correcting a Few Decision Tokens

arXiv cs.AI

This paper shows that the reasoning gap between base LLMs and large reasoning models is concentrated on a small set of early planning tokens. It introduces disagreement-guided token intervention, where replacing only those critical tokens with a reasoning model's outputs allows a base model to nearly match the reasoning model's performance.