What Effort Level Should I Choose? On the Economics of Reasoning Effort in LLM Agents

Hugging Face Daily Papers Papers
reasoning-effort llm-agents inference-compute cost-efficiency evaluation agentic-tasks

Summary

该论文将生成模型的推理 effort 视为'信任品'，通过反事实实验量化不同 effort 级别对成本与准确率的影响，发现从低 effort 起步、失败后再升级的策略在多数模型和基准上以更低成本达到或超过固定最高 effort 的效果。

Generative models increasingly expose a reasoning effort setting that lets users scale inference-time compute, trading higher cost for potentially improved performance. Users must choose an effort level before knowing whether extra reasoning will help, and even a successful run does not reveal whether the effort purchased was necessary. Reasoning effort is therefore a credence good: buyers observe the outcome but cannot assess the marginal value of the computation they paid for. In this work, we measure the counterfactual effect of effort levels on both proprietary and open-weight models on four benchmarks, spanning single-call reasoning and long-horizon agentic tasks. Effort, it turns out, does not consistently preserve tasks that lower effort solves, can overspend, and can change trajectory behavior. First, we find that although higher effort can raise accuracy in aggregate, it frequently fails on tasks that lower effort solved. Second, on the subset of easy tasks that every effort level solves, higher effort is generally more costly, with the highest effort level costing up to 42x as much as the lowest. Lastly, effort changes not only how many tokens are used, but also how agents solve tasks: at higher effort, agents verify their own work more heavily and more often search for solutions than work through problems directly, both of which lengthen trajectories without necessarily altering the outcome. These findings show that naively choosing the highest level is not only cost-inefficient but also offers no guarantee of the best instance-level performance. A simple alternative, starting at the lowest effort and escalating only after a verified failure, matches or improves on fixed top effort across all models and benchmarks at lower cost for most proprietary models.
Original Article
View Cached Full Text

Cached at: 10/28/26, 05:28 AM

Paper page - What Effort Level Should I Choose? On the Economics of Reasoning Effort in LLM Agents

Source: https://huggingface.co/papers/2610.34491

Abstract

Reasoning effort in generative models acts as a credence good where higher compute does not guarantee better instance-level results, and an escalation strategy from low effort improves cost-efficiency.

Generative models increasingly expose areasoning effortsetting that lets users scaleinference-time compute, trading higher cost for potentially improved performance. Users must choose an effort level before knowing whether extra reasoning will help, and even a successful run does not reveal whether the effort purchased was necessary.Reasoning effortis therefore a credence good: buyers observe the outcome but cannot assess the marginal value of the computation they paid for. In this work, we measure thecounterfactual effectof effort levels on both proprietary and open-weight models on four benchmarks, spanning single-call reasoning and long-horizonagentic tasks. Effort, it turns out, does not consistently preserve tasks that lower effort solves, can overspend, and can changetrajectory behavior. First, we find that although higher effort can raise accuracy in aggregate, it frequently fails on tasks that lower effort solved. Second, on the subset of easy tasks that every effort level solves, higher effort is generally more costly, with the highest effort level costing up to 42x as much as the lowest. Lastly, effort changes not only how many tokens are used, but also how agents solve tasks: at higher effort, agents verify their own work more heavily and more often search for solutions than work through problems directly, both of which lengthen trajectories without necessarily altering the outcome. These findings show that naively choosing the highest level is not only cost-inefficient but also offers no guarantee of the best instance-level performance. A simple alternative, starting at the lowest effort and escalating only after a verified failure, matches or improves on fixed top effort across all models and benchmarks at lower cost for most proprietary models.

View arXiv pageView PDFGitHub34Add to collection

Get this paper in your agent:

hf papers read 2610\.34491

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.34491 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.34491 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.34491 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Controlling Reasoning Effort in LLMs

Hacker News Top

The article discusses controlling reasoning effort in LLMs, including the release of OpenAI's GPT-5.6 with multiple reasoning-effort settings, and provides guidance on developing reasoning models with adjustable effort modes.

@ZeroZ_JQ: https://x.com/ZeroZ_JQ/status/2079504278922891568

X AI KOLs Timeline

This article provides an in-depth explanation of the principle of reasoning effort in large language models: by adjusting the reasoning effort level (low/medium/high) on the same model, more intermediate reasoning trajectories (serial autoregressive computation) are allowed before outputting the final answer, thereby making trade-offs among answer quality, response speed, and computational cost. The key is that the model has learned to organize longer generation processes into effective computation, rather than simply increasing parameters or network layers.