ATEST: Adaptive Resource-Aware Test-Time Scaling for LLM Inference

Hugging Face Daily Papers Papers
test-time-scaling llm-inference energy-efficiency resource-aware gpu-contention reasoning

Summary

ATEST: Adaptive Resource-Aware Test-Time Scaling for LLM Inference 提出了一种资源感知的测试时缩放框架，联合考虑额外采样的预期价值与当前 GPU 成本，在 GPU 负载变化下动态决定采样何时停止，从而在满足能耗预算的同时保持准确率，并显著减少 GPU 时间与能耗。

Test-time scaling (TTS) improves large language model (LLM) reasoning by sampling multiple answers, but fixed sampling budgets ignore both query difficulty and the changing energy cost of inference under GPU contention. We measure up to a 4.1-fold increase in energy for the same four-sample query as GPU load rises. We present ATEST, a resource-aware TTS framework that jointly considers the expected value of additional samples and their current GPU cost. ATEST-Online, its default mode, requires no offline calibration: it incrementally generates samples, estimates the stability of the plurality answer, monitors live GPU conditions, and learns step energy cost online to decide when to stop. ATEST-Calibrated instead selects a per-query budget using offline accuracy and cost models, favoring deployments with stable loads. We evaluate both modes on four reasoning and question-answering datasets using Phi-3-mini on an NVIDIA A100 under controlled varying load, and test transfer to an NVIDIA L40S and Qwen2.5-3B. Across the four A100 datasets, ATEST-Online answers every query without exceeding the energy budget, while achieving the highest or statistically tied accuracy and reducing sampling as load increases. On GSM8K at high load, it achieves 60.0% accuracy with no deferrals, versus 44.0% accuracy and 42.4% deferrals for ATEST-Calibrated. Ablations show that online cost learning drives adaptation to load. Including offline profiling, ATEST-Online uses 42 times less GPU time and 48 times less energy than ATEST-Calibrated for the evaluated workload. Online control trades higher tail latency and lower throughput on short tasks for robustness under varying load; offline calibration remains competitive under stable conditions.
Original Article
View Cached Full Text

Cached at: 10/22/26, 03:49 PM

Paper page - ATEST: Adaptive Resource-Aware Test-Time Scaling for LLM Inference

Source: https://huggingface.co/papers/2610.25519

Abstract

Test-timescaling(TTS)improveslargelanguagemodel(LLM)reasoningbysamplingmultipleanswers,butfixedsamplingbudgetsignorebothquerydifficultyandthechangingenergycostofinferenceunderGPUcontention.Wemeasureuptoa4.1-foldincreaseinenergyforthesamefour-samplequeryasGPUloadrises.WepresentATEST,aresource-awareTTSframeworkthatjointlyconsiderstheexpectedvalueofadditionalsamplesandtheircurrentGPUcost.ATEST-Online,itsdefaultmode,requiresnoofflinecalibration:itincrementallygeneratessamples,estimatesthestabilityofthepluralityanswer,monitorsliveGPUconditions,andlearnsstepenergycostonlinetodecidewhentostop.ATEST-Calibratedinsteadselectsaper-querybudgetusingofflineaccuracyandcostmodels,favoringdeploymentswithstableloads.Weevaluatebothmodesonfourreasoningandquestion-answeringdatasetsusingPhi-3-minionanNVIDIAA100undercontrolledvaryingload,andtesttransfertoanNVIDIAL40SandQwen2.5-3B.AcrossthefourA100datasets,ATEST-Onlineanswerseveryquerywithoutexceedingtheenergybudget,whileachievingthehighestorstatisticallytiedaccuracyandreducingsamplingasloadincreases.OnGSM8Kathighload,itachieves60.0%accuracywithnodeferrals,versus44.0%accuracyand42.4%deferralsforATEST-Calibrated.Ablationsshowthatonlinecostlearningdrivesadaptationtoload.Includingofflineprofiling,ATEST-Onlineuses42timeslessGPUtimeand48timeslessenergythanATEST-Calibratedfortheevaluatedworkload.Onlinecontroltradeshighertaillatencyandlowerthroughputonshorttasksforrobustnessundervaryingload;offlinecalibrationremainscompetitiveunderstableconditions.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2610\.25519

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.25519 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.25519 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.25519 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling

Hugging Face Daily Papers

This paper introduces AutoTTS, an environment-driven framework that automates the discovery of test-time scaling strategies for LLMs by formulating it as controller synthesis. It demonstrates improved accuracy-cost tradeoffs on mathematical reasoning benchmarks with minimal computational overhead.