HALO: Hybrid Adaptive Latent Reasoning for Language Models

# HALO: Hybrid Adaptive Latent Reasoning for Language Models
Source: [https://arxiv.org/html/2607.08775](https://arxiv.org/html/2607.08775)
###### Abstract

We study how to improve a frozen pretrained language model with a small amount of adaptive extra computation\. A simple approach is to add additional refinement steps on top of the backbone hidden states, but fixed extra refinement can be wasteful: a one\-step refinement head may be too weak, while forcing a second full\-sequence refinement step everywhere can increase compute without improving transfer\. We introduce HALO, a hybrid adaptive latent\-refinement method that combines a coarse refinement stage with selective second\-stage latent refinement on a subset of tokens chosen by token scoring and monotonic token halting\. On the main public benchmark comparison built from MMLU\-Pro and GPQA\-Diamond, HALO achieves the best overall average among the paper\-facing methods, outperforming the frozen backbone, fixed\-1, and fixed\-2\. Internal analysis further shows that HALO reaches nearly the same token\-accuracy level as fixed\-2 while using fewer average applied refine steps than fixed\-1 and far fewer than fixed\-2\. These results suggest that the key advantage is not simply more refinement, but a better allocation of refinement: HALO achieves the strongest paper\-facing result while also using less measured controller compute than either fixed baseline\.

## 1Introduction

Frozen pretrained language models are strong general\-purpose systems, but some reasoning problems benefit from additional inference\-time computation\. A natural way to add such computation is to refine backbone hidden states after the main forward pass\. This can improve reasoning behavior without full\-model finetuning, but the extra computation must be allocated carefully\.

We introduceHALO\(HybridAdaptiveLatent reasOning\), a lightweight adaptive latent\-refinement method for frozen language models\. HALO uses a coarse refinement stage together with token scoring and monotonic token halting to choose a subset of tokens for a second\-stage latent refinement block\. Skipped tokens bypass this expensive path and are merged back afterward\. The resulting design concentrates the more expensive second\-stage computation on only part of the sequence rather than spending a uniform refinement budget everywhere\. Importantly, the coarse refinement stage is an architectural path, whereas the paper’s compute metric counts the refinement updates actually executed by the controller at runtime\. In the winning configuration, these are not the same thing: the initial adaptive keep decision is made token by token from cheap gate features computed from the current logits and, when available, the change from the previous step’s logits, so the controller can skip refinement before the later budgeted token\-halting stage is reached\.

We study a focused question: when extending a frozen language model with extra latent computation, is selective second\-step refinement a better use of compute than either stopping after one refinement step or forcing a second refinement step everywhere? We compare HALO against three paper\-facing baselines: the frozen backbone, a one\-step full\-sequence refinement baseline \(fixed\-1\), and a two\-step full\-sequence refinement baseline \(fixed\-2\)\.

Our results support a clear answer\. On the public comparison built from MMLU\-Pro and GPQA\-Diamond, HALO achieves the best overall average among the paper\-facing methods\. Internally, it reaches nearly the same token\-accuracy level as fixed\-2 while using fewer average applied refine steps than fixed\-1 and far fewer than fixed\-2\. In other words, HALO achieves the strongest paper\-facing performance while also using less measured controller compute than either fixed baseline\. The gain is not uniform across benchmarks—HALO’s strongest advantage comes from GPQA\-Diamond while MMLU\-Pro remains strongest for the frozen backbone—so the paper makes a focused quality–compute claim rather than a claim of universal dominance\. HALO does not win every metric or benchmark column: fixed\-2 is marginally strongest on the internal token\-accuracy metric, and the frozen backbone remains strongest on MMLU\-Pro\.

Our contributions are:

- •We introduce HALO, a hybrid adaptive latent\-refinement framework for extending frozen language models with selective second\-stage computation\.
- •We show that HALO achieves the best overall public average among the paper\-facing methods on MMLU\-Pro and GPQA\-Diamond\.
- •We show that HALO achieves the strongest paper\-facing public result while using fewer average applied refine steps than fixed\-1 and far fewer than fixed\-2\.

## 2Method

### 2\.1Problem setup

We extend a frozen pretrained language model with a lightweight trainable refinement module\. Given an input sequence, the frozen backbone produces contextual token representations and next\-token logits\. The goal is to improve prediction quality with a small amount of extra computation without paying the cost of applying the same expensive refinement uniformly to every token\. Our design target is therefore a better*quality–compute tradeoff*, not simply a larger refinement head\.

### 2\.2HALO

Figure[1](https://arxiv.org/html/2607.08775#S2.F1)shows the overall architecture\. Starting from the frozen backbone hidden states, HALO enters a coarse refinement stage\. In the winning evaluated configuration, this is an architectural refinement path rather than a universally applied first update on every token\. At each refinement step, the model computes token\-level gate features from the current logits and, when available, the change from the previous step’s logits; a small gate head converts those features into a keep probability, and a token is refined only if that keep probability exceeds threshold and its prediction is not already stable\. Because the minimum forced refinement count is zero, the controller can skip the first adaptive refinement update on some token positions\.

![Refer to caption](https://arxiv.org/html/2607.08775v1/x1.png)Figure 1:HALO architecture\.Left:overall model pipeline\. A frozen pretrained language model produces base hidden states, which are passed to HALO before final prediction\. HALO uses a coarse refinement stage together with token scoring and monotonic token halting to decide where additional computation is needed\. In the winning evaluated configuration, the first adaptive keep decision is made token by token from gate features derived from the current logits and the change from the previous step’s logits, so the coarse refinement stage denotes an architectural path rather than a universally forced first update on every token\. Selected tokens undergo latent scratchpad refinement, while bypassed tokens skip the expensive path and are merged back before producing final logits\.Right:selected\-token latent refinement block\. Selected token states are combined with a latent scratchpad state and updated through a latent refinement step to produce refined selected representations\. This design concentrates extra computation on a subset of tokens rather than forcing additional full\-sequence refinement everywhere\.HALO then performs token scoring and monotonic token halting\. In the winning configuration, this selected\-token routing is separate from the earlier first\-step keep decision\. After the coarse stage, HALO assigns a score to candidate tokens for the selected\-token path and applies a budgeted monotonic halting rule: only the top\-scoring fraction of eligible tokens, subject to a minimum token count, are routed into the expensive second stage\. The halting rule is monotonic: once a token is dropped from further refinement within a forward pass, it is not reactivated later\. Tokens that are not selected therefore bypass the expensive second stage entirely\.

The winning configuration uses a learned\-gain token score together with a budget fraction of0\.350\.35and a minimum of88selected tokens\. Operationally, the controller forms the eligible\-token set, scores those tokens, and routes only the top\-scoring budgeted fraction into the second stage while still enforcing the minimum token count\. Intuitively, this score is meant to prioritize tokens for which an additional latent update is predicted to be most useful, rather than spending the same second\-stage budget everywhere\. This implements the selective\-compute intuition behind HALO by concentrating the second\-stage budget on a subset of token positions\.

A second important design choice is train–test consistency\. Here this means that the same selected\-token routing rule used in adaptive evaluation is also applied when fixed evaluation invokes the second\-stage refinement path, rather than sending all eligible tokens through a different second\-stage regime\. In the winning configuration, the same budgeted learned\-gain token\-halting rule is applied in fixed evaluation as well\. In practice, this consistency matters because it makes the selective second\-step computation behave like a genuine mechanism rather than a training\-only regularizer\.

### 2\.3Selected\-token latent refinement

For the subset of tokens chosen by the halting mechanism, HALO maintains two representations: a selected\-token state and a latent scratchpad state\. The selected\-token block updates these jointly, then writes the refined selected\-token state back into the sequence\. The latent scratchpad acts as a small auxiliary working memory for the routed tokens, enabling a richer update than a plain one\-step residual transformation while keeping the refinement module lightweight relative to the frozen backbone\. In the implementation used here, this selected\-token refiner is a compact latent refinement block with a single latent recursion rather than a long inner recurrent rollout\.

### 2\.4Inference modes

We evaluate HALO against three paper\-facing baselines that differ only in how much additional refinement they use\.

#### Frozen backbone\.

The frozen backbone performs no extra refinement\.

#### Fixed\-1\.

The one\-step single\-state baseline applies one full\-sequence refinement step to all tokens\.

#### Fixed\-2\.

The two\-step single\-state baseline applies a second full\-sequence refinement step everywhere\.

#### HALO \(adaptive\)\.

HALO uses a coarse refinement stage together with token scoring and monotonic token halting, but allocates the latent second\-stage refinement only to selected tokens, while bypassed tokens skip that path and are merged back afterward\. In the winning adaptive configuration, the first\-step keep decision is made token by token using a gate probability computed from cheap logit\-based features together with a stability check, so the controller can skip the first adaptive refinement update on some tokens before the later selected\-token budget is applied\. The later budgeted token\-halting mechanism is a second, distinct routing rule that restricts which eligible tokens enter the selected\-token latent path\. As a result, the runtime compute can fall below one average applied refinement update per token\.

This comparison is intentionally narrow\. Fixed\-1 asks whether one uniform refinement step is enough\. Fixed\-2 asks whether simply adding more full\-sequence refinement improves results\. HALO asks whether selective latent refinement can turn a limited extra\-compute budget into a better quality–compute tradeoff than either fixed alternative\.

## 3Experimental setup

### 3\.1Model family and checkpoints

All experiments use a frozen pretrained language model as the backbone, extended with lightweight trainable refinement heads rather than full\-model finetuning\. In the training and evaluation harness, the default frozen backbone ismicrosoft/Phi\-4\-mini\-instruct\(Abdinet al\.,[2025](https://arxiv.org/html/2607.08775#bib.bib20)\)\. The winning HALO configuration uses the selected\-token latent refinement variant together with budgeted token halting, a token\-selection budget of0\.350\.35, and train–test consistency in fixed evaluation\.

We train only the refinement/controller parameters while keeping the backbone frozen\. Training uses a supervised next\-token objective on instruction\-tuning data together with auxiliary losses that shape adaptive\-compute behavior, including anytime supervision across refinement steps, controller regularization toward a target refinement budget, and consistency training across budget settings\. The winning configuration is trained in this frozen\-backbone refinement setup on a 4k\-example subset of the HuggingFaceH4 UltraChat\-200k instruction\-tuning data, a filtered derivative of UltraChat\(Dinget al\.,[2023](https://arxiv.org/html/2607.08775#bib.bib35)\), with sequence length 1024, batch size 1 with gradient accumulation of 8, one epoch, and learning rate10−410^\{\-4\}\.

For the public results, trainable methods are evaluated across six independently trained checkpoints\. The benchmark launch matrix runs the frozen backbone once, then evaluates the trainable checkpoints in fixed and adaptive modes on the benchmark suite using lm\-eval\(Bidermanet al\.,[2024](https://arxiv.org/html/2607.08775#bib.bib19)\)\. For the internal analysis, we evaluate the same family of checkpoints with a separate self\-contained internal evaluation matrix that reports quality and compute summaries from the model’s native evaluation harness\.

### 3\.2Public and internal evaluation

Our public evaluation centers on MMLU\-Pro and GPQA\-Diamond\. These are the two tasks used in the public benchmark matrix for the paper\-facing comparison, and we treat them as the main transfer test because they probe complementary forms of challenging reasoning while remaining standard enough for direct comparison across methods\. We do not claim that two benchmarks exhaustively characterize the mechanism; rather, they provide a focused but challenging public test for the narrow comparison studied in this paper\. For trainable methods, we report mean and standard deviation across six seeds; for the frozen backbone, we report a single deterministic score because the backbone itself is not retrained\. Following the main\-paper comparison, we summarize public performance using MMLU\-Pro, GPQA\-Diamond, and their simple average\. We use the average only as a compact summary statistic for the joint public picture, while keeping the benchmark\-specific columns explicit because the methods do not behave identically across the two tasks\.

In addition to the public benchmarks, we run an internal evaluation designed to expose the quality–compute tradeoff more directly than lm\-eval alone\. The internal harness reports token accuracy, token\-accuracy lift over the frozen backbone, and average applied refine steps, along with related adaptive and fixed evaluation summaries\. It evaluates each checkpoint through the model’s native fixed and adaptive evaluation modes and records controller\-side quantities such as per\-token used\-step counts, gate statistics, and prediction\-delta traces, from which the reported internal quality and compute summaries are derived\. The internal evaluation matrix runs the trained checkpoints with up to 512 evaluation samples, sequence length 1024, and batch size 1\. We focus on three internal quantities: token accuracy, token\-accuracy lift over the frozen backbone, and average applied refine steps as a compute proxy\. This metric counts the refinement updates actually executed by the controller at token positions during evaluation rather than a fixed architectural stage count\. Operationally, the model maintains a per\-token counter of executed refinement updates: a token skipped at all steps contributes0, a token refined once contributes11, and a token refined twice contributes22, and the reported quantity averages that count over predicted token positions\. Because the winning HALO configuration allows the first adaptive refinement update to be skipped on some tokens, HALO can fall below 1\.0 on this metric even though the architecture contains a coarse refinement stage\. Throughout the paper, this is the load\-bearing efficiency metric: it is the quantity on which HALO is more compute\-efficient than both fixed baselines\. We use average applied refine steps as the primary compute proxy because it is measured directly by the model’s own refinement controller and is available consistently across fixed and adaptive modes\.

### 3\.3Baselines and reporting

The paper compares four methods: the frozen backbone, single\-state fixed\-1, single\-state fixed\-2, and HALO \(adaptive\)\. These baselines are intentionally narrow: the goal is not a broad family sweep, but a focused test of whether selective second\-step latent refinement is a better use of compute than either stopping after one full\-sequence refinement step or forcing a second step everywhere\.

All table values in the paper are taken from the consolidated verified results workbook used to merge the public and internal evaluations into a single checked artifact\. The frozen backbone is reported once as a deterministic reference; the trainable public and internal rows report means and standard deviations across the six trainable seeds\. Average applied refine steps are reported as the main method\-level compute proxy\.

### 3\.4Reporting protocol

The public benchmark matrix evaluates the frozen backbone in teacher mode and the trainable checkpoints in fixed and adaptive modes\. The internal evaluation matrix analogously re\-evaluates the trained checkpoints through the model’s own evaluation path\. Overall, the experimental setup is designed to make the comparison as controlled as possible: the frozen backbone is shared, the refinement families are evaluated under matched benchmark protocols, and the main comparison isolates whether selective latent refinement provides a better quality–compute tradeoff than fixed one\-step or fixed two\-step alternatives\.

## 4Results

We evaluate HALO against three paper\-facing comparators: the frozen backbone, a one\-step single\-state baseline \(fixed\-1\), and a two\-step single\-state baseline \(fixed\-2\)\. The public\-benchmark results establish the main transfer claim, while the internal results clarify the quality–efficiency tradeoff behind that claim\. The central empirical result of the paper is that HALO achieves the strongest paper\-facing public average while also using fewer average applied refine steps than fixed\-1 and far fewer than fixed\-2\. Across these experiments, the central question is therefore not whether extra refinement always helps, but whether*selective*extra refinement yields a better quality–compute tradeoff than simply forcing additional full\-sequence refinement everywhere\.

### 4\.1Public benchmark results

Table 1:Public benchmark results on MMLU\-Pro and GPQA\-Diamond\. We report mean±\\pmstandard deviation across six seeds for trainable methods and a single deterministic score for the frozen backbone\. HALO achieves the best overall average, outperforming all paper\-facing baselines while avoiding the extra full\-sequence second step of fixed\-2\. The gains are driven primarily by GPQA\-Diamond, whereas MMLU\-Pro remains strongest for the frozen backbone\.Table[1](https://arxiv.org/html/2607.08775#S4.T1)presents the main public\-benchmark results on MMLU\-Pro and GPQA\-Diamond\. HALO achieves the best overall average, improving over the frozen backbone, the one\-step single\-state baseline, and the two\-step single\-state baseline\.

The pattern across benchmarks is asymmetric\. On MMLU\-Pro, the frozen backbone remains strongest at38\.5438\.54, while HALO reaches38\.33±0\.3738\.33\\pm 0\.37\. The public gain therefore does not come from uniform improvement across all benchmarks, and the paper should not be read as claiming that it does\. Instead, it is driven primarily by GPQA\-Diamond, where HALO reaches33\.00±0\.9933\.00\\pm 0\.99, outperforming the frozen backbone \(31\.8231\.82\), fixed\-1 \(32\.32±2\.0232\.32\\pm 2\.02\), and fixed\-2 \(32\.07±1\.6532\.07\\pm 1\.65\)\.

The comparison between fixed\-1 and fixed\-2 is also informative\. If the benefit of HALO were mainly due to using more refinement steps, then forcing a second full\-sequence refinement step everywhere should have improved the single\-state baseline\. Instead, fixed\-2 performs slightly worse than fixed\-1 on the overall public average \(35\.25±0\.7835\.25\\pm 0\.78versus35\.41±1\.0135\.41\\pm 1\.01\)\. This suggests that the key advantage is not simply “more refinement,” but rather*where*and*how*the additional computation is applied\. HALO allocates second\-step computation selectively, whereas fixed\-2 pays for an unconditional second full\-sequence pass\.

Overall, the public results support a clear conclusion: HALO provides the best paper\-facing transfer result on this focused public benchmark pair while leaving the benchmark\-level asymmetry visible rather than hiding it\.

### 4\.2Internal quality–efficiency comparison

Table 2:Internal quality and efficiency comparison for the paper\-facing methods\. We report internal token accuracy, token\-accuracy lift over the frozen backbone, and average applied refine steps across six seeds for the trainable methods\. Average applied refine steps count the refinement updates actually executed by the controller\. Among trainable methods, HALO uses fewer applied refinement updates than fixed\-1 and much less compute than fixed\-2, while fixed\-2 is only marginally higher than HALO on this internal token\-accuracy metric\.Table[2](https://arxiv.org/html/2607.08775#S4.T2)complements the public results by comparing internal quality and compute for the same paper\-facing methods\. Here the picture is more nuanced, and that nuance is useful\. The fixed\-2 baseline achieves the highest internal token accuracy at0\.7067±0\.00080\.7067\\pm 0\.0008, with HALO essentially tied at0\.7066±0\.00040\.7066\\pm 0\.0004\. The absolute difference between them is extremely small\. By contrast, the compute difference is large: fixed\-2 uses exactly2\.0002\.000average applied refine steps, whereas HALO uses only0\.776±0\.0090\.776\\pm 0\.009\.

This comparison is central to the paper’s interpretation\. On the internal metric, fixed\-2 can match or slightly exceed HALO in raw token accuracy, but it does so by always paying for a second full\-sequence refinement step\. HALO reaches nearly the same internal quality while using far less additional computation\. More strikingly, HALO also uses fewer average applied refine steps than fixed\-1\. This is possible because the reported compute metric counts executed token\-level refinement updates, not architectural stages\. In the winning HALO configuration, the first adaptive refinement update is not forced on every token: the controller computes a keep probability from cheap logit\-based features and skips tokens whose keep probability stays below threshold or whose predictions are already stable\. In other words, HALO is not best understood as maximizing the internal metric at all costs; rather, it offers a substantially more efficient way to obtain nearly the same internal quality\.

The internal results also help contextualize fixed\-1\. Relative to the frozen backbone, fixed\-1 improves token accuracy by0\.0089±0\.00120\.0089\\pm 0\.0012with exactly one average applied refine step, while fixed\-2 improves by0\.0107±0\.00080\.0107\\pm 0\.0008with two average applied refine steps\. HALO improves by0\.0106±0\.00040\.0106\\pm 0\.0004while using less than one average applied refine step\. Thus, HALO clearly exceeds the one\-step baseline in internal quality while also using less measured controller compute, while remaining far below the two\-step baseline in compute\.

Taken together, Table[2](https://arxiv.org/html/2607.08775#S4.T2)supports the paper’s core efficiency claim\. A simple second full\-sequence step can be competitive on the internal metric, but HALO reaches comparable internal quality with much lower average compute and translates that tradeoff into the strongest public average\.

### 4\.3Public quality versus compute

![Refer to caption](https://arxiv.org/html/2607.08775v1/x2.png)Figure 2:Public quality versus compute\. The y\-axis shows the average of MMLU\-Pro and GPQA\-Diamond from the verified public benchmark table, and the x\-axis shows average applied refine steps as a compute proxy, i\.e\. the average number of refinement updates actually executed by the controller\. Error bars show variation across seeds\. HALO achieves higher public\-benchmark average than the one\-step single\-state baseline while using fewer applied refinement updates than fixed\-1 and far fewer than fixed\-2, and it exhibits lower seed\-to\-seed variability than the single\-state baselines\.Figure[2](https://arxiv.org/html/2607.08775#S4.F2)summarizes the public quality–compute tradeoff visually\. The x\-axis shows average applied refine steps as a compute proxy, and the y\-axis shows the average of MMLU\-Pro and GPQA\-Diamond from Table[1](https://arxiv.org/html/2607.08775#S4.T1)\. The frozen backbone anchors the zero\-compute point\. Fixed\-1 is weaker than HALO and, under this controller\-compute metric, also uses more applied refinement than HALO\. Fixed\-2 uses substantially more compute but does not improve the public average over fixed\-1\. HALO occupies the most attractive region of the plot: it attains the best public average while lying to the left of both fixed baselines\.

This figure makes the paper’s main systems\-style point especially clear\. If the goal were only to maximize public accuracy with no regard for compute, then one might expect the best method to sit at the far right of the curve, where the most computation is spent\. Instead, the two\-step single\-state baseline shows that simply spending more full\-sequence compute is not enough\. HALO’s advantage comes from*selective*computation: it improves over fixed\-1 while using fewer applied refinement updates than fixed\-1, and without inheriting the cost profile of fixed\-2\. Because the public average is computed from only two benchmarks, the figure should be read as a focused quality–compute summary rather than a universal frontier over all tasks\.

Figure[2](https://arxiv.org/html/2607.08775#S4.F2)also reinforces the variance pattern already visible in Table[1](https://arxiv.org/html/2607.08775#S4.T1)\. The HALO point is accompanied by smaller error bars than the single\-state baselines, indicating lower seed\-to\-seed variation in the public average\.

### 4\.4Summary of the main finding

Across the public and internal evaluations, a consistent picture emerges\. The one\-step single\-state baseline is weaker than HALO; the two\-step single\-state baseline spends more compute but does not improve the public average; and HALO achieves the strongest public result while using fewer average applied refine steps than fixed\-1 and substantially less compute than fixed\-2\. This is the paper’s central result: HALO is simultaneously the best\-performing paper\-facing method on the main public average and the most compute\-efficient of the trainable paper\-facing methods under the reported controller\-compute metric\. The most important takeaway is therefore not that more refinement is always better, but that*selective latent refinement*is a more effective use of extra computation than applying additional full\-sequence refinement everywhere\.

### 4\.5Discussion and limitations

The results support a specific claim about*how*extra computation should be used in this setting\. HALO is not strongest because it wins every metric or every benchmark column\. Rather, it delivers the best overall public transfer result while using substantially less compute than the unconditional two\-step baseline\. The public improvement is asymmetric: HALO’s gain is driven primarily by GPQA\-Diamond, while MMLU\-Pro remains strongest for the frozen backbone\.

The internal results sharpen the same point\. Fixed\-2 is marginally stronger than HALO on raw internal token accuracy, but only by a very small amount and at much higher compute\. HALO therefore should not be read as maximizing the internal metric at all costs; its value lies in reaching nearly the same internal quality with far fewer average applied refine steps and converting that efficiency advantage into the strongest public average\.

This also clarifies one of the most important negative results in the paper\. If HALO’s gain came primarily from simply adding another refinement step, then fixed\-2 should have been a stronger public baseline than fixed\-1\. Instead, fixed\-2 spends more computation than fixed\-1 by applying a second full\-sequence refinement step everywhere, yet it does not improve the public average\. The public results therefore suggest that indiscriminately spending more full\-sequence compute is not enough\. What matters is whether the additional computation is targeted\.

The internal and public results should not be treated as interchangeable\. The internal metric exposes quality and compute with fine granularity, but the public benchmarks are load\-bearing because they test whether the refinement mechanism transfers beyond the model’s native harness\.

Our evidence also has clear limits\. The public story is built on two benchmarks, the overall win is modest, and the main comparison is intentionally narrow\. The benchmark\-level gain is also asymmetric, with the strongest advantage coming from GPQA\-Diamond rather than from a uniform improvement across both public tasks\. We therefore present HALO as evidence that selective latent refinement can improve the quality–compute tradeoff of frozen language models, not as evidence that adaptive reasoning for frozen language models is solved\.

A broader implication of HALO is that better quality–compute efficiency could reduce the deployment cost of language\-model reasoning for beneficial applications\. At the same time, improved inference efficiency may also make capable generative systems cheaper to run and easier to scale, which could amplify misuse risks already associated with large language models\. We therefore view HALO as a methods contribution whose societal effects depend on the downstream deployment context of the underlying pretrained model\.

These limitations point naturally to future directions\. One next step is to test whether the same selective refinement mechanism holds up across a broader benchmark set, including tasks with different mixtures of recall, structured reasoning, and long\-context dependence\. Another is to measure efficiency more directly, for example through wall\-clock latency or throughput in addition to average applied refine steps\. A third is to improve the refinement policy itself, since the present results already suggest that where computation is allocated matters more than simply increasing it\.

Overall, we view HALO as evidence for a particular design principle\. When extending a frozen pretrained language model with extra reasoning compute, the key question is not just whether additional refinement helps, but whether that refinement is allocated selectively enough to justify its cost\. Our results suggest that the answer is yes: selective latent refinement can outperform a simpler one\-step baseline while also using fewer applied refinement updates, and while avoiding the public inefficiency of a fully applied second\-step baseline\. That is a narrower claim than universal superiority, but it is also the one most directly supported by the evidence in this paper\.

## 5Related work

HALO is related to several lines of work on adaptive computation, recurrent refinement, token\-level halting, and frozen\-model extension\.

#### Adaptive computation\.

Adaptive Computation Time, Universal Transformers, and PonderNet study how neural sequence models can allocate variable amounts of computation rather than using a fixed budget everywhere\(Graves,[2016](https://arxiv.org/html/2607.08775#bib.bib1); Dehghaniet al\.,[2019](https://arxiv.org/html/2607.08775#bib.bib2); Baninoet al\.,[2021](https://arxiv.org/html/2607.08775#bib.bib4)\)\. HALO belongs to this broad tradition, but targets frozen language\-model extension rather than end\-to\-end recurrent sequence modeling\.

#### Recurrent and latent refinement\.

Recent recurrent\-depth and latent\-reasoning language models argue that repeated latent updates can provide a form of inference\-time scaling distinct from explicit chain\-of\-thought generation\(Geipinget al\.,[2025](https://arxiv.org/html/2607.08775#bib.bib9); Haoet al\.,[2024](https://arxiv.org/html/2607.08775#bib.bib8); Saunshiet al\.,[2025](https://arxiv.org/html/2607.08775#bib.bib10); Zenget al\.,[2025](https://arxiv.org/html/2607.08775#bib.bib12)\)\. Closely related small\-model work such as TRM also studies recursive latent refinement through repeated latent\-state and answer\-state updates\(Jolicoeur\-Martineau,[2025](https://arxiv.org/html/2607.08775#bib.bib16)\)\. HALO is related in spirit to this line of work, but differs in focusing on a lightweight refinement head on top of a frozen pretrained backbone rather than recurrently training the full model\.

#### Selective computation and token\-level halting\.

Mixture\-of\-Depths, early\-exit methods, AdaPonderLM, and ANIRA study token\-level adaptive depth or halting in Transformers and language models\(Raposoet al\.,[2024](https://arxiv.org/html/2607.08775#bib.bib5); Elhoushiet al\.,[2024](https://arxiv.org/html/2607.08775#bib.bib6); Shanet al\.,[2024](https://arxiv.org/html/2607.08775#bib.bib7); Songet al\.,[2026](https://arxiv.org/html/2607.08775#bib.bib13); Moosaet al\.,[2026](https://arxiv.org/html/2607.08775#bib.bib14)\)\. HALO is most directly aligned with this line of work: its core mechanism is budgeted monotonic token halting that routes only a selected subset of tokens into a second\-stage latent refinement block\.

#### Frozen\-model refinement\.

HALO is also connected to methods that improve a pretrained language model by adding an auxiliary refinement or self\-improvement stage at inference time, including output\-space self\-refinement and latent\-space reasoning\(Madaanet al\.,[2023](https://arxiv.org/html/2607.08775#bib.bib18); Haoet al\.,[2024](https://arxiv.org/html/2607.08775#bib.bib8); Geipinget al\.,[2025](https://arxiv.org/html/2607.08775#bib.bib9); Zenget al\.,[2025](https://arxiv.org/html/2607.08775#bib.bib12)\)\. HALO differs in its emphasis on*selective*hidden\-state refinement over a frozen backbone\.

Overall, HALO sits at the intersection of these threads: learned halting, latent hidden\-state refinement, and frozen\-model extension with a small trainable module\. Our empirical focus is narrower than most related work: a controlled comparison showing that selective second\-step latent refinement is a better use of extra compute than unconditional additional full\-sequence refinement in the frozen\-backbone setting studied here\.

## 6Conclusion

We introduced HALO, a hybrid adaptive latent\-refinement method for extending a frozen pretrained language model with selective extra computation\. HALO combines a coarse refinement stage with a selected\-token latent refinement stage, allowing the model to spend additional computation only where it appears most useful rather than forcing extra full\-sequence refinement everywhere\.

Across the public benchmarks used in our primary comparison, HALO achieves the best overall average among the paper\-facing methods, outperforming the frozen backbone, fixed\-1, and fixed\-2 on this focused two\-benchmark comparison\.

The internal analysis clarifies this tradeoff further\. HALO reaches nearly the same internal token\-accuracy level as the two\-step baseline while using fewer average applied refine steps than the one\-step baseline and far fewer than the two\-step baseline\. Taken together, these results suggest that the key advantage is not merely more refinement, but better allocation of refinement\. More broadly, HALO provides focused evidence that selective latent refinement can improve the quality–compute tradeoff of frozen language models\.

## Acknowledgments

I thank Nathan Rigoni for his support of this project and for giving me the space to pursue this research and develop it into a paper\.

## References

- M\. Abdin, J\. Aneja, H\. Awadalla, A\. Awan, N\. Bach, A\. Bahree, A\. Bakhtiari, H\. Behl, A\. Benhaim, M\. Bilenko,et al\.\(2025\)Phi\-4\-mini technical report: compact yet powerful multimodal language models via mixture\-of\-loras\.arXiv preprint arXiv:2503\.01743\.External Links:[Link](https://arxiv.org/abs/2503.01743)Cited by:[§B\.2](https://arxiv.org/html/2607.08775#A2.SS2.p3.1),[§3\.1](https://arxiv.org/html/2607.08775#S3.SS1.p1.1)\.
- M\. Abdinet al\.\(2024\)Phi\-4 technical report\.arXiv preprint arXiv:2412\.08905\.External Links:[Link](https://arxiv.org/abs/2412.08905)Cited by:[§B\.2](https://arxiv.org/html/2607.08775#A2.SS2.p3.1)\.
- S\. Bae, Y\. Kim, R\. Bayat, S\. Kim, J\. Ha, T\. Schuster, A\. Fisch, H\. Harutyunyan, Z\. Ji, A\. Courville, and S\. Yun \(2025\)Mixture\-of\-recursions: learning dynamic recursive depths for adaptive token\-level computation\.arXiv preprint arXiv:2507\.10524\.External Links:[Link](https://arxiv.org/abs/2507.10524)Cited by:[§B\.2](https://arxiv.org/html/2607.08775#A2.SS2.p3.1)\.
- A\. Banino, J\. Balaguer, P\. Bojanowski, C\. Bunne, A\. Schwarzschild, E\. Lockhart, M\. Engelcke, A\. Ibrahim, C\. Clopath, S\. Denève, T\. George,et al\.\(2021\)PonderNet: learning to ponder\.International Conference on Learning Representations\.External Links:[Link](https://arxiv.org/abs/2107.05407)Cited by:[§5](https://arxiv.org/html/2607.08775#S5.SS0.SSS0.Px1.p1.1)\.
- S\. Biderman, H\. Schoelkopf, L\. Sutawika, L\. Gao, J\. Tow, B\. Abbasi, A\. F\. Aji, P\. S\. Ammanamanchi, S\. Black, J\. Clive, A\. DiPofi, J\. Etxaniz, B\. Fattori, J\. Z\. Forde, C\. Foster, J\. Hsu, M\. Jaiswal, W\. Y\. Lee, H\. Li, C\. Lovering, N\. Muennighoff, E\. Pavlick, J\. Phang, A\. Skowron, S\. Tan, X\. Tang, K\. A\. Wang, G\. I\. Winata, F\. Yvon, and A\. Zou \(2024\)Lessons from the trenches on reproducible evaluation of language models\.arXiv preprint arXiv:2405\.14782\.External Links:[Link](https://arxiv.org/abs/2405.14782)Cited by:[§B\.2](https://arxiv.org/html/2607.08775#A2.SS2.p3.1),[§3\.1](https://arxiv.org/html/2607.08775#S3.SS1.p3.1)\.
- Y\. Chen, J\. Shang, Z\. Zhang, Y\. Xie, J\. Sheng, T\. Liu, S\. Wang, Y\. Sun, H\. Wu, and H\. Wang \(2025\)Inner thinking transformer: leveraging dynamic depth scaling to foster adaptive internal thinking\.arXiv preprint arXiv:2502\.13842\.External Links:[Link](https://arxiv.org/abs/2502.13842)Cited by:[§B\.2](https://arxiv.org/html/2607.08775#A2.SS2.p3.1)\.
- M\. Dehghani, S\. Gouws, O\. Vinyals, J\. Uszkoreit, and Ł\. Kaiser \(2019\)Universal transformers\.International Conference on Learning Representations\.External Links:[Link](https://arxiv.org/abs/1807.03819)Cited by:[§5](https://arxiv.org/html/2607.08775#S5.SS0.SSS0.Px1.p1.1)\.
- N\. Ding, Y\. Chen, B\. Xu, Y\. Qin, Z\. Zheng, S\. Hu, Z\. Liu, M\. Sun, and B\. Zhou \(2023\)Enhancing chat language models by scaling high\-quality instructional conversations\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 3028–3054\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.183/)Cited by:[§A\.2](https://arxiv.org/html/2607.08775#A1.SS2.p1.1),[§3\.1](https://arxiv.org/html/2607.08775#S3.SS1.p2.1)\.
- M\. Elhoushi, A\. Shrivastava, D\. Liskovich, B\. Hosmer, B\. Wasti, L\. Lai, A\. Mahmoud, B\. Acun, S\. Agarwal, A\. Roman, A\. A\. Aly, B\. Chen, and C\. Wu \(2024\)LayerSkip: enabling early exit inference and self\-speculative decoding\.Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics\.External Links:[Link](https://arxiv.org/abs/2404.16710)Cited by:[§5](https://arxiv.org/html/2607.08775#S5.SS0.SSS0.Px3.p1.1)\.
- J\. Geiping, S\. McLeish, N\. Jain, J\. Kirchenbauer, S\. Singh, B\. R\. Bartoldson, B\. Kailkhura, A\. Bhatele, and T\. Goldstein \(2025\)Scaling up test\-time compute with latent reasoning: a recurrent depth approach\.arXiv preprint arXiv:2502\.05171\.External Links:[Link](https://arxiv.org/abs/2502.05171)Cited by:[§5](https://arxiv.org/html/2607.08775#S5.SS0.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.08775#S5.SS0.SSS0.Px4.p1.1)\.
- S\. Goyal, Z\. Ji, A\. S\. Rawat, A\. K\. Menon, S\. Kumar, and V\. Nagarajan \(2023\)Think before you speak: training language models with pause tokens\.arXiv preprint arXiv:2310\.02226\.External Links:[Link](https://arxiv.org/abs/2310.02226)Cited by:[§B\.2](https://arxiv.org/html/2607.08775#A2.SS2.p3.1)\.
- A\. Graves \(2016\)Adaptive computation time for recurrent neural networks\.arXiv preprint arXiv:1603\.08983\.External Links:[Link](https://arxiv.org/abs/1603.08983)Cited by:[§5](https://arxiv.org/html/2607.08775#S5.SS0.SSS0.Px1.p1.1)\.
- S\. Hao, S\. Sukhbaatar, D\. Su, X\. Li, Z\. Hu, J\. Weston, and Y\. Tian \(2024\)Training large language models to reason in a continuous latent space\.arXiv preprint arXiv:2412\.06769\.External Links:[Link](https://arxiv.org/abs/2412.06769)Cited by:[§5](https://arxiv.org/html/2607.08775#S5.SS0.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.08775#S5.SS0.SSS0.Px4.p1.1)\.
- D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. Steinhardt \(2021\)Measuring massive multitask language understanding\.International Conference on Learning Representations\.External Links:[Link](https://arxiv.org/abs/2009.03300)Cited by:[§B\.2](https://arxiv.org/html/2607.08775#A2.SS2.p3.1)\.
- A\. Jeddi, M\. Ciccone, and B\. Taati \(2026\)LoopFormer: elastic\-depth looped transformers for latent reasoning via shortcut modulation\.arXiv preprint arXiv:2602\.11451\.External Links:[Link](https://arxiv.org/abs/2602.11451)Cited by:[§B\.2](https://arxiv.org/html/2607.08775#A2.SS2.p3.1)\.
- A\. Jolicoeur\-Martineau \(2025\)Less is more: recursive reasoning with tiny networks\.arXiv preprint arXiv:2510\.04871\.External Links:[Link](https://arxiv.org/abs/2510.04871)Cited by:[§5](https://arxiv.org/html/2607.08775#S5.SS0.SSS0.Px2.p1.1)\.
- A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang, S\. Gupta, B\. Majumder, K\. Hermann, S\. Welleck, A\. Yazdanbakhsh, and P\. Clark \(2023\)Self\-refine: iterative refinement with self\-feedback\.Advances in Neural Information Processing Systems\.External Links:[Link](https://arxiv.org/abs/2303.17651)Cited by:[§5](https://arxiv.org/html/2607.08775#S5.SS0.SSS0.Px4.p1.1)\.
- I\. M\. Moosa, S\. Lohit, Y\. Wang, M\. Chatterjee, and W\. Yin \(2026\)Understanding dynamic compute allocation in recurrent transformers\.arXiv preprint arXiv:2602\.08864\.External Links:[Link](https://arxiv.org/abs/2602.08864)Cited by:[§5](https://arxiv.org/html/2607.08775#S5.SS0.SSS0.Px3.p1.1)\.
- D\. Raposo, S\. Ritter, B\. Richards, T\. Lillicrap, P\. C\. Humphreys, and A\. Santoro \(2024\)Mixture\-of\-depths: dynamically allocating compute in transformer\-based language models\.arXiv preprint arXiv:2404\.02258\.External Links:[Link](https://arxiv.org/abs/2404.02258)Cited by:[§5](https://arxiv.org/html/2607.08775#S5.SS0.SSS0.Px3.p1.1)\.
- P\. Rauba, C\. Fanconi, and M\. van der Schaar \(2026\)Tiny autoregressive recursive models\.arXiv preprint arXiv:2603\.08082\.External Links:[Link](https://arxiv.org/abs/2603.08082)Cited by:[§B\.2](https://arxiv.org/html/2607.08775#A2.SS2.p3.1)\.
- D\. Rein, B\. L\. Hou, A\. C\. Stickland, J\. Petty, R\. Y\. Pang, J\. Dirani, J\. Michael, and S\. R\. Bowman \(2023\)GPQA: a graduate\-level google\-proof q&a benchmark\.arXiv preprint arXiv:2311\.12022\.External Links:[Link](https://arxiv.org/abs/2311.12022)Cited by:[§B\.2](https://arxiv.org/html/2607.08775#A2.SS2.p3.1)\.
- N\. Saunshi, N\. Dikkala, Z\. Li, S\. Kumar, and S\. J\. Reddi \(2025\)Reasoning with latent thoughts: on the power of looped transformers\.arXiv preprint arXiv:2502\.17416\.External Links:[Link](https://arxiv.org/abs/2502.17416)Cited by:[§5](https://arxiv.org/html/2607.08775#S5.SS0.SSS0.Px2.p1.1)\.
- W\. Shan, L\. Meng, T\. Zheng, Y\. Luo, B\. Li, J\. Wang, T\. Xiao, and J\. Zhu \(2024\)Early exit is a natural capability in transformer\-based models: an empirical study on early exit without joint optimization\.arXiv preprint arXiv:2412\.01455\.External Links:[Link](https://arxiv.org/abs/2412.01455)Cited by:[§5](https://arxiv.org/html/2607.08775#S5.SS0.SSS0.Px3.p1.1)\.
- D\. Sheshanarayana, R\. S\. Pal, M\. Sinha, and T\. Dasgupta \(2026\)Thinking in latents: adaptive anchor refinement for implicit reasoning in llms\.arXiv preprint arXiv:2603\.15051\.External Links:[Link](https://arxiv.org/abs/2603.15051)Cited by:[§B\.2](https://arxiv.org/html/2607.08775#A2.SS2.p3.1)\.
- C\. Snell, J\. Lee, K\. Xu, and A\. Kumar \(2024\)Scaling llm test\-time compute optimally can be more effective than scaling model parameters\.arXiv preprint arXiv:2408\.03314\.External Links:[Link](https://arxiv.org/abs/2408.03314)Cited by:[§B\.2](https://arxiv.org/html/2607.08775#A2.SS2.p3.1)\.
- S\. Song, H\. Li, Z\. Wang, B\. Zeng, F\. Song, Y\. Wang, Z\. J\. Xu, Z\. He, and Z\. Lin \(2026\)AdaPonderLM: gated pondering language models with token\-wise adaptive depth\.arXiv preprint arXiv:2603\.01914\.External Links:[Link](https://arxiv.org/abs/2603.01914)Cited by:[§5](https://arxiv.org/html/2607.08775#S5.SS0.SSS0.Px3.p1.1)\.
- Y\. Wang, X\. Ma, G\. Zhang, Y\. Ni, A\. Chandra, S\. Guo, W\. Ren, A\. Arulraj, X\. He, Z\. Jiang, T\. Li, M\. Ku, K\. Wang, A\. Zhuang, R\. Fan, X\. Yue, and W\. Chen \(2024\)MMLU\-pro: a more robust and challenging multi\-task language understanding benchmark\.arXiv preprint arXiv:2406\.01574\.External Links:[Link](https://arxiv.org/abs/2406.01574)Cited by:[§B\.2](https://arxiv.org/html/2607.08775#A2.SS2.p3.1)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. H\. Chi, Q\. V\. Le, and D\. Zhou \(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.Advances in Neural Information Processing Systems\.External Links:[Link](https://arxiv.org/abs/2201.11903)Cited by:[§B\.2](https://arxiv.org/html/2607.08775#A2.SS2.p3.1)\.
- B\. Zeng, Y\. Hao, H\. Li, S\. Song, F\. Song, Z\. Wang, S\. Huang, Y\. Xu, Z\. He, X\. Wang, and Z\. Lin \(2026\)Pretraining with token\-level adaptive latent chain\-of\-thought\.arXiv preprint arXiv:2602\.08220\.External Links:[Link](https://arxiv.org/abs/2602.08220)Cited by:[§B\.2](https://arxiv.org/html/2607.08775#A2.SS2.p3.1)\.
- B\. Zeng, H\. Li, Z\. Wang, F\. Song, Y\. Wang, Z\. J\. Xu, Z\. He, Z\. Lin,et al\.\(2025\)PonderLM: pretraining language models to ponder in continuous space\.arXiv preprint arXiv:2505\.20674\.External Links:[Link](https://arxiv.org/abs/2505.20674)Cited by:[§5](https://arxiv.org/html/2607.08775#S5.SS0.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.08775#S5.SS0.SSS0.Px4.p1.1)\.

## Appendix AAdditional experimental details

### A\.1Winning HALO configuration

The appendix is intended to support auditability of the paper’s narrow comparison rather than to introduce a broader family sweep\. Accordingly, we report only the exact paper\-facing adaptive configuration and the corresponding fixed baselines\.

The winning HALO configuration used in the paper \(internally labeled v139\) uses selected\-token latent refinement together with budgeted token halting, a token\-selection budget of0\.350\.35, a minimum of88selected tokens, and train–test consistency in fixed evaluation\. It also uses multibudget consistency training and a learned token\-scoring rule for selecting which tokens receive additional latent refinement\. Operationally, the selected\-token controller scores eligible tokens, keeps only the top\-scoring budgeted fraction subject to the minimum token count, and routes only those tokens into the selected\-token latent block\. In this configuration, monotonic token halting means that once a token is dropped from further selected\-token refinement within a forward pass, it is not reactivated later\. Train–test consistency means that this same budgeted learned\-gain routing rule is also applied when fixed evaluation invokes the second\-stage path\.

The paper\-facing fixed baselines are intentionally simple\. The frozen backbone performs no refinement\. The fixed\-1 baseline applies one full\-sequence single\-state refinement step\. The fixed\-2 baseline applies two full\-sequence single\-state refinement steps\. These rows are included because they answer the paper’s central question directly: whether selective second\-stage latent refinement is a better use of limited extra computation than either stopping after one refinement step or forcing a second full\-sequence refinement step everywhere\.

### A\.2Training recipe

The winning HALO checkpoints are trained in the harness’s frozen\-backbone refinement setting, which freezes the pretrained backbone and optimizes only the refinement/controller module\. The training data path is an instruction\-tuning chat setup using the HuggingFaceH4 UltraChat\-200k data, a filtered derivative of UltraChat\(Dinget al\.,[2023](https://arxiv.org/html/2607.08775#bib.bib35)\), and the winning run uses 4,000 training examples, 256 held\-out evaluation examples during training, sequence length 1024, batch size 1 with gradient accumulation of 8, one epoch, BF16, activation checkpointing, and learning rate10−410^\{\-4\}\.

The training objective is multi\-term rather than plain next\-token cross\-entropy alone\. In addition to supervised next\-token learning, the winning setup includes anytime supervision across refinement steps, regularization toward a target adaptive\-compute budget, gate warm\-start/oracle\-style controller supervision, and multibudget consistency training\. These ingredients matter because HALO’s controller behavior is learned rather than hand\-specified: they encourage the refinement head to improve predictions while also learning when to spend or withhold extra computation\.

### A\.3Public benchmark protocol

The public benchmark comparison is built from MMLU\-Pro and GPQA\-Diamond\. In the public lm\-eval matrix, the frozen backbone is evaluated once in teacher mode, and the trainable checkpoints are evaluated in fixed and adaptive modes\. The public matrix explicitly lists MMLU\-Pro and GPQA\-Diamond as the paper\-facing tasks, and the main paper reports the frozen backbone, fixed\-1, fixed\-2, and HALO rows derived from the consolidated verified workbook\.

For trainable methods, public results are aggregated across six independently trained checkpoints with seeds 1234, 2234, 3234, 4234, 5234, and 6234\. The frozen backbone is deterministic and is therefore reported once\. In the paper, we summarize public performance with MMLU\-Pro, GPQA\-Diamond, and their simple average\. Table[3](https://arxiv.org/html/2607.08775#A2.T3)reports the underlying per\-seed values used to form Table[1](https://arxiv.org/html/2607.08775#S4.T1)\.

### A\.4Internal evaluation protocol

The internal evaluation uses the model’s native evaluation path\. The self\-contained internal evaluation matrix re\-evaluates the paper\-facing fixed\-baseline and HALO checkpoints across the same six seeds used in the public table\. Each call uses up to 512 evaluation samples, sequence length 1024, and batch size 1\. As in the main text, we focus on token accuracy, token\-accuracy lift over the frozen backbone, and average applied refine steps as the primary method\-level compute proxy\.

Average applied refine steps count the refinement updates actually executed by the controller rather than a fixed architectural stage count\. This is the quantity used in the paper’s quality–compute analysis and in the appendix quality–compute scatter\.

### A\.5Compute resources and external assets

Training and evaluation runs used a single NVIDIA A100 GPU per run; multiple runs were sometimes executed in parallel, but each individual model was trained and evaluated on one A100\. The experiments use the MIT\-licensed Phi\-4\-mini\-instruct model, the MIT\-licensed lm\-evaluation\-harness, the Apache\-2\.0\-licensed MMLU\-Pro repository, GPQA assets released under CC BY 4\.0 for the dataset card and MIT for the code repository, and the HuggingFaceH4 UltraChat\-200k training data released under the OpenRAIL license\.

## Appendix BPer\-seed results

### B\.1Per\-seed public benchmark results

Table[3](https://arxiv.org/html/2607.08775#A2.T3)reports the per\-seed public benchmark values underlying the aggregated results in the main public table\.

Table 3:Per\-seed public benchmark results for the paper\-facing methods\. Values are percentages\. The frozen backbone is deterministic and is reported once\. This table is the appendix companion to Table[1](https://arxiv.org/html/2607.08775#S4.T1): it makes the reported means and standard deviations directly auditable\.Figure[3](https://arxiv.org/html/2607.08775#A2.F3)provides a compact visual view of the per\-seed public averages reported in Table[3](https://arxiv.org/html/2607.08775#A2.T3)\. Across these six seeds, every HALO checkpoint remains above the frozen\-backbone public average, whereas both fixed baselines include seeds that fall below the frozen\-backbone reference\. This is consistent with the smaller public\-average standard deviation reported for HALO in Table[1](https://arxiv.org/html/2607.08775#S4.T1)and suggests a more stable public transfer gain for HALO on this benchmark pair, but not that the effect would necessarily continue to grow with more training or a larger backbone\.

![Refer to caption](https://arxiv.org/html/2607.08775v1/x3.png)Figure 3:Seed\-level public benchmark averages for the paper\-facing methods\. Each point corresponds to one seed for a trainable method, and the frozen backbone is shown as a reference\. This plot is the visual companion to Table[3](https://arxiv.org/html/2607.08775#A2.T3)and makes the public variance pattern easier to inspect\.
### B\.2Per\-seed internal quality and efficiency results

Table[4](https://arxiv.org/html/2607.08775#A2.T4)reports the per\-seed internal quality and efficiency values underlying the aggregated results in Table[2](https://arxiv.org/html/2607.08775#S4.T2)\.

Table 4:Per\-seed internal quality and efficiency results for the paper\-facing methods\. Average applied refine steps count the refinement updates actually executed by the controller\. The frozen backbone is deterministic and is reported once\. This table is the appendix companion to Table[2](https://arxiv.org/html/2607.08775#S4.T2)\.Figure[4](https://arxiv.org/html/2607.08775#A2.F4)gives the corresponding seed\-level internal quality–compute view\.

![Refer to caption](https://arxiv.org/html/2607.08775v1/x4.png)Figure 4:Seed\-level internal quality versus compute\. Each point corresponds to one seed, the x\-axis shows average applied refine steps, and the y\-axis shows internal token accuracy\. The frozen backbone is shown as a reference\. This plot is the seed\-level appendix analogue of the paper’s compute story: HALO clusters near the fixed\-2 token\-accuracy region while using fewer applied refinement updates than fixed\-1 and much fewer than fixed\-2\.