The Hard Decision Layer: Evidence for Committed Inference in Transformers
Summary
This paper identifies the Hard Decision Layer (HDL) in transformer language models, where answer option rankings stabilize abruptly during inference, and shows it is invariant to fine-tuning and consistent across models and datasets.
arXiv:2607.21613v1 Announce Type: cross
Abstract: We investigate where and how transformer-based language models commit to predictions in multiple-choice question answering. We identify the _Hard Decision Layer_ (HDL), a natural architectural property where answer option rankings stabilize abruptly during inference. Empirical validation across four language models (Qwen, Llama, Granite, Mistral) and four benchmark datasets demonstrates consistent HDL emergence without learned routing policies. We also show that the HDL is invariant to fine-tuning. Our results reveal striking accuracy improvements at the HDL: up to +0.61 (Qwen on CommonsenseQA), after which performance stabilizes. Systematic ablations on label formats and problem complexity confirm the phenomenon is fundamental to model architecture. These findings offer mechanistic insights into transformer inference and suggest opportunities for efficient reasoning and model steering. All code and results required to reproduce this work are available in https://github.com/Mystic-Slice/hard-decision-layer
View Cached Full Text
Cached at: 07/27/26, 07:41 AM
# The Hard Decision Layer: Evidence for Committed Inference in Transformers
Source: [https://arxiv.org/html/2607.21613](https://arxiv.org/html/2607.21613)
Ashwath Vaithinathan Aravindan University of Southern California Los Angeles, 90007, California, United States of America vaithina@usc\.edu &Mayank Kejriwal Information Sciences Institute 4676 Admiralty Way \#1001, Los Angeles, 90292, California, United States of America kejriwal@isi\.edu
###### Abstract
We investigate where and how transformer\-based language models commit to predictions in multiple\-choice question answering\. We identify the*Hard Decision Layer*\(HDL\), a natural architectural property where answer option rankings stabilize abruptly during inference\. Empirical validation across four language models \(Qwen, Llama, Granite, Mistral\) and four benchmark datasets demonstrates consistent HDL emergence without learned routing policies\. We also show that the HDL is invariant to fine\-tuning\. Our results reveal striking accuracy improvements at the HDL: up to \+0\.61 \(Qwen on CommonsenseQA\), after which performance stabilizes\. Systematic ablations on label formats and problem complexity confirm the phenomenon is fundamental to model architecture\. These findings offer mechanistic insights into transformer inference and suggest opportunities for efficient reasoning and model steering\. All code and results required to reproduce this work are available in[https://github\.com/Mystic\-Slice/hard\-decision\-layer](https://github.com/Mystic-Slice/hard-decision-layer)
The Hard Decision Layer: Evidence for Committed Inference in Transformers
Ashwath Vaithinathan AravindanUniversity of Southern CaliforniaLos Angeles, 90007, California,United States of Americavaithina@usc\.eduMayank KejriwalInformation Sciences Institute4676 Admiralty Way \#1001,Los Angeles, 90292, California,United States of Americakejriwal@isi\.edu
Figure 1:Average token rankings for answer option tokens across transformer layers of the Qwen model\. Each line represents one of the four answer options, with average rankings computed at each layer over all questions in the QASC dataset\. The faded regions surrounding each ranking line indicate the standard deviation of rankings across the dataset\. Per our definition, thehard decision layer\(HDL\) is identified at Layer 25\. Following this layer, the average rankings are stabilized all the way until the last layer\.## 1Introduction
Understanding where and how transformer\-based language models commit to their predictions is fundamental to both mechanistic interpretabilityBereska and Gavves \([2024](https://arxiv.org/html/2607.21613#bib.bib29)\)and computational efficiency\. Recent work shows that models allocate computational depth dynamically: information flows through the residual streamElhageet al\.\([2021](https://arxiv.org/html/2607.21613#bib.bib10)\)and is iteratively refined by attention and feed\-forward networksGevaet al\.\([2022](https://arxiv.org/html/2607.21613#bib.bib27)\), with early layers performing vocabulary filtering while later layers refine candidatesGuptaet al\.\([2025](https://arxiv.org/html/2607.21613#bib.bib3)\)\. A key insight is that model predictions stabilize mid\-networkLioubashevskiet al\.\([2024](https://arxiv.org/html/2607.21613#bib.bib28)\), motivating research into when and how models make their final decisions\.
Existing approaches to understanding layer\-wise decision\-making fall into two categories: dynamic methods that identify when to exit early through learned routing policiesDinet al\.\([2024](https://arxiv.org/html/2607.21613#bib.bib22)\), and static analyses that examine post\-hoc layer pruningFanet al\.\([2024](https://arxiv.org/html/2607.21613#bib.bib26)\)\. However, neither characterizes how models inherently organize their reasoning\. What is missing is an understanding of whether there exists a natural architectural property that governs when predictions stabilize, independent of learned routing mechanisms or post\-hoc modifications\.
We investigate this question in the context of multiple\-choice question answering \(MCQA\)Hendryckset al\.\([2021](https://arxiv.org/html/2607.21613#bib.bib30)\), a controlled setting that isolates discrete reasoning from open\-ended generation\. We discover that a fixedHard Decision Layer\(HDL\) emerges in transformer models, beyond which the relative ranking of answer options remains stable\. The HDL represents a natural, static property of how models organize their reasoning, consistent across diverse datasets and model architectures\.
We make the following key contributions: \(i\) we identify and characterize theHard Decision Layer \(HDL\), a natural architectural property in transformer models where answer option rankings stabilize during multiple\-choice reasoning, emerging without learned routing policies\. \(ii\) Through a comprehensive experimental study, we demonstrate that HDL is invariant to model fine\-tuning \(via LoRA\) and robust to the number of options, confirming it reflects an intrinsic architectural property rather than a training artifact\. \(iii\) Through systematic evaluation across four language models and four datasets, we also show that HDL prominence depends critically on option label representation, with alphabetic and arabic numeral labels producing sharper decision boundaries than roman numerals\.
## 2Related Work
Understanding how LLMs process and generate predictions is fundamental to both mechanistic interpretability and computational efficiency\. Our work bridges these two areas by examining layer\-wise decision\-making in transformers, specifically through the lens of multiple\-choice question answering \(MCQA\)\.
### 2\.1Mechanistic Interpretability
Mechanistic interpretability seeks to understand the internal computations underlying transformer predictionsBereska and Gavves \([2024](https://arxiv.org/html/2607.21613#bib.bib29)\)\. Information flows through theresidual streamElhageet al\.\([2021](https://arxiv.org/html/2607.21613#bib.bib10)\), iteratively refined by attention and feed\-forward networks \(FFN\) across layers\. Projection\-based techniques like thelogit lensnostalgebraist \([2020](https://arxiv.org/html/2607.21613#bib.bib17)\)andtuned lensBelroseet al\.\([2023](https://arxiv.org/html/2607.21613#bib.bib23)\)enable “peeking” at intermediate predictions\. Complementary approaches such as activation patching and causal tracingMenget al\.\([2022](https://arxiv.org/html/2607.21613#bib.bib21)\)identify causal components and factual associationsHewitt and Manning \([2019](https://arxiv.org/html/2607.21613#bib.bib13)\); Havivet al\.\([2023](https://arxiv.org/html/2607.21613#bib.bib18)\); Daret al\.\([2023](https://arxiv.org/html/2607.21613#bib.bib19)\)\.
In transformers, feed\-forward layers function as key\-value memoriesGevaet al\.\([2022](https://arxiv.org/html/2607.21613#bib.bib27),[2021](https://arxiv.org/html/2607.21613#bib.bib20)\), promoting concepts to affect outputs\. Early layers perform vocabulary filtering while later layers refine candidatesGuptaet al\.\([2025](https://arxiv.org/html/2607.21613#bib.bib3)\)\. Notably, models allocate computational depth dynamically: tokens at the start of factual sequences use more layers for decision\-making, while those naturally following use fewer layers, suggesting that models self\-organize reasoning based on task context\.
### 2\.2Token Saturation Phenomenon
Model predictions stabilize mid\-network, motivating early exit research\.Dinet al\.\([2024](https://arxiv.org/html/2607.21613#bib.bib22)\)use linear transformations to shortcut intermediate computation, achieving 13\.8% layer savings on GPT\-2, though this requires dynamic decisions at each layer\. More structured approaches show that token ranking saturation follows a predictable order across architecturesLioubashevskiet al\.\([2024](https://arxiv.org/html/2607.21613#bib.bib28)\), suggesting principled decision progression and natural boundaries\.Fanet al\.\([2024](https://arxiv.org/html/2607.21613#bib.bib26)\)further demonstrate that not all layers are equally essential, achieving layer pruning ratios up to 17\.8% in large models like Llama2 and OPT\.
### 2\.3Multiple\-Choice Question Answering and Text Classification using LLMs
MCQA benchmarks have become standard for evaluating language modelsHendryckset al\.\([2021](https://arxiv.org/html/2607.21613#bib.bib30)\), offering a balanced evaluation framework that avoids open\-ended generation complexities while probing model capabilities\. The use of large language models \(LLMs\) as classifiers through in\-context learning \(ICL\) was established byBrownet al\.\([2020](https://arxiv.org/html/2607.21613#bib.bib6)\), who demonstrated that GPT\-3 \(175B parameters\) could perform sentiment analysis, natural language inference, and topic classification by conditioning on a few labeled demonstrations in the prompt and reading the next predicted token as the class label, without any gradient updates\. This one\-token prediction paradigm, however, was shown byZhaoet al\.\([2021](https://arxiv.org/html/2607.21613#bib.bib7)\)to suffer from systematic biases: the model’s probability distribution over candidate label tokens is skewed by majority\-label bias \(favoring whichever class appears most in the demonstrations\), recency bias \(favoring the label of the last demonstration\), and common\-token bias \(favoring label words that are more frequent in pretraining\)\. Their proposed fix,*contextual calibration*, estimates these biases by feeding a content\-free input \(e\.g\., “N/A”\) and learning an affine transformation on the label logits, recovering up to 30 percentage points of accuracy\.
Our central research question is: How do modern LLMs make their answer choice in MCQA tasks? This study also extends to classification tasks using LLMs\. We address this question by identifying a fixedHard Decision Layer\(HDL\)—the layer beyond which answer option rankings remain stable\. Unlike prior work that relies on dynamic early exit policies or post\-hoc layer pruning, the HDL emerges as a natural, static property of transformer inference without requiring learned routing mechanisms\. We demonstrate that this phenomenon generalizes consistently across model architectures and evaluation datasets\. These findings reveal that transformers inherently organize their inference according to architectural and task characteristics, offering mechanistic insights into how depth is utilized during reasoning\.
## 3Methodology
### 3\.1Preliminary
#### 3\.1\.1Logit Lens
Logit Lens is a technique where intermediate layer representations from a transformer model are projected into the vocabulary space, allowing us to observe what output the model "predicts" at each layer\. This technique enables us to track how the model’s predictions evolve across layers and identify key decision points in the reasoning process\.
Letℳ\\mathcal\{M\}denote a transformer\-based large language model withLLlayers, hidden dimensiondd, and vocabulary𝒱\\mathcal\{V\}of size\|𝒱\|=V\|\\mathcal\{V\}\|=V\. Given an input sequence ofTTtokens𝒙=\(x1,x2,…,xT\)\\bm\{x\}=\(x\_\{1\},x\_\{2\},\\dots,x\_\{T\}\)withxt∈𝒱x\_\{t\}\\in\\mathcal\{V\}, the embedding layer produces an initial representation𝒉\(0\)∈ℝT×d\\bm\{h\}^\{\(0\)\}\\in\\mathbb\{R\}^\{T\\times d\}\. Each transformer blockℓ∈\{1,…,L\}\\ell\\in\\\{1,\\dots,L\\\}updates the residual stream as
𝒉\(ℓ\)=fℓ\(𝒉\(ℓ−1\)\),\\bm\{h\}^\{\(\\ell\)\}=f\_\{\\ell\}\\\!\\bigl\(\\bm\{h\}^\{\(\\ell\-1\)\}\\bigr\),\(1\)wherefℓf\_\{\\ell\}encapsulates multi\-head self\-attention followed by a position\-wise feedforward network, with residual connections and layer normalization applied according to the model’s specific architecture\. We capture𝒉\(ℓ\)\\bm\{h\}^\{\(\\ell\)\}for everyℓ\\ell, storing the output tensor during a single forward pass with gradient computation disabled\.
To interpret the intermediate representations, we project each layer’s residual stream into vocabulary space using the model’s unembedding head\. For the token at positionTT, we first extract the hidden\-state vector𝒉t\(ℓ\)∈ℝd\\bm\{h\}\_\{t\}^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{d\}from layerℓ\\ell\. Because the unembedding matrix𝑾U∈ℝV×d\\bm\{W\}\_\{U\}\\in\\mathbb\{R\}^\{V\\times d\}is trained to operate on the output of the final layer normalization, we apply the model’s final\-layer normalization functionLNfinal\(⋅\)\\text\{LN\}\_\{\\text\{final\}\}\(\\cdot\)before projection:
𝒉^t\(ℓ\)=LNfinal\(𝒉t\(ℓ\)\)\.\\hat\{\\bm\{h\}\}\_\{t\}^\{\(\\ell\)\}=\\text\{LN\}\_\{\\text\{final\}\}\\\!\\bigl\(\\bm\{h\}\_\{t\}^\{\(\\ell\)\}\\bigr\)\.\(2\)The logit vector over the vocabulary at layerℓ\\elland positionttis then
𝒛t\(ℓ\)=𝑾U𝒉^t\(ℓ\)\+𝒃U,\\bm\{z\}^\{\(\\ell\)\}\_\{t\}=\\bm\{W\}\_\{U\}\\,\\hat\{\\bm\{h\}\}\_\{t\}^\{\(\\ell\)\}\+\\bm\{b\}\_\{U\},\(3\)where𝒃U∈ℝV\\bm\{b\}\_\{U\}\\in\\mathbb\{R\}^\{V\}is the bias term if present\. For each layer, we record the logits for the answer option tokensS=\{A,B,C,D,…\}S=\\\{A,B,C,D,\.\.\.\\\}, yielding the pairs\{\(v,zt,v\(ℓ\)\):v∈𝒮\}\\bigl\\\{\\\!\\bigl\(v,\\,z^\{\(\\ell\)\}\_\{t,v\}\\bigr\):v\\in\\mathcal\{S\}\\bigr\\\}\.
#### 3\.1\.2Logit Lens Extraction and Analysis
To investigate how models arrive at answers across layers, we apply Logit Lens to extract layer\-wise logits, and thereby, token rankings for the answer options in response to the prompt111We reproduce a standard multiple\-choice prompt template in Figure[3](https://arxiv.org/html/2607.21613#A1.F3)in Appendix[A](https://arxiv.org/html/2607.21613#A1), demonstrating the structured format used in our experiments\.\. For each layer, we record the logits for the answer option tokens and rank them to determine the model’s preference at that depth\. These layer\-wise rankings allow us to compute accuracy at each layer and track how the model’s answer evolves throughout the network, providing mechanistic insight into the internal reasoning process\.
We now formalize the ranking and accuracy computations used in our analysis\. For a questionsswith four answer options represented by vocabulary tokensv1,v2,v3,v4∈𝒱v\_\{1\},v\_\{2\},v\_\{3\},v\_\{4\}\\in\\mathcal\{V\}, we extract logits at each layerℓ\\ellusing the Logit Lens procedure\.
At layerℓ\\elland positionTT\(the position of the answer token in the sequence\), let𝒛T\(ℓ\)\\bm\{z\}^\{\(\\ell\)\}\_\{T\}denote the full logit vector over vocabulary, as defined in Equation \(3\)\. We extract the logits corresponding to the four answer option tokens as
𝐳ℓ=\(zT,v1\(ℓ\),zT,v2\(ℓ\),zT,v3\(ℓ\),zT,v4\(ℓ\)\),\\mathbf\{z\}\_\{\\ell\}=\\bigl\(z^\{\(\\ell\)\}\_\{T,v\_\{1\}\},z^\{\(\\ell\)\}\_\{T,v\_\{2\}\},z^\{\(\\ell\)\}\_\{T,v\_\{3\}\},z^\{\(\\ell\)\}\_\{T,v\_\{4\}\}\\bigr\),\(4\)wherezT,vi\(ℓ\)z^\{\(\\ell\)\}\_\{T,v\_\{i\}\}is theviv\_\{i\}\-th component of𝒛T\(ℓ\)\\bm\{z\}^\{\(\\ell\)\}\_\{T\}\. We compute the ranking by sorting these logits in descending order:
rankℓ\(vi\)=argsortdescending\(𝐳ℓ\)\(vi\),\\text\{rank\}\_\{\\ell\}\(v\_\{i\}\)=\\text\{argsort\}\_\{\\text\{descending\}\}\(\\mathbf\{z\}\_\{\\ell\}\)\(v\_\{i\}\),\(5\)whererankℓ\(vi\)∈\{1,2,3,4\}\\text\{rank\}\_\{\\ell\}\(v\_\{i\}\)\\in\\\{1,2,3,4\\\}represents the position of option tokenviv\_\{i\}in the sorted logits at layerℓ\\ell, with rank 1 indicating the highest logit\.
To aggregate layer\-wise rankings across the dataset, we group tokens by their rank at the final output layer\. Let𝒟\\mathcal\{D\}denote the set of all questions, and for each questions∈𝒟s\\in\\mathcal\{D\}, letoutput\_rank\(s,vi\)\\text\{output\\\_rank\}\(s,v\_\{i\}\)denote the output option rank \(position in the final\-layer ranking\) of tokenviv\_\{i\}\. LetIk\(s,vi\):=𝟙\[output\_rank\(s,vi\)=k\]I\_\{k\}\(s,v\_\{i\}\):=\\mathds\{1\}\[\\text\{output\\\_rank\}\(s,v\_\{i\}\)=k\], where𝟙\[⋅\]\\mathds\{1\}\[\\cdot\]is the indicator function, be the indicator that tokenviv\_\{i\}achieves rankkkin questionss\. The average rank of tokens with output option rankkkat layerℓ\\ellis computed as:
rank¯ℓ\(k\)=1\|𝒟\|∑s∈𝒟∑i=14rankℓ\(vi\)⋅Ik\(s,vi\),\\overline\{\\text\{rank\}\}\_\{\\ell\}\(k\)=\\frac\{1\}\{\|\\mathcal\{D\}\|\}\\sum\_\{s\\in\\mathcal\{D\}\}\\sum\_\{i=1\}^\{4\}\\text\{rank\}\_\{\\ell\}\(v\_\{i\}\)\\cdot I\_\{k\}\(s,v\_\{i\}\),\(6\)with standard deviationσℓ\(k\)\\sigma\_\{\\ell\}\(k\)computed across questions to capture variability\.
Finally, layer\-wise accuracy measures the fraction of questions for which the correct answer token achieves the highest ranking at layerℓ\\ell\. Letvcorrect\(s\)v\_\{\\text\{correct\}\}\(s\)denote the correct answer token for questionss\. The accuracy at layerℓ\\ellis:
Accℓ=1\|𝒟\|∑s∈𝒟𝟙\[rankℓ\(vcorrect\(s\)\)=1\],\\text\{Acc\}\_\{\\ell\}=\\frac\{1\}\{\|\\mathcal\{D\}\|\}\\sum\_\{s\\in\\mathcal\{D\}\}\\mathds\{1\}\[\\text\{rank\}\_\{\\ell\}\(v\_\{\\text\{correct\}\}\(s\)\)=1\],\(7\)where𝟙\[⋅\]\\mathds\{1\}\[\\cdot\]is the indicator function\.
#### 3\.1\.3Hard\-Decision Layer
We identify a unique layer \- called the Hard\-Decision Layer \(HDL\) \- in the modern LLMs tested, which causes the steepest drop in the average ranking of the answer option that eventually ends up as the predicted answer by the model\. Letoutput\_rank\(s,vi\)=1\\text\{output\\\_rank\}\(s,v\_\{i\}\)=1identify the final predicted answer option for questionss\. HDL is formally defined as the layer where the largest decrease in average rank occurs:
ℓHDL=argmaxℓ∈\{1,…,L\}\[rank¯ℓ\(1\)−rank¯ℓ−1\(1\)\],\\ell\_\{\\text\{HDL\}\}=\\arg\\max\_\{\\ell\\in\\\{1,\\ldots,L\\\}\}\\left\[\\overline\{\\text\{rank\}\}\_\{\\ell\}\(1\)\-\\overline\{\\text\{rank\}\}\_\{\\ell\-1\}\(1\)\\right\],\(8\)whererank¯ℓ\(1\)\\overline\{\\text\{rank\}\}\_\{\\ell\}\(1\)is the average rank of the predicted answer token \(output rank 1\) at layerℓ\\ell, as defined in Equation[6](https://arxiv.org/html/2607.21613#S3.E6)\. This layer represents the critical point in the network where the model commits to its final answer, with the steepest improvement in answer ranking indicating the strongest refinement in the decision process\.
### 3\.2Experimental Setup
#### 3\.2\.1Models
We evaluate four instruction\-tuned language models: Mistral\-7B\-Instruct\-v0\.3Jianget al\.\([2023](https://arxiv.org/html/2607.21613#bib.bib32)\), Llama\-3\.1\-8B\-InstructGrattafioriet al\.\([2024](https://arxiv.org/html/2607.21613#bib.bib33)\), IBM Granite\-3\.3\-2B\-Instruct222[https://huggingface\.co/ibm\-granite/granite\-3\.3\-2b\-instruct](https://huggingface.co/ibm-granite/granite-3.3-2b-instruct), and Qwen3\-4B\-Instruct\-2507Yanget al\.\([2025](https://arxiv.org/html/2607.21613#bib.bib36)\)\. These models range from 2B to 8B parameters and represent diverse architectures and training methodologies\. More information about the models used is provided in Appendix[A](https://arxiv.org/html/2607.21613#A1)\(Table[5](https://arxiv.org/html/2607.21613#A1.T5)\)\.
#### 3\.2\.2Datasets
We evaluate model behavior across four multiple\-choice question\-answering datasets: CommonsenseQATalmoret al\.\([2019](https://arxiv.org/html/2607.21613#bib.bib37)\), a benchmark for evaluating commonsense reasoning; QASCKhotet al\.\([2020](https://arxiv.org/html/2607.21613#bib.bib38)\), which requires compositional reasoning and multi\-step inference; MMLU\-ProWanget al\.\([2024](https://arxiv.org/html/2607.21613#bib.bib39)\), an enhanced MMLU benchmark covering STEM, social sciences, and humanities; and SuperGPQADuet al\.\([2025](https://arxiv.org/html/2607.21613#bib.bib40)\), a graduate\-level benchmark covering advanced concepts in physics, chemistry, and biology\. For each model and dataset, we sample 100 random questions and normalize them to contain exactly four options per question\. When a question contains more than four options, we randomly select four while ensuring the correct answer remains in the pool\. A detailed summary of the datasets is provided in Appendix[A](https://arxiv.org/html/2607.21613#A1)\(Table[6](https://arxiv.org/html/2607.21613#A1.T6)\)\.
#### 3\.2\.3Fine\-tuning with Low\-Rank Adaptation
To understand whether HDL persists after fine\-tuning, we fine\-tune two models \(Qwen and Llama\) using low\-rank adaptation \(LoRA\)Huet al\.\([2022](https://arxiv.org/html/2607.21613#bib.bib2)\)for 1 epoch on a separate training set consisting of80008000to1000010000samples for each evaluated dataset\. Given base parameters𝑾0\\bm\{W\}\_\{0\}and low\-rank matrices𝑨∈ℝd×r\\bm\{A\}\\in\\mathbb\{R\}^\{d\\times r\},𝑩∈ℝr×d\\bm\{B\}\\in\\mathbb\{R\}^\{r\\times d\}with rankr≪dr\\ll d, a model with the LoRA adapter behaves identically to the model with𝑾=𝑾0\+𝑨𝑩\\bm\{W\}=\\bm\{W\}\_\{0\}\+\\bm\{A\}\\bm\{B\}for each adapted weight matrix\. Details regarding the hyperparameters used for model training are provided in Appendix[B](https://arxiv.org/html/2607.21613#A2)\.
#### 3\.2\.4Task Variants
To examine whether other aspects of the multiple\-choice format affect the HDL, we conduct experiments varying both the option labeling scheme and the number of available choices\. We test three labeling formats on the QASC dataset for both Qwen and Llama models: alphabetic labels \(A/B/C/D\), arabic numerals \(1/2/3/4\), and roman numerals \(i/ii/iii/iv\)\. It was ensured that, for the models tested, the new option labels are still tokenized to 1 token\. We additionally vary the number of options from three to five choices \(using a subset or extension of the original options\) to understand how the HDL manifests across different problem complexities\. For each configuration, we extract layer\-wise logits and compute both option token rankings and layer\-wise accuracy\. This allows us to isolate the effects of notational convention and problem structure from the underlying model reasoning and assess whether the HDL is robust across different task variants\.
#### 3\.2\.5Open\-ended Generation
To investigate whether HDL as a detectable phenomenon extends beyond multiple\-choice reasoning to general sequence generation, we conduct preliminary experiments on open\-ended question answering using the GSM8KCobbeet al\.\([2021](https://arxiv.org/html/2607.21613#bib.bib35)\)dataset of mathematical word problems\. Rather than tracking option token rankings, we extract residual states during generation of thenn\-th output token at various generation steps \(n=10,50,100,200n=10,50,100,200\), allowing us to observe layer\-wise dynamics across different points in the generation process\. For each step, we capture the top\-10 most likely output tokens at each layer by applying Logit Lens to the intermediate representations\. This analysis allows us to assess whether intermediate layers exhibit stabilization patterns similar to those observed in multiple\-choice settings and to examine how the rank of the generated token evolves across layers in a free\-form generation context\.
## 4Results
Finding 1: HDL exists in all models and is invariant across MCQA datasets\.
ModelDatasetHDL Layer\(Predicted\)Accuracy\(Pre\-HDL\)Accuracy\(Post\-HDL\)Accuracy\(Final\)Option\-1 Rank\(Pre\-HDL\)Option\-1 Rank\(Post\-HDL\)QwenQASC250\.310\.70 \(\+0\.39\)0\.72 \(\+0\.02\)2\.31±\\pm1\.091\.04±\\pm0\.20MMLU\-Pro250\.230\.58 \(\+0\.35\)0\.61 \(\+0\.03\)2\.52±\\pm1\.171\.09±\\pm0\.35CommonsenseQA250\.260\.87 \(\+0\.61\)0\.86 \(\-0\.01\)2\.42±\\pm1\.071\.01±\\pm0\.10SuperGPQA250\.240\.40 \(\+0\.16\)0\.41 \(\+0\.01\)2\.22±\\pm1\.111\.05±\\pm0\.26LlamaQASC180\.280\.59 \(\+0\.31\)0\.74 \(\+0\.15\)2\.35±\\pm1\.011\.42±\\pm0\.62MMLU\-Pro180\.250\.45 \(\+0\.20\)0\.59 \(\+0\.14\)2\.72±\\pm1\.111\.97±\\pm1\.05CommonsenseQA180\.240\.63 \(\+0\.39\)0\.80 \(\+0\.17\)2\.54±\\pm1\.081\.52±\\pm0\.81SuperGPQA230\.290\.41 \(\+0\.12\)0\.32 \(\-0\.09\)2\.11±\\pm0\.961\.58±\\pm0\.89GraniteQASC320\.270\.51 \(\+0\.24\)0\.67 \(\+0\.16\)2\.41±\\pm1\.211\.56±\\pm0\.88MMLU\-Pro320\.320\.45 \(\+0\.13\)0\.47 \(\+0\.02\)2\.54±\\pm1\.121\.90±\\pm1\.01CommonsenseQA320\.300\.56 \(\+0\.26\)0\.72 \(\+0\.16\)2\.45±\\pm1\.201\.73±\\pm1\.01SuperGPQA330\.270\.36 \(\+0\.09\)0\.41 \(\+0\.05\)2\.12±\\pm1\.181\.74±\\pm1\.05MistralQASC200\.380\.56 \(\+0\.18\)0\.62 \(\+0\.06\)1\.86±\\pm0\.961\.32±\\pm0\.72MMLU\-Pro200\.300\.46 \(\+0\.16\)0\.49 \(\+0\.03\)2\.09±\\pm1\.031\.35±\\pm0\.77CommonsenseQA200\.450\.68 \(\+0\.23\)0\.76 \(\+0\.08\)1\.97±\\pm1\.081\.36±\\pm0\.66SuperGPQA200\.270\.33 \(\+0\.06\)0\.37 \(\+0\.04\)2\.08±\\pm0\.891\.50±\\pm0\.69
Table 1:Descriptive HDL summary across models and datasets\. Accuracy \(Pre\-HDL\), Accuracy \(Post\-HDL\), and Accuracy \(Final\) are mean accuracies of the logit\-lens projection at layers before the predicted HDL, at and after the HDL, and at the final layer, respectively\. The HDL Layer \(Predicted\) is predicted as the layer that causes the largest drop in Option 1’s average rank\. Parenthetical values in the accuracy columns indicate absolute increases: Accuracy \(Post\-HDL\) shows the increase over Accuracy \(Pre\-HDL\), and Accuracy \(Final\) shows the increase over Accuracy \(Post\-HDL\)\. A small final\-post gap indicates that post\-HDL behavior already matches the model’s final output\. Option\-1 Rank \(Pre\-HDL\) and Option\-1 Rank \(Post\-HDL\) show the mean and standard deviation of the rank of Option 1 at the layer before HDL and at the HDL layer, respectively\. Lower rank values indicate greater concentration on the correct answer, demonstrating the rank collapse that shows the model committing to an answer\.As shown in Table[1](https://arxiv.org/html/2607.21613#S4.T1), all models have a fixed HDL for the MCQA datasets tested\. Figure[4](https://arxiv.org/html/2607.21613#A3.F4)and Figure[5](https://arxiv.org/html/2607.21613#A3.F5)in Appendix[C](https://arxiv.org/html/2607.21613#A3)show the generalization of this behaviour across datasets and models respectively\. Some models position their HDL early in the network \(e\.g\., Llama at layer 18 of 32 total layers, or 56\.25% depth\), while others place it much later \(e\.g\., Granite at layer 32 of 40 total layers, or 80% depth\)\. Table[2](https://arxiv.org/html/2607.21613#S4.T2)summarizes the model parameter counts and the fraction of layers up to the HDL, representing where most computation occurs\. A trend is observed where parameter count appears inversely related to the fraction of layers until the HDL\. This suggests that higher\-capacity models accomplish answer selection in earlier layers\.
Table 2:Depth of Hard Decision Layer \(as percentage of layers\) across modelsModelParametersDepth of HDL\(as % of layers\)Llama8B56\.25% \(18/32\)Mistral7B62\.50% \(20/32\)Qwen4B69\.44% \(25/36\)Granite2B80\.00% \(32/40\)Table[1](https://arxiv.org/html/2607.21613#S4.T1)also demonstrates a sharp jump in the accuracy of intermediate layer outputs at the HDL \(also illustrated in Figures[7](https://arxiv.org/html/2607.21613#A4.F7)and[9](https://arxiv.org/html/2607.21613#A4.F9)in Appendix[D](https://arxiv.org/html/2607.21613#A4)\)\. For some models, such as Qwen, the accuracy at the HDL nearly matches that of the final layer\. The accuracy spike at HDL is much larger than the accuracy improvement from HDL till the final output layer\. Remarkably, the accuracy of layers before the HDL hovers around 25%, which equals random chance for four\-option multiple\-choice questions\. This suggests the model essentially selects its answer, for most questions, at the HDL\.
Citing a specific example of one dataset and model, Figure[1](https://arxiv.org/html/2607.21613#S0.F1)shows the aggregated layer\-wise rankings of the answer option tokens by the Qwen model across the entire QASC dataset\. In the plot, Options 1, 2, 3, and 4 represent the options ranked 1st, 2nd, 3rd, and 4th respectively in the model’s final output layer\. For most questions, the model initially exhibits frequent fluctuations in the ordering of answer tokens\. However, at the HDL \(layer 25 for the Qwen model\), the model produces a decisive ranking that remains largely stable through the remaining layers\. On average, option tokens maintain their ranking from the HDL onward, suggesting that the model has completed most of its reasoning by this point\. Notably, the top answer token shows greater stability than other option choices\. As shown in Figure[4](https://arxiv.org/html/2607.21613#A3.F4), the HDL position for a given model remains consistent across different datasets\. This suggests the HDL is unlikely to be a memory storage layer, in which case we would expect its position to vary with different datasets\. Rather, the HDL likely plays a role specific to multiple\-choice question answering\. The results also show a sharp drop in the mean and standard deviation of Option 1’s rank at the HDL which shows the model committing to a specific answer\.
Finding 2: HDL is invariant both to finetuning and to the number of choice options\.The Qwen and Llama models were finetuned on a separate training set for each of the datasets and evaluated again\. The HDL position for these models remains stable even after finetuning\. While the models demonstrate clear accuracy improvements \(as expected\) following finetuning, the HDL itself does not shift across base and finetuned variants, as confirmed quantitatively in Table[3](https://arxiv.org/html/2607.21613#S4.T3), and visualized in Figures[6](https://arxiv.org/html/2607.21613#A4.F6)and[8](https://arxiv.org/html/2607.21613#A4.F8)\(Appendix[D](https://arxiv.org/html/2607.21613#A4)\)\.
ModelDatasetVariantHDL Layer\(Predicted\)Accuracy\(Pre\-HDL\)Accuracy\(Post\-HDL\)Accuracy\(Final\)LlamaCommonsenseQABase180\.240\.63 \(\+0\.39\)0\.80 \(\+0\.17\)Finetuned180\.250\.60 \(\+0\.35\)0\.86 \(\+0\.26\)LlamaMMLU\-ProBase180\.250\.45 \(\+0\.20\)0\.59 \(\+0\.14\)Finetuned180\.210\.40 \(\+0\.19\)0\.65 \(\+0\.25\)LlamaQASCBase180\.280\.59 \(\+0\.31\)0\.74 \(\+0\.15\)Finetuned180\.290\.61 \(\+0\.32\)0\.77 \(\+0\.16\)LlamaSuperGPQABase230\.290\.41 \(\+0\.12\)0\.32 \(\-0\.09\)Finetuned180\.230\.31 \(\+0\.08\)0\.40 \(\+0\.09\)QwenCommonsenseQABase250\.260\.87 \(\+0\.61\)0\.86 \(\-0\.01\)Finetuned250\.270\.86 \(\+0\.59\)0\.86 \(\+0\.00\)QwenMMLU\-ProBase250\.230\.58 \(\+0\.35\)0\.61 \(\+0\.03\)Finetuned250\.220\.65 \(\+0\.43\)0\.69 \(\+0\.04\)QwenQASCBase250\.310\.70 \(\+0\.39\)0\.72 \(\+0\.02\)Finetuned250\.300\.79 \(\+0\.49\)0\.80 \(\+0\.01\)QwenSuperGPQABase250\.240\.40 \(\+0\.16\)0\.41 \(\+0\.01\)Finetuned250\.230\.45 \(\+0\.22\)0\.42 \(\-0\.03\)Table 3:Descriptive HDL summary for base vs fine\-tuned models\. Each \(model, dataset\) pair appears on two rows: base and finetuned variants\. HDL Layer \(Predicted\), Accuracy \(Pre\-HDL\), Accuracy \(Post\-HDL\), and Accuracy \(Final\) are mean logit\-lens accuracies at layers before the predicted HDL, at and after the HDL, and at the final layer, respectively\. Parenthetical values in the accuracy columns indicate absolute increases: Accuracy \(Post\-HDL\) shows the increase over Accuracy \(Pre\-HDL\), and Accuracy \(Final\) shows the increase over Accuracy \(Post\-HDL\)\. Comparing base and finetuned variants within a \(model, dataset\) pair reveals whether fine\-tuning shifted the HDL position or only its accuracy levels\.Similarly, when examining whether the number of options in multiple\-choice questions affects the HDL position by comparing three\-option and five\-option variants on the QASC dataset for both Qwen and Llama models, we found the HDL position to remain invariant across different option counts \(Figure[10](https://arxiv.org/html/2607.21613#A5.F10)in Appendix[E](https://arxiv.org/html/2607.21613#A5)\)\. While the pre\-HDL accuracy differs according to random chance \(33% for three options, 20% for five options\), the HDL location and the accuracy patterns \(Figure[11](https://arxiv.org/html/2607.21613#A5.F11)in Appendix[E](https://arxiv.org/html/2607.21613#A5)\) do not significantly differ across these conditions \- quantified in Table[4](https://arxiv.org/html/2607.21613#S4.T4)\.
ModelNumber of OptionsHDL Layer\(Predicted\)Accuracy\(Pre\-HDL\)Accuracy\(Post\-HDL\)Accuracy\(Final\)Qwen3250\.320\.78 \(\+0\.46\)0\.77 \(−\-0\.01\)Qwen4250\.310\.70 \(\+0\.39\)0\.72 \(\+0\.02\)Qwen5250\.220\.64 \(\+0\.42\)0\.64 \(\+0\.00\)Llama3180\.340\.62 \(\+0\.28\)0\.78 \(\+0\.16\)Llama4180\.280\.59 \(\+0\.31\)0\.74 \(\+0\.15\)Llama5180\.240\.55 \(\+0\.31\)0\.70 \(\+0\.15\)Table 4:Descriptive HDL summary across QASC option\-count variants\. All variants use alphabetic labels \(A/B/C/D\) on the same QASC questions; only the Number of Options differs\. The default QASC cohort uses 4 options\. Each row shows the HDL Layer \(Predicted\) and accuracies at the layer just before HDL \(Accuracy \(Pre\-HDL\)\), at the HDL layer \(Accuracy \(Post\-HDL\)\), and at the final layer \(Accuracy \(Final\)\) for one \(model, number of options\) combination; parenthetical values on the accuracy columns are absolute increases over the previous column\. Results are restricted to Qwen and Llama because the 3\- and 5\-option residual\-trace pickles are only available for those models\.Finding 3: HDL prominence is affected by option labels used\.Previous experiments used alphabets \(A/B/C/D\) as option labels in the multiple\-choice questions\. To determine whether the choice of option labels affects the location or prominence of the HDL, we tested alternatives: arabic numerals \(1/2/3/4\) and roman numerals \(i/ii/iii/iv\)\. As shown in both Figure[2](https://arxiv.org/html/2607.21613#S4.F2)\(also in Table[7](https://arxiv.org/html/2607.21613#A5.T7)and Figure[12](https://arxiv.org/html/2607.21613#A5.F12)in Appendix[D](https://arxiv.org/html/2607.21613#A4)\), the HDL appears at the same position across all labeling schemes\. However, the pattern differs in clarity: the HDL is well\-pronounced for both arabic numerals and alphabets, whereas for roman numerals it is less distinct\. The top\-1 option remains stable post\-HDL across all schemes, but other option probabilities show greater volatility with roman numerals compared to the other formats\. The layer\-wise accuracy plots in Figure[13](https://arxiv.org/html/2607.21613#A5.F13)\- also in Appendix[E](https://arxiv.org/html/2607.21613#A5)\- reveal more pronounced differences\. Arabic numerals exhibit the steepest accuracy jump at the HDL, with alphabets showing nearly identical behavior\. In contrast, roman numerals display a more gradual accuracy increase beginning at the HDL\. This might be indicative of MCQs in training data having more of alphabets and arabic numerals for option labels\.
\(a\)Qwen \(Arabic\)
\(b\)Qwen \(Roman\)
Figure 2:Impact of option labeling schemes on token rankings across layers\. Each subfigure shows aggregate token ranking plots for the Qwen model across alternate labeling schemes: Arabic numerals \(1/2/3/4\), and Roman numerals \(i/ii/iii/iv\), all evaluated on the QASC dataset\. The plots reveal whether the location and prominence of the Hard Decision Layer \(HDL\) varies across different labeling schemes\. Generally, the HDL manifests at a similar layer position regardless of labeling scheme, though its clarity differs: Alphabets \(in Figure[1](https://arxiv.org/html/2607.21613#S0.F1)\) and Arabic numerals produce a well\-pronounced HDL, while Roman numerals show a less distinct pattern\.### 4\.1Open\-ended generation
We also briefly explore open\-ended question answering by evaluating models on GSM8K math word problems\. Residuals during generation ofnthn^\{th\}token were captured \(n=10,50,100,200n=10,50,100,200\)\. Figure[14](https://arxiv.org/html/2607.21613#A6.F14)in Appendix[F](https://arxiv.org/html/2607.21613#A6)displays average token rankings for the top 10 output tokens from the Qwen model; tokens not in the top\-10 at intermediate layers appear blank\. The HDL is less pronounced in these plots than in multiple\-choice settings\. However, knowing that layer 25 is the HDL for Qwen, we observe that the top\-1 token stabilizes around this layer across all values ofnn, while other tokens show no clear pattern\. This suggests the HDL may extend beyond multiple\-choice answering to general token generation, though further investigation is needed\.
## 5Discussion
Our findings reveal interesting patterns in how language models process multiple\-choice questions across their layers\. At first glance, the observed behavior might be attributable to models storing dataset\-specific information in particular layers, as suggested by prior work on model internalsMenget al\.\([2022](https://arxiv.org/html/2607.21613#bib.bib21)\)\. However, this explanation appears unlikely for several reasons\. The behavior remains consistent across diverse datasets, suggesting it is not tied to specific memorization of training data\. Furthermore, the phenomenon occurs consistently across four different independently\-trained models, making it improbable that every model independently chose to localize all dataset\-specific knowledge for four different datasets in one layer\. Additionally, the inclusion of SuperGPQA dataset, released after the Llama and Mistral models were trained, rules out dataset memorization as the primary driver, while still allowing for the remote possibility that relevant knowledge is localized within these layers\.
A more compelling interpretation is that this behavior reflects the inherent structure of multiple\-choice question answering itself\. If this hypothesis holds, it would suggest that models may self\-organize their computations in a task\-specific manner, with later layers dedicated to selecting among the available options\. Consistent with this view, our analysis of open\-ended generation shows that while this decision\-making pattern is less pronounced, it remains present\. This aligns with recent findings in the literature showing that models identify likely candidate tokens in earlier layers and refine token distributions progressively through deeper layersGuptaet al\.\([2025](https://arxiv.org/html/2607.21613#bib.bib3)\)\.
Our results have significant implications for model compression and efficiency\. Since the HDL achieves accuracy comparable to the final layer, layers beyond the HDL can be pruned without sacrificing performance on multiple\-choice tasks\. Larger models like Llama demonstrate a shallower HDL position, enabling up to 43\.75% of layers to be removed while retaining competitive accuracy\. This reveals an interesting interplay between model capacity and computational depth: a larger model with substantial layer pruning post\-HDL may outperform a smaller, naturally efficient model with a deeper HDL\. These results suggest that model size and HDL position are jointly important considerations for inference efficiency, providing a novel approach to trading off computational cost against model capacity\. However, the findings also suggest a critical caveat to layer pruning: when prompt formatting differs, the final layers become essential for maintaining accuracy, and aggressive pruning post\-HDL could substantially degrade performance\.
## 6Limitations
This study has several important limitations that should be considered when interpreting the results\. First, our evaluation restricts models to direct answer selection without access to chain\-of\-thought reasoning, which is a more natural and often more effective way for models to approach complex problems\. Additionally, our analysis focuses only on the canonical answer labels \(A/B/C/D\), but intermediate layers may activate alternative surface forms that correspond to the same option \(e\.g\., “b”, “second”, “B\)” and other variants\), which are not accounted for in the current analysis\. Furthermore, while our analysis includes some exploration of open\-ended generation, this setting requires substantially more investigation to understand whether the patterns we observe in multiple\-choice tasks generalize to free\-form generation\. Future work should extend this analysis to chain\-of\-thought reasoning, more diverse generation tasks, account for answer form variations, and investigate the underlying mechanisms driving the observed layer\-wise decision patterns\.
## References
- N\. Belrose, I\. Ostrovsky, L\. McKinney, Z\. Furman, L\. Smith, D\. Halawi, S\. Biderman, and J\. Steinhardt \(2023\)Eliciting latent predictions from transformers with the tuned lens\.arXiv preprint arXiv:2303\.08112\.Cited by:[§2\.1](https://arxiv.org/html/2607.21613#S2.SS1.p1.1)\.
- Mechanistic interpretability for ai safety: a review\.Note:arXiv preprint arXiv:2404\.14082Cited by:[§1](https://arxiv.org/html/2607.21613#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.21613#S2.SS1.p1.1)\.
- T\. B\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell, S\. Agarwal, A\. Herbert\-Voss, G\. Krueger, T\. Henighan, R\. Child, A\. Ramesh, D\. M\. Ziegler, J\. Wu, C\. Winter, C\. Hesse, M\. Chen, E\. Sigler, M\. Litwin, S\. Gray, B\. Chess, J\. Clark, C\. Berner, S\. McCandlish, A\. Radford, I\. Sutskever, and D\. Amodei \(2020\)Language models are few\-shot learners\.InAdvances in Neural Information Processing Systems 33 \(NeurIPS\),External Links:[Link](https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html)Cited by:[§2\.3](https://arxiv.org/html/2607.21613#S2.SS3.p1.1)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. Schulman \(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§3\.2\.5](https://arxiv.org/html/2607.21613#S3.SS2.SSS5.p1.2)\.
- G\. Dar, M\. Geva, A\. Gupta, and J\. Berant \(2023\)Analyzing transformers in embedding space\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics,pp\. 16124–16170\.Cited by:[§2\.1](https://arxiv.org/html/2607.21613#S2.SS1.p1.1)\.
- A\. Y\. Din, T\. Karidi, L\. Choshen, and M\. Geva \(2024\)Jump to conclusions: short\-cutting transformers with linear transformations\.InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING 2024\),pp\. 9615–9625\.Cited by:[§1](https://arxiv.org/html/2607.21613#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.21613#S2.SS2.p1.1)\.
- X\. Du, Y\. Yao, K\. Ma, B\. Wang, T\. Zheng, K\. Zhu, M\. Liu, Y\. Liang, X\. Jin, Z\. Wei,et al\.\(2025\)Supergpqa: scaling llm evaluation across 285 graduate disciplines\.arXiv preprint arXiv:2502\.14739\.Cited by:[§3\.2\.2](https://arxiv.org/html/2607.21613#S3.SS2.SSS2.p1.1)\.
- N\. Elhage, N\. Nanda, C\. Olsson, T\. Henighan, N\. Joseph, B\. Mann, A\. Askell, Y\. Bai, A\. Chen, T\. Conerly,et al\.\(2021\)A mathematical framework for transformer circuits\.Transformer Circuits Thread\.External Links:[Link](https://transformer-circuits.pub/2021/framework/index.html)Cited by:[§1](https://arxiv.org/html/2607.21613#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.21613#S2.SS1.p1.1)\.
- S\. Fan, X\. Jiang, X\. Li, X\. Meng, P\. Han, S\. Shang, A\. Sun, Y\. Wang, and Z\. Wang \(2024\)Not all layers of llms are necessary during inference\.arXiv preprint arXiv:2403\.02181\.Cited by:[§1](https://arxiv.org/html/2607.21613#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.21613#S2.SS2.p1.1)\.
- M\. Geva, A\. Caciularu, K\. Wang, and Y\. Goldberg \(2022\)Transformer feed\-forward layers build predictions by promoting concepts in the vocabulary space\.InProceedings of the 2022 conference on empirical methods in natural language processing,pp\. 30–45\.Cited by:[§1](https://arxiv.org/html/2607.21613#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.21613#S2.SS1.p2.1)\.
- M\. Geva, R\. Schuster, J\. Berant, and O\. Levy \(2021\)Transformer feed\-forward layers are key\-value memories\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,pp\. 5484–5495\.Cited by:[§2\.1](https://arxiv.org/html/2607.21613#S2.SS1.p2.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§3\.2\.1](https://arxiv.org/html/2607.21613#S3.SS2.SSS1.p1.1)\.
- A\. Gupta, J\. Yeung, G\. Anumanchipalli, and A\. Ivanova \(2025\)How do llms use their depth?\.arXiv preprint arXiv:2510\.18871\.Cited by:[§1](https://arxiv.org/html/2607.21613#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.21613#S2.SS1.p2.1),[§5](https://arxiv.org/html/2607.21613#S5.p2.1)\.
- A\. Haviv, I\. Cohen, J\. Gidron, R\. Schuster, Y\. Goldberg, and M\. Geva \(2023\)Understanding transformer memorization recall through idioms\.InProceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics,pp\. 248–264\.Cited by:[§2\.1](https://arxiv.org/html/2607.21613#S2.SS1.p1.1)\.
- D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. Steinhardt \(2021\)Measuring massive multitask language understanding\.arXiv preprint arXiv:2009\.03300\.Cited by:[§1](https://arxiv.org/html/2607.21613#S1.p3.1),[§2\.3](https://arxiv.org/html/2607.21613#S2.SS3.p1.1)\.
- J\. Hewitt and C\. D\. Manning \(2019\)A structural probe for finding syntax in word representations\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,pp\. 4129–4138\.Cited by:[§2\.1](https://arxiv.org/html/2607.21613#S2.SS1.p1.1)\.
- E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, W\. Chen,et al\.\(2022\)Lora: low\-rank adaptation of large language models\.\.Iclr1\(2\),pp\. 3\.Cited by:[§3\.2\.3](https://arxiv.org/html/2607.21613#S3.SS2.SSS3.p1.7)\.
- A\. Q\. Jiang, A\. Sablayrolles, A\. Mensch, C\. Bamford, D\. S\. Chaplot, D\. de Las Casas, F\. Bressand, G\. Lengyel, G\. Lample, L\. Saulnier, L\. R\. Lavaud, M\. Lachaux, P\. Stock, T\. L\. Scao, T\. Lavril, T\. Wang, T\. Lacroix, and W\. E\. Sayed \(2023\)Mistral 7b\.ArXivabs/2310\.06825\.External Links:[Link](https://api.semanticscholar.org/CorpusID:263830494)Cited by:[§3\.2\.1](https://arxiv.org/html/2607.21613#S3.SS2.SSS1.p1.1)\.
- T\. Khot, P\. Clark, M\. Guerquin, P\. Jansen, and A\. Sabharwal \(2020\)QASC: a dataset for question answering via sentence composition\.arXiv:1910\.11473v2\.Cited by:[§3\.2\.2](https://arxiv.org/html/2607.21613#S3.SS2.SSS2.p1.1)\.
- D\. Lioubashevski, T\. Schlank, G\. Stanovsky, and A\. Goldstein \(2024\)Looking beyond the top\-1: transformers determine top tokens in order\.arXiv preprint arXiv:2410\.20210\.Cited by:[§1](https://arxiv.org/html/2607.21613#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.21613#S2.SS2.p1.1)\.
- K\. Meng, D\. Bau, A\. Andonian, and Y\. Belinkov \(2022\)Locating and editing factual associations in gpt\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 17359–17372\.Cited by:[§2\.1](https://arxiv.org/html/2607.21613#S2.SS1.p1.1),[§5](https://arxiv.org/html/2607.21613#S5.p1.1)\.
- nostalgebraist \(2020\)Interpreting gpt: the logit lens\.LessWrong\.External Links:[Link](https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens)Cited by:[§2\.1](https://arxiv.org/html/2607.21613#S2.SS1.p1.1)\.
- A\. Talmor, J\. Herzig, N\. Lourie, and J\. Berant \(2019\)CommonsenseQA: a question answering challenge targeting commonsense knowledge\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),Minneapolis, Minnesota,pp\. 4149–4158\.External Links:[Link](https://aclanthology.org/N19-1421),[Document](https://dx.doi.org/10.18653/v1/N19-1421),1811\.00937Cited by:[§3\.2\.2](https://arxiv.org/html/2607.21613#S3.SS2.SSS2.p1.1)\.
- Y\. Wang, X\. Ma, G\. Zhang, Y\. Ni, A\. Chandra, S\. Guo, W\. Ren, A\. Arulraj, X\. He, Z\. Jiang,et al\.\(2024\)Mmlu\-pro: a more robust and challenging multi\-task language understanding benchmark\.Advances in Neural Information Processing Systems37,pp\. 95266–95290\.Cited by:[§3\.2\.2](https://arxiv.org/html/2607.21613#S3.SS2.SSS2.p1.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§3\.2\.1](https://arxiv.org/html/2607.21613#S3.SS2.SSS1.p1.1)\.
- T\. Z\. Zhao, E\. Wallace, S\. Feng, D\. Klein, and S\. Singh \(2021\)Calibrate before use: improving few\-shot performance of language models\.InProceedings of the 38th International Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.139,pp\. 12697–12706\.External Links:[Link](http://proceedings.mlr.press/v139/zhao21c.html)Cited by:[§2\.3](https://arxiv.org/html/2607.21613#S2.SS3.p1.1)\.
## Appendix Contents
- APrompt, Models and Datasets\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.[A](https://arxiv.org/html/2607.21613#A1)
- BLoRA Hyperparameter Details\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.[B](https://arxiv.org/html/2607.21613#A2)
- CSupplementary Results for Finding 1\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.[C](https://arxiv.org/html/2607.21613#A3)
- DSupplementary Results for Finding 2\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.[D](https://arxiv.org/html/2607.21613#A4)
- ESupplementary Results for Finding 3\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.[E](https://arxiv.org/html/2607.21613#A5)
- FOpen\-Ended Generation\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.[F](https://arxiv.org/html/2607.21613#A6)
## Appendix APrompt, Models and Datasets
Figure[3](https://arxiv.org/html/2607.21613#A1.F3)shows the prompt template used for the multiple\-choice question answering used in experiments\. Tables[5](https://arxiv.org/html/2607.21613#A1.T5)and[6](https://arxiv.org/html/2607.21613#A1.T6)provide more information about the models and datasets used respectively\.
You are given a question and some options\. Output the correct option letter only and nothing else\.<question\>For the two linear equations 2 \* x \+ 3 \* y = 10 and 4 \* x \+ 4 \* y = 12 with variables x and y\. Use cramer’s rule to solve these two variables\.</question\><options\>\(A\) \[4, 1\]\(B\) \[\-2, 6\]\(C\) \[3, 2\]\(D\) \[\-1, 4\]</options\>The correct option is: \(Figure 3:Prompt template used in the MCQA tasks\.Table 5:List of models evaluated in this study\. Release dates are approximate and sourced from Hugging Face and arXiv\.Table 6:Summary of datasets used in this study\. Release dates are approximate and sourced from Hugging Face and arXiv\.DatasetDescriptionRelease DateLicense[CommonsenseQA](https://huggingface.co/datasets/tau/commonsense_qa)Benchmark for evaluating commonsense reasoning capabilities through multiple\-choice questions\.November 2, 2018MIT[QASC](https://huggingface.co/datasets/allenai/qasc)Question\-answering dataset requiring compositional reasoning and multi\-step inference across multiple facts\.October 21, 2019Creative Commons Attribution 4\.0[MMLU\-Pro](https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro)Enhanced MMLU benchmark covering STEM, social sciences, and humanities with challenging multiple\-choice questions\.May 8, 2024MIT[SuperGPQA](https://huggingface.co/datasets/m-a-p/SuperGPQA)Graduate\-level question\-answering benchmark covering advanced concepts in physics, chemistry, and biology\.February 20, 2025Open Data Commons License Attribution
## Appendix BLoRA Hyperparameter Details
##### Training Hyperparameters
Hyperparameters for all LoRA model trainings are set as follows:
- •Learning rate:2×10−42\\times 10^\{\-4\}
- •Maximum prompt length:10241024
- •Batch size:6464
- •LoRA rank:3232
##### Implementation
We use the Tinker library for implementing LoRA fine\-tuning\.
##### Dataset\-specific Sample Counts
The number of samples used for fine\-tuning varies by dataset:
- •CommonsenseQA:90009000samples
- •MMLU\-Pro:1000010000samples
- •QASC:80008000samples
- •SuperGPQA:1000010000samples
## Appendix CSupplementary Results for Finding 1
Figure[4](https://arxiv.org/html/2607.21613#A3.F4)shows the average token rankings through the layers for the Qwen model on all the four datasets evaluated\. Figure[5](https://arxiv.org/html/2607.21613#A3.F5)shows the average token rankings through the layers for all four models on the QASC dataset\.
\(a\)CommonsenseQA
\(b\)MMLU\-Pro
\(c\)QASC
\(d\)SuperGPQA
Figure 4:Average token rankings plots of the Qwen model when evaluated on different datasets\.\(a\)Granite
\(b\)Llama
\(c\)Mistral
\(d\)Qwen
Figure 5:Average token rankings plots across different models when evaluated on the QASC dataset\.
## Appendix DSupplementary Results for Finding 2
Figure[6](https://arxiv.org/html/2607.21613#A4.F6)and Figure[7](https://arxiv.org/html/2607.21613#A4.F7)show the average token ranking and accuracy through the layers for the Qwen model in comparison to its finetuned versions on QASC and MMLU\-Pro datasets, while Figure[8](https://arxiv.org/html/2607.21613#A4.F8)and Figure[9](https://arxiv.org/html/2607.21613#A4.F9)show the corresponding results for the Llama model\.
\(a\)QASC Qwen \(Base\)
\(b\)MMLU\-Pro Qwen \(Base\)
\(c\)QASC Qwen \(Ft\)
\(d\)MMLU\-Pro Qwen \(Ft\)
Figure 6:Average token rankings plots comparing base and finetuned \(Ft\) Qwen models across datasets\.\(a\)Qwen QASC
\(Base\)
\(b\)Qwen CommonsenseQA
\(Base\)
\(c\)Qwen QASC
\(Ft\)
\(d\)Qwen CommonsenseQA
\(Ft\)
Figure 7:Accuracy of intermediate layer outputs across model layers for base and finetuned \(Ft\) Qwen models\.\(a\)QASC Llama \(Base\)
\(b\)MMLU\-Pro Llama \(Base\)
\(c\)QASC Llama \(Ft\)
\(d\)MMLU\-Pro Llama \(Ft\)
Figure 8:Average token rankings plots comparing base and finetuned \(Ft\) Llama models across datasets\.\(a\)Llama QASC
\(Base\)
\(b\)Llama CommonsenseQA
\(Base\)
\(c\)Llama QASC
\(Ft\)
\(d\)Llama CommonsenseQA
\(Ft\)
Figure 9:Accuracy of intermediate layer outputs across model layers for base and finetuned \(Ft\) Llama models\.
## Appendix ESupplementary Results for Finding 3
Figures[10](https://arxiv.org/html/2607.21613#A5.F10)and[11](https://arxiv.org/html/2607.21613#A5.F11)show the average token rankings and accuracy through the layers for the Qwen and Llama models when evaluated on the QASC dataset with varying number of answer choices\. Figures[12](https://arxiv.org/html/2607.21613#A5.F12)and[13](https://arxiv.org/html/2607.21613#A5.F13)along with Table[7](https://arxiv.org/html/2607.21613#A5.T7)show the results of Qwen and Llama models when the option labels were varied\.
\(a\)Qwen \(3 Options\)
\(b\)Qwen \(5 Options\)
\(c\)Llama \(3 Options\)
\(d\)Llama \(5 Options\)
Figure 10:Average token ranking plots for varying numbers in the multiple\-choice options for Qwen and Llama models on the QASC dataset\.\(a\)Qwen \(3 Options\)
\(b\)Qwen \(5 Options\)
\(c\)Llama \(3 Options\)
\(d\)Llama \(5 Options\)
Figure 11:Layer\-wise accuracy comparison across different numbers of multiple\-choice options\.\(a\)Qwen \(Alphabets\)
\(b\)Llama \(Alphabets\)
\(c\)Qwen \(Arabic\)
\(d\)Llama \(Arabic\)
\(e\)Qwen \(Roman\)
\(f\)Llama \(Roman\)
Figure 12:Impact of option labeling schemes on token rankings across layers\. Each subfigure shows aggregate token ranking plots for either Qwen or Llama across three labeling schemes: Alphabets \(A/B/C/D\), Arabic numerals \(1/2/3/4\), and Roman numerals \(i/ii/iii/iv\), all evaluated on the QASC dataset\. The plots reveal whether the location and prominence of the Hard Decision Layer \(HDL\) varies across different labeling schemes\. Generally, the HDL manifests at a similar layer position regardless of labeling scheme, though its clarity differs: Alphabets and Arabic numerals produce a well\-pronounced HDL, while Roman numerals show a less distinct pattern\.ModelLabel TypeHDL Layer\(Predicted\)Accuracy\(Pre\-HDL\)Accuracy\(Post\-HDL\)Accuracy\(Final\)Qwenalpha250\.310\.70 \(\+0\.39\)0\.72 \(\+0\.02\)Qwenarabic250\.310\.73 \(\+0\.42\)0\.75 \(\+0\.02\)Qwenroman250\.270\.57 \(\+0\.30\)0\.77 \(\+0\.20\)Llamaalpha180\.280\.59 \(\+0\.31\)0\.74 \(\+0\.15\)Llamaarabic180\.430\.68 \(\+0\.25\)0\.71 \(\+0\.03\)Llamaroman180\.420\.56 \(\+0\.14\)0\.73 \(\+0\.17\)Table 7:Descriptive HDL summary across QASC option\-label variants\. All variants present 4 options on the same QASC questions; only the Label Type differs \(alpha = A/B/C/D, arabic = 1/2/3/4, roman = i/ii/iii/iv\)\. Each row shows the HDL Layer \(Predicted\) and accuracies at the layer just before HDL \(Accuracy \(Pre\-HDL\)\), at the HDL layer \(Accuracy \(Post\-HDL\)\), and at the final layer \(Accuracy \(Final\)\) for one \(model, label type\) combination; parenthetical values on the accuracy columns are absolute increases over the previous column\.\(a\)Qwen \(Alphabets\)
\(b\)Llama \(Alphabets\)
\(c\)Qwen \(Arabic\)
\(d\)Llama \(Arabic\)
\(e\)Qwen \(Roman\)
\(f\)Llama \(Roman\)
Figure 13:Layer\-wise accuracy comparison across different option label types\.
## Appendix FPlots for Open\-Ended Generation
Figure[14](https://arxiv.org/html/2607.21613#A6.F14)shows the average token rankings through the layers for the Qwen model on the open\-ended generation task evaluated on different token positionsn=10,50,100,200n=\{10,50,100,200\}
\(a\)n=10n=10
\(b\)n=50n=50
\(c\)n=100n=100
\(d\)n=200n=200
Figure 14:Aggregate plots ofnnth token generation in the open\-ended GSM8K evaluation for Qwen\.Similar Articles
The Scaling Properties of Implicit Deductive Reasoning in Transformers
This research examines how deep Transformers with bidirectional masking achieve implicit deductive reasoning comparable to explicit chain-of-thought methods. The study demonstrates that algorithmically aligned models can scale reasoning capabilities across diverse graph topologies and problem widths.
The Spectral Geometry of Thought: Phase Transitions, Instruction Reversal, Token-Level Dynamics, and Perfect Correctness Prediction in How Transformers Reason
A comprehensive spectral analysis across 11 LLMs revealing that transformers exhibit phase transitions in hidden activation spaces during reasoning versus factual recall, with seven fundamental phenomena including spectral compression, instruction-tuning reversal, and perfect correctness prediction (AUC=1.0) based solely on spectral properties.
Transformers Linearly Represent Highly Structured World Models
This paper demonstrates that transformers trained on Sudoku solving traces build structured world models organized by domain constraints, and identifies a sparse, monosemantic circuit responsible for the naked-single decision rule. The work provides a fully interpretable algorithmic account of transformer reasoning on a combinatorial task.
Transformers Learn the Mestre-Nagao Heuristic
This paper trains a two-layer transformer encoder to classify rational elliptic curves by rank from Frobenius traces, achieving >99% accuracy. Mechanistic interpretability reveals the model learns the Mestre-Nagao heuristic and concentrates attention on prime positions, demonstrating that transformers can learn number-theoretic algorithms.
We are hitting a wall trying to force transformers to do actual logic [D]
The author expresses frustration with the industry's reliance on prompt engineering and scaling to fix logical reasoning deficits in transformer-based LLMs, arguing that these probabilistic models fundamentally lack the architecture for deterministic logic.