追踪大语言模型中的知识与推理

arXiv cs.CL 论文

摘要

该论文提出一个以任务为中心的框架,结合注意力/激活信号与表示度量,刻画LLM在基准测试中的内部处理过程,将检索主导与推理主导的任务置于连续谱上,从而揭示任务级分数所掩盖的失败模式;作者在24个基准上评估了16个LLM并开源了代码。

arXiv:2611.17461v1 Announce Type: new Abstract: Current evaluations of Large Language Models (LLMs) report performance at the dataset or task level, revealing what a model achieves but offering little insight into the internal processing associated with its performance. We propose a task-centric framework for characterising the model-internal processing elicited by a benchmark. It combines attention- and activation-based signals with representation-based measures to characterise benchmarks spanning a continuum from retrieval-dominant to reasoning-dominant and to surface failure modes that task-level scores conceal. We evaluate 16 LLMs across 24 diverse benchmarks and conduct targeted internal-state analyses on representative models and tasks, observing systematic differences in processing between knowledge-centric and reasoning-centric tasks. We shift evaluation from asking what score a model achieves toward characterising the processing patterns associated with that score, potentially supporting more targeted benchmark selection and more informed model choice. We open source our code at https://github.com/extraordinarylab/trace-kr.
查看原文
查看缓存全文

缓存时间: 2026/11/16 08:18

# Tracing Knowledge and Reasoning in LLMs
Source: [https://arxiv.org/html/2611.17461](https://arxiv.org/html/2611.17461)
Yang Wang,Xiao Wang,Yiqi Liu,Chenghao Xiao,Eleni Vasilaki,Goran Nenadic,Chenghua LinUniversity of Manchester University of SheffieldShanghai University of Finance and Economics\{yang\.wang\-2, xiao\.wang\-4, chenghua\.lin\}@manchester\.ac\.uk

###### Abstract

Current evaluations of Large Language Models \(LLMs\) report performance at the dataset or task level, revealing what a model achieves but offering little insight into the internal processing associated with its performance\. We propose a task\-centric framework for characterising the model\-internal processing elicited by a benchmark\. It combines attention\- and activation\-based signals with representation\-based measures to characterise benchmarks spanning a continuum from retrieval\-dominant to reasoning\-dominant and to surface failure modes that task\-level scores conceal\. We evaluate 16 LLMs across 24 diverse benchmarks and conduct targeted internal\-state analyses on representative models and tasks, observing systematic differences in processing between knowledge\-centric and reasoning\-centric tasks\. We shift evaluation from asking what score a model achieves toward characterising the processing patterns associated with that score, potentially supporting more targeted benchmark selection and more informed model choice\. We open source our code at[https://github\.com/extraordinarylab/trace\-kr](https://github.com/extraordinarylab/trace-kr)\.

††footnotetext:†Equal contribution\.††footnotetext:‡Corresponding author\.## 1Introduction

Large Language Models \(LLMs\) have demonstrated remarkable capabilities across a diverse spectrum of tasks, achieving state\-of\-the\-art results on numerous benchmarks[Wang et al\. \(2024a\)](https://arxiv.org/html/2611.17461#bib.bib1);[Jimenez et al\. \(2024\)](https://arxiv.org/html/2611.17461#bib.bib72);[M\-A\-P Team et al\. \(2026\)](https://arxiv.org/html/2611.17461#bib.bib70)\. This progress has been largely measured by aggregate performance scores on static leaderboards\. While invaluable for tracking progress, this evaluation paradigm is reaching its limits[Zhao et al\. \(2026\)](https://arxiv.org/html/2611.17461#bib.bib62);[Shi et al\. \(2026\)](https://arxiv.org/html/2611.17461#bib.bib69)\. A single accuracy number on a benchmark tells us what a model can do, but reveals little about how it achieves the result or why it fails, because it cannot tell whether a correct answer reflects memorised knowledge or actual reasoning\. As models become more powerful, simply measuring their aggregate success yields diminishing insight into their underlying architectural strengths and weaknesses\. The field requires a more fine\-grained approach to evaluation that can disentangle the fundamental capabilities underpinning performance\.

The challenge of disentangling a model’s knowledge from its reasoning is widely acknowledged in the community[Jin et al\. \(2025\)](https://arxiv.org/html/2611.17461#bib.bib29);[Wu et al\. \(2025\)](https://arxiv.org/html/2611.17461#bib.bib28)\. Prior work has largely focused on analysing these capabilities in isolation, which has significant limitations\. Research into knowledge boundaries \(i\.e\., what an LLM knows versus what it does not\) primarily uses techniques like uncertainty estimation, confidence calibration, and internal state probing[Li et al\. \(2025\)](https://arxiv.org/html/2611.17461#bib.bib30);[Xiao et al\. \(2025\)](https://arxiv.org/html/2611.17461#bib.bib4)\. Recent work nevertheless shows that reliable boundary estimation remains difficult: uncertainty can be imperfectly calibrated, while poor awareness of parametric knowledge boundaries can lead to inappropriate reliance on internal or retrieved information[Ni et al\. \(2025\)](https://arxiv.org/html/2611.17461#bib.bib68);[Li et al\. \(2026a\)](https://arxiv.org/html/2611.17461#bib.bib71);[Feng et al\. \(2026a\)](https://arxiv.org/html/2611.17461#bib.bib73)\. Studies of mathematical and compositional reasoning reveal that performance can degrade under seemingly superficial perturbations[Mirzadeh et al\. \(2025\)](https://arxiv.org/html/2611.17461#bib.bib31)\. More recent evidence further shows that models may possess the factual information needed to solve a task yet still rely on surface heuristics rather than applying that knowledge through principled reasoning[Lehmann et al\. \(2026\)](https://arxiv.org/html/2611.17461#bib.bib74)\. While valuable, these largely isolated approaches leave open how knowledge retrieval and reasoning interact within hybrid tasks\. Recent work[Feng et al\. \(2026b\)](https://arxiv.org/html/2611.17461#bib.bib75)has begun to study this interaction, showing that failures can arise either from inadequate knowledge integration or from downstream reasoning even when the relevant knowledge is available\. A high score on a “reasoning benchmark” does not reveal whether success comes from robust reasoning or from easier retrieval shortcuts\. For example, a science benchmark may contain questions that require recalling a specific fact before applying it in a multi\-step inference; strong aggregate performance does not show which of these two capabilities the model actually relies on\. Recent evidence further suggests that access to relevant knowledge does not guarantee its effective use: models may attend to shortcut\-like cues or fall back on parametric memory even when sufficient external knowledge is provided[Wu et al\. \(2025\)](https://arxiv.org/html/2611.17461#bib.bib28);[Li et al\. \(2026b\)](https://arxiv.org/html/2611.17461#bib.bib76)\.

Recent studies have begun to address this gap by moving beyond passive observation toward active decoupling of these capabilities\. For instance, some frameworks propose decomposing the inference process into explicit steps by fine\-tuning models with special tokens like<memory\>and<reason\>[Jin et al\. \(2025\)](https://arxiv.org/html/2611.17461#bib.bib29), or by prompting models to adoptfast\(knowledge\-based\) versusslow\(reasoning\-based\) thinking modes inspired by dual\-system theory[Yang et al\. \(2025a\)](https://arxiv.org/html/2611.17461#bib.bib32)\. Another line of work focuses on evaluating the final reasoning chain, introducing metrics to judge the correctness of knowledge and the logical quality of each step[Wu et al\. \(2025\)](https://arxiv.org/html/2611.17461#bib.bib28)\. Although effective in their respective settings, these methods either alter the model’s natural inference process or rely solely on analysing the generated text\. Consequently, they do not provide a task\-level account of the relative knowledge\-retrieval and reasoning demands reflected in the internal processing of an unmodified LLM\.

To address this limitation, we draw on prior findings concerning the internal structure of Transformers\. Prior work[Yang et al\. \(2025a\)](https://arxiv.org/html/2611.17461#bib.bib32);[Song et al\. \(2025\)](https://arxiv.org/html/2611.17461#bib.bib61);[Ashuach et al\. \(2026\)](https://arxiv.org/html/2611.17461#bib.bib77)suggests partial functional specialisation across Transformer layers\. Lower\-to\-middle feed\-forward layers often support factual associations[Geva et al\. \(2021\)](https://arxiv.org/html/2611.17461#bib.bib33), while upper layers are important for some multi\-step and arithmetic computations[Stolfo et al\. \(2023\)](https://arxiv.org/html/2611.17461#bib.bib34);[Hou et al\. \(2023\)](https://arxiv.org/html/2611.17461#bib.bib35)\. These findings do not imply a strict separation between knowledge and reasoning\. Instead, they motivate the hypothesis that tasks with different balances of retrieval and intermediate computation may produce different internal patterns\. Motivated by this evidence, we test for systematic associations between these patterns and task demands without making causal claims\.

In this paper, we distinguish knowledge retrieval from reasoning by the amount of intermediate transformation required during inference\. Knowledge retrieval refers to identifying or recalling task\-relevant factual or conceptual information, with little or no transformation of that information\. Reasoning refers to deriving task\-relevant information by composing or transforming available information through intermediate operations such as inference, calculation, comparison, or state tracking\. As most tasks require both processes, we therefore use “knowledge\-centric” and “reasoning\-centric” to describe dominant tendencies along a continuum rather than mutually exclusive task categories\. This distinction allows us to examine whether similar task\-level performance can arise from different underlying processing demands and failure modes\. Our investigation is structured around two core research questions:

- •RQ1: Do tasks with different balances of knowledge retrieval and reasoning produce systematically different layer\-wise and decoding\-time patterns in an unmodified LLM?
- •RQ2: How do model family, parameter scale, and inference\-time prompting affect performance across the knowledge\-reasoning continuum?

Our contribution is not a new activation or probing metric in isolation\. Instead, we introduce a task\-level diagnostic framework that combines existing internal signals within a unified knowledge\-reasoning analysis\. Specifically, we: \(i\) introduce a task\-centric framework, which combines static layer\-wise activation statistics, decoding\-time trajectories, and representation\-based measures; \(ii\) use these signals to characterise benchmark demands and construct a knowledge\-reasoning capability landscape across 16 models and 24 benchmarks; and \(iii\) connect the capability landscape to behavioural stress tests that reveal how prompting affects performance and robustness to shortcut\-inducing perturbations\.

## 2Related Work

Our research is situated at the intersection of two critical areas: the effort to disentangle core abilities like knowledge and reasoning, and the field of mechanistic interpretability, which seeks to explain model behaviour through its internal structure\.

### 2\.1Behavioural Approaches

A significant body of research has sought to separate knowledge and reasoning by analysing a model’s inputs and outputs\. The most prominent example is Chain\-of\-Thought \(CoT\) prompting[Wei et al\. \(2022\)](https://arxiv.org/html/2611.17461#bib.bib27), which encourages models to externalise their reasoning steps\. Subsequent frameworks have built on this by evaluating the logical quality and factual correctness of the generated reasoning chain[Wu et al\. \(2025\)](https://arxiv.org/html/2611.17461#bib.bib28), often revealing that models can produce specious or unfaithful explanations[Press et al\. \(2023\)](https://arxiv.org/html/2611.17461#bib.bib40);[Turpin et al\. \(2023\)](https://arxiv.org/html/2611.17461#bib.bib37);[Lanham et al\. \(2023\)](https://arxiv.org/html/2611.17461#bib.bib50);[Chen et al\. \(2025\)](https://arxiv.org/html/2611.17461#bib.bib52)\. While these methods provide valuable insight into a model’s problem\-solving process, they are fundamentally analyses of the final text output, not the internal computations that produced it\. Another line of work actively alters the model’s inference process to force a separation\. Some approaches fine\-tune models with special tokens \(e\.g\.,<memory\>and<reason\>\) to explicitly direct the model’s processing path[Jin et al\. \(2025\)](https://arxiv.org/html/2611.17461#bib.bib29)\. Others[Zhang et al\. \(2026\)](https://arxiv.org/html/2611.17461#bib.bib41);[Yang et al\. \(2025a\)](https://arxiv.org/html/2611.17461#bib.bib32);[Jia et al\. \(2025\)](https://arxiv.org/html/2611.17461#bib.bib39)draw inspiration from dual\-process theory[Groves and Thompson \(1970\)](https://arxiv.org/html/2611.17461#bib.bib49), prompting models to adopt distinctfast\(System 1\) andslow\(System 2\) thinking modes[Kahneman \(2011\)](https://arxiv.org/html/2611.17461#bib.bib38)to disentangle intuitive knowledge retrieval from deliberate reasoning\. These are powerful interventionist techniques, but they do not characterise the intrinsic behaviour of an unmodified LLM\. Our work differs by aiming to diagnose the model’s unaltered inference process\.

### 2\.2Mechanistic Foundations

A distinct and complementary field, mechanistic interpretability, suggests that the conceptual separation between knowledge and reasoning is mirrored by a structural separation within the Transformer architecture itself[Zhang et al\. \(2024\)](https://arxiv.org/html/2611.17461#bib.bib45);[Shi et al\. \(2024\)](https://arxiv.org/html/2611.17461#bib.bib54);[Yang et al\. \(2025b\)](https://arxiv.org/html/2611.17461#bib.bib46);[Gromov et al\. \(2025\)](https://arxiv.org/html/2611.17461#bib.bib53)\. This provides the theoretical foundation for our empirical approach\.[Geva et al\. \(2021\)](https://arxiv.org/html/2611.17461#bib.bib33)demonstrated that Feed\-Forward Networks \(FFNs\) function as key\-value memories, storing factual associations learned during training, particularly in the lower\-to\-mid layers of the model\. This finding has been extended by research that successfully edits specific factual knowledge within the FFN weights of mid\-layer transformers[Meng et al\. \(2022\)](https://arxiv.org/html/2611.17461#bib.bib42);[Meng et al\. \(2023\)](https://arxiv.org/html/2611.17461#bib.bib43), providing a mechanistic basis for where parametric knowledge resides\. In contrast, causal analysis and probing studies have identified the upper layers of the network as being critical for multi\-step, compositional, and arithmetic reasoning[Stolfo et al\. \(2023\)](https://arxiv.org/html/2611.17461#bib.bib34);[Hou et al\. \(2023\)](https://arxiv.org/html/2611.17461#bib.bib35)\. For instance, some studies interpret LLMs by focusing on how the information is processed between layers[Olsson et al\. \(2022\)](https://arxiv.org/html/2611.17461#bib.bib44);[Nanda et al\. \(2023\)](https://arxiv.org/html/2611.17461#bib.bib51)\. Related work further suggests that information encoded in internal representations may not always be reflected in observable model behaviour[Liu et al\. \(2026\)](https://arxiv.org/html/2611.17461#bib.bib78)\. These higher\-level processes are understood to integrate and manipulate information that has been retrieved or processed by the lower layers\. These findings motivate a testable hypothesis rather than a strict localisation claim: tasks that rely more on direct retrieval and tasks that require more intermediate computation may produce different patterns across layers\. Because representations can become less distinguishable in deeper causal Transformers[Hou et al\. \(2023\)](https://arxiv.org/html/2611.17461#bib.bib35), final\-layer states may obscure differences that appear earlier\. We therefore analyse the full layer\-wise evolution of internal states to identify where task\-related differences emerge and whether they persist\.

### 2\.3Towards Holistic Evaluation

Behavioural methods analyse the model from the outside, while mechanistic work “reverse engineer” the function of internal components in isolation[Hou et al\. \(2023\)](https://arxiv.org/html/2611.17461#bib.bib35);[Zhao et al\. \(2024\)](https://arxiv.org/html/2611.17461#bib.bib36);[Zhu et al\. \(2025\)](https://arxiv.org/html/2611.17461#bib.bib55);[Zhao et al\. \(2025\)](https://arxiv.org/html/2611.17461#bib.bib59);[Wang et al\. \(2025b\)](https://arxiv.org/html/2611.17461#bib.bib80);[Hong et al\. \(2026\)](https://arxiv.org/html/2611.17461#bib.bib79)\. Our work bridges this divide by operationalising the findings from mechanistic interpretability\. Instead of focusing on model\-centric properties, such as a model’s self\-awareness of its knowledge gaps[Yin et al\. \(2023\)](https://arxiv.org/html/2611.17461#bib.bib3);[Manakul et al\. \(2023\)](https://arxiv.org/html/2611.17461#bib.bib57)or the procedural limits of its CoT capabilities[Dziri et al\. \(2023\)](https://arxiv.org/html/2611.17461#bib.bib56);[Chen et al\. \(2024\)](https://arxiv.org/html/2611.17461#bib.bib5), our approach is task\-centric\. We use internal activation patterns to characterise the relative retrieval and reasoning demands that a task elicits in an LLM\. We then compare models according to how their internal states and behavioural performance respond to these demands\. This provides a unified view of model performance across the knowledge\-reasoning continuum\. Unlike prior studies that use individual interpretability techniques to localise specific behaviours, we combine complementary internal signals to compare task demands across benchmarks and model families\. Rather than introducing a new metric, its novelty lies in using these signals together to capture different aspects of task\-level processing\.

## 3Method and Experimental Setup

### 3\.1Benchmarks and Taxonomy

We evaluate 24 benchmarks covering factual and biomedical QA, scientific knowledge, commonsense and logical reasoning, reading comprehension, symbolic reasoning, and mathematical problem solving\. Appendix[A\.1](https://arxiv.org/html/2611.17461#A1.SS1)reports the complete benchmark list, evaluation formats, numbers of examples, and citations\. We classify each benchmark by its dominant task demand\. A benchmark isknowledge\-centricwhen most items primarily require factual retrieval or verification with limited intermediate transformation\. A benchmark isreasoning\-centricwhen most items require composition, inference, calculation, comparison, or state tracking\. This assignment is based on the benchmark design and required operations and is made independently of the analyses conducted in this paper\. The labels describe dominant tendencies rather than disjoint categories, as many benchmarks require both knowledge and reasoning\. Appendix[A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1)provides the complete assignment and discusses mixed cases\.

##### Scope\.

The capability landscape includes both multiple\-choice and free\-form benchmarks, but probing, representation similarity, andNota\(none of the above\) analyses use multiple\-choice tasks because fixed candidate sets enable controlled representation extraction and answer\-option perturbation\. We restrict the study to English to avoid confounding task demands with differences in tokenisation, language coverage, and translation\. These choices provide a controlled diagnostic setting\.

### 3\.2Profiling Representations and Internal States of Models

We propose a task\-centric framework for characterising the model\-internal processing elicited by a task in an unmodified LLM\. It does not directly measure knowledge or reasoning capability; rather, it compares internal patterns across tasks with different balances of retrieval and intermediate transformation\. We combine layer\-wise attention entropy and feed\-forward activation sparsity, their trajectories during decoding, and representation\-based evidence from probing and correct–incorrect answer comparisons\.

##### Static Profile\.

We define a static profile as the benchmark\-level pattern of layer\-wise attention entropy and activation sparsity measured before answer generation\. For each benchmark, we process all available questions with Qwen2\.5\-7B\-Instruct without generating an answer\. We select Qwen2\.5\-7B\-Instruct as the reference model because it provides a tractable mid\-sized, open\-weight setting for extracting layer\-wise internal states across all 24 benchmarks\. To reduce the risk that the observed representational patterns are specific to this model family, the subsequent probing and representation analyses additionally examine Llama3\.1\-8B\-Instruct and OLMo\-3\-7B\-Instruct\. We extract the layer\-output hidden state and attention distribution at the final input\-token position\. No prompt template or instruction is added in this analysis, so all 24 benchmarks share the same input format\. We use attention entropy to characterise how concentrated or diffuse the model’s attention is across the input at each layer\. The layer\-wise entropy profile therefore serves as a descriptive measure of how selectively the model allocates attention when processing tasks with different retrieval and reasoning demands\.

Letai,j\(l,h\)a^\{\(l,h\)\}\_\{i,j\}denote the attention weight assigned to input positionjjby headhhin layerll, when the query is the final input token of exampleii\. We first average the attention distribution across theMMheads:

a¯i,j\(l\)=1M​∑h=1Mai,j\(l,h\)\.\\bar\{a\}^\{\(l\)\}\_\{i,j\}=\\frac\{1\}\{M\}\\sum\_\{h=1\}^\{M\}a^\{\(l,h\)\}\_\{i,j\}\.\(1\)
We normalise this distribution as

pi,j\(l\)=a¯i,j\(l\)∑ka¯i,k\(l\)\+δ,p^\{\(l\)\}\_\{i,j\}=\\frac\{\\bar\{a\}^\{\(l\)\}\_\{i,j\}\}\{\\sum\_\{k\}\\bar\{a\}^\{\(l\)\}\_\{i,k\}\+\\delta\},\(2\)whereδ=10−10\\delta=10^\{\-10\}\. The layer\-wise attention entropy is then

Hl=−1N∑i=1N∑jpi,j\(l\)log\(pi,j\(l\)\+δ\),H\_\{l\}=\-\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\sum\_\{j\}p^\{\(l\)\}\_\{i,j\}\\log\\left\(p^\{\(l\)\}\_\{i,j\}\+\\delta\\right\),\(3\)whereNNis the number of examples\.

We use activation sparsity to characterise how selectively the model’s hidden representation is activated at each layer\. Higher sparsity indicates that a larger proportion of hidden dimensions remain near zero, providing a descriptive measure of how concentrated or distributed the layer\-wise representation is across tasks\.

Letui,r\(l\)u^\{\(l\)\}\_\{i,r\}denote dimensionrrof the layer\-output hidden state at the final input\-token position, and letdddenote the model hidden size\. We compute hidden state sparsity as

Sl=1N​∑i=1N1d​∑r=1d𝕀⁡\(\|ui,r\(l\)\|<0\.01\)\.S\_\{l\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\frac\{1\}\{d\}\\sum\_\{r=1\}^\{d\}\\mathbb\{I\}\\left\(\\left\|u^\{\(l\)\}\_\{i,r\}\\right\|<0\.01\\right\)\.\(4\)
The dataset\-level static feature vector is

Fd=\[H1,…,HL,S1,…,SL\],F\_\{d\}=\\left\[H\_\{1\},\\ldots,H\_\{L\},S\_\{1\},\\ldots,S\_\{L\}\\right\],\(5\)whereLLis the number of Transformer layers\. These features summarise the final\-token attention distribution and layer\-output hidden state\. They are descriptive and are not interpreted as causal measures of knowledge retrieval or reasoning\.

##### Decoding\-Time Trajectories\.

This analysis is intended to characterise how internal processing evolves during answer generation, rather than only how it differs across layers at input time\. We trace the same metrics over generated tokens for all available SciQ examples using Qwen2\.5\-7B\-Instruct, and repeat the analysis on two reasoning\-centric benchmarks, MathQA and LogiQA\. We use SciQ as the primary controlled diagnostic dataset because its questions are short and relatively homogeneous, reducing confounds from variable generation lengths and heterogeneous reasoning strategies\. The two reasoning\-centric benchmarks test whether the resulting signal extends beyond knowledge retrieval\. All three benchmarks use the retrieve\-then\-answer prompt given in Figure[12](https://arxiv.org/html/2611.17461#A1.F12)\. We compare the token\-wise mean trajectories and within\-trajectory variability of attention entropy and activation sparsity between correct and incorrect generations\. We report layers 3, 14, and 22 as representative lower, middle, and upper layers\. Generation uses temperature0\.70\.7, top\-pp0\.80\.8, and a maximum of 1,024 new tokens; generations that are truncated or contain no parsable answer are excluded\. Signals are read at the positions of the generated tokens: after sampling, the prompt and sampled tokens are passed through the model once, which, under causal masking, reproduces the internal state at each decoding step\. To compare responses of different lengths, we sampleP=101P=101uniformly spaced positions from each generation of lengthGG\. ForG\>1G\>1, the selected positions are

ik=⌊k⁡\(G−1\)P−1⌋i\_\{k\}=\\left\\lfloor\\frac\{k\(G\-1\)\}\{P\-1\}\\right\\rfloor\(6\)wherek∈\{0,…,P−1\}k\\in\\\{0,\\ldots,P\-1\\\}\.

### 3\.3Probing and Representation Analysis

To complement the static profile and decoding\-time trajectory analyses, we use linear probing and representation similarity analysis to examine two questions: \(1\) at which layers is the correct answer option linearly recoverable from pre\-generation hidden states, and \(2\) how distinctly does the model represent a question paired with its correct answer versus the same question paired with an incorrect distractor?

We conduct these probing and representation analyses on a balanced diagnostic subset of four multiple\-choice benchmarks: two knowledge\-centric tasks \(BioMixQA and SciQ\) and two reasoning\-centric tasks \(LogiQA and MathQA\)\. We use multiple\-choice benchmarks because their fixed candidate sets enable controlled extraction and comparison of correct and incorrect answer representations\. The analyses are conducted using Llama3\.1\-8B\-Instruct, Qwen2\.5\-7B\-Instruct, and OLMo\-3\-7B\-Instruct\.

##### Layer\-Wise Probing\.

For each question, we perform one forward pass using the templates in Figure[13](https://arxiv.org/html/2611.17461#A1.F13)and extract the hidden state at the final input position before answer generation\. At each layer, we train a logistic regression classifier to predict the correct answer option using 5\-fold cross\-validation and report average held\-out accuracy\. We compare this accuracy with the majority\-class baseline, obtained by always predicting the most frequent correct answer label in each benchmark\. We track accuracy across layers to examine when the correct answer becomes linearly recoverable in each task group\. Accuracy close to this baseline does not show that reasoning information is absent\. It indicates only that the selected linear probe does not reliably recover the answer under the given prompt and token position\. The task groups use different templates, so task type and template choice are confounded\. We therefore run every benchmark under both templates to compare the task groups under the same prompting conditions\. The model, questions, probing procedure, and token position are held fixed\. From these forward passes, we record the head\-averaged attention of the final input token \(Eq\.[1](https://arxiv.org/html/2611.17461#S3.E1)\)\. We report the share assigned to the question and answer options, whose text is identical in both templates\.

##### Representation Similarity\.

For each questionqq, we append either the correct answera\+a^\{\+\}or an incorrect distractora−a^\{\-\}:

x\+=\[q;a\+\],x−=\[q;a−\]\.x^\{\+\}=\[q;a^\{\+\}\],\\qquad x^\{\-\}=\[q;a^\{\-\}\]\.\(7\)
We extract their final\-token hidden states at each layer and compute an adjusted cosine similarity:

simadj⁡\(h\+,h−\)=cos⁡\(h\+,h−\)−𝔼i,j​\[cos⁡\(hi,hj\)\],\\operatorname\{sim\}\_\{\\mathrm\{adj\}\}\(h^\{\+\},h^\{\-\}\)=\\cos\(h^\{\+\},h^\{\-\}\)\-\\mathbb\{E\}\_\{i,j\}\\left\[\\cos\(h\_\{i\},h\_\{j\}\)\\right\],\(8\)where the expectation is computed over randomly paired completions from the same benchmark\. We use this measure to characterise how distinctly the model represents the correct answer relative to a plausible incorrect alternative\. Comparing this separation across task types allows us to test whether knowledge\-centric and reasoning\-centric benchmarks differ systematically in the geometry of their answer representations\. This measure captures relative representational similarity and does not establish causal separation between correct and incorrect answers\.

### 3\.4Capability Scores and Evaluation

Letsm,p,ds\_\{m,p,d\}denote the performance of modelmm, under prompting conditionpp, on benchmarkdd\. We compute the knowledge and reasoning scores as unweighted averages over these reported evaluation settings:

Km,p\\displaystyle K\_\{m,p\}=1\|𝒟K\|​∑d∈𝒟Ksm,p,d,\\displaystyle=\\frac\{1\}\{\|\\mathcal\{D\}\_\{K\}\|\}\\sum\_\{d\\in\\mathcal\{D\}\_\{K\}\}s\_\{m,p,d\},\(9\)Rm,p\\displaystyle R\_\{m,p\}=1\|𝒟R\|​∑d∈𝒟Rsm,p,d\.\\displaystyle=\\frac\{1\}\{\|\\mathcal\{D\}\_\{R\}\|\}\\sum\_\{d\\in\\mathcal\{D\}\_\{R\}\}s\_\{m,p,d\}\.\(10\)where \(\|𝒟K\|=10\|\\mathcal\{D\}\_\{K\}\|=10\) and \(\|𝒟R\|=20\|\\mathcal\{D\}\_\{R\}\|=20\)\. Each reported setting therefore contributes equally to the corresponding capability score, irrespective of the number of examples it contains\.

For either dimensionX∈\{K,R\}X\\in\\\{K,R\\\}, let

Xmin\\displaystyle X\_\{\\min\}=minm,p⁡Xm,p,\\displaystyle=\\min\_\{m,p\}X\_\{m,p\},\(11\)Xmax\\displaystyle X\_\{\\max\}=maxm,p⁡Xm,p\.\\displaystyle=\\max\_\{m,p\}X\_\{m,p\}\.\(12\)and apply min\-max normalisation:

X~m,p=Xm,p−XminXmax−Xmin\.\\widetilde\{X\}\_\{m,p\}=\\frac\{X\_\{m,p\}\-X\_\{\\min\}\}\{X\_\{\\max\}\-X\_\{\\min\}\}\.\(13\)where min/max are taken jointly over model×\\timesprompting condition \(Standard and CoT\), separately for each dimension\. Evaluation scores are expressed on a common\[0,1\]\[0,1\]scale\. We use accuracy for classification and multiple\-choice settings and exact match for the free\-form mathematical benchmarks GSM8K, AIME, MATH500, and AMC\.

We evaluate open\-weight, instruction\-tuned models from the Qwen, Llama, Gemma, OLMo, GLM, and GPT\-OSS families\. The models range from 2B to more than 30B parameters and include both dense and Mixture\-of\-Experts architectures\. Full model details are provided in Appendix[A\.2](https://arxiv.org/html/2611.17461#A1.SS2)\.

All evaluations use EvalScope[ModelScope \(2024\)](https://arxiv.org/html/2611.17461#bib.bib48), with SGLang[Zheng et al\. \(2024\)](https://arxiv.org/html/2611.17461#bib.bib47)as the inference backend\. We follow the benchmark\-specific zero\-shot and few\-shot settings provided by the evaluation framework\. We use accuracy for classification and multiple\-choice tasks and exact match for free\-form mathematical tasks\. Standard and CoT conditions use the same evaluation configuration except for the prompting instruction\.

![Refer to caption](https://arxiv.org/html/2611.17461v1/figure1.png)Figure 1:Layer\-wise static internal profiles of Qwen2\.5\-7B\-Instruct across 24 benchmarks\. The heatmaps show attention entropy \(left\), where higher values indicate more diffuse attention, and activation sparsity \(right\), defined as the proportion of near\-zero feed\-forward activations\. Benchmarks are ordered by the first principal component of the activation sparsity matrix to expose cross\-benchmark variation in their layer\-wise profiles\.Figure 2:Controls for the lower\-layer activation sparsity in Figure[1](https://arxiv.org/html/2611.17461#S3.F1)\(Qwen2\.5\-7B\-Instruct; mean over layers 0\-3 at the final input token\)\. Left and middle: benchmark means against mean prompt length and against the share of prompt tokens that contain a digit or mathematical symbol\. Right: benchmark means under two controls against the original values\. Length\-adjusted values are predicted at a fixed length from a within\-benchmark regression on log length; same\-final\-token values append “\\nAnswer:” to every prompt\. The dashed line marks equality\.Figure 3:Within\-trajectory variability of attention entropy for Qwen2\.5\-7B\-Instruct on SciQ, MathQA, and LogiQA\. For each generation, the standard deviation of attention entropy is computed over the 101 aligned generated\-token positions at the lower, middle, and upper depths \(layers 3, 14, and 22\)\.Figure 4:Decoding\-time trajectory analysis on SciQ using Qwen2\.5\-7B\-Instruct\. Rows show attention entropy \(top\) and activation sparsity \(bottom\)\. Columns separate correct \(left\) and incorrect \(right\) predictions\. Curves show token\-wise mean trajectories at representative lower, middle, and upper layers \(layers 3, 14, and 22\)\. Shaded regions show±1\\pm 1standard deviation across examples within each correctness group\. Generated sequences are length\-normalised to 101 aligned token positions by uniform subsampling\. Incorrect generations are associated with greater within\-trajectory variability in attention entropy, particularly at the lower and upper layers\. The one\-sided tests do not detect greater within\-trajectory variability in activation sparsity for incorrect generations \(Table[3](https://arxiv.org/html/2611.17461#A1.T3)\)\.Figure 5:Layer\-wise linear probing accuracy under the*multiple\-choice*setting across three model families\. Knowledge\-centric datasets show increasing probing accuracy in the mid\-to\-late layers, indicating that the correct option becomes more linearly recoverable\. Prompts are shown in Figure[13](https://arxiv.org/html/2611.17461#A1.F13)\.Figure 6:Template\-swap control for the probing analysis\. Each benchmark is run under both the knowledge\-centric and the reasoning\-centric template of Figure[13](https://arxiv.org/html/2611.17461#A1.F13)\. Bars show the mean over Llama3\.1\-8B\-Instruct, Qwen2\.5\-7B\-Instruct, and OLMo\-3\-7B\-Instruct; markers show individual models\. Left: peak layer\-wise probing accuracy; dashed lines mark the majority\-class baseline\. Right: share of the final token’s attention, averaged over heads and layers, on the question and answer options\.Figure 7:Representation geometry of correct and distractor answers\. Curves show the adjusted cosine similarity between hidden states for prompts appended with the ground\-truth answer versus an incorrect distractor; shaded regions indicate variability across examples\. Lower values indicate greater representational separation\. Knowledge\-centric benchmarks \(BioMixQA and SciQ\) generally show lower similarity than reasoning\-centric benchmarks \(LogiQA and MathQA\) across the three models\.Figure 8:Robustness toNotaperturbations on MathQA under Standard and CoT prompting\. Hollow markers \(∘\\circ\) indicate Base accuracy with the original answer options, while solid markers \(∙\\bullet\) show accuracy after theNota\-as\-answer orNota\-as\-distractor perturbation; connecting lines indicate the resulting performance change\.Nota\-as\-answer produces substantially larger degradation thanNota\-as\-distractor for most models, with the effect varying across prompting conditions\.Figure 9:Robustness toNotaperturbations on SciQ under Standard and CoT prompting\. Hollow markers \(∘\\circ\) indicate Base accuracy with the original answer options, while solid markers \(∙\\bullet\) show accuracy under theNota\-as\-answer orNota\-as\-distractor condition; connecting lines indicate the performance change\.Nota\-as\-answer causes a pronounced accuracy drop across models, whereas addingNotaas a distractor has comparatively little effect\.Figure 10:For each model and prompting condition, raw performance is first averaged over the corresponding knowledge or reasoning evaluation settings \(Eqs\.[9](https://arxiv.org/html/2611.17461#S3.E9)\-[10](https://arxiv.org/html/2611.17461#S3.E10)\)\. Each resulting dimension is then min\-max normalised jointly across all model and prompting\-condition pairs \(Eq\.[13](https://arxiv.org/html/2611.17461#S3.E13)\)\. Hollow \(∘\\circ\) and solid \(∙\\bullet\) markers denote Standard and CoT prompting, respectively, and arrows indicate the change induced by CoT\.

## 4Knowledge\-Reasoning Patterns

##### RQ1: Do tasks with different balances of knowledge retrieval and reasoning produce systematically different layer\-wise and decoding\-time patterns in an unmodified LLM?

##### Internal Mechanics\.

We first examine whether an LLM exhibits a consistent, data\-driven internal pattern that differentiates tasks\. To establish a baseline internal profile, we perform aquestion\-onlyanalysis using Qwen2\.5\-7B\-Instruct\. For each dataset, we feed the model only the input question \(without generating an answer\) and summarise its layer\-wise internal activity into a single static profile\. This design isolates task comprehension and retrieval demands from decoding\-dependent effects such as variable generation length, sampling stochasticity, or branching during answer construction\. As a result, differences in the profile reflect how the model processes a fixed input, rather than decoding\-dependent variability\.

We compute attention entropy and activation sparsity at each layer\. Attention entropy measures how concentrated the attention distribution is\. Higher values indicate more diffuse attention\. Activation sparsity is the proportion of feed\-forward activations that are close to zero\. We concatenate these values across layers to form a feature vector for each dataset\. Figure[1](https://arxiv.org/html/2611.17461#S3.F1)shows these profiles for all 24 datasets\. We order the datasets by the first principal component of the activation sparsity matrix\. This produces a smooth progression in sparsity across datasets and layers\. Under this ordering, activation sparsity shows a clearer pattern in the lower layers than attention entropy\. LogiQA, TruthfulQA, and PIQA have the highest sparsity in the lower layers\. The mathematical benchmarks AIME, GSM8K, MATH500, AMC, and MathQA fall in the middle of the range\. The lowest values occur in both task groups\. These include knowledge\-centric benchmarks such as SciQ and MedMCQA and reasoning\-centric benchmarks such as CommonsenseQA and SIQA\. Figure[1](https://arxiv.org/html/2611.17461#S3.F1)describes how the model processes the input of each benchmark\. The controls below show that adding the same suffix to every input reduces the differences in lower\-layer sparsity across benchmarks\. This indicates that these measurements are sensitive to how the input ends\. We therefore use probing, decoding trajectories, representation geometry, and behavioural analyses to compare knowledge and reasoning\.

##### Prompt Length and Token\-Composition Controls\.

Because benchmarks differ in input length, vocabulary, and formatting, we test whether these input properties explain the differences in lower\-layer sparsity in Figure[1](https://arxiv.org/html/2611.17461#S3.F1)\. We run the model again on the same questions from each benchmark, one question at a time\. We record the prompt length and token composition for each question \(Figure[2](https://arxiv.org/html/2611.17461#S3.F2)\)\. Across the 24 benchmarks, lower\-layer sparsity \(layers 0 to 3\) has a moderate correlation with mean prompt length \(Spearmanρ=0\.43\\rho=0\.43\)\. It shows little correlation with the proportion of tokens containing digits or mathematical symbols \(ρ=−0\.09\\rho=\-0\.09\)\. We fit a regression of sparsity on log prompt length within each benchmark\. We then estimate sparsity at a common prompt length, which is the median across all questions\. This adjustment leaves the benchmark ordering nearly unchanged \(ρ=0\.92\\rho=0\.92with the original ordering\)\. We measure sparsity at the final input token\. To test whether this token affects the measurements, we append the same suffix \(“\\nAnswer:”\) to every prompt\. This makes all prompts end with the same token\. The standard deviation of lower\-layer sparsity across benchmarks decreases\. The rank correlation with the original benchmark ordering falls toρ=0\.37\\rho=0\.37\. We find no statistically significant difference in lower\-layer sparsity between knowledge\-centric and reasoning\-centric benchmarks\. This holds both before and after the suffix is added \(Mann\-Whitneyp=\.19p=\.19andp=\.21p=\.21, respectively\)\. These results indicate that the lower\-layer differences in Figure[1](https://arxiv.org/html/2611.17461#S3.F1)depend on how each benchmark’s input ends\. We therefore do not draw conclusions about knowledge and reasoning from lower\-layer sparsity alone\.

##### Decoding\-time Trajectories\.

Using the controlled SciQ setup described in §[3\.2](https://arxiv.org/html/2611.17461#S3.SS2), we examine how attention entropy and activation sparsity evolve during generation for correct and incorrect predictions\. SciQ is additionally evaluated with a structured retrieve\-then\-answer prompt \(Figure[12](https://arxiv.org/html/2611.17461#A1.F12)\), providing a consistent temporal scaffold for interpreting these trajectories\.

Figure[4](https://arxiv.org/html/2611.17461#S3.F4)visualises the evolution of internal signals during generation on SciQ\. Here, themean trajectoryrepresents the token\-wise average of the metric across examples after length normalisation\. A key observation is thatmean levels aloneoften fail to distinguish correct from incorrect outputs\. The diagnostic signal lies primarily in differences inwithin\-trajectory variability\. Correct predictions display a clearer late\-stage drop in attention entropy \(reflecting answer consolidation\)\. However, the separation in mean levels is modest\. Incorrect generations show greater within\-trajectory variability in attention entropy\. On SciQ, incorrect generations are associated with greater within\-trajectory variability in attention entropy, particularly at the lower and upper layers \(Table[3](https://arxiv.org/html/2611.17461#A1.T3)\)\. The one\-sided tests do not detect greater within\-trajectory variability in activation sparsity for incorrect generations than for correct ones\.

Figure[3](https://arxiv.org/html/2611.17461#S3.F3)shows that this pattern is not specific to SciQ\. As on SciQ, incorrect generations on MathQA and LogiQA show a larger within\-trajectory standard deviation of attention entropy than correct ones\. Under the one\-sided Mann\-Whitney test used in Table[3](https://arxiv.org/html/2611.17461#A1.T3), the difference is significant at the lower and upper layers on all three benchmarks \(q<\.01q<\.01\)\. The difference is also significant at the middle layer on MathQA and LogiQA\. Activation sparsity provides no consistent signal across benchmarks\. Its within\-trajectory variability differs between correct and incorrect generations at some layers on MathQA and LogiQA\. On SciQ, the one\-sided tests do not detect greater sparsity variability in incorrect generations\.

##### Probing and Geometric Analysis\.

The trajectory analysis suggests that correct and incorrect generations differ more in within\-trajectory variability than in mean signal levels, particularly for attention entropy\. We next examine how reliably a linear probe can recover the correct answer option from astatichidden state before generation in a multiple\-choice setting\.

Figure[5](https://arxiv.org/html/2611.17461#S3.F5)reveals a clear contrast in pre\-generation answer recoverability across the two task groups\. On the knowledge\-centric benchmarks BioMixQA and SciQ, probing accuracy rises consistently through the mid\-to\-late layers across the three model families, indicating that the correct option becomes progressively more linearly accessible before answer generation\. In contrast, probing accuracy on LogiQA and MathQA stays close to the majority\-class baseline for Llama3\.1\-8B\-Instruct and OLMo\-3\-7B\-Instruct \(peak 0\.29\-0\.34 against baselines of 0\.24 and 0\.28\)\. Qwen2\.5\-7B\-Instruct is the exception: its accuracy rises above chance from around layer 20, peaking at 0\.46 on LogiQA and 0\.35 on MathQA, although its gains over the baseline \(0\.18 and 0\.11\) remain smaller than on BioMixQA and SciQ \(0\.22 and 0\.69\)\.

Under the probing protocol defined in §[3\.3](https://arxiv.org/html/2611.17461#S3.SS3), these results indicate that answer\-discriminative information is more linearly recoverable from pre\-generation hidden states for the two knowledge\-centric tasks than for the two reasoning\-centric tasks\. This should not, however, be interpreted as evidence that reasoning information is absent from the latter representations; rather, the correct answer is not reliably recoverable from them using the selected prompt, final\-token representation, and linear probe\.

##### Effect of Prompt Structure\.

Figure[6](https://arxiv.org/html/2611.17461#S3.F6)shows that using different templates for the two task groups does not fully explain the probing contrast in Figure[5](https://arxiv.org/html/2611.17461#S3.F5)\. Averaged over Llama3\.1\-8B\-Instruct, Qwen2\.5\-7B\-Instruct, and OLMo\-3\-7B\-Instruct, BioMixQA and SciQ retain higher peak probing accuracy than LogiQA and MathQA under both templates\. The reasoning\-centric template attracts more attention to its instructions and response cues\. Attention to the question and answer options remains nearly unchanged\. The extra instructions therefore change how the model allocates attention without closing the gap in answer recoverability\. This supports the knowledge\-reasoning comparison for these benchmarks under the two tested templates\.

##### Distinctness of Representations\.

While linear probing reveals whether correctness isseparablefrom hidden states, it does not quantify howfar apartcorrect and incorrect completions lie in representation space\. We therefore compare the adjusted correct–incorrect representation similarity defined in §[3\.3](https://arxiv.org/html/2611.17461#S3.SS3)across the four diagnostic benchmarks\. In Figure[7](https://arxiv.org/html/2611.17461#S3.F7), BioMixQA and SciQ show lower adjusted cosine similarity, whereas LogiQA and MathQA show higher similarity\. This pattern is consistent with greater semantic separation between factual answer options and greater lexical or structural overlap between reasoning options\. It may reflect both task demands and distractor construction\. The result therefore shows an association between benchmark type and representation geometry, not a causal or structural separation between truth and falsehood\. One possible explanation is that factual distractors often differ from correct answers in entity or semantic content, whereas reasoning distractors are frequently near\-miss solutions with similar wording and structure\. The observed geometry may therefore reflect both task demands and answer\-option construction\. It should not be interpreted as evidence that truth and falsehood are causally or structurally separated in the early layers\.

The analyses in this section provide complementary forms of evidence\. The static analysis compares all benchmarks at input time, the trajectory analysis examines correct and incorrect generations on SciQ, MathQA, and LogiQA, and the probing and geometric analyses examine four multiple\-choice benchmarks\. Their agreement supports an association between benchmark demands and internal processing patterns\. However, prompt format, answer semantics, distractor construction, and dataset composition may also contribute to the observed differences\. Because we do not perform activation patching, layer ablation, or causal tracing, these results identify diagnostic associations rather than causal mechanisms\.

## 5Knowledge\-Reasoning Landscape

We now turn from internal task characterisation to model\-level capability profiling\. RQ2 examines how model family, scale, and prompting shift models along the knowledge–reasoning dimensions and whether apparent gains remain robust to answer\-option perturbations\.

##### RQ2\.1: How do the knowledge and reasoning profiles of LLMs vary with model family and scale?

##### Model Family\.

Figure[10](https://arxiv.org/html/2611.17461#S3.F10)shows the patterns of different model families in the knowledge\-reasoning plane: models from the same family tend to occupy similar regions, indicating that successive architectural revisions can influence performance as much as, and sometimes more than, model size\. More recent families tend to achieve higher scores on both axes, placing them nearer the upper\-right region of the plane\. In particular, Qwen3 variants are among the strongest overall performers, while Gemma3 variants consistently outperform earlier Gemma versions along both knowledge and reasoning axes\. Within the Gemma family, newer models exhibit a near\-linear progression in the knowledge\-reasoning plane\.

##### Scaling\.

Within a given family, increasing model size improves performance, but the gains are uneven across the two axes\. Larger models move rightward more consistently, indicating that knowledge scores tend to increase more reliably with scale\. Improvements in reasoning are less monotonic and often exhibit smaller marginal returns at larger sizes\. The Gemma3 series illustrates this pattern: scaling from 3B to 27B yields a substantial gain in knowledge but only a modest improvement in reasoning\. At the same time, smaller models \(e\.g\., Gemma3\-4B, Qwen2\.5\-3B\) already achieve competitive reasoning scores\. Within the evaluated model families, knowledge scores increase more consistently with parameter count than reasoning scores\. However, model size co\-varies with architecture, training data, instruction tuning, and other design choices\. These comparisons therefore do not isolate a causal effect of scale\. The result should be interpreted as a descriptive pattern: larger models show more reliable gains on the knowledge axis, whereas reasoning gains vary more across model families\.

##### RQ2\.2: What is the impact of CoT prompting on the internal knowledge\-reasoning balance, and does it introduce performance trade\-offs?

##### Knowledge\-Reasoning Trade\-off\.

Figure[10](https://arxiv.org/html/2611.17461#S3.F10)compares each model under standard prompting \(hollow marker∘\\circ\) and CoT prompting \(solid marker∙\\bullet\), with an arrow \(→\\rightarrow\) showing the change\. The main pattern is simple:*most arrows point upward*, meaning CoT usually increases the reasoning score\. The change in knowledge is smaller and less consistent: for many models the arrow is close to vertical \(little change in knowledge\), while for several models the arrow also moves to the right \(a small gain in knowledge alongside improved reasoning\)\. However, CoT is not always beneficial\. A clear counterexample is Gemma\-7B, which moves down\-left under CoT, indicating a drop in both reasoning and knowledge\. CoT increases reasoning scores in 14 of the 16 checkpoints and knowledge scores in 9\. Paired model\-level tests detect a significant increase in reasoning scores but do not detect a significant change in knowledge scores \(Appendix[A\.4](https://arxiv.org/html/2611.17461#A1.SS4), Table[4](https://arxiv.org/html/2611.17461#A1.T4)\)\. CoT can still reduce performance for individual models, as illustrated by Gemma\-7B\.

##### RQ2\.3: Does CoT prompting mitigate parametric shortcuts?

##### Experiment Settings\.

To disentangle*parametric shortcuts*\(answering by matching memorised option patterns\) from*dynamic reasoning*\(deriving an answer independent of the option set\), we adopt a “none of the above” \(Nota\) perturbation stress test, inspired by[Wang et al\. \(2025a\)](https://arxiv.org/html/2611.17461#bib.bib60)\. We evaluate two complementary perturbations under zero\-shot multiple\-choice evaluation: \(i\)Nota\-as\-answer, where we remove the ground\-truth option and makeNotathe correct choice; and \(ii\)Nota\-as\-distractor, where we injectNotaas an additional incorrect option\. We compare Standard prompting against CoT prompting across six instruction\-tuned LLMs \(Llama3\.1, Qwen2\.5, Olmo3, Gemma3 4B/12B/27B\) on MathQA and SciQ\. We report accuracy and visualise robustness as the gap between Base \(original options\) and each perturbed condition \(Figures[8](https://arxiv.org/html/2611.17461#S3.F8)and[9](https://arxiv.org/html/2611.17461#S3.F9)\)\.

##### Robustness to Option Manipulation\.

On the MathQA dataset, CoT frequently improves robustness to option\-set manipulations\. For several models \(notably Qwen2\.5 and OLMo3\), CoT both increases Base accuracy and reduces the drop underNota\-as\-answer, consistent with reduced reliance on specific answer options\. However, this effect is not universal: Llama3\.1 still shows a large gap between Base andNota\-as\-answer even with CoT, indicating that correctness continues to depend on the presence of the original correct option\. This residual gap is consistent with partially option\-dependent CoT, in which plausible rationales are produced while some reliance on option matching remains\. SciQ exhibits a more consistent pattern across models\. AddingNotaas a distractor has only a small effect on accuracy under both Standard and CoT, suggesting that merely inserting an additional wrong option does not substantially disrupt model behaviour\. In contrast,Nota\-as\-answer causes a pronounced drop across models, directly revealing reliance on the correct option’s presence in the candidate set\. Paired item\-level tests confirm this asymmetry across model, dataset, and prompting conditions \(Appendix[A\.4](https://arxiv.org/html/2611.17461#A1.SS4); Table[5](https://arxiv.org/html/2611.17461#A1.T5)\)\. CoT can raise Base accuracy and sometimes narrows this gap, but the remaining degradation indicates that option dependence is reduced rather than eliminated\. Overall, CoT mitigates parametric shortcutsto the extent that it reduces option dependence\. MathQA shows that CoT can improve robustness for some models, while SciQ confirms a strong and broadly shared failure mode: performance collapses when the ground\-truth option is removed, and CoT only partially alleviates this\. Therefore,Notastress tests provide a necessary complement to standard leaderboard accuracy for diagnosing whether CoT gains reflect genuinely option\-invariant reasoning\.

## 6Conclusion

In this paper, we introduced a task\-centric framework that combines layer\-wise, decoding\-time, representation\-based, and behavioural signals to characterise the demands that benchmarks place on LLMs\. Across 16 models and 24 benchmarks, we find that knowledge\- and reasoning\-centric tasks exhibit different internal processing patterns, and that these differences are not fully captured by aggregate task performance alone\. At the behavioural level, knowledge performance scales more consistently than reasoning performance\. Paired model\-level tests detect a significant increase in reasoning scores under CoT\. They do not detect a significant change in knowledge scores\. At the same time, theNotaperturbation analysis shows that improved performance under CoT does not necessarily imply option\-invariant reasoning: models can remain strongly dependent on the presence of the original ground\-truth answer\. Taken together, these results illustrate the value of analysing model behaviour along multiple complementary axes\. Static layer\-wise profiles characterise how task inputs are processed before generation\. Decoding\-time trajectories reveal differences in within\-trajectory variability\. Representation\-based and perturbation analyses expose distinctions that may remain hidden in final task accuracy\. Rather than treating knowledge and reasoning as disjoint capabilities, we provide a view of how their relative demands vary across tasks, models, and prompting conditions\.

## Code and Data Availability

Our codebase for reproducing the figures and tables are available at[https://github\.com/extraordinarylab/trace\-kr](https://github.com/extraordinarylab/trace-kr)\. To ensure accessibility, all figures in this work utilise the colour\-blind safe Okabe\-Ito palette\.

## Limitations

Our study is limited to English benchmarks, and the probing, representation\-similarity, andNotaanalyses rely on multiple\-choice answer structure\. Cross\-lingual differences in tokenisation and model coverage may change the observed footprints, while open\-ended generation may exhibit different reasoning paths and shortcut behaviours\. These settings define the scope of our conclusions and motivate future multilingual and open\-ended evaluation\.

## Acknowledgments

We are grateful to the action editor and the anonymous reviewers for their thoughtful feedback and valuable suggestions, which have helped improve the quality and clarity of this work\. This work was supported by UKRI Metascience\-funded RISC project \(APP111963\)\. Eleni Vasilaki was supported by the UK Neuromorphic Computing Hardware Semiconductor IKC \(Neuroware; UKRI2784\), funded by EPSRC and Innovate UK\.

## References

- A\. Amini, S\. Gabriel, S\. Lin, R\. Koncel\-Kedziorski, Y\. Choi, and H\. HajishirziMathQA: towards interpretable math word problem solving with operation\-based formalisms\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),J\. Burstein, C\. Doran, and T\. Solorio \(Eds\.\),Minneapolis, Minnesota,pp\. 2357–2367\.External Links:[Link](https://aclanthology.org/N19-1245/),[Document](https://dx.doi.org/10.18653/v1/N19-1245)Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px18.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.19.1.1)\.
- Ashuachet al\.\(2026\)T\. Ashuach, S\. Gretz, Y\. Katz, Y\. Belinkov, and L\. Ein\-DorMasked by consensus: disentangling privileged knowledge in LLM correctness\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 10577–10596\.External Links:[Link](https://aclanthology.org/2026.acl-long.483/),[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.483),ISBN 979\-8\-89176\-390\-6Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p4.1)\.
- Benjamini and Hochberg \(1995\)Y\. Benjamini and Y\. HochbergControlling the false discovery rate: a practical and powerful approach to multiple testing\.Journal of the Royal Statistical Society: Series B \(Methodological\)57\(1\),pp\. 289–300\.External Links:ISSN 0035\-9246,[Document](https://dx.doi.org/10.1111/j.2517-6161.1995.tb02031.x),[Link](https://doi.org/10.1111/j.2517-6161.1995.tb02031.x),https://academic\.oup\.com/jrsssb/article\-pdf/57/1/289/49173396/jrsssb\_57\_1\_289\.pdfCited by:[§A\.4](https://arxiv.org/html/2611.17461#A1.SS4.SSS0.Px2.p1.1),[Table 3](https://arxiv.org/html/2611.17461#A1.T3)\.
- Bisket al\.\(2020\)Y\. Bisk, R\. Zellers, R\. L\. Bras, J\. Gao, and Y\. ChoiPiqa: reasoning about physical commonsense in natural language\.InProceedings of the AAAI conference on artificial intelligence,External Links:[Link](https://cdn.aaai.org/ojs/6239/6239-13-9464-1-10-20200516.pdf)Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px15.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.16.1.1)\.
- Chenet al\.\(2024\)Q\. Chen, L\. Qin, J\. Wang, J\. Zhou, and W\. CheUnlocking the capabilities of thought: a reasoning boundary framework to quantify and optimize chain\-of\-thought\.Advances in Neural Information Processing Systems37,pp\. 54872–54904\.Cited by:[§2\.3](https://arxiv.org/html/2611.17461#S2.SS3.p1.1)\.
- Chenet al\.\(2025\)Y\. Chen, J\. Benton, A\. Radhakrishnan, J\. Uesato, C\. Denison, J\. Schulman, A\. Somani, P\. Hase, M\. Wagner, F\. Roger, V\. Mikulik, S\. R\. Bowman, J\. Leike, J\. Kaplan, and E\. PerezReasoning models don’t always say what they think\.External Links:2505\.05410,[Link](https://arxiv.org/abs/2505.05410)Cited by:[§2\.1](https://arxiv.org/html/2611.17461#S2.SS1.p1.1)\.
- Clarket al\.\(2018\)P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. TafjordThink you have solved question answering? try arc, the ai2 reasoning challenge\.arXiv preprint arXiv:1803\.05457\.Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px7.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.8.1.1)\.
- Cliff \(1993\)N\. CliffDominance statistics: ordinal analyses to answer ordinal questions\.\.Psychological bulletin114\(3\),pp\. 494\.Cited by:[Table 3](https://arxiv.org/html/2611.17461#A1.T3)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. SchulmanTraining verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px19.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.20.1.1)\.
- Dziriet al\.\(2023\)N\. Dziri, X\. Lu, M\. Sclar, X\. L\. Li, L\. Jiang, B\. Y\. Lin, S\. Welleck, P\. West, C\. Bhagavatula, R\. L\. Bras, J\. D\. Hwang, S\. Sanyal, X\. Ren, A\. Ettinger, Z\. Harchaoui, and Y\. ChoiFaith and fate: limits of transformers on compositionality\.InThirty\-seventh Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=Fkckkr3ya8)Cited by:[§2\.3](https://arxiv.org/html/2611.17461#S2.SS3.p1.1)\.
- Fenget al\.\(2026a\)T\. Feng, X\. Jiang, X\. Hu, Y\. Zhang, Z\. Tao, W\. Zhang, B\. Liu, W\. Jiang, and C\. WuTrust within? seek beyond? knowledge boundary aware policy optimization for agentic search\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 27664–27682\.External Links:[Link](https://aclanthology.org/2026.acl-long.1276/),[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.1276),ISBN 979\-8\-89176\-390\-6Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p2.1)\.
- Fenget al\.\(2026b\)Y\. Feng, Z\. Chen, H\. Wu, J\. Zhou, and A\. BosselutTracking the limits of knowledge propagation: how LLMs fail at multi\-step reasoning with conflicting knowledge\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\),V\. Demberg, K\. Inui, and L\. Marquez \(Eds\.\),Rabat, Morocco,pp\. 5813–5847\.External Links:[Link](https://aclanthology.org/2026.eacl-long.273/),[Document](https://dx.doi.org/10.18653/v1/2026.eacl-long.273),ISBN 979\-8\-89176\-380\-7Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p2.1)\.
- Gevaet al\.\(2021\)M\. Geva, R\. Schuster, J\. Berant, and O\. LevyTransformer feed\-forward layers are key\-value memories\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,M\. Moens, X\. Huang, L\. Specia, and S\. W\. Yih \(Eds\.\),Online and Punta Cana, Dominican Republic,pp\. 5484–5495\.External Links:[Link](https://aclanthology.org/2021.emnlp-main.446/),[Document](https://dx.doi.org/10.18653/v1/2021.emnlp-main.446)Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p4.1),[§2\.2](https://arxiv.org/html/2611.17461#S2.SS2.p1.1)\.
- Gromovet al\.\(2025\)A\. Gromov, K\. Tirumala, H\. Shapourian, P\. Glorioso, and D\. RobertsThe unreasonable ineffectiveness of the deeper layers\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=ngmEcEer8a)Cited by:[§2\.2](https://arxiv.org/html/2611.17461#S2.SS2.p1.1)\.
- Groves and Thompson \(1970\)P\. M\. Groves and R\. F\. ThompsonHabituation: a dual\-process theory\.\.Psychological review77\(5\),pp\. 419\.Cited by:[§2\.1](https://arxiv.org/html/2611.17461#S2.SS1.p1.1)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. SteinhardtMeasuring mathematical problem solving with the MATH dataset\.InThirty\-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track \(Round 2\),External Links:[Link](https://openreview.net/forum?id=7Bywt2mQsCe)Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px21.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.22.1.1)\.
- Honget al\.\(2026\)H\. Hong, C\. Xiao, Y\. Wang, Y\. Liu, W\. Rong, and C\. LinBeyond one\-size\-fits\-all: inversion learning for highly effective nlg evaluation prompts\.Transactions of the Association for Computational Linguistics14,pp\. 689–710\.External Links:ISSN 2307\-387X,[Document](https://dx.doi.org/10.1162/TACL.a.617),[Link](https://doi.org/10.1162/TACL.a.617),https://direct\.mit\.edu/tacl/article\-pdf/doi/10\.1162/TACL\.a\.617/2598657/tacl\.a\.617\.pdfCited by:[§2\.3](https://arxiv.org/html/2611.17461#S2.SS3.p1.1)\.
- Houet al\.\(2023\)Y\. Hou, J\. Li, Y\. Fei, A\. Stolfo, W\. Zhou, G\. Zeng, A\. Bosselut, and M\. SachanTowards a mechanistic interpretation of multi\-step reasoning capabilities of language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 4902–4919\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.299/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.299)Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p4.1),[§2\.2](https://arxiv.org/html/2611.17461#S2.SS2.p1.1),[§2\.3](https://arxiv.org/html/2611.17461#S2.SS3.p1.1)\.
- Jiaet al\.\(2025\)X\. Jia, J\. Li, Z\. Wang, J\. Li, X\. Zeng, Y\. Wang, W\. Zhang, Y\. Yu, and W\. LiuFast, slow, and tool\-augmented thinking for llms: a review\.arXiv preprint arXiv:2508\.12265\.Cited by:[§2\.1](https://arxiv.org/html/2611.17461#S2.SS1.p1.1)\.
- Jimenezet al\.\(2024\)C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. R\. NarasimhanSWE\-bench: can language models resolve real\-world github issues?\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=VTF8yNQM66)Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p1.1)\.
- Jinet al\.\(2025\)M\. Jin, W\. Luo, S\. Cheng, X\. Wang, W\. Hua, R\. Tang, W\. Y\. Wang, and Y\. ZhangDisentangling memory and reasoning ability in large language models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 1681–1701\.External Links:[Link](https://aclanthology.org/2025.acl-long.84/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.84),ISBN 979\-8\-89176\-251\-0Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p2.1),[§1](https://arxiv.org/html/2611.17461#S1.p3.1),[§2\.1](https://arxiv.org/html/2611.17461#S2.SS1.p1.1)\.
- Jinet al\.\(2019\)Q\. Jin, B\. Dhingra, Z\. Liu, W\. Cohen, and X\. LuPubMedQA: a dataset for biomedical research question answering\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\),K\. Inui, J\. Jiang, V\. Ng, and X\. Wan \(Eds\.\),Hong Kong, China,pp\. 2567–2577\.External Links:[Link](https://aclanthology.org/D19-1259/),[Document](https://dx.doi.org/10.18653/v1/D19-1259)Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px4.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.5.1.1)\.
- Kahneman \(2011\)D\. KahnemanThinking, fast and slow\.Farrar, Straus and Giroux\.Cited by:[§2\.1](https://arxiv.org/html/2611.17461#S2.SS1.p1.1)\.
- Kerby \(2014\)D\. S\. KerbyThe simple difference formula: an approach to teaching nonparametric correlation\.Comprehensive Psychology3,pp\. 11–IT\.Cited by:[§A\.4](https://arxiv.org/html/2611.17461#A1.SS4.SSS0.Px1.p1.1),[Table 4](https://arxiv.org/html/2611.17461#A1.T4)\.
- Khotet al\.\(2020\)T\. Khot, P\. Clark, M\. Guerquin, P\. Jansen, and A\. SabharwalQasc: a dataset for question answering via sentence composition\.InProceedings of the AAAI Conference on Artificial Intelligence,Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px6.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.7.1.1)\.
- Laiet al\.\(2017\)G\. Lai, Q\. Xie, H\. Liu, Y\. Yang, and E\. HovyRACE: large\-scale ReAding comprehension dataset from examinations\.InProceedings of the 2017 Conference on Empirical Methods in Natural Language Processing,M\. Palmer, R\. Hwa, and S\. Riedel \(Eds\.\),Copenhagen, Denmark,pp\. 785–794\.External Links:[Link](https://aclanthology.org/D17-1082/),[Document](https://dx.doi.org/10.18653/v1/D17-1082)Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px10.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.11.1.1)\.
- Lanhamet al\.\(2023\)T\. Lanham, A\. Chen, A\. Radhakrishnan, B\. Steiner, C\. Denison, D\. Hernandez, D\. Li, E\. Durmus, E\. Hubinger, J\. Kernion, K\. Lukošiūtė, K\. Nguyen, N\. Cheng, N\. Joseph, N\. Schiefer, O\. Rausch, R\. Larson, S\. McCandlish, S\. Kundu, S\. Kadavath, S\. Yang, T\. Henighan, T\. Maxwell, T\. Telleen\-Lawton, T\. Hume, Z\. Hatfield\-Dodds, J\. Kaplan, J\. Brauner, S\. R\. Bowman, and E\. PerezMeasuring faithfulness in chain\-of\-thought reasoning\.External Links:2307\.13702,[Link](https://arxiv.org/abs/2307.13702)Cited by:[§2\.1](https://arxiv.org/html/2611.17461#S2.SS1.p1.1)\.
- Lehmannet al\.\(2026\)H\. H\. Lehmann, J\. H\. Lee, S\. Schockaert, and S\. WermterKnowing the facts but choosing the shortcut: understanding how large language models compare entities\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\),V\. Demberg, K\. Inui, and L\. Marquez \(Eds\.\),Rabat, Morocco,pp\. 4788–4821\.External Links:[Link](https://aclanthology.org/2026.eacl-long.222/),[Document](https://dx.doi.org/10.18653/v1/2026.eacl-long.222),ISBN 979\-8\-89176\-380\-7Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p2.1)\.
- Liet al\.\(2023\)J\. Li, X\. Cheng, X\. Zhao, J\. Nie, and J\. WenHaluEval: a large\-scale hallucination evaluation benchmark for large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 6449–6464\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.397/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.397)Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px12.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.13.1.1)\.
- Liet al\.\(2025\)M\. Li, Y\. Zhao, W\. Zhang, S\. Li, W\. Xie, S\. Ng, T\. Chua, and Y\. DengKnowledge boundary of large language models: a survey\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 5131–5157\.External Links:[Link](https://aclanthology.org/2025.acl-long.256/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.256),ISBN 979\-8\-89176\-251\-0Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p2.1)\.
- Liet al\.\(2026a\)P\. Li, L\. Ding, Z\. Zhou, C\. Zhang, J\. Fu, H\. Li, Y\. Yuan, and G\. WangDemystifying uncertainty in LLMs: active calibration between concepts and human evaluations\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 5726–5759\.External Links:[Link](https://aclanthology.org/2026.acl-long.259/),[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.259),ISBN 979\-8\-89176\-390\-6Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p2.1)\.
- Liet al\.\(2026b\)S\. Li, J\. Han, Y\. Wang, Y\. Zhu, Z\. Song, L\. He, K\. K\. A\. Alghythee, and P\. S\. YuWhy LLMs hallucinate on structured knowledge: a mechanistic analysis of reasoning over linearized representations\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 19943–19956\.External Links:[Link](https://aclanthology.org/2026.acl-long.914/),[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.914),ISBN 979\-8\-89176\-390\-6Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p2.1)\.
- Linet al\.\(2022\)S\. Lin, J\. Hilton, and O\. EvansTruthfulQA: measuring how models mimic human falsehoods\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),S\. Muresan, P\. Nakov, and A\. Villavicencio \(Eds\.\),Dublin, Ireland,pp\. 3214–3252\.External Links:[Link](https://aclanthology.org/2022.acl-long.229/),[Document](https://dx.doi.org/10.18653/v1/2022.acl-long.229)Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px11.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.12.1.1)\.
- Liuet al\.\(2020\)J\. Liu, L\. Cui, H\. Liu, D\. Huang, Y\. Wang, and Y\. ZhangLogiQA: a challenge dataset for machine reading comprehension with logical reasoning\.InProceedings of the Twenty\-Ninth International Joint Conference on Artificial Intelligence, IJCAI\-20,C\. Bessiere \(Ed\.\),pp\. 3622–3628\.External Links:[Document](https://dx.doi.org/10.24963/ijcai.2020/501),[Link](https://doi.org/10.24963/ijcai.2020/501)Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px24.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.25.1.1)\.
- Liuet al\.\(2026\)Y\. Liu, Y\. Wang, S\. Wang, C\. Xiao, and C\. LinBroken symmetry in llm refusal: answer release is more local than refusal restoration\.arXiv preprint arXiv:2608\.15772\.Cited by:[§2\.2](https://arxiv.org/html/2611.17461#S2.SS2.p1.1)\.
- M\-A\-P Teamet al\.\(2026\)M\-A\-P Team, X\. Du, Y\. Yao, K\. Ma, B\. Wang, T\. Zheng, K\. Zhu, M\. Liu, Y\. Liang, X\. Jin, Z\. Wei, C\. Zheng, K\. Deng, S\. Guo, S\. Jia, S\. Jiang, Y\. Liao, R\. Li, Q\. Li, S\. Li, Y\. Li, Y\. Li, D\. Ma, Y\. Ni, H\. Que, Q\. Wang, Z\. Wen, S\. Wu, T\. Xing, M\. Xu, Z\. Yang, Z\. M\. Wang, J\. Zhou, Y\. Bai, X\. Bu, C\. Cai, L\. Chen, Y\. Chen, C\. Cheng, T\. Cheng, K\. Ding, S\. Huang, Y\. Huang, Y\. Li, Y\. Li, Z\. Li, T\. Liang, C\. Lin, H\. Lin, Y\. Ma, Z\. Peng, Z\. Peng, Q\. Qi, S\. Qiu, X\. Qu, Y\. Tan, Z\. Wang, C\. Wang, H\. Wang, Y\. Wang, Y\. Wang, J\. Xu, K\. Yang, R\. Yuan, Y\. Yue, T\. Zhan, C\. Zhang, J\. Zhang, X\. Zhang, X\. Zhang, Y\. Zhang, Y\. Zhao, X\. Zheng, C\. Zhong, Y\. Gao, Z\. Li, D\. Liu, Q\. Liu, T\. Liu, S\. Ni, J\. Peng, Y\. Qin, W\. Su, G\. Wang, S\. Wang, J\. Yang, M\. Yang, M\. Cao, X\. Yue, Z\. Zhang, W\. Zhou, J\. Liu, Q\. Lin, W\. Huang, and G\. ZhangSuperGPQA: scaling LLM evaluation across 285 graduate disciplines\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track,External Links:[Link](https://openreview.net/forum?id=6WgflzYQpf)Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p1.1)\.
- MAA \(2025a\)MAAAIME problems and solutions\.Note:[https://artofproblemsolving\.com/wiki/index\.php/AIME\_Problems\_and\_Solutions](https://artofproblemsolving.com/wiki/index.php/AIME_Problems_and_Solutions)Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px20.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.21.1.1)\.
- MAA \(2025b\)MAAAMC 12 problems and solutions\.Note:[https://artofproblemsolving\.com/wiki/index\.php/AMC\_12\_Problems\_and\_Solutions](https://artofproblemsolving.com/wiki/index.php/AMC_12_Problems_and_Solutions)Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px22.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.23.1.1)\.
- Manakulet al\.\(2023\)P\. Manakul, A\. Liusie, and M\. GalesSelfCheckGPT: zero\-resource black\-box hallucination detection for generative large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 9004–9017\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.557/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.557)Cited by:[§2\.3](https://arxiv.org/html/2611.17461#S2.SS3.p1.1)\.
- McNemar \(1947\)Q\. McNemarNote on the sampling error of the difference between correlated proportions or percentages\.Psychometrika12\(2\),pp\. 153–157\.Cited by:[§A\.4](https://arxiv.org/html/2611.17461#A1.SS4.SSS0.Px2.p1.1),[Table 5](https://arxiv.org/html/2611.17461#A1.T5)\.
- Menget al\.\(2022\)K\. Meng, D\. Bau, A\. Andonian, and Y\. BelinkovLocating and editing factual associations in gpt\.Advances in neural information processing systems35,pp\. 17359–17372\.Cited by:[§2\.2](https://arxiv.org/html/2611.17461#S2.SS2.p1.1)\.
- Menget al\.\(2023\)K\. Meng, A\. S\. Sharma, A\. J\. Andonian, Y\. Belinkov, and D\. BauMass\-editing memory in a transformer\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=MkbcAHIYgyS)Cited by:[§2\.2](https://arxiv.org/html/2611.17461#S2.SS2.p1.1)\.
- Mirzadehet al\.\(2025\)S\. I\. Mirzadeh, K\. Alizadeh, H\. Shahrokhi, O\. Tuzel, S\. Bengio, and M\. FarajtabarGSM\-symbolic: understanding the limitations of mathematical reasoning in large language models\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=AjXkRZIvjB)Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p2.1)\.
- ModelScope \(2024\)ModelScopeEvalScope: evaluation framework for large models\.External Links:[Link](https://github.com/modelscope/evalscope)Cited by:[§3\.4](https://arxiv.org/html/2611.17461#S3.SS4.p4.1)\.
- Nandaet al\.\(2023\)N\. Nanda, L\. Chan, T\. Lieberum, J\. Smith, and J\. SteinhardtProgress measures for grokking via mechanistic interpretability\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=9XFSbDPmdW)Cited by:[§2\.2](https://arxiv.org/html/2611.17461#S2.SS2.p1.1)\.
- Niet al\.\(2025\)S\. Ni, K\. Bi, J\. Guo, L\. Yu, B\. Bi, and X\. ChengTowards fully exploiting LLM internal states to enhance knowledge boundary perception\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 24315–24329\.External Links:[Link](https://aclanthology.org/2025.acl-long.1184/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.1184),ISBN 979\-8\-89176\-251\-0Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p2.1)\.
- Olssonet al\.\(2022\)C\. Olsson, N\. Elhage, N\. Nanda, N\. Joseph, N\. DasSarma, T\. Henighan, B\. Mann, A\. Askell, Y\. Bai, A\. Chen, T\. Conerly, D\. Drain, D\. Ganguli, Z\. Hatfield\-Dodds, D\. Hernandez, S\. Johnston, A\. Jones, J\. Kernion, L\. Lovitt, K\. Ndousse, D\. Amodei, T\. Brown, J\. Clark, J\. Kaplan, S\. McCandlish, and C\. OlahIn\-context learning and induction heads\.arXiv preprint arXiv:2209\.11895\.Cited by:[§2\.2](https://arxiv.org/html/2611.17461#S2.SS2.p1.1)\.
- Palet al\.\(2022\)A\. Pal, L\. K\. Umapathi, and M\. SankarasubbuMedmcqa: a large\-scale multi\-subject multi\-choice dataset for medical domain question answering\.InConference on health, inference, and learning,pp\. 248–260\.Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px2.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.3.1.1)\.
- Presset al\.\(2023\)O\. Press, M\. Zhang, S\. Min, L\. Schmidt, N\. Smith, and M\. LewisMeasuring and narrowing the compositionality gap in language models\.InFindings of the Association for Computational Linguistics: EMNLP 2023,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 5687–5711\.External Links:[Link](https://aclanthology.org/2023.findings-emnlp.378/),[Document](https://dx.doi.org/10.18653/v1/2023.findings-emnlp.378)Cited by:[§2\.1](https://arxiv.org/html/2611.17461#S2.SS1.p1.1)\.
- Reinet al\.\(2024\)D\. Rein, B\. L\. Hou, A\. C\. Stickland, J\. Petty, R\. Y\. Pang, J\. Dirani, J\. Michael, and S\. R\. BowmanGpqa: a graduate\-level google\-proof q&a benchmark\.InFirst Conference on Language Modeling,Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px8.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.9.1.1)\.
- Sakaguchiet al\.\(2021\)K\. Sakaguchi, R\. L\. Bras, C\. Bhagavatula, and Y\. ChoiWinogrande: an adversarial winograd schema challenge at scale\.Communications of the ACM64\(9\),pp\. 99–106\.Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px14.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.15.1.1)\.
- Sapet al\.\(2019\)M\. Sap, H\. Rashkin, D\. Chen, R\. Le Bras, and Y\. ChoiSocial IQa: commonsense reasoning about social interactions\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\),K\. Inui, J\. Jiang, V\. Ng, and X\. Wan \(Eds\.\),Hong Kong, China,pp\. 4463–4473\.External Links:[Link](https://aclanthology.org/D19-1454/),[Document](https://dx.doi.org/10.18653/v1/D19-1454)Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px16.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.17.1.1)\.
- Sedano \(2025\)O\. M\. SedanoMRI\-mcqa\.Note:[https://huggingface\.co/datasets/HPAI\-BSC/MRI\-MCQA](https://huggingface.co/datasets/HPAI-BSC/MRI-MCQA)Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px3.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.4.1.1)\.
- Shiet al\.\(2024\)G\. Shi, Z\. Lu, X\. Dong, W\. Zhang, X\. Zhang, Y\. Feng, and X\. WuUnderstanding layer significance in llm alignment\.CoRRabs/2410\.17875\.External Links:[Link](https://doi.org/10.48550/arXiv.2410.17875)Cited by:[§2\.2](https://arxiv.org/html/2611.17461#S2.SS2.p1.1)\.
- Shiet al\.\(2026\)Z\. Shi, X\. Jiang, C\. Xu, C\. Yao, S\. Ma, Y\. Shen, Z\. Li, J\. Guo, and Y\. WangJudgeAgent: beyond static benchmarks for knowledge\-driven and dynamic LLM evaluation\.InFindings of the Association for Computational Linguistics: ACL 2026,M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 13004–13030\.External Links:[Link](https://aclanthology.org/2026.findings-acl.634/),[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.634),ISBN 979\-8\-89176\-395\-1Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p1.1)\.
- Somanet al\.\(2024\)K\. Soman, P\. W\. Rose, J\. H\. Morris, R\. E\. Akbas, B\. Smith, B\. Peetoom, C\. Villouta\-Reyes, G\. Cerono, Y\. Shi, A\. Rizk\-Jackson, S\. Israni, C\. A\. Nelson, S\. Huang, and S\. E\. BaranziniBiomedical knowledge graph\-optimized prompt generation for large language models\.Bioinformatics40\(9\),pp\. btae560\.Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px1.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.2.1.1)\.
- Songet al\.\(2025\)X\. Song, K\. Wang, P\. Li, L\. Yin, and S\. LiuDemystifying the roles of llm layers in retrieval, knowledge, and reasoning\.External Links:2510\.02091,[Link](https://arxiv.org/abs/2510.02091)Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p4.1)\.
- Stolfoet al\.\(2023\)A\. Stolfo, Y\. Belinkov, and M\. SachanA mechanistic interpretation of arithmetic reasoning in language models using causal mediation analysis\.InThe 2023 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://openreview.net/forum?id=aB3Hwh4UzP)Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p4.1),[§2\.2](https://arxiv.org/html/2611.17461#S2.SS2.p1.1)\.
- Talmoret al\.\(2019\)A\. Talmor, J\. Herzig, N\. Lourie, and J\. BerantCommonsenseQA: a question answering challenge targeting commonsense knowledge\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),J\. Burstein, C\. Doran, and T\. Solorio \(Eds\.\),Minneapolis, Minnesota,pp\. 4149–4158\.External Links:[Link](https://aclanthology.org/N19-1421/),[Document](https://dx.doi.org/10.18653/v1/N19-1421)Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px13.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.14.1.1)\.
- Turpinet al\.\(2023\)M\. Turpin, J\. Michael, E\. Perez, and S\. R\. BowmanLanguage models don’t always say what they think: unfaithful explanations in chain\-of\-thought prompting\.InThirty\-seventh Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=bzs4uPLXvi)Cited by:[§2\.1](https://arxiv.org/html/2611.17461#S2.SS1.p1.1)\.
- Wanget al\.\(2025a\)Y\. Wang, C\. Xiao, C\. Hsiao, Z\. Y\. Chang, C\. Chen, T\. Loakman, and C\. LinDrivel\-ology: challenging LLMs with interpreting nonsense with depth\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 23074–23096\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.1177/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1177),ISBN 979\-8\-89176\-332\-6Cited by:[§5](https://arxiv.org/html/2611.17461#S5.SS0.SSS0.Px7.p1.1)\.
- Wanget al\.\(2025b\)Y\. Wang, C\. Xiao, Y\. Li, S\. E\. Middleton, N\. A\. Moubayed, and C\. LinAdversarial defense without adversarial defense: enhancing language model robustness via instance\-level principal component removal\.Transactions of the Association for Computational Linguistics13,pp\. 1381–1409\.External Links:ISSN 2307\-387X,[Document](https://dx.doi.org/10.1162/TACL.a.43),[Link](https://doi.org/10.1162/TACL.a.43),https://direct\.mit\.edu/tacl/article\-pdf/doi/10\.1162/TACL\.a\.43/2561616/tacl\.a\.43\.pdfCited by:[§2\.3](https://arxiv.org/html/2611.17461#S2.SS3.p1.1)\.
- Wang \(2025\)Y\. WangMusic trivia\.Note:[https://huggingface\.co/datasets/extraordinarylab/music\-trivia](https://huggingface.co/datasets/extraordinarylab/music-trivia)Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px9.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.10.1.1)\.
- Wanget al\.\(2024a\)Y\. Wang, X\. Ma, G\. Zhang, Y\. Ni, A\. Chandra, S\. Guo, W\. Ren, A\. Arulraj, X\. He, Z\. Jiang, T\. Li, M\. Ku, K\. Wang, A\. Zhuang, R\. Fan, X\. Yue, and W\. ChenMmlu\-pro: a more robust and challenging multi\-task language understanding benchmark\.Advances in Neural Information Processing Systems37,pp\. 95266–95290\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/ad236edc564f3e3156e1b2feafb99a24-Abstract-Datasets_and_Benchmarks_Track.html)Cited by:[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.18.1.1),[§1](https://arxiv.org/html/2611.17461#S1.p1.1)\.
- Wanget al\.\(2024b\)Y\. Wang, X\. Ma, G\. Zhang, Y\. Ni, A\. Chandra, S\. Guo, W\. Ren, A\. Arulraj, X\. He, Z\. Jiang, T\. Li, M\. Ku, K\. Wang, A\. Zhuang, R\. Fan, X\. Yue, and W\. ChenMMLU\-pro: a more robust and challenging multi\-task language understanding benchmark\.InProceedings of the 38th International Conference on Neural Information Processing Systems,NIPS ’24,Red Hook, NY, USA\.External Links:ISBN 9798331314385Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px17.p1.1)\.
- Weiet al\.\(2022\)J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, brian ichter, F\. Xia, E\. H\. Chi, Q\. V\. Le, and D\. ZhouChain of thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems,A\. H\. Oh, A\. Agarwal, D\. Belgrave, and K\. Cho \(Eds\.\),External Links:[Link](https://openreview.net/forum?id=_VjQlMeSB_J)Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px23.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.24.1.1),[§2\.1](https://arxiv.org/html/2611.17461#S2.SS1.p1.1)\.
- Welblet al\.\(2017\)J\. Welbl, N\. F\. Liu, and M\. GardnerCrowdsourcing multiple choice science questions\.InProceedings of the 3rd Workshop on Noisy User\-generated Text,L\. Derczynski, W\. Xu, A\. Ritter, and T\. Baldwin \(Eds\.\),Copenhagen, Denmark,pp\. 94–106\.External Links:[Link](https://aclanthology.org/W17-4413/),[Document](https://dx.doi.org/10.18653/v1/W17-4413)Cited by:[§A\.1\.1](https://arxiv.org/html/2611.17461#A1.SS1.SSS1.Px5.p1.1),[Table 1](https://arxiv.org/html/2611.17461#A1.T1.2.1.6.1.1)\.
- Wilcoxon \(1945\)F\. WilcoxonIndividual comparisons by ranking methods\.Biometrics bulletin1\(6\),pp\. 80–83\.Cited by:[§A\.4](https://arxiv.org/html/2611.17461#A1.SS4.SSS0.Px1.p1.1),[Table 4](https://arxiv.org/html/2611.17461#A1.T4)\.
- Wuet al\.\(2025\)J\. Wu, S\. Liu, H\. Tu, H\. Yu, X\. Huang, J\. Y\. Zou, C\. Xie, and Y\. ZhouKnowledge or reasoning? a close look at how llms think across domains\.CoRRabs/2506\.02126\.External Links:[Link](https://doi.org/10.48550/arXiv.2506.02126)Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p2.1),[§1](https://arxiv.org/html/2611.17461#S1.p3.1),[§2\.1](https://arxiv.org/html/2611.17461#S2.SS1.p1.1)\.
- Xiaoet al\.\(2025\)C\. Xiao, H\. P\. Chan, H\. Zhang, M\. Aljunied, L\. Bing, N\. Al Moubayed, and Y\. RongAnalyzing LLMs’ knowledge boundary cognition across languages through the lens of internal representations\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 24099–24115\.External Links:[Link](https://aclanthology.org/2025.acl-long.1174/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.1174),ISBN 979\-8\-89176\-251\-0Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p2.1)\.
- Yanget al\.\(2025a\)M\. Yang, J\. Gao, and J\. WuDecoupling knowledge and reasoning in llms: an exploration using cognitive dual\-system theory\.arXiv preprint arXiv:2507\.18178\.Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p3.1),[§1](https://arxiv.org/html/2611.17461#S1.p4.1),[§2\.1](https://arxiv.org/html/2611.17461#S2.SS1.p1.1)\.
- Yanget al\.\(2025b\)S\. Yang, N\. Kassner, E\. Gribovskaya, S\. Riedel, and M\. GevaDo large language models perform latent multi\-hop reasoning without exploiting shortcuts?\.InFindings of the Association for Computational Linguistics: ACL 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 3971–3992\.External Links:[Link](https://aclanthology.org/2025.findings-acl.205/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.205),ISBN 979\-8\-89176\-256\-5Cited by:[§2\.2](https://arxiv.org/html/2611.17461#S2.SS2.p1.1)\.
- Yinet al\.\(2023\)Z\. Yin, Q\. Sun, Q\. Guo, J\. Wu, X\. Qiu, and X\. HuangDo large language models know what they don’t know?\.InFindings of the Association for Computational Linguistics: ACL 2023,A\. Rogers, J\. Boyd\-Graber, and N\. Okazaki \(Eds\.\),Toronto, Canada,pp\. 8653–8665\.External Links:[Link](https://aclanthology.org/2023.findings-acl.551/),[Document](https://dx.doi.org/10.18653/v1/2023.findings-acl.551)Cited by:[§2\.3](https://arxiv.org/html/2611.17461#S2.SS3.p1.1)\.
- Zhanget al\.\(2026\)D\. Zhang, Z\. Li, M\. Zhang, J\. Zhang, Z\. Liu, Y\. Yao, H\. Xu, J\. Zheng, X\. Chen, Y\. Zhang, F\. Yin, J\. Dong, Z\. Guo, L\. Song, and C\. LiuFrom system 1 to system 2: a survey of reasoning large language models\.IEEE Transactions on Pattern Analysis and Machine Intelligence48\(3\),pp\. 3335–3354\.External Links:[Document](https://dx.doi.org/10.1109/TPAMI.2025.3637037)Cited by:[§2\.1](https://arxiv.org/html/2611.17461#S2.SS1.p1.1)\.
- Zhanget al\.\(2024\)Y\. Zhang, Y\. Dong, and K\. KawaguchiInvestigating layer importance in large language models\.InThe 7th BlackboxNLP Workshop,External Links:[Link](https://openreview.net/forum?id=kjZIIvFtmK)Cited by:[§2\.2](https://arxiv.org/html/2611.17461#S2.SS2.p1.1)\.
- Zhaoet al\.\(2026\)B\. Zhao, R\. Wheeler, K\. Bacho, \{\. M\. Ponti, B\. Rabern, and O\. BohdalA benchmark of expert\-level academic questions to assess ai capabilities\.Nature649\(8099\),pp\. 1139–1146\(English\)\.Note:The research is supported by Center for AI Safety and Scale AI\. \_Contributions\_ All authors have contributed to the dataset creation process\. The Center for AI Safety and Scale AI consortia jointly designed the dataset premise and pipeline; operated the data collection platform \(https://lastexam\.ai\); and provided funding, inference infrastructure for LLMs and review/auditing resources\. The authors in the HLE Contributors Consortium contributed to the dataset in various ways, including submitting at least one accepted question to one of the dataset versions, contributing to dataset refinement or assisting with evaluations\. In the Center for AI Safety and Scale AI, Long Phan, Alice Gatti, Ziwen Han and Nathaniel Li led the project, and Summer Yue, Alexandr Wang and Dan Hendrycks provided senior supervision\.External Links:[Document](https://dx.doi.org/10.1038/s41586-025-09962-4),ISSN 0028\-0836Cited by:[§1](https://arxiv.org/html/2611.17461#S1.p1.1)\.
- Zhaoet al\.\(2025\)H\. Zhao, F\. Yang, B\. Shen, A\. Payani, H\. Lakkaraju, and M\. DuTowards uncovering how large language modelswork: an interpretability perspective\.ACM SIGKDD Explorations Newsletter27\(2\),pp\. 79–87\.Cited by:[§2\.3](https://arxiv.org/html/2611.17461#S2.SS3.p1.1)\.
- Zhaoet al\.\(2024\)Y\. Zhao, W\. Zhang, G\. Chen, K\. Kawaguchi, and L\. BingHow do large language models handle multilingualism?\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=ctXYOoAgRy)Cited by:[§2\.3](https://arxiv.org/html/2611.17461#S2.SS3.p1.1)\.
- Zhenget al\.\(2024\)L\. Zheng, L\. Yin, Z\. Xie, C\. Sun, J\. Huang, C\. H\. Yu, S\. Cao, C\. Kozyrakis, I\. Stoica, J\. E\. Gonzalez, C\. Barrett, and Y\. ShengSGLang: efficient execution of structured language model programs\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=VqkAKQibpq)Cited by:[§3\.4](https://arxiv.org/html/2611.17461#S3.SS4.p4.1)\.
- Zhuet al\.\(2025\)R\. Zhu, T\. Peng, T\. Cheng, X\. Qu, J\. Huang, D\. Zhu, H\. Wang, K\. Xue, X\. Zhang, Y\. Shan, T\. Cai, T\. Kergan, A\. Kembay, A\. Smith, C\. Lin, B\. Nguyen, Y\. Pan, Y\. Chou, Z\. Cai, Z\. Wu, Y\. Zhao, T\. Liu, J\. Yang, W\. Zhou, C\. Zheng, C\. Li, Y\. Zhou, Z\. Li, Z\. Zhang, J\. Liu, G\. Zhang, W\. Huang, and J\. EshraghianA survey on latent reasoning\.External Links:2507\.06203,[Link](https://arxiv.org/abs/2507.06203)Cited by:[§2\.3](https://arxiv.org/html/2611.17461#S2.SS3.p1.1)\.

## Appendix AExperiments Details

Table 1:Statistics of datasets used for knowledge and reasoning analysis, grouped by domain\.Table 2:Overview of the models evaluated in this work, including their HuggingFace links, licensing terms, and release time\. Models from the same family share the same background colour\.### A\.1Datasets

#### A\.1\.1Benchmark Taxonomy

Following the operational criteria in the main text, the knowledge\-centric group comprises BioMixQA, MedMCQA, MRI\-MCQA, SciQ, QASC, MusicTrivia, TruthfulQA, and HaluEval\. These benchmarks predominantly test retrieval or verification of biomedical, scientific, cultural, or factual information\. The reasoning\-centric group comprises AI2\-ARC, GPQA\-Diamond, RACE, PubMedQA, CommonsenseQA, Winogrande, PIQA, SIQA, MMLU\-Pro, MathQA, GSM8K, AIME, MATH500, AMC, CoinFlip, and LogiQA\. Their items predominantly require composition, inference from a supplied context, calculation, or state tracking\. This grouping records the dominant demand of a benchmark rather than asserting that every item is pure: GPQA and MMLU\-Pro require substantial domain knowledge, while SciQ and biomedical QA can require local inference\. These mixed cases are precisely why we analyse a graded spectrum and report deviations from the group\-level trend\.

##### BioMixQA

[Soman et al\. \(2024\)](https://arxiv.org/html/2611.17461#bib.bib19)is a biomedical MCQA dataset created to evaluate Knowledge Graph\-based Retrieval\-Augmented Generation \(KG\-RAG\) frameworks\.

##### MedMCQA

[Pal et al\. \(2022\)](https://arxiv.org/html/2611.17461#bib.bib8)is a large\-scale MCQA dataset with over 194,000 questions from Indian medical entrance exams, testing complex reasoning across 21 medical subjects\.

##### MRI\-MCQA

[Sedano \(2025\)](https://arxiv.org/html/2611.17461#bib.bib22)is a specialised MCQA benchmark covering Magnetic Resonance Imaging \(MRI\) topics such as cardiac imaging, pulse sequences, and physics\.

##### PubMedQA

[Jin et al\. \(2019\)](https://arxiv.org/html/2611.17461#bib.bib14)is a biomedical QA dataset requiring yes/no/maybe answers to research questions based on PubMed abstracts, testing the understanding of quantitative results\.

##### SciQ

[Welbl et al\. \(2017\)](https://arxiv.org/html/2611.17461#bib.bib24)is a crowdsourced dataset of 13,679 multiple\-choice science questions in Physics, Chemistry, and Biology, most of which include a supporting text with evidence\.

##### QASC

[Khot et al\. \(2020\)](https://arxiv.org/html/2611.17461#bib.bib6)is a grade\-school science dataset with nearly 10,000 questions designed to test multi\-hop reasoning by composing facts\.

##### AI2\-ARC

[Clark et al\. \(2018\)](https://arxiv.org/html/2611.17461#bib.bib16)is a dataset of 7,787 grade\-school science questions\. Its Challenge Set of 2,590 questions is designed to require complex reasoning beyond simple retrieval or word association\.

##### GPQA\-Diamond

[Rein et al\. \(2024\)](https://arxiv.org/html/2611.17461#bib.bib12)is a Google\-Proof dataset of 448 challenging science questions created by PhD\-level experts, designed to be difficult even with internet access\.

##### MusicTrivia

[Wang \(2025\)](https://arxiv.org/html/2611.17461#bib.bib23)is an MCQA dataset testing a wide range of musical knowledge, from classical composers and periods to modern pop/rock artists and general music history\.

##### RACE

[Lai et al\. \(2017\)](https://arxiv.org/html/2611.17461#bib.bib25)is a large\-scale reading comprehension dataset with nearly 100,000 questions sourced from English exams for middle and high school students\.

##### TruthfulQA

[Lin et al\. \(2022\)](https://arxiv.org/html/2611.17461#bib.bib7)is a benchmark designed to measure a model’s truthfulness using 817 adversarial questions that probe common human misconceptions\.

##### HaluEval

[Li et al\. \(2023\)](https://arxiv.org/html/2611.17461#bib.bib15)is a large\-scale benchmark for evaluating an LLM’s ability to recognize and avoid generating factually incorrect or unverifiable content \(hallucinations\)\.

##### CommonsenseQA

[Talmor et al\. \(2019\)](https://arxiv.org/html/2611.17461#bib.bib11)is a challenging MCQA dataset designed to evaluate commonsense reasoning ability beyond simple fact retrieval\.

##### Winogrande

[Sakaguchi et al\. \(2021\)](https://arxiv.org/html/2611.17461#bib.bib13)is a large\-scale and challenging dataset for commonsense reasoning, designed as a more robust successor to the Winograd Schema Challenge\.

##### PIQA

[Bisk et al\. \(2020\)](https://arxiv.org/html/2611.17461#bib.bib17)is a benchmark for physical commonsense, requiring models to choose the more sensible of two solutions to achieve a practical, real\-world goal\.

##### SIQA

[Sap et al\. \(2019\)](https://arxiv.org/html/2611.17461#bib.bib26)is an MCQA benchmark for testing social and emotional commonsense by requiring reasoning about people’s motivations and actions in social situations\.

##### MMLU\-Pro

[Wang et al\. \(2024b\)](https://arxiv.org/html/2611.17461#bib.bib2)is an enhanced version of MMLU that increases difficulty with more complex questions and expands from four to ten multiple\-choice options for a more robust evaluation\.

##### MathQA

[Amini et al\. \(2019\)](https://arxiv.org/html/2611.17461#bib.bib10)is a large\-scale dataset for math word problems where each problem is paired with an operational program, promoting interpretable, step\-by\-step solutions\.

##### GSM8K

[Cobbe et al\. \(2021\)](https://arxiv.org/html/2611.17461#bib.bib9)is a dataset of 8,500 high\-quality grade school math word problems designed to evaluate multi\-step reasoning, with solutions provided as natural language explanations\.

##### AIME

[MAA \(2025a\)](https://arxiv.org/html/2611.17461#bib.bib21)is a benchmark of problems from the American Invitational Mathematics Examination, a challenging high school competition used to evaluate advanced mathematical reasoning\.

##### MATH500

[Hendrycks et al\. \(2021\)](https://arxiv.org/html/2611.17461#bib.bib58)is a benchmark of thousands of competition math problems from pre\-algebra to calculus, used to evaluate mathematical problem\-solving via step\-by\-step solutions\.

##### AMC

[MAA \(2025b\)](https://arxiv.org/html/2611.17461#bib.bib20)consists of problems from the American Mathematics Competitions, used to test pre\-collegiate proficiency in algebra, geometry, number theory, and combinatorics\.

##### CoinFlip

[Wei et al\. \(2022\)](https://arxiv.org/html/2611.17461#bib.bib27)is a symbolic reasoning benchmark that tests a model’s ability to track the binary state of an object through a sequence of actions\.

##### LogiQA

[Liu et al\. \(2020\)](https://arxiv.org/html/2611.17461#bib.bib18)is a reading comprehension benchmark from civil service exams that tests formal logical reasoning \(e\.g\., categorical, conditional\) based on provided premises\.

### A\.2Models

Figure 11:Cross\-benchmark variability in adjusted cosine similarity for the Llama, Qwen, and OLMo model families\. The plotted value is the population standard deviation of those four benchmark means\. Dispersion is greatest in early layers and decreases toward a lower, relatively stable level in middle and later layers\.Prompt Template \(Retrieve→\\rightarrowReason\)\.Identify key facts and technical principles related to the following question\. Based only on the knowledge retrieved above, analyze the following question\. The last line of your response should be of the following format: ’ANSWER: \[LETTER\]’ \(without quotes\) where \[LETTER\] is one of letters\.Question: \{question\}Choices: \{choices\}Figure 12:Prompt used for decoding\-time trajectory analysis\.Table[2](https://arxiv.org/html/2611.17461#A1.T2)provides an overview of the open\-source LLMs evaluated in this work\. Models from the same family share a common background colour\.

\(a\) Prompt for knowledge\-centric datasetsQuestion: \{question\}Choices: \{choices\}Respond with one letter\. The answer is \_

\(b\) Prompt for reasoning\-centric datasetsPlease enclose your thinking process in<think\></think\>and the final answer in<final\_answer\></final\_answer\>\.Question: \{question\}Choices: \{choices\}Respond with one letter\. The answer is\_ <think\></think\><final\_answer\>

Figure 13:Prompt templates for extracting representations under the multiple\-choice setting\. We query the LLM with the appropriate template and extract the last\-token representation from each layer as input to the linear probing classifiers\.
### A\.3Prompts

Our prompt templates are designed to \(i\) standardise the interface across benchmarks for representation extraction, \(ii\) avoid confounds from variable\-length decoding\. Importantly, we donotgenerate free\-form outputs for probing\. Instead, we run a single forward pass on the full prompt and extract the hidden state at the last input token from each layer, which serves as the feature for the layer\-wise linear classifiers\.

Table 3:Correct\-incorrect differences in within\-trajectory variability across representative lower, middle, and upper layers\.qqdenotes the Benjamini\-Hochberg adjustedpp\-value[Benjamini and Hochberg \(1995\)](https://arxiv.org/html/2611.17461#bib.bib63), with correction across the three layer\-wise comparisons within each metric\.δ\\deltadenotes Cliff’s delta[Cliff \(1993\)](https://arxiv.org/html/2611.17461#bib.bib64); positive values indicate greater within\-trajectory variability for incorrect than correct generations\.Boldvalues indicateq<\.05q<\.05\.Table 4:Effect of CoT prompting on the knowledge and reasoning dimensions\.Δ~\\tilde\{\\Delta\}is the median paired change from Standard to CoT prompting, and\#⁡\(Δ\>0\)\\\#\(\\Delta\>0\)reports the number of evaluated configurations with a positive change\.ppis obtained from a two\-sided Wilcoxon signed\-rank test[Wilcoxon \(1945\)](https://arxiv.org/html/2611.17461#bib.bib65), andrrbr\_\{\\mathrm\{rb\}\}denotes the matched\-pairs rank\-biserial correlation[Kerby \(2014\)](https://arxiv.org/html/2611.17461#bib.bib66)\.Table 5:Robustness toNotaperturbations across model, dataset, and prompting conditions\.Δ~acc\\tilde\{\\Delta\}\_\{\\mathrm\{acc\}\}denotes the median decrease in accuracy relative to the corresponding Base condition\. The final column reports the proportion of the 24 comparisons that remain significant after Benjamini\-Hochberg correction over all 48 exact McNemar tests[McNemar \(1947\)](https://arxiv.org/html/2611.17461#bib.bib67)\.
### A\.4Statistical Validation

We complement the descriptive comparisons in RQ2 with paired non\-parametric tests targeting the corresponding experimental units\. For the CoT analysis, comparisons are paired across model checkpoints\. For theNotaanalysis, comparisons are paired at the item level within each model, dataset, and prompting condition\.

##### CoT prompting\.

To assess whether the shifts in Figure[10](https://arxiv.org/html/2611.17461#S3.F10)are systematic across models, we compare the Standard and CoT scores separately for the knowledge and reasoning dimensions using two\-sided Wilcoxon signed\-rank tests\([Wilcoxon, 1945](https://arxiv.org/html/2611.17461#bib.bib65)\)\. The model checkpoint is treated as the paired unit; theThinkandNo\-Thinkconfigurations of Qwen3\-8B are aggregated within the shared checkpoint before testing\. We report the median paired change, the number of models with a positive change, and the matched\-pairs rank\-biserial correlationrrbr\_\{\\mathrm\{rb\}\}\([Kerby, 2014](https://arxiv.org/html/2611.17461#bib.bib66)\)as an effect size\. Table[4](https://arxiv.org/html/2611.17461#A1.T4)shows a systematic positive shift in reasoning, whereas the corresponding shift in knowledge is not supported\.

##### Notaperturbations\.

For Figures[8](https://arxiv.org/html/2611.17461#S3.F8)and[9](https://arxiv.org/html/2611.17461#S3.F9), we test changes in item\-level correctness between each Base condition and its correspondingNotaperturbation using two\-sided exact McNemar tests\([McNemar, 1947](https://arxiv.org/html/2611.17461#bib.bib67)\)\. Tests are conducted separately for each model×\\timesdataset×\\timesprompting condition, yielding 24 comparisons forNota\-as\-answer and 24 forNota\-as\-distractor\. We control the false discovery rate across the resulting 48 tests using the Benjamini\-Hochberg procedure\([Benjamini and Hochberg, 1995](https://arxiv.org/html/2611.17461#bib.bib63)\)\. As summarised in Table[5](https://arxiv.org/html/2611.17461#A1.T5), theNota\-as\-answer effect is consistently detected across conditions, whereas theNota\-as\-distractor effect is comparatively limited\.

Table 6:Performance of all models across knowledge\-based benchmarks\. For each dataset, we report accuracy under both standard \(Std\.\) and CoT prompting\. The best\-performing model in each setting is highlighted inbold\.
Table 7:Performance on Mathematical Reasoning benchmarks \(Part 1\)\. Scores are reported as accuracy\. The number of few\-shot examples used is in parentheses\. The best\-performing model in each setting is highlighted inbold\.
Table 8:Performance on General and Commonsense Reasoning benchmarks \(Part 2\)\. Scores are reported as accuracy\. The number of few\-shot examples used is in parentheses\. The best\-performing model is highlighted inbold\.
Table 9:Performance on Reading Comprehension and Multi\-Discipline Reasoning benchmarks \(Part 3\)\. Scores are reported as accuracy\. The number of few\-shot examples is in parentheses\. The best model is inbold\.

相似文章

PRIME:迈向 LLM 智能体中可验证的证据支撑推理

arXiv cs.AI

PRIME(Precommitted Rules for Inference from Minimal Evidence,基于最小证据的预承诺推理规则)通过预承诺证据规则和检查账本约束 LLM 智能体,使其只报告可被验证证据支持的结论,在 480 个公共记录任务上将无依据结论从 26.7% 降至 0,并将有依据结论的平均分从 30.1 提升至满分 59.2/100。

在推理型大语言模型中实现摊销式序贯实验设计

Hugging Face Daily Papers

该论文提出将序贯贝叶斯实验设计(BED)通过信息增益奖励训练摊销进 LLM 的推理过程中,使 LLM 学会自适应信息收集,并在 Twenty Questions 与临床对话基准 Medi-Q 上超越基线且泛化良好。