大语言模型中知识可达性的几何结构

# The Geometry of Knowledge Accessibility in Large Language Models
Source: [https://arxiv.org/html/2610.03052](https://arxiv.org/html/2610.03052)
###### Abstract

Large language models \(LLMs\) contain broad knowledge, but they cannot access all of it reliably\. We study this problem through*knowledge accessibility*, which describes whether the knowledge needed for a query can be recalled from the model\. We find that knowledge accessibility has a simple geometric structure in the model’s representation of the query alone, before any generation\. More accessible queries are closer to a center in the representation space, while less accessible queries are farther away\. This geometry reveals a*knowledge boundary*that separates more accessible queries from less accessible ones\. Accessibility consistently decreases with distance from the center, and this distance\-based ordering transfers across datasets even when the centers differ\. Controlled experiments further show that the centered geometry is more closely related to knowledge accessibility than to reasoning difficulty\. The geometry also reveals when different interventions are useful\. Query rewriting helps more for accessible queries, chain\-of\-thought reasoning helps more near the boundary, and retrieval gives larger gains beyond the boundary\. These findings not only provide a new geometric view of how knowledge is organized in language models, but also suggest a useful pre\-generation signal for adaptive inference\.

## 1Introduction

Large language models \(LLMs\) store a large amount of knowledge during pre\-training\([Petroni et al\., 2019](https://arxiv.org/html/2610.03052#bib.bib1)\), yet this knowledge is not equally accessible for every query\. This limitation is often described by the*knowledge boundary*, which separates queries that a model can answer from those that it cannot\([Kadavath et al\., 2022](https://arxiv.org/html/2610.03052#bib.bib18);[Li et al\., 2025](https://arxiv.org/html/2610.03052#bib.bib20)\)\. Recent work shows that this boundary can be estimated from the internal states of a query before any token is generated\([Ni et al\., 2025](https://arxiv.org/html/2610.03052#bib.bib23);[Chen et al\., 2026](https://arxiv.org/html/2610.03052#bib.bib24)\)\. Such generation\-free signals are potentially useful for deciding how much additional computation or external information a query may require\.

However, existing work often defines the knowledge boundary in terms of observed answer correctness\. This conflates knowledge accessibility with reasoning, as a model may access the required knowledge but still answer incorrectly due to reasoning failures\([Press et al\., 2023](https://arxiv.org/html/2610.03052#bib.bib36);[Yang et al\., 2026](https://arxiv.org/html/2610.03052#bib.bib34)\)\. Moreover, prior methods mainly ask whether knowledge is encoded in hidden states\([Dai et al\., 2022](https://arxiv.org/html/2610.03052#bib.bib38);[Chen et al\., 2025](https://arxiv.org/html/2610.03052#bib.bib39)\), but they do not reveal how knowledge accessibility itself is organized in representation space\.

In this work, we study the problem through*knowledge accessibility*, which describes whether the knowledge required by a query is accessible from the model’s parameters\. To isolate knowledge recall from reasoning, we focus on single\-fact queries, such as “The capital of France is?”, whose answers require one factual association rather than multi\-step reasoning\([Petroni et al\., 2019](https://arxiv.org/html/2610.03052#bib.bib1);[Press et al\., 2023](https://arxiv.org/html/2610.03052#bib.bib36)\)\. This provides a controlled setting for studying the representation of accessibility\. We then learn the geometry of knowledge accessibility in query representations and ask a different question:

*What is the geometry of knowledge accessibility in an LLM’s representation space?*

We propose the*accessibility center hypothesis*\. Inspired by center\-based structures in representation learning\([Wen et al\., 2016](https://arxiv.org/html/2610.03052#bib.bib14);[Ruff et al\., 2018](https://arxiv.org/html/2610.03052#bib.bib16)\), we hypothesize that reliably accessible knowledge tends to occupy a localized region of representation space\. We model this region with an accessibility center and measure each query by its distance from the center, with smaller distances indicating higher knowledge accessibility\. This distance naturally defines a*knowledge boundary*separating regions of higher and lower accessibility\.

Across two LLMs and four QA datasets, we find consistent support for this centered organization\. Simple centered geometries predict knowledge accessibility better than competitive probing baselines on knowledge\-focused tasks, and accessibility consistently decreases with distance from the learned center\. In contrast, on reasoning\-heavy tasks, a flexible MLP gains an advantage by capturing additional reasoning signals, while the centered geometry remains more closely associated with factual accessibility\. Controlled experiments further support this distinction between accessibility and reasoning\.

The geometric view also reveals several properties that are difficult to see from correctness prediction alone\. Queries tend to be less stable in their correctness as they approach the boundary, while their distance\-based accessibility ordering transfers across datasets even when the precise centers differ\. More importantly, the geometry suggests how different interventions benefit the model\. Query rewriting is most effective for accessible queries, chain\-of\-thought reasoning becomes more useful near the boundary, and retrieval provides larger gains beyond the boundary\. Thus, the knowledge boundary is not only a signal of whether a model may need help, its geometry also provides information about what kind of help is likely to be useful\. These results provide a geometric perspective on how knowledge is organized and accessed in LLMs, and suggest that knowledge accessibility may serve as a useful signal for adaptive and more efficient inference\([Yong et al\., 2025a](https://arxiv.org/html/2610.03052#bib.bib40);[Chen et al\., 2026](https://arxiv.org/html/2610.03052#bib.bib24)\)\.

## 2Knowledge Accessibility

We conceptually break down query correctness asC≈f⁡\(K,R,O\)C\\approx f\(K,R,O\), whereKK,RR, andOOdenote*knowledge accessibility*,*reasoning capability*, and*other task\-specific capabilities*\. That is, answering a query correctly requires both access to the needed knowledge and the ability to use it appropriately\. This decomposition is motivated by prior work showing that access to relevant knowledge and the ability to reason with that knowledge are distinct factors\([Press et al\., 2023](https://arxiv.org/html/2610.03052#bib.bib36);[Yang et al\., 2026](https://arxiv.org/html/2610.03052#bib.bib34)\)\. We treat knowledge accessibility \(KK\) and reasoning capability \(RR\) as the primary factors of interest for query correctness, whileOOcaptures other task\-specific factors whose influence is expected to be limited in our setting\.

In this work, we focus on knowledge accessibility, which we define as the ability to recall the factual knowledge required by a query from the model’s parameters\. To study knowledge accessibility separately from reasoning capability, we focus on*single\-fact factual queries*\. Answers of these queries mainly depend on recalling a single target fact\([Petroni et al\., 2019](https://arxiv.org/html/2610.03052#bib.bib1);[Roberts et al\., 2020](https://arxiv.org/html/2610.03052#bib.bib2)\)\. This setting allows us to isolate factual recall from multi\-step reasoning\.

### 2\.1Query Correctness and Factual Recall

For a single\-fact factual queryxx, let𝕐⁡\(x\)\{\\mathbb\{Y\}\}\(x\)denote the set of valid answers and lety^π​\(x\)\\hat\{y\}\_\{\\pi\}\(x\)be the model response under decoding strategyπ\\pi\. We define

cπ\(x\)≜𝟏\[y^π\(x\)∈𝕐\(x\)\]\.c\_\{\\pi\}\(x\)\\triangleq\\bm\{1\}\[\\hat\{y\}\_\{\\pi\}\(x\)\\in\{\\mathbb\{Y\}\}\(x\)\]\.\(1\)In our single\-fact setting, correctness mainly reflects whether the target fact is successfully recalled\. We therefore usecπ​\(x\)c\_\{\\pi\}\(x\)as a behavioral observation of factual recall under decoding strategyπ\\pi, with correctness potentially varying across decoding strategies\.

More generally, we view knowledge accessibility as the model’s ability to recall the required factual knowledge from its parameters under a given elicitation and decoding setting\. Low accessibility may reflect either knowledge that is not reliably stored or knowledge that is stored but not successfully accessed\. This work does not distinguish between these two cases\.

For more complex tasks, such as mathematical or multi\-step reasoning, correctness depends on more than knowledge accessibility\. A model may have access to the required knowledge but still fail because of reasoning or other task\-specific capabilities\. This motivates our use of single\-fact factual queries, where correctness provides a cleaner proxy for knowledge accessibility\.

### 2\.2A Geometric Hypothesis of Knowledge Accessibility

Let𝒉ℓ​\(x\)∈ℝd\{\\bm\{h\}\}\_\{\\ell\}\(x\)\\in\\mathbb\{R\}^\{d\}denote the hidden representation at the final query token at layerℓ\\ell\. We compute𝒉ℓ​\(x\)\{\\bm\{h\}\}\_\{\\ell\}\(x\)from the query alone, before generating any answer tokens\. For notational simplicity, we omit the layer indexℓ\\ellin the following\. We therefore ask whether knowledge accessibility is already reflected in representation space before generation, and, if so, what geometric structure it takes\.

Prior work has mostly used linear probing to study this question\. These studies ask whether hidden representations can separate queries with different model outcomes\([Xiao et al\., 2025](https://arxiv.org/html/2610.03052#bib.bib42);[Chen et al\., 2026](https://arxiv.org/html/2610.03052#bib.bib24)\)\. This shows that hidden states may contain information about knowledge accessibility\. However, it does not tell us how this information is organized in representation space\.

A simple center\-based structure has been observed in learned representations\. In particular, representations with shared properties may become more concentrated around a center in representation space\([Papyan et al\., 2020](https://arxiv.org/html/2610.03052#bib.bib12);[Wu and Papyan, 2024](https://arxiv.org/html/2610.03052#bib.bib13)\)\. We ask whether a similar effect also appears for knowledge accessibility\. Similar queries may develop a shared geometric structure during training\. Queries based on frequently seen knowledge may form a common center, while queries based on rarely seen or unseen knowledge receive less shared training signal and may lie farther from this center\.

#### Accessibility Center Hypothesis\.

We propose that queries may be organized around an*accessibility center*, with distance from this center related to how easily the model can recall the required knowledge\. Specifically, we posit an accessibility center𝒄∈ℝd\{\\bm\{c\}\}\\in\\mathbb\{R\}^\{d\}and define the knowledge distance asD⁡\(x\)≜d⁡\(𝒉⁡\(x\),𝒄\)\.D\(x\)\\triangleq d\(\{\\bm\{h\}\}\(x\),\{\\bm\{c\}\}\)\.We hypothesize that factual recall becomes less likely as knowledge distance increases\. LetXXdenote a query drawn from the data distribution\. Formally,

r1<r2⟹𝔼⁡\[c⁡\(X\)∣D⁡\(X\)=r1\]≥𝔼⁡\[c⁡\(X\)∣D⁡\(X\)=r2\]r\_\{1\}<r\_\{2\}\\quad\\Longrightarrow\\quad\\mathbb\{E\}\[c\(X\)\\mid D\(X\)=r\_\{1\}\]\\geq\\mathbb\{E\}\[c\(X\)\\mid D\(X\)=r\_\{2\}\]\(2\)
The accessibility center should not be interpreted as a location where individual facts are stored\. Rather, it summarizes a region of representation space associated with more reliable access to factual knowledge\.

## 3Modeling the Geometry of Knowledge Accessibility

### 3\.1Centered Accessibility Geometry

Center\-based structures have been widely used in representation learning\([Snell et al\., 2017](https://arxiv.org/html/2610.03052#bib.bib15);[Ruff et al\., 2018](https://arxiv.org/html/2610.03052#bib.bib16)\)\. For example, center loss encourages representations sharing the same label to concentrate around a learned feature center\([Wen et al\., 2016](https://arxiv.org/html/2610.03052#bib.bib14)\)\. Motivated by this view, we instantiate the accessibility\-center hypothesis with a simple hyperspherical geometry\. Specifically, we defineD⁡\(x\)≜‖𝒉⁡\(x\)−𝒄‖2,D\(x\)\\triangleq\\\|\{\\bm\{h\}\}\(x\)\-\{\\bm\{c\}\}\\\|\_\{2\},so that points with the same knowledge distance lie on a hypersphere centered at𝒄\{\\bm\{c\}\}\.

This formulation makes an isotropic assumption\. Distance from the center is measured equally in all representation dimensions\. Representations, however, may vary along different dimensions at different scales\([Ethayarajh, 2019](https://arxiv.org/html/2610.03052#bib.bib17)\)\. We therefore relax this assumption and consider an anisotropic extension,

D⁡\(x\)≜\(𝒉⁡\(x\)−𝒄\)⊤​𝑨​\(𝒉⁡\(x\)−𝒄\),D\(x\)\\triangleq\\bigl\(\{\\bm\{h\}\}\(x\)\-\{\\bm\{c\}\}\\bigr\)^\{\\top\}\{\\bm\{A\}\}\\bigl\(\{\\bm\{h\}\}\(x\)\-\{\\bm\{c\}\}\\bigr\),\(3\)where𝑨⪰0\{\\bm\{A\}\}\\succeq 0is*dimension weights*, which is diagonal and controls the relative scaling of different representation dimensions\. The resulting level sets are hyperellipsoids rather than hyperspheres, while retaining the assumption that accessibility is organized around a common center\.

To test whether accessibility is organized around a center, rather than merely encoded in the representation, we compare against two references\. Linear probing is widely used to assess whether a concept is encoded in representations\. We therefore use the score𝒘⊤​𝒉​\(x\)\+b\{\\bm\{w\}\}^\{\\top\}\{\\bm\{h\}\}\(x\)\+bas a linear baseline\. We additionally consider an MLP, with more parameters and nonlinear activations, which provides a more expressive reference\. These models contrast a centered inductive bias with linear separability and flexible nonlinear predictability, respectively, as summarized in Table[1](https://arxiv.org/html/2610.03052#S3.T1)\.

HypothesisAccessibility scoreGeometryParametersLinear𝒘⊤​𝒉\+b\{\\bm\{w\}\}^\{\\top\}\{\\bm\{h\}\}\+bHyperplane𝒪⁡\(d\)\\mathcal\{O\}\(d\)Centered−∥𝒉−𝒄∥22\-\\lVert\{\\bm\{h\}\}\-\{\\bm\{c\}\}\\rVert\_\{2\}^\{2\}Hypersphere𝒪⁡\(d\)\\mathcal\{O\}\(d\)Anisotropic centered−\(𝒉−𝒄\)⊤​𝑨​\(𝒉−𝒄\)\-\(\{\\bm\{h\}\}\-\{\\bm\{c\}\}\)^\{\\top\}\{\\bm\{A\}\}\(\{\\bm\{h\}\}\-\{\\bm\{c\}\}\)Hyperellipsoid𝒪⁡\(d\)\\mathcal\{O\}\(d\)Flexible nonlinearMLP⁡\(𝒉\)\\operatorname\{MLP\}\(\{\\bm\{h\}\}\)MLP reference𝒪⁡\(d​m\)\\mathcal\{O\}\(dm\)
Table 1:Geometric hypotheses of knowledge accessibility in representation space\. Here,ddis the query representation dimension andmmis the MLP hidden dimension\.
DatasetKKRRTask TypeTriviaQAHighLowFactual QASciQHighMediumScience QAMuSiQueHighHighMulti\-hop QAGSM8KMediumHighMath reasoning
Table 2:Decomposing answerability across benchmarks\. Correctness reflects knowledge accessibility \(KK\) and reasoning capability \(RR\)\.

### 3\.2The Knowledge Boundary

Under the accessibility\-center hypothesis, queries farther from the accessibility center are less likely to support successful factual recall\. This radial ordering naturally induces a transition between regions of higher and lower knowledge accessibility\. We refer to this transition as the*knowledge boundary*\.

Letτ\\taudenote a learned boundary radius and letϵ\>0\\epsilon\>0control the width of the transition region\. Under this geometry, we define three regions:

D⁡\(x\)\\displaystyle D\(x\)<τ−ϵ\\displaystyle<\\tau\-\\epsilon\(*interior*\),\|D⁡\(x\)−τ\|\\displaystyle\\text\{\(\\emph\{interior\}\)\},\\ \|D\(x\)\-\\tau\|≤ϵ\\displaystyle\\leq\\epsilon\(*boundary*\),D⁡\(x\)\\displaystyle\\text\{\(\\emph\{boundary\}\)\},\\ D\(x\)\>τ\+ϵ\\displaystyle\>\\tau\+\\epsilon\(*exterior*\)\.\\displaystyle\\text\{\(\\emph\{exterior\}\)\}\.\(4\)
For subsequent analysis, we define the signed distance to the boundary asδ⁡\(x\)≜D⁡\(x\)−τ\.\\delta\(x\)\\triangleq D\(x\)\-\\tau\.Accordingly, the three regimes can equivalently be expressed as

δ⁡\(x\)\\displaystyle\\delta\(x\)<−ϵ\\displaystyle<\-\\epsilon\(*interior*\),\|δ⁡\(x\)\|\\displaystyle\\text\{\(\\emph\{interior\}\)\},\\ \|\\delta\(x\)\|≤ϵ\\displaystyle\\leq\\epsilon\(*boundary*\),δ⁡\(x\)\\displaystyle\\text\{\(\\emph\{boundary\}\)\},\\ \\delta\(x\)\>ϵ\\displaystyle\>\\epsilon\(*exterior*\)\.\\displaystyle\\text\{\(\\emph\{exterior\}\)\}\.\(5\)Queries in the interior are expected to have higher knowledge accessibility, while those in the exterior have lower accessibility\. The boundary marks an unstable transition region between the two\.

## 4Evaluating the Accessibility Geometry

### 4\.1Experimental Setup

We evaluate two instruction\-tuned language models,Llama\-3\.1\-8B\-Instruct\([Grattafiori et al\., 2024](https://arxiv.org/html/2610.03052#bib.bib28)\)andQwen\-2\.5\-14B\-Instruct\([Hui et al\., 2024](https://arxiv.org/html/2610.03052#bib.bib29)\), across four QA benchmarks with different demands on knowledge accessibility and reasoning \(Table[2](https://arxiv.org/html/2610.03052#S3.T2)\)\.

TriviaQA\([Joshi et al\., 2017](https://arxiv.org/html/2610.03052#bib.bib30)\)and SciQ\([Welbl et al\., 2017](https://arxiv.org/html/2610.03052#bib.bib31)\)provide relatively clean settings for studying knowledge accessibility, as correctness mainly reflects factual recall\. MuSiQue\([Trivedi et al\., 2022](https://arxiv.org/html/2610.03052#bib.bib32)\)and GSM8K\([Cobbe et al\., 2021](https://arxiv.org/html/2610.03052#bib.bib33)\)require more reasoning\. They let us test whether the centered geometry still works when correctness depends less directly on knowledge access\. In such settings, a more flexible nonlinear model may become advantageous\. For each query, we obtain the observed answer using the greedy decoding policyπ\\pi\(Equation[1](https://arxiv.org/html/2610.03052#S2.E1)\), and useMistral\-Largeto judge its correctness\.

For each query, we extract the hidden state at the final query token from the query alone, without generating any answer tokens\. We use a 40%/10%/50% train/validation/test split and use the same split for all layers within each random seed\. All results are averaged over five random seeds\.

For the layer\-wise analysis, we fit each geometry independently at every layer\. For all subsequent analyses, we use the layer with the highest mean validation AUROC and keep this layer fixed\. We setτ\\tauon the validation set as the distance where the correctness probability is0\.50\.5\. We choose the boundary widthϵ\\epsilonsuch that the boundary region corresponds to predicted correctness probabilities between0\.450\.45and0\.550\.55\. Full training details and hyperparameters are provided in Appendix[A](https://arxiv.org/html/2610.03052#A1)\.

\(a\)Llama\-8B: TriviaQA\(b\)Llama\-8B: SciQ\(c\)Llama\-8B: MuSiQue\(d\)Llama\-8B: GSM8K\(e\)Qwen\-14B: TriviaQA\(f\)Qwen\-14B: SciQ\(g\)Qwen\-14B: MuSiQue\(h\)Qwen\-14B: GSM8K
Figure 1:Layer\-wise comparison of different geometry hypotheses across four datasets and two language models\. Peak performance is reported in the legend\.
### 4\.2Comparing Knowledge Accessibility Geometries

Table[1](https://arxiv.org/html/2610.03052#S3.T1)compares four hypotheses for how answerability is organized in representation space: a linear geometry, an isotropic centered geometry, an anisotropic centered geometry \(*A\-Centered*\), and a flexible nonlinear model \(MLP\)\. Figure[1](https://arxiv.org/html/2610.03052#S4.F1)evaluates these hypotheses layer by layer across the four benchmarks and two language models\.

On TriviaQA and SciQ, A\-Centered is the strongest method on three of the four model–dataset pairs\. Notably, even the simple Centered geometry achieves performance close to the flexible MLP baseline\. In contrast, the Linear model performs substantially worse than the centered models\. Thus, when correctness is more directly driven by knowledge accessibility, a structured centered geometry is sufficient to match or outperform a flexible nonlinear predictor\. These results suggest that much of the accessibility signal can be captured by a simple centered geometry, rather than a complex nonlinear model\.

On MuSiQue and GSM8K, where correctness depends more strongly on reasoning capability, the flexible MLP performs best overall\. This result is expected because correctness on these tasks depends on both knowledge accessibility and reasoning capability\. Our hypothesis focuses on the geometric structure of knowledge accessibility rather than correctness in general\. Even so, A\-Centered remains competitive while using only𝒪⁡\(d\)\\mathcal\{O\}\(d\)parameters, compared with𝒪⁡\(d​m\)\\mathcal\{O\}\(dm\)for the MLP, as shown in Table[1](https://arxiv.org/html/2610.03052#S3.T1)\. This suggests that the centered geometry still captures a large part of the correctness signal with a much simpler model\.

Figure 2:AUROC on TriviaQA with Llama\-8B\.One may ask whether Centered performs better than Linear simply because it uses the squared norm of the representation, rather than because of the centered structure itself\. The Centered score can be expanded as−‖𝒉−𝒄‖22=2​𝒄⊤​𝒉−‖𝒉‖22−‖𝒄‖22,\-\\\|\{\\bm\{h\}\}\-\{\\bm\{c\}\}\\\|\_\{2\}^\{2\}=2\{\\bm\{c\}\}^\{\\top\}\{\\bm\{h\}\}\-\\\|\{\\bm\{h\}\}\\\|\_\{2\}^\{2\}\-\\\|\{\\bm\{c\}\}\\\|\_\{2\}^\{2\},which contains both a linear term and a squared norm term\. We therefore introduce*Linear\+Norm*,s⁡\(𝒉\)=𝒘⊤​𝒉\+α​‖𝒉‖22\+b,s\(\{\\bm\{h\}\}\)=\{\\bm\{w\}\}^\{\\top\}\{\\bm\{h\}\}\+\\alpha\\\|\{\\bm\{h\}\}\\\|\_\{2\}^\{2\}\+b,which learns these terms freely without enforcing a centered structure\. For AUROC, Centered depends onddcenter parameters, while Linear\+Norm depends onddlinear weights and one norm coefficientα\\alpha, giving it at least as much effective capacity\.

As shown in Figure[2](https://arxiv.org/html/2610.03052#S4.F2), Linear\+Norm improves over Linear but remains below Centered\. This suggests that the benefit of Centered is not explained by the additional norm term or greater model capacity, but is consistent with a useful centered inductive bias\. We further analyze the learned coefficientα\\alphain Appendix[B\.1](https://arxiv.org/html/2610.03052#A2.SS1), which shows that the Linear\+Norm model also tends to learn a centered structure\.

### 4\.3Separating Knowledge Accessibility from Reasoning

GroupK / RnnDist\.PositionSuccess✓/✓3127\.85InteriorReasoning Failure✓/✗3115\.33InteriorKnowledge Failure✗/✗2811\.05ExteriorRecovered Success✗/✓965\.29InteriorTable 3:MQuAKE groups by absolute distance to the knowledge boundary, where K and R denote knowledge accessibility \(*single\-hop*\) and reasoning \(*multi\-hop*\) success, respectively\.To further test more directly whether the centered geometry captures knowledge accessibility rather than reasoning capability, we construct a controlled evaluation on MQuAKE\([Zhong et al\., 2023](https://arxiv.org/html/2610.03052#bib.bib3)\)\. MQuAKE provides multi\-hop questions together with their single\-hop subquestions, which allows us to separate knowledge accessibility from reasoning capability\. The single\-hop questions test whether the individual facts needed for the question can be recalled, while the multi\-hop question tests whether the model can use reasoning to combine these facts and produce the final answer\. Thus, if the single\-hop questions are answered correctly but the multi\-hop question is answered incorrectly, the failure is more likely due to reasoning rather than missing knowledge\. In contrast, failure on the required single\-hop facts provides evidence of limited knowledge accessibility\. We then apply the A\-Centered geometry learned on TriviaQA directly to MQuAKE without relearning, which makes MQuAKE an independent test of the learned geometry\. \(see Appendix[B\.2](https://arxiv.org/html/2610.03052#A2.SS2)for details\)\.

As shown in Table[3](https://arxiv.org/html/2610.03052#S4.T3), geometric position is more closely aligned with knowledge accessibility than with final correctness\. Both*Success*and*Reasoning Failure*cases have accessible single\-hop facts, and both remain inside the boundary, even though the latter fail on the final multi\-hop question\. In contrast,*Knowledge Failure*cases, where at least one required fact cannot be recalled, move into the exterior\. This pattern suggests that queries with similar knowledge accessibility remain close in distance despite different reasoning outcomes\. Interestingly,*Recovered Success*cases also remain inside the boundary\. This suggests that failure on an isolated single\-hop probe does not always mean that the required knowledge is inaccessible\. Some of these accessibility failures may instead reflect sensitivity to prompting or decoding, which we study later\.

![Refer to caption](https://arxiv.org/html/2610.03052v1/ellipsoidal_knowledge_quantile_shells.png)\(a\)Accuracy vs\. distance
\(b\)Correctness stability\(c\)Answer consistency
Figure 3:Behavior of knowledge accessibility for Llama\-8B on Trivia QA\. \(a\) Accessibility decreases with distance from the center\. \(b\) Answerability is least stable near the boundary\. \(c\) Consistency decreases with distance from the center\. Results for Qwen14B are provided in[B\.2](https://arxiv.org/html/2610.03052#A2.F2)in Appendix\.
### 4\.4Behavior of Knowledge Accessibility

We next test whether the learned geometry captures meaningful behavior beyond binary correctness\. In particular, we study how knowledge accessibility changes with distance and how stable model behavior is around the learned boundary\.

#### Accessibility Monotonicity\.

We next visualize how knowledge accessibility changes across the learned geometry\. Figure[3\(a\)](https://arxiv.org/html/2610.03052#S4.F3.sf1)shows a low\-dimensional projection of the hidden representations and the learned boundary on TriviaQA\. Since TriviaQA mainly tests factual recall, accuracy provides a proxy for knowledge accessibility\. Queries closer to the center tend to have higher accessibility, while accessibility decreases toward and beyond the boundary\. This pattern is consistent with the learned accessibility structure\.

#### Correctness Stability\.

We next study how stable correctness is around the learned boundary\. For each queryxx, we sampleNNresponses under a fixed stochastic decoding strategyπ\\pi\. Letcπ\(i\)​\(x\)c\_\{\\pi\}^\{\(i\)\}\(x\)denote the correctness of theii\-th sampled response\. We define

p^π​\(x\)=1N​∑i=1Ncπ\(i\)​\(x\),Sπ​\(x\)≜\|2​p^π​\(x\)−1\|\.\\hat\{p\}\_\{\\pi\}\(x\)=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}c\_\{\\pi\}^\{\(i\)\}\(x\),\\qquad S\_\{\\pi\}\(x\)\\triangleq\\left\|2\\hat\{p\}\_\{\\pi\}\(x\)\-1\\right\|\.\(6\)Here,Sπ​\(x\)∈\[0,1\]S\_\{\\pi\}\(x\)\\in\[0,1\]is the correctness stability\. A value close to11means that repeated generations are consistently correct or consistently incorrect, while a value close to00means that correctness varies across generations\.

Figure[3\(b\)](https://arxiv.org/html/2610.03052#S4.F3.sf2)shows a U\-shaped pattern of stability around the learned boundary\. Queries deep in the interior are stably answered correctly\. Stability decreases toward the boundary, where the same query can be answered correctly in some generations and incorrectly in others\. Beyond the boundary, stability increases again, but now mainly reflects repeated failure\. Together with the decrease in accuracy, this gives the following behavioral pattern:stable access⟶\\;\\longrightarrow\\;unstable access⟶\\;\\longrightarrow\\;stable failure\. The boundary therefore marks a region where access to the required knowledge is least stable\. Since repeated generations are not used to learn the boundary, this provides independent evidence for the learned geometry\.

#### Answer Consistency\.

Figure[3\(c\)](https://arxiv.org/html/2610.03052#S4.F3.sf3)provides a complementary view through answer consistency\. On average, repeated answers become less consistent as distance increases, which suggests that knowledge access becomes less reliable farther from the accessibility center\. Some uncertainty methods rely on answer consistency across repeated generations\. A limitation of such signals is that they cannot identify*confidently wrong*samples\([Farquhar et al\., 2024](https://arxiv.org/html/2610.03052#bib.bib4)\), while the centered geometry provides richer signals\.

Figure 4:Transferability of the distance and accuracy relation for Llama\-8B\. Accuracy generally decreases with distance across target datasets\. Results for Qwen\-14B are provided in Figure[B\.3](https://arxiv.org/html/2610.03052#A2.F3)in Appendix\.

### 4\.5Transferability of Accessibility Distance

We next study whether distance to an accessibility center learned on one dataset can transfer to other datasets\. We do not require the learned center to be the same across datasets\. Instead, we ask whether distance under a geometry learned on one dataset can still preserve the accessibility ordering of queries from another dataset\. Full experimental details are provided in Appendix[B\.4](https://arxiv.org/html/2610.03052#A2.SS4)\.

For each source dataset, we learn an accessibility center𝒄\{\\bm\{c\}\}and its dimension weights𝑨\{\\bm\{A\}\}, and apply them directly to the target datasets without relearning\. As shown in Figure[4](https://arxiv.org/html/2610.03052#S4.F4), task performance generally decreases as the distance from the source accessibility center increases across different source\-target pairs\. This transfer is stronger on datasets where correctness mainly reflects knowledge accessibility\. On reasoning datasets \(e\.g\., GSM8K\), the distance still preserves a useful ordering, but the relation is weaker\. Overall, queries farther from the source center tend to have lower knowledge accessibility\.

However, the accessibility centers𝒄\{\\bm\{c\}\}and the dimension weights𝑨\{\\bm\{A\}\}do not transfer across datasets, as shown in Figure[B\.4](https://arxiv.org/html/2610.03052#A2.F4)in the Appendix\. This suggests that different types of task have their own accessibility centers\. Even so, the monotonic relation between distance and knowledge accessibility remains consistent across datasets\. Queries farther from the source center still tend to have lower knowledge accessibility\.

## 5Knowledge Accessibility Interventions

So far, we have shown that distance is related to knowledge accessibility\. Importantly, this distance is computed solely from the query representation, before any answer tokens are generated\. For a queryxx, we extract its hidden representation𝒉⁡\(x\)\{\\bm\{h\}\}\(x\)and measure its distance to the learned accessibility center,D⁡\(x\)=d⁡\(𝒉⁡\(x\),𝒄\)D\(x\)=d\(\{\\bm\{h\}\}\(x\),\{\\bm\{c\}\}\)\.

We next ask whether effective knowledge accessibility can be improved at inference time, without retraining or modifying the model parameters\. We consider three complementary interventions that operate through different mechanisms\. \(1\)*Query rewriting*changes the wording of the query without changing or adding new information, \(2\)*chain\-of\-thought \(CoT\)*provides additional computation, and \(3\)*retrieval\-augmented generation \(RAG\)*provides external information\.

We study these interventions on TriviaQA, where factual recall provides a relatively clean setting for examining their interaction with knowledge accessibility\. Figure[5](https://arxiv.org/html/2610.03052#S5.F5)shows that these interventions are effective in different accessibility regions\. Query rewriting yields its largest average gain in the interior, additional computation around the boundary, and external retrieval in the exterior\. Experimental details are provided in Appendix[C](https://arxiv.org/html/2610.03052#A3)\.

\(a\)Llama\-8B: Rewriting\(b\)Llama\-8B: CoT\(c\)Llama\-8B: RAG\(d\)Qwen\-14B: Rewriting\(e\)Qwen\-14B: CoT\(f\)Qwen\-14B: RAG
Figure 5:Intervention effects across the knowledge boundary on TriviaQA\.#### Query rewriting is most useful when knowledge is already accessible\.

Query rewriting does not add new information or additional reasoning computation\. Instead, it changes how the query accesses knowledge already stored in the model’s parameters\. On TriviaQA, Figure[5\(a\)](https://arxiv.org/html/2610.03052#S5.F5.sf1)shows that rewriting has its largest average gain in the interior, where the required knowledge is more likely to be parametrically accessible\.

This suggests that some failures in the interior are from unsuccessful access rather than missing knowledge\. Paraphrasing the same query can provide an alternative access path and recover the correct answer\. As queries approach the boundary, access becomes more sensitive to wording\. Correctness varies more across semantically equivalent paraphrases, and paraphrase stability decreases\. Farther into the exterior, rewriting becomes less effective because changing the access path alone cannot compensate for increasingly unreliable parametric access\.

Prior work has shown that failure under one prompt does not necessarily imply missing knowledge, since paraphrased prompts can recover correct responses\([Jiang et al\., 2020](https://arxiv.org/html/2610.03052#bib.bib5)\)\. Our results further relate this phenomenon to knowledge accessibility\. Query rewriting is most effective when the required knowledge is already within the model’s accessible region, but its benefit diminishes as parametric access weakens\.

#### Additional computation is most useful near the boundary\.

Unlike query rewriting, CoT does not change the query provided to the model\. Instead, it allocates additional computation to accessing and using knowledge already available from the model’s parameters\. On TriviaQA, Figure[5\(b\)](https://arxiv.org/html/2610.03052#S5.F5.sf2)shows that CoT yields its largest average accuracy gain near the knowledge boundary\.

This pattern suggests that extra computation is most useful near the knowledge boundary, where the required knowledge is still accessible but access is unstable\. Deep in the interior, direct access is already reliable, so there is little room for improvement\. Near the boundary, the required knowledge is still available, but direct access is unstable\. CoT gives the model more steps to retrieve and use this knowledge, so it has more room to improve than in the interior\. In the exterior, parametric access becomes unreliable\. Since CoT adds computation but no new information, it cannot recover knowledge that the model cannot reliably access, so the gain decreases even as more tokens are generated\.

Existing work improves reasoning efficiency and reduces inference cost by adapting compute to the amount of reasoning or thinking required by a query\([Wang et al\., 2025](https://arxiv.org/html/2610.03052#bib.bib6);[Yong et al\., 2025b](https://arxiv.org/html/2610.03052#bib.bib7);[Li et al\., 2026](https://arxiv.org/html/2610.03052#bib.bib8)\)\. Our findings complement this line of work by showing that knowledge accessibility can serve as a pre\-generation signal for when additional computation is likely to help\.

#### External retrieval is most useful outside the boundary\.

RAG differs from the previous interventions because it does not rely only on knowledge already available from the model’s parameters\. Instead, it supplies external information when parametric access is weak\. On TriviaQA, Figure[5\(c\)](https://arxiv.org/html/2610.03052#S5.F5.sf3)shows that the average benefit of RAG increases with knowledge distance and is largest in the exterior\. As queries move farther outside the accessible region, the required knowledge becomes less reliably available from the model’s parameters, which makes external information increasingly useful\.

This finding is closely related to adaptive RAG, which aims to decide when retrieval should be invoked at inference time\. Existing methods make this decision using signals such as generation confidence, self\-reflection, or query complexity\([Gao et al\., 2023](https://arxiv.org/html/2610.03052#bib.bib9);[Jiang et al\., 2023](https://arxiv.org/html/2610.03052#bib.bib10);[Jeong et al\., 2024](https://arxiv.org/html/2610.03052#bib.bib11)\)\. Our results complement this line of work by showing that, in factual QA, the geometry of knowledge accessibility provides a pre\-generation signal for when external retrieval is likely to help\.

#### A unified pattern of accessibility interventions\.

The three interventions affect knowledge access in different ways, and their benefits vary across the accessibility space\. In the*interior*, where parametric access is reliable, query rewriting can recover failures caused by how the query is expressed\. Near the*boundary*, where access becomes unstable, additional reasoning yields its largest gains\. In the*exterior*, where parametric access is unreliable, external retrieval becomes more useful\.

These results reveal a clear shift\. As queries move from the*interior*\(reliable access\), through the*boundary*\(unstable access\), to the*exterior*\(unreliable access\), the useful strategy shifts from query rewriting→\\rightarrowadditional computation→\\rightarrowexternal retrieval\. Because knowledge distance is available before generation, this geometry may provide a simple signal for deciding how a model should intervene on its own inference process, which may help reduce inference cost\([Wang et al\., 2026](https://arxiv.org/html/2610.03052#bib.bib37)\)\.

## 6Related Work

Prior work has studied the knowledge boundary of large language models, i\.e\., whether a model can distinguish queries it can answer from those it cannot\([Kadavath et al\., 2022](https://arxiv.org/html/2610.03052#bib.bib18);[Yin et al\., 2023](https://arxiv.org/html/2610.03052#bib.bib21);[Yin et al\., 2024](https://arxiv.org/html/2610.03052#bib.bib19);[Li et al\., 2025](https://arxiv.org/html/2610.03052#bib.bib20)\)\. Several studies have further used LLM internal states to probe knowledge boundaries\([Slobodkin et al\., 2023](https://arxiv.org/html/2610.03052#bib.bib22);[Xiao et al\., 2025](https://arxiv.org/html/2610.03052#bib.bib42)\)\. Some of them estimate the boundary from the query alone before generation, which provides a useful signal for deciding how to answer a query\([Ni et al\., 2025](https://arxiv.org/html/2610.03052#bib.bib23);[Chen et al\., 2026](https://arxiv.org/html/2610.03052#bib.bib24)\)\.

However, these studies predict observed correctness directly, without decoupling whether errors are from limited knowledge recall or from failures in reasoning steps\([Press et al\., 2023](https://arxiv.org/html/2610.03052#bib.bib36);[Li and Goyal, 2025](https://arxiv.org/html/2610.03052#bib.bib35);[Yang et al\., 2026](https://arxiv.org/html/2610.03052#bib.bib34);[Cheang et al\., 2026](https://arxiv.org/html/2610.03052#bib.bib25);[Wang et al\., 2026](https://arxiv.org/html/2610.03052#bib.bib37)\)\. More importantly, they focus mainly on whether knowledge boundary information is encoded in hidden states, rather than how such information is organized geometrically in representation space\. In this work, we separate knowledge accessibility from reasoning and study the geometry of knowledge accessibility in query representations\.

## 7Conclusion

This work shifts the study of the knowledge accessibility from whether accessibility is encoded in LLM representations to how it is geometrically organized\. We find that accessible queries lie close to a common center\. Accessibility drops as queries move farther from it\. It also tracks what the model can access rather than whether it reasons correctly\. Across datasets, the centers differ but the accessibility ordering transfers\. The geometry also indicates what kind of help a query needs\. Rewriting helps most in the interior, reasoning helps most near the boundary, and retrieval helps most beyond it\. Because this signal is available before generation, it suggests that models could choose whether to answer directly, reason more, or retrieve information\([Chen et al\., 2026](https://arxiv.org/html/2610.03052#bib.bib24)\)\.

Our study has limitations that suggest several directions for future work\. We focus on single\-fact queries and use correctness as a behavioral proxy for accessibility\. We do not separate knowledge that is never stored from knowledge that is stored but not reached\([Allen\-Zhu and Li, 2024](https://arxiv.org/html/2610.03052#bib.bib41)\)\. Our evidence is also correlational\. Whether moving a representation toward the accessibility center can causally impact recall remains an open question\. Other open questions are how the center emerges from pre\-training\([Papyan et al\., 2020](https://arxiv.org/html/2610.03052#bib.bib12)\)and whether the geometry can be extended to reasoning capability\.

## References

- Allen\-Zhu and Li \(2024\)Z\. Allen\-Zhu and Y\. LiPhysics of language models: part 3\.1, knowledge storage and extraction\.InForty\-first International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=5x788rqbcj)Cited by:[§7](https://arxiv.org/html/2610.03052#S7.p2.1)\.
- Cheanget al\.\(2026\)C\. S\. Cheang, H\. P\. Chan, W\. Zhang, and Y\. DengDo LLMs really know what they don’t know? internal states mainly reflect knowledge recall rather than truthfulness\.InFindings of the Association for Computational Linguistics: ACL 2026,M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 713–730\.External Links:[Link](https://aclanthology.org/2026.findings-acl.34/),[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.34),ISBN 979\-8\-89176\-395\-1Cited by:[§6](https://arxiv.org/html/2610.03052#S6.p2.1)\.
- Chenet al\.\(2026\)L\. Chen, G\. de Melo, F\. M\. Suchanek, and G\. VaroquauxQuery\-level uncertainty in large language models\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=11QZITAMUO)Cited by:[§1](https://arxiv.org/html/2610.03052#S1.p1.1),[§1](https://arxiv.org/html/2610.03052#S1.p6.1),[§2\.2](https://arxiv.org/html/2610.03052#S2.SS2.p2.1),[§6](https://arxiv.org/html/2610.03052#S6.p1.1),[§7](https://arxiv.org/html/2610.03052#S7.p1.1)\.
- Chenet al\.\(2025\)L\. Chen, A\. Dejl, and F\. ToniIdentifying query\-relevant neurons in large language models for long\-form texts\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.39,pp\. 23595–23604\.Cited by:[§1](https://arxiv.org/html/2610.03052#S1.p2.1)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano,et al\.Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§4\.1](https://arxiv.org/html/2610.03052#S4.SS1.p2.1)\.
- Daiet al\.\(2022\)D\. Dai, L\. Dong, Y\. Hao, Z\. Sui, B\. Chang, and F\. WeiKnowledge neurons in pretrained transformers\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),S\. Muresan, P\. Nakov, and A\. Villavicencio \(Eds\.\),Dublin, Ireland,pp\. 8493–8502\.External Links:[Link](https://aclanthology.org/2022.acl-long.581/),[Document](https://dx.doi.org/10.18653/v1/2022.acl-long.581)Cited by:[§1](https://arxiv.org/html/2610.03052#S1.p2.1)\.
- Ethayarajh \(2019\)K\. EthayarajhHow contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT\-2 embeddings\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\),K\. Inui, J\. Jiang, V\. Ng, and X\. Wan \(Eds\.\),Hong Kong, China,pp\. 55–65\.External Links:[Link](https://aclanthology.org/D19-1006/),[Document](https://dx.doi.org/10.18653/v1/D19-1006)Cited by:[§3\.1](https://arxiv.org/html/2610.03052#S3.SS1.p2.1)\.
- Farquharet al\.\(2024\)S\. Farquhar, J\. Kossen, L\. Kuhn, and Y\. GalDetecting hallucinations in large language models using semantic entropy\.Nature630\(8017\),pp\. 625–630\.Cited by:[§4\.4](https://arxiv.org/html/2610.03052#S4.SS4.SSS0.Px3.p1.1)\.
- Gaoet al\.\(2023\)Y\. Gao, Y\. Xiong, X\. Gao, K\. Jia, J\. Pan, Y\. Bi, Y\. Dai, J\. Sun, M\. Wang, and H\. WangRetrieval\-augmented generation for large language models: a survey\.arXiv preprint arXiv:2312\.10997\.Cited by:[§5](https://arxiv.org/html/2610.03052#S5.SS0.SSS0.Px3.p2.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§4\.1](https://arxiv.org/html/2610.03052#S4.SS1.p1.1)\.
- Huiet al\.\(2024\)B\. Hui, J\. Yang, Z\. Cui, J\. Yang, D\. Liu, L\. Zhang, T\. Liu, J\. Zhang, B\. Yu, K\. Lu,et al\.Qwen2\. 5\-coder technical report\.arXiv preprint arXiv:2409\.12186\.Cited by:[§4\.1](https://arxiv.org/html/2610.03052#S4.SS1.p1.1)\.
- Jeonget al\.\(2024\)S\. Jeong, J\. Baek, S\. Cho, S\. J\. Hwang, and J\. ParkAdaptive\-RAG: learning to adapt retrieval\-augmented large language models through question complexity\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),K\. Duh, H\. Gomez, and S\. Bethard \(Eds\.\),Mexico City, Mexico,pp\. 7036–7050\.External Links:[Link](https://aclanthology.org/2024.naacl-long.389/),[Document](https://dx.doi.org/10.18653/v1/2024.naacl-long.389)Cited by:[§5](https://arxiv.org/html/2610.03052#S5.SS0.SSS0.Px3.p2.1)\.
- Jianget al\.\(2020\)Z\. Jiang, F\. F\. Xu, J\. Araki, and G\. NeubigHow can we know what language models know?\.Transactions of the Association for Computational Linguistics8,pp\. 423–438\.External Links:[Link](https://aclanthology.org/2020.tacl-1.28/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00324)Cited by:[§5](https://arxiv.org/html/2610.03052#S5.SS0.SSS0.Px1.p3.1)\.
- Jianget al\.\(2023\)Z\. Jiang, F\. Xu, L\. Gao, Z\. Sun, Q\. Liu, J\. Dwivedi\-Yu, Y\. Yang, J\. Callan, and G\. NeubigActive retrieval augmented generation\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 7969–7992\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.495/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.495)Cited by:[§5](https://arxiv.org/html/2610.03052#S5.SS0.SSS0.Px3.p2.1)\.
- Joshiet al\.\(2017\)M\. Joshi, E\. Choi, D\. Weld, and L\. ZettlemoyerTriviaQA: a large scale distantly supervised challenge dataset for reading comprehension\.InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),R\. Barzilay and M\. Kan \(Eds\.\),Vancouver, Canada,pp\. 1601–1611\.External Links:[Link](https://aclanthology.org/P17-1147/),[Document](https://dx.doi.org/10.18653/v1/P17-1147)Cited by:[§4\.1](https://arxiv.org/html/2610.03052#S4.SS1.p2.1)\.
- Kadavathet al\.\(2022\)S\. Kadavath, T\. Conerly, A\. Askell, T\. Henighan, D\. Drain, E\. Perez, N\. Schiefer, Z\. Hatfield\-Dodds, N\. DasSarma, E\. Tran\-Johnson,et al\.Language models \(mostly\) know what they know\.arXiv preprint arXiv:2207\.05221\.Cited by:[§1](https://arxiv.org/html/2610.03052#S1.p1.1),[§6](https://arxiv.org/html/2610.03052#S6.p1.1)\.
- Kingma and Ba \(2014\)D\. P\. Kingma and J\. BaAdam: a method for stochastic optimization\.arXiv preprint arXiv:1412\.6980\.Cited by:[§A\.2](https://arxiv.org/html/2610.03052#A1.SS2.SSS0.Px5.p1.1)\.
- Li and Goyal \(2025\)A\. O\. Li and T\. GoyalMemorization vs\. reasoning: updating LLMs with new knowledge\.InFindings of the Association for Computational Linguistics: ACL 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 25853–25874\.External Links:[Link](https://aclanthology.org/2025.findings-acl.1326/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.1326),ISBN 979\-8\-89176\-256\-5Cited by:[§6](https://arxiv.org/html/2610.03052#S6.p2.1)\.
- Liet al\.\(2025\)M\. Li, Y\. Zhao, W\. Zhang, S\. Li, W\. Xie, S\. Ng, T\. Chua, and Y\. DengKnowledge boundary of large language models: a survey\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 5131–5157\.External Links:[Link](https://aclanthology.org/2025.acl-long.256/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.256),ISBN 979\-8\-89176\-251\-0Cited by:[§1](https://arxiv.org/html/2610.03052#S1.p1.1),[§6](https://arxiv.org/html/2610.03052#S6.p1.1)\.
- Liet al\.\(2026\)Z\. Li, Q\. Dong, J\. Ma, D\. Zhang, K\. Jia, and Z\. SuiSelfBudgeter: adaptive token allocation for efficient LLM reasoning\.InFindings of the Association for Computational Linguistics: ACL 2026,M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 21135–21156\.External Links:[Link](https://aclanthology.org/2026.findings-acl.1063/),[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.1063),ISBN 979\-8\-89176\-395\-1Cited by:[§5](https://arxiv.org/html/2610.03052#S5.SS0.SSS0.Px2.p3.1)\.
- Niet al\.\(2025\)S\. Ni, K\. Bi, J\. Guo, L\. Yu, B\. Bi, and X\. ChengTowards fully exploiting LLM internal states to enhance knowledge boundary perception\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 24315–24329\.External Links:[Link](https://aclanthology.org/2025.acl-long.1184/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.1184),ISBN 979\-8\-89176\-251\-0Cited by:[§1](https://arxiv.org/html/2610.03052#S1.p1.1),[§6](https://arxiv.org/html/2610.03052#S6.p1.1)\.
- Papyanet al\.\(2020\)V\. Papyan, X\. Y\. Han, and D\. L\. DonohoPrevalence of neural collapse during the terminal phase of deep learning training\.Proceedings of the National Academy of Sciences117\(40\),pp\. 24652–24663\.Cited by:[§2\.2](https://arxiv.org/html/2610.03052#S2.SS2.p3.1),[§7](https://arxiv.org/html/2610.03052#S7.p2.1)\.
- Petroniet al\.\(2019\)F\. Petroni, T\. Rocktäschel, S\. Riedel, P\. Lewis, A\. Bakhtin, Y\. Wu, and A\. MillerLanguage models as knowledge bases?\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\),K\. Inui, J\. Jiang, V\. Ng, and X\. Wan \(Eds\.\),Hong Kong, China,pp\. 2463–2473\.External Links:[Link](https://aclanthology.org/D19-1250/),[Document](https://dx.doi.org/10.18653/v1/D19-1250)Cited by:[§1](https://arxiv.org/html/2610.03052#S1.p1.1),[§1](https://arxiv.org/html/2610.03052#S1.p3.1),[§2](https://arxiv.org/html/2610.03052#S2.p2.1)\.
- Presset al\.\(2023\)O\. Press, M\. Zhang, S\. Min, L\. Schmidt, N\. Smith, and M\. LewisMeasuring and narrowing the compositionality gap in language models\.InFindings of the Association for Computational Linguistics: EMNLP 2023,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 5687–5711\.External Links:[Link](https://aclanthology.org/2023.findings-emnlp.378/),[Document](https://dx.doi.org/10.18653/v1/2023.findings-emnlp.378)Cited by:[§1](https://arxiv.org/html/2610.03052#S1.p2.1),[§1](https://arxiv.org/html/2610.03052#S1.p3.1),[§2](https://arxiv.org/html/2610.03052#S2.p1.1),[§6](https://arxiv.org/html/2610.03052#S6.p2.1)\.
- Robertset al\.\(2020\)A\. Roberts, C\. Raffel, and N\. ShazeerHow much knowledge can you pack into the parameters of a language model?\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),B\. Webber, T\. Cohn, Y\. He, and Y\. Liu \(Eds\.\),Online,pp\. 5418–5426\.External Links:[Link](https://aclanthology.org/2020.emnlp-main.437/),[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.437)Cited by:[§2](https://arxiv.org/html/2610.03052#S2.p2.1)\.
- Ruffet al\.\(2018\)L\. Ruff, R\. Vandermeulen, N\. Goernitz, L\. Deecke, S\. A\. Siddiqui, A\. Binder, E\. Müller, and M\. KloftDeep one\-class classification\.InInternational conference on machine learning,pp\. 4393–4402\.Cited by:[§1](https://arxiv.org/html/2610.03052#S1.p4.1),[§3\.1](https://arxiv.org/html/2610.03052#S3.SS1.p1.1)\.
- Slobodkinet al\.\(2023\)A\. Slobodkin, O\. Goldman, A\. Caciularu, I\. Dagan, and S\. RavfogelThe curious case of hallucinatory \(un\)answerability: finding truths in the hidden states of over\-confident large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 3607–3625\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.220/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.220)Cited by:[§6](https://arxiv.org/html/2610.03052#S6.p1.1)\.
- Snellet al\.\(2017\)J\. Snell, K\. Swersky, and R\. ZemelPrototypical networks for few\-shot learning\.Advances in neural information processing systems30\.Cited by:[§3\.1](https://arxiv.org/html/2610.03052#S3.SS1.p1.1)\.
- Srivastavaet al\.\(2014\)N\. Srivastava, G\. Hinton, A\. Krizhevsky, I\. Sutskever, and R\. SalakhutdinovDropout: a simple way to prevent neural networks from overfitting\.The journal of machine learning research15\(1\),pp\. 1929–1958\.Cited by:[§A\.2](https://arxiv.org/html/2610.03052#A1.SS2.SSS0.Px2.p1.1)\.
- Trivediet al\.\(2022\)H\. Trivedi, N\. Balasubramanian, T\. Khot, and A\. SabharwalMuSiQue: multihop questions via single\-hop question composition\.Transactions of the Association for Computational Linguistics10,pp\. 539–554\.External Links:[Link](https://aclanthology.org/2022.tacl-1.31/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00475)Cited by:[§4\.1](https://arxiv.org/html/2610.03052#S4.SS1.p2.1)\.
- Wanget al\.\(2025\)X\. Wang, S\. Feng, Y\. Li, P\. Yuan, Y\. Zhang, C\. Tan, B\. Pan, Y\. Hu, and K\. LiMake every penny count: difficulty\-adaptive self\-consistency for cost\-efficient reasoning\.InFindings of the Association for Computational Linguistics: NAACL 2025,L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),Albuquerque, New Mexico,pp\. 6919–6932\.External Links:[Link](https://aclanthology.org/2025.findings-naacl.383/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-naacl.383),ISBN 979\-8\-89176\-195\-7Cited by:[§5](https://arxiv.org/html/2610.03052#S5.SS0.SSS0.Px2.p3.1)\.
- Wanget al\.\(2026\)Z\. Wang, Y\. Ding, Z\. Wang, J\. Guo, Z\. Zhou, J\. Dong, and C\. ChenX\-router: decoupling knowledge and reasoning for cost\-effective LLM inference\.InFindings of the Association for Computational Linguistics: ACL 2026,M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 19856–19874\.External Links:[Link](https://aclanthology.org/2026.findings-acl.994/),[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.994),ISBN 979\-8\-89176\-395\-1Cited by:[§5](https://arxiv.org/html/2610.03052#S5.SS0.SSS0.Px4.p2.1),[§6](https://arxiv.org/html/2610.03052#S6.p2.1)\.
- Welblet al\.\(2017\)J\. Welbl, N\. F\. Liu, and M\. GardnerCrowdsourcing multiple choice science questions\.InProceedings of the 3rd Workshop on Noisy User\-generated Text,L\. Derczynski, W\. Xu, A\. Ritter, and T\. Baldwin \(Eds\.\),Copenhagen, Denmark,pp\. 94–106\.External Links:[Link](https://aclanthology.org/W17-4413/),[Document](https://dx.doi.org/10.18653/v1/W17-4413)Cited by:[§4\.1](https://arxiv.org/html/2610.03052#S4.SS1.p2.1)\.
- Wenet al\.\(2016\)Y\. Wen, K\. Zhang, Z\. Li, and Y\. QiaoA discriminative feature learning approach for deep face recognition\.InEuropean conference on computer vision,pp\. 499–515\.Cited by:[§1](https://arxiv.org/html/2610.03052#S1.p4.1),[§3\.1](https://arxiv.org/html/2610.03052#S3.SS1.p1.1)\.
- Wu and Papyan \(2024\)R\. Wu and V\. PapyanLinguistic collapse: neural collapse in \(large\) language models\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=G0LfcMiRkc)Cited by:[§2\.2](https://arxiv.org/html/2610.03052#S2.SS2.p3.1)\.
- Xiaoet al\.\(2025\)C\. Xiao, H\. P\. Chan, H\. Zhang, M\. Aljunied, L\. Bing, N\. Al Moubayed, and Y\. RongAnalyzing LLMs’ knowledge boundary cognition across languages through the lens of internal representations\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 24099–24115\.External Links:[Link](https://aclanthology.org/2025.acl-long.1174/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.1174),ISBN 979\-8\-89176\-251\-0Cited by:[§2\.2](https://arxiv.org/html/2610.03052#S2.SS2.p2.1),[§6](https://arxiv.org/html/2610.03052#S6.p1.1)\.
- Yanget al\.\(2026\)M\. Yang, J\. Gao, and J\. WuDecoupling knowledge and reasoning in llms: an exploration using cognitive dual\-system theory\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 34268–34276\.Cited by:[§1](https://arxiv.org/html/2610.03052#S1.p2.1),[§2](https://arxiv.org/html/2610.03052#S2.p1.1),[§6](https://arxiv.org/html/2610.03052#S6.p2.1)\.
- Yinet al\.\(2024\)X\. Yin, X\. Zhang, J\. Ruan, and X\. WanBenchmarking knowledge boundary for large language models: a different perspective on model evaluation\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 2270–2286\.External Links:[Link](https://aclanthology.org/2024.acl-long.124/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.124)Cited by:[§6](https://arxiv.org/html/2610.03052#S6.p1.1)\.
- Yinet al\.\(2023\)Z\. Yin, Q\. Sun, Q\. Guo, J\. Wu, X\. Qiu, and X\. HuangDo large language models know what they don’t know?\.InFindings of the Association for Computational Linguistics: ACL 2023,A\. Rogers, J\. Boyd\-Graber, and N\. Okazaki \(Eds\.\),Toronto, Canada,pp\. 8653–8665\.External Links:[Link](https://aclanthology.org/2023.findings-acl.551/),[Document](https://dx.doi.org/10.18653/v1/2023.findings-acl.551)Cited by:[§6](https://arxiv.org/html/2610.03052#S6.p1.1)\.
- Yonget al\.\(2025a\)X\. Yong, X\. Zhou, Y\. Zhang, J\. Li, Y\. Zheng, and X\. WuThink or not? exploring thinking efficiency in large reasoning models via an information\-theoretic lens\.InAdvances in Neural Information Processing Systems,D\. Belgrave, C\. Zhang, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, and N\. Chen \(Eds\.\),Vol\.38, Main Conference,pp\. 2787–2827\.External Links:[Document](https://dx.doi.org/10.52202/085713-0098),[Link](https://proceedings.neurips.cc/paper_files/paper/2025/file/04185b5ae2d450ef39bd53c0ec4802cb-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2610.03052#S1.p6.1)\.
- Yonget al\.\(2025b\)X\. Yong, X\. Zhou, Y\. Zhang, J\. Li, Y\. Zheng, and X\. WuThink or not? exploring thinking efficiency in large reasoning models via an information\-theoretic lens\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=DpOSndSOZz)Cited by:[§5](https://arxiv.org/html/2610.03052#S5.SS0.SSS0.Px2.p3.1)\.
- Zhonget al\.\(2023\)Z\. Zhong, Z\. Wu, C\. Manning, C\. Potts, and D\. ChenMQuAKE: assessing knowledge editing in language models via multi\-hop questions\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 15686–15702\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.971/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.971)Cited by:[§4\.3](https://arxiv.org/html/2610.03052#S4.SS3.p1.1)\.

## Appendix AImplementation Details

### A\.1Representations and Correctness Labels

#### Query representations\.

For each query, the model receives the query alone, and we extract the hidden state𝒉⁡\(x\)\{\\bm\{h\}\}\(x\)at the final non\-padding query token before any answer token is generated\.

#### Standardization\.

Before fitting any geometry, each representation dimension is standardized with the mean𝝁\\bm\{\\mu\}and standard deviation𝝈\\bm\{\\sigma\}computed on the training split\. All geometries operate on standardized representations, so throughout the paper𝒉⁡\(x\)\{\\bm\{h\}\}\(x\)denotes the standardized representation\.

#### Correctness labels\.

Correctnesscπ​\(x\)c\_\{\\pi\}\(x\)\(Eq\.[1](https://arxiv.org/html/2610.03052#S2.E1)\) is decided byMistral\-Large, used as a semantic\-equivalence judge with temperature zero\. Given the queryxx, a response, and the reference answer set𝕐⁡\(x\)\{\\mathbb\{Y\}\}\(x\)including its aliases, the judge decides whether the response expresses one of the reference answers\. Unless stated otherwise,π\\piis greedy decoding\.

### A\.2Geometry Models and Training

The four accessibility scores are listed in Table[1](https://arxiv.org/html/2610.03052#S3.T1)\. They are trained as follows\.

#### Linear\.

Logistic regression with inverse regularization strengthC=1C=1and the L\-BFGS solver\.

#### MLP\.

One hidden layer withm=128m=128units, ReLU activation, and dropout\([Srivastava et al\., 2014](https://arxiv.org/html/2610.03052#bib.bib27)\)of0\.10\.1\. We use a batch size of 128 and early stopping with a patience of 10 epochs based on validation AUROC\.

#### Centered\.

The center𝒄\{\\bm\{c\}\}is initialized to the mean representation of the correctly answered training queries and trained for 600 epochs with a soft\-margin objective\. The margin is selected from\{5,7,10,15,20\}\\\{5,7,10,15,20\\\}by validation AUROC, separately for each layer and random seed\.

#### A\-Centered\.

A\-Centered uses the distance in Eq\.[3](https://arxiv.org/html/2610.03052#S3.E3)with𝑨\{\\bm\{A\}\}\. The dimension weights are positive and have geometric mean one, which removes their arbitrary global scale\. Training has two stages\. We first fit Centered and use its learned center to initialize A\-Centered\. We then train A\-Centered for 300 epochs\. The margin is inherited from the validation\-selected Centered model\.

#### Optimization\.

MLP, Centered, and A\-Centered are trained with Adam\([Kingma and Ba, 2014](https://arxiv.org/html/2610.03052#bib.bib26)\)using a learning rate of10−210^\{\-2\}\.

#### Data splits and seeds\.

The 40%/10%/50% train/validation/test splits \(Section[4](https://arxiv.org/html/2610.03052#S4)\) are stratified by the correctness label\. For each seed, the same split is shared by all layers and all methods\. The five seeds are\{42,43,44,45,46\}\\\{42,43,44,45,46\\\}\.

## Appendix BDetails of the Geometry Analyses

This section follows the order of Section[4](https://arxiv.org/html/2610.03052#S4)\.

### B\.1Testing the Centered Inductive Bias

Figure B\.1:Learned norm coefficientα\\alphain Linear\+Norm across layers on TriviaQA with Llama\-8B\. The line shows the mean over five random seeds and the shaded region shows±1\\pm 1standard deviation\.This section complements the Linear\+Norm comparison in Section[4\.2](https://arxiv.org/html/2610.03052#S4.SS2)\. Linear\+Norm uses the same splits as the other geometries, and all preprocessing statistics are fitted on the training split only\. It is trained as a regularized logistic model, with the inverse regularization strength selected fromC∈\{0\.01,0\.1,1,10\}C\\in\\\{0\.01,0\.1,1,10\\\}by validation AUROC\.

Figure[B\.1](https://arxiv.org/html/2610.03052#A2.F1)shows the learned coefficientα\\alphaacross layers\. Althoughα\\alphavaries across layers, its average is negative\. Forα<0\\alpha<0, completing the square gives

𝒘⊤​𝒉\+α​‖𝒉‖22\+b=−\|α\|​‖𝒉−𝒄~‖22\+const,𝒄~=−𝒘2​α,\{\\bm\{w\}\}^\{\\top\}\{\\bm\{h\}\}\+\\alpha\\\|\{\\bm\{h\}\}\\\|\_\{2\}^\{2\}\+b=\-\|\\alpha\|\\,\\bigl\\\|\{\\bm\{h\}\}\-\\tilde\{\{\\bm\{c\}\}\}\\bigr\\\|\_\{2\}^\{2\}\+\\mathrm\{const\},\\qquad\\tilde\{\{\\bm\{c\}\}\}=\-\\frac\{\{\\bm\{w\}\}\}\{2\\alpha\},\(B\.1\)which is equivalent to the Centered score after positive rescaling, with an implied center𝒄~\\tilde\{\{\\bm\{c\}\}\}\. Thus, even without an explicit center, Linear\+Norm tends to learn a centered structure\. Together with the stronger performance of Centered, this supports the centered structure as a useful inductive bias rather than an effect of greater model capacity\.

### B\.2Controlled MQuAKE Evaluation

We use MQuAKE to separate failures in knowledge accessibility from failures in reasoning \(Table[3](https://arxiv.org/html/2610.03052#S4.T3)\)\. Each example contains a multi\-hop query together with the single\-hop questions that probe the facts required to answer it\. We evaluate 1,000 examples withLlama\-3\.1\-8B\-Instruct\.

#### Behavioral groups\.

All single\-hop questions and the multi\-hop query are answered with the same prompt and greedy decoding\. Correct recall of all single\-hop questions is used as a behavioral proxy for accessible knowledge \(KK\), and correctness of the multi\-hop query as reasoning success \(RR\)\. Combining the two outcomes gives four groups:*Success*\(all single\-hop questions and the multi\-hop query correct\),*Reasoning Failure*\(all single\-hop questions correct, multi\-hop query incorrect\),*Knowledge Failure*\(at least one single\-hop question and the multi\-hop query incorrect\), and*Recovered Success*\(at least one single\-hop question incorrect, multi\-hop query correct\)\.

#### Frozen TriviaQA geometry\.

No geometry is fitted on MQuAKE\. We freeze the A\-Centered geometry learned on TriviaQA\. Each canonical MQuAKE query is represented exactly as in Appendix[A\.1](https://arxiv.org/html/2610.03052#A1.SS1)at the selected TriviaQA layer, and its distance is computed under the frozen geometry\.

### B\.3Behavior of Knowledge Accessibility

![Refer to caption](https://arxiv.org/html/2610.03052v1/qwen14b_ellipsoidal_knowledge_quantile_shells.png)\(a\)Accuracy vs\. distance
\(b\)Correctness stability\(c\)Answer consistency
Figure B\.2:Behavior of knowledge accessibility for Qwen\-14B on TriviaQA \(counterpart of Figure[3](https://arxiv.org/html/2610.03052#S4.F3)\)\. \(a\) Accessibility decreases with distance from the center\. \(b\) Correctness is least stable near the boundary\. \(c\) Consistency decreases with distance from the center\.This section gives the details for Section[4\.4](https://arxiv.org/html/2610.03052#S4.SS4); the results for Qwen\-14B are shown in Figure[B\.2](https://arxiv.org/html/2610.03052#A2.F2)\.

#### Frozen geometry on held\-out queries\.

All analyses use test queries only\. Their distances are computed under the geometry learned on the training split\.

#### Repeated sampling and correctness stability\.

For each test query, we drawNNindependent responses under a fixed stochastic decoding strategyπ\\pi\. Each response is judged independently as in Appendix[A\.1](https://arxiv.org/html/2610.03052#A1.SS1), which givescπ\(i\)​\(x\)c\_\{\\pi\}^\{\(i\)\}\(x\)\. The empirical correctness probabilityp^π​\(x\)\\hat\{p\}\_\{\\pi\}\(x\)and the correctness stabilitySπ​\(x\)S\_\{\\pi\}\(x\)then follow Eq\.[6](https://arxiv.org/html/2610.03052#S4.E6)\.

#### Answer consistency\.

Answer consistency measures whether repeated generations produce the same surface\-form answer, regardless of whether that answer is correct\. For visualization, queries withκπ​\(x\)≥0\.9\\kappa\_\{\\pi\}\(x\)\\geq 0\.9are marked as*confidently correct*if their modal normalized answer is judged correct, and as*confidently wrong*otherwise\. These categories are only used for visualization\.

### B\.4Cross\-Dataset Transfer and Dataset\-Specific Geometry

Figure B\.3:Transferability of the distance and accuracy relation for Qwen\-14B \(counterpart of Figure[4](https://arxiv.org/html/2610.03052#S4.F4)\)\. Accuracy generally decreases with distance across target datasets\.![Refer to caption](https://arxiv.org/html/2610.03052v1/llama8b_layer18_four_geometry_heatmaps.png)Figure B\.4:Cross\-dataset diagnostics for Llama\-8B \(layer 18\)\. \(a\) AUROC retention, \(b\) distance–accuracy correlation, \(c\) distance between learned centers, and \(d\) correlation between learned dimension weights\. The distance\-based ordering transfers across datasets even though the centers and dimension weights are dataset\-specific\.This section gives the details for Section[4\.5](https://arxiv.org/html/2610.03052#S4.SS5)\. We separate two questions: whether the*relation*between distance and correctness transfers across datasets, and whether the*geometric parameters*themselves \(center and dimension weights\) are shared\.

#### Transfer protocol\.

We use TriviaQA, SciQ, MuSiQue, and GSM8K, with the best\-performing layer selected for Llama\-8B\. Each dataset is used in turn as the sourcess\. Its A\-Centered geometry is applied without relearning\.

#### Distance–accuracy relation\.

Queries of a target datasetttare grouped into distance binsBkB\_\{k\}under the source geometry\. For each bin we compute the mean distance and the accuracy,

D¯k=1\|Bk\|​∑x∈BkDs​\(x\),Acc⁡\(Bk\)=1\|Bk\|​∑x∈Bkc⁡\(x\),\\bar\{D\}\_\{k\}=\\frac\{1\}\{\|B\_\{k\}\|\}\\sum\_\{x\\in B\_\{k\}\}D\_\{s\}\(x\),\\qquad\\mathrm\{Acc\}\(B\_\{k\}\)=\\frac\{1\}\{\|B\_\{k\}\|\}\\sum\_\{x\\in B\_\{k\}\}c\(x\),\(B\.2\)which gives the curves in Figures[4](https://arxiv.org/html/2610.03052#S4.F4)and[B\.3](https://arxiv.org/html/2610.03052#A2.F3)\. The relation is summarized by the Spearman correlation

ρs→t=ρSpearman​\(\{D¯k\}k,\{Acc⁡\(Bk\)\}k\)\.\\rho\_\{s\\rightarrow t\}=\\rho\_\{\\mathrm\{Spearman\}\}\\bigl\(\\\{\\bar\{D\}\_\{k\}\\\}\_\{k\},\\\{\\mathrm\{Acc\}\(B\_\{k\}\)\\\}\_\{k\}\\bigr\)\.\(B\.3\)A negative value means that target queries farther from the source center are answered less accurately\. This only requires the source geometry to preserve the relative ordering of target queries, not to share their center or shape\.

#### Cross\-dataset diagnostics\.

Figure[B\.4](https://arxiv.org/html/2610.03052#A2.F4)reports four pairwise matrices\. In \(a\) and \(b\), rows are source datasets and columns are target datasets\. In \(c\) and \(d\), entries compare the geometries learned independently on each pair of datasets\.

- \(a\)AUROC retention\.The transfer AUROC is normalized by the AUROC of the target’s own geometry,Retentions→t=AUROCs→t/AUROCt→t\\mathrm\{Retention\}\_\{s\\rightarrow t\}=\\mathrm\{AUROC\}\_\{s\\rightarrow t\}/\\mathrm\{AUROC\}\_\{t\\rightarrow t\}\.
- \(b\)Distance–accuracy monotonicity\.Entry\(s,t\)\(s,t\)isρs→t\\rho\_\{s\\rightarrow t\}from Eq\.[B\.3](https://arxiv.org/html/2610.03052#A2.E3)\.
- \(c\)Center variation\.Since each geometry is fitted after dataset\-specific standardization, we first map the learned centers back to the original representation space\. We then measure the distance between centers and normalize it by the overall scale of the representations\. Smaller values indicate more similar centers\.
- \(d\)Dimension\-weight variation\.We similarly map the learned dimension weights back to the original representation space and compare them using Spearman correlation\. A high correlation means that the two geometries emphasize similar dimensions\.

#### Interpretation\.

Panels \(a\) and \(b\) show that transferred geometries retain much of the target\-specific AUROC and that distance–accuracy correlations remain negative for all source–target pairs\. Panels \(c\) and \(d\) show that the learned centers and dimension weights differ across datasets\.

## Appendix CIntervention Experiments

This section provides the experimental details for Section[5](https://arxiv.org/html/2610.03052#S5)\.

### C\.1Shared Setup

#### Geometry and queries\.

All interventions are evaluated on the TriviaQA test set\. We use the same frozen A\-Centered geometry and the same selected layer as in the main experiments\.

#### Evaluation\.

For each query, we compare the correctness of the original greedy answer with the correctness after an intervention\. Correctness is judged as in Appendix[A\.1](https://arxiv.org/html/2610.03052#A1.SS1)\. For CoT and RAG, only the final extracted answer is given to the judge, without the reasoning trace or retrieved context\.

#### Binning\.

Queries are grouped into 20 equal\-frequency bins according to their signed distance to the learned boundary\. Negative distance corresponds to the interior side, while positive distance corresponds to the exterior side\. All curves report raw bin\-level statistics without smoothing or interpolation\.

### C\.2Query Rewriting

#### Paraphrase generation\.

We useMistral\-Largewith temperature0\.80\.8to generate five semantically equivalent paraphrases for each query\. The paraphrases are instructed to preserve the original meaning, while changing wording and sentence structure\.

#### Answering and evaluation\.

The model answers each paraphrase independently using only its internal knowledge\. We use greedy decoding with at most 128 new tokens and do not use chain\-of\-thought or retrieval\. We keep queries for which all five paraphrases receive valid correctness judgments\.

For each original query, we compute the fraction of paraphrases answered correctly\. We also measure paraphrase stability from the consistency of the five correctness outcomes\. The rewriting gain reported in Figure[5](https://arxiv.org/html/2610.03052#S5.F5)is the change in correctness relative to the original query\.

### C\.3Chain\-of\-Thought

The model is prompted to reason step by step using only its internal knowledge\. We allow at most five short reasoning steps before the final answer\. Generation uses greedy decoding with at most 2,048 new tokens\.

This intervention adds computation but no external information\. We record both answer correctness and the number of generated tokens, which allows us to compare the benefit and cost of additional reasoning across different knowledge\-boundary regions\.

### C\.4Retrieval\-Augmented Generation

For retrieval, we use the evidence provided by TriviaQA\. Each test question is matched to its corresponding example in the TriviaQArcvalidation split\. We use the highest\-ranked non\-emptysearch\_contextand truncate it to at most 4,096 tokens\.

The retrieved context is prepended to the question, and the model is instructed to answer using it as external evidence\. We do not request chain\-of\-thought so that the intervention mainly reflects the effect of additional information rather than additional reasoning\. Generation uses greedy decoding with at most 256 new tokens\.

Queries without a matching TriviaQA example or usable retrieval context are excluded from the RAG analysis\.