Polar probe linearly decodes semantic structures from LLMs
Summary
This paper proposes a Polar Probe that linearly recovers semantic structures from LLM activations by representing entity relations through distance and direction in a learned subspace. Testing across arithmetic, visual scenes, family trees, metro maps, and social interactions shows the code emerges in middle layers, generalizes to new entities, and causally influences model predictions.
arXiv:2605.14125v1 Announce Type: new
Abstract: How do artificial neural networks bind concepts to form complex semantic structures? Here, we propose a simple neural code, whereby the existence and the type of relations between entities are represented by the distance and the direction between their embeddings, respectively. We test this hypothesis in a variety of Large Language Models (LLMs), each input with natural-language descriptions of minimalist tasks from five different domains: arithmetic, visual scenes, family trees, metro maps and social interactions. Results show that the true semantic structures can be linearly recovered with a Polar Probe targeting a subspace of LLMs' layer activations. Second, this code emerges mostly in middle layers and improves with LLM performance. Third, these Polar Probes successfully generalize to new entities and relation types, but degrades with the size of the semantic structure. Finally, the quality of the polar representation correlates with the LLM's ability to answer questions about the semantic structure. Together, these findings suggest that LLMs learn to build complex semantic structures by binding representations with a simple geometrical principle.
View Cached Full Text
Cached at: 05/15/26, 06:19 AM
# Polar probe linearly decodes semantic structures from LLMs
Source: [https://arxiv.org/html/2605.14125](https://arxiv.org/html/2605.14125)
Pablo J\. Diego\-Simón1Pierre Orhan2Yair Lakretz1Jean\-Rémi King3
1LSCP, ENS, PSL, EHESS, CNRS, Paris, France 2Paris Brain Institute, Paris, France 3Meta AI, Paris, France
###### Abstract
How do artificial neural networks bind concepts to form complex semantic structures? Here, we propose a simple neural code, whereby the existence and the type of relations between entities are represented by the distance and the direction between their embeddings, respectively\. We test this hypothesis in a variety of Large Language Models \(LLMs\), each input with natural\-language descriptions of minimalist tasks from five different domains: arithmetic, visual scenes, family trees, metro maps and social interactions\. Results show that the true semantic structures can be linearly recovered with a Polar Probe targeting a subspace of LLMs’ layer activations\. Second, this code emerges mostly in middle layers and improves with LLM performance\. Third, these Polar Probes successfully generalize to new entities and relation types, but degrades with the size of the semantic structure\. Finally, the quality of the polar representation correlates with the LLM’s ability to answer questions about the semantic structure, and intervening on this subspace causally shifts the model’s predictions\. Together, these findings suggest that LLMs learn to build complex semantic structures by binding representations with a simple geometrical principle\.
## 1Introduction
Figure 1:Polar probes linearly read out semantic structures from LLM activations\.A:A natural\-language description specifies a set of entities and their typed relations \(illustrated here for*spatial layout*, where entities are objects and relations are spatial predicates \(left of/right, top of/ below\)\.B:The description corresponds to a semantic structure, formalized as a relational graph whose nodes are entities and whose edges are typed, directed relations\.C:The LLM contextualizes each entity token, yielding a high\-dimensional entity representation in its residual stream\.D:A polar probe, a learned linear transformation from activation space to a probe\-space recovers the relational graph: Euclidean distance between entity representations codes for edge*existence*, while relative direction codes for relation*type*\.Bottom panels:The same scheme applies to four additional domains: spatial layouts, family trees, metro maps, and thematic roles\.##### Compositional representations\.
Human languages constantly require combining words into rich semantic structures – such as family ties, spatial arrangements, or part\-whole connections\. Consider the sentence ‘Bob is Alice’s father, and Mary is her mother’: understanding it requires constructing, on the fly, a representation of the underlying family tree\. Yet, simply allocating a one\-hot feature for each possible combination quickly becomes impractical, and indeed prevents generalization\(Fodor & Pylyshyn,[1988](https://arxiv.org/html/2605.14125#bib.bib12)\)\. As large language models become increasingly able to combine new concepts\(Brown et al\.,[2020](https://arxiv.org/html/2605.14125#bib.bib4)\), it is thus critical to understand*how*the geometry of their activations bind entities to represent compositional structures\.
##### Probing syntactic structures\.
A simple binding principle has recently been evidenced in the context of syntactic representations\. Indeed,Hewitt & Manning \([2019](https://arxiv.org/html/2605.14125#bib.bib18)\)showed with a*Structural Probe*that words that are linked syntactically \(e\.g\., subject \- verb\) are represented more closely in a specific subspace of the LLMs’ activations than words that aren’t \(e\.g\., subject \- object\)\. Building on this proposal,Diego\-Simon et al\. \([2024](https://arxiv.org/html/2605.14125#bib.bib8)\)further showed with a*Polar Probe*that the*type*of syntactic relation can be recovered from the relative*direction*between the two words in this subspace\. In sum, the structure of syntax can be explicitly represented through the relative distances and directions between contextualized word embeddings\.
##### Remaining challenge\.
This binding principle, however, is currently limited to*syntax*\(Müller\-Eberstein et al\.,[2022](https://arxiv.org/html/2605.14125#bib.bib28); Eisape et al\.,[2022](https://arxiv.org/html/2605.14125#bib.bib9); Limisiewicz & Mareček,[2021](https://arxiv.org/html/2605.14125#bib.bib23)\)\. Consequently, it is unclear whether a similar principle may also be at play in*semantic*binding\.
##### Approach\.
To test this hypothesis, we evaluate whether LLMs build subspaces of activations where the relative distance and direction between word embeddings linearly represent the corresponding semantic structure\. To evaluate the generality of our approach, we introduce a synthetic dataset spanning five semantic domains: variable ordinality, spatial layouts, thematic roles, family trees and metro maps, each with distinctive properties\. Each sample in the dataset consists of a text that describes a semantic structure \(e\.g\.,The dog is to the left of the cat\. The cat is below the table\.etc\)\. For each semantic structure, we generate multiple validtextual descriptions, where we randomize the order of relations and entities, to break any correlation between word order and graph structure\. We then input these text descriptions to a variety of LLMs, differing in size and pretraining stage, and train a Polar Probe\(Diego\-Simon et al\.,[2024](https://arxiv.org/html/2605.14125#bib.bib8)\)on their activations, one per layer and per domain\. To verify that the identified representations effectively generalize, we evaluate out\-of\-domain \(OOD\) samples with new entity names and relation surface forms\. Finally, to evaluate the functional role of the polar probe, we test whether the quality of these polar representations predicts the LLMs’ ability to answer questions about the semantic structure\.
## 2Methods
### 2\.1Problem formalization
##### Semantic structures as directed graphs\.
We formalize a semantic structure as a relational graph with labeled nodes \(entities\) and typed directed edges \(relations\)\.
Let𝒱=\{v1,…,v\|𝒱\|\}\\mathcal\{V\}=\\\{v\_\{1\},\\dots,v\_\{\|\\mathcal\{V\}\|\}\\\}be a finite set of entities and𝒯=\{t1,…,t\|𝒯\|\}\\mathcal\{T\}=\\\{t\_\{1\},\\dots,t\_\{\|\\mathcal\{T\}\|\}\\\}a finite set of relation types\. Define
ℰ=\{\(vi,vj,tr\)∈𝒱×𝒱×𝒯:i≠j\}\.\\mathcal\{E\}=\\\{\(v\_\{i\},v\_\{j\},t\_\{r\}\)\\in\\mathcal\{V\}\\times\\mathcal\{V\}\\times\\mathcal\{T\}:\\ i\\neq j\\\}\.A semantic structure is then a graphG=\(𝒱G,ℰG,𝒯G\)G=\(\\mathcal\{V\}\_\{G\},\\mathcal\{E\}\_\{G\},\\mathcal\{T\}\_\{G\}\), where𝒱G⊆𝒱\\mathcal\{V\}\_\{G\}\\subseteq\\mathcal\{V\},𝒯G⊆𝒯\\mathcal\{T\}\_\{G\}\\subseteq\\mathcal\{T\}, and
ℰG=\{\(vi,vj,tr\)∈ℰ:vi,vj∈𝒱G,tr∈𝒯G\}\.\\mathcal\{E\}\_\{G\}=\\\{\(v\_\{i\},v\_\{j\},t\_\{r\}\)\\in\\mathcal\{E\}:\\ v\_\{i\},v\_\{j\}\\in\\mathcal\{V\}\_\{G\},\\ t\_\{r\}\\in\\mathcal\{T\}\_\{G\}\\\}\.
We representGGalgebraically by two objects\. First, the distance matrix
MGρ∈ℕ0\|𝒱G\|×\|𝒱G\|,\(MGρ\)ij=dG\(vi,vj\),M\_\{G\}^\{\\rho\}\\in\\mathbb\{N\}\_\{0\}^\{\|\\mathcal\{V\}\_\{G\}\|\\times\|\\mathcal\{V\}\_\{G\}\|\},\\qquad\(M\_\{G\}^\{\\rho\}\)\_\{ij\}=d\_\{G\}\(v\_\{i\},v\_\{j\}\),wheredGd\_\{G\}is the shortest\-path distance\. Second, the incidence tensor
MGϕ∈\{−1,0,1\}\|𝒱G\|×\|𝒱G\|×\|𝒯\|,\(MGϕ\)ijr=𝟏\[\(vi,vj,tr\)∈ℰG\]−𝟏\[\(vj,vi,tr\)∈ℰG\]\.M\_\{G\}^\{\\phi\}\\in\\\{\-1,0,1\\\}^\{\|\\mathcal\{V\}\_\{G\}\|\\times\|\\mathcal\{V\}\_\{G\}\|\\times\|\\mathcal\{T\}\|\},\\qquad\(M\_\{G\}^\{\\phi\}\)\_\{ijr\}=\\mathbf\{1\}\[\(v\_\{i\},v\_\{j\},t\_\{r\}\)\\in\\mathcal\{E\}\_\{G\}\]\-\\mathbf\{1\}\[\(v\_\{j\},v\_\{i\},t\_\{r\}\)\\in\\mathcal\{E\}\_\{G\}\]\.
Figure 2:Polar probe geometry mirrors the gold semantic structure\.Top:Expected polar probe geometry for semantic structures from every domain\.Bottom:2D PCA of probe\-space entity representations from 10 different descriptions of a semantic structure in the test set; large markers denote entity centroids and lines indicate gold relations\. The projections tend to follow the polar code: direction encodes relation*type*, and Euclidean distance encodes relation*existence*and proximity between entities in the relational graph\.
##### Semantic structures as LLM activations\.
A semantic structureGGcan also be described in natural language\. For simplicity, we consider that each entityvi∈𝒱Gv\_\{i\}\\in\\mathcal\{V\}\_\{G\}is represented by a unique tokenwiw\_\{i\}\. If an LLMs input with such a textual description representsGG, then this structure should be retrievable from its hidden activations𝐡i∈ℝd\\mathbf\{h\}\_\{i\}\\in\\mathbb\{R\}^\{d\}\(Vaswani et al\.,[2017](https://arxiv.org/html/2605.14125#bib.bib36)\)\.
In sum, we seek to identify how the symbolic/graphical representations of semantic structures are represented in the vectorial activations of neural networks\.
##### Polar Probe\.
The probed pairwise distance matrixM^Gρ∈ℝ\|𝒱G\|×\|𝒱G\|\\hat\{M\}\_\{G\}^\{\\rho\}\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\_\{G\}\|\\times\|\\mathcal\{V\}\_\{G\}\|\}is computed as
\(M^Gρ\)ij=∥𝜹ij∥2,\(\\hat\{M\}\_\{G\}^\{\\rho\}\)\_\{ij\}=\\lVert\\boldsymbol\{\\delta\}\_\{ij\}\\rVert\_\{2\},where∥⋅∥2\\lVert\\cdot\\rVert\_\{2\}is theℓ2\\ell\_\{2\}\-norm\.
To assign each relation to a specific direction of the probed space, we learn a prototype vector𝐩r\\mathbf\{p\}\_\{r\}for each relation typer∈𝒯r\\in\\mathcal\{T\}and train the probe so that, whenever a relation of typerrholds between\(vi,vj\)\(v\_\{i\},v\_\{j\}\), the probed difference𝜹ij\\boldsymbol\{\\delta\}\_\{ij\}aligns with𝐩r\\mathbf\{p\}\_\{r\}\. The probed incidence tensorM^Gϕ∈ℝ\|𝒱G\|×\|𝒱G\|×\|𝒯\|\\hat\{M\}\_\{G\}^\{\\phi\}\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\_\{G\}\|\\times\|\\mathcal\{V\}\_\{G\}\|\\times\|\\mathcal\{T\}\|\}records, for each token pair\(wi,wj\)\(w\_\{i\},w\_\{j\}\), the cosine similarity between𝜹ij\\boldsymbol\{\\delta\}\_\{ij\}and every prototype vector:
\(M^Gϕ\)ijr=𝜹ij⋅𝐩r‖𝜹ij‖2‖𝐩r‖2\.\(\\hat\{M\}\_\{G\}^\{\\phi\}\)\_\{ijr\}=\\frac\{\\boldsymbol\{\\delta\}\_\{ij\}\\cdot\\mathbf\{p\}\_\{r\}\}\{\\\|\\boldsymbol\{\\delta\}\_\{ij\}\\\|\_\{2\}\\,\\\|\\mathbf\{p\}\_\{r\}\\\|\_\{2\}\}\.
Figure 3:Semantic structures are most linearly decodable in the middle layers, only in pretrained LLMs\.Spearman’sρ\\rhofor relation existence \(blue\) and type \(orange\) decoded by a polar probe from Llama3\-8B across layers in five domains\. In pretrained models \(solid\), decoding peaks around layers 12–15 and remains high in late layers\. In randomly initialized models \(dashed\), both scores remain close to chance across all layers and domains\.
##### Learning objective\.
Consequently, the loss of the Polar Probe consists of two loss terms\. The structural lossℒs\\mathcal\{L\}\_\{s\}\(Hewitt & Manning,[2019](https://arxiv.org/html/2605.14125#bib.bib18)\)is
ℒs=1\|ℬ\|∑G∈ℬ\(1−Ψ\(M^Gρ,MGρ\)\)\.\\mathcal\{L\}\_\{s\}=\\frac\{1\}\{\|\\mathcal\{B\}\|\}\\sum\_\{G\\in\\mathcal\{B\}\}\\bigl\(1\-\\Psi\(\\hat\{M\}\_\{G\}^\{\\rho\},M\_\{G\}^\{\\rho\}\)\\bigr\)\.
Ψ\(⋅,⋅\)\\Psi\(\\cdot,\\cdot\)is the differentiable Spearman rank correlation over the vectorized upper\-triangular entries of the two distance \(MGρM\_\{G\}^\{\\rho\}andM^Gρ\\hat\{M\}\_\{G\}^\{\\rho\}\); ranks are computed via the Sinkhorn\-based soft\-sorting operator ofBlondel et al\. \([2020](https://arxiv.org/html/2605.14125#bib.bib3)\)\.Ψ\\Psiassesses agreement in relative distance orderings \(higher is better\) and does not assume a common scale or equal\-interval spacing between distances\.
The angular lossℒa\\mathcal\{L\}\_\{a\}\(Diego\-Simon et al\.,[2024](https://arxiv.org/html/2605.14125#bib.bib8)\)is
ℒa=1\|ℬ\|∑G∈ℬ1\|EG\|\|𝒯\|∑\(i,j\)∈EG∑r∈𝒯\(\(M^Gϕ\)ijr−\(MGϕ\)ijr\)2\.\\mathcal\{L\}\_\{a\}=\\frac\{1\}\{\|\\mathcal\{B\}\|\}\\sum\_\{G\\in\\mathcal\{B\}\}\\frac\{1\}\{\|E\_\{G\}\|\\,\|\\mathcal\{T\}\|\}\\sum\_\{\(i,j\)\\in E\_\{G\}\}\\sum\_\{r\\in\\mathcal\{T\}\}\\bigl\(\(\\hat\{M\}^\{\\phi\}\_\{G\}\)\_\{ijr\}\-\(M^\{\\phi\}\_\{G\}\)\_\{ijr\}\\bigr\)^\{2\}\.
The polar probe and the relation prototypes are jointly trained to minimize a weighted \(λ∈ℝ\\lambda\\in\\mathbb\{R\}\) sum of the structural \(ℒs\\mathcal\{L\}\_\{s\}\) and angular \(ℒa\\mathcal\{L\}\_\{a\}\) losses:
\(B∗,\{𝐩r∗\}r∈𝒯\)=argminB,\{𝐩r\}r∈𝒯\(ℒs\+λℒa\),λ\>0\.\(B^\{\*\},\\\{\\mathbf\{p\}\_\{r\}^\{\*\}\\\}\_\{r\\in\\mathcal\{T\}\}\)=\\operatorname\*\{arg\\,min\}\_\{B,\\\{\\mathbf\{p\}\_\{r\}\\\}\_\{r\\in\\mathcal\{T\}\}\}\\bigl\(\\mathcal\{L\}\_\{s\}\+\\lambda\\mathcal\{L\}\_\{a\}\\bigr\),\\qquad\\lambda\>0\.
##### Implementation details\.
For simplicity, we systematically train and evaluate the Polar Probe independently on each of the layers of a given LLM\.111Code and data will be made publicly available with the camera\-ready version of the paper\.After a grid search, we setλ=5\.0\\lambda=5\.0and the learning rate to1×10−51\\times 10^\{\-5\}\. Training runs for 100 epochs, and the probe rank is 512 by default unless stated otherwise\. For each semantic domain, the training set contains 30 graphs, each described in 20 distinct ways\. The validation set contains 50 unseen graphs, each with described 20 times\. All graphs in a dataset have a fixed number of entities; entity names are drawn from a pool of 13 items\. A disjoint pool of 13 entities and relation surface forms is reserved for OOD evaluation\.
### 2\.2Datasets
##### Semantic domains\.
We synthesize five datasets, each containing relational graphs from a different semantic domain\. We group semantic domains into*Euclidean*\(spatial layouts, variable ordinality and thematic roles\) and*non\-Euclidean*\(metro networks and family trees\)\. The criterion is whether the graphs can be faithfully coded with a polar code in a flat \(zero–curvature\) space, or instead require nonzero curvature\.
##### Prompting\.
We use a short, domain\-specific prompt to introduce the LLM to the semantic domain and to incline it to infer the relations between entities\. For more details refer to[Table1](https://arxiv.org/html/2605.14125#S5.T1)\. For example, \(e\.g\.,I am going to describe a family tree, you need to understand the how all family members relate to each other\)\. After this prompt, the relations in the graph are exhaustively described in a random order\. For each relation, the order of the entities is also randomized between both equivalent options \(e\.g\.,James is the dad of JosephorJoseph is the son of James\)\. Therefore, for a given relational graph, there exists multiple textual descriptions\.
##### Post\-prompt\.
Entities that are related tend to be relatively close in the textual descriptions, because we do not have any sentences that describes lack of relations\. Consequently, to ensure that the distance between probed entities is not confounded by the distance between words, we append the list of entities in a fixed order \(e\.g\.,Who are James, Joseph, and Amelia?\) and use those as probed tokens\. To guarantee a one\-to\-one mapping between entities and tokens, entity names where chose to ensure that they are coded by a single\-token\.
#### 2\.2\.1Euclidean graphs\.
We generate Euclidean relational graphs for three semantic domains via[Algorithm1](https://arxiv.org/html/2605.14125#alg1)\.
##### Arithmetic \(Variable ordinality\)\.
First, we investigate ordinality of mathematical variables, such semantic domain consists of a single relation type\. Different variable names are randomly placed on a magnitude axis\. Relational graphs are constructed where entities are mathematical variables \(e\.g\.,*x*,*y*,*z*\) and relations denote relative order between adjacent variables \(e\.g\., greater than/less than\)\.
“xxis greater thanzz\.zzis less thanyy\.”
##### Spatial arrangement \(2D layouts\)\.
Second, we investigate spatial layouts in two dimensions\. Such layouts consist of placing objects on a two\-dimensional regular grid\. Relational graphs are constructed where entities are objects \(e\.g\.,*ball*,*shirt*,*plane*\) and relations denote relative position between adjacent objects \(e\.g\., left of/right of; above/below\)\.
“The*ball*is left of the*plane*\. The*ball*is below the*shirt*\.”
Figure 4:Polar probe performance grows with pretraining, falls with the number of entities in the relational graph, and degrades with out\-of\-distribution \(OOD\) entities and relation surface forms\.Top:Spearman’sρ\\rhofor relation existence \(blue\) and type \(orange\) vs\. pretraining steps at the best layer of OLMo\-7B\.Middle:Polar probe performance vs\. number of entities in the graph at the best layer of Llama3\.1\-8B\.Bottom:Generalization analysis on the best layer of Llama3\.1\-8B to OOD relation surface forms, OOD entity names, and without domain\-specific prompting\.
##### Social interaction \(Thematic roles\)\.
Third, we turn to abstract social interactions, considering only cases where the relational graph admits an exact two\-dimensional Euclidean embedding under a polar code\. We model thematic roles that capture agent–patient interactions within a group of people\. Relational graphs are constructed where entities are professionals \(e\.g\.,*farmer*,*pilot*,*teacher*\), and relations denote interaction types between an agent and a patient \(e\.g\., follows/followed by; helps/helped by\)\.
“The*pilot*follows the*teacher*\. The*farmer*is helped by the*teacher*”
#### 2\.2\.2Non\-Euclidean graphs\.
Some semantic domains are not exactly representable in an Euclidean space following a polar code\. This occurs in two settings; \(1\) when relation composition is non\-commutative \(2\) when relations are many\-to\-many or \(3\) when the underlying graph has non\-Euclidean topology\. Nevertheless, we ask whether a polar code can hold*locally*—within small neighborhoods—even when if not globally coherent\.
##### Family trees\.
Then, we study a semantic domain with non\-commutative and many\-to\-many relations: family trees\. Parent relations compose non\-commutatively \(e\.g\.,*mother*∘\\circ*father*≠\\neq*father*∘\\circ*mother*\), and kinship ties such as*sibling of*are many\-to\-many \(e\.g\., three sisters are each siblings of the others\)\. Relational graphs are constructed where entities are people \(e\.g\.,*Joseph*,*Amelia*,*James*,*Emily*\) and relations are kinship ties \(e\.g\., mother of/father of; daughter of/son of; sister of/brother of\)\. Valid relational graphs are sampled while enforcing classical genealogical constraints \(no cycles through parent links, consistent parentage, and gendered inverse relations\)\. For the “sibling of” relation, because of its many\-to\-many nature, we do not assign it to a prototypical direction; however, we include it in the structural loss computation\.
“*Amelia*is the mother of*James*\.*Joseph*is the father of*James*\.*Emily*is the sister of*James*”
##### Metro maps\.
Finally, we investigate a semantic domain where the graph’s distance metric is not Euclidean\. Metro networks are a canonical example, since the distance between two stops on different lines is determined by network path length rather than Euclidean geometry\. Relational graphs are constructed where entities are metro stops corresponding to city landmarks \(e\.g\.,*lake*,*mall*,*hospital*\) and relations denote the line and direction connecting adjacent stops \(e\.g\., next on line A / previous on line A; next on line B / previous on line B\)\. Transfer hubs occur where the lines intersect, and distances follow shortest\-path length along the metro lines \(including transfers\)\.
“The*mall*is one stop after the*lake*on line A\.The*hospital*is one stop before the*mall*on line B”
### 2\.3Evaluation details
The test set for each dataset comprises 50 held\-out graphs, each described 30 times\. We report Spearman’sρ\\rho, following prior work\(Hewitt & Manning,[2019](https://arxiv.org/html/2605.14125#bib.bib18)\), for \(i\)*relation existence*predictions \(rank correlation between the vectorized upper triangle of predicted vs\. gold pairwise distance matrices\) and \(ii\)*relation type*predictions \(rank correlation between the vectorized prototype\-based predicted incidence tensor vs\. the gold incidence tensor\)\.
Figure 5:Polar probe performance increases with model size and saturates at low probe rank\.Top:Spearman’sρ\\rhofor relation existence \(blue\) and type \(orange\) as a function of model size across the Pythia family, at the middle layer\. Solid lines indicate pretrained models; dashed lines indicate randomly initialized ones\.Bottom:Polar probe relation existence and type scores as a function of probe rank at the best layer of Llama3\.1\-8B\.
### 2\.4Correlation with downstream predictions\.
We evaluate whether probe\-space errors correlate with LLM downstream performance on a Question\-Answering task\. We present500500relational graphs one\-by\-one to the LLM and, in each graph, query a specific relation \(e\.g\.,Which variable is immediately greater thanyy?\)\. For each query, we record the logit assigned to the correct answer\. To account for varying graph complexity, we normalize probe errors within each graph and report Spearman’sρ\\rhobetween the normalized probe error and the correct answer’s logit\. A statistically significant negative correlation \(Spearman’sρ<0\\rho<0\) would be consistent with larger probe errors being associated with lower logit values, suggesting that probe errors may provide a partial diagnostic of the model’s reasoning performance\.
##### Causal interventions
Beyond correlations, we test whether the geometric structure identified by the Polar Probe can*causally*steer model behavior\(Nanda et al\.,[2023](https://arxiv.org/html/2605.14125#bib.bib29); Turner et al\.,[2025](https://arxiv.org/html/2605.14125#bib.bib35)\)\. We use the same QA setup as in the correlation analysis and measure the probability of predicting the correct entity token\. Given a Polar ProbeBB, we map a prototype vector𝐩r\\mathbf\{p\}\_\{r\}back to model space via𝐯=B†𝐩r\\mathbf\{v\}=B^\{\\dagger\}\\mathbf\{p\}\_\{r\}\. We then intervene on the hidden state of the correct token by adding a scaled direction:𝐡←𝐡\+αs𝐯\\mathbf\{h\}\\leftarrow\\mathbf\{h\}\+\\alpha\\,s\\,\\mathbf\{v\}, whereα\\alphaandsscontrol the intervention strength and the directionality of the intervention respectively\. We compare two conditions: \(i\) positive\-prototype steering \(s=1s=1\), \(ii\) negative\-prototype steering \(s=−1s=\-1\)\.
### 2\.5Pretrained Large Language Models\.
We use publicly available, text\-only LLMs\. Most analyses are conducted with Llama3\.1\-8B\(Grattafiori et al\.,[2024](https://arxiv.org/html/2605.14125#bib.bib15)\)\. For the emergence analysis, we use OLMo\-7B\(Groeneveld et al\.,[2024](https://arxiv.org/html/2605.14125#bib.bib16)\), whose training checkpoints are publicly available \. For model\-size analyses, we consider members of the Pythia family\(Biderman et al\.,[2023](https://arxiv.org/html/2605.14125#bib.bib2)\)with varying parameter counts ranging from 10 million to 6\.8 billion\.
## 3Results
##### Layer analysis\.
Across domains, polar probes decode semantic structures most accurately from the*middle*layers of Llama3\-8B\(Grattafiori et al\.,[2024](https://arxiv.org/html/2605.14125#bib.bib15)\)\([Figure3](https://arxiv.org/html/2605.14125#S2.F3)\)\. Spearman’sρ\\rhofor relation existence peaks at∼0\.80\\sim 0\.80and for relation*type*at∼0\.50\\sim 0\.50–0\.700\.70around layers 12–15\. Probes trained on a randomly initialized Llama3\-8B yield scores near0\.00\.0across layers, matching a random baseline \(see also additional baselines in[Figure9](https://arxiv.org/html/2605.14125#S5.F9)\)\. Unlike prior results in syntax\(Hewitt & Manning,[2019](https://arxiv.org/html/2605.14125#bib.bib18); Diego\-Simon et al\.,[2024](https://arxiv.org/html/2605.14125#bib.bib8)\), performance does not fully collapse in deeper layers\. Domain\-wise, type scores diverge: family trees reach comparatively high type accuracy already in shallow layers, likely reflecting lexical gender cues in names \(encoded in the vocabulary embeddings\), whereas metro maps show lower type scores overall and peak later, consistent with their non\-Euclidean, network\-based structure\. We observe similar results when evaluating polar probes on both naturalistic and multilingual variants of the spatial layouts domain \([Figure10](https://arxiv.org/html/2605.14125#S5.F10)\)
##### Emergence during pretraining\.
How does training shape their subspaces to represent semantic structures? To address this issue, we apply the Polar Probe to 9 checkpoints of OLMo\-7B\(Groeneveld et al\.,[2024](https://arxiv.org/html/2605.14125#bib.bib16)\)\. Polar probe performance increases with pretraining steps at the best\-performing layer of OLMo\-7B \([Figure4](https://arxiv.org/html/2605.14125#S2.F4)\)\. Scores for both relation existence and type strengthen gradually and remain largely unsaturated across available checkpoints, suggesting further gains with longer pretraining\. Consistent with the depth analysis, decoding for metro maps emerges later during pretraining than for other domains and attains lower type scores overall\. Overall, these results suggests that the polar coordinate principle is not a trivial property of high dimensional connectionist models, but directly depends on their ability to learn to store, represent, and manipulate knowledge\.
##### Graph complexity\.
Are all semantic structures equally represented in the LLMs? To address this question we train and evaluate polar probes on relational graphs with varying number of entities\. Both existence and type scores decline as the number of entities in the relational graph increases \(measured at the best\-performing layer of Llama3\-8B\) \([Figure4](https://arxiv.org/html/2605.14125#S2.F4)\)\. The drop is especially pronounced for relation*type*in thematic roles and metro maps, exceeding 50% when just two entities are added\. As the entity count grows, the combinatorial space of possible graphs expands rapidly, making the decoding problem substantially harder\.
##### Generalization to new entities and relations\.
While the above analyses are systematically evaluated on semantic structures absent from the training set, the Polar Probe may learn some relations by heart \(e\.g\. plane left of ball = dimension 42\)\. Consequently, we perform the same analyses on semantic structures, for which every entity name or relation surface form is absent from the training set\.[Figure4](https://arxiv.org/html/2605.14125#S2.F4)shows that, at the best\-performing layer of Llama3\-8B, polar probe performance degrades only modestly when the domain\-specific prompt is removed\. Using OOD relation surface forms produces an additional but modest drop\. In contrast, OOD entity names have a substantially larger impact\. Across all settings, polar probe performance remains well above the random baseline, which thus indicates that this polar coordinate system reliably generalizes to new structures\.
Figure 6:Polar probe prototypes steer LLM predictions: Probability of a correct answer under steering at layer 11 of Llama3\-8B\.
##### LLM size\.
To evaluate whether the capacity of the LLMs influenced the geometry of semantic structures, we trained and evaluated polar probes on models from the Pythia suite spanning 14M–6\.9B parameters\(Biderman et al\.,[2023](https://arxiv.org/html/2605.14125#bib.bib2)\)\. Polar probe performance increases with model size, for each of the 5 domains, even thought the probe size is fixed at 128 \([Figure5](https://arxiv.org/html/2605.14125#S2.F5)\)\. This result is not trivially explained by the LLM dimensionality: when trained on randomly initialized LLMs, the polar probes remain close to chance across all sizes\.
##### Polar probe rank\.
To assess whether the Polar Probe relies on dense or sparse representations, we trained and evaluated polar probes for each semantic domain with ranks logarithmically spaced from 2 to 1024\. Polar probe performance saturates at low ranks for both relation existence and type scores\. Across domains, a rank of roughly 32 captures most of the achieved performance \([Figure5](https://arxiv.org/html/2605.14125#S2.F5)\)\. Notably, for variable ordinality, relation type scores peak with only 2 dimensions; beyond these values, increasing rank yields no consistent gains\. Overall, this suggests that semantic structures are represented in a compact subspace of the LLMs\.
##### Causal interventions\.
To examine whether polar probes can*causally*influence downstream predictions, we intervene on Llama3\-8B by steering its activations along the learned positive and negative prototype directions\. Steering shifts the probability of a correct answer in a QA setting \([Figure6](https://arxiv.org/html/2605.14125#S3.F6)\), with the strongest effects observed in the model’s middle layers \([Figures11](https://arxiv.org/html/2605.14125#S5.F11)and[12](https://arxiv.org/html/2605.14125#S5.F12)\)\.
##### Subspace superposition\.
Across domains, the learned subspaces are largely disjoint in the LLM’s activation space \([Figure7](https://arxiv.org/html/2605.14125#S5.F7)\)\. A clear exception is a pronounced overlap between spatial layouts and variable ordinality, consistent with the superposition hypothesis\(Elhage et al\.,[2022](https://arxiv.org/html/2605.14125#bib.bib10)\)\.
## 4Discussion
##### Specific contributions\.
This work demonstrates that the textual description of a new semantic structure can be recovered from the hidden activations of a large language model \(LLM\) with a simple geometric principle\. Specifically, the*existence*and the*type*of a semantic relation between two entities are encoded by the*distance*and the*direction*, respectively, between their embeddings in a subspace of the LLM\.
##### Beyond syntactic trees\.
Our approach extends earlier work on syntactic tree representations in LLMs\(Hewitt & Manning,[2019](https://arxiv.org/html/2605.14125#bib.bib18); Diego\-Simon et al\.,[2024](https://arxiv.org/html/2605.14125#bib.bib8)\), which focused exclusively on universal\-dependency relations between words\. By contrast, we here show that the Polar Probe is not limited to \(1\) syntax or to \(2\) to tree structures, but also extends to a broader class of structures, namely, directed and labeled Euclidean graphs\.
##### Beyond knowledge retrieval\.
Linear probing of language models has been widely explored\(Humphrey et al\.,[1970](https://arxiv.org/html/2605.14125#bib.bib19); Georgopoulos et al\.,[1986](https://arxiv.org/html/2605.14125#bib.bib14); Alain & Bengio,[2017](https://arxiv.org/html/2605.14125#bib.bib1); Conneau et al\.,[2018](https://arxiv.org/html/2605.14125#bib.bib6)\), to show that a broad range of features are linearly decodable from their activations: linguistic properties\(Tenney et al\.,[2019](https://arxiv.org/html/2605.14125#bib.bib34); Jawahar et al\.,[2019](https://arxiv.org/html/2605.14125#bib.bib20); Liu et al\.,[2019](https://arxiv.org/html/2605.14125#bib.bib24)\), spatial and temporal knowledge\(Gurnee & Tegmark,[2024](https://arxiv.org/html/2605.14125#bib.bib17); Chen et al\.,[2023](https://arxiv.org/html/2605.14125#bib.bib5)\), lexical semantics\(Mikolov et al\.,[2013](https://arxiv.org/html/2605.14125#bib.bib27); Park et al\.,[2025b](https://arxiv.org/html/2605.14125#bib.bib31)\), political stances and factuality\(Kim et al\.,[2025](https://arxiv.org/html/2605.14125#bib.bib21); Marks & Tegmark,[2024](https://arxiv.org/html/2605.14125#bib.bib26)\), and numeric values\(Levy & Geva,[2025](https://arxiv.org/html/2605.14125#bib.bib22)\)\. These studies, however, primarily investigate how models retrieve knowledge acquired during training\. By contrast, our present work targets compositional structures that, by design, could not be learned by heart during training\. In this sense, this study closely relates toin\-context learning\(Brown et al\.,[2020](https://arxiv.org/html/2605.14125#bib.bib4); Park et al\.,[2025a](https://arxiv.org/html/2605.14125#bib.bib30)\)and binding\(Feng & Steinhardt,[2024](https://arxiv.org/html/2605.14125#bib.bib11); Dai et al\.,[2024](https://arxiv.org/html/2605.14125#bib.bib7)\), providing a geometric principle for how compositional representations may be structured in neural activations\.
##### The limit of Euclidean graphs\.
As noted inSimon et al\. \([2025](https://arxiv.org/html/2605.14125#bib.bib32)\), the Polar Probe relies on linear distances, and is therefore tailored to capture*Euclidean graphs*, i\.e\., structures whose nodes and edges can be faithfully embedded in a flat vector space and where distances satisfy Euclidean geometry\. As a result, our method cannot accurately represent entire non\-commutative, many\-to\-many relations and shortest\-path distances \(e\.g\. family trees and metro networks\)\.
##### Larger impact
Overall, this study clarifies how symbolic structures can be represented within vectorial systems, thereby refining the solution to the long\-standing — and sometimes overstated — tension between symbolic and connectionist approaches to AI\(Fodor & Pylyshyn,[1988](https://arxiv.org/html/2605.14125#bib.bib12); Smolensky,[1991](https://arxiv.org/html/2605.14125#bib.bib33); Marcus,[2003](https://arxiv.org/html/2605.14125#bib.bib25); Gayler,[2004](https://arxiv.org/html/2605.14125#bib.bib13)\)\.
## References
- Alain & Bengio \(2017\)Guillaume Alain and Yoshua Bengio\.Understanding intermediate layers using linear classifier probes, 2017\.URL[https://openreview\.net/forum?id=ryF7rTqgl](https://openreview.net/forum?id=ryF7rTqgl)\.
- Biderman et al\. \(2023\)Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar Van Der Wal\.Pythia: a suite for analyzing large language models across training and scaling\.In*Proceedings of the 40th International Conference on Machine Learning*, ICML’23\. JMLR\.org, 2023\.
- Blondel et al\. \(2020\)Mathieu Blondel, Olivier Teboul, Quentin Berthet, and Josip Djolonga\.Fast differentiable sorting and ranking\.In*Proceedings of the 37th International Conference on Machine Learning*, ICML’20\. JMLR\.org, 2020\.
- Brown et al\. \(2020\)Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert\-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei\.Language models are few\-shot learners\.In H\. Larochelle, M\. Ranzato, R\. Hadsell, M\.F\. Balcan, and H\. Lin \(eds\.\),*Advances in Neural Information Processing Systems*, volume 33, pp\. 1877–1901\. Curran Associates, Inc\., 2020\.URL[https://proceedings\.neurips\.cc/paper\_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a\-Paper\.pdf](https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf)\.
- Chen et al\. \(2023\)Yida Chen, Yixian Gan, Sijia Li, Li Yao, and Xiaohan Zhao\.More than correlation: Do large language models learn causal representations of space?*arXiv preprint arXiv:2312\.16257*, 2023\.
- Conneau et al\. \(2018\)Alexis Conneau, German Kruszewski, Guillaume Lample, Loïc Barrault, and Marco Baroni\.What you can cram into a single $&\!\#\* vector: Probing sentence embeddings for linguistic properties\.In Iryna Gurevych and Yusuke Miyao \(eds\.\),*Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pp\. 2126–2136, Melbourne, Australia, July 2018\. Association for Computational Linguistics\.doi:10\.18653/v1/P18\-1198\.URL[https://aclanthology\.org/P18\-1198/](https://aclanthology.org/P18-1198/)\.
- Dai et al\. \(2024\)Qin Dai, Benjamin Heinzerling, and Kentaro Inui\.Representational analysis of binding in language models\.In Yaser Al\-Onaizan, Mohit Bansal, and Yun\-Nung Chen \(eds\.\),*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, pp\. 17468–17493, Miami, Florida, USA, November 2024\. Association for Computational Linguistics\.doi:10\.18653/v1/2024\.emnlp\-main\.967\.URL[https://aclanthology\.org/2024\.emnlp\-main\.967/](https://aclanthology.org/2024.emnlp-main.967/)\.
- Diego\-Simon et al\. \(2024\)Pablo J\. Diego\-Simon, Stéphane d’Ascoli, Emmanuel Chemla, Yair Lakretz, and Jean\-Remi King\.A polar coordinate system represents syntax in large language models\.In*The Thirty\-eighth Annual Conference on Neural Information Processing Systems*, 2024\.URL[https://openreview\.net/forum?id=x2780VcMOI](https://openreview.net/forum?id=x2780VcMOI)\.
- Eisape et al\. \(2022\)Tiwalayo Eisape, Vineet Gangireddy, Roger Levy, and Yoon Kim\.Probing for incremental parse states in autoregressive language models\.In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang \(eds\.\),*Findings of the Association for Computational Linguistics: EMNLP 2022*, pp\. 2801–2813, Abu Dhabi, United Arab Emirates, December 2022\. Association for Computational Linguistics\.doi:10\.18653/v1/2022\.findings\-emnlp\.203\.URL[https://aclanthology\.org/2022\.findings\-emnlp\.203/](https://aclanthology.org/2022.findings-emnlp.203/)\.
- Elhage et al\. \(2022\)Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield\-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah\.Toy models of superposition\.*Transformer Circuits Thread*, 2022\.
- Feng & Steinhardt \(2024\)Jiahai Feng and Jacob Steinhardt\.How do language models bind entities in context?In*The Twelfth International Conference on Learning Representations*, 2024\.URL[https://openreview\.net/forum?id=zb3b6oKO77](https://openreview.net/forum?id=zb3b6oKO77)\.
- Fodor & Pylyshyn \(1988\)Jerry A\. Fodor and Zenon W\. Pylyshyn\.Connectionism and cognitive architecture: A critical analysis\.*Cognition*, 28\(1\):3–71, 1988\.ISSN 0010\-0277\.doi:https://doi\.org/10\.1016/0010\-0277\(88\)90031\-5\.URL[https://www\.sciencedirect\.com/science/article/pii/0010027788900315](https://www.sciencedirect.com/science/article/pii/0010027788900315)\.
- Gayler \(2004\)Ross W Gayler\.Vector symbolic architectures answer jackendoff’s challenges for cognitive neuroscience\.*arXiv preprint cs/0412059*, 2004\.
- Georgopoulos et al\. \(1986\)Apostolos P\. Georgopoulos, Andrew B\. Schwartz, and Ronald E\. Kettner\.Neuronal population coding of movement direction\.*Science*, 233\(4771\):1416–1419, September 1986\.ISSN 1095\-9203\.doi:10\.1126/science\.3749885\.URL[http://dx\.doi\.org/10\.1126/science\.3749885](http://dx.doi.org/10.1126/science.3749885)\.
- Grattafiori et al\. \(2024\)Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al\-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia\-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind Thattai, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El\-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala\-Yeary, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao, Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do, Vish Vogeti, Vítor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang, Changkyu Kim, Chao Zhou, Chester Hu, Ching\-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Duc Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Eric\-Tuan Le, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel, Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Ilias Leontiadis, Irina\-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher, Jean\-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Junjie Wang, Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L\. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Rodriguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto, Sharadh Ramaswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar, Shuqiang Zhang, Shuqiang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vlad Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang, Xiaojian Wu, Xiaolan Wang, Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary DeVito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, Zhiwei Zhao, and Zhiyu Ma\.The llama 3 herd of models, 2024\.URL[https://arxiv\.org/abs/2407\.21783](https://arxiv.org/abs/2407.21783)\.
- Groeneveld et al\. \(2024\)Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E\. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah A\. Smith, and Hannaneh Hajishirzi\.Olmo: Accelerating the science of language models\.*Preprint*, 2024\.
- Gurnee & Tegmark \(2024\)Wes Gurnee and Max Tegmark\.Language models represent space and time\.In*The Twelfth International Conference on Learning Representations*, 2024\.URL[https://openreview\.net/forum?id=jE8xbmvFin](https://openreview.net/forum?id=jE8xbmvFin)\.
- Hewitt & Manning \(2019\)John Hewitt and Christopher D\. Manning\.A structural probe for finding syntax in word representations\.In Jill Burstein, Christy Doran, and Thamar Solorio \(eds\.\),*Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\)*, pp\. 4129–4138, Minneapolis, Minnesota, June 2019\. Association for Computational Linguistics\.doi:10\.18653/v1/N19\-1419\.URL[https://aclanthology\.org/N19\-1419/](https://aclanthology.org/N19-1419/)\.
- Humphrey et al\. \(1970\)Donald R\. Humphrey, E\. M\. Schmidt, and W\. D\. Thompson\.Predicting measures of motor performance from multiple cortical spike trains\.*Science*, 170\(3959\):758–762, November 1970\.ISSN 1095\-9203\.doi:10\.1126/science\.170\.3959\.758\.URL[http://dx\.doi\.org/10\.1126/science\.170\.3959\.758](http://dx.doi.org/10.1126/science.170.3959.758)\.
- Jawahar et al\. \(2019\)Ganesh Jawahar, Benoît Sagot, and Djamé Seddah\.What does BERT learn about the structure of language?In Anna Korhonen, David Traum, and Lluís Màrquez \(eds\.\),*Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics*, pp\. 3651–3657, Florence, Italy, July 2019\. Association for Computational Linguistics\.doi:10\.18653/v1/P19\-1356\.URL[https://aclanthology\.org/P19\-1356/](https://aclanthology.org/P19-1356/)\.
- Kim et al\. \(2025\)Junsol Kim, James Evans, and Aaron Schein\.Linear representations of political perspective emerge in large language models\.In*The Thirteenth International Conference on Learning Representations*, 2025\.URL[https://openreview\.net/forum?id=rwqShzb9li](https://openreview.net/forum?id=rwqShzb9li)\.
- Levy & Geva \(2025\)Amit Arnold Levy and Mor Geva\.Language models encode numbers using digit representations in base 10\.In Luis Chiruzzo, Alan Ritter, and Lu Wang \(eds\.\),*Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 2: Short Papers\)*, pp\. 385–395, Albuquerque, New Mexico, April 2025\. Association for Computational Linguistics\.ISBN 979\-8\-89176\-190\-2\.doi:10\.18653/v1/2025\.naacl\-short\.33\.URL[https://aclanthology\.org/2025\.naacl\-short\.33/](https://aclanthology.org/2025.naacl-short.33/)\.
- Limisiewicz & Mareček \(2021\)Tomasz Limisiewicz and David Mareček\.Introducing orthogonal constraint in structural probes\.In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli \(eds\.\),*Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\)*, pp\. 428–442, Online, August 2021\. Association for Computational Linguistics\.doi:10\.18653/v1/2021\.acl\-long\.36\.URL[https://aclanthology\.org/2021\.acl\-long\.36/](https://aclanthology.org/2021.acl-long.36/)\.
- Liu et al\. \(2019\)Nelson F\. Liu, Matt Gardner, Yonatan Belinkov, Matthew E\. Peters, and Noah A\. Smith\.Linguistic knowledge and transferability of contextual representations\.In Jill Burstein, Christy Doran, and Thamar Solorio \(eds\.\),*Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\)*, pp\. 1073–1094, Minneapolis, Minnesota, June 2019\. Association for Computational Linguistics\.doi:10\.18653/v1/N19\-1112\.URL[https://aclanthology\.org/N19\-1112/](https://aclanthology.org/N19-1112/)\.
- Marcus \(2003\)Gary F Marcus\.*The algebraic mind: Integrating connectionism and cognitive science*\.MIT press, 2003\.
- Marks & Tegmark \(2024\)Samuel Marks and Max Tegmark\.The geometry of truth: Emergent linear structure in large language model representations of true/false datasets\.In*First Conference on Language Modeling*, 2024\.URL[https://openreview\.net/forum?id=aajyHYjjsk](https://openreview.net/forum?id=aajyHYjjsk)\.
- Mikolov et al\. \(2013\)Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean\.Efficient estimation of word representations in vector space\.*arXiv preprint arXiv:1301\.3781*, 2013\.
- Müller\-Eberstein et al\. \(2022\)Max Müller\-Eberstein, Rob van der Goot, and Barbara Plank\.Probing for labeled dependency trees\.In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio \(eds\.\),*Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pp\. 7711–7726, Dublin, Ireland, May 2022\. Association for Computational Linguistics\.doi:10\.18653/v1/2022\.acl\-long\.532\.URL[https://aclanthology\.org/2022\.acl\-long\.532/](https://aclanthology.org/2022.acl-long.532/)\.
- Nanda et al\. \(2023\)Neel Nanda, Andrew Lee, and Martin Wattenberg\.Emergent linear representations in world models of self\-supervised sequence models\.In Yonatan Belinkov, Sophie Hao, Jaap Jumelet, Najoung Kim, Arya McCarthy, and Hosein Mohebbi \(eds\.\),*Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP*, pp\. 16–30, Singapore, December 2023\. Association for Computational Linguistics\.doi:10\.18653/v1/2023\.blackboxnlp\-1\.2\.URL[https://aclanthology\.org/2023\.blackboxnlp\-1\.2/](https://aclanthology.org/2023.blackboxnlp-1.2/)\.
- Park et al\. \(2025a\)Core Francisco Park, Andrew Lee, Ekdeep Singh Lubana, Yongyi Yang, Maya Okawa, Kento Nishi, Martin Wattenberg, and Hidenori Tanaka\.ICLR: In\-context learning of representations\.In*The Thirteenth International Conference on Learning Representations*, 2025a\.URL[https://openreview\.net/forum?id=pXlmOmlHJZ](https://openreview.net/forum?id=pXlmOmlHJZ)\.
- Park et al\. \(2025b\)Kiho Park, Yo Joong Choe, Yibo Jiang, and Victor Veitch\.The geometry of categorical and hierarchical concepts in large language models\.In*The Thirteenth International Conference on Learning Representations*, 2025b\.URL[https://openreview\.net/forum?id=bVTM2QKYuA](https://openreview.net/forum?id=bVTM2QKYuA)\.
- Simon et al\. \(2025\)Pablo J\. Diego Simon, Emmanuel Chemla, Jean\-Remi King, and Yair Lakretz\.Probing syntax in large language models: Successes and remaining challenges\.In*Second Conference on Language Modeling*, 2025\.URL[https://openreview\.net/forum?id=nrZysNmJ0n](https://openreview.net/forum?id=nrZysNmJ0n)\.
- Smolensky \(1991\)Paul Smolensky\.*Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems*, pp\. 159–216\.The MIT Press, October 1991\.ISBN 9780262256360\.doi:10\.7551/mitpress/2102\.003\.0006\.URL[http://dx\.doi\.org/10\.7551/mitpress/2102\.003\.0006](http://dx.doi.org/10.7551/mitpress/2102.003.0006)\.
- Tenney et al\. \(2019\)Ian Tenney, Dipanjan Das, and Ellie Pavlick\.BERT rediscovers the classical NLP pipeline\.In Anna Korhonen, David Traum, and Lluís Màrquez \(eds\.\),*Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics*, pp\. 4593–4601, Florence, Italy, July 2019\. Association for Computational Linguistics\.doi:10\.18653/v1/P19\-1452\.URL[https://aclanthology\.org/P19\-1452/](https://aclanthology.org/P19-1452/)\.
- Turner et al\. \(2025\)Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid\.Steering language models with activation engineering, 2025\.URL[https://openreview\.net/forum?id=2XBPdPIcFK](https://openreview.net/forum?id=2XBPdPIcFK)\.
- Vaswani et al\. \(2017\)Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin\.Attention is all you need\.In I\. Guyon, U\. Von Luxburg, S\. Bengio, H\. Wallach, R\. Fergus, S\. Vishwanathan, and R\. Garnett \(eds\.\),*Advances in Neural Information Processing Systems*, volume 30\. Curran Associates, Inc\., 2017\.URL[https://proceedings\.neurips\.cc/paper\_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa\-Paper\.pdf](https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf)\.
- Åke Björck & Golub \(1973\)Åke Björck and Gene H\. Golub\.Numerical methods for computing angles between linear subspaces\.*Mathematics of Computation*, 27\(123\):579–594, 1973\.ISSN 00255718, 10886842\.URL[http://www\.jstor\.org/stable/2005662](http://www.jstor.org/stable/2005662)\.
## 5Appendix
### 5\.1Subspace alignment score\.
LetBi∈ℝki×dB\_\{i\}\\\!\\in\\\!\\mathbb\{R\}^\{k\_\{i\}\\times d\}andPi∈ℝki×𝒯iP\_\{i\}\\\!\\in\\\!\\mathbb\{R\}^\{k\_\{i\}\\times\\mathcal\{T\}\_\{i\}\}be a trained polar probe and its prototype matrix \(columns are prototype vectors\)\. We project prototypes to model space:
Vi=Bi†Pi∈ℝd×𝒯i,V\_\{i\}\\;=\\;B\_\{i\}^\{\\dagger\}P\_\{i\}\\in\\mathbb\{R\}^\{d\\times\\mathcal\{T\}\_\{i\}\},and take the reduced QR factorizationVi=QiRiV\_\{i\}=Q\_\{i\}R\_\{i\}withQi⊤Qi=I𝒯iQ\_\{i\}^\{\\top\}Q\_\{i\}=I\_\{\\mathcal\{T\}\_\{i\}\},Qi∈ℝd×𝒯iQ\_\{i\}\\in\\mathbb\{R\}^\{d\\times\\mathcal\{T\}\_\{i\}\}\.
We report the mean squared cosines of the principal angles between two subspaces\(Åke Björck & Golub,[1973](https://arxiv.org/html/2605.14125#bib.bib37)\):
Alignment\(Bi,Bj\)=1N‖Qi⊤Qj‖F2\.\\mathrm\{Alignment\}\(B\_\{i\},B\_\{j\}\)=\\frac\{1\}\{N\}\\,\\\|Q\_\{i\}^\{\\top\}Q\_\{j\}\\\|\_\{F\}^\{2\}\.
where\{θn\}\\\{\\theta\_\{n\}\\\}are the principal angles andN=min\(𝒯1,𝒯2\)N=\\min\(\\mathcal\{T\}\_\{1\},\\mathcal\{T\}\_\{2\}\)\. This score lies in\[0,1\]\[0,1\], equals0for orthogonal subspaces, and increases with alignment\.
Figure 7:Semantic domain subspaces are largely disjoint, with a spatial–ordinal overlap\.Cross\-domain alignment at the best layer of Llama3\-8B, quantified via the principal angles in LLM space \(higher = more overlap\)\.
### 5\.2Correlation with downstream predictions
To determine whether polar representations are merely epiphenomenal or instead reflect representations used by the LLM, we conduct a representation–behavior analysis\. Specifically, we compare the reliability of these representations \(Polar Probe performance\) to LLMs’ ability to effectively answer questions about the semantic structure\. We find that higher type errors in probe\-space correlate with the worse Llama3\-8B downstream performance \([Figure8](https://arxiv.org/html/2605.14125#S5.F8)\)\. No analogous correlation is observed for existence errors\.
Figure 8:Type probe errors predict LLM’s downstream performance\.Layerwise Spearman correlation between existence \(blue\) and type \(orange\) probe\-space errors and logit of the correct answer on a Question\-Answering task over semantic structures\.
### 5\.3Additional baselines
Figure 9:Semantic structures are encoded in linear subspaces of LLM activations\.Training prototype vectors with a fixed identity probe yields Existence scores close to chance\. Type scores are high in the spatial layout and family tree domains but remain near chance elsewhere\. Finally, the linear baseline, shown as a horizontal line, performs very close to chance across all domains\.
### 5\.4Naturalistic evaluation
We extend the evaluation to more naturalistic settings by synthesizing a dataset from the spatial layouts domain, in which relation surface forms vary\. We further synthesize a multilingual version comprising Italian, Spanish, French, and English\.
Figure 10:Polar probes trained on the Spatial Layout controlled dataset generalize to a naturalistic and multilingual sentences\.Polar probes are trained on the controlled Spatial Layout dataset and evaluated on an LLM\-generated naturalistic dataset within the same semantic domain\. Probe performance substantially exceeds both chance level and the untrained baseline\.
### 5\.5Causal interventions
Figure 11:Interventions along polar probe directions causally modulate model predictions, with the strongest effects in middle layers\.\. Probability of the correct token under positive and negative direction steering\. In middle layers, positive\-prototype interventions reliably increase the probability and negative\-prototype interventions decrease it\.Figure 12:Middle layers show maximal response to causal interventions\.Layerwise mean difference in probability between positive\-signed and negative\-signed prototypes\.
### 5\.6Alignment with uncontextualized embeddings
Figure 13:Semantic domain subspaces and uncontextualized embeddings are disjoint\.Cross\-domain alignment at the best layer of Llama3\-8B, quantified via the principal angles in LLM space \(higher = more overlap\)
### 5\.7Predicted vs\.Ground\-truth distances
Figure 14:Semantic distance and Probe distance used to calculate Spearman’sρ\\rho
### 5\.8Dataset generation
Algorithm 1Grid\-based Euclidean graph sampler1:entities
nn, dimension
dd
2:
G=\(𝒱G,ℰG,𝒯G\)G=\(\\mathcal\{V\}\_\{G\},\\mathcal\{E\}\_\{G\},\\mathcal\{T\}\_\{G\}\)
3:
𝒱G=\{v1,…,vn\}\\mathcal\{V\}\_\{G\}=\\\{v\_\{1\},\\dots,v\_\{n\}\\\}
4:
𝒯G=\{t1,…,td\}\\mathcal\{T\}\_\{G\}=\\\{t\_\{1\},\\dots,t\_\{d\}\\\}
5:
𝖴d←\{𝐞1,…,𝐞d\}\\mathsf\{U\}\_\{d\}\\leftarrow\\\{\\mathbf\{e\}\_\{1\},\\dots,\\mathbf\{e\}\_\{d\}\\\}⊳\\trianglerightpositive unit steps
6:
𝗉𝗈𝗌𝗂𝗍𝗂𝗈𝗇𝗌←\{𝟎\}\\mathsf\{positions\}\\leftarrow\\\{\\mathbf\{0\}\\\}⊳\\trianglerightpositions inℤd\\mathbb\{Z\}^\{d\}
7:while
\|𝗉𝗈𝗌𝗂𝗍𝗂𝗈𝗇𝗌\|<n\|\\mathsf\{positions\}\|<ndo⊳\\trianglerightrandom signed unit\-step growth
8:pick
𝐱∈𝗉𝗈𝗌𝗂𝗍𝗂𝗈𝗇𝗌\\mathbf\{x\}\\in\\mathsf\{positions\}uniformly
9:pick
𝐮∈𝖴d\\mathbf\{u\}\\in\\mathsf\{U\}\_\{d\}uniformly
10:pick
s∈\{−1,\+1\}s\\in\\\{\-1,\+1\\\}uniformly
11:
𝗉𝗈𝗌𝗂𝗍𝗂𝗈𝗇𝗌←𝗉𝗈𝗌𝗂𝗍𝗂𝗈𝗇𝗌∪\{𝐱\+s𝐮\}\\mathsf\{positions\}\\leftarrow\\mathsf\{positions\}\\cup\\\{\\mathbf\{x\}\+s\\mathbf\{u\}\\\}⊳\\trianglerightreject if already occupied
12:endwhile
13:pick a bijection
f:𝗉𝗈𝗌𝗂𝗍𝗂𝗈𝗇𝗌→∼𝒱Gf:\\mathsf\{positions\}\\xrightarrow\{\\sim\}\\mathcal\{V\}\_\{G\}
14:pick a bijection
g:𝖴d→∼𝒯Gg:\\mathsf\{U\}\_\{d\}\\xrightarrow\{\\sim\}\\mathcal\{T\}\_\{G\}
15:
ℰG←∅\\mathcal\{E\}\_\{G\}\\leftarrow\\varnothing
16:for allordered pairs
\(vi,vj\)\(v\_\{i\},v\_\{j\}\)do
17:
𝚫←f−1\(vj\)−f−1\(vi\)\\boldsymbol\{\\Delta\}\\leftarrow f^\{\-1\}\(v\_\{j\}\)\-f^\{\-1\}\(v\_\{i\}\)
18:if\(
∥𝚫∥2=1\)∧\(𝚫∈𝖴d\\\|\\boldsymbol\{\\Delta\}\\\|\_\{2\}=1\)\\;\\land\\;\(\\boldsymbol\{\\Delta\}\\in\\mathsf\{U\}\_\{d\}\)then
19:
t←g−1\(𝚫\)∈𝒯Gt\\leftarrow g^\{\-1\}\(\\boldsymbol\{\\Delta\}\)\\in\\mathcal\{T\}\_\{G\}
20:
ℰG←ℰG∪\{\(vi,vj,t\)\}\\mathcal\{E\}\_\{G\}\\leftarrow\\mathcal\{E\}\_\{G\}\\cup\\\{\(v\_\{i\},v\_\{j\},t\)\\\}⊳\\trianglerightAdd to graph
21:endif
22:endfor
23:return
G=\(𝒱G,ℰG,𝒯G\)G=\(\\mathcal\{V\}\_\{G\},\\mathcal\{E\}\_\{G\},\\mathcal\{T\}\_\{G\}\)
### 5\.9Setup details
Table 1:Entity names, relations and prompt used for each semantic domainSimilar Articles
What are They Thinking? Delineation, Probing and Tracking of Concepts in LLMs
This paper presents a methodology for delineating concepts and training linear probes to detect them in LLM embeddings, using four example concepts across three models. The work aims to enable scalable monitoring of LLM internal representations.
Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States
This paper demonstrates that linear probes on LLM hidden states detect task format confounds (e.g., source identity, response length) rather than distinct reasoning modes, using residualization and causal steering to show that high probe accuracy is due to superficial features, not computational structure.
Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representations
This paper systematically tests linear probes for deception detection in large language models, finding they fail under distributional shifts but style-augmented probes recover performance, and revealing that deception is encoded through distributed sub-threshold features.
Tracing Relational Knowledge Recall in Large Language Models
Researchers trace how LLMs recall relational facts by probing per-head attention contributions, showing these are strong linear features whose fidelity correlates with relation specificity and entity connectedness.
Causal Probing for Internal Visual Representations in Multimodal Large Language Models
This paper proposes a causal framework for probing internal visual representations in Multimodal Large Language Models, revealing differences in how entities and abstract concepts are encoded. The study highlights that increasing model depth is crucial for encoding abstract concepts and uncovers a disconnect between perception and reasoning in current MLLMs.