Comparing Linear Probes with Mahalanobis Cosine Similarity
Summary
This paper extends empirical findings that the Mahalanobis cosine similarity (MCS) between linear probes linearly predicts out-of-distribution AUROC, and proves this relationship theoretically under Gaussian assumptions.
View Cached Full Text
Cached at: 06/23/26, 09:44 PM
Paper page - Comparing Linear Probes with Mahalanobis Cosine Similarity
Source: https://huggingface.co/papers/2606.19603
Abstract
The Mahalanobis cosine similarity provides a theoretically grounded method for comparing linear probes that correlates strongly with out-of-distribution performance metrics.
Linear probesare widely used in interpretability research and often compared by cosine similarity. TheMahalanobis cosine similarity(MCS) between two directions, which reweights the inner product bytest data covariance, is a natural task-aware refinement. Ying et al. (2026) report that a probe’s MCS to areference probetrained on the out-of-distribution (OOD) data near-perfectly linearly predicts the probe’s OOD AUROC (R^2 = 0.98). Here, we extend this empirical finding across models, layers, and concept domains, and prove this general phenomenon in closed form: For balanced classes whose projections are Gaussian, OOD AUROC and MCS to thereference probeare linear because both aresigmoid-shaped functionsof the probe’ssignal-to-noise ratio(SNR) on the test data. The theory also predicts when this linearity fails, which we verify empirically. MCS offers a theoretically grounded and empirically effective alternative to Euclidean cosine similarity for comparinglinear probes.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2606\.19603
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.19603 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.19603 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.19603 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Mean-Pooled Cosine Similarity is Not Length-Invariant: Theory and Cross-Domain Evidence for a Length-Invariant Alternative
This paper demonstrates that mean-pooled cosine similarity is not length-invariant under anisotropic representations, showing it artificially inflates similarity with sequence length. It argues for using Centered Kernel Alignment (CKA) as a default metric to correct biases in cross-lingual and cross-representation analysis.
Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representations
This paper systematically tests linear probes for deception detection in large language models, finding they fail under distributional shifts but style-augmented probes recover performance, and revealing that deception is encoded through distributed sub-threshold features.
Probing the Misaligned Thinking Process of Language Models
This paper proposes monitoring LLM misalignment by decomposing it into fine-grained cognitive processes (misalignment indicators) and detecting them via linear probes on internal activations, achieving high AUROC on out-of-distribution transcripts.
Polar probe linearly decodes semantic structures from LLMs
This paper proposes a Polar Probe that linearly recovers semantic structures from LLM activations by representing entity relations through distance and direction in a learned subspace. Testing across arithmetic, visual scenes, family trees, metro maps, and social interactions shows the code emerges in middle layers, generalizes to new entities, and causally influences model predictions.
Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States
This paper demonstrates that linear probes on LLM hidden states detect task format confounds (e.g., source identity, response length) rather than distinct reasoning modes, using residualization and causal steering to show that high probe accuracy is due to superficial features, not computational structure.