Knowing, and Saying It Only When Asked: LLM Endognostics and the Schizognosis of Minerva-7B

arXiv cs.CL Papers
large-language-models internal-representation evaluation-framework causal-intervention ai-safety white-box-auditing auditing-method

Summary

This paper introduces LLM endognostics, a white-box framework for auditing latent knowledge in language models, showing that behavioral evaluations can fail to reflect internal knowledge distinctions, as demonstrated on Minerva-7B-Instruct-v1.0.

arXiv:2609.22219v1 Announce Type: new Abstract: Evaluating an aligned language model by reading its answers assumes the answers carry the distinction the evaluator cares about. We introduce LLM endognostics, a white-box internal auditing framework designed to extract and causally manipulate latent knowledge within the residual stream. Applied to Minerva-7B-Instruct-v1.0 on 124 minimal prompt pairs over 12 categories of professional risk, behavioral evaluation fails on most of the set: the model acts identically on 63.7% of the pairs (95% CI [55.0%, 71.6%]), complying with or refusing both members. Yet, projecting the residual stream onto the vocabulary by a Jacobian lens reveals a statistically significant Contrastive Endognostic Margin, proving the model maintains robust risk differentiation internally. In a second protocol crossing 25 facts with five linguistic framings, we show that the model conforms to presupposed falsehoods in 72% of cases, despite representing the true entity in its latent layers. Surgically ablating the direction of the planted falsehood restores the correct answer in 11 of 25 suppressed cases (McNemar p = 0.0010), validated by blind human annotation (binary agreement kappa = 0.68). In contrast, an out-of-sample linear probe achieves 77% accuracy at layer 10, but its orthogonal ablation yields a 0% recovery rate. This establishes a fundamental theoretical dissociation: abstract linear representation does not imply causal control over verbalization. Our main contribution is the formalization of endognostics to prove that behavioral evaluation and internal reading systematically disagree in the common case, and that linear decodability is decoupled from causal control.
Original Article
View Cached Full Text

Cached at: 09/22/26, 09:09 AM 

# Knowing, and Saying It Only When Asked: LLM Endognostics and the Schizognosis of Minerva-7B
Source: [https://arxiv.org/abs/2609.22219](https://arxiv.org/abs/2609.22219)
[View PDF](https://arxiv.org/pdf/2609.22219)

> Abstract:Evaluating an aligned language model by reading its answers assumes the answers carry the distinction the evaluator cares about\. We introduce LLM endognostics, a white\-box internal auditing framework designed to extract and causally manipulate latent knowledge within the residual stream\. Applied to Minerva\-7B\-Instruct\-v1\.0 on 124 minimal prompt pairs over 12 categories of professional risk, behavioral evaluation fails on most of the set: the model acts identically on 63\.7% of the pairs \(95% CI \[55\.0%, 71\.6%\]\), complying with or refusing both members\. Yet, projecting the residual stream onto the vocabulary by a Jacobian lens reveals a statistically significant Contrastive Endognostic Margin, proving the model maintains robust risk differentiation internally\. In a second protocol crossing 25 facts with five linguistic framings, we show that the model conforms to presupposed falsehoods in 72% of cases, despite representing the true entity in its latent layers\. Surgically ablating the direction of the planted falsehood restores the correct answer in 11 of 25 suppressed cases \(McNemar p = 0\.0010\), validated by blind human annotation \(binary agreement kappa = 0\.68\)\. In contrast, an out\-of\-sample linear probe achieves 77% accuracy at layer 10, but its orthogonal ablation yields a 0% recovery rate\. This establishes a fundamental theoretical dissociation: abstract linear representation does not imply causal control over verbalization\. Our main contribution is the formalization of endognostics to prove that behavioral evaluation and internal reading systematically disagree in the common case, and that linear decodability is decoupled from causal control\.

## Submission history

From: Fabrizio Davide Dr \[[view email](https://arxiv.org/show-email/dfed2bb3/2609.22219)\] **\[v1\]**Wed, 2 Sep 2026 09:38:04 UTC \(690 KB\)

Similar Articles

Do LLMs Really Know What They Don't Know? Internal States Mainly Reflect Knowledge Recall Rather Than Truthfulness

arXiv cs.CL

This paper challenges the assumption that LLMs can reliably distinguish between hallucinated and factual outputs through internal signals, arguing that internal states primarily reflect knowledge recall rather than truthfulness. The authors propose a taxonomy of hallucinations (associated vs. unassociated) and show that associated hallucinations exhibit hidden-state geometries overlapping with factual outputs, making standard detection methods ineffective.