Do All LLMs Know When They're Being Harmful? A Reproducibility Study of Latent-Space Safety Probes Across Model Families

# Do All LLMs Know When They’re Being Harmful? A Reproducibility Study of Latent-Space Safety Probes Across Model Families
Source: [https://arxiv.org/html/2608.08029](https://arxiv.org/html/2608.08029)
###### Abstract

Khatriet al\.\([2026](https://arxiv.org/html/2608.08029#bib.bib1)\)show that lightweight MLP probes on final\-layer activations of a single 8B model \(LLaMA\-3\.1\-8B\) detect harmful prompts at F1 competitive with guard models 1000×\\timeslarger, using one probe per benchmark\. We reproduce this pipeline end\-to\-end and extend it along two axes the original study leaves open\. First, we test whether the result generalizes across other model*architecture and scale*by training identical probes on activations from models like Gemma\-4\-E4B, Mistral\-7B\-v0\.3, and Qwen2\-7B, using the three benchmarks \(WildJailbreak, BeaverTails, AEGIS 2\.0\)\. Second, we test how much of the reported performance is affected by*non\-determinism*during inference by repeating extraction under five random seeds and measuring the variance of F1 scores\. Our results reproduce the original LLaMA model benchmarks within 0\.37 percentage points of the original F1 scores \(and within 0\.2 points on BeaverTails\)\. We find that the original MLP probe architecture extends to other model families with F1 scores within a point of the values reported for LLaMA\-3\.1\-8B\. Our experiments varying seed values reveals an interesting observation \- final token latent vectors remained the same for all tested architectures irrespective of the seed values used\.

Do All LLMs Know When They’re Being Harmful? A Reproducibility Study of Latent\-Space Safety Probes Across Model Families

Alizishaan KhatriWrynx Incresearch@wrynx\.comDun Li ChanINTI International College Penang

## 1Introduction

Runtime safety moderation for large language models \(LLMs\) is dominated by*external guardrails*: separate classifier models that screen prompts and responses\(Inanet al\.,[2023](https://arxiv.org/html/2608.08029#bib.bib5); Zenget al\.,[2024](https://arxiv.org/html/2608.08029#bib.bib6); Hanet al\.,[2024](https://arxiv.org/html/2608.08029#bib.bib7)\)\.Khatriet al\.\([2026](https://arxiv.org/html/2608.08029#bib.bib1)\)argue that this architecture is unnecessarily expensive, since safety\-relevant information is already linearly decodable from an LLM’s own hidden states\(Zouet al\.,[2023](https://arxiv.org/html/2608.08029#bib.bib3); Saglamet al\.,[2025](https://arxiv.org/html/2608.08029#bib.bib2); Chenet al\.,[2025](https://arxiv.org/html/2608.08029#bib.bib4)\)\. They support this claim with a single model family \(LLaMA\-3\.1\-8B\) and a single extraction configuration \(final layer, last\-token after prefill\), reporting F1 scores of 99\.1%, 82\.7%, and 83\.5% on WildJailbreak, BeaverTails, and AEGIS 2\.0 respectively\(Jianget al\.,[2024](https://arxiv.org/html/2608.08029#bib.bib9); Jiet al\.,[2023](https://arxiv.org/html/2608.08029#bib.bib8); Ghoshet al\.,[2025](https://arxiv.org/html/2608.08029#bib.bib10)\)\.

This leaves two questions the original paper does not address, and which are central to whether latent\-space probing is a viable general\-purpose alternative to external guardrails:

1. 1\.Generality across architectures\.Does the finding that “safety lives in a linearly separable subspace” hold for models trained with different architectures, pretraining data, alignment recipes, and hidden dimensionality, or is it an artifact ofLLaMA\-3\.1\-8B’s specific alignment procedure?
2. 2\.Sensitivity to non\-determinism\.The reported metrics come from a single extraction\-and\-training run per dataset\. Activation extraction involves stochastic components \(sampling order, weight initialization, non\-deterministic GPU kernels, etc\)\. In a practical setting, one probe would be deployed one model per category\. However, a user must be able to use different seed values for the same model\. How sensitive are the probe and guard\-model baselines in Table 3 ofKhatriet al\.\([2026](https://arxiv.org/html/2608.08029#bib.bib1)\)to non\-determinism of latent states during inference?

We address both questions directly\. We reimplement the two\-stage pipeline ofKhatriet al\.\([2026](https://arxiv.org/html/2608.08029#bib.bib1)\)\(activation extraction, then MLP probe training\) and \(1\) apply it unchanged to additional open\-weight model families at comparable parameter counts, and \(2\) repeat extraction under multiple seeds for each model/ data set pair to report probe variance along with point estimates\. This positions our contribution squarely within The 9th BlackboxNLP Workshop Special Track: Reproducibility and Reliability in Interpretability Analyses: we apply the original method without modification to previously untested models and add the statistical controls \(multi\-seed variance\) the original study omits\.

#### Contributions\.

- •A faithful reproduction of theKhatriet al\.\([2026](https://arxiv.org/html/2608.08029#bib.bib1)\)probe pipeline, validated against the original paper’s reported numbers onLLaMA\-3\.1\-8B\.
- •Empirical benchmarks for three additional model families, testing whether probe performance and probe\-vs\-guard\-model competitiveness hold outside the original model family\.
- •A variance analysis across five independent extraction runs per model/dataset pair, reporting standard deviation of F1 score, and quantifying what fraction of the original paper’s reported margin over baseline guard models survives this noise\.
- •Released code, activations, and per\-run results\.

## 2Related Work

#### External guardrails\.

Production LLM systems commonly rely on dedicated guard models such as Llama Guard\(Inanet al\.,[2023](https://arxiv.org/html/2608.08029#bib.bib5)\), ShieldGemma\(Zenget al\.,[2024](https://arxiv.org/html/2608.08029#bib.bib6)\), Latent Guard\(Zhaoet al\.,[2025](https://arxiv.org/html/2608.08029#bib.bib26)\), and WildGuard\(Hanet al\.,[2024](https://arxiv.org/html/2608.08029#bib.bib7)\)to screen prompts and responses\(Donget al\.,[2024](https://arxiv.org/html/2608.08029#bib.bib19)\)\. These add inference\-time latency and remain vulnerable to attacks that exploit the gap between what the guard sees and what happens inside the primary model\(Hunget al\.,[2025](https://arxiv.org/html/2608.08029#bib.bib20)\)\.

#### Latent safety representations\.

A growing line of work argues that safety\-relevant concepts are linearly represented in hidden LLM states\(Zouet al\.,[2023](https://arxiv.org/html/2608.08029#bib.bib3); Chenet al\.,[2025](https://arxiv.org/html/2608.08029#bib.bib4); Chiaet al\.,[2025](https://arxiv.org/html/2608.08029#bib.bib21)\), and that lightweight probes on these representations can match or approach external classifiers at a fraction of the parameter count\(Saglamet al\.,[2025](https://arxiv.org/html/2608.08029#bib.bib2); Cunninghamet al\.,[2026](https://arxiv.org/html/2608.08029#bib.bib11); Zhanget al\.,[2024](https://arxiv.org/html/2608.08029#bib.bib14)\)\. Closely related,Zhaoet al\.\([2025](https://arxiv.org/html/2608.08029#bib.bib26)\)show that harmfulness and refusal occupy separate directions in latent space, and that only the harmfulness direction, not the refusal direction, governs the model’s judgment of harm\. This distinction is directly relevant here, since the probes we study target harmfulness rather than refusal\.Khatriet al\.\([2026](https://arxiv.org/html/2608.08029#bib.bib1)\), the paper we reproduce and extend, is the most direct precedent: a 13\.9M\-parameter MLP probe onLLaMA\-3\.1\-8Bfinal\-layer activations, evaluated on WildJailbreak, BeaverTails, and AEGIS 2\.0\. Concurrent work extends the cross\-architecture question we ask in Section[3\.2](https://arxiv.org/html/2608.08029#S3.SS2):Llorente\-Saguer \([2026](https://arxiv.org/html/2608.08029#bib.bib27)\)show that harmful intent is linearly separable from residual\-stream activations across 12 models spanning four architectural families and three alignment variants \(base, instruction\-tuned, and abliterated\), including cases where the refusal mechanism has been surgically removed, evidence that harm recognition and refusal generation are separable mechanisms at the representation level, consistent withZhaoet al\.\([2025](https://arxiv.org/html/2608.08029#bib.bib26)\)above\. Our contribution focuses on cross\-architecture generalization and run\-to\-run variance rather than head\-to\-head latency comparisons\.

#### Reproducibility and robustness in interpretability\.

Our extensions follow the spirit of recent calls for stress\-testing interpretability claims across models and against random/non\-deterministic baselines rather than reporting single\-run point estimates on a single model\(Albrethsenet al\.,[2026](https://arxiv.org/html/2608.08029#bib.bib22)\)\. We are not aware of prior work that specifically tests cross\-architecture generalization or extraction non\-determinism for latent safety probes\.

## 3Methodology

We reuse the two\-stage pipeline ofKhatriet al\.\([2026](https://arxiv.org/html/2608.08029#bib.bib1)\)unchanged \(Figure[1](https://arxiv.org/html/2608.08029#S3.F1)\) and vary only \(a\) the backbone LLM from which activations are extracted, and \(b\) the random seed governing extraction order and batch sampling\.

### 3\.1Original pipeline \(reproduced\)

Stage 1: Activation extraction\.Each promptxix\_\{i\}is passed through a frozen backbone LLMℳ\\mathcal\{M\}, and the final\-layer hidden state at the last token after prompt prefill position is stored:hi=ℳ​\(xi\)​\[−1\]∈ℝdh\_\{i\}=\\mathcal\{M\}\(x\_\{i\}\)\[\-1\]\\in\\mathbb\{R\}^\{d\}, whereddis the backbone’s hidden dimension \(4096 forLLaMA\-3\.1\-8B\)\. Extraction is implemented withnnsight\(Fiotto\-Kaufmanet al\.,[2024](https://arxiv.org/html/2608.08029#bib.bib29)\), using an extraction batch size of 5 \(we also tested a batch size of 100 and found identical activations\), a maximum sequence length of 512 tokens, and extraction seed 42 for the base run\.

Stage 2: Probe training\.A 6\-layer MLPfθ:ℝd→\{0,1\}f\_\{\\theta\}:\\mathbb\{R\}^\{d\}\\rightarrow\\\{0,1\\\}\(Table[1](https://arxiv.org/html/2608.08029#S3.T1)\) is trained with cross\-entropy loss via AdamW\(Loshchilov and Hutter,[2019](https://arxiv.org/html/2608.08029#bib.bib13)\), learning rate2\.5×10−42\.5\\times 10^\{\-4\}, weight decay1×10−21\\times 10^\{\-2\}, dropout0\.10\.1, and aReduceLROnPlateaulearning\-rate schedule \(factor 0\.5, patience 2\), batch size 1024, for 50 epochs, to reproduce the original hyperparameters exactly\.

Table 1:MLP probe architecture, reproduced fromKhatriet al\.\([2026](https://arxiv.org/html/2608.08029#bib.bib1)\)\.ddis the backbone hidden size and varies by model \(Table[2](https://arxiv.org/html/2608.08029#S3.T2)\)\.![Refer to caption](https://arxiv.org/html/2608.08029v1/extraction_pipeline.png)Figure 1:Reproduced extraction\-and\-probing pipeline\. Only the backboneℳ\\mathcal\{M\}and the random seed vary across our experiments\.
### 3\.2Extension 1: Cross\-architecture generalization

We apply the pipeline of Section[3\.1](https://arxiv.org/html/2608.08029#S3.SS1)without modification to a set of additional open\-weight backbone LLMs spanning different model families, pretraining corpora, alignment procedures, and parameter counts \(Table[2](https://arxiv.org/html/2608.08029#S3.T2)\)\. For each backbone we extract final\-layer, last\-token activations on the same three datasets and splits asKhatriet al\.\([2026](https://arxiv.org/html/2608.08029#bib.bib1)\), and train an MLP probe with the architecture in Table[1](https://arxiv.org/html/2608.08029#S3.T1), adjusting only the input dimensionddto match the backbone’s hidden size\. All other hyperparameters \(learning rate, batch size, epochs, optimizer\) are held fixed to isolate the effect of the backbone\.

Table 2:Backbones used in the cross\-architecture extension\. First row is the original paper’s setting, included for calibration\.
### 3\.3Extension 2: Sensitivity to non\-determinism

For select \(model, dataset\) pair we repeat extractionK=5K\{=\}5times \(seeds 42, 75, 456, 789, 1024\), varying:

- •the order in which prompts are batched during extraction \(relevant when the inference backend uses non\-deterministic batched attention kernels\)

Dataset splits are held fixed across seeds so that variance reflects extraction/training stochasticity rather than data partitioning\. Probes are trained on the train set extracted with seed=42\. These probes are then evaluated on test seeds with all 5 seed values\. We report standard deviation \(computed via non\-parametric bootstrap over theKKruns\) for F1 scores\.

As a complementary diagnostic, we additionally treat the extracted activation tensors themselves as the unit of comparison, independent of any downstream metric\. For a few select \(model, dataset, seed\) triples we compute a SHA\-256 digest over the serialized activation tensor file produced by Stage 1, and compare digests across seeds within the same \(model, dataset\) pair\. This lets us distinguish two qualitatively different sources of “no variance”: \(i\) Stage 1 produces different activations across seeds, but Stage 2 happens to converge to indistinguishable probes regardless; or \(ii\) Stage 1 itself is seed\-invariant, so Stage 2 never sees different inputs in the first place\. Section[5\.4](https://arxiv.org/html/2608.08029#S5.SS4)reports the result of this check and discusses its implications\.

## 4Experimental Setup

#### Datasets\.

We evaluate on three datasets: WildJailbreak \(vanilla subset\), BeaverTails \(QA pairs with binaryis\_safelabels\), and AEGIS 2\.0\. For WildJailbreak and AEGIS 2\.0 we use an 80/10/10 train/validation/test split; for BeaverTails we adopt the standard split provided with the dataset\. Table[3](https://arxiv.org/html/2608.08029#S4.T3)summarises the datasets\.

Table 3:Datasets used in our experiments with train/validation/test splits, total size, label column, and label values\.
#### Compute and reproducibility artifacts\.

#### Evaluation\.

We report F1 scores on the held\-out test split of each dataset, matching the original paper’s protocol\.

## 5Results

### 5\.1Reproduction of the original result

Table[4](https://arxiv.org/html/2608.08029#S5.T4)compares our reproducedLLaMA\-3\.1\-8Bnumbers against those reported byKhatriet al\.\([2026](https://arxiv.org/html/2608.08029#bib.bib1)\)\. We find that we are able to reproduce results within a delta of 0\.37% F1 of the values reported in the original work\.

Table 4:Reproduction ofKhatriet al\.\([2026](https://arxiv.org/html/2608.08029#bib.bib1)\), Table 2, using identical hyperparameters and splits\.
### 5\.2Cross\-architecture results

Table[5](https://arxiv.org/html/2608.08029#S5.T5)reports probe F1 score for each backbone in Table[2](https://arxiv.org/html/2608.08029#S3.T2)on each dataset\. We find that the probes’ performance on the newly introduced model architectures is comparable to their performance to the values reported in the original work\. We observe an increase of about 0\.4 percentage points in F1 on the BeaverTails dataset for the new models compared to the numbers on LLaMA\-3\.1\-8B\.

Table 5:F1 score across backbones and datasets\. First row reproduces the original paper’s setting\.
### 5\.3Effect of non\-determinism

Table[6](https://arxiv.org/html/2608.08029#S5.T6)reports the full per\-seed breakdown for every \(model, dataset\) pair we evaluated underK=5K\{=\}5extraction seeds\. Across every pair, F1 is identical to at least two decimal places across all five seeds: standard deviation is0throughout, and every row of Table[6](https://arxiv.org/html/2608.08029#S5.T6)is constant across columns\. Precision and recall \(not shown in the main table; full values are in Appendix[B](https://arxiv.org/html/2608.08029#A2)\) show the same pattern\.

Table 6:Per\-seed test\-set F1 score for every \(model, dataset\) pair we evaluated underK=5K\{=\}5extraction seeds\. Columns S42–S1024 denote the five seed values\. Every row is constant across columns to the precision shown; see Appendix[B](https://arxiv.org/html/2608.08029#A2)for precision/recall\.\(Table[5](https://arxiv.org/html/2608.08029#S5.T5)\)\.
### 5\.4Latent state determinism: a hash\-matching diagnostic

Given the zero\-variance F1 pattern in Section[5\.3](https://arxiv.org/html/2608.08029#S5.SS3), we tested whether the extracted activation tensors from Stage 1 were themselves identical across seeds, rather than merely producing indistinguishable downstream probes\. For each \(model, dataset\) pair we computed a SHA\-256 digest of the serialized activation tensor file produced under each of the five extraction seeds and compared digests pairwise against the seed\-42 run\. Table[7](https://arxiv.org/html/2608.08029#S5.T7)reports the result: for all four model architectures, the digests are identical across all five seeds for every dataset we checked\. In other words, the latent state tensors extracted from LLaMA\-3\.1\-8B, Gemma\-4\-E4B, Mistral\-7B\-v0\.3, and Qwen2\-7B are byte\-for\-byte identical regardless of which of the four non\-default seed values \(75, 456, 789, 1024\) we compared against the seed\-42 baseline\.

Table 7:SHA\-256 digests \(first 8 hex chars\) of the serialized Stage\-1 activation tensor file for each \(model, dataset\) pair\. “Match” counts, out of the four non\-default seeds \(75/456/789/1024\), how many produced a digest identical to the seed\-42 run\.

## 6Discussion

Cross\-architecture generality\.The results in Table[5](https://arxiv.org/html/2608.08029#S5.T5)are consistent with the hypothesis in Section[3\.2](https://arxiv.org/html/2608.08029#S3.SS2): probe F1 score on WildJailbreak and BeaverTails stays within roughly half a point of the original LLaMA\-3\.1\-8B numbers for all three additional backbones \(Gemma\-4\-E4B, Mistral\-7B\-v0\.3, Qwen2\-7B\), despite these models differing in pretraining corpus, alignment recipe, and hidden dimensionality \(see Appendix\)\. This is consistent with the claim inKhatriet al\.\([2026](https://arxiv.org/html/2608.08029#bib.bib1)\); Saglamet al\.\([2025](https://arxiv.org/html/2608.08029#bib.bib2)\); Zhaoet al\.\([2025](https://arxiv.org/html/2608.08029#bib.bib26)\)that safety\-relevant information is linearly decoded in final\-layer, last\-token representations being a property of how current aligned LLMs represent language in general, rather than an artifact specific to the Llama alignment pipeline\. We caution, however, that all four backbones we have tested to date are open\-weight, English\-centric, RLHF\- or DPO\-aligned chat models in a similar 4B–8B parameter range; we have not yet tested whether the pattern holds at very different scales, for base \(non\-aligned\) models, or for models aligned with substantially different objectives\.

Non\-determinism and the guard\-model comparison\.Different sub\-stages of an LLM behave differently with respect to seed sensitivity, and it is important to keep them separate when interpreting the zero\-variance result\. From our results, we observe that activation vectors extracted at the last hidden layer of the last prefill token remained invariant to seed variations\. This is not the same as claiming that the LLMs generated identical tokens for different seed values\.

Non\-determinism in LLM inference can be introduced due to various factorsYuanet al\.\([2026](https://arxiv.org/html/2608.08029#bib.bib31)\)\. While a majority of the factors causing non\-deterministic outcomes come into play after the first token has been generated, it remains possible for non\-deterministic outcomes to occur at the prefill stage, e\.g use of non\-deterministic algorithms like Chunked Prefill\.Agrawalet al\.\([2025](https://arxiv.org/html/2608.08029#bib.bib30)\)\. We interpret the data to infer that it is likely that our current experimental settings were inadequate at recreating non\-deterministic behavior in the LLMs across several inference runs\. As a result, the complete relationship between non\-deterministic LLM behavior and probe performance remains an open question\. However, we show that prefill stage activations remain unchanged to batch order despite changes in seed values\.

As Table[7](https://arxiv.org/html/2608.08029#S5.T7)confirms, activation extraction, under current experimental settings, is a pure function of the frozen weights and input tokens\. If the variance in F1 patterns were to survive non\-determinism in latent activation, it would suggest that the probe F1 scores are robust to noise in latent space activations, strengthening the original contribution\. We do not yet draw this conclusion\. The defensible claim is narrower: our protocol has not detected variance attributable to minibatch order at the last token last layer level\. This is weaker than a general claim that the probe is variance\-free across different implementations\.

Practical implications\.Taken together, the cross\-architecture and non\-determinism results are cautiously encouraging for latent\-space probing as a lightweight alternative or supplement to external guardrails: performance looks portable across at least a handful of open\-weight models, across modalities, and post\-prefill last token activations appear deterministic with respect to variations in seed and batch order\.

## 7Limitations

As with the original paper, we extract only final\-layer, last\-token activations; earlier layers or alternative pooling strategies may behave differently and are left to future work\. Concurrent work diagnoses exactly this failure mode directly:Doda \([2026](https://arxiv.org/html/2608.08029#bib.bib28)\)show that final\-token safety probes can miss jailbreak prompts whose unsafe evidence is distributed across earlier user\-token representations rather than concentrated at the final prefill position, and that naive fixes \(wider probe bottlenecks, naive max\-pooling over tokens\) do not reliably resolve the gap\. Our reproduction inherits this limitation by design, since faithfully reproducingKhatriet al\.\([2026](https://arxiv.org/html/2608.08029#bib.bib1)\)requires keeping the extraction configuration unchanged\. Our non\-determinism analysis \(Section[5\.4](https://arxiv.org/html/2608.08029#S5.SS4)\) found that extractions of the last hidden state of the last token was deterministic under our experimental settings, confirmed by the digest matches in Table[7](https://arxiv.org/html/2608.08029#S5.T7)\. The zero\-variance F1 result should be read as “no variance detected by our current protocol” rather than as a general claim about probe stability, and should not be used to argue that single\-seed point estimates are reliable outside our specific setup\. Our cross\-architecture extension is limited to the backbones in Table[2](https://arxiv.org/html/2608.08029#S3.T2)and does not cover closed\-weight models, non\-English data, or multimodal inputs\. Our non\-determinism analysis captures extraction\- and training\-time stochasticity under fixed data splits; it does not capture variance from alternative train/validation/test partitions, which would require a separate cross\-validation study\. Finally, both the original paper and this reproduction restrict evaluation to prompt\-level binary harmfulness classification; findings may not transfer to response\-level moderation or finer\-grained, multi\-category risk scoring\.

## Ethics Statement

This work analyzes existing safety classification datasets and does not collect new human data\. As with any safety\-classifier research, probes of the kind studied here could in principle be repurposed for overly broad or opaque content filtering if deployed without transparency; we do not release any capability beyond what is already available in the original paper’s public artifacts and the datasets’ original releases\. We do not reproduce or redistribute any redacted or sensitive content from AEGIS 2\.0 beyond what the dataset’s original terms permit\.

## References

- A\. Agrawal, N\. Kedia, A\. Panwar, J\. Mohan, N\. Kwatra, B\. S\. Gulavani, A\. Tumanov, and R\. Ramjee \(2025\)Efficient llm inference via chunked prefills\.ACM SIGOPS Operating Systems Review59\(1\),pp\. 9–16\.Cited by:[§6](https://arxiv.org/html/2608.08029#S6.p3.1)\.
- J\. Albrethsen, Y\. Datta, K\. Kumar, and S\. Rajasekar \(2026\)DeepContext: stateful real\-time detection of multi\-turn adversarial intent drift in LLMs\.arXiv preprint arXiv:2602\.16935\.Cited by:[§2](https://arxiv.org/html/2608.08029#S2.SS0.SSS0.Px3.p1.1)\.
- J\. Chen, X\. Wang, Z\. Yao, Y\. Bai, L\. Hou, and J\. Li \(2025\)Towards understanding safety alignment: a mechanistic perspective from safety neurons\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2608.08029#S1.p1.1),[§2](https://arxiv.org/html/2608.08029#S2.SS0.SSS0.Px2.p1.1)\.
- X\. W\. Chia, S\. L\. Wong, and J\. Pan \(2025\)Probing latent subspaces in LLM for AI security: identifying and manipulating adversarial states\.arXiv preprint arXiv:2503\.09066\.Cited by:[§2](https://arxiv.org/html/2608.08029#S2.SS0.SSS0.Px2.p1.1)\.
- H\. Cunningham, J\. Wei, Z\. Wang, A\. Persic, A\. Peng, J\. Abderrachid, R\. Agarwal, B\. Chen, A\. Cohen, A\. Dau,et al\.\(2026\)Constitutional classifiers\+\+: efficient production\-grade defenses against universal jailbreaks\.arXiv preprint arXiv:2601\.04603\.Cited by:[§2](https://arxiv.org/html/2608.08029#S2.SS0.SSS0.Px2.p1.1)\.
- S\. Doda \(2026\)Before the last token: diagnosing final\-token safety probe failures\.arXiv preprint arXiv:2605\.12726\.Cited by:[§7](https://arxiv.org/html/2608.08029#S7.p1.1)\.
- Y\. Dong, R\. Mu, G\. Jin, Y\. Qi, J\. Hu, X\. Zhao, J\. Meng, W\. Ruan, and X\. Huang \(2024\)Building guardrails for large language models\.arXiv preprint arXiv:2402\.01822\.Cited by:[§2](https://arxiv.org/html/2608.08029#S2.SS0.SSS0.Px1.p1.1)\.
- J\. Fiotto\-Kaufman, A\. R\. Loftus, E\. Todd, J\. Brinkmann, K\. Pal, D\. Troitskii, M\. Ripa, A\. Belfki, C\. Rager, C\. Juang, A\. Mueller, S\. Marks, A\. S\. Sharma, F\. Lucchetti, N\. Prakash, C\. Brodley, A\. Guha, J\. Bell, B\. C\. Wallace, and D\. Bau \(2024\)NNsight and NDIF: democratizing access to foundation model internals\.arXiv preprint arXiv:2407\.14561\.Cited by:[§3\.1](https://arxiv.org/html/2608.08029#S3.SS1.p1.4)\.
- S\. Ghosh, P\. Varshney, M\. N\. Sreedhar, A\. Padmakumar, T\. Rebedea, J\. R\. Varghese, and C\. Parisien \(2025\)AEGIS 2\.0: a diverse AI safety dataset and risks taxonomy for alignment of LLM guardrails\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 5992–6026\.Cited by:[§1](https://arxiv.org/html/2608.08029#S1.p1.1)\.
- S\. Han, K\. Rao, A\. Ettinger, L\. Jiang, B\. Y\. Lin, N\. Lambert, Y\. Choi, and N\. Dziri \(2024\)WildGuard: open one\-stop moderation tools for safety risks, jailbreaks, and refusals of LLMs\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2608.08029#S1.p1.1),[§2](https://arxiv.org/html/2608.08029#S2.SS0.SSS0.Px1.p1.1)\.
- K\. Hung, C\. Ko, A\. Rawat, I\. Chung, W\. H\. Hsu, and P\. Chen \(2025\)Attention tracker: detecting prompt injection attacks in LLMs\.InFindings of the Association for Computational Linguistics: NAACL 2025,pp\. 2309–2322\.Cited by:[§2](https://arxiv.org/html/2608.08029#S2.SS0.SSS0.Px1.p1.1)\.
- H\. Inan, K\. Upasani, J\. Chi, R\. Rungta, K\. Iyer, Y\. Mao, M\. Tontchev, Q\. Hu, B\. Fuller, D\. Testuggine, and M\. Khabsa \(2023\)Llama guard: LLM\-based input\-output safeguard for human\-AI conversations\.arXiv preprint arXiv:2312\.06674\.Cited by:[§1](https://arxiv.org/html/2608.08029#S1.p1.1),[§2](https://arxiv.org/html/2608.08029#S2.SS0.SSS0.Px1.p1.1)\.
- J\. Ji, M\. Liu, J\. Dai, X\. Pan, C\. Zhang, C\. Bian, B\. Chen, R\. Sun, Y\. Wang, and Y\. Yang \(2023\)BeaverTails: towards improved safety alignment of LLM via a human\-preference dataset\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2608.08029#S1.p1.1)\.
- L\. Jiang, K\. Rao, S\. Han, A\. Ettinger, F\. Brahman, S\. Kumar, N\. Mireshghallah, X\. Lu, M\. Sap, Y\. Choi,et al\.\(2024\)WildTeaming at scale: from in\-the\-wild jailbreaks to \(adversarially\) safer language models\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 47094–47165\.Cited by:[§1](https://arxiv.org/html/2608.08029#S1.p1.1)\.
- A\. Khatri, C\. Prabhu, and O\. Neogi \(2026\)Safety beyond the interface: detecting harm via latent states in large language models\.In2026 56th Annual IEEE International Conference on Dependable Systems and Networks Workshops \(DSN\-W\),Vol\.,pp\. 48–52\.External Links:[Document](https://dx.doi.org/10.1109/DSN-W70714.2026.00027)Cited by:[item 2](https://arxiv.org/html/2608.08029#S1.I1.i2.p1.1),[1st item](https://arxiv.org/html/2608.08029#S1.I2.i1.p1.1),[§1](https://arxiv.org/html/2608.08029#S1.p1.1),[§1](https://arxiv.org/html/2608.08029#S1.p4.1),[§2](https://arxiv.org/html/2608.08029#S2.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2608.08029#S3.SS2.p1.1),[Table 1](https://arxiv.org/html/2608.08029#S3.T1),[§3](https://arxiv.org/html/2608.08029#S3.p1.1),[§5\.1](https://arxiv.org/html/2608.08029#S5.SS1.p1.1),[Table 4](https://arxiv.org/html/2608.08029#S5.T4),[§6](https://arxiv.org/html/2608.08029#S6.p1.1),[§7](https://arxiv.org/html/2608.08029#S7.p1.1)\.
- I\. Llorente\-Saguer \(2026\)Harmful intent as a geometrically recoverable feature of LLM residual streams\.arXiv preprint arXiv:2604\.18901\.Cited by:[§2](https://arxiv.org/html/2608.08029#S2.SS0.SSS0.Px2.p1.1)\.
- I\. Loshchilov and F\. Hutter \(2019\)Decoupled weight decay regularization\.InInternational Conference on Learning Representations,Cited by:[§3\.1](https://arxiv.org/html/2608.08029#S3.SS1.p2.4)\.
- B\. Saglam, P\. Kassianik, B\. Nelson, S\. Weerawardhena, Y\. Singer, and A\. Karbasi \(2025\)Large language models encode semantics and alignment in linearly separable representations\.InProceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia\-Pacific Chapter of the Association for Computational Linguistics,pp\. 2282–2303\.Cited by:[§1](https://arxiv.org/html/2608.08029#S1.p1.1),[§2](https://arxiv.org/html/2608.08029#S2.SS0.SSS0.Px2.p1.1),[§6](https://arxiv.org/html/2608.08029#S6.p1.1)\.
- J\. Yuan, H\. Li, X\. Ding, W\. Xie, Y\. Li, W\. Zhao, K\. Wan, J\. Shi, X\. Hu, and Z\. Liu \(2026\)Understanding and mitigating numerical sources of nondeterminism in llm inference\.Advances in Neural Information Processing Systems38,pp\. 169819–169851\.Cited by:[§6](https://arxiv.org/html/2608.08029#S6.p3.1)\.
- W\. Zeng, Y\. Liu, R\. Mullins, L\. Peran, J\. Fernandez, H\. Harkous, K\. Narasimhan, D\. Proud, P\. Kumar, B\. Radharapu,et al\.\(2024\)ShieldGemma: generative AI content moderation based on gemma\.arXiv preprint arXiv:2407\.21772\.Cited by:[§1](https://arxiv.org/html/2608.08029#S1.p1.1),[§2](https://arxiv.org/html/2608.08029#S2.SS0.SSS0.Px1.p1.1)\.
- M\. Zhang, K\. K\. Goh, P\. Zhang, J\. Sun, R\. L\. Xin, and H\. Zhang \(2024\)LLMScan: causal scan for LLM misbehavior detection\.arXiv preprint arXiv:2410\.16638\.Cited by:[§2](https://arxiv.org/html/2608.08029#S2.SS0.SSS0.Px2.p1.1)\.
- J\. Zhao, J\. Huang, Z\. Wu, D\. Bau, and W\. Shi \(2025\)LLMs encode harmfulness and refusal separately\.Advances in Neural Information Processing Systems \(NeurIPS\)\.Note:arXiv preprint arXiv:2507\.11878Cited by:[§2](https://arxiv.org/html/2608.08029#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2608.08029#S2.SS0.SSS0.Px2.p1.1),[§6](https://arxiv.org/html/2608.08029#S6.p1.1)\.
- A\. Zou, L\. Phan, S\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski,et al\.\(2023\)Representation engineering: a top\-down approach to AI transparency\.arXiv preprint arXiv:2310\.01405\.Cited by:[§1](https://arxiv.org/html/2608.08029#S1.p1.1),[§2](https://arxiv.org/html/2608.08029#S2.SS0.SSS0.Px2.p1.1)\.

## Appendix AAdditional Implementation Details

*Extraction Hardware used*: Nvidia T4 \(Azure VMs\), Nvidia L4 \(Google Colab\)

Table[8](https://arxiv.org/html/2608.08029#A1.T8)lists the training and extraction hyperparameters used throughout, matching Section[3\.1](https://arxiv.org/html/2608.08029#S3.SS1)and held fixed across all backbones\.

Table[9](https://arxiv.org/html/2608.08029#A1.T9)reports the exact parameter count of the MLP probe for each backbone, following directly from the fixed architecture in Table[1](https://arxiv.org/html/2608.08029#S3.T1)and each backbone’s hidden dimensiondd\(Table[2](https://arxiv.org/html/2608.08029#S3.T2)\)\.

Table 8:Training and extraction hyperparameters, held fixed across all backbones and datasets\.Table 9:Exact parameter count of the MLP probe for each backbone, determined by the fixed architecture in Table[1](https://arxiv.org/html/2608.08029#S3.T1)and the backbone’s hidden dimension\.#### Architectural diversity\.

Table[10](https://arxiv.org/html/2608.08029#A1.T10)summarizes the architectural diversity across our four backbone models\. Despite significant differences in hidden dimensionality \(2560–4096\), depth \(28–42 layers\), attention mechanism \(full vs\. interleaved sliding\-window in Gemma\-4\-E4B\), KV head counts, and modality \(Gemma\-4\-E4B is natively multimodal with text, image, and audio inputs, while the remaining three are text\-only\), our MLP probes achieve consistent F1 score across all backbones \(Table[5](https://arxiv.org/html/2608.08029#S5.T5)\)\. This suggests the safety\-relevant signal captured by latent\-space probes is not tied to any specific architectural choice\.

Table 10:Architectural summary of the four backbone models\. All four use RMSNorm\. Gemma\-4\-E4B uses interleaved sliding\-window \(window 512\) and full\-attention layers \(1 full per 6 sliding\), per\-layer embeddings \(PLE\), and is natively multimodal; the other three are text\-only dense decoders\.

## Appendix BPer\-Seed Raw Results

The experiment results reported in Table[11](https://arxiv.org/html/2608.08029#A2.T11)focus on testing variance in F1 with respect to seed\. They might not use the most efficient probe model checkpoint, and thus F1 values may differ slightly from those reported in the main paper\.

Table 11:Precision, recall, and F1 for each model / dataset / seed triplet