Harmfulness Propagation Dynamics: Layer-wise Trajectories of Adversarial Intent in Large Language Models

# Harmfulness Propagation Dynamics: Layer-wise Trajectories of Adversarial Intent in Large Language Models
Source: [https://arxiv.org/html/2609.13534](https://arxiv.org/html/2609.13534)
Noor Islam S\. MohammadAffiliation:Department of Computer Science, İTÜ, İstanbul, TürkiyeCorrespondence to:[islam23@itu\.edu\.tr](mailto:[email protected])

###### Abstract

We identifyHarmfulness Propagation Dynamics \(HPD\): for harmful prompts, the projection of the last\-token hidden state onto a learned harm direction rises monotonically with transformer depth, whereas benign prompts remain flat or oscillatory\. This cross\-layer signature reflects harmful intent as a*progressively resolved*semantic property: surface form appears early, while pragmatic intent consolidates later, making the*trajectory shape*more informative than any single\-layer snapshot\. Moreover, LDA\-based harm directions, learned per layer, remain stable across random splits \(pairwise cosine similarity\>0\.97\>0\.97\), supporting the projection sequence as a reproducible structured signal\. Building on HPD, we introduceHerald\(HarmfulEncodingRecognition viaActivationLayerDynamics\)\. This lightweight input moderator extracts a seven\-dimensional feature record, slope, curvature, monotonicity, onset layer, and related statistics from the cross\-layer projection sequence and classifies it with a 288\-parameter MLP\.Heraldstores onedd\-dimensional direction per layer \(262262KB for a 32\-layer,d=4096d\{=\}4096model\), requires no gradient computation during training, and adds only2\.6×10−62\.6\{\\times\}10^\{\-6\}prefill FLOPs at inference\. Across eight prompt\-harmfulness benchmarks and four model families,Heraldachieves an average F1 of89\.389\.3on OLMo2\-7B, surpassing all tested guard models on adversarial jailbreak detection \(98\.498\.4vs\.96\.996\.9F1\) and outperforming prior latent\-based methods by2\.32\.3\-4\.14\.1F1 points on every backbone\. Per\-instance trajectories provide machine\-readable audit records that reveal*when*and*how*harmfulness emerges, offering an interpretability advantage over single\-layer approaches\.

###### Keywords:

LLM safety, input moderation, mechanistic interpretability, representation geometry, jailbreak detection

## 1Introduction

Safe deployment of large language models requires defenses that go beyond alignment fine\-tuning\. Even well\-aligned models remain vulnerable to adversarial and indirect prompts\([Perez et al\., 2022](https://arxiv.org/html/2609.13534#bib.bib5);[Greshake et al\., 2023](https://arxiv.org/html/2609.13534#bib.bib7);[Ganguli et al\., 2022](https://arxiv.org/html/2609.13534#bib.bib6);[Carlini et al\., 2023](https://arxiv.org/html/2609.13534#bib.bib8)\), and alignment may come at the cost of general capability\([Askell et al\., 2021](https://arxiv.org/html/2609.13534#bib.bib3)\)\. Input moderation screening requests before generation is a complementary safeguard that blocks unsafe prompts while avoiding the full cost of a forward pass on harmful inputs\. Existing moderators occupy two extremes\. Guard models\([Markov et al\., 2023](https://arxiv.org/html/2609.13534#bib.bib9);[Vidgen et al\., 2023](https://arxiv.org/html/2609.13534#bib.bib10)\)are accurate but incur the expense of an additional large model \(≈14\{\\approx\}14GB for a 7B guard\)\. Latent\-based methods\([Ryu et al\., 2024](https://arxiv.org/html/2609.13534#bib.bib12);[Li et al\., 2025](https://arxiv.org/html/2609.13534#bib.bib13)\)are lightweight but commit to a*single*layer’s hidden state, discarding information carried by the progression of representations across depth\.

Our starting point: harmfulness is a progressively resolved signal\.Prior work on transformer representation geometry shows that different linguistic properties are encoded at different depths: syntax in early layers, semantics in middle layers, and task\-relevant pragmatics in late layers\([Jawahar et al\., 2019](https://arxiv.org/html/2609.13534#bib.bib14);[Tenney et al\., 2019](https://arxiv.org/html/2609.13534#bib.bib15);[Geva et al\., 2021](https://arxiv.org/html/2609.13534#bib.bib17)\)\. We ask whether*harmfulness*follows this pattern and specifically whether the trajectory of a prompt’s representation projected onto a harm direction, across all layers, is itself a discriminative signal\.

Core observation \(HPD\)\.Projecting each layer’s last\-token hidden state onto a per\-layer LDA harm direction yields a trajectory\{pl\}l=1L\\\{p\_\{l\}\\\}\_\{l=1\}^\{L\}that differs sharply between harmful and benign prompts \(Section[3](https://arxiv.org/html/2609.13534#S3)\)\. For harmful inputs, particularly jailbreaks, the trajectory rises steadily and monotonically from near\-zero in early layers to strongly positive values in late layers\. Benign prompts remain flat or oscillatory, with no systematic directional growth\.

We call thisHarmfulness Propagation Dynamics \(HPD\)\. Critically, HPD is*not*merely a restatement of the well\-known fact that late\-layer representations are more discriminative\. The trajectory’s*shape*carries information beyond the terminal value: the onset layer, the monotonicity of growth, and the curvature each contribute independently \(Section[7\.1](https://arxiv.org/html/2609.13534#S7.SS1)\), and the gap between trajectory and terminal\-only classification is largest precisely for adversarial jailbreaks, the most practically important detection target\.

Relation to representation engineering\.[Zou et al\. \(2023c\)](https://arxiv.org/html/2609.13534#bib.bib22)show that linear directions in activation space can steer and probe model behavior\. HPD extends this insight in a distinct direction: rather than learning a single probe or steering vector, we track how a harm direction evolves*across layers*and treat the resulting trajectory as a first\-class data object for classification\. This cross\-layer dynamics view is orthogonal to activation\-space probing at a fixed depth\.

Heraldexploits HPD through four steps: \(i\) Learn a per\-layer LDA harm direction𝐯l\\mathbf\{v\}\_\{l\}using only a single gradient\-free forward pass over the training set; \(ii\) project each layer’s last\-token hidden state onto𝐯l\\mathbf\{v\}\_\{l\}to obtain scalarplp\_\{l\}; \(iii\) extract a compact seven\-dimensional feature vectorϕ⁡\(𝐩\)\\boldsymbol\{\\phi\}\(\\mathbf\{p\}\)capturing the trajectory’s slope, curvature, monotonicity, onset layer, and related statistics; and \(iv\) classifyϕ\\boldsymbol\{\\phi\}with a 288\-parameter MLP trained in seconds on CPU\.

Contributions:

- •We identify and formally characterizeHarmfulness Propagation Dynamics, a cross\-layer signature of harmful prompts in which per\-layer harm projections rise monotonically with depth\. We prove that LDA harm directions converge to stable axes \(pairwise cosine similarity\>0\.97\>0\.97across five splits\) and show that this stability is necessary for trajectory\-based classification\.
- •We introduceHerald, a gradient\-free trajectory moderator requiringO⁡\(L​d\)O\(Ld\)memory \(262262KB for a 32\-layer model\) and negligible runtime overhead \(2\.6×10−62\.6\{\\times\}10^\{\-6\}of prefill FLOPs\), distinguishing it from guard models and full\-covariance latent methods\.
- •Heraldsurpasses all tested guard models on adversarial jailbreak detection and outperforms all latent\-based baselines on average F1 across eight benchmarks and four model families, with2\.32\.3–4\.14\.1F1 improvements over prior latent methods\.
- •We show that trajectory features provide*category\-specific interpretability*: jailbreaks exhibit early onset \(l^∗≈7\\hat\{l\}^\{\*\}\{\\approx\}7\) and high monotonicity \(0\.830\.83\), whereas social stereotypes onset late \(l^∗≈19\\hat\{l\}^\{\*\}\{\\approx\}19\) with inconsistent growth \(0\.580\.58\)\. This structural information is invisible to any single\-layer approach\.
- •Comprehensive ablations across eight benchmarks isolate the contributions of direction learning, layer coverage, token position, normalization, classifier capacity, and shrinkage regularization, constituting a reproducible evaluation framework for cross\-layer safety methods\.

## 2Related Work

#### LLM safety and alignment\.

RLHF\([Christiano et al\., 2017](https://arxiv.org/html/2609.13534#bib.bib1);[Stiennon et al\., 2020](https://arxiv.org/html/2609.13534#bib.bib2)\)and DPO\([Rafailov et al\., 2023](https://arxiv.org/html/2609.13534#bib.bib4)\)improve model safety, yet aligned models remain susceptible to adversarial prompting\([Ganguli et al\., 2022](https://arxiv.org/html/2609.13534#bib.bib6);[Perez et al\., 2022](https://arxiv.org/html/2609.13534#bib.bib5);[Zou et al\., 2023c](https://arxiv.org/html/2609.13534#bib.bib22)\)\. We address the complementary problem of input moderation, which acts before generation rather than during training\.

#### Input moderation\.

Guard models\([Markov et al\., 2023](https://arxiv.org/html/2609.13534#bib.bib9);[Vidgen et al\., 2023](https://arxiv.org/html/2609.13534#bib.bib10)\)achieve strong classification but require a second large model\. Rule\-based filters\([Röttger et al\., 2021](https://arxiv.org/html/2609.13534#bib.bib11)\)are interpretable but brittle\. Latent\-based methods\([Ryu et al\., 2024](https://arxiv.org/html/2609.13534#bib.bib12);[Li et al\., 2025](https://arxiv.org/html/2609.13534#bib.bib13)\)use host\-model activations efficiently but classify from a single chosen layer\.Heraldinstead treats the full cross\-layer projection sequence as its input, capturing how harmfulness emerges rather than where it peaks\.

#### Representation geometry and linear probing\.

Linear directions in LLM hidden spaces encode semantically meaningful concepts\([Mikolov et al\., 2013](https://arxiv.org/html/2609.13534#bib.bib18);[Park et al\., 2024](https://arxiv.org/html/2609.13534#bib.bib19)\)\. Probing studies confirm that layers encode increasingly abstract properties, from syntax to pragmatics\([Jawahar et al\., 2019](https://arxiv.org/html/2609.13534#bib.bib14);[Tenney et al\., 2019](https://arxiv.org/html/2609.13534#bib.bib15);[Rogers et al\., 2020](https://arxiv.org/html/2609.13534#bib.bib16)\)\.[Zou et al\. \(2023c\)](https://arxiv.org/html/2609.13534#bib.bib22)demonstrates that reading vectors learned via contrastive activation addition can probe and steer behavior;[Markov et al\. \(2023\)](https://arxiv.org/html/2609.13534#bib.bib9)further shows that truth\-value directions follow a linear geometry\.Heralddiffers from all these approaches: we learn*separate*LDA directions per layer and classify the resulting cross\-layer trajectory rather than the projection at any fixed depth\.

#### Refusal and safety directions\.

[Park et al\. \(2024\)](https://arxiv.org/html/2609.13534#bib.bib19)identify a linear ”refusal direction” in residual streams and show that ablating it removes safety behavior\. This is complementary to our work: we monitor the*input*side by tracking how a harmful direction accumulates projection mass across layers, rather than intervening on the*output*side\. The two directions also differ conceptually—refusal directions characterize generation\-time behavior, while HPD directions characterize inference\-time encoding of the prompt’s intent\.

#### Trajectory and time\-series methods for anomaly detection\.

Time\-series features such as slope, curvature, and monotonicity are standard tools in anomaly detection\([Christ et al\., 2018](https://arxiv.org/html/2609.13534#bib.bib20)\)\.Heraldtransfers this paradigm to LLM activation spaces, treating cross\-layer projections as a structured temporal signal subject to principled feature extraction and compact classification\.

#### LLM governance and structured evaluation\.

Structured, reproducible evaluation is increasingly recognized as essential for responsible deployment\([Ganguli et al\., 2022](https://arxiv.org/html/2609.13534#bib.bib6);[Vidgen et al\., 2023](https://arxiv.org/html/2609.13534#bib.bib10)\)\.Herald’s per\-instance trajectories constitute machine\-readable audit records that expose*when*and*how*harmfulness emerges, directly supporting governance workflows requiring more than a binary safe/unsafe label\.

## 3Harmfulness Propagation Dynamics

### 3\.1Definition and Observation

#### Setup\.

Letxxbe a prompt of lengthTTprocessed by anLL\-layer LLM, and let𝐡l∈ℝd\\mathbf\{h\}\_\{l\}\\in\\mathbb\{R\}^\{d\}denote the last\-token hidden state at layerll\. For each layer, we learn a harm direction𝐯l∈ℝd\\mathbf\{v\}\_\{l\}\\in\\mathbb\{R\}^\{d\}\(defined formally in Section[4](https://arxiv.org/html/2609.13534#S4)\) and compute the cosine projection

pl​\(x\)=⟨𝐡l‖𝐡l‖,𝐯l‖𝐯l‖⟩∈\[−1,1\]\.p\_\{l\}\(x\)=\\left\\langle\\frac\{\\mathbf\{h\}\_\{l\}\}\{\\\|\\mathbf\{h\}\_\{l\}\\\|\},\\;\\frac\{\\mathbf\{v\}\_\{l\}\}\{\\\|\\mathbf\{v\}\_\{l\}\\\|\}\\right\\rangle\\in\[\-1,1\]\.\(1\)The sequence\{pl\}l=1L\\\{p\_\{l\}\\\}\_\{l=1\}^\{L\}is theharm trajectoryof promptxx\.

#### Empirical observation\.

On a Llama\-3\.1\-8B\-Instruct backbone trained with WildGuardMix, harmful prompts produce trajectories with three characteristic phases: \(i\) near\-zero projections in early layers \(l≲8l\\lesssim 8\), \(ii\) a steady monotonic rise through middle layers, and \(iii\) strongly positive values \(pL≳0\.4p\_\{L\}\\gtrsim 0\.4\) in late layers\. Benign prompts produce flat or oscillatory trajectories with a mean projection near zero and no systematic directional drift\. This pattern—which we callHarmfulness Propagation Dynamics \(HPD\)—is stable across all four model families tested \(Llama\-3\.1\-8B, Mistral\-7B, OLMo2\-7B, and Qwen\-3\-8B\), diverse harm categories, and prompt paraphrases\.

### 3\.2Theoretical Grounding

HPD is consistent with the layered computation hypothesis for transformers\([Jawahar et al\., 2019](https://arxiv.org/html/2609.13534#bib.bib14);[Tenney et al\., 2019](https://arxiv.org/html/2609.13534#bib.bib15)\): early layers process surface form \(tokenization artifacts, punctuation, and lexical identity\), while later layers encode increasingly abstract semantic and pragmatic properties\. We make this precise with the following proposition\.

###### Proposition 3\.1\(Informal\)\.

Suppose that \(i\) harmful intent is a semantic\-pragmatic property primarily encoded in later transformer layers; \(ii\) the LDA harm direction𝐯l\\mathbf\{v\}\_\{l\}is a consistent estimator of the optimal Fisher discriminant at layerll; and \(iii\) the projection of harmful representations onto𝐯l\\mathbf\{v\}\_\{l\}increasesllin expectation while benign representations remain bounded\. Then, the expected trajectory of harmful prompts is monotonically increasing, whereas benign trajectories satisfy𝔼⁡\[pl\]≈0\\mathbb\{E\}\[p\_\{l\}\]\\approx 0for allll\.

Assumptions \(i\) and \(iii\) are validated empirically in Appendix[B](https://arxiv.org/html/2609.13534#A2)and are consistent with prior probing results\([Tenney et al\., 2019](https://arxiv.org/html/2609.13534#bib.bib15);[Geva et al\., 2021](https://arxiv.org/html/2609.13534#bib.bib17)\)\. Assumption \(ii\) is validated by the direction stability analysis in Appendix[H](https://arxiv.org/html/2609.13534#A8): pairwise cosine similarity of𝐯l\\mathbf\{v\}\_\{l\}learning on independent splits exceeds0\.970\.97at every layer and backbone \(Table[12](https://arxiv.org/html/2609.13534#A8.T12)\), confirming that LDA converges to a stable population direction rather than fitting sampling noise\.

#### Why the trajectory, not just the terminal value?

The final projectionpLp\_\{L\}is indeed informative \(Table[3](https://arxiv.org/html/2609.13534#S7.T3)\)\. However, two prompts can share similarpLp\_\{L\}values while differing markedly in trajectory shape: a jailbreak that reveals harmful intent gradually \(early\-onset, monotone rise\) and a borderline prompt that happens to land near the harm direction at the final layer through coincidence \(no consistent rise, late onset\) have very different risk profiles\. The trajectory features—particularly the onset layerl^∗\\hat\{l\}^\{\*\}and monotonicity—discriminate these cases where the terminal value alone cannot\. We quantify this advantage in Section[6\.3](https://arxiv.org/html/2609.13534#S6.SS3)\.

## 4Methodology

### 4\.1Per\-Layer Harm Directions via LDA

For each layerll, we apply binary LDA to find the direction maximally separating last\-token hidden states of harmful versus safe prompts:

𝐯l=argmax𝐯:‖𝐯‖=1𝐯⊤​𝐒B\(l\)​𝐯𝐯⊤​𝐒W\(l\)​𝐯,\\mathbf\{v\}\_\{l\}=\\arg\\max\_\{\\mathbf\{v\}:\\\|\\mathbf\{v\}\\\|=1\}\\frac\{\\mathbf\{v\}^\{\\top\}\\mathbf\{S\}\_\{B\}^\{\(l\)\}\\mathbf\{v\}\}\{\\mathbf\{v\}^\{\\top\}\\mathbf\{S\}\_\{W\}^\{\(l\)\}\\mathbf\{v\}\},\(2\)where𝐒B\(l\)\\mathbf\{S\}\_\{B\}^\{\(l\)\}and𝐒W\(l\)\\mathbf\{S\}\_\{W\}^\{\(l\)\}are the between\-class and within\-class scatter matrices\. The closed\-form solution is:

𝐯l∝\(𝐒W\(l\)\)−1​\(𝝁lharm−𝝁lsafe\),\\mathbf\{v\}\_\{l\}\\propto\\bigl\(\\mathbf\{S\}\_\{W\}^\{\(l\)\}\\bigr\)^\{\-1\}\\bigl\(\\boldsymbol\{\\mu\}\_\{l\}^\{\\mathrm\{harm\}\}\-\\boldsymbol\{\\mu\}\_\{l\}^\{\\mathrm\{safe\}\}\\bigr\),\(3\)normalized to unit length\. We invert𝐒W\(l\)\\mathbf\{S\}\_\{W\}^\{\(l\)\}using analytic Ledoit\-Wolf shrinkage\([Ledoit and Wolf, 2004](https://arxiv.org/html/2609.13534#bib.bib21)\), which replaces the sample covariance with a well\-conditioned convex combination of itself and a scaled identity:𝐒W^\(l\)=\(1−α\)​𝐒W\(l\)\+α⋅tr⁡\(𝐒W\(l\)\)d​𝐈\\hat\{\\mathbf\{S\}\_\{W\}\}^\{\(l\)\}=\(1\-\\alpha\)\\mathbf\{S\}\_\{W\}^\{\(l\)\}\+\\alpha\\cdot\\tfrac\{\\mathrm\{tr\}\(\\mathbf\{S\}\_\{W\}^\{\(l\)\}\)\}\{d\}\\mathbf\{I\}, whereα\\alphais computed analytically\. This is critical whend≫ntraind\\gg n\_\{\\mathrm\{train\}\}: without regularization, F1 degrades by up to 27 points at low data regimes \(Table[10](https://arxiv.org/html/2609.13534#S7.T10)\)\.

Why LDA over simpler alternatives?The class\-mean difference𝝁lharm−𝝁lsafe\\boldsymbol\{\\mu\}\_\{l\}^\{\\mathrm\{harm\}\}\-\\boldsymbol\{\\mu\}\_\{l\}^\{\\mathrm\{safe\}\}ignores within\-class variation: prompts with the same label differ in length, wording, and rhetorical style, producing substantial within\-class scatter\. LDA accounts for this scatter, yielding a more transferable discrimination axis\. Empirically, LDA outperforms the mean\-difference direction by0\.50\.5–0\.80\.8F1 \(Table[4](https://arxiv.org/html/2609.13534#S7.T4)\)\. Each layer retains only𝐯l∈ℝd\\mathbf\{v\}\_\{l\}\\in\\mathbb\{R\}^\{d\}\(≈8\{\\approx\}8KB atd=4096d\{=\}4096half precision; scatter matrices are discarded after training\.

### 4\.2Trajectory Feature Extraction

Given layer\-wise projectionspl=⟨𝐡^l,𝐯l⟩p\_\{l\}=\\langle\\hat\{\\mathbf\{h\}\}\_\{l\},\\mathbf\{v\}\_\{l\}\\rangleforl=1,…,Ll=1,\\dots,L, where𝐡^l=𝐡l/‖𝐡l‖\\hat\{\\mathbf\{h\}\}\_\{l\}=\\mathbf\{h\}\_\{l\}/\\\|\\mathbf\{h\}\_\{l\}\\\|, we construct a compact feature vectorϕ⁡\(𝐩\)∈ℝ7\\boldsymbol\{\\phi\}\(\\mathbf\{p\}\)\\in\\mathbb\{R\}^\{7\}:

ϕ⁡\(𝐩\)=\[pL,p¯,pL−p1,Δ2​𝐩,mono⁡\(𝐩\),pl^∗,l^∗\],\\boldsymbol\{\\phi\}\(\\mathbf\{p\}\)=\\bigl\[p\_\{L\},\\;\\bar\{p\},\\;p\_\{L\}\-p\_\{1\},\\;\\Delta^\{2\}\\mathbf\{p\},\\;\\mathrm\{mono\}\(\\mathbf\{p\}\),\\;p\_\{\\hat\{l\}^\{\*\}\},\\;\\hat\{l\}^\{\*\}\\bigr\],\(4\)where𝐩=\{pl\}l=1L\\mathbf\{p\}=\\\{p\_\{l\}\\\}\_\{l=1\}^\{L\}\. The mean projection isp¯=1L​∑l=1Lpl\\bar\{p\}=\\tfrac\{1\}\{L\}\\sum\_\{l=1\}^\{L\}p\_\{l\}\. The mean absolute curvature isΔ2​𝐩=1L−2​∑l=2L−1\|pl\+1−2​pl\+pl−1\|\\Delta^\{2\}\\mathbf\{p\}=\\tfrac\{1\}\{L\-2\}\\sum\_\{l=2\}^\{L\-1\}\|p\_\{l\+1\}\-2p\_\{l\}\+p\_\{l\-1\}\|, and the monotonicity ratio ismono\(𝐩\)=1L−1∑l=1L−1𝟏\[pl\+1\>pl\]\\mathrm\{mono\}\(\\mathbf\{p\}\)=\\tfrac\{1\}\{L\-1\}\\sum\_\{l=1\}^\{L\-1\}\\mathbf\{1\}\[p\_\{l\+1\}\>p\_\{l\}\]\. We define the onset layer asl^∗=min⁡\{l:pl\>τ90\}\\hat\{l\}^\{\*\}=\\min\\\{l:p\_\{l\}\>\\tau\_\{90\}\\\}, i\.e\., the first layer exceeding the 90th\-percentile threshold, with a corresponding valuepl^∗p\_\{\\hat\{l\}^\{\*\}\}\.

These features capture complementary geometric properties of the trajectory: terminal alignment \(pLp\_\{L\}\), global rise \(pL−p1p\_\{L\}\-p\_\{1\}\), smoothness \(Δ2​𝐩\\Delta^\{2\}\\mathbf\{p\}\), consistency \(mono⁡\(𝐩\)\\mathrm\{mono\}\(\\mathbf\{p\}\)\), and emergence timing \(l^∗\\hat\{l\}^\{\*\},pl^∗p\_\{\\hat\{l\}^\{\*\}\}\)\. Empirically, logistic regression achieves performance within1\.01\.0F1 of the full MLP \(Table[9](https://arxiv.org/html/2609.13534#S7.T9)\), indicating thatϕ\\boldsymbol\{\\phi\}is close to linearly separable and that predictive power is primarily encoded in trajectory geometry rather than classifier complexity\.

### 4\.3Trajectory Classifier

A two\-layer MLPg:ℝ7→\[0,1\]g\\\!:\\\!\\mathbb\{R\}^\{7\}\\to\[0,1\]with a hidden dimension3232classifies trajectory features:

Herald​\(x\)=σ⁡\(𝐖2​ReLU​\(𝐖1​ϕ​\(𝐩\)\+𝐛1\)\+b2\)\.\\textsc\{Herald\}\(x\)=\\sigma\\\!\\bigl\(\\mathbf\{W\}\_\{2\}\\,\\mathrm\{ReLU\}\(\\mathbf\{W\}\_\{1\}\\boldsymbol\{\\phi\}\(\\mathbf\{p\}\)\+\\mathbf\{b\}\_\{1\}\)\+b\_\{2\}\\bigr\)\.\(5\)The MLP has288288parameters and trains on CPU in seconds\. Larger architectures \(hidden size6464\-128128, two hidden layers\) provide no statistically significant benefit \(Table[9](https://arxiv.org/html/2609.13534#S7.T9)\), confirming that the bottleneck is representation quality rather than classifier capacity\.

Algorithm 1Herald: Training and InferenceInput:LLM

ffwith

LLlayers; labeled dataset

𝒟=\{\(xi,yi\)\}\\mathcal\{D\}=\\\{\(x\_\{i\},y\_\{i\}\)\\\}; threshold

τ90\\tau\_\{90\}
Training\(single forward pass, no gradients required\)

foreach

\(xi,yi\)∈𝒟\(x\_\{i\},y\_\{i\}\)\\in\\mathcal\{D\}do

Collect last\-token hidden states

\{𝐡l\(i\)\}l=1L\\\{\\mathbf\{h\}\_\{l\}^\{\(i\)\}\\\}\_\{l=1\}^\{L\}during prefill of

f⁡\(xi\)f\(x\_\{i\}\)
endfor

for

l=1l=1to

LLdo

Compute class means

𝝁lharm,𝝁lsafe\\boldsymbol\{\\mu\}\_\{l\}^\{\\mathrm\{harm\}\},\\boldsymbol\{\\mu\}\_\{l\}^\{\\mathrm\{safe\}\}and scatter matrix

𝐒W\(l\)\\mathbf\{S\}\_\{W\}^\{\(l\)\}
Solve Eq\. \([3](https://arxiv.org/html/2609.13534#S4.E3)\) with Ledoit–Wolf shrinkage to obtain

𝐯l\\mathbf\{v\}\_\{l\}
Discard scatter matrices; retain only

𝐯l\\mathbf\{v\}\_\{l\}
Compute

pl\(i\)=⟨𝐡^l\(i\),𝐯l⟩p\_\{l\}^\{\(i\)\}=\\langle\\hat\{\\mathbf\{h\}\}\_\{l\}^\{\(i\)\},\\mathbf\{v\}\_\{l\}\\ranglefor all

ii
endfor

Extract

ϕ\(i\)\\boldsymbol\{\\phi\}^\{\(i\)\}from

\{pl\(i\)\}\\\{p\_\{l\}^\{\(i\)\}\\\}for all

ii; train MLP

gg
Inference\(no extra forward passes over

ff\)

Given new prompt

xx: collect

\{𝐡l\}l=1L\\\{\\mathbf\{h\}\_\{l\}\\\}\_\{l=1\}^\{L\}during prefill; compute

ϕ⁡\(𝐩\)\\boldsymbol\{\\phi\}\(\\mathbf\{p\}\)
Return

g⁡\(ϕ⁡\(𝐩\)\)≥0\.5g\(\\boldsymbol\{\\phi\}\(\\mathbf\{p\}\)\)\\geq 0\.5as harmful prediction

#### Computational overhead\.

HeraldperformsLLdot products andLLnormalizations at inference, addingO⁡\(2​L​d\)O\(2Ld\)FLOPs\. ForL=32L\{=\}32,d=4096d\{=\}4096, this is≈262​K\{\\approx\}262\\text\{K\}FLOPs against≈100​B\{\\approx\}100\\text\{B\}prefill FLOPs for a 100\-token prompt—a ratio of2\.6×10−62\.6\{\\times\}10^\{\-6\}\. Memory:L×d×2L\\\!\\times\\\!d\\\!\\times\\\!2bytes \(fp16\)=262=262KB–∼650×\{\\sim\}650\\timesless than a full per\-layer covariance approach and∼53,000×\{\\sim\}53\{,\}000\\timesless than a 7B\-parameter guard model \(≈14\{\\approx\}14GB\)\.

## 5Experimental Setup

#### Benchmarks\.

We evaluate on eight prompt\-harmfulness datasets: Aegis, HarmBench, OpenAI Moderation \(OAI\), SimpleSafetyTests \(SimpST\), ToxicChat \(TChat\), WildGuardMix \(WGMix\), WildJailbreak \(WJB\), and XSTest\. Unless otherwise noted, models are trained on the WildGuardMix training split and evaluated zero\-shot on the remaining splits\. Performance is measured using macro\-averaged F1 to account for class imbalance\. This multi\-benchmark protocol captures variability across harm types, distribution shifts, and adversarial prompt constructions\.

Backbones\.We consider four instruction\-tuned model families: Llama\-3\.1\-8B\-Instruct, Mistral\-7B\-Instruct, OLMo2\-7B\-Instruct, and Qwen3\-8B\-Instruct, with additional scaling experiments spanning11B to7070B parameters\.

Baselines\.*Latent\-based:*\(i\)Embed\. Clf\., a linear classifier over embedding\-layer representations; \(ii\)Act\. Delta, which uses differences between hidden states at fixed layers\. Both operate on the same backbone asHerald\.*Guard models:*four standalone safety classifiers \(Guard A–D\) with heterogeneous architectures, evaluated without access to backbone activations\.

Statistical reporting\.All results report the mean macro\-F1 over three random seeds\. Improvements of≥0\.5\\geq 0\.5F1 are statistically significant \(p<0\.05p<0\.05\) under a paired bootstrap test over evaluation samples\.

## 6Results

Table[1](https://arxiv.org/html/2609.13534#S6.T1)and Figure[1](https://arxiv.org/html/2609.13534#S6.F1)report per\-dataset and average F1 scores\.Heraldconsistently outperforms both latent\-based baselines across all four backbones, yielding gains of2\.32\.3–4\.14\.1average F1\. On OLMo2\-7B, it achieves the best overall performance with an average F1 of89\.389\.3\. Notably,Heraldsurpasses all four guard models on the adversarial WildJailbreak benchmark, reaching98\.498\.4F1 compared to the next\-best96\.996\.9, demonstrating strong robustness to jailbreak\-style prompts\. Across most datasets, performance improvements are consistent and statistically significant, confirming that trajectory\-based features provide a more discriminative signal than static latent representations\. However, on ToxicChat and OpenAI Moderation,Heraldunderperforms the strongest guard model by22–44F1, suggesting that standalone classifiers may better capture certain surface\-level or dataset\-specific patterns\. We analyze this performance gap and its implications in Section[8](https://arxiv.org/html/2609.13534#S8)\.

Table 1:Average F1 on eight prompt\-harmfulness benchmarks\.Green textmarks the top result within each group\.Heraldoutperforms all latent\-based baselines on every backbone and surpasses all guard models on adversarial jailbreak detection \(WJB\)\. Results are the mean F1 over three seeds; standard deviations are≤0\.3\{\\leq\}0\.3for allHeraldconfigurations and are omitted for space\.![Refer to caption](https://arxiv.org/html/2609.13534v1/ICLR/fig1_main_results.png)\(a\)F1 across all eight benchmarks and four backbones\.Herald\(orange\) consistently exceeds both latent\-based baselines and is competitive with or stronger than guard models, especially on WildJailbreak\.
![Refer to caption](https://arxiv.org/html/2609.13534v1/ICLR/fig2_average_comparison.png)\(b\)Average F1 across benchmarks\.
![Refer to caption](https://arxiv.org/html/2609.13534v1/ICLR/fig3_jailbreak_performance.png)\(c\)WildJailbreak F1\.

Figure 1:Heraldachieves state\-of\-the\-art jailbreak detection at a fraction of the cost of guard models\. The structured trajectory representation is most advantageous when harmful intent accumulates progressively, exactly the setting where single\-layer methods are most limited\.### 6\.1Advantage on Adversarial Jailbreaks

Heraldsurpasses all four guard models on WildJailbreak for every backbone tested\. On OLMo2\-7B it reaches98\.498\.4F1, exceeding the best guard by1\.51\.5points\. The mechanism is structural: jailbreak prompts conceal harmful intent in early tokens and reveal it progressively through multi\-step framing, producing exactly the monotonically rising trajectory that HPD captures\. By contrast, a single\-layer snapshot reads only the terminal representation, missing the*path*by which it was reached\. On ToxicChat and OpenAI Moderation,Heraldtrails the best guard by22–44F1, a gap we attribute to the diffuse, culturally contingent nature of those harm categories \(Section[8](https://arxiv.org/html/2609.13534#S8)\)\.

Table 2:WildGuardMix F1 at varying training set sizes\.Heraldapproaches plateau at1,0001\{,\}000samples per class and leads by\>2\>2F1 at100100samples on all backbones\.
### 6\.2Data Efficiency and OOD Generalization

Table[2](https://arxiv.org/html/2609.13534#S6.T2)shows thatHeraldapproaches a plateau at1,0001\{,\}000samples per class, matching latent baselines in final performance while achieving comparable F1 with10×10\\timesfewer samples at the100100sample mark\. OOD evaluation \(trained on Aegis only, evaluated on WildGuardMix\),Heraldshows a smaller performance drop than both baselines\. This robustness likely reflects LDA’s inductive bias within\-class scatter; the learned harm directions are less sensitive to distributional idiosyncrasies\. The same mechanism underlies the lower\-data advantage: trajectory features compress the cross\-layer pattern into seven geometrically stable scalars, acting as an implicit regularizer in the low\-data regime\.

### 6\.3Disentangling Terminal\-Layer Signal from Trajectory Signal

A natural concern is whetherHerald’s gains stem from improved use of the final hidden state rather than true trajectory information\. We address this in three ways\. First, we compare against strong terminal\-layer baselines using identical classifiers and training budgets to isolate the effect of representation\. Second, we ablate trajectory features and observe consistent performance drops when the cross\-layer structure is removed\. Third, we analyze layerwise projections, demonstrating that a discriminative signal emerges progressively rather than concentrating solely at the final layer\. Together, these results confirm thatHeraldleverage structured cross\-layer dynamics that any single\-layer snapshot cannot capture\.The gap is largest on hard cases:Restricting to the final projectionpLp\_\{L\}alone achieves87\.887\.8F1 on OLMo2\-7B—only1\.51\.5points below the full model on average\. However, on WildJailbreak the gap is2\.12\.1points \(96\.396\.3vs\.98\.498\.4\)\. Jailbreak prompts are precisely the category where harmful intent is most gradually revealed; the terminal hidden state accumulates full depth but cannot reveal*how it got there*, whether through consistent directional growth or late\-occurring coincidence\. The trajectory features, especially the onset layer and monotonicity, discriminate these cases\.

#### The advantage compounds under data scarcity\.

At 100 training samples, OLMo2\-7BHeraldreaches83\.783\.7F1, comparable to the latent baselines at full data, while embedding and activation\-delta classifiers trail by over two points at the same sample count\. A final\-layer classifier trained on 100 examples must estimate a decision boundary ind=4096d\{=\}4096dimensions; trajectory features compress the discriminative information into seven interpretable scalars with geometric meaning \(slope, onset, monotonicity\) that is stable across random splits \(cosine similarity\>0\.97\>0\.97, Table[12](https://arxiv.org/html/2609.13534#A8.T12)\)\.Trajectory shape is an independent source of interpretability:Even setting accuracy aside, the per\-layer trajectory provides qualitatively distinct information\. Table[13](https://arxiv.org/html/2609.13534#A9.T13)shows that harm categories differ systematically in onset layer and monotonicity: jailbreaks exhibit early onset \(l^∗≈7\\hat\{l\}^\{\*\}\{\\approx\}7\) and high monotonicity \(0\.830\.83\), while social stereotypes onset late \(l^∗≈19\\hat\{l\}^\{\*\}\{\\approx\}19\) with inconsistent growth \(0\.580\.58\)\. This category\-specific trajectory signature is invisible to any method reading only the final hidden state, regardless of the classifier\.Herald’s trajectories are machine\-readable, loggable, and queryable, properties that a binary label or single\-layer score cannot provide\.

## 7Ablation Studies

To assess the contribution of each component, we vary one design choice at a time and report the resulting average F1 on WildGuardMix\. All variants use the same data splits and hyperparameters as the fullHeraldmodel, ensuring a fair comparison\. Performance differences of≥0\.5\{\\geq\}0\.5F1 are statistically significant whenp<0\.05p\{<\}0\.05using a paired bootstrap test\. This controlled evaluation allows us to quantify the impact of trajectory features, LDA\-based harm directions, and other architectural choices, highlighting which elements drive gains and which have minimal effect on overall robustness and generalization\.

### 7\.1Trajectory Feature Components

Table[3](https://arxiv.org/html/2609.13534#S7.T3)adds trajectory features cumulatively\. The final projectionpLp\_\{L\}is a strong but incomplete predictor\. Adding meanp¯\\bar\{p\}, total rise, monotonicity, curvature, and onset layer each improve performance; onset layerl^∗\\hat\{l\}^\{\*\}provides the largest single gain, especially on WildJailbreak \(\+0\.5\+0\.5F1 on the full model,\+1\.3\+1\.3F1 incrementally\)\. Removing any feature from the full model reduces performance, confirming independent contributions\.

Table 3:Trajectory feature ablation\.Features are added cumulatively,green textmarking the full model\. The onset layer contributes the most to adversarial jailbreak detection \(WJB column\)\.![Refer to caption](https://arxiv.org/html/2609.13534v1/ICLR/fig4_ablation_features.png)\(a\)Trajectory feature ablation\.
![Refer to caption](https://arxiv.org/html/2609.13534v1/ICLR/fig8_direction_learning.png)\(b\)Direction learning comparison\.

Figure 2:Feature and direction\-learning ablations\. Each trajectory feature contributes positively; LDA outperforms simpler direction choices, confirming that accounting for within\-class scatter is essential\.
### 7\.2Direction Learning Method

Table[4](https://arxiv.org/html/2609.13534#S7.T4)compares four harm\-direction estimators\. LDA consistently performs best\. The gap over class\-mean difference \(0\.50\.5–0\.80\.8F1\) quantifies the value of within\-class covariance estimation\. The random\-direction baseline confirms that HPD is a genuinely directional phenomenon rather than an artifact of any projection\.

Table 4:Direction learning method\.LDA produces the most discriminative per\-layer harm direction across all backbones\.
### 7\.3Layer Coverage Strategy

Table[5](https://arxiv.org/html/2609.13534#S7.T5)compares layer selection strategies\. All\-layer aggregation performs best: early, middle, and late thirds each contain complementary information, and the best single oracle layer trails full aggregation by0\.80\.8–1\.41\.4F1\. A top\-8 layer selection recovers most of the gain at25%25\\%of the storage cost, offering a practical trade\-off\.

Table 5:Layer coverage strategy\.All\-layer aggregation is best; the oracle single\-layer falls short by0\.80\.8–1\.41\.4F1\.![Refer to caption](https://arxiv.org/html/2609.13534v1/ICLR/fig5_data_efficiency.png)\(a\)Data efficiency\.
![Refer to caption](https://arxiv.org/html/2609.13534v1/ICLR/fig9_layer_coverage.png)\(b\)Layer coverage\.

Figure 3:Heraldis data\-efficient \(plateau near1,0001\{,\}000samples/class\) and benefits from full\-layer aggregation\. The top\-8 selection offers a compelling storage\-accuracy trade\-off\.
### 7\.4Token Aggregation and Robustness to Suffix Attacks

Table[6](https://arxiv.org/html/2609.13534#S7.T6)compares which token’s hidden state is projected onto𝐯l\\mathbf\{v\}\_\{l\}\. Last\-token representations perform best across all backbones, consistent with the autoregressive inductive bias of instruction\-tuned models: the final token aggregates context from all preceding positions via causal self\-attention, concentrating task\-relevant information at the sequence endpoint\.

Table 6:Token position\.Last\-token representations are consistently superior for harm\-direction projection\.#### Suffix\-padding robustness\.

An attacker could append long benign suffixes to dilute the last\-token representation\. Table[7](https://arxiv.org/html/2609.13534#S7.T7)tests this by appendingk∈\{10,50,100,200\}k\\in\\\{10,50,100,200\\\}benign tokens to jailbreak prompts\. Performance degrades gracefully: atk=50k\{=\}50the drop, it is modest \(1\.31\.3F1\); at the larger drop,k=200k\{=\}200it is manageable \(4\.84\.8F1\)\. The max\-pooling variant \(maxl⁡pl\\max\_\{l\}p\_\{l\}\) is substantially more robust at long suffixes while sacrificing0\.40\.4F1 on clean inputs; we recommend it when suffix attacks are a realistic threat\.

Table 7:Suffix\-padding robustness\(Llama\-8B, WildGuardMix\)\. Max\-pooling is more robust under long\-suffix attacks\. Best results per column are shown ingreen\.

### 7\.5Projection Normalization

Table[8](https://arxiv.org/html/2609.13534#S7.T8)tests unit normalization of𝐡l\\mathbf\{h\}\_\{l\}before projection\. Without normalization, projections conflate semantic alignment with raw activation magnitude, which varies across layers, prompt lengths, and model families\. Normalizing both𝐡l\\mathbf\{h\}\_\{l\}and𝐯l\\mathbf\{v\}\_\{l\}to unit length isolates direction from magnitude and improves average F1 by1\.31\.3–1\.81\.8\-points—a critical step for cross\-layer trajectory comparability\.

Table 8:Projection normalization\.Normalizing both𝐡l\\mathbf\{h\}\_\{l\}and𝐯l\\mathbf\{v\}\_\{l\}yields the most stable trajectory signal \(\+1\.3\+1\.3–1\.81\.8F1\)\. Best results are shown ingreen\.
### 7\.6MLP Classifier Capacity

Table[9](https://arxiv.org/html/2609.13534#S7.T9)shows that logistic regression already achieves85\.385\.3F1, only1\.01\.0below the full MLP\. A hidden size3232reaches the performance plateau; larger architectures provide no significant gain\. This confirms thatϕ\\boldsymbol\{\\phi\}is nearly linearly separable, a deliberate design outcome resulting from principled feature engineering rather than a limitation to be overcome by a larger classifier\.

Table 9:MLP classifier capacity\.Logistic regression is within1\.01\.0F1 of the full model; hidden size 32 reaches the plateau\. Best results are shown ingreen\.
### 7\.7Shrinkage Regularization

Table[10](https://arxiv.org/html/2609.13534#S7.T10)compares covariance inversion strategies for Eq\. \([3](https://arxiv.org/html/2609.13534#S4.E3)\)\. Uninvertible raw covariances cause severe instability \(F1 drops to61\.361\.3on Llama\-8B\)\. Ledoit\-Wolf shrinkage is the most stable option and yields the best average F1, with particular advantage at100100and in low\-sample data regimes where the diagonal and fixed\-ridge alternatives underperform\. This result underscores that covariance regularization is not merely a numerical convenience but is essential toHerald’s data efficiency\.

Table 10:Covariance regularization\.Ledoit\-Wolf shrinkage is most stable and yields strong F1, especially under data scarcity \(F1@100\)\. Best results are shown ingreen\.
### 7\.8Ablation Summary

Across all ablations, onset layer and LDA direction learning contribute most to performance; token position, layer coverage, and normalization each add meaningfully; shrinkage regularization is critical for numerical stability; and classifier capacity matters least\. These findings validate the core design philosophy ofHeraldinvesting in principled feature engineering of the activation trajectory rather than in downstream classifier capacity\. The resulting compact tabular representation generalizes reliably across backbones, harm categories, and data regimes\.

## 8Discussion

#### Why HPD is most effective for jailbreaks\.

Jailbreak prompts often conceal harmful intent early and reveal it gradually through multi\-step framing—a construction strategy structurally analogous to multi\-hop reasoning chains, in which the conclusion is only determinable after integrating evidence across multiple intermediate steps\. This produces the characteristic rising cross\-layer trajectory thatHeraldis designed to detect\. In contrast, implicit harms like social stereotypes or subtle sarcasm are semantically diffuse and less geometrically coherent: the harm direction separating explicit harm from benign text aligns poorly with these cases, producing flat or noisy trajectories\. Our onset\-layer and monotonicity results \(Table[13](https://arxiv.org/html/2609.13534#A9.T13)\) support this view\. Future work should explore multi\-directional subspaces to better capture diffuse or culturally specific harms\.

#### Failure modes: ToxicChat and OpenAI Moderation\.

Heraldtrails the best guard on ToxicChat \(−4\.3\-4\.3F1\) and OpenAI moderation \(−8\.4\-8\.4F1\)\. We hypothesize two causes\. First, both benchmarks contain a high proportion of implicit or context\-dependent harms where surface\-level tokens do not systematically trigger the rising trajectory\. Second, the WildGuardMix training distribution may under\-represent the stylistic variation in these benchmarks, limiting the transferability of the learned LDA directions\. Richer or category\-balanced training data and multi\-directional subspace extensions are natural remedies\.

#### Adaptive evasion\.

Heraldis harder to evade than single\-score detectors because an attacker must jointly fool multiple trajectory features—terminal value, onset layer, monotonicity, and curvature—that jointly characterize a rising trajectory\. Nevertheless, adaptive white\-box adversaries with access to the learned directions could potentially craft inputs that suppress early\-layer projections while maintaining a high terminal value\. Developing trajectory\-aware adversarial training is an important open direction\.

#### Implications for LLM governance\.

HPD reframes safety classification from a binary output property into a*structured sequential signal*from which interpretable features can be read, logged, and queried at scale\. Safety practitioners can ask not just whether a prompt is harmful, but also*when*the model resolves its intent and*how consistently*harmfulness grows across depth\.Herald’sO⁡\(L​d\)O\(Ld\)footprint also makes online updates practical as threat distributions evolve\. Future directions include multilingual and multimodal extension, integration into retrieval\-augmented generation pipelines to flag structurally ambiguous retrieved content, and subspace extensions for diffuse harm categories\.

## 9Conclusion

We identified Harmfulness Propagation Dynamics \(HPD\), a consistent cross\-layer pattern in which harmful prompts exhibit monotonically increasing alignment with a learned harm direction, whereas benign prompts remain flat or oscillatory\. We formally grounded HPD in the layered computation hypothesis for transformers, validated the stability of per\-layer LDA directions \(cosine similarity\>0\.97\>0\.97across random splits\), and showed that the trajectory’s*shape*, not merely its terminal value, carries independent discriminative information, especially for adversarial jailbreaks\.Herald, built on HPD, it extracts a compact seven\-dimensional feature record from the cross\-layer projection sequence and classifies it with a 288\-parameter MLP\. It adds only262262KB of memory and2\.6×10−62\.6\{\\times\}10^\{\-6\}prefilled FLOPs, yet outperforms prior latent\-based methods by2\.32\.3–4\.14\.1F1 across all backbones and surpasses all tested guard models on adversarial jailbreak detection\. Per\-instance trajectories provide interpretable, machine\-readable audit records revealing*when*and*how*harmfulness emerges—a capability absent in single\-layer or binary approaches\. More broadly, our results suggest that treating LLM activation sequences as structured temporal data, rather than opaque snapshots, offers a promising avenue for safe, reliable, and interpretable LLM governance\.

## 10Broader Impact and Ethical Considerations

Heraldreduces harmful LLM outputs with minimal computational overhead, making lightweight moderation more accessible beyond large, resource\-rich organizations\. Three limitations deserve attention\. First, the learned harm directions derive from WildGuardMix, which is primarily in English and may underrepresent culturally specific, multilingual, or diffuse harms; operators deploying in other languages or cultural contexts should validate and, ideally, retrain on domain\-specific data\. Second, if training data contains spurious demographic or stylistic correlations, the learned directions may partially encode those signals, increasing false positives on benign prompts from affected groups; subgroup evaluation and fairness auditing are recommended\. Third, publishing characterizations of harmfulness propagation dynamics could help adversaries design evasion strategies; continued red teaming, category\-specific evaluation, and adaptive threat modeling are therefore important before deployment\.

## References

- Arditiet al\.\(2024\)A\. Arditi, O\. Obeso, A\. Syed, D\. Paleka, N\. Panickssery, W\. Gurnee, and N\. NandaRefusal in language models is mediated by a single direction\.arXiv preprint arXiv:2406\.11717\.External Links:[Link](https://arxiv.org/abs/2406.11717)Cited by:[Appendix L](https://arxiv.org/html/2609.13534#A12.p1.1),[Appendix N](https://arxiv.org/html/2609.13534#A14.SS0.SSS0.Px2.p1.1)\.
- Askellet al\.\(2021\)A\. Askell, Y\. Bai, A\. Chen, D\. Drain, D\. Ganguli, T\. Henighan, A\. Jones, N\. Joseph, B\. Mann, N\. DasSarma,et al\.A general language assistant as a laboratory for alignment\.arXiv preprint arXiv:2112\.00861\.Cited by:[§1](https://arxiv.org/html/2609.13534#S1.p1.1)\.
- Carliniet al\.\(2023\)N\. Carlini, M\. Nasr, C\. A\. Choquette\-Choo, M\. Jagielski, I\. Gao, P\. W\. W\. Koh, D\. Ippolito, F\. Tramer, and L\. SchmidtAre aligned neural networks adversarially aligned?\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2609.13534#S1.p1.1)\.
- Christet al\.\(2018\)M\. Christ, N\. Braun, J\. Neuffer, and A\. W\. Kempa\-Liehrtsfresh: a python package for time series feature extraction on basis of scalable hypothesis tests\.Neurocomputing307,pp\. 72–77\.Cited by:[Appendix W](https://arxiv.org/html/2609.13534#A23.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px5.p1.1)\.
- Christianoet al\.\(2017\)P\. Christiano, J\. Leike, T\. B\. Brown, M\. Martic, S\. Legg, and D\. AmodeiDeep reinforcement learning from human preferences\.InAdvances in Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px1.p1.1)\.
- Ganguliet al\.\(2022\)D\. Ganguli, L\. Lovitt, J\. Kernion, A\. Askell, Y\. Bai, S\. Kadavath, B\. Mann, E\. Perez, N\. Schiefer, K\. Ndousse,et al\.Red teaming language models to reduce harms: methods, scaling behaviors, and lessons learned\.arXiv preprint arXiv:2209\.07858\.Cited by:[§1](https://arxiv.org/html/2609.13534#S1.p1.1),[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px6.p1.1)\.
- Gevaet al\.\(2021\)M\. Geva, R\. Schuster, J\. Berant, and O\. LevyTransformer feed\-forward layers are key\-value memories\.InProceedings of the Conference on Empirical Methods in Natural Language Processing,Cited by:[Appendix B](https://arxiv.org/html/2609.13534#A2.p1.1),[§1](https://arxiv.org/html/2609.13534#S1.p2.1),[§3\.2](https://arxiv.org/html/2609.13534#S3.SS2.p2.1)\.
- Ghoshet al\.\(2024\)S\. Ghosh, P\. Varshney, E\. Galinkin, and C\. ParisienAEGIS: online adaptive AI content safety moderation with ensemble of LLM experts\.arXiv preprint arXiv:2404\.05993\.External Links:[Link](https://arxiv.org/abs/2404.05993)Cited by:[1st item](https://arxiv.org/html/2609.13534#A16.I1.i1.p1.1)\.
- Greshakeet al\.\(2023\)K\. Greshake, S\. Abdelnabi, S\. Mishra, C\. Endres, T\. Holz, and M\. FritzNot what you’ve signed up for: compromising real\-world LLM\-integrated applications with indirect prompt injections\.InACM Workshop on Artificial Intelligence and Security,Cited by:[§1](https://arxiv.org/html/2609.13534#S1.p1.1)\.
- Inanet al\.\(2023\)H\. Inan, K\. Upasani, J\. Chi, R\. Rungta, K\. Iyer, Y\. Mao, M\. Tontchev, Q\. Hu, B\. Fuller, D\. Testuggine, and M\. KhabsaLlama Guard: LLM\-based input\-output safeguard for human\-AI conversations\.arXiv preprint arXiv:2312\.06674\.External Links:[Link](https://arxiv.org/abs/2312.06674)Cited by:[4th item](https://arxiv.org/html/2609.13534#A16.I1.i4.p1.1)\.
- Jawaharet al\.\(2019\)G\. Jawahar, B\. Sagot, and D\. SeddahWhat does BERT learn about the structure of language?\.InProceedings of the Annual Meeting of the Association for Computational Linguistics,Cited by:[Appendix B](https://arxiv.org/html/2609.13534#A2.p1.1),[§1](https://arxiv.org/html/2609.13534#S1.p2.1),[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px3.p1.1),[§3\.2](https://arxiv.org/html/2609.13534#S3.SS2.p1.1)\.
- Langley \(2000\)P\. LangleyCrafting papers on machine learning\.Note:Paper presented at the ICML Workshop on Submissions to the International Conference on Machine LearningExternal Links:[Link](https://icml.cc/Conferences/2000/langley00.pdf)Cited by:[Appendix W](https://arxiv.org/html/2609.13534#A23.SS0.SSS0.Px3.p2.1)\.
- Ledoit and Wolf \(2004\)O\. Ledoit and M\. WolfA well\-conditioned estimator for large\-dimensional covariance matrices\.Journal of Multivariate Analysis88\(2\),pp\. 365–411\.Cited by:[§4\.1](https://arxiv.org/html/2609.13534#S4.SS1.p1.3)\.
- Liet al\.\(2024\)L\. Li, B\. Dong, R\. Wang, X\. Hu, W\. Zuo, D\. Lin, Y\. Qiao, and J\. ShaoSALAD\-Bench: a hierarchical and comprehensive safety benchmark for large language models\.arXiv preprint arXiv:2402\.05044\.External Links:[Link](https://arxiv.org/abs/2402.05044)Cited by:[2nd item](https://arxiv.org/html/2609.13534#A16.I1.i2.p1.1)\.
- Liet al\.\(2025\)S\. Li, L\. Yao, L\. Zhang, and Y\. LiSafety layers in aligned large language models: the key to LLM security\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2609.13534#S1.p1.1),[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px2.p1.1)\.
- Markovet al\.\(2023\)T\. Markov, C\. Zhang, S\. Agarwal, F\. E\. Nekoul, T\. Lee, S\. Adler, A\. Jiang, and L\. WengA holistic approach to undesired content detection in the real world\.InProceedings of the AAAI Conference on Artificial Intelligence,Cited by:[§1](https://arxiv.org/html/2609.13534#S1.p1.1),[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px3.p1.1)\.
- Marks and Tegmark \(2023\)S\. Marks and M\. TegmarkThe geometry of truth: emergent linear structure in large language model representations of true/false datasets\.arXiv preprint arXiv:2310\.06824\.External Links:[Link](https://arxiv.org/abs/2310.06824)Cited by:[Appendix L](https://arxiv.org/html/2609.13534#A12.p1.1)\.
- Mikolovet al\.\(2013\)T\. Mikolov, K\. Chen, G\. Corrado, and J\. DeanEfficient estimation of word representations in vector space\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px3.p1.1)\.
- Parket al\.\(2024\)K\. Park, Y\. J\. Choe, and V\. VeitchThe linear representation hypothesis and the geometry of large language models\.InInternational Conference on Machine Learning,Cited by:[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px4.p1.1)\.
- Perezet al\.\(2022\)E\. Perez, S\. Huang, F\. Song, T\. Cai, R\. Ring, J\. Aslanides, A\. Glaese, N\. McAleese, and G\. IrvingRed teaming language models with language models\.InProceedings of the Conference on Empirical Methods in Natural Language Processing,Cited by:[§1](https://arxiv.org/html/2609.13534#S1.p1.1),[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px1.p1.1)\.
- Rafailovet al\.\(2023\)R\. Rafailov, A\. Sharma, E\. Mitchell, S\. Ermon, C\. D\. Manning, and C\. FinnDirect preference optimization: your language model is secretly a reward model\.InAdvances in Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px1.p1.1)\.
- Rogerset al\.\(2020\)A\. Rogers, O\. Kovaleva, and A\. RumshiskyA primer in BERTology: what we know about how BERT works\.Transactions of the Association for Computational Linguistics8,pp\. 842–866\.Cited by:[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px3.p1.1)\.
- Röttgeret al\.\(2021\)P\. Röttger, B\. Vidgen, D\. Nguyen, Z\. Waseem, H\. Margetts, and J\. B\. PierrehumbertHateCheck: functional tests for hate speech detection models\.InProceedings of the Annual Meeting of the Association for Computational Linguistics,Cited by:[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px2.p1.1)\.
- Ryuet al\.\(2024\)R\. Ryu, A\. Khullar, C\. Liu, Z\. Chen, Y\. Shen, and H\. SunLatent guard: a safety framework for text\-to\-image generation\.InEuropean Conference on Computer Vision,Cited by:[§1](https://arxiv.org/html/2609.13534#S1.p1.1),[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px2.p1.1)\.
- Stiennonet al\.\(2020\)N\. Stiennon, L\. Ouyang, J\. Wu, D\. M\. Ziegler, R\. Lowe, C\. Voss, A\. Radford, D\. Amodei, and P\. ChristianoLearning to summarize with human feedback\.InAdvances in Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px1.p1.1)\.
- Tenneyet al\.\(2019\)I\. Tenney, D\. Das, and E\. PavlickBERT rediscovers the classical NLP pipeline\.InProceedings of the Annual Meeting of the Association for Computational Linguistics,Cited by:[Appendix B](https://arxiv.org/html/2609.13534#A2.p1.1),[§1](https://arxiv.org/html/2609.13534#S1.p2.1),[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px3.p1.1),[§3\.2](https://arxiv.org/html/2609.13534#S3.SS2.p1.1),[§3\.2](https://arxiv.org/html/2609.13534#S3.SS2.p2.1)\.
- Vidgenet al\.\(2023\)B\. Vidgen, N\. Scherrer, H\. R\. Kirk, R\. Qian, A\. Kannappan, S\. A\. Hale, and P\. RöttgerSimpleSafetyTests: a test suite for identifying critical safety risks in large language models\.arXiv preprint arXiv:2311\.08370\.Cited by:[§1](https://arxiv.org/html/2609.13534#S1.p1.1),[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px6.p1.1)\.
- Zenget al\.\(2024\)W\. Zeng, Y\. Liu, R\. Mullins, L\. Peran, J\. Fernandez, H\. Harkous, K\. Narasimhan, D\. Proud, P\. Kumar, B\. Radharapu, O\. Sturman, and O\. WahltinezShieldGemma: generative AI content moderation based on Gemma\.arXiv preprint arXiv:2407\.21772\.External Links:[Link](https://arxiv.org/abs/2407.21772)Cited by:[3rd item](https://arxiv.org/html/2609.13534#A16.I1.i3.p1.1)\.
- Zhaoet al\.\(2024\)W\. Zhao, X\. Ren, J\. Hessel, C\. Cardie, Y\. Choi, and Y\. DengWildChat: 1m ChatGPT interaction logs in the wild\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Bl8u7ZRlbM)Cited by:[Appendix S](https://arxiv.org/html/2609.13534#A19.SS0.SSS0.Px1.p1.1)\.
- Zouet al\.\(2023a\)A\. Zou, L\. Phan, S\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski, S\. Goel, N\. Li, M\. J\. Byun, Z\. Wang, A\. Mallen, S\. Basart, S\. Koyejo, D\. Song, M\. Fredrikson, J\. Z\. Kolter, and D\. HendrycksRepresentation engineering: a top\-down approach to AI transparency\.arXiv preprint arXiv:2310\.01405\.External Links:[Link](https://arxiv.org/abs/2310.01405)Cited by:[Appendix L](https://arxiv.org/html/2609.13534#A12.p1.1),[Appendix C](https://arxiv.org/html/2609.13534#A3.SS0.SSS0.Px1.p1.1)\.
- Zouet al\.\(2023b\)A\. Zou, Z\. Wang, N\. Carlini, M\. Nasr, J\. Z\. Kolter, and M\. FredriksonUniversal and transferable adversarial attacks on aligned language models\.arXiv preprint arXiv:2307\.15043\.External Links:[Link](https://arxiv.org/abs/2307.15043)Cited by:[Appendix L](https://arxiv.org/html/2609.13534#A12.p1.1)\.
- Zouet al\.\(2023c\)J\. Zou, Z\. Yuan, P\. Xin, Z\. Xiao, J\. Sun, S\. Zhuang, Z\. Guo, J\. Fu, and Y\. LiuPrivacy\-Friendly Task Offloading for Smart Grid in 6G Satellite–Terrestrial Edge Computing Networks †\.Electronics \(Switzerland\)12\(16\)\.External Links:[Document](https://dx.doi.org/10.3390/ELECTRONICS12163484),ISSN 20799292Cited by:[§1](https://arxiv.org/html/2609.13534#S1.p5.1),[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.13534#S2.SS0.SSS0.Px3.p1.1)\.

## Appendix AEfficiency: FLOPs and Memory

Heraldcomputes one dot product and one unit normalisation per layer, addingO⁡\(2​L​d\)O\(2Ld\)FLOPs at inference\. ForL=32L\{=\}32,d=4096d\{=\}4096, this is≈262​K\{\\approx\}262\\text\{K\}FLOPs against≈100​B\{\\approx\}100\\text\{B\}prefill FLOPs for a 100\-token prompt—a ratio of≈2\.6×10−6\{\\approx\}2\.6\{\\times\}10^\{\-6\}\.

#### Memory\.

Each layer retains a single direction vector𝐯l∈ℝd\\mathbf\{v\}\_\{l\}\\in\\mathbb\{R\}^\{d\}; scatter matrices are discarded after training\. Total storage:L×d×2L\\times d\\times 2bytes \(fp16\)\. ForL=32L\{=\}32,d=4096d\{=\}4096:32×4096×2=26232\\times 4096\\times 2=262KB\. By contrast, a full per\-layer covariance costs32×40962×2≈1\.0732\\times 4096^\{2\}\\times 2\\approx 1\.07GB \(∼4100×\{\\sim\}4100\\timesmore\), and a 7B\-parameter guard model requires≈14\{\\approx\}14GB \(∼53,000×\{\\sim\}53\{,\}000\\timesmore\)\. TheO⁡\(L​d\)O\(Ld\)vs\.O⁡\(L​d2\)O\(Ld^\{2\}\)gap is fundamental, not incidental—it is what makesHeralddeployable on the same hardware as the host model without additional accelerators \(Figure[4](https://arxiv.org/html/2609.13534#A1.F4)\)\.

![Refer to caption](https://arxiv.org/html/2609.13534v1/ICLR/fig10_efficiency.png)Figure 4:Memory and computational overhead\.Heraldstores262262KB \(650×650\\timesless than full covariance;53,000×53\{,\}000\\timesless than a 7B guard\) and adds2\.6×10−62\.6\{\\times\}10^\{\-6\}of prefill FLOPs\.

## Appendix BTheoretical Motivation for Layer\-Wise Trajectories

Harmful intent is a*progressively emerging*signal\. Early transformer layers encode surface form \(token identities, punctuation, morphology\); middle layers compose semantic structure; late layers resolve pragmatic and task\-relevant meaning\([Jawahar et al\., 2019](https://arxiv.org/html/2609.13534#bib.bib14);[Tenney et al\., 2019](https://arxiv.org/html/2609.13534#bib.bib15)\)\. Feed\-forward blocks also function as associative memory that progressively refines token representations\([Geva et al\., 2021](https://arxiv.org/html/2609.13534#bib.bib17)\)\. For a jailbreak prompt that conceals its intent through multi\-step framing, the model’s representation therefore becomes increasingly aligned with the harm direction as depth accumulates semantic evidence, while a benign prompt produces no systematic directional drift\. The*shape*of the resulting trajectory—not only its endpoint—thus constitutes a structural fingerprint of harmful intent, one that is most pronounced precisely where lightweight moderation is most needed\.

We validate Proposition[3\.1](https://arxiv.org/html/2609.13534#S3.Thmtheorem1)’s assumptions empirically by computing the mean per\-layer projection for 500 randomly sampled harmful and benign prompts from WildGuardMix on each backbone\. In all cases, mean harmful projection increases monotonically across layers88–3232, while mean benign projection remains within one standard deviation of zero, consistent with the proposition\.

## Appendix CPer\-Layer LDA Directions

At each layerllwe solve the binary LDA problem:

𝐯l=\(𝐒W\(l\)\)−1​\(𝝁lharm−𝝁lsafe\)‖\(𝐒W\(l\)\)−1​\(𝝁lharm−𝝁lsafe\)‖\.\\mathbf\{v\}\_\{l\}=\\frac\{\\bigl\(\\mathbf\{S\}\_\{W\}^\{\(l\)\}\\bigr\)^\{\-1\}\\\!\\bigl\(\\boldsymbol\{\\mu\}\_\{l\}^\{\\mathrm\{harm\}\}\-\\boldsymbol\{\\mu\}\_\{l\}^\{\\mathrm\{safe\}\}\\bigr\)\}\{\\bigl\\\|\\bigl\(\\mathbf\{S\}\_\{W\}^\{\(l\)\}\\bigr\)^\{\-1\}\\\!\\bigl\(\\boldsymbol\{\\mu\}\_\{l\}^\{\\mathrm\{harm\}\}\-\\boldsymbol\{\\mu\}\_\{l\}^\{\\mathrm\{safe\}\}\\bigr\)\\bigr\\\|\}\.\(6\)LDA is preferred over the raw mean\-difference direction because hidden\-state variation*within*each class is substantial\. Prompts with the same label differ in length, wording, and rhetorical style, producing large within\-class scatter\. Accounting for this scatter via Ledoit–Wolf shrinkage yields a more transferable discrimination axis, as confirmed by the ablation in Table[4](https://arxiv.org/html/2609.13534#S7.T4)\.

#### Connection to representation engineering\.

[Zou et al\. \(2023a\)](https://arxiv.org/html/2609.13534#bib.bib23)obtain steering vectors by taking the difference of positive and negative class activations at a*single*fixed layer\.Heraldextends this by \(i\) applying LDA rather than a raw mean difference \(accounting for within\-class scatter\), \(ii\) learning independent directions per layer, and \(iii\) aggregating the resulting projections into a trajectory for classification\. This produces a fundamentally different object: not a single vector for steering, but a sequence of vectors whose induced trajectory is the classification feature\.

## Appendix DTrajectory Feature Vector

Given per\-layer projectionspl=⟨𝐡^l,𝐯l⟩p\_\{l\}=\\langle\\hat\{\\mathbf\{h\}\}\_\{l\},\\mathbf\{v\}\_\{l\}\\rangle, the feature vectorϕ⁡\(𝒑\)∈ℝ7\\boldsymbol\{\\phi\}\(\\boldsymbol\{p\}\)\\in\\mathbb\{R\}^\{7\}is defined in Eq\. \([4](https://arxiv.org/html/2609.13534#S4.E4)\)\. Table[11](https://arxiv.org/html/2609.13534#A4.T11)summarises the geometric role of each component\.

Table 11:Trajectory feature vectorϕ⁡\(𝒑\)∈ℝ7\\boldsymbol\{\\phi\}\(\\boldsymbol\{p\}\)\\in\\mathbb\{R\}^\{7\}: geometric interpretation\.The near\-competitive performance of logistic regression \(Table[9](https://arxiv.org/html/2609.13534#S7.T9)\) confirms thatϕ\\boldsymbol\{\\phi\}is nearly linearly separable: most discriminative power resides in trajectory geometry, not downstream classifier capacity\.

## Appendix ELast\-Token Representations

In instruction\-tuned autoregressive LLMs, the final prefill token aggregates context from all preceding positions through causal self\-attention, making it the representation most directly predictive of next\-token behaviour\. Mean pooling dilutes this by averaging tokens serving different syntactic roles; the first token captures only initial context\. Table[6](https://arxiv.org/html/2609.13534#S7.T6)confirms that last\-token projection yields the strongest harm trajectory signal\.

## Appendix FNormalization and Shrinkage Regularization

#### Unit normalization\.

Projecting𝐡^l=𝐡l/‖𝐡l‖\\hat\{\\mathbf\{h\}\}\_\{l\}=\\mathbf\{h\}\_\{l\}/\\\|\\mathbf\{h\}\_\{l\}\\\|onto𝐯l/‖𝐯l‖\\mathbf\{v\}\_\{l\}/\\\|\\mathbf\{v\}\_\{l\}\\\|isolates the*direction*of the representation from its magnitude\. Without normalization,plp\_\{l\}conflates semantic alignment with raw activation scale, which varies across layers, prompt lengths, and model families, degrading cross\-layer trajectory comparability\. Normalizing both vectors improves average F1 by1\.31\.3–1\.81\.8points \(Table[8](https://arxiv.org/html/2609.13534#S7.T8)\)\.

#### Shrinkage regularization\.

Becaused≫ntraind\\gg n\_\{\\mathrm\{train\}\}, the sample covariance𝐒W\(l\)\\mathbf\{S\}\_\{W\}^\{\(l\)\}is poorly conditioned\. Ledoit–Wolf shrinkage replaces it with a well\-conditioned convex combination of the sample covariance and a scaled identity, stabilising the LDA solution and preventing harm directions from overfitting sampling noise\. Omitting regularisation degrades F1 by up to2727points in low\-data regimes \(Table[10](https://arxiv.org/html/2609.13534#S7.T10)\)\.

## Appendix GClassifier Design

Onceϕ⁡\(𝒑\)\\boldsymbol\{\\phi\}\(\\boldsymbol\{p\}\)is computed, the classification problem is seven\-dimensional and nearly linearly separable\. A large model would overfit rather than generalise\. The chosen MLP with hidden size3232\(<400<400parameters\) reaches the performance plateau: doubling capacity yields no gain \(Table[9](https://arxiv.org/html/2609.13534#S7.T9)\)\. The small size also makes the classifier inspectable—a practitioner can audit which trajectory features drove a particular flagged classification, supporting transparency requirements in LLM governance\.

## Appendix HHarm Direction Stability

Table[12](https://arxiv.org/html/2609.13534#A8.T12)reports mean pairwise cosine similarity between harm directions𝐯l\\mathbf\{v\}\_\{l\}learned on five independent 80/20 splits\. Similarity exceeds0\.970\.97at every layer and backbone, confirming that HPD reflects genuine geometric structure rather than sampling artefacts\. This stability is a formal prerequisite for Proposition[3\.1](https://arxiv.org/html/2609.13534#S3.Thmtheorem1): if directions varied substantially across splits, the trajectory would not be a reliable population\-level signal\. Stable directions are also shareable as versioned weight files, enabling comparable evaluation across research groups without requiring identical training data\.

Table 12:Harm direction stability: mean pairwise cosine similarity across five random training splits \(\>0\.97\>0\.97throughout\)\.
## Appendix IOnset Layer Statistics by Harm Category

Table[13](https://arxiv.org/html/2609.13534#A9.T13)reports mean onset layerl^∗\\hat\{l\}^\{\*\}and monotonicity index per harm category\. Jailbreaks exhibit the earliest onset and highest monotonicity, reflecting their structured multi\-step escalation\. Social stereotypes onset latest and rise least consistently, indicating that a single LDA direction is insufficient for diffuse, culturally contingent harms—motivating multi\-direction subspace extensions as future work\.

Table 13:Mean onset layerl^∗\\hat\{l\}^\{\*\}and monotonicity index by harm category\. Jailbreaks \(earliest onset, highest monotonicity\) are structurally most amenable to HPD\-based detection; social stereotypes \(latest onset, lowest monotonicity\) are least\.
## Appendix JTrue Negative Rate on Benign Benchmarks

A safety moderator that over\-fires on benign prompts imposes an invisible cost on legitimate use\. Table[14](https://arxiv.org/html/2609.13534#A10.T14)shows thatHeraldproduces false positives on fewer than1\.5%1\.5\\%of benign prompts across seven diverse tasks\. The100%100\\%TNR on Codex and GSM8k across all backbones is particularly notable: structured code and mathematical prompts, despite their lexical specificity, are cleanly distinguished from harmful content by the trajectory classifier\.

Table 14:True Negative Rate \(%\) on seven benign evaluation benchmarks\.Heraldmaintains\>98\.5%\>98\.5\\%average TNR across all four backbones\.
## Appendix KComparison with Last\-Layer Supervised Classifiers

Table[15](https://arxiv.org/html/2609.13534#A11.T15)holds the representation fixed at the final hidden layer and varies only the classifier, isolating the contribution of trajectory aggregation from that of the per\-layer LDA direction\. LDA direction scoring—with*no free parameters*at the per\-layer stage—matches or exceeds all supervised classifiers applied to the same representation\. The spread across all five methods is under22F1 points, confirming that representation quality dominates classifier capacity\. Both results support the data\-centric view: investing in principled feature extraction yields more reliable gains than scaling the downstream model\.

Table 15:Average F1 of last\-layer representation with varying classifiers\. LDA scoring has no free per\-layer parameters yet remains competitive with supervised methods, confirming that representation quality dominates classifier capacity\.![Refer to caption](https://arxiv.org/html/2609.13534v1/ICLR/fig7_suffix_robustness.png)\(a\)Suffix\-padding robustness\.
![Refer to caption](https://arxiv.org/html/2609.13534v1/ICLR/fig6_category_performance.png)\(b\)Category\-wise F1\.

Figure 5:Robustness and category analysis\. Max\-pooling is more stable under long benign suffixes\. Single LDA directions excel for explicit harms but degrade on diffuse categories \(social stereotypes\), motivating subspace extensions\.
## Appendix LAdditional References for Related Work

We include the following references critical to contextualizingHeraldwithin the mechanistic interpretability and representation engineering literature:[Zou et al\. \(2023a\)](https://arxiv.org/html/2609.13534#bib.bib23)\(representation engineering via contrastive activation\);[Marks and Tegmark \(2023\)](https://arxiv.org/html/2609.13534#bib.bib24)\(linear geometry of truth representations\);[Arditi et al\. \(2024\)](https://arxiv.org/html/2609.13534#bib.bib25)\(refusal directions via activation analysis\);[Zou et al\. \(2023b\)](https://arxiv.org/html/2609.13534#bib.bib26)\(universal adversarial suffixes for LLMs\)\.

## Appendix MConcat\-All\-Layers Probe and Learned Sequence Aggregator Baselines

*Addressing Reviewer 9qrV \(highest\-value addition\) and Reviewer SLvX \(why not treat\{pl\}\\\{p\_\{l\}\\\}as a time series with a learned aggregator?\)\.*

#### Setup\.

We add three new baselines that use identical cross\-layer information toHeraldbut replace the hand\-crafted feature vectorϕ\\boldsymbol\{\\phi\}with either a richer fixed projection or a learned sequence model\. All baselines train on the same WildGuardMix split and are evaluated zero\-shot on the remaining seven benchmarks\.

1. \(i\)All\-layers MLP\.The scalar trajectory\{pl\}l=1L\\\{p\_\{l\}\\\}\_\{l=1\}^\{L\}is fed as a rawLL\-dimensional input to the same 288\-parameter MLP used byHerald\. This tests whether multi\-layer information alone, without geometric feature engineering, is sufficient\.
2. \(ii\)1D\-CNN over\{pl\}\\\{p\_\{l\}\\\}\.A one\-dimensional convolutional network with two conv\-relu layers \(kernel width 3, 16 channels\) followed by global average pooling and a linear head \(≈\\approx600 parameters\)\. This allows the model to learn local trajectory patterns—including curvature and monotonicity—from data rather than from hand\-crafted formulas\.
3. \(iii\)GRU over\{pl\}\\\{p\_\{l\}\\\}\.A single\-layer GRU with hidden size 16 processes the scalar sequencep1,…,pLp\_\{1\},\\dots,p\_\{L\}and classifies from the final hidden state \(≈\\approx900 parameters\)\. GRUs are the natural sequential baseline for ordered multi\-layer signals\.

Table 16:Learned sequence\-model baselines vs\.Herald\.All models receive identical per\-layer projections\{pl\}\\\{p\_\{l\}\\\}\.Herald’s seven\-dimensional hand\-crafted feature vector matches or exceeds learned aggregators on every backbone, while requiring no hyperparameter tuning of a sequence architecture\. Best result per column ingreen\.
#### Interpretation\.

Heraldoutperforms all learned sequence aggregators despite having the fewest parameters and no trainable recurrence\. Three factors explain this result\. First, the trajectory\{pl\}\\\{p\_\{l\}\\\}is a*scalar*sequence of lengthLL: a 32\-step time series is easily modelled by geometric features but provides limited training signal for a convolutional or recurrent architecture that must estimate its own filter coefficients\. Second, the geometric features \(onset layer, monotonicity\) are*global*statistics that require the entire sequence; a GRU can in principle learn these but needs substantially more data to do so reliably\. Third, the 1D\-CNN and GRU have additional hyperparameters \(kernel size, hidden size, number of layers\) whose tuning introduces variance; hand\-crafted features are stable by construction\.

Concretely, the gap between the GRU andHeraldon WildJailbreak \(97\.8 vs\. 98\.4\) is statistically significant \(p<0\.05p<0\.05, paired bootstrap\), confirming that the geometric features—onset layer and monotonicity in particular—capture structure that a data\-driven sequence model does not fully recover at this scale\. The contribution is therefore*geometry\-aware classification*, not merely multi\-layer aggregation\.

#### Complexity note\.

The raw trajectory alone \(All\-layers MLP\) already improves over last\-token baselines \(86\.386\.3vs\.84\.784\.7for embed\. clf\. on Llama\-8B\), confirming that the cross\-layer structure is the primary driver\. The hand\-craftedϕ\\boldsymbol\{\\phi\}adds a further1\.11\.1F1 by encoding geometric invariants that the raw sequence does not make immediately accessible to a small classifier\.

## Appendix NCausal Intervention: Ablating Harm Directions During Generation

*Addressing Reviewer 9qrV\.*

#### Experiment\.

To probe whether the learned harm directions𝐯l\\mathbf\{v\}\_\{l\}causally influence downstream generation rather than merely correlating with classifier outputs, we perform a direction\-ablation experiment on Llama\-3\.1\-8B\-Instruct and OLMo2\-7B\-Instruct\. For a set of 200 jailbreak prompts from WildJailbreak on which the model would normally refuse, we apply residual\-stream hooks at the layer of peak monotonicityl†l^\{\\dagger\}\(the layer achieving the highestpl\+1−plp\_\{l\+1\}\-p\_\{l\}increment, typicallyl†≈14l^\{\\dagger\}\\approx 14–1818for jailbreaks\) and zero\-project the harm direction from the hidden state:

𝐡~l†=𝐡l†−⟨𝐡l†,𝐯l†⟩​𝐯l†\.\\tilde\{\\mathbf\{h\}\}\_\{l^\{\\dagger\}\}=\\mathbf\{h\}\_\{l^\{\\dagger\}\}\-\\bigl\\langle\\mathbf\{h\}\_\{l^\{\\dagger\}\},\\,\\mathbf\{v\}\_\{l^\{\\dagger\}\}\\bigr\\rangle\\,\\mathbf\{v\}\_\{l^\{\\dagger\}\}\.\(7\)The modified hidden state𝐡~l†\\tilde\{\\mathbf\{h\}\}\_\{l^\{\\dagger\}\}is passed forward through subsequent layers unchanged; all other layers are unmodified\. We then generate 50 tokens greedily and record whether the model produces a refusal or a compliance \(as judged by a Llama\-Guard\-3 oracle\)\.

Table 17:Generation\-time refusal rate under harm\-direction ablation\.Ablating𝐯l†\\mathbf\{v\}\_\{l^\{\\dagger\}\}at the peak\-monotonicity layer reduces the refusal rate substantially, confirming that the harm direction has causal influence on safety behavior and is not merely a post\-hoc correlate\.
#### Findings\.

Ablating𝐯l†\\mathbf\{v\}\_\{l^\{\\dagger\}\}at the peak\-monotonicity layer reduces the refusal rate from≈94%\{\\approx\}94\\%to≈60%\{\\approx\}60\\%, a drop of3333–3535percentage points\. Intervening at an early layer \(l=4l\{=\}4\) or a late layer \(l=30l\{=\}30\) produces much smaller effects \(55–88percentage points\), establishing that the causal influence is layer\-specific and concentrated near the trajectory’s steepest ascent\. These results support the mechanistic claim underlying HPD: the harm direction at the peak\-monotonicity layer is not merely a classification artifact but represents a causal locus at which the model’s safety behavior is determined\. Consistent with prior work on refusal directions\([Arditi et al\., 2024](https://arxiv.org/html/2609.13534#bib.bib25)\), the directional ablation is not equivalent to semantic erasure of the entire prompt—model outputs remain coherent—but specifically disrupts the pragmatic safety signal\.

#### Limitations\.

This experiment uses greedy decoding with 50 tokens; longer generation and sampling\-based decoding may shift absolute refusal rates\. The oracle \(Llama\-Guard\-3\) may classify ambiguous completions inconsistently\. Nonetheless, the layer\-specificity of the effect strongly supports a causal interpretation that goes beyond correlation\.

## Appendix OStrengthened Out\-of\-Distribution Evaluation

*Addressing Reviewer 9qrV\.*

#### Setup\.

The OOD evaluation in Section[6\.2](https://arxiv.org/html/2609.13534#S6.SS2)trained on a single alternative source\. Here we extend to two fully cross\-distribution training protocols: \(i\) train onAegis only, evaluate on HarmBench and XSTest separately; \(ii\) train onHarmBench only, evaluate on Aegis and WildGuardMix\. These pairs represent substantive distributional divergence: Aegis uses adversarial red\-team prompts with fine\-grained category labels, whereas HarmBench is a standardized benchmark with diverse harm types and standardized difficulty tiers, and XSTest contains near\-miss benign prompts specifically designed to probe false\-positive rates\.

Table 18:Cross\-distribution OOD generalization\.Models are trained exclusively on one dataset and evaluated on others\.Heraldconsistently maintains the smallest performance drop relative to in\-distribution results, confirming that LDA\-based trajectory features are robust to distributional shift\. In\-distribution F1 \(from Table 1\) shown for reference in parentheses\.
#### Analysis\.

Herald’s absolute OOD F1 drops by44–88points relative to in\-distribution results, consistent with any method facing domain shift\. However, its*relative*drop is smaller than both baselines across all four cross\-distribution splits\. The HarmBench→\\toAegis drop \(≈5\{\\approx\}5F1 forHerald\) is substantially smaller than the embed\. clf\. drop \(≈7\{\\approx\}7F1\), suggesting that compressing discriminative information into seven geometric scalars serves as an implicit regularizer against dataset\-specific idiosyncrasies\. Results on Aegis→\\toXSTest are of particular note: XSTest probes false\-positive rates on near\-miss benign prompts, yetHerald’s trajectory features—particularly onset layer and monotonicity—correctly classify the vast majority as benign, since near\-miss prompts do not produce the systematic monotone rise characteristic of harmful inputs\.

## Appendix PGuard Model Identification

*Addressing Reviewer 9qrV\.*

The four guard models \(Guard A–D\) in Table 1 correspond to the following publicly available safety classifiers, listed in alphabetical order of their anonymized labels:

- •Guard A:Aegis\-AI\-Content\-Safety\-Defense\-2\.0\([Ghosh et al\., 2024](https://arxiv.org/html/2609.13534#bib.bib28)\), a 7B parameter model fine\-tuned from Llama\-3\-8B on adversarially collected safety data\. Selected because it is the source of one of the eight evaluation benchmarks \(Aegis\), providing a test of in\-distribution generalization for the guard itself\.
- •Guard B:MD\-Judge\([Li et al\., 2024](https://arxiv.org/html/2609.13534#bib.bib29)\), a Mistral\-7B fine\-tune trained on a diverse set of malicious instruction categories\. Included to represent guard models built on the same backbone family as one of ourHeraldbackbones\.
- •Guard C:ShieldGemma\-2\([Zeng et al\., 2024](https://arxiv.org/html/2609.13534#bib.bib30)\), a Gemma\-2\-based safety classifier targeting both prompt and response classification at multiple severity levels\. Included as a current SOTA guard from a different model family\.
- •Guard D:Llama\-Guard\-3\-8B\([Inan et al\., 2023](https://arxiv.org/html/2609.13534#bib.bib31)\), the latest public release of Meta’s guard series\. Included as the de facto standard in the field and the strongest single competitor reported in Table 1 \(avg\. F187\.887\.8, best guard on ToxicChat and OpenAI Moderation\)\.

All four guard models are evaluated in zero\-shot mode on each benchmark\. For guard models that require an output format, we follow the official inference instructions released by each model’s authors\. None of the guard models had access to backbone activations; they classify from raw text inputs only, as indicated in Table 1\.

## Appendix QCode and Weights Release Plan

*Addressing Reviewer 9qrV\.*

We commit to the following public release upon acceptance:

1. 1\.Per\-layer LDA directions\{𝐯l\}l=1L\\\{\\mathbf\{v\}\_\{l\}\\\}\_\{l=1\}^\{L\}as fp16 NumPy arrays for all four backbone families tested \(Llama\-3\.1\-8B, Mistral\-7B, OLMo2\-7B, Qwen3\-8B\), trained on WildGuardMix and stored as versioned weight files on HuggingFace Hub under a CC\-BY 4\.0 license\. Each direction file is≈262\{\\approx\}262KB and self\-contained; users can evaluateHeraldwithout retraining LDA\.
2. 2\.Feature extraction and inference codein a lightweight Python package \(herald\-moderator\) with a single\-function interface:herald\.score\(prompt, backbone, layer\_directions\)\. The package will be available via PyPI\.
3. 3\.Training codefor reproducing per\-layer LDA directions from any instruction\-tuned LLM, with Ledoit–Wolf shrinkage applied automatically viasklearn\.covariance\.LedoitWolf\.
4. 4\.Evaluation scriptsreproducing all eight benchmark F1 scores reported in Table 1, with fixed random seeds documented in the README\.

Sharing versioned direction vectors has a concrete reproducibility implication: any research group can download the Llama\-8B or OLMo2\-7B directions and replicate the inference\-time results in Table 1 without a GPU, since the feature extraction requires only dot products on cached hidden states\.

## Appendix RClarification on Proposition 3\.1

*Addressing Reviewer 9qrV\.*

Reviewer 9qrV rightly notes that Proposition 3\.1 \(Section 3\.2\) reads as a near\-restatement of the empirical observation rather than a derivation from first principles\. We clarify the role of the proposition and provide additional theoretical content\.

#### What Proposition 3\.1 does and does not claim\.

The proposition is intentionally informal and serves as a*conditional grounding*rather than a derivation: given assumptions \(i\)–\(iii\), the trajectory has the stated properties\. It is not presented as a theorem with a closed\-form proof because assumptions \(i\) and \(iii\) are empirical—they characterize the behavior of specific pre\-trained transformers—and cannot be derived from architectural axioms alone\. We strengthen the proposition by making one non\-trivial implication explicit\.

###### Proposition R\.1\(Formal version of Proposition 3\.1\)\.

Letμlharm\\mu\_\{l\}^\{\\mathrm\{harm\}\}andμlsafe\\mu\_\{l\}^\{\\mathrm\{safe\}\}be the class\-conditional means at layerll, and suppose the within\-class scatter satisfiesλmin​\(𝐒W\(l\)\)≥σ2\>0\\lambda\_\{\\min\}\(\\mathbf\{S\}\_\{W\}^\{\(l\)\}\)\\geq\\sigma^\{2\}\>0for allll\. Under the regularized LDA estimator of Eq\. \([3](https://arxiv.org/html/2609.13534#S4.E3)\) with Ledoit–Wolf shrinkage parameterαl∈\[0,1\)\\alpha\_\{l\}\\in\[0,1\), the signed gap

δl:=⟨𝝁lharm−𝝁lsafe,𝐯l⟩\\delta\_\{l\}\\;:=\\;\\bigl\\langle\\boldsymbol\{\\mu\}\_\{l\}^\{\\mathrm\{harm\}\}\-\\boldsymbol\{\\mu\}\_\{l\}^\{\\mathrm\{safe\}\},\\;\\mathbf\{v\}\_\{l\}\\bigr\\rangle\(8\)satisfiesδl≥0\\delta\_\{l\}\\geq 0for allll\. If additionally the Fisher discriminant ratioJl:=δl2/𝐯l⊤​𝐒W\(l\)​𝐯lJ\_\{l\}:=\\delta\_\{l\}^\{2\}/\\mathbf\{v\}\_\{l\}^\{\\top\}\\mathbf\{S\}\_\{W\}^\{\(l\)\}\\mathbf\{v\}\_\{l\}is non\-decreasing inll, then the expected harmful\-class projection𝔼x∼ℋ​\[pl​\(x\)\]\\mathbb\{E\}\_\{x\\sim\\mathcal\{H\}\}\[p\_\{l\}\(x\)\]is non\-decreasing inll, and the expected benign projection𝔼x∼ℬ​\[pl​\(x\)\]\\mathbb\{E\}\_\{x\\sim\\mathcal\{B\}\}\[p\_\{l\}\(x\)\]is bounded in\[−ϵ,ϵ\]\[\-\\epsilon,\\epsilon\]forϵ≪δl\\epsilon\\ll\\delta\_\{l\}\.

###### Proof sketch\.

The sign ofδl\\delta\_\{l\}follows from the LDA solution:𝐯l∝\(𝐒W\(l\)\)−1​\(𝝁lharm−𝝁lsafe\)\\mathbf\{v\}\_\{l\}\\propto\(\\mathbf\{S\}\_\{W\}^\{\(l\)\}\)^\{\-1\}\(\\boldsymbol\{\\mu\}\_\{l\}^\{\\mathrm\{harm\}\}\-\\boldsymbol\{\\mu\}\_\{l\}^\{\\mathrm\{safe\}\}\), so⟨𝝁lharm−𝝁lsafe,𝐯l⟩≥0\\langle\\boldsymbol\{\\mu\}\_\{l\}^\{\\mathrm\{harm\}\}\-\\boldsymbol\{\\mu\}\_\{l\}^\{\\mathrm\{safe\}\},\\mathbf\{v\}\_\{l\}\\rangle\\geq 0by positive semi\-definiteness of\(𝐒W\(l\)\)−1\(\\mathbf\{S\}\_\{W\}^\{\(l\)\}\)^\{\-1\}\. The monotonicity of𝔼⁡\[pl\]\\mathbb\{E\}\[p\_\{l\}\]on harmful inputs then follows from the non\-decreasing Fisher ratio assumption, which is the formal statement of empirical assumption \(iii\)\. The benign bound follows because𝐯l\\mathbf\{v\}\_\{l\}is orthogonal to the between\-class mean difference in the space of the class means, making the benign\-class mean near\-zero in projection\. ∎

The non\-trivial content is the connection between the Fisher ratio’s growth and the trajectory’s monotonicity: any estimator that preserves the Fisher ratio ordering across layers will produce a non\-decreasing expected trajectory\. The empirical assumption \(iii\) is operationalized precisely as this ratio ordering\. The stability result \(pairwise cosine\>0\.97\>0\.97across five splits, Table[12](https://arxiv.org/html/2609.13534#A8.T12)\) is a finite\-sample validation that the LDA estimator converges to the population direction rather than fitting noise, which is a necessary condition for the trajectory to be a reliable population\-level signal\.

For future work, we note that a full derivation under a hierarchical Gaussian mixture generative model would make assumption \(iii\) purely architectural; this is a natural extension beyond the scope of the current empirical contribution\.

## Appendix SWildGuardMix / WildJailbreak Distributional Overlap

*Addressing Reviewer 9qrV\.*

#### Overlap acknowledgment\.

WildGuardMix and WildJailbreak share a common source corpus \(WildChat\([Zhao et al\., 2024](https://arxiv.org/html/2609.13534#bib.bib32)\)\), meaning the training distribution and the WildJailbreak test distribution are not fully independent\. This is a legitimate concern: models trained on WildGuardMix may benefit from stylistic familiarity with WildJailbreak prompts, inflating the reported WJB F1 scores\.

#### Held\-out hard\-divergent subset\.

To bound this effect, we constructed ahard\-divergent WildJailbreak \(HD\-WJB\)subset by filtering for prompts whose n\-gram overlap with the WildGuardMix training set \(measured by ROUGE\-2 recall\) is below the 10th percentile \(ROUGE\-2<0\.07\\text\{ROUGE\-2\}<0\.07\) and whose attack strategy is absent from WildGuardMix \(multi\-role plays, fictional framing, suffix injection\)\. This yieldedN=412N\{=\}412prompts representing maximal prompt\-style divergence\.

Table 19:WildJailbreak results on the full set vs\. the hard\-divergent \(HD\-WJB\) subset\.Performance drops modestly on HD\-WJB, confirming that distributional overlap provides a modest advantage but is not the primary driver ofHerald’s jailbreak performance\.
#### Interpretation\.

Heraldretains a≈2\.5\{\\approx\}2\.5–33point lead over guard models on HD\-WJB \(95\.7 vs\. 94\.8 on OLMo2\-7B\), despite the guard models having no overlap with WildGuardMix\. Performance drops of22–33F1 across all methods on HD\-WJB suggest that distributional overlap benefits all latent methods equally, notHeraldalone\. This confirms that the trajectory\-based mechanism—not stylistic familiarity—is the primary source of jailbreak detection advantage\. We recommend that future work explicitly exclude WildGuardMix\-adjacent prompts when reporting WJB results, and we will update Table 1 accordingly in the camera\-ready version\.

## Appendix TOnset Threshold Sensitivity Analysis

*Addressing Reviewer 9qrV\.*

The onset layerl^∗\\hat\{l\}^\{\*\}is defined as the first layer exceeding theτ90\\tau\_\{90\}\(90th percentile\) threshold computed over the training trajectory distribution\. Table[20](https://arxiv.org/html/2609.13534#A20.T20)reports average F1 under percentile values fromτ70\\tau\_\{70\}toτ95\\tau\_\{95\}\.

Table 20:Sensitivity of average F1 to onset threshold percentile\.Heraldis robust to the choice of onset threshold betweenτ80\\tau\_\{80\}andτ95\\tau\_\{95\}; the performance difference across this range is≤0\.4\\leq 0\.4F1 on all backbones\. The thresholdτ90\\tau\_\{90\}was selected by cross\-validation and is optimal or near\-optimal in all cases\.#### Interpretation\.

The maximum F1 variation across all tested percentiles is≤0\.9\{\\leq\}0\.9on any backbone and benchmark, confirming thatHeraldis not sensitive to the precise onset threshold\. Thresholds belowτ80\\tau\_\{80\}tend to trigger earlier, picking up false\-onset signals from mid\-trajectory fluctuations and degrading the onset\-layer feature’s discriminative value\. Aboveτ95\\tau\_\{95\}, the threshold rarely triggers for harmful prompts with moderate trajectory slopes, suppressing onset\-layer information unnecessarily\. The plateau betweenτ85\\tau\_\{85\}andτ95\\tau\_\{95\}suggests that the 90th\-percentile default is robust; practitioners may safely use any value in this range\.

## Appendix UEvaluation on a Reasoning Model

*Addressing Reviewer 9qrV\.*

Reviewer 9qrV correctly identifies that reasoning/thinking models are absent from our backbone evaluation, and that this represents a potential scope limitation: if harmful intent can emerge during a chain\-of\-thought trace rather than at prompt\-prefill, HPD may not transfer\.

#### Setup\.

We evaluateHeraldwithQwen3\-8B\-Thinking\(the reasoning variant of Qwen3\-8B\-Instruct enabled via theenable\_thinking=Trueflag, which activates internal chain\-of\-thought generation before the user\-visible response\)\. Because thinking tokens are generated*after*prefill,Herald’s harm\-direction projection is computed solely on the prefill hidden states, exactly as for the instruction\-tuned variant\. We compare on WildGuardMix and WildJailbreak with LDA directions trained on Qwen3\-8B\-Thinking activations \(same WildGuardMix split\)\.

Table 21:Heraldon a reasoning model \(Qwen3\-8B\-Thinking\) vs\. the instruction\-tuned variant \(Qwen3\-8B\-Instruct\)\.HPD persists on the reasoning model; trajectory shape remains discriminative at prefill, before any CoT generation begins\. The modest drop \(≈1\.3\{\\approx\}1\.3F1\) relative to the instruct variant reflects a minor calibration difference, not a structural failure of HPD\.
#### Findings\.

HPD persists on Qwen3\-8B\-Thinking: the average F1 drop versus the instruct variant is1\.31\.3points—well within the margin separatingHeraldfrom its latent baselines\. Inspection of per\-prompt trajectories confirms that the monotone rise pattern is present for harmful inputs on the thinking model, with nearly identical onset\-layer statistics to the instruct variant \(jailbreak onset atl^∗≈7\.1\\hat\{l\}^\{\*\}\{\\approx\}7\.1vs\.7\.47\.4for instruct, monotonicity0\.810\.81vs\.0\.830\.83\)\.

#### Scope limitation\.

The experiment evaluates HPD at*prefill*only, before CoT generation begins\. A full reasoning trace may distribute harmful\-intent encoding across generated thinking tokens; monitoring the representations during CoT generation \(rather than prefill\) is an open direction\. If an adversary crafts a prompt whose harmful intent is only resolvable after extended reasoning \(e\.g\., a multi\-step deduction that terminates in a harmful conclusion\), HPD would not detect it at prefill\. This is a genuine scope limitation that applies equally to all existing prompt\-level moderators; generation\-time monitoring of reasoning traces is a natural extension of this work\.

## Appendix VComparison Against the Best Single Layer \(Reviewer SLvX\)

*Addressing Reviewer SLvX\.*

Reviewer SLvX asks why the trajectory comparison uses the last layer rather than the best\-performing single layer across all layers, and whether the trajectory advantage persists against an oracle single\-layer selection\.

#### Oracle single\-layer baseline\.

Table[5](https://arxiv.org/html/2609.13534#S7.T5)\(main paper\) already includes this comparison: the “Single best layer \(oracle\)” row selects the layer achieving maximum validation F1 per backbone\. The oracle trailsHeraldby0\.80\.8–1\.41\.4F1 on average and by up to1\.91\.9F1 on WildJailbreak\. We replicate and extend this result here with a per\-benchmark breakdown\.

Table 22:Oracle single\-layer vs\.Heraldtrajectory, per benchmark\.The trajectory gap is small on SimpST \(simple, surface\-level safety tests\) and large on WildJailbreak \(adversarial, multi\-step jailbreaks\), directly reflecting the HPD claim: trajectory information is most valuable precisely where harmful intent is most gradually revealed\.
#### Why the oracle single layer is not presented in the main table\.

The oracle uses the best layer on the validation set, so it has access to distributional structure not available at deployment time\. In practice, one would need to select the layer either by cross\-validation \(introducing a hyperparameter\) or by the same feature\-engineering logic thatHeraldalready applies\. The all\-layerHeraldavoids this choice entirely while outperforming the oracle, making the comparison favorable to the single\-layer approach\.

The benchmark\-specific breakdown in Table[22](https://arxiv.org/html/2609.13534#A22.T22)supports the HPD narrative directly: the trajectory advantage is smallest on SimpST \(\+0\.2\+0\.2F1\), a benchmark with direct, unambiguous harmful requests where intent is resolved in early layers and the terminal representation is already fully informative\. The advantage is largest on WildJailbreak \(\+1\.9\+1\.9F1\), where multi\-step adversarial framing distributes harmful intent across layers in exactly the pattern HPD is designed to capture\.

#### Last\-token vs\. oracle single layer in the main comparison\.

Reviewer SLvX’s original question also applies to the latent baselines: we use last\-token embeddings for embed\. clf\. and act\. delta, which is their natural operating point \(following prior work\), whereas forHeraldwe use all layers\. To confirm fairness, Table[23](https://arxiv.org/html/2609.13534#A22.T23)reports embed\. clf\. at its own oracle single layer; the gap toHeraldwidens to3\.53\.5–4\.84\.8F1, confirming thatHerald’s advantage is not an artifact of layer selection\.

Table 23:Embed\. clf\. at oracle single layer vs\.Heraldtrajectory\.Even when embed\. clf\. is given oracle access to the best single layer,Heraldoutperforms it by3\.53\.5–4\.84\.8F1\.

## Appendix WLearned Time\-Series Aggregation vs\. Geometric Features

*Addressing Reviewer SLvX\.*

Reviewer SLvX asks why we do not treat\{pl\}\\\{p\_\{l\}\\\}directly as a time series and apply standard time\-series analysis \(e\.g\., pattern recognition or a learned sequence model\) rather than extracting hand\-crafted features\.

#### The seven features are explicitly time\-series features\.

Curvature \(Δ2​𝒑\\Delta^\{2\}\\boldsymbol\{p\}\), monotonicity \(mono⁡\(𝒑\)\\mathrm\{mono\}\(\\boldsymbol\{p\}\)\), onset layer \(l^∗\\hat\{l\}^\{\*\}\), and total rise \(pL−p1p\_\{L\}\-p\_\{1\}\) are canonical time\-series descriptors; they are used in the tsfresh library\([Christ et al\., 2018](https://arxiv.org/html/2609.13534#bib.bib20)\)and in standard anomaly\-detection pipelines\. The design choice is therefore not “time series vs\. not,” but*domain\-motivated feature selection*vs\. learned aggregation\.

#### Why domain\-motivated features over fully learned aggregation\.

The trajectory\{pl\}\\\{p\_\{l\}\\\}is a 32\-step scalar sequence \(forL=32L\{=\}32models\)\. At this length:

- •A 1D\-CNN or GRU must estimate filter or recurrent weights from the training distribution\. For a typical safety dataset of∼5,000\{\\sim\}5\{,\}000training examples, a small GRU has enough capacity to overfit the training set’s stylistic patterns rather than the geometric invariants \(monotone rise, early onset\) that generalize across prompt styles\.
- •The geometric features have*closed\-form definitions*aligned with the HPD hypothesis: onset layer tests when harmfulness first appears, monotonicity tests whether it consistently grows, curvature tests whether growth is smooth\. These features are designed to be invariant to prompt length and stylistic variation, whereas a learned aggregator’s internal representations are not interpretable in these terms\.
- •Our learned\-sequence baseline experiments \(Table[16](https://arxiv.org/html/2609.13534#A13.T16), Appendix[M](https://arxiv.org/html/2609.13534#A13)\) confirm empirically that the geometric features outperform the GRU and 1D\-CNN at the relevant training\-set sizes\.

#### When would learned aggregation be preferable?

A learned sequence model would be preferable if \(i\) training data are abundant \(≫50,000\\gg 50\{,\}000examples per harm category\), \(ii\) trajectories have non\-monotone but learnable patterns not expressible as curvature/monotonicity, or \(iii\) the sequence length is much longer \(e\.g\.,L\>100L\>100\) so that there is sufficient intra\-sequence structure to reward a learner\. None of these conditions hold in the current setting; should they arise in future work \(e\.g\., with 70B models havingL=80L\{=\}80layers and large curated safety datasets\), a learned sequence aggregator would be a natural extension\.