被提示所引导的歧视：在真实场景中泛化恶意输入探测

# GeneralizingMalicious-Input Probes in the Wild
Source: [https://arxiv.org/html/2610.02413](https://arxiv.org/html/2610.02413)
\\workshoptitle

Agents in the Wild: Safety, Security, and Beyond

## Prompted to Discriminate: Generalizing Malicious\-Input Probes in the Wild

Max FominAffiliation:ZenityAffiliation:Tel Aviv, IsraelEmail:[maxf@zenity\.io](mailto:)

###### Abstract

LLM agents increasingly rely on activation probes as runtime monitors for prompt injection, jailbreaks, and unsafe requests, reading the model’s own hidden state to catch a harmful input before the agent acts on it\. A cheap and increasingly common move, borrowed from LLM\-as\-judge prompting, is to append a short classification instruction after the user’s turn and read the probe at that point, to*try*to sharpen it: the instruction asks the model to represent the incoming request*as a class*, concentrating the signal the probe must separate, at negligible serving cost\. But does the wording of that suffix matter, and does its benefit hold*in the wild*, on attack types the probe never saw in training? That out\-of\-distribution regime is the one a deployed monitor faces\. We test this with a controlled ladder of post\-user suffixes under strict leave\-one\-dataset\-out \(LODO\) evaluation across 13 safety benchmarks \(jailbreak, injection, and benign chat\) and three open\-weight model families \(Llama\-3\.1\-8B, Qwen3\.5\-9B, Gemma\-4\-12B\)\. On a single\-position probe, a classification suffix*consistently improves*out\-of\-distribution detection over the no\-suffix baseline \(up to∼4\{\\sim\}4AUC points\); yet*which*suffix matters: prompting the model to*classify*the input, even into content\-free labels, reliably wins, whereas an off\-topic or merely\-attentive suffix helps little\. The gain comes from the classification*format*, not the named criterion: a content\-free suffix does as well as the real malicious/benign one, with the criterion adding precision only at strict thresholds\. This is not an artifact of the single\-position read: the benefit carries to the multi\-position pooling probes used in production \(attention, multi\-max, MLP\), though the best\-performing suffix there is readout\-dependent\. Served through a KV\-cache fork, it is a cheap drop\-in for any activation\-probe monitor, though not an automatic win:*which*suffix helps, and by how much, depends on the model and the readout\.

## 1Introduction

LLM agents increasingly rely on activation\-based probes as runtime monitors for prompt injection, unsafe intent, and policy violations\([Zou et al\., 2023](https://arxiv.org/html/2610.02413#bib.bib11);[Abdelnabi et al\., 2025](https://arxiv.org/html/2610.02413#bib.bib1);[Zou et al\., 2025](https://arxiv.org/html/2610.02413#bib.bib12)\): a probe on the model’s hidden state flags a malicious request before the agent acts, riding the model’s own forward pass rather than a separate classifier; recent production monitors pool over many positions\([Kramár et al\., 2026](https://arxiv.org/html/2610.02413#bib.bib4)\)\. A central question for deploying such a monitor is how to make it read well\.[Tillman and Mossing \(2025\)](https://arxiv.org/html/2610.02413#bib.bib9)show that appending a task\-description instruction \(*prompted probing*\) improves out\-of\-distribution safety monitoring, but Tillman & Mossing’s instructions are semantically meaningful throughout and the mechanism is left open\. We ask the question that determines how the monitor should be built:*what part of the instruction does the work: the fact that it induces a classification, or the specific criterion it names?*

We answer it with adesigned ladder of post\-user suffixes, each isolating one ingredient \(Table[1](https://arxiv.org/html/2610.02413#S1.T1)\)\. We study*post\-user*suffixes specifically because they deploy for free: appended after the user turn on a KV\-cache fork \(adopted for the simplicity and perturbation\-robustness[David et al\. \(2026\)](https://arxiv.org/html/2610.02413#bib.bib2)report\), read once, then reverted \(§[3\.1](https://arxiv.org/html/2610.02413#S3.SS1)\), so the monitor adds no separate forward pass and inherits that vehicle’s position stability and robustness to input perturbations\.

We make three contributions:

1. 1\.We extend prompted probing into the wild, and decompose it\(§[4](https://arxiv.org/html/2610.02413#S4)\)\.[Tillman and Mossing \(2025\)](https://arxiv.org/html/2610.02413#bib.bib9)showed a task instruction improves out\-of\-distribution monitoring on one closed model; we carry that analysis to three open\-weight families under strict leave\-one\-dataset\-out, and go further with a length\-matched suffix*ladder*that isolates which ingredient of the instruction does the work\. The classification suffix consistently improves detection \(up to∼4\{\\sim\}4AUC points; positive on2222of2727model–dataset cells, App\.[A](https://arxiv.org/html/2610.02413#A1)\)\.
2. 2\.The classification*format*, not the named criterion, carries the ranking gain\(§[4](https://arxiv.org/html/2610.02413#S4)\)\. Being*asked to classify*is what transfers: a content\-free A/B suffix is statistically equivalent to the intent suffix \(<1\{<\}1AUC point apart; TOSTp≤0\.005p\{\\leq\}0\.005; Llama and Gemma\), while off\-topic or attend\-only suffixes do no better than no suffix\. The named criterion adds precision only at strict thresholds\.
3. 3\.The benefit holds across readout architectures\(§[4\.2](https://arxiv.org/html/2610.02413#S4.SS2)\)\. It survives the move from a single\-position read to the multi\-position pooling probes \(attention, multi\-max, MLP\) used at scale \(92%92\\%of readout×\\timessuffix cells beat no\-suffix\), so the gain is a property of the representation the suffix induces, not of one probe head\. The best\-performing suffix is readout\-dependent, making suffix choice a per\-probe design knob\.

Table 1:The suffix ladder \(experimental design\)\.Each suffix adds one ingredient; comparing adjacent suffixes isolates it\.#### Scope\.

Building on prompted probing\([Tillman and Mossing, 2025](https://arxiv.org/html/2610.02413#bib.bib9)\)and the KV\-fork suffix\([David et al\., 2026](https://arxiv.org/html/2610.02413#bib.bib2)\), our contribution is to establish, by controlled ablation,*what in the suffix*drives the gain and how broadly it holds: across suffixes, models, and probe architectures\. Table[2](https://arxiv.org/html/2610.02413#S1.T2)summarizes what we add over Tillman & Mossing\.

Table 2:Delta over[Tillman and Mossing \(2025\)](https://arxiv.org/html/2610.02413#bib.bib9)\.Every row is backed in this paper\. \(T&M report AUROC on a single closed model, GPT\-4o, with limited within\-task distribution shift on one dataset\.\)Concretely, every suffix shares the stemCRITICAL REMINDER \-\-and differs only in what follows\. Four verbatim examples \(the full set is in App\.[A](https://arxiv.org/html/2610.02413#A1)\):

> intent:…start your response with ‘Input is⟨\\langleclass⟩\\rangle’ where classes are malicious/benign\. A/B:…classes are A/B\. reflect:…take a moment to read the message above carefully and answer completely\. irrelevant:…here is an unrelated fact: the ocean covers most of the planet\.

## 2Background and Related Work

#### Prompted probing\.

[Tillman and Mossing \(2025\)](https://arxiv.org/html/2610.02413#bib.bib9)introduce instruction\-conditioned probing and show it improves OOD safety monitoring on a GPT\-4o snapshot \(AUROC; linear vs\. prompted vs\. SAE probing\)\. Activation probing has progressed from reading a concept as a linear direction in the hidden state\([Zou et al\., 2023](https://arxiv.org/html/2610.02413#bib.bib11)\)\(as single\-position injection detectors still do,\([Abdelnabi et al\., 2025](https://arxiv.org/html/2610.02413#bib.bib1);[Zou et al\., 2025](https://arxiv.org/html/2610.02413#bib.bib12)\)\) to the multi\-position*pooling*probes \(attention, multi\-max, MLP\) that[Kramár et al\. \(2026\)](https://arxiv.org/html/2610.02413#bib.bib4)deploy at production scale\. That line develops the probe*architecture*; prompted probing is orthogonal, acting on the input\. We combine the two, taking prompted probing across these SOTA readout architectures, from a single position to the advanced multi\-position probes\.

#### Do these probes read the concept, or surface features?

[Wang et al\. \(2025\)](https://arxiv.org/html/2610.02413#bib.bib10)show malicious\-input probes can reach\>\>98% in\-distribution yet learn trigger words and instructional patterns and fail to generalize\. This bears on us two ways\. Our content\-freeA/Bcontrol answers the trigger\-word half: the gain survives when the criterion carries no attack vocabulary, so it is not that shortcut\. On the instructional\-pattern half, our suffix*is*such a pattern, so we make no claim that the probe reads ground\-truth harmfulness, only that a classification*format*yields a more transferable readout on our confound\-controlled benchmarks\.

#### Format over content\.

[Min et al\. \(2022\)](https://arxiv.org/html/2610.02413#bib.bib6)show in\-context learning is largely insensitive to label*content*\(format over content\), behaviorally, via demonstrations; prompts are broadly format\-sensitive\([Sclar et al\., 2024](https://arxiv.org/html/2610.02413#bib.bib8);[Mizrahi et al\., 2024](https://arxiv.org/html/2610.02413#bib.bib7)\)\. Our result is the activation\-level analogue for a safety probe: prompting the model to*classify*the input \(even into content\-free labels\) reshapes its representation to be more separable across datasets, largely independent of the criterion named\. We state this for the models with a gain to carry \(Llama, Gemma\); on Qwen it is small \(near\-ceiling\)\.

#### Cross\-dataset evaluation\.

[Fomin \(2026\)](https://arxiv.org/html/2610.02413#bib.bib3)show malicious classifiers collapse under LODO relative to in\-distribution CV \(pooled AUC gap88–1616points\) on overlapping model families, but explicitly find that SAE\-feature dataset shortcuts do*not*causally account for the gap and never vary how the classifier is prompted\. We adopt their LODO rigor but ask a different question: how the readout is prompted, not which trained\-probe features it exploits\. Our aim is a better\-ranking probe, not a claim to eliminate the in\-distribution gap they measure\.

#### The deployment driver\.

[David et al\. \(2026\)](https://arxiv.org/html/2610.02413#bib.bib2)introduce the KV\-fork post\-user suffix \(a stable single\-position readout at near\-zero cost\) and frame agent monitoring around the low\-FPR operating point rather than aggregate AUROC\. They show a*generic*suffix makes the readout robust to input perturbations, but do not ask what a*classification*instruction adds\. We sit between the two: we take the prompted\-probing question of[Tillman and Mossing \(2025\)](https://arxiv.org/html/2610.02413#bib.bib9)and the suffix vehicle and operating\-point lens of[David et al\. \(2026\)](https://arxiv.org/html/2610.02413#bib.bib2), and resolve the instruction into a controlled ladder separating format from content under LODO\.

## 3Setup

#### Models and probes\.

Llama\-3\.1\-8B, Qwen3\.5\-9B, and Gemma\-4\-12B, read at late layers \(∼2/3\{\\sim\}2/3of depth, where the malicious/benign distinction is most linearly accessible; exact layers in App\.[C](https://arxiv.org/html/2610.02413#A3)\)\. Our default readout is a frozen single\-position linear probe \(L2\-logistic, standardized,C=0\.1C\{=\}0\.1\) at the last suffix token; §[4\.2](https://arxiv.org/html/2610.02413#S4.SS2)additionally evaluates the advanced multi\-position*pooling*probes \(attention, multi\-max, MLP\)\. We check robustness across two layers per model \(§[5](https://arxiv.org/html/2610.02413#S5)\); full probe and layer settings in App\.[C](https://arxiv.org/html/2610.02413#A3)\.

#### Suffix ladder\.

Table[1](https://arxiv.org/html/2610.02413#S1.T1); all suffixes share theCRITICAL REMINDER \-\-stem and are length\-controlled, so adjacent comparisons isolate a single ingredient rather than length or mere presence of text\. The main suffix \(intent, or its content\-free twinA/B\) is 29 tokens\.

#### Benchmarks\.

13 chat safety, jailbreak, and injection benchmarks \(full list in App\.[B](https://arxiv.org/html/2610.02413#A2)\), evaluated LODO: each dataset’s malicious examples are scored against one shared, balanced benign pool\.N=2000N\{=\}2000prompts per dataset\(fewer where a dataset is smaller\)\.

### 3\.1The suffix and its deployment vehicle

The suffix rides the post\-user KV\-cache fork of Figure[1](https://arxiv.org/html/2610.02413#S3.F1): a short classification instruction is appended after the user turn, the probe reads at end\-of\-suffix, and the cache is reverted so the served response is byte\-identical to no\-suffix serving: monitoring rides the model’s existing forward pass, not a second one\.

We*inherit*this vehicle’s benefits rather than re\-establish them: negligible overhead \(a 29\-token suffix and a few MB of KV cache, with no second forward pass\), perturbation\-robustness, and a readout\-stabilization gain for a*generic*suffix\([David et al\., 2026](https://arxiv.org/html/2610.02413#bib.bib2)\), and ask the orthogonal question: what the suffix should*say*\. Adopting it is a one\-line change to an existing probe pipeline \(same architecture, same training data, one appended suffix\) that keeps serving and monitoring in a single forward pass, unlike prompt\-reformatting\([Tillman and Mossing, 2025](https://arxiv.org/html/2610.02413#bib.bib9)\)\.

serve\+monitorEOTsysENDprobe readsrevert KV→\\togenerateclassification suffix \(post\-user role; same forward pass\)Figure 1:The classification suffix on the KV\-cache fork\.A short classification instruction is appended after the user turn; the probe reads at end\-of\-suffix; the cache is reverted so generation proceeds unchanged\. We adopt the KV\-fork vehicle from[David et al\. \(2026\)](https://arxiv.org/html/2610.02413#bib.bib2)and study what the suffix should*say*\.#### Metrics\.

Primary =shared\-benign AUC: each malicious\-containing dataset’s malicious examples are ranked against one fixed, shared benign pool\. This is threshold\-free and defined for all 9 malicious\-containing datasets\. PlusTPR@1% FPR, the strict operating point a deployed guardrail runs at\([David et al\., 2026](https://arxiv.org/html/2610.02413#bib.bib2);[McKenzie et al\., 2025](https://arxiv.org/html/2610.02413#bib.bib5)\)\. We tabulate deltas versusnone\(no suffix\), but the deployment\-relevant comparison is versus a*neutral*suffix \(the relocation a monitor may already run; §[4](https://arxiv.org/html/2610.02413#S4)\)\. All numbers are LODO out\-of\-fold,N=2000N\{=\}2000/dataset; we reportper\-dataset; any aggregate is an equal\-weight mean across datasets\.

## 4Results and Analysis

We first analyze the single\-position probe \(§[4\.1](https://arxiv.org/html/2610.02413#S4.SS1)\), then show the benefit extends to the multi\-position pooling probes used in production \(§[4\.2](https://arxiv.org/html/2610.02413#S4.SS2)\)\.

### 4\.1Single\-position probe

A classification suffix consistently improves out\-of\-distribution detection: up to∼4\{\\sim\}4shared\-benign AUC points over no suffix, positive on 22 of 27 \(model, dataset\) cells \(Figure[2](https://arxiv.org/html/2610.02413#S4.F2)\)\.*What in it*does the work splits into two ingredients: the classification*format*carries the cross\-dataset*ranking*gain \(shared\-benign AUC\), while naming the real*criterion*adds*precision*at a strict false\-positive rate\. The two dissociate cleanly: a content\-free format matches the full suffix on ranking yet leaves the precision gain on the table \(App\.[C\.3](https://arxiv.org/html/2610.02413#A3.SS3)\)\. Throughout, we compare each classification suffix both to*no suffix*and to a*neutral*suffix \(reflect\): since appending any generic suffix already relocates the readout, the neutral suffix is the stricter baseline for what*classification*adds\.

![Refer to caption](https://arxiv.org/html/2610.02413v1/figures/ladder_result.png)Figure 2:A classification suffix lifts OOD detection; format, not criterion, carries the ranking gain\.Δ\\Deltashared\-benign AUC vs\. no suffix \(mean over 9 LODO datasets,±\\pmSEM\)\. A neutral suffix \(reflect\) barely moves ranking; asking the model to*classify*lifts it \(Llama, Gemma\), and content\-freeA/Bmatches namedintent\. Qwen is near\-ceiling\. Precision \(the criterion’s low\-FPR gain\): App\.[A\.1](https://arxiv.org/html/2610.02413#A1.SS1)\.The ranking gain \(Table[3](https://arxiv.org/html/2610.02413#S4.T3)\) differs by model in magnitude: large on Llama and Gemma, near\-ceiling on Qwen\. Naming the real criterion additionally sharpens recall at a strict1%1\\%\-FPR operating point \(App\.[A\.1](https://arxiv.org/html/2610.02413#A1.SS1)\), the regime a deployed monitor runs at\([David et al\., 2026](https://arxiv.org/html/2610.02413#bib.bib2)\); our recall there is in range of concurrent probe\-monitoring under shift \([McKenzie et al\., 2025](https://arxiv.org/html/2610.02413#bib.bib5):43%43\\%at1%1\\%FPR vs\.∼55%\{\\sim\}55\\%for their best LLM monitor\)\.

Table 3:Suffix ladder:Δ\\Deltashared\-benign AUC vs\.none\(points; LODO,N=2000N\{=\}2000; 9 malicious\-containing datasets; paired Wilcoxon∗p<0\.05\{\}^\{\*\}p\{<\}0\.05\)\. Content\-freeA/B≈\\approxnamedintentacross models; on Llama the null suffixes sit at baseline, while Gemma responds to*any*post\-user suffix \(relocation, §[4](https://arxiv.org/html/2610.02413#S4)\)\. Qwen is near\-ceiling\.#### Classification, not the criterion, carries the ranking gain\.

Two comparisons isolate the format\. First, suffixes that add*no*classification \(off\-topic text, or an instruction merely to attend\) leave the probe unmoved \(Llamairrelevant−0\.14\-0\.14,reflect−0\.44\-0\.44; n\.s\.\), whereas asking the model to classify lifts it\. Second, once it classifies, the*criterion*does not matter: a content\-freeA/Bsuffix is statistically equivalent to the realintentone on shared\-benign AUC \(TOST,±2\\pm 2\-pt bound; Llamap=0\.004p\{=\}0\.004, Gemmap=0\.005p\{=\}0\.005\)\. So the classification*format*, not the named criterion, produces the transferable ranking\. \(The criterion is not*entirely*inert: two arbitrary criteria disagree on Llama,classify\_grammar\+2\.0\+2\.0vs\.classify\_length−1\.6\-1\.6; but a meaningful one adds nothing over a content\-free one\.\) Naming the real classes does buy low\-FPR precision on top, at a single noisy operating point \(App\.[A\.1](https://arxiv.org/html/2610.02413#A1.SS1)\)\.

#### The right baseline is a neutral suffix, not the raw probe\.

A neutral post\-user suffix is essentially the KV\-fork relocation[David et al\. \(2026\)](https://arxiv.org/html/2610.02413#bib.bib2)already credit for robustness \(we usereflect\); it, not the raw probe, is the baseline for “what does a*classification*suffix add\.” Over it the ranking increment is modest but consistent: the classification suffix wins73%73\\%of per\-dataset comparisons \(38/52; sign\-testp=6×10−4p\{=\}6\{\\times\}10^\{\-4\}; Llama median\+1\.8\+1\.8, the larger mean inflated bymosscap\); the clean win over neutral is at strict precision \(App\.[A\.1](https://arxiv.org/html/2610.02413#A1.SS1)\)\.

#### Magnitude tracks headroom, with a ceiling caveat\.

The improvement is consistent: 22 of 27 \(model, dataset\) cells \(sign\-testp<10−3p\{<\}10^\{\-3\}\), strong on Llama and Gemma, smaller on Qwen, and concentrates where the base probe is weakest: the intent gain falls as the no\-suffix baseline rises \(pooledr=−0\.78r\{=\}\{\-\}0\.78,p=1\.4×10−6p\{=\}1\.4\{\\times\}10^\{\-6\}; Fig\.[3](https://arxiv.org/html/2610.02413#S4.F3)\)\. Part of this slope is mechanical \(the gain is bounded above\), and*within*Qwen the trend is weak \(r=−0\.29r\{=\}\{\-\}0\.29\), so headroom is only a partial account of Qwen’s smaller effect; its readout regime \(thinking mode\) is a confound we do not disentangle\.

![Refer to caption](https://arxiv.org/html/2610.02413v1/figures/headroom.png)Figure 3:The gain concentrates where the probe has headroom\.Each point is one \(model, dataset\): no\-suffix shared\-benign AUC \(xx\) vs\. the intent gain \(yy\)\. Pooledr=−0\.78r\{=\}\{\-\}0\.78\(Llama−0\.77\-0\.77, Gemma−0\.92\-0\.92, Qwen−0\.29\-0\.29\); part of the slope is a ceiling effect \(bounded AUC\), so we read it as a headroom regularity, not a law\.The gain lands on non\-saturated datasets \(Llamadeepset80→9080\\to 90,harmbench81→9181\\to 91\) and not on near\-ceiling ones, and the mean can mislead:reflect’s aggregate “null” is a cancellation that helpsdeepset/harmbenchbut crashesmosscap\(83→6383\\to 63\), so we report per\-dataset throughout \(App\.[A](https://arxiv.org/html/2610.02413#A1)\)\.

Where the probe has headroom, we can characterize*why*the suffix helps: it aligns an otherwise\-scattered per\-source malicious direction into a shared axis \(on Gemma, leave\-one\-source\-out transfer AUC0\.74→0\.930\.74\\to 0\.93; the classes visibly separate in a t\-SNE of the readout\), whereas on models whose malicious axis is already formed without a suffix \(Llama, Qwen; transfer0\.930\.93,0\.960\.96\) there is little to add, a representational reading of the headroom\-gating seen throughout, detailed in App\.[C\.2](https://arxiv.org/html/2610.02413#A3.SS2)\.

### 4\.2Multi\-position \(pooling\) probes

Multi\-position monitors increasingly read the hidden state not at a single position but by*pooling*over many \(attention, multi\-max, or an MLP head\)\([Kramár et al\., 2026](https://arxiv.org/html/2610.02413#bib.bib4)\)\. If the suffix’s gain were an artifact of the single\-position readout we \(and[Tillman and Mossing, 2025](https://arxiv.org/html/2610.02413#bib.bib9)\) use, it need not survive that shift\. It does\. Table[4](https://arxiv.org/html/2610.02413#S4.T4)reports, for each readout, the mean suffix gain over no\-suffix across the five\-rung ladder \(best single rung in parentheses\), holding the probe’s training data and the LODO protocol fixed: a suffix lifts*every*readout we test, from the single\-position read to the advanced multi\-position probes\.

Table 4:The suffix helps across readout architectures\.Suffix gain over no\-suffix in shared\-benign LODO AUC \(pp\), primary layers, per pooling readout\([Kramár et al\., 2026](https://arxiv.org/html/2610.02413#bib.bib4)\)\. Each cell: mean over the five\-rung ladder \(best rung in parentheses\)\. Averaged over the ladder,22/24model×\\timesreadout configs and110/120\(readout×\\timesrung\) cells beat no\-suffix \(92%\), needing no rung selection\. Full grid: Table[14](https://arxiv.org/html/2610.02413#A3.T14)\.The lift is model\-modulated, largest where the single\-position probe has headroom \(Gemma, Llama\), smallest where it is already saturated \(Qwen, baselines∼95\{\\sim\}95; its one negative cell, in the full grid, is the most\-saturated attention readout at its second layer\)\. Two things follow\. First, the benefit is a property of the*representation*, not of one probe head: any reasonable readout can exploit it\. The format/criterion split holds here too, not just at the single position: a content\-freeA/Bsuffix matches the namedintentone within0\.30\.3AUC points on average across pooling readouts \(23 of 24 cells within11pt; Table[14](https://arxiv.org/html/2610.02413#A3.T14)\)\. Second, and more practically,the best\-performing rung is readout\-dependent\(lengthandreflecttop more columns thanintent, Table[14](https://arxiv.org/html/2610.02413#A3.T14)\), so while*some*classification suffix helps universally, the optimal suffix is a per\-probe design choice rather than a single fixed string\. This per\-readout tuning is a small residual on top of the shared\-axis effect, not a contradiction of it: the large gain is the no\-suffix→\\tosuffix alignment jump \(on Gemma,0\.74→0\.930\.74\\to 0\.93; App\.[C\.2](https://arxiv.org/html/2610.02413#A3.SS2)\), and among*effective*suffixes the cross\-readout differences are≲1\{\\lesssim\}1pt\.

## 5Limitations

#### Where the effect is weaker\.

The classification\-*specific*benefit is a Llama/Gemma result\. Qwen is positive vs\. no\-suffix \(6/9 datasets, so not a null\), but its gain over a*neutral*suffix is only 4/9 and its precision increment is flat, so we do not claim the classification\-specific effect for it\. Per\-model readout differs \(Llama L31, non\-thinking; Qwen L31, thinking; Gemma L47\), so cross\-model magnitudes are confounded with readout regime\. The content\-tailoring result is preliminary \(one genre, source confound\), and we do not claim the probe reads ground\-truth harmfulness \(cf\.[Wang et al\., 2025](https://arxiv.org/html/2610.02413#bib.bib10)\): a content\-free format yields a more transferable*readout*, not a verified concept\.

#### Measurement\.

The gain is a ranking improvement, not recalibration: it appears in threshold\-free shared\-benign AUC, and accuracy@0\.5 understates it \(Gemma\+1\.8\+1\.8–2\.82\.8vs\. AUC\+2\.7\+2\.7–3\.23\.2\); compare prompted detectors on a threshold\-free metric, read per\-dataset\. We prefer shared\-benign AUC over within\-dataset AUC, which exists for only the 4 mixed\-class benchmarks \(too few to resolve the dissociation; it spuriously readsreflectas\+2\.4\+2\.4on Qwen\); the shared benign pool recovers all 9 malicious datasets\.

#### Significance\.

Withn=9n\{=\}9datasets the minimum two\-sided Wilcoxonppis0\.0040\.004, so per\-cell tests have limited power and we do not rest claims on them\. We rely instead on the dataset\-level sign\-test \(22/27 cells positive,p<10−3p\{<\}10^\{\-3\}\), the directional TPR@1% tests \(one\-sidedp=0\.002p\{=\}0\.002\), and the A/B\-vs\-intent equivalence \(TOST,p≤0\.005p\{\\leq\}0\.005; Llama and Gemma\)\.

#### Robustness to layer\.

The gain does not depend on the exact readout layer: on Gemma it is essentially unchanged at a second late layer \(\+3\.1\+3\.1at L40 vs\.\+3\.2\+3\.2at L47\), and on Qwen it is near\-zero at both L27 and L31, so Qwen’s weakness is intrinsic to the model, not the layer\.

## 6Conclusion

#### Takeaway\.

For prompted activation probes, a classification suffix consistently improves malicious\-input detection; the classification*format*carries the ranking gain \(a content\-free criterion suffices\), while naming the real criterion*adds*the low\-FPR precision that deployment reads\.

Deploying this\.\(1\) append a 29\-token post\-user classification suffix via the KV\-fork; \(2\) train the probe on the end\-of\-suffix activation as in standard probing—same cost as a no\-suffix probe, no model fine\-tuning, no second prefill \(the KV\-fork reuses the user prefill; the backbone only supplies the embedding\); \(3\)*name the real classes*\(malicious/benign\), not placeholders: where the probe has low\-FPR headroom \(Llama, Gemma\) this is what earns the recall; \(4\) evaluate per\-dataset on a threshold\-free metric atN≥2N\{\\geq\}2k; \(5\) keep the criterion*broad*: benchmark\-specific content overfits out\-of\-genre\.

#### On “in the wild\.”

This targets agent monitoring: activation probes read the model’s own forward pass on the incoming request to flag malicious inputs before the agent acts\([Abdelnabi et al\., 2025](https://arxiv.org/html/2610.02413#bib.bib1);[Zou et al\., 2025](https://arxiv.org/html/2610.02413#bib.bib12);[Kramár et al\., 2026](https://arxiv.org/html/2610.02413#bib.bib4)\), reusing that pass at roughly six orders of magnitude less compute than a separate LLM monitor\([McKenzie et al\., 2025](https://arxiv.org/html/2610.02413#bib.bib5)\)\. We test under strict LODO, the deployment condition: every number is on a benchmark the probe never saw in training\. One scope caveat: our benchmarks are chat\-style, and we do not vary input*packaging or genre*\(single\-turn vs\. multi\-turn, tool\-call transcripts, or heavily formatted content such as tables or code\), so extension to those inputs is a conjecture, not a result\.

#### Future work\.

The decomposition points to three directions\. First,principled suffix optimization: since a per\-readout best rung already beats a fixed string \(§[4\.2](https://arxiv.org/html/2610.02413#S4.SS2)\) and naive benchmark\-specific tailoring overfits out\-of\-genre, searching*broad*, transferable attack\-family content under out\-of\-distribution validation is the natural way to push the gain without overfitting\. Second,agentic tool\-call and multi\-turn inputs, where the deployment payoff is largest and which our single\-turn setup does not yet test\. Third,a deeper representational account: our geometry \(App\.[C\.2](https://arxiv.org/html/2610.02413#A3.SS2)\) shows*where*the suffix helps but not*why*a content\-free classification*format*reshapes the input’s representation to be more separable\. Relating this to the format\-over\-content effect in in\-context learning\([Min et al\., 2022](https://arxiv.org/html/2610.02413#bib.bib6)\)\(what asking the model to*classify*does to its representation of the input, and why the named criterion is largely inert\) is an open question our ablation frames but does not resolve\. Because the content\-free suffix \(A/B\) names no criterion, asking the model to*classify*might make the input easier to read along*any*axis it encodes, or it might merely recruit the model’s strong safety prior—an effect specific to maliciousness\. Reading only the malicious axis we cannot separate these two accounts; applying the same suffix to a non\-safety target, such as task type, would\.

## Acknowledgements

We thank the open\-source community for the models and datasets this work builds on\.

## References

- Abdelnabi et al\. \[2025\]Sahar Abdelnabi, Aideen Fay, Giovanni Cherubin, Ahmed Salem, Mario Fritz, and Andrew Paverd\.Get My Drift? Catching LLM Task Drift with Activation Deltas\.In*2025 IEEE Conference on Secure and Trustworthy Machine Learning \(SaTML\)*, pages 43–67\. IEEE Computer Society, 2025\.doi:10\.1109/SaTML64287\.2025\.00011\.
- David et al\. \[2026\]Elad David, Max Fomin, and Amit LeVi\.Latent undertow: How ordinary typos break probes\.[https://github\.com/eladd\-ai/latent\-undertow](https://github.com/eladd-ai/latent-undertow), 2026\.Zenity\.
- Fomin \[2026\]Max Fomin\.When benchmarks lie: Evaluating malicious prompt classifiers under true distribution shift, 2026\.URL[https://arxiv\.org/abs/2602\.14161](https://arxiv.org/abs/2602.14161)\.
- Kramár et al\. \[2026\]János Kramár, Joshua Engels, Zheng Wang, Bilal Chughtai, Rohin Shah, Neel Nanda, and Arthur Conmy\.Building production\-ready probes for gemini, 2026\.URL[https://arxiv\.org/abs/2601\.11516](https://arxiv.org/abs/2601.11516)\.
- McKenzie et al\. \[2025\]Alex McKenzie, Urja Pawar, Phil Blandfort, William Bankes, David Krueger, Ekdeep Singh Lubana, and Dmitrii Krasheninnikov\.Detecting high\-stakes interactions with activation probes\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2025\.URL[https://arxiv\.org/abs/2506\.10805](https://arxiv.org/abs/2506.10805)\.
- Min et al\. \[2022\]Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer\.Rethinking the role of demonstrations: What makes in\-context learning work?In*Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*, 2022\.URL[https://arxiv\.org/abs/2202\.12837](https://arxiv.org/abs/2202.12837)\.
- Mizrahi et al\. \[2024\]Moran Mizrahi, Guy Kaplan, Dan Malkin, Rotem Dror, Dafna Shahaf, and Gabriel Stanovsky\.State of what art? a call for multi\-prompt LLM evaluation\.*Transactions of the Association for Computational Linguistics*, 12:933–949, 2024\.doi:10\.1162/tacl\_a\_00681\.
- Sclar et al\. \[2024\]Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr\.Quantifying language models’ sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting\.In*The Twelfth International Conference on Learning Representations \(ICLR\)*, 2024\.URL[https://openreview\.net/forum?id=RIu5lyNXjT](https://openreview.net/forum?id=RIu5lyNXjT)\.
- Tillman and Mossing \[2025\]Henk Tillman and Dan Mossing\.Investigating task\-specific prompts and sparse autoencoders for activation monitoring, 2025\.URL[https://arxiv\.org/abs/2504\.20271](https://arxiv.org/abs/2504.20271)\.
- Wang et al\. \[2025\]Cheng Wang, Zeming Wei, Qin Liu, and Muhao Chen\.False sense of security: Why probing\-based malicious input detection fails to generalize, 2025\.URL[https://arxiv\.org/abs/2509\.03888](https://arxiv.org/abs/2509.03888)\.
- Zou et al\. \[2023\]Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann\-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J\. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J\. Zico Kolter, and Dan Hendrycks\.Representation engineering: A top\-down approach to ai transparency, 2023\.URL[https://arxiv\.org/abs/2310\.01405](https://arxiv.org/abs/2310.01405)\.
- Zou et al\. \[2025\]Wei Zou, Yupei Liu, Yanting Wang, Ying Chen, Neil Gong, and Jinyuan Jia\.Pishield: Detecting prompt injection attacks via intrinsic llm features, 2025\.URL[https://arxiv\.org/abs/2510\.14005](https://arxiv.org/abs/2510.14005)\.

## Appendix APer\-dataset results and cross\-model ROC

All values LODO out\-of\-fold,N=2000N\{=\}2000/dataset\.*sb\-AUC*= each dataset’s malicious examples vs\. the shared benign pool \(absolute; per\-fold calibration offsets make deltas, not absolutes, the unit of inference, §[5](https://arxiv.org/html/2610.02413#S5)\)\.

Table 5:Llama: per\-dataset shared\-benign AUC by suffix\.Table 6:Qwen: per\-dataset shared\-benign AUC by suffix\.Table 7:Gemma: per\-dataset shared\-benign AUC by suffix\.Across models the pattern is the same: the classification suffixes \(A/B,intent\) lift the non\-saturated datasets most \(deepset80→9080\{\\to\}90andharmbench81→9181\{\\to\}91on Llama\) and leave near\-ceiling sets \(advbench,safeguard≈98\{\\approx\}98\) unmoved, while Qwen barely moves anywhere\. The macro\-ROC curves \(Figs\.[7](https://arxiv.org/html/2610.02413#A3.F7),[4](https://arxiv.org/html/2610.02413#A1.F4)\) show the same ranking gain as area under the curve\.

![Refer to caption](https://arxiv.org/html/2610.02413v1/figures/roc_qwen.png)

![Refer to caption](https://arxiv.org/html/2610.02413v1/figures/roc_gemma.png)

Figure 4:Qwen \(left\) and Gemma \(right\) macro\-ROC, malicious vs\. shared\-benign over the 9 LODO folds \(cf\. Llama, Fig\.[7](https://arxiv.org/html/2610.02413#A3.F7)\)\. Qwen curves bunch near a 0\.94 baseline; Gemma separates like Llama on area\.### A\.1Precision at a strict operating point \(TPR@1%\)

At a strict1%1\\%\-FPR operating point the named criterion earns its keep:intentexceeds both the content\-freeA/Bsuffix and the neutralreflectat TPR@1% on Llama \(≈\\approx\+28\+28pp, 9/9 datasets, one\-sidedp=0\.002p\{=\}0\.002\) and Gemma \(\+6\.6\+6\.6pp vs\. A/B, 9/9,p=0\.002p\{=\}0\.002\), while Qwen is flat \(\+0\.3\+0\.3pp, n\.s\.\)\. Format alone can lift low\-FPR recall \(on Gemmareflectdoes,\+17\+17pp\), but naming the real classes adds precision on top\. TPR@1% is a single, high\-variance operating point, so we read this increment as directional \(consistent 9/9\) rather than a calibrated size, and state the claim as*content adds precision*, not*content is required*\(on Llama,A/Beven trends below no\-suffix here\)\.

Table 8:Suffix ladder:Δ\\DeltaTPR@FPR==1% vs\.none\(pp\)\.intentpulls ahead of content\-free A/B on Llama/Gemma; A/B*underperforms*no\-suffix on Llama; Qwen stays flat\. The Gemmareflectfigure averages 8 datasets \(wildjailbreakhas noreflectrun\)\.Per\-dataset TPR@1% \(below\) shows the increment is broad on Llama and Gemma but dataset\-specific in size; on several Llama datasets the content\-freeA/Bsuffix falls below no\-suffix, whereintentinstead lifts\.

Table 9:Llama: per\-dataset TPR@1% \(%\) by suffix\.Table 10:Qwen: per\-dataset TPR@1% \(%\) by suffix\.Table 11:Gemma: per\-dataset TPR@1% \(%\) by suffix\.

## Appendix BBenchmarks

The 13 evaluation benchmarks, with role and sizes in the primary panel \(up toN=2000N\{=\}2000/dataset; counts vary slightly by model\)\. Nine contain malicious examples \(scored against the shared benign pool\); the four benign\-source sets contribute to that pool\.

Table 12:Evaluation benchmarks\.Role and number of benign/malicious examples per benchmark in the primary panel\.
## Appendix CProbe architectures and settings

All probes are trained per LODO fold on frozen, standardized residual\-stream activations at the readout layer\. The single\-position probe is L2\-logistic \(C=0\.1C\{=\}0\.1\) at the last suffix token; the multi\-position pooling probes follow the production “Gemini probe” family\[[Kramár et al\., 2026](https://arxiv.org/html/2610.02413#bib.bib4)\]\(over all token positions, Adam\)\. Readout layers: Llama L31, Qwen L27/L31, Gemma L40/L47\.

Table 13:Probe readouts\.Token positions each readout uses and how it is fit\.### C\.1Full architecture×\\timessuffix grid

Table[14](https://arxiv.org/html/2610.02413#A3.T14)reports every cell behind the architecture\-agnostic result \(§[4\.2](https://arxiv.org/html/2610.02413#S4.SS2)\): six suffix conditions×\\timesfour pooling readouts×\\timesthree families×\\timestwo layers\. A classification suffix lifts almost every column overnonewhere the baseline has headroom; the best\-performing rung is readout\-dependent \(length/reflecttop more columns thanintent\), and Qwen is near\-ceiling throughout\.

Table 14:Full architecture×\\timessuffix grid\.Shared\-benign LODO AUC \(%\),N=2000N\{=\}2000/dataset, 9 malicious datasets vs\. shared benign pool\. Each block is one model/layer; rows are the suffix ladder, columns the pooling readouts;noneis the matched baseline, bold = best rung per column\. Almost every column beatsnonewhere the baseline has headroom; the winning rung is readout\-dependent\.
### C\.2What the suffix does to the representation \(where there is headroom\)

This appendix expands the one\-sentence mechanism in §[4](https://arxiv.org/html/2610.02413#S4)\. We look two ways at the single\-position readout, both independent of the trained probe\.\(i\) Cross\-source axis\.We form a malicious direction as the mean difference \(malicious−\-benign\) over all\-but\-one attack source and test whether it separates the held\-out source’s malicious examples from the shared benign pool: a leave\-one\-*source*\-out transfer of the*axis itself*, not of a fitted probe\. On Gemma this transfer AUC climbs from0\.740\.74with no suffix to∼0\.93\{\\sim\}0\.93once*any*suffix is appended \(Fig\.[5](https://arxiv.org/html/2610.02413#A3.F5), right\): without a suffix the per\-source malicious directions are nearly orthogonal \(mean pairwise cosine0\.180\.18\), and the suffix rotates them into near\-alignment \(0\.900\.90for the classification rungs\)\.\(ii\) A direct look\.A t\-SNE of Gemma’s readout colored by label \(Fig\.[5](https://arxiv.org/html/2610.02413#A3.F5), left\) shows the same thing: with no suffix, malicious and benign interleave; under the classification suffix they separate\.

![Refer to caption](https://arxiv.org/html/2610.02413v1/figures/mech_tsne_gemma.png)

![Refer to caption](https://arxiv.org/html/2610.02413v1/figures/mech_transfer_ladder.png)

Figure 5:Where it has headroom, the suffix aligns the malicious representation \(Gemma\)\.*Top:*t\-SNE of Gemma’s single\-position readout across the ladder, colored malicious/benign; classes interleave with no suffix and separate under the classification suffixes\.*Bottom:*leave\-one\-source\-out transfer of a mean\-difference malicious axis\. Gemma climbs0\.74→0\.930\.74\\to 0\.93; Llama and Qwen start already\-aligned \(0\.930\.93,0\.960\.96\) and barely move\.This is a*headroom*effect, not a universal transformation\. Llama and Qwen already carry a well\-aligned malicious axis with no suffix \(transfer AUC0\.930\.93and0\.960\.96; Fig\.[5](https://arxiv.org/html/2610.02413#A3.F5), bottom\), so there is little left for a suffix to build, precisely why their detection gains are the smallest of the three \(§[4\.2](https://arxiv.org/html/2610.02413#S4.SS2)\)\. The t\-SNE is illustrative; the transfer AUC is the quantitative claim\.

### C\.3Format/content dissociation and ROC detail

Figure[6](https://arxiv.org/html/2610.02413#A3.F6)plots each suffix by its mean change vs\. no\-suffix in shared\-benign AUC \(ranking\) and TPR@1% \(precision\), one panel per model: the content\-freeA/Band namedintentsuffixes share the same ranking gain \(xx\) butintentis far higher on precision \(yy\): the format/content dissociation\. Figure[7](https://arxiv.org/html/2610.02413#A3.F7)shows the Llama macro\-ROC:A/Bmatchesintenton area yet drops below no\-suffix at the strict1%1\\%\-FPR operating point, where naming the real criterion helps most\.

![Refer to caption](https://arxiv.org/html/2610.02413v1/figures/dissociation.png)Figure 6:The format/content dissociation\.Each point is a suffix, by its mean change vs\. no\-suffix in shared\-benign AUC \(xx, ranking\) and TPR@1% \(yy, precision\); one panel per model\.A/Bandintentshare the samexxbutintentis far higher onyy\(≈\\approx2828pp on Llama\); neutralreflectsits near the origin \(except Gemma, where it already lifts precision\)\.![Refer to caption](https://arxiv.org/html/2610.02413v1/figures/roc_llama.png)Figure 7:Llama macro\-ROC \(malicious vs\. shared benign, 9 LODO folds\)\.intent\(0\.936\) andA/B\(0\.934\) lift over no\-suffix \(0\.898\);reflect\(0\.893\) does not\. Curves cross near FPR==1% \(dotted\):A/Bmatchesintenton area but drops below no\-suffix at the strict operating point\.±1​σ\\pm 1\\sigma==cross\-dataset spread\.