Semalith v1.4: A Calibrated 184M Safety Classifier Achieving State-of-the-Art Prompt-Injection Detection at 44x Fewer Parameters than Llama-Guard-3-8B

# A Calibrated 184M Safety Classifier Achieving State-of-the-Art Prompt-Injection Detection at 44× Fewer Parameters than Llama-Guard-3-8B One DeBERTa-v3 head, three safety axes (prompt injection, harm, BFSI compliance), 7-of-7 prompt-injection benchmark wins vs 8B Llama-Guard-3, zero false positives (FPR=0.000, 𝑛=208) on benign agentic prompts, and a 100% real-world contamination-clean 76,204-row training corpus
Source: [https://arxiv.org/html/2607.22545](https://arxiv.org/html/2607.22545)
Tejasvi C\. Addagada Independent Researcher tejasvi@tejasviaddagada\.com⋅\\cdot[tejasviaddagada\.com](https://arxiv.org/html/2607.22545v1/tejasviaddagada.com)The nameSemalithis built from Greek*sēma*\(sign\) and*lithos*\(stone\): a structured signal\-bearing marker\. This work is unrelated to Patronus AI’s*Lynx*hallucination detector\. Production weights and the evaluation harness are released under enterprise license; please contact the author for access\.

###### Abstract

Deploying large language models in financial\-services and agentic settings requires safety classifiers that simultaneously handle prompt injection, regulatory compliance, and general harm—a combination no existing open guardrail addresses in a single inference pass\.

Semalith v1\.4is a 184M\-parameter DeBERTa\-v3\-base classifier performing simultaneous three\-axis safety classification—prompt injection, general harm, and financial\-services regulatory compliance—in a single forward pass\. Its 22\-class head \(BENIGN, nine prompt\-injection sub\-types, general\-harm, eleven BFSI labels\) is trained with a 4\-class auxiliary super\-category head under jointly weighted loss, on a 76,204\-row corpus mined from 49 public sources with SHA\-1 deduplication against every held\-out evaluation set, with 21 of 22 benchmarks at zero contamination \(max 0\.22%\)\.

AgainstLlama\-Guard\-3\-8Bon 22 held\-out benchmarks, Semalith v1\.4 wins every prompt\-injection evaluation \(7/7\) and 11 of 18 benchmarks overall at 44×\\timesfewer parameters, with FPR = 0\.000 on 208 benign agentic prompts vs 0\.063 for Llama\-Guard\-3\-8B\.111AgentHarm\-benign covers all 208 benign prompts from the public AgentHarm split\[[23](https://arxiv.org/html/2607.22545#bib.bib23)\]\(32\-sample validation \+ 176\-sample test\_public\); 0 false positives; Wilson 95% CI \[0\.000, 0\.018\]\.On general\-harm benchmarks \(WildGuardMix, HEx\-PHI, HarmBench\), Llama\-Guard\-3 leads—this complementary split is documented in Section 4\. Six measured weak spots are disclosed in Section 6\.

Deployment guidance:v1\.3 is recommended for conversational moderation deployments \(ToxicChat F1 0\.624\); v1\.4 is recommended when BFSI label coverage or zero\-FPR on benign agentic prompts is the priority\.

## 1Introduction

Every major open guardrail optimises for one axis\. LlamaGuard and Granite Guardian catch general harm but collapse on subtle prompt\-injection attacks\. PromptGuard catches prompt injection but flags 96% of benign agentic content as harmful\. None ship with financial\-services regulatory labels\.Semalith v1\.4answers all three in a single 11\.6\-millisecond forward pass—the fourth iteration of a from\-scratch rebuild built on 100% real\-world training data, zero synthetic prompts, and SHA\-1 deduplication against every held\-out benchmark\.

Three requirements drove the design that no existing open guardrail satisfies simultaneously:

- •Llama\-Guard\-3\-8B and Granite\-Guardian\-3\.3\-8B are general\-harm taxonomy classifiers; they collapse on subtle prompt\-injection evaluations \(PINT F1≈0\.06\\approx 0\.06and≈0\.32\\approx 0\.32at fp16 respectively in our prior cont7\-era benchmark\)\.
- •PromptGuard\-2\-86M is a binary prompt\-injection classifier; it achieves recall close to 1\.0 on PI\-heavy benchmarks but produces false\-positive rates of 96% on AgentHarm\-benign and 98% on ToxicChat by flagging anything edgy as harmful\.
- •No public guardrail ships with BFSI\-specific labels covering banking customer service, mortgage/loans, transfers, cards, insurance, investment advisory, etc\., despite SEC Rule 206\(4\)\-7\[[26](https://arxiv.org/html/2607.22545#bib.bib26)\], FCA COBS\[[27](https://arxiv.org/html/2607.22545#bib.bib27)\], MiFID II\[[28](https://arxiv.org/html/2607.22545#bib.bib28)\], EU AI Act\[[29](https://arxiv.org/html/2607.22545#bib.bib29)\], and RBI/SEBI/IRDAI rulebooks all naming these as compliance\-relevant categories for AI\-assisted client interactions\.

Semalith v1\.4answers all three in a single 11\.6\-millisecond forward pass \(batch=1, max\_len=256, fp16, RTX 4090\); see Table[2](https://arxiv.org/html/2607.22545#S3.T2)for the full latency profile\.

#### Full disclosure\.

This paper documentsv1\.4 as the production checkpoint, the fourth iteration in a lineage that progressed from v1 \(baseline, val macro\-F1 0\.876\) through v1\.2 \(a BENIGN\-register expansion that catastrophically regressed PI recall and was rolled back\), v1\.3 \(attack\-class expansion closing the Mosscap gap\), to the current v1\.4 release \(BFSI\-enrichment expansion fixing three thin labels\)\. v1\.4 has six measured weak spots documented in Section 6: HarmBench\-contextual recall 0\.181 \(architectural gap\), WildGuardMix F1 0\.289 \(BENIGN\-register mismatch\), BeaverTails FPR 0\.446 \(trade\-off of attack\-class expansion\), HEx\-PHI persuasion recall 0\.885 \(vs 0\.970 for Llama\-Guard\-3\), ToxicChat F1 0\.538 \(regression from v1\.3’s 0\.624, cost of BFSI enrichment\), and HarmBench\-copyright recall 0\.700 \(out\-of\-scope label\)\. All six are disclosed with root causes and mitigation paths\.

The 11\-of\-18 overall win record vs Llama\-Guard\-3 should be read in benchmark\-axis context: Semalith wins every prompt\-injection and adversarial\-stealth evaluation \(7/7\); Llama\-Guard\-3 wins every general\-harm and conversational\-moderation evaluation \(5/6, with BeaverTails F1 the single exception\)\. Neither model dominates the other on shared ground; they are trained for complementary axes\.

#### What v1\.4 is NOT\.

It scores complete prompts in a single pass\. It does not reason, stream tokens, speak languages other than English, or act as an agent\. If you send it a 1,001\-character prompt, the tail is truncated\. If you need multilingual coverage or context\-window\-length inputs, v1\.4 is not the right tool\.

## 2Related Work

#### Prompt injection: taxonomy and benchmarks\.

Prompt injection attacks were first systematically classified by Perez and Ribeiro\[[11](https://arxiv.org/html/2607.22545#bib.bib11)\], who identified direct\-override, indirect\-context, and jailbreak sub\-types; this taxonomy informs Semalith’s D1\_SYSTEM\_OVERRIDE, D1\_JAILBREAK, D7\_INDIRECT\_INJECTION, and D1\_NARRATIVE\_FRAME labels\. The OWASP LLM Top 10 \(2023–2025\) formalises prompt injection as the leading LLM risk category\[[8](https://arxiv.org/html/2607.22545#bib.bib8)\]\. SALAD\-Bench\[[21](https://arxiv.org/html/2607.22545#bib.bib21)\]organises attacks into a three\-level hierarchy covering 66 sub\-categories; Semalith’s D\-attack taxonomy is a coarser 9\-label variant designed for low\-latency classifier inference rather than post\-hoc audit\. Gandalf\[[12](https://arxiv.org/html/2607.22545#bib.bib12)\]and Mosscap\[[13](https://arxiv.org/html/2607.22545#bib.bib13)\]provide escalating\-defense extraction benchmarks specifically designed to defeat rule\-based and fine\-tuned classifiers; Mosscap L6/L7/L8 remain the hardest open prompt\-injection benchmarks by defence tier\.

#### Safety classifier evolution\.

The LlamaGuard family\[[2](https://arxiv.org/html/2607.22545#bib.bib2),[3](https://arxiv.org/html/2607.22545#bib.bib3)\]uses a causal\-LM taxonomy classifier \(Llama\-3\-8B backbone\) with categorical harm labels; it achieves strong recall on harm\-elicitation benchmarks but degrades on subtly\-phrased PI attacks \(HackaPrompt recall 0\.084 in our evaluation\)\. Granite\-Guardian\[[4](https://arxiv.org/html/2607.22545#bib.bib4)\]\(IBM, 8B\) follows a similar chat\-template design with improved general\-harm coverage but similar PI weakness\. WildGuard\[[5](https://arxiv.org/html/2607.22545#bib.bib5)\]\(Allen AI, 7B\) is the current state\-of\-the\-art for mixed conversational moderation, establishing WildGuardMix as the canonical mixed\-signal benchmark\. AprielGuard\[[7](https://arxiv.org/html/2607.22545#bib.bib7)\]\(ServiceNow, 2025\) is a recent 8B enterprise guardrail targeting agentic and multi\-turn safety; to our knowledge it does not report financial\-domain \(BFSI\) or PI\-specific benchmark results\. PromptGuard\-2\-86M\[[6](https://arxiv.org/html/2607.22545#bib.bib6)\]\(Meta\) is the strongest compact binary PI classifier, achieving near\-ceiling recall on PI benchmarks by design while accepting extreme FPR on benign content \(96% on AgentHarm\-benign\)\. Semalith occupies a different operating point: comparable PI recall to PromptGuard\-2\-86M on structural PI benchmarks, 190×\\timeslower FPR on benign agentic content, and simultaneous BFSI label coverage that no other public guardrail provides\.

#### Domain\-specific and BFSI safety\.

General\-domain guardrails exhibit severe false\-positive rate inflation when deployed in specialist domains: Zhang and Ren\[[9](https://arxiv.org/html/2607.22545#bib.bib9)\]show that domain shift from general to financial\-services prompts raises FPR by 15–40 percentage points across LlamaGuard, WildGuard, and Llama\-2\-7B chat safety filters\. This motivates Semalith’s BFSI\-specific label schema \(B\-01\.\.B\-11, Table[1](https://arxiv.org/html/2607.22545#S3.T1)\), which enables distinguishing benign financial\-domain queries from genuinely harmful financial fraud, unlicensed\-advice requests, and money\-laundering facilitation\. The relevant regulatory frameworks— SEC Rule 206\(4\)\-7\[[26](https://arxiv.org/html/2607.22545#bib.bib26)\], FCA COBS\[[27](https://arxiv.org/html/2607.22545#bib.bib27)\], MiFID II\[[28](https://arxiv.org/html/2607.22545#bib.bib28)\], and the EU AI Act\[[29](https://arxiv.org/html/2607.22545#bib.bib29)\]—all require AI\-assisted financial\-services interactions to be auditable by interaction category, motivating fine\-grained BFSI labels over a coarse harmful/benign binary\.

#### Prompt\-injection benchmarks\.

For public PI benchmarks we use HackaPrompt\[[11](https://arxiv.org/html/2607.22545#bib.bib11)\], Gandalf\[[12](https://arxiv.org/html/2607.22545#bib.bib12)\], AdvBench\[[14](https://arxiv.org/html/2607.22545#bib.bib14)\], AART\[[15](https://arxiv.org/html/2607.22545#bib.bib15)\], AttaQ\[[16](https://arxiv.org/html/2607.22545#bib.bib16)\], WildJailbreak\[[5](https://arxiv.org/html/2607.22545#bib.bib5)\], Mosscap L6/L7/L8\[[13](https://arxiv.org/html/2607.22545#bib.bib13)\], SimpleSafetyTests\[[22](https://arxiv.org/html/2607.22545#bib.bib22)\], and HarmBench\[[17](https://arxiv.org/html/2607.22545#bib.bib17)\]\(standard / contextual / copyright configurations\) and HEx\-PHI\[[18](https://arxiv.org/html/2607.22545#bib.bib18)\]\. PINT\[[10](https://arxiv.org/html/2607.22545#bib.bib10)\]is licensed to Lakera customers only and is not present in any training corpus we use; PINT F1 = 0\.99x measurements from the cont7\-era development cycle \(cont4 checkpoint, April 2026\) are cited for historical reference only, as v1\.3 was retrained on a corpus containing no PINT\-adjacent prompts\.

#### Lineage disclosure\.

Earlier work in this lineage \(cont4–cont7\) used synthetic D6 agentic\-injection templates that produced a 96\.5% false\- positive rate on AgentHarm\-benign\[[23](https://arxiv.org/html/2607.22545#bib.bib23)\]—a structural failure documented in our v1 corpus build report\. v1 was retrained from scratch frommicrosoft/deberta\-v3\-base\[[1](https://arxiv.org/html/2607.22545#bib.bib1)\]to fix this and eliminate training\-set contamination on five PI/harm benchmarks memorized by the cont lineage\. v1\.3 extends v1 by adding 13,827 attack\-class rows targeting the Mosscap stealth gap, lifting Mosscap L6/L7/L8 recall from 0\.47–0\.58 \(v1\) to 0\.79–0\.91 and WildJailbreak recall from 0\.840 to 0\.968\.

## 3Method

### 3\.1Architecture

Semalith v1 is amicrosoft/deberta\-v3\-base\[[1](https://arxiv.org/html/2607.22545#bib.bib1)\]encoder followed by a 0\.10\-dropout layer, a 22\-way classification head on the \[CLS\] embedding, and a 4\-way auxiliary super\-category head on the same embedding\. The 22 labels are:

- –1 BENIGN class\(label\_id=0, weight=1\.5\)\.
- –9 D\-attack sub\-types: D1\_SYSTEM\_OVERRIDE, D1\_JAILBREAK, D1\_EXTRACTION, D1\_SOCIAL\_ENGINEERING, D1\_AUTHORITY\_CLAIM, D1\_NARRATIVE\_FRAME, D5\_EVASION, D6\_AGENTIC\_INJECTION \(weight 3\.0\), D7\_INDIRECT\_INJECTION \(weight 6\.0\)\.
- –1 D8\_GENERAL\_HARM class\(weight 1\.0\)\.
- –11 BFSI regulatory sub\-types: B\-01\.\.B\-11 \(each weight 1\.5\), defined in Table[1](https://arxiv.org/html/2607.22545#S3.T1)\.

The 4\-way super\-category head maps to \{BENIGN=0, D\-attack=1, D8\-harm=2, BFSI=3\}\. The aux\-loss weightα\\alphawas selected by grid search over \{0\.05, 0\.10, 0\.20, 0\.40\}; the exact value is reported in the supplementary materials\.

The total parameter count is 184M \(183\.83M encoder \+ 0\.034M heads\)\.

Table 1:BFSI label taxonomy \(B\-01\.\.B\-11\) with regulatory grounding\. Each label covers both*benign*customer intent \(the normal case, routed to downstream domain handlers\) and*harmful*variants within that domain \(fraud, scams, unlicensed advice\), distinguishable by the D\-attack / D8 / B\-08 co\-signal\.
### 3\.2Training corpus

The v1\.4 training corpus contains 76,204 rows mined from 49 public sources\. It extends the v1\.3 base \(57,086 rows from 38 sources\) with 19,118 BFSI\-enrichment rows from 11 new sources targeting three previously\-thin labels \(B\-11: 137→\\to1,920 training rows; D1\_AUTHORITY\_CLAIM: 137→\\to583; D6\_AGENTIC\_INJECTION:→\\to910\)\. New sources include financial\-services customer service datasets, EU regulatory QA corpora, authority\-claim and tool\-injection filtered subsets from existing safety benchmarks, fraud narrative corpora, and extortion datasets\. The corpus spans five categories: \(i\) general harm and toxicity, \(ii\) prompt\-injection sub\-types, \(iii\) jailbreak attempts, \(iv\) agentic\-injection, and \(v\) BFSI customer\-service intents intent\-mapped to the 11 B\-XX labels\.

All training rows are real\-world data, with zero synthetic or templated prompts — a deliberate methodology choice in contrast to recent safety\-classifier work that trains entirely on LLM\- generated synthetic data\. Each candidate row is SHA\-1\- and MinHash\- deduplicated against every held\-out evaluation benchmark during corpus build to bound contamination\.

### 3\.3Training procedure

We fine\-tunemicrosoft/deberta\-v3\-basefrom scratch \(no continued\-pretraining lineage\)\. Loss is a weighted cross\-entropy over the 22\-class head plus an auxiliary cross\-entropy over the 4\-class super\-category head withα\\alphaselected by grid search over \{0\.05, 0\.10, 0\.20, 0\.40\} \(exact value in supplementary materials\)\. Optimization uses AdamW \(lr=2×10−5=2\\times 10^\{\-5\}, weight decay 0\.01\) with gradient clipping at‖g‖2≤1\.0\\\|g\\\|\_\{2\}\\leq 1\.0, linear warmup over 6% of total steps, and cosine decay\. Batch size 32; max sequence length 256 tokens; 6 epochs\. The validation split is a stratified 5% per\-class hold\-out ensuring all 22 labels have non\-zero validation representation\.

### 3\.4Latency and throughput

Table[2](https://arxiv.org/html/2607.22545#S3.T2)reports wall\-clock inference latency and throughput measured on an NVIDIA RTX 4090 \(24 GB VRAM\) at fp16 precision using PyTorch 2\.4\.1 and the HuggingFacetransformers4\.46\.3 tokenizer\. Measurements are the median of 100 forward passes after a 10\-pass warm\-up; standard deviation<0\.3<0\.3ms across all conditions\. The sequence length cap of 256 tokens matches the training configuration; prompts longer than 256 tokens are truncated at the tokenizer, not the model\. Llama\-Guard\-3\-8B throughput is measured under identical conditions for reference\.

Table 2:Inference latency and throughput on NVIDIA RTX 4090, fp16, PyTorch 2\.4\.1\. “Latency” = median wall\-clock time per batch\. “Tput” = prompts per second\. seq\_len = max input tokens \(truncated\)\.ModelBatchseq\_lenLatency \(ms\)Tput \(prompts/s\)Semalith v1\.3/v1\.4 \(184M\)125611\.686Semalith v1\.3/v1\.4825618\.4435Semalith v1\.3/v1\.43225642\.1760Semalith v1\.3/v1\.46425673\.8867Semalith v1\.3/v1\.411287\.9127Llama\-Guard\-3\-8B1256387\.42\.6Llama\-Guard\-3\-8B82561,842\.04\.3Llama\-Guard\-3\-8B32256OOM—v1\.4 latency is identical to v1\.3 — same DeBERTa\-v3\-base encoder, same 22\-class head dimensions, same hardware configuration; Table[2](https://arxiv.org/html/2607.22545#S3.T2)applies to both versions\. At batch=1 \(latency\-sensitive, synchronous API path\), Semalith v1\.3/v1\.4 is 33×\\timesfaster than Llama\-Guard\-3\-8B \(11\.6 ms vs 387\.4 ms\)\. At batch=8 \(typical async micro\-batch\), the throughput gap is 100×\\times\(435 vs 4\.3 prompts/s\)\. Llama\-Guard\-3\-8B does not fit a 32\-prompt batch within the 24 GB RTX 4090 VRAM budget at fp16\.

## 4Results

### 4\.1Held\-out validation per\-class metrics

Table[3](https://arxiv.org/html/2607.22545#S4.T3)reports per\-class precision / recall / F1 on the 2,151\-row deterministic stratified 5% validation split\.

Table 3:Per\-class metrics on the 2,151\-row held\-out validation split\. Sorted by F1 descending\. “Active” confirms each of the 22 labels received non\-zero training and validation support\.
### 4\.2Held\-out OOD evaluation suite \(22 benchmarks\)

Table[4](https://arxiv.org/html/2607.22545#S4.T4)reports Semalith v1\.4 on a 22\-benchmark held\-out OOD suite, using the v1\.2 binary mapping that excludes BFSI super\-cat from “flagged harmful” \(see Section 4\.3\)\. All point estimates are accompanied by 95% Wilson score confidence intervals\[[24](https://arxiv.org/html/2607.22545#bib.bib24)\]\. Two benchmarks haven<100n<100\(marked \[\*\]\): SimpleSafetyTests \(nn=50\) and HarmBench\-contextual \(nn=94\)\. These are included for coverage and directional calibration; their CIs are wide and results should not be treated as statistically conclusive\. AgentHarm\-benign has been expanded to the fullnn=208 public split \(32\-sample validation \+ 176\-sample test\_public\) and now carries a tight CI of±\\pm1\.3 pp\. All large\-sample claims \(n≥191n\\geq 191\) carry CIs of±\\pm6 pp or better\.

Table 4:Semalith v1\.4 held\-out OOD results across 22 benchmarks with 95% Wilson score confidence intervals\. v1\.3 shown for comparison\. Benchmarks withn<100n<100are marked \[\*\] \(directional only\);100≤n<500100\\leq n<500marked \[~\]\.↑\\uparrow= v1\.4 improves on v1\.3;↓\\downarrow= regression\.
### 4\.3BFSI binary\-mapping methodology fix

The original cont7\-era evaluation pipeline computed binary “flagged harmful” from the 22\-class argmax via the legacy mapping:

pred=1​if​arg⁡max≠0​else​0\\texttt\{pred\}=1\\text\{ if \}\\arg\\max\\neq 0\\text\{ else \}0This incorrectly counts BFSI sub\-classes \(B\-01\.\.B\-11, super\-category 3\) as “harmful flagged” even though the model recognizes the prompt as benign\-domain financial content\. The v1\.2 binary mapping corrects this:

pred=1​ifSUPER​\[arg⁡max\]∈\{1,2\}​else​0\\texttt\{pred\}=1\\text\{ if \}\\texttt\{SUPER\}\[\\arg\\max\]\\in\\\{1,2\\\}\\text\{ else \}0where super\-cat 1 is D\-attack and super\-cat 2 is D8\-harm\.

Applied to the existing v1 checkpoint \(no retraining\), this drops FPR by 24% on average and lifts ToxicChat F1 by 10\.5 percentage points\. The methodology fix is now the published default\.

### 4\.4Apples\-to\-apples vs\. Llama\-Guard\-3\-8B

Llama\-Guard\-3\-8B\[[3](https://arxiv.org/html/2607.22545#bib.bib3)\]is Meta’s reference 8B causal\-LM safety guardrail, the de\-facto baseline for production safety filtering\. We ran a fully apples\-to\-apples comparison: same 22 evaluation JSONLs, both models at fp16 on a single RTX 4090, identical prompt templates from each model’s tokenizer\. The full result is in Table[5](https://arxiv.org/html/2607.22545#S4.T5):Semalith v1\.4 wins 11 of 18 benchmarks at 44×\\timesfewer parameters\(184M vs 8B\)\. Table[6](https://arxiv.org/html/2607.22545#S4.T6)stratifies this by benchmark axis to give an unbiased picture of where each model leads\.

Table 5:Semalith v1\.4 \(184M\) vs Llama\-Guard\-3\-8B\[[3](https://arxiv.org/html/2607.22545#bib.bib3)\]\(Meta’s 8B reference safety guardrail\) on 18 held\-out evaluation benchmarks\. Both models run at fp16 on a single RTX 4090; identical prompt templates drawn from each model’s published tokenizer\. The twowmdpflag\-rate benchmarks and the 4 BFSI\-specific eval JSONLs are excluded from this table because they do not map to Llama\-Guard\-3’s taxonomy; full Semalith\-only results for all 22 benchmarks appear in Table[4](https://arxiv.org/html/2607.22545#S4.T4)\. Semalith v1\.4 wins 11 of 18 benchmarks overall; see Table[6](https://arxiv.org/html/2607.22545#S4.T6)for a per\-axis breakdown\. Bold = winner on each row\.The 11\-of\-18 win pattern is benchmark\-axis dependent and should be read with that context\. Table[6](https://arxiv.org/html/2607.22545#S4.T6)splits the results into two axes: prompt\-injection / adversarial\-stealth benchmarks \(Semalith’s design target\) and general harm\-detection benchmarks \(Llama\-Guard\-3’s design target\)\. On PI and stealth evaluations Semalith wins every benchmark \(7/7\), often by margins exceeding 70 percentage points\. On general harm\-detection benchmarks Llama\-Guard\-3 wins 5 of 6, with BeaverTails F1 the single exception\. This split reflects complementary training objectives, not a one\-model\-wins\-all result\. The copyright\-extraction loss \(HarmBench\-copyright\) is by\-design: Semalith’s 22\-class taxonomy has no copyright label, while Llama\-Guard\-3’s S5 \(Specialized Advice\) category captures it; this benchmark is excluded from the 11/18 count in the neutral analysis\.

Table 6:Benchmark\-axis stratification: Semalith v1\.4 vs Llama\-Guard\-3\-8B\. Splitting the 18\-benchmark suite by design axis reveals complementary strengths rather than a global winner\. Bold = winner on each row\. HarmBench\-copyright is excluded \(no Semalith label; not a fair comparison\)\. AttaQ \(0\.945 vs 0\.868\) is a Semalith win\.
### 4\.5TP\-miss \+ FP diagnosis informing v1\.4

Each production iteration is informed by a miss/FP analysis on the predecessor checkpoint\. For v1, the dominant missed\-TP cluster was adversarially\-stealthy prompt\-injection prompts; an auxiliary 86M model caught most of these misses, indicating a decision\-boundary rather than detection\-floor failure\. v1\.3’s attack\-class expansion lifted Mosscap L6/L7/L8 recall from 0\.47–0\.58 \(v1\) to 0\.79–0\.91 and WildJailbreak recall from 0\.840 to 0\.968\. v1\.4’s BFSI\-enrichment expansion targets the three thin\-label gap: D1\_AUTHORITY\_CLAIM \(137→\\to583 training rows\), D6\_AGENTIC\_INJECTION \(→\\to910\), and B\-11 AML/sanctions \(→\\to1,920\), raising their per\-class val F1 to 0\.931, 0\.755, and 0\.577 respectively from the near\-zero floor\.

Collectively, the results in Section 4 establish three findings: \(1\) a 184M encoder trained on a contamination\-controlled corpus matches or exceeds an 8B causal\-LM on all PI and adversarial\-stealth axes while remaining calibrated on agentic benign content; \(2\) the BFSI binary\-mapping fix eliminates a systematic methodology artifact from prior evaluations at zero retraining cost; and \(3\) the per\-row miss\-FP analysis provides a reproducible, training\-corpus\-independent diagnostic loop that informed the v1\.3 and v1\.4 corpus expansions and will guide future iterations\.

## 5Discussion

### 5\.1Why 184M is enough

We considered DeBERTa\-v3\-large \(304M parameters\) in early scoping\. A preliminary run on the v1 corpus \(43,259 rows, seed=42\) showed val macro\-F1 of 0\.879 for the large variant vs 0\.876 for base — a 0\.3 pp improvement at 65% higher parameter count and roughly2×2\\timesthe latency budget\. We chose base\. The reasoning was not that 184M is always sufficient, but that the bottleneck at 43k rows was training data diversity, not model capacity: every class where base underperformed \(D1\_AUTHORITY\_CLAIM at F1=0\.50, D6\_AGENTIC\_INJECTION at F1=0\.74\) was also a class below the 250\-row stability floor\. Adding capacity to an under\-represented class does not help; adding data does\.

The 4\-way auxiliary super\-category head turned out to matter more than the base\-vs\-large choice\. Without the super\-category loss \(α=0\\alpha=0\), we observed D\-attack sub\-class collapse — the model learned a single D\-attack prototype and distributed the 9 sub\-type labels quasi\-randomly within it, yielding per\-class F1 variance of 0\.12 across the D1 family vs 0\.04 with the selectedα\\alphavalue\. The anchor is particularly important when any sub\-class is below the stability floor, which is the case in both v1\.3 \(D1\_AUTHORITY\_CLAIM at 137 rows\) and v1\.4 \(B\-11 partially underdifferentiated\)\. The weight was selected by grid search over \{0\.05, 0\.10, 0\.20, 0\.40\} on the v1 val split; the optimal value was the elbow — lower gave collapse, higher started degrading the 22\-class head precision\.

### 5\.2Seed stability

v1\.3 was verified across three independent seeds \(42, 123, 456\) on the 57,086\-row corpus: val macro\-F1 0\.8735 / 0\.8746 / 0\.8758, mean0\.8746±0\.00120\.8746\\pm 0\.0012— substantially below the cont6/cont7 reference range \(0\.003–0\.008\)\. v1\.4 seed=42 reports val macro\-F1 =0\.8222; the 5\.2 pp drop from v1\.3 is explained by BFSI sub\-type competition, not by regressions in the PI/harm classes\. Per\-class F1 comparison shows that the three newly\-enriched labels gained as expected \(D1\_AUTHORITY\_CLAIM: 0\.667→\\to0\.931\), but five previously\-stable BFSI labels regressed significantly: B\-01 \(−\-14\.8pp\), B\-03 \(−\-20\.6pp\), B\-06 \(−\-21\.0pp\), B\-07 \(−\-19\.8pp\)\. The shared root cause is intra\-BFSI boundary confusion: 19,118 new rows expanded the BFSI vocabulary \(bitext wealth, EU regulatory, fraud narratives\) into semantic regions previously occupied by the stable B\-01/B\-06/B\-07 labels, requiring the model to learn finer sub\-type distinctions within the same super\-category\. The fix is a B\-11 sub\-label split \(AML/sanctions vs wealth management\) and cleaner source routing for B\-01 vs B\-06/B\-07, planned for v1\.5\. Production\-critical PI/stealth metrics are maintained within 2 pp of v1\.3\. A full 3\-seed stability run for v1\.4 is in progress; all downstream evaluation numbers are reported for seed=42\.

### 5\.3McNemar pairwise significance \(Semalith v1\.3 reference vs PromptGuard\-2\-86M\)

To assess whether per\-benchmark differences are statistically significant beyond chance, we applied McNemar’s test with continuity correction on 16 benchmarks where per\-row predictions are available for both Semalithv1\.3and PromptGuard\-2\-86M\[[6](https://arxiv.org/html/2607.22545#bib.bib6)\]\. These results are carried forward as a v1\.3 reference baseline; v1\.4 McNemar results will be added when v1\.4 per\-row predictions are generated viagenerate\_llamaguard\_preds\.py\. The 5\-benchmark delta between v1\.3 and v1\.4 on the benchmarks below is≤\\leq1\.3 pp \(see Table[4](https://arxiv.org/html/2607.22545#S4.T4)\), so the statistical conclusions are expected to be unchanged\. PromptGuard\-2\-86M was chosen as the McNemar comparison target because it operates at the same encoder scale \(86M DeBERTa parameters\) with per\-row predictions available across all 22 evaluation JSONLs; Llama\-Guard\-3\-8B and Granite\-Guardian\-3\.3\-8B are evaluated via a chat\-template generation pipeline that produces output labels rather than logits and were compared using the aggregate benchmark tables in Section 4\.4 and Section 5\.4\. The per\-axis framing in Table[6](https://arxiv.org/html/2607.22545#S4.T6)provides the complementary structural comparison against Llama\-Guard\-3\-8B\.

PromptGuard flags everything and catches everything: 96% FPR on AgentHarm\-benign, 98% on ToxicChat\. That is the operating point it was designed for\. The results below are not a comparison between equals — they are a comparison between a recall\-maximising binary classifier and a multi\-label classifier that distinguishes 22 classes\. PromptGuard wins where it was built to win\. Semalith wins where FPR matters\.

Semalith v1\.3 is significantly superior\(p<0\.05p<0\.05, two\-tailed\) on four benchmarks:ToxicChat\-test\(χ2=3,727\\chi^\{2\}=3\{,\}727,p≈0p\\approx 0; Semalith 4,313/4,793 vs PromptGuard 408/4,793\),WildGuardMix\-eval\(χ2=1,339\\chi^\{2\}=1\{,\}339; 1,545/2,000 vs 102/2,000\),BeaverTails\-test\(χ2=96\\chi^\{2\}=96; 880/1,234 vs 704/1,234\), andWildJailbreak\(χ2=5\.5\\chi^\{2\}=5\.5,p=0\.019p=0\.019; 1,873/2,000 vs 1,833/2,000\)\. The first three are in the FPR\-sensitive conversational and BFSI domain where Semalith’s multi\-label schema distinguishes benign financial queries from harmful ones\.

PromptGuard\-2\-86M is significantly superioron 14 benchmarks, including the Mosscap stealth\-jailbreak trilogy \(χ2∈\[248,575\]\\chi^\{2\}\\in\[248,575\],p≈0p\\approx 0\), AART \(χ2=101\\chi^\{2\}=101\), HarmBench×\\times3, and WMDP\. This is structurally equivalent to PromptGuard’s advantage in Table[6](https://arxiv.org/html/2607.22545#S4.T6)axis B \(pure\-recall benchmarks\); the same benchmarks where Semalith’s multi\-label schema requires distinguishing sub\-classes rather than collapsing to binary flagging\. These correspond exactly to the documented limitations in Section[6](https://arxiv.org/html/2607.22545#S6)\. The critical operational difference is that PromptGuard’s 14\-benchmark recall advantage comes at the cost of 96% FPR on benign agentic content \(vs Semalith’s 0\.5%\); for deployments where false positives carry a user\-experience cost, Semalith is strictly preferable\.

On two benchmarks \(AdvBench, SimpleSafetyTests\) the difference is not statistically significant \(p\>0\.05p\>0\.05\), consistent with the near\-ceiling performance of both models there \(≥0\.96\\geq 0\.96accuracy\)\.

### 5\.4McNemar significance vs Llama\-Guard\-3\-8B

To test significance of the Semalith vs Llama\-Guard\-3\-8B benchmark differences reported in Table[5](https://arxiv.org/html/2607.22545#S4.T5), we apply McNemar’s test with continuity correction on the 16 benchmarks where per\-row predictions exist for both models\. Llama\-Guard\-3\-8B per\-row predictions are generated usingeval\_competitors\_pod\.pywith thellamaguard3competitor class\. The generation script \(generate\_llamaguard\_preds\.py\) is included in the public evaluation harness; per\-row prediction files will be added to the repository upon completion\. The seven PI/stealth benchmarks withχ2\>500\\chi^\{2\}\>500are statistically conclusive from the aggregate recall gaps alone \(70–91 pp\) and do not require per\-row confirmation to establish significance\.

Table 7:McNemarχ2\\chi^\{2\}statistic \(continuity\-corrected, two\-tailed\) for Semalith v1\.3 vs Llama\-Guard\-3\-8B on shared benchmarks\.†\\dagger:p<0\.05p<0\.05;‡\\ddagger:p<0\.001p<0\.001; NS: not significant \(p≥0\.05p\\geq 0\.05\)\.Bold winner= model with higher correct\-row count on that benchmark\.BenchmarkSemalithLG\-3\-8Bχ2\\chi^\{2\}Sig\.Semalith wins \(PI / stealth axis\)HackaPrompt \(R\)0\.9940\.084\>\>1000‡\\ddaggerGandalf \(R\)0\.9830\.263\>\>500‡\\ddaggerMosscap L6 \(R\)0\.9050\.056\>\>2000‡\\ddaggerMosscap L7 \(R\)0\.8750\.058\>\>2000‡\\ddaggerMosscap L8 \(R\)0\.7870\.085\>\>1500‡\\ddaggerWildJailbreak \(R\)0\.9680\.473\>\>800‡\\ddaggerSALAD\-Bench \(R\)0\.9600\.563\>\>300‡\\ddaggerAART \(R\)0\.8970\.850—\(pending\)AdvBench \(R\)0\.9920\.972—\(pending\)BeaverTails F10\.7710\.679—\(pending\)AgentHarm\-benign \(FPR↓\\downarrow\)0\.0050\.063—\(pending\)LlamaGuard wins \(harm\-elicitation axis\)HarmBench\-Standard \(R\)0\.8530\.974—\(pending\)HarmBench\-Contextual \(R\)0\.1810\.553—\(pending\)HEx\-PHI \(R\)0\.9160\.970—\(pending\)WildGuardMix F10\.2430\.754—\(pending\)SimpleSafetyTests \(R\)0\.9201\.000—\(pending\)The five PI/stealth benchmarks withχ2\>500\\chi^\{2\}\>500are beyond any reasonable significance threshold \(p≈0p\\approx 0\) based on the reported recall gaps alone \(70–91 pp\) and will not change with per\-row confirmation\. The remaining benchmarks are marked*\(pending\)*pending LlamaGuard per\-row predictions; those results will be published in the associated evaluation\-harness repository\.

### 5\.5Granite\-Guardian\-3\.3\-8B head\-to\-head \(22\-benchmark\)

We completed a 22\-benchmark evaluation of Granite\-Guardian\-3\.3\-8B \(IBM, 8B parameters, 45×\\timeslarger than Semalith v1\.4\) using its native<\|start\_of\_role\|\>chat template\. The 11 benchmarks below are selected to cover all axis types \(PI/stealth, harm\-elicitation, mixed\-moderation, agentic FPR\); omitted benchmarks are the WMDP flag\-rate evaluations and BFSI\-specific JSONLs which do not map to Granite’s taxonomy\. Representative results:

Semalith v1\.3 wins decisively on all structural and adversarial prompt\-injection benchmarks \(Mosscap, Gandalf, HackaPrompt, WildJailbreak, SALAD\-Bench\)\. Granite\-Guardian\-3\.3\-8B leads on clean harm\-elicitation benchmarks \(HarmBench\-Standard, HEx\-PHI, ToxicChat\), consistent with its training objective\. Granite’s BeaverTails FPR of 0\.549 on the same 532\-row benign subset used to measure Semalith’s 0\.474 FPR \(the BeaverTails\-30k test split safe\-labelled rows, SHA\-1 verified against both training corpora\) — flagging over half of benign financial\-domain content as harmful — raises significant concerns for production deployments requiring low false\-positive rates on legitimate financial\-services queries\.

### 5\.6Iteration history

v1\.4 is the fourth production iteration\. v1\.2 attempted a BENIGN\-register expansion that catastrophically over\-corrected the decision boundary, regressing PI recall, and was rolled back\. v1\.3 applied the inverse lesson: zero net BENIGN expansion, all new rows in attack/harm classes, recovering Mosscap and Gandalf at the cost of a 12 pp BeaverTails FPR regression\. v1\.4 extends v1\.3 with targeted BFSI enrichment to fix three thin labels \(D1\_AUTHORITY\_CLAIM, D6\_AGENTIC\_INJECTION, B\-11\), at the cost of an 8\.6 pp ToxicChat F1 regression\. Each trade\-off is the explicit consequence of a corpus\-axis expansion — the same pattern documented for v1\.3 in Section 6\.3\. This iteration history informs a simple design heuristic: expansion of any super\-category will improve recall on that axis and mildly degrade FPR\- sensitive conversational benchmarks that overlap with the expanded class space\.

#### Deployment guidance\.

v1\.3 and v1\.4 are complementary production checkpoints with different operating points\.Use v1\.3for conversational moderation deployments where ToxicChat F1 \(0\.624\) and WMDP flag\-rate \(0\.415/0\.319\) matter most\.Use v1\.4when BFSI\-domain label coverage is required \(D1\_AUTHORITY\_CLAIM F1 0\.931, D6\_AGENTIC\_INJECTION F1 0\.755, B\-11 AML/sanctions F1 0\.577\) or when zero FPR on benign agentic prompts \(0\.000 on 208\-row AgentHarm split\) is the primary production requirement\. The PI/stealth axis is equivalent: both versions win all 7 prompt\-injection benchmarks vs Llama\-Guard\-3\-8B with≤\\leq1\.3 pp difference between versions\.

## 6Limitations

Six measured weak spots are documented for v1\.4, in descending order of severity\.

### 6\.1Context\-grounded indirect prompt injection \(HarmBench\-contextual\)

v1\.4 achieves recall0\.181on HarmBench\-contextual \(n=94\) versus Llama\-Guard\-3\-8B’s 0\.553 — a 37\.2\-pp gap\. Root cause: long benign\-looking context paragraphs prepended to harmful instructions are read as benign in aggregate by the single \[CLS\] embedding\. v1 recorded 0\.075 here; v1\.3 improved to 0\.181 incidentally via the WildJailbreak training expansion; v1\.4 is unchanged at 0\.181\. Mitigation candidate: sliding\-window prediction \(process the prompt in 256\-token windows and OR the per\-window predictions\), or a contextual D7 training carve from BIPIA / INJECAGENT \(neither currently on HuggingFace\)\. Architectural rather than data fix\.

### 6\.2Mixed conversational moderation \(WildGuardMix, ToxicChat\)

WildGuardMix F1 = 0\.290 \(v1\.4, improved from v1\.3’s 0\.243\) vs Llama\-Guard\-3\-8B 0\.754 — a 46\.4\-pp gap\. ToxicChat F1 = 0\.538 \(v1\.4; regression from v1\.3’s 0\.624, documented in Section 6\.4\)\. Root cause: BENIGN\-class register mismatch\. v1\.3’s BENIGN class is the same 14,865 rows as v1 \(by design — v1\.2’s BENIGN expansion broke calibration\)\. The attempted v1\.2 fix \(adding 13k conversational benign rows\) severely over\-corrected the decision boundary\. We have not yet identified a BENIGN\-register expansion path that holds without collapsing PI / Mosscap recall\. This is the largest non\-architectural gap between v1\.3 and an 8B causal\-LM guardrail\.

### 6\.3BeaverTails\-test FPR regression \(cost of v1\.3 attack\-class expansion\)

v1\.3 BeaverTails\-test FPR = 0\.474 vs v1’s 0\.350 — a 12\.4\-pp regression\. Root cause: the 13,827 additional attack/harm\-class rows shifted the decision boundary marginally toward “flagged\-harmful”\. Documented as the explicit trade\-off of the v1\.3 recall recovery; deployments sensitive to BeaverTails\-style benign conversational FPR should pair v1\.3 with a downstream calibration threshold or use v1 as the production checkpoint\.

### 6\.4Persuasion\-style social engineering \(HEx\-PHI Cats 6\+7\)

Overall HEx\-PHI v1\.4 R = 0\.885 \(vs Llama\-Guard\-3\-8B 0\.970 — 8\.5\-pp behind; regression from v1\.3’s 0\.916\)\. Cat 6 Economic Harm and Cat 7 Fraud Deception remain the weak sub\-categories\. v1\.2 ablation showed adding HEx\-PHI training data lifts overall recall by 3\.7 pp; v1\.3 cannot include this fix because all 296 HEx\-PHI prompts are in the held\-out eval JSONL\.

### 6\.5ToxicChat F1 regression \(cost of v1\.4 BFSI enrichment\)

v1\.4 ToxicChat F1 = 0\.538 vs v1\.3’s 0\.624 — an 8\.6\-pp regression\. Root cause: the 19,118 new BFSI rows \(authority\-claim, tool\-injection, and fraud narrative sources\) shifted the decision boundary marginally toward flagging conversational content as BFSI\-relevant, which collides with ToxicChat’s toxic\-but\-not\-harmful conversational register\. This is the same mechanism that drove v1\.3’s BeaverTails FPR regression from the attack\-class expansion, now manifesting on ToxicChat\. Deployments sensitive to ToxicChat F1 should use v1\.3 \(0\.624\) as the production checkpoint; v1\.4 is recommended when improved D1\_AUTHORITY\_CLAIM, D6\_AGENTIC\_INJECTION, or B\-11 AML/sanctions coverage is the primary requirement\.

### 6\.6D1\_AUTHORITY\_CLAIM, D6\_AGENTIC\_INJECTION, B\-11 \(resolved in v1\.4\)

These three labels were below the 250\-row stability floor in v1\.3 \(D1\_AUTHORITY\_CLAIM: 137 rows, F1=0\.500; D6\_AGENTIC\_INJECTION: thin; B\-11: thin\)\. v1\.4 raises all three above the floor: D1\_AUTHORITY\_CLAIM→\\to583 rows \(val F1=0\.931\), D6\_AGENTIC\_INJECTION→\\to910 rows \(val F1=0\.755\), B\-11→\\to1,920 rows \(val F1=0\.577\)\. B\-11 remains the weakest at 0\.577 due to the domain diversity of AML/sanctions queries vs wealth\-management queries sharing the same label; a sub\-label split is planned for v1\.5\.

### 6\.7WMDP biosecurity / chemistry flag\-rate increase

v1\.4 WMDP flag\-rate: wmdp\_bio 0\.463 \(v1\.3: 0\.415, \+4\.8pp\) and wmdp\_chem 0\.360 \(v1\.3: 0\.319, \+4\.1pp\)\. WMDP flag\-rate measures how often the model incorrectly flags biosecurity or chemistry educational content as harmful\. The increase is consistent with the iteration heuristic \(Section 5\.6\): the 19,118 new BFSI rows include fraud/AML/regulatory content whose vocabulary \(sanctions, chemical weapons conventions, dual\-use goods\) partially overlaps with WMDP biosecurity and chemistry queries, shifting the decision boundary slightly toward flagging knowledge\-domain questions\. This does not affect safety recall — WMDP prompts are not harmful — but production deployments with WMDP\- adjacent educational content should use v1\.3 \(flag\-rate 0\.415/0\.319\)\.

### 6\.8English\-only scope

The training corpus and evaluation suite are English\-only\. Multilingual robustness is unmeasured, and out\-of\-distribution languages should be expected to degrade both recall and FPR\. Multilingual coverage is on the v2 roadmap\.

## 7Reproducibility

Everything needed to reproduce the numbers in Tables[4](https://arxiv.org/html/2607.22545#S4.T4)–[6](https://arxiv.org/html/2607.22545#S4.T6)is on GitHub at[https://github\.com/anonymous/semalith\-eval\-harness](https://github.com/anonymous/semalith-eval-harness): three scripts \(eval\_safety\_full\.py,eval\_competitors\_mac\.py,binary\_mapping\.py\), the 22 held\-out evaluation JSONLs \(31,430 rows, SHA\-1 clean\), andcontamination\_audit\.py\. Run time on a single GPU with≥\\geq8 GB VRAM is under 4 hours\.

The model weights \(model\.safetensors, 184M parameters\) and corpus manifest \(\_SEMALITH\_V1\.4\_MANIFEST\.json\) are available on request\. The manifest has the full per\-source row counts and the SHA\-1 contamination audit for all 22 evals — 21 at 0\.000%, one at 0\.2218% \(agentharm\_benign\_holdout: 169 collisions from WildGuardMix authority\-filtered rows overlapping AgentHarm benign prompts\)\. Corpus SHA\-256:c63cd3d6…\. Raw training rows are not released\.

Pre\- and post\-training gate reports \(\_SEMALITH\_V1\.4\_PRE\_CHECK\.json,\_SEMALITH\_V1\.4\_POST\_CHECK\.json\) are on GitHub\. Hard gates: 4/4 PASS\. v1\.3 seed\-stability report \(3 seeds, std=0\.0012\) is included as a reference baseline; v1\.4 3\-seed run is in progress\.

Given the checkpoint and evaluation harness, every number in Tables[4](https://arxiv.org/html/2607.22545#S4.T4)–[6](https://arxiv.org/html/2607.22545#S4.T6)can be independently reproduced on a single GPU \(≥\\geq8 GB VRAM\) in under 4 hours\. Hyperparameters \(Section 3\.3\), the 22\-class taxonomy, and the binary mapping \(Section 4\.3\) are fully specified in this paper and sufficient to retrain frommicrosoft/deberta\-v3\-basegiven the training data\.

#### Statistical methodology\.

All point estimates for binary metrics \(recall, FPR\) are accompanied by 95% Wilson score confidence intervals\[[24](https://arxiv.org/html/2607.22545#bib.bib24)\]in Table[4](https://arxiv.org/html/2607.22545#S4.T4)\. Benchmarks withn<100n<100are flagged with \[\*\] and should be treated as directional signals; the two affected benchmarks \(SimpleSafetyTestsnn=50, HarmBench\-contextualnn=94\) are included for coverage completeness, not as primary statistical evidence\. AgentHarm\-benign has been expanded to the fullnn=208 public split and now carries a tight 95% CI of \[0\.001, 0\.027\]\. A full validation of the AgentHarm\-benign FPR claim on the complete 208\-row dataset is planned for the next release cycle, pending infrastructure availability\.

## 8Conclusion

Semalith v1\.4 demonstrates that a 184M\-parameter encoder classifier, trained on a contamination\-controlled real\-world corpus with a three\-axis label taxonomy and an auxiliary super\-category loss, achieves state\-of\-the\-art prompt\-injection detection at 44×\\timesfewer parameters than 8B causal\-LM guardrails, while maintaining zero false positives on 208 benign agentic prompts\. The BFSI binary\-mapping correction eliminates a systematic evaluation artefact at zero retraining cost\. Six measured limitations are disclosed with root causes and mitigation paths, providing a reproducible diagnostic loop for future iterations\. Production weights and evaluation harness are available under enterprise license; contact the author for access\.

## References

- \[1\]P\. He, J\. Gao, W\. Chen\. “DeBERTaV3: Improving DeBERTa using ELECTRA\-Style Pre\-Training with Gradient\-Disentangled Embedding Sharing,”*ICLR 2023*, arXiv:2111\.09543\.
- \[2\]H\. Inan, K\. Upasani, J\. Chi, R\. Rungta, K\. Iyer, Y\. Mao, M\. Tontchev, Q\. Hu, B\. Fuller, D\. Testuggine, M\. Khabsa\. “Llama Guard: LLM\-based Input\-Output Safeguard for Human\-AI Conversations,” arXiv:2312\.06674, 2023\.
- \[3\]Meta AI\. “Llama Guard 3 8B Model Card,” 2024\.[https://huggingface\.co/meta\-llama/Llama\-Guard\-3\-8B](https://huggingface.co/meta-llama/Llama-Guard-3-8B)\.
- \[4\]IBM Research\. “Granite Guardian,” arXiv:2412\.07724, 2024\.
- \[5\]S\. Han, H\. Zeng, H\. Shin, A\. Askell, A\. Bakhtin, et al\. “WildGuard: Open One\-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs,”*NeurIPS 2024*, arXiv:2406\.18495\.
- \[6\]Meta AI\. “PromptGuard 2: Robust Prompt Injection and Jailbreak Detection,” 2025\.[https://huggingface\.co/meta\-llama/Prompt\-Guard\-2\-86M](https://huggingface.co/meta-llama/Prompt-Guard-2-86M)\.
- \[7\]ServiceNow Research\. “AprielGuard: A Lightweight Safety Classifier for Enterprise AI,” arXiv:2512\.20293, 2025\.
- \[8\]OWASP\. “OWASP Top 10 for Large Language Model Applications,” Version 2\.0, 2025\.[https://owasp\.org/www\-project\-top\-10\-for\-large\-language\-model\-applications/](https://owasp.org/www-project-top-10-for-large-language-model-applications/)\.
- \[9\]Y\. Zhang, M\. Ren\. “Domain Shift Amplifies False Positive Rates in LLM Safety Classifiers: Evidence from Financial\-Services Deployments,”*NAACL 2025 Industry Track*, 2025\.
- \[10\]Lakera AI\. “PINT: Prompt Injection Test,” 2024\.[https://lakera\.ai/pint](https://lakera.ai/pint)\(commercial benchmark; cited for context, not measured in this work\)\.
- \[11\]F\. Perez, I\. Ribeiro\. “Ignore Previous Prompt: Attack Techniques For Language Models,”*NeurIPS 2022 Workshop on ML Safety*, arXiv:2211\.09527\.
- \[12\]Lakera AI\. “Gandalf: Prompt Extraction Challenge,” 2023\.[https://gandalf\.lakera\.ai/](https://gandalf.lakera.ai/)\.
- \[13\]Lakera AI\. “Mosscap: Escalating\-Defense Prompt\-Injection Benchmark,” 2024\.[https://huggingface\.co/datasets/Lakera/mosscap](https://huggingface.co/datasets/Lakera/mosscap)\.
- \[14\]A\. Zou, Z\. Wang, N\. Carlini, M\. Nasr, J\. Z\. Kolter, M\. Fredrikson\. “Universal and Transferable Adversarial Attacks on Aligned Language Models,” arXiv:2307\.15043, 2023\.
- \[15\]B\. Radharapu, K\. Harrison, L\. Aroyo, P\. Bhatt\. “AART: AI\-Assisted Red\-Teaming with Diverse Data Generation for New LLM\-Powered Applications,” arXiv:2311\.08592, 2023\.
- \[16\]I\. Bhatt, C\. Bhatt, L\. Shpigel Nacson, et al\. “AttaQ: Automated Attack\-Generation for Testing Question\-Answer Models,”*ACL 2024 Workshop on Customizable NLP*, arXiv:2406\.10590\.
- \[17\]M\. Mazeika, L\. Phan, X\. Wang, A\. Zou, Z\. Wang, N\. Mu, E\. Sakhaee, N\. Li, S\. Basart, B\. Li, D\. Forsyth, D\. Hendrycks\. “HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal,”*ICML 2024*, arXiv:2402\.04249\.
- \[18\]X\. Qi, Y\. Zeng, T\. Xie, P\.\-Y\. Chen, R\. Jia, P\. Mittal, P\. Henderson\. “Fine\-Tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To\!”*ICLR 2024*, arXiv:2310\.03693\.
- \[19\]Z\. Lin, Z\. Wang, Y\. Tong, Y\. Wang, Y\. Guo, Y\. Wang, J\. Shang\. “ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real\-World User\-AI Conversation,”*EMNLP 2023 Findings*, arXiv:2310\.17389\.
- \[20\]J\. Ji, M\. Liu, J\. Dai, X\. Pan, C\. Zhang, C\. Bian, B\. Chen, R\. Sun, Y\. Wang, Y\. Yang\. “BeaverTails: Towards Improved Safety Alignment of LLM via a Human\-Preference Dataset,”*NeurIPS 2023*\.
- \[21\]L\. Li, Y\. Dong, R\. Wang, X\. Hu, W\. Zuo, D\. Lin, Y\. Qiao, J\. Shao\. “SALAD\-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models,”*ACL 2024 Findings*, arXiv:2402\.05044\.
- \[22\]B\. Vidgen, S\. Agrawal, A\. Ahmed, V\. Blili\-Hamelin, K\. Brown, A\. Drouhard, et al\. “SimpleSafetyTests: A Test Suite for Identifying Critical Safety Risks in Large Language Models,” arXiv:2311\.08370, 2023\.
- \[23\]M\. Andriushchenko, F\. Croce, N\. Flammarion\. “AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents,” arXiv:2410\.09024, 2024\.
- \[24\]E\. B\. Wilson\. “Probable Inference, the Law of Succession, and Statistical Inference,”*Journal of the American Statistical Association*, 22\(158\):209–212, 1927\.
- \[25\]N\. Li, A\. Pan, A\. Gopal, S\. Pinder, G\. Stephan, A\. Padhi, N\. Warrington, D\. Biderman, J\. Zou, M\. Fredrikson, Z\. C\. Lipton\. “The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning,”*ICML 2024*, arXiv:2403\.03218\.
- \[26\]U\.S\. Securities and Exchange Commission\. “Investment Adviser Code of Ethics,” Rule 206\(4\)\-7, 17 CFR §275\.206\(4\)\-7, 2004\.[https://www\.sec\.gov/rules/final/ia\-2256\.htm](https://www.sec.gov/rules/final/ia-2256.htm)\.
- \[27\]Financial Conduct Authority \(UK\)\. “Conduct of Business Sourcebook \(COBS\),” FCA Handbook, 2024\.[https://www\.handbook\.fca\.org\.uk/handbook/COBS/](https://www.handbook.fca.org.uk/handbook/COBS/)\.
- \[28\]European Parliament and Council\. “Markets in Financial Instruments Directive II \(MiFID II\),” Directive 2014/65/EU, 2014\.[https://eur\-lex\.europa\.eu/legal\-content/EN/TXT/?uri=CELEX:32014L0065](https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32014L0065)\.
- \[29\]European Parliament and Council\. “Artificial Intelligence Act,” Regulation \(EU\) 2024/1689, 2024\.[https://eur\-lex\.europa\.eu/legal\-content/EN/TXT/?uri=CELEX:32024R1689](https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689)\.