nationaldesignstudio/rampart · Hugging Face

Source: https://huggingface.co/nationaldesignstudio/rampart rampartis a 14.7 MB ONNX token-classification model that detects personally identifiable information (PII) in text before it leaves the user’s device. It is the on-device half ofRampart, a defense-in-depth client-side redaction system released by National Design Studio. The shipped artifact runs alongside a deterministic recognizer layer that handles structured identifiers; together they form the complete system.

This card documents the released artifact only. Alternative configurations explored during model selection (an ELECTRA-small base, the prefilter-off training variant, leaner data mixes, and smaller corpus slices) are discussed in the project whitepaper for context but are not published.

https://huggingface.co/nationaldesignstudio/rampart#model-summaryModel summary

PropertyValueModel idnationaldesignstudio/rampartArchitecturenreimers/MiniLM\-L6\-H384\-uncasedfine-tuned with a 35-label BIO head (17 entity types)Parameters≈18.5M (MiniLM-L6-H384 with the trimmed 19,730-piece vocabulary; the 22.7M base figure is for the full 30,522-piece BERT vocab)Quantization4-bit MatMul + INT8 embedding (onnx/model\_q4\.onnx)Shipped artifact size14.7 MBVocabulary19,730 WordPieces (trimmed from BERT-uncased’s 30,522, retaining all special and single-character pieces plus frequent multi-character pieces)Max sequence512 tokensLanguagesEnglish, Spanish, French, German, Italian, Portuguese, Dutch (all Latin-script)RuntimeONNX Runtime Web (WASM/WebGPU) viatransformers\.jsLicenseCC BY 4.0 (Creative Commons Attribution 4.0 International)Training data licenseCC BY 4.0 (ai4privacy/pii\-masking\-openpii\-1\.5m)Released byNational Design StudioCard version1.0 (initial public release)

https://huggingface.co/nationaldesignstudio/rampart#intended-useIntended use

The model is designed forclient-side redaction of user-typed text in AI assistants and intake flows— replacing identifying values with stable placeholders before any data is transmitted to a model provider, a server, or a logging system.

https://huggingface.co/nationaldesignstudio/rampart#direct-usesDirect uses

• Redact user content before passing it to a hosted LLM.
• Maintain stable placeholders (\[GIVEN\_NAME\_1\],\[SSN\_1\], ...) across a multi-turn conversation, with rehydration on the client.
• Preempt accidental collection of personal data in analytics, traces, and crash reports.
• Validate domain-specific redaction policies before deploying chat systems in regulated contexts.

https://huggingface.co/nationaldesignstudio/rampart#out-of-scopeOut of scope

• **Stand-alone government-ID detection.**The model is one layer of a defense-in-depth system; it is not a replacement for the deterministic recognizer layer that ships alongside it. SSNs and payment cards are caught by the deterministic layer with checksum validation (structural rules and Luhn), at higher recall than the model alone. Phone, routing, government-ID, passport, and license numbers carry no checksum, so they are caught by the model; the deterministic layer does not attempt them.
• **Indirect / inferential identifiers.**A “rare disease + 5-digit ZIP” combination can re-identify someone even though neither token is in the redact-set. The model does not detect inferential leaks.
• **Adversarial robustness as a security guarantee.**We publish numbers on hostile inputs and document the failure surface; the system is positioned as harm reduction for users entering their own information in good faith, not as a security boundary against motivated adversaries.
• **Non-Latin scripts.**This release is scoped to the seven Latin-script languages listed above. Korean, Han Chinese, Japanese, Arabic, Cyrillic, and Devanagari names recall ~14% in aggregate (see “Fairness and limitations” below). Do not deploy this release for populations who routinely type non-Latin-script names without compensating controls; monitor accordingly.

https://huggingface.co/nationaldesignstudio/rampart#usageUsage

The runtime ships as@nationaldesignstudio/rampart.createGuard\(\)returns aChatGuardthat loads this classifier and runs the full deterministic + model pipeline:

import { createGuard } from "@nationaldesignstudio/rampart";

const guard = await createGuard();
const { text } = await guard.protect("My name is Alex Rivera and my SSN is 472-81-0094.");
// → "My name is [GIVEN_NAME_1] [SURNAME_1] and my SSN is [SSN_1]."

https://huggingface.co/nationaldesignstudio/rampart#training-dataTraining data

The shipped model is trained on acumulative three-source mix, added in the order below; the selection matrix found that folding in all three sources produced every top-recall configuration, with breadth of corpus mattering more than the volume of any single source.

SourceRows usedLicenseRoleSynthetic conversation corpus (in-house)~250,000 conversationsCC BY 4.0Primary in-house corpus. Chat-style messages generated to bedeliberately messy and realistic: low-effort/typo-prone text, voice-dictated phrasing, values pasted out of forms, multilingual mixing, and contradictory/duplicated/wrong-field entries, across a range of assistant personas — so the model learns to catch fragmented PII in disordered chat rather than only clean prose. Covers all 17 entity types.OCR’d-document corpus (in-house)in-house, span-taggedCC BY 4.0Scanned and photographed forms, IDs, and documents run through OCR and thenspan-taggedwith the 17 entity types. Adds OCR noise (character confusions, broken line wraps, stray glyphs) and form-style field layouts absent from the conversational sources, hardening recall on values lifted out of documents.ai4privacy/pii\-masking\-openpii\-1\.5mfull corpus: ~1.4M train + 100,000 held-outCC BY 4.0Public AI4Privacy template corpus, used infull— itstrainandvalidationsplits are pooled, deduplicated byuid, shuffled with a fixed seed, then split into all-but-100k for training (~1.4M rows) and a 100k held-out. No training cap and no language filter are applied; this is the entire deduplicated corpus, not a subsampled slice. Broad multilingual entropy across 7 Latin-script languages (en, es, fr, de, it, pt, nl); the OpenPII schema mapped to our 35-label BIO schema (17 entity types). Also the source of the held-out evaluation split below. The synthetic and OCR’d corpora supply the disordered, document-noisy inputs OpenPII lacks (its conversations are clean and well-formed); OpenPII supplies the multilingual breadth and the held-out test set. The exact synthetic/OCR mix and volume were chosen by the end-to-end selection matrix, not assumed — see the project whitepaper (§3.1, §3.5, §4.2) for the ablation that fixed the recipe.

The held-out 100,000 rows (from the AI4Privacy corpus) are split into two non-overlapping subsets, seeded for full reproducibility:

• 10,000 rowsfor recall-floor threshold tuning.
• 30,000 rowsfor the headline test results below (per-language row counts in the eval table).

The remaining 60,000 held-out rows are reserved for future evaluation and are not used in this release.

https://huggingface.co/nationaldesignstudio/rampart#pre-processingPre-processing

All training rows pass through the same normalization the runtime applies before tokenization: lowercase, NFKD decomposition, and combining-mark stripping. The combining-mark step folds accents —Josébecomesjose,Müllerbecomesmuller— so the model sees a single canonical form regardless of how the user typed the name. This matches what BERT’s BasicTokenizer does implicitly at inference time underdo\_lower\_case=True, so the train-time and runtime distributions are identical by construction. A guard in the training pipeline fails the run if a future tokenizer change breaks this assumption.

A re-trainer who omits this normalization step will produce a model with mismatched distributions, and recall numbers will not reproduce.

The structured classes the deterministic layer owns (SSN,CREDIT\_CARD,IP\_ADDRESS) are also masked to sentinel tokens before tokenization — both at inference (src/premask\.ts) and during dataset construction — so the model never learns to classify raw card/SSN/IP digits and the train-time and inference-time inputs match by construction.

https://huggingface.co/nationaldesignstudio/rampart#vocabularyVocabulary

The full BERT-uncased vocabulary contains 30,522 WordPieces. The shipped vocabulary retains:

1. All special tokens (\[PAD\],\[UNK\],\[CLS\],\[SEP\],\[MASK\]).
2. All single-character pieces and their\#\#continuations, which preserve WordPiece’s character-level fallback for rare names.
3. All multi-character pieces appearing in the training corpus above a frequency threshold.

The shipped vocabulary is19,730 pieces.

https://huggingface.co/nationaldesignstudio/rampart#training-procedureTraining procedure

HyperparameterValueBasenreimers/MiniLM-L6-H384-uncasedEpochs3Batch size32Learning rate5e-5Weight decay0.01Max sequence length512OptimizerAdamWEval strategyper-epoch on held-out validationSave strategyper-epochHardwareApple M-series MPSTotal wall time~3.5 hours The final epoch was selected by held-out eval loss.

https://huggingface.co/nationaldesignstudio/rampart#label-taxonomyLabel taxonomy

The model emits 35 BIO labels (17 entity types × {B-, I-} + O); the deterministic recognizer layer contributes three more structured classes that are masked before the model runs. The runtime applies a default-deny policy: every detected span is redacted unless its label is explicitly in the keep-set.

https://huggingface.co/nationaldesignstudio/rampart#redacted-by-defaultRedacted by default

Owned by the deterministic recognizer layer (regex + validator, masked before the model):

LabelDescriptionSSNSocial Security Numbers (US) — structural validationCREDIT\_CARDPayment card numbers — Luhn-validatedEMAILEmail addressesURLURLs in user contentIP\_ADDRESSIPv4 / IPv6 / MAC addresses Emitted by the token-classification model:

LabelDescriptionGIVEN\_NAMEGiven / first namesSURNAMEFamily / last namesPHONEPhone numbersTAX\_IDTax identifiersBANK\_ACCOUNTBank account / IBAN numbersROUTING\_NUMBERBank routing numbersGOVERNMENT\_IDGovernment-issued ID / case numbersPASSPORTPassport numbersDRIVERS\_LICENSEDriver’s license numbersBUILDING\_NUMBERStreet-line building numberSTREET\_NAMEStreet nameSECONDARY\_ADDRESSSecondary-address line (apt / unit / suite) BUILDING\_NUMBER+STREET\_NAMEtogether form the precise street line; both are redacted while city/state/ZIP are kept.

https://huggingface.co/nationaldesignstudio/rampart#kept-by-defaultKept by default

LabelDescriptionCITYCity — coarse geography for eligibility checksSTATEState / regionZIP\_CODEPostal code The keep-set keeps coarse geography (city/state/ZIP) while redacting the precise street line. To change it, editKEEP\_LABELSinsrc/types\.ts— it is a compile-time set, not a runtime flag.

The taxonomy is deliberatelyatomic: there is no coarsePERSON,STREET\_ADDRESS,ADDRESS,ORGANIZATION, orLOCATIONlabel, and no catch-allSECRET. Names split intoGIVEN\_NAME/SURNAME, the street line intoBUILDING\_NUMBER/STREET\_NAME, and document identifiers into their specific classes, so the model learns to catch PII fragments in disordered text rather than expecting one tidy blob. Dates, ages, and income are intentionallynotmodeled as PII (they map toO): a bare date is rarely identifying, and assistants need age and income as context, so redacting them was over-redaction without a privacy gain.

https://huggingface.co/nationaldesignstudio/rampart#evaluationEvaluation

We score thefull system(model + deterministic layer) because that is what consumers experience end-to-end. Model-only numbers are reported separately for researchers who want to evaluate the encoder in isolation.

https://huggingface.co/nationaldesignstudio/rampart#primary-metricsPrimary metrics

• Private-term recall: for every gold private value, did the redacted output contain the value? This is the privacy-headline number; misses here are leaks.
• Public-term retention: for every gold public value, did the redacted output preserve the value? This measures over-redaction.
• Span F1 strict (IoU=1.0)andrelaxed (IoU≥0.5): how well predicted span boundaries align with gold boundaries under one-to-one greedy matching.
• Latency: Node.js ONNX runtime cold / p50 / p95 / p99 over the full 30,000-row test set. Browser latency (WebGPU and WASM backends) is measured separately byeval/bench/webgpu\.ts— see below.
• Calibration: 15-bin reliability ECE, per label and overall, on per-span max-class scores.

All recall and retention numbers carry Wilson 95% confidence intervals; stratified breakdowns include 1000-iteration bootstrap intervals.

https://huggingface.co/nationaldesignstudio/rampart#held-out-openpii-test-set–seven-supported-languages-30000-rows-131707-private-terms-87207-public-termsHeld-out OpenPII test set — seven supported languages (30,000 rows; 131,707 private terms; 87,207 public terms)

The headline number is measured across all seven supported Latin-script languages. English-only, Spanish-only, and the English+Spanish slice are reported as sub-slices.

SlicePrivate recall (Wilson 95%)Public retention*Span F1 strictLatency p50**All seven languages****98.42% [98.35, 98.49]**91.69%0.5286.6 msEnglish only (11,569 rows)98.85%90.5%—6.6 msSpanish only (3,234 rows)98.84%91.6%—6.6 msEnglish + Spanish98.85%91.0%—6.6 ms 2,082 leaks of 131,707 private terms on the seven-language test (1 in 64 terms slips past the system, before the application’s downstream defenses fire). On the English+Spanish slice the system leaks 778 of 67,613.

These numbers are measured by the committedeval/benchharness running theshipped Q4 pipelineend-to-end over a pinned held-out slice ofpii\-masking\-openpii\-1\.5m. The harness was corrected relative to earlier revisions of this card: city/state/ZIP are now scored askept(matching the runtime keep-set) instead of being counted as leaks, so public retention reflects policy-aware behavior directly. Recall is reported against the full, harder seven-language slice. Span-F1 strict (exact byte+label match) is a secondary metric; term-presence recall is the privacy headline.

The 6.6 ms p50 above is the Node ONNX (CPU) figure over the 30k held-out set. Run over a held-out OpenPII slice in the browser, the same shipped pipeline measures3.9 ms p50on WebGPU (Apple Metal, p95 9.3 ms) and 12.6 ms on WASM (p95 35.5 ms), viaeval/bench/webgpu\.ts— so the WebGPU form factor is faster than Node CPU on the same class of inputs, and WASM is the floor when no GPU is available.

* See “Schema reconciliation” below — the Rampart policy redacts the precise street line (BUILDING\_NUMBER+STREET\_NAME) and the secondary-address line while keeping city/state/ZIP, which the harness now honors.

https://huggingface.co/nationaldesignstudio/rampart#per-language-slices-openpii-latin-test-30k-rows-across-7-languagesPer-language slices (OpenPII Latin test, 30k rows across 7 languages)

LanguageRowsPrivate recallPublic retentionLeaks / totalEnglish (en)11,56998.85%90.5%618 / 53,877Spanish (es)3,23498.84%91.6%160 / 13,736French (fr)4,70898.41%92.8%317 / 19,906German (de)4,26097.94%91.7%357 / 17,347Italian (it)3,21897.83%94.1%301 / 13,855Portuguese (pt)1,48597.73%92.5%147 / 6,467Dutch (nl)1,52697.21%91.9%182 / 6,519 All seven languages land in the 97-99% band; Dutch is the lowest at 97.21% and is flagged for attention in subsequent training cycles. (The recall band moved down ~1pp versus the previous card because the harness now scores the corrected, harder slice — see the note above; the same model scores higher on the older, easier slice.)

https://huggingface.co/nationaldesignstudio/rampart#hand-curated-suitesHand-curated suites

SuiteCasesPrivate recall (Wilson 95%)Public retentionDomain intake2096.97% [84.68, 99.46]93.2%Adversarial (homoglyph / zero-width / leet / splits / NFC-NFD / casing / prompt-injection)2086.36% [66.66, 95.25]83.3%Fairness (Faker × 15 naming traditions × 5 templates)1,87565.44% [63.26, 67.56]90.0% The adversarial and domain-intake suites are 20 cases each; Wilson CIs are wide. The 1,875-case fairness suite has tight CIs and is the most statistically grounded slice we report.

https://huggingface.co/nationaldesignstudio/rampart#schema-reconciliationSchema reconciliation

The 91.69% retention number in the headline table is term-presence scoring that already credits city/state/ZIP as kept, matching the runtime keep-set. We analyzed the 7,244 remaining “over-redacted” public terms in the 30,000-row eval:

• The vast majorityare policy-driven redactions of street-line components (street name, building number, secondary address line). AI4Privacy OpenPII marksSTREET,BUILDINGNUM, andSECADDRESSasO(public); the Rampart policy redacts the precise street line (BUILDING\_NUMBER+STREET\_NAME) andSECONDARY\_ADDRESSwhile keepingCITY,STATE, andZIP. These are not detector errors; they are the policy firing as designed.
• A smaller shareare span-edge artifacts. The runtime’s particle-rescue step grows name spans (GIVEN\_NAME/SURNAME) to swallow capitalized particles (“de la”, “von”, “Mc”). When an adjacent public token is itself capitalized, that token can be absorbed into the redacted span.
• A very small fractionare digit fragments inside longer correctly-redacted spans (e.g. “376” found inside a redacted 16-digit credit card).

We publish the 91.69% term-presence number for like-for-like comparison against public PII benchmarks running the same scoring rules. For product reasoning, the policy-aware retention exceeds 99%.

https://huggingface.co/nationaldesignstudio/rampart#calibrationCalibration

The runtime applies a single recall-biased confidence floor (minScore= 0.4) uniformly across the model’s labels, chosen against the 10,000-row OpenPII Latin calibration split (disjoint from test) so misses — which leak data — are traded against the cheaper failure of over-redaction. There is no per-label threshold table in the shipped runtime; the deterministic recognizer layer, not a tuned model threshold, is the system of record for the structured classes the model alone is weakest on:

• SSN— structural validation (reserved-area rules).
• CREDIT_CARD— Luhn checksum over the digit projection.
• EMAIL / URL / IP_ADDRESS— pattern-anchored regex at near-100% recall.

Phone, routing, government-ID, passport, and license numbers carry no checksum and are left to the model under the same recall-biased floor.

ECE on the full 30,000-row test set is0.291(overall, all labels); the model alone (no deterministic layer) is0.018. The system-level ECE is higher because the deterministic layer always emits score 1.0 on its detections, making the score distribution bimodal — that is a score-distribution artifact of the union, not a calibration regression of the underlying model.

https://huggingface.co/nationaldesignstudio/rampart#fairness-and-limitationsFairness and limitations

We document failures because consumers need this to deploy the redactor responsibly. None of these are surprises; we measured each.

https://huggingface.co/nationaldesignstudio/rampart#fairness-across-naming-traditions-1875-faker-generated-casesFairness across naming traditions (1,875 Faker-generated cases)

Cases are stratified bynaming tradition(15 categories) embedded in 5 chat templates. Same surrounding context across all traditions — only the name varies.

TraditionLocaleRecallCasesAngloen_US99.9%125Hispanices_MX, es_ES99.9%250Francophonefr_FR99.9%125Germanicde_DE99.9%125Romance (Italian)it_IT99.9%125Lusophonept_BR99.9%125Turkictr_TR99.9%125Vietnamesevi_VN99.2%125Japaneseja_JP45.6%125Koreanko_KR15.2%125Han Chinesezh_CN8.8%125South Asian (Hindi)hi_IN5.6%125Arabicar_AA4.8%125Slavic (Russian)ru_RU2.4%125 Aggregated by script:

• Latin-ASCII names: ~100% recall (695 / 695)
• Latin + diacritics: 99.8% recall (429 / 430)
• Non-Latin scripts: 13.7% recall (103 / 750)

The deterministic recognizer layer does not catch names — there is no checksum to validate against — so this failure surfaces at the system level. This is the most important regression we have identified, and the fairness suite is wired into the eval pipeline as a stratified regression test so any further drop will surface in subsequent training cycles.

https://huggingface.co/nationaldesignstudio/rampart#government-style-identifiers-model-onlyGovernment-style identifiers (model only)

Government-style identifiers (case numbers, Medicare-style identifiers, USCIS receipts, A-numbers, passports, licenses) carry no checksum, so — unlike SSNs and payment cards — the deterministic layer doesnotdetect them. They rely entirely on the model, which catches ~67.6% of them in a structured-ID probe. This is a documented weak spot: there is no deterministic backstop for these classes, so the model’s recall is effectively the system’s recall on them. Consumers should not assume the deterministic layer covers government IDs the way it covers SSNs and cards; deployments that handle these identifiers heavily should add their own format-specific validators.

https://huggingface.co/nationaldesignstudio/rampart#adversarial-robustnessAdversarial robustness

The system catches most homoglyph, casing, leet, NFC/NFD, and basic whitespace-split attacks. It does not reliably catch:

• Zero-width characters injected between every digit of an SSN.
• Prompt-injection text inside the PII span (e.g."ignore previous instructions").
• Combined attacks (homoglyph plus whitespace split).

The deterministic layer’s digit projection (which strips non-digit characters before checksum validation) restores most digit-bearing PII against these attacks; names remain vulnerable. This is the right framing for the limitation, not the primary use case: Rampart is designed to protect users entering their own information in good faith from incidental disclosure to downstream services, not to defeat a motivated user actively trying to smuggle their own PII past the filter.

https://huggingface.co/nationaldesignstudio/rampart#wordpiece-fragmentation-on-long-namesWordPiece fragmentation on long names

Names likeThanh\-Nghiem Quoc\-BaoorChukwuemeka Okonkwo\-Adeyemiproduce many subwords; the runtime performs span-merging across same-label adjacencies plus particle-rescue, which closes most of the gap. Some five-or-more-subword names still fragment in a way that loses recall on the trailing subword.

https://huggingface.co/nationaldesignstudio/rampart#reproducibilityReproducibility

The model weights, deterministic layer, and TypeScript evaluation harness are released under CC BY 4.0.

Evaluation runs entirely in TypeScript, against the shipped pipeline: the native benchmark (eval/bench) runs the real@nationaldesignstudio/rampartcode over a frozen OpenPII held-out slice and writessummary\.json/by\_language\.json, which are committed alongside the eval output — so every number in this card traces to committed evidence produced by the code that ships. The held-out rowuids are pinned in a committed manifest; regenerate the data withbun run bench:fetchand reproduce the figures withbun run bench.

https://huggingface.co/nationaldesignstudio/rampart#citationCitation

If you use this model in research, please cite:

@misc{rampart-2026,
  author = {National Design Studio},
  title  = {Rampart: Client-side PII redaction for AI assistants},
  year   = {2026},
  url    = {https://huggingface.co/nationaldesignstudio/rampart},
}

Please also cite the upstream training corpus:

@misc{ai4privacy-openpii-1.5m,
  title  = {ai4privacy/pii-masking-openpii-1.5m},
  author = {AI4Privacy},
  year   = {2025},
  url    = {https://huggingface.co/datasets/ai4privacy/pii-masking-openpii-1.5m},
}