Arc Sentry outperformed LLM Guard 92% vs 70% detection on a head to head benchmark. Here is how it works.
Summary
Arc Sentry is a new pre-generation prompt-injection detector that reads a model’s internal residual stream, achieving 92% detection with 0% false positives versus LLM Guard’s 70%/3.3% on a 130-prompt benchmark.
Similar Articles
LLM Guard scored 0/8 on a USENIX 2025 multi-turn jailbreak. Here’s what caught it instead.
Arc Sentry detects multi-turn jailbreaks like Crescendo by reading model internal state rather than text output, catching attacks that text-based monitors miss entirely.
Gate AI: LLM Security Benchmark Evaluation Methodology and Results
This paper presents an evaluation methodology for LLM security detectors that addresses systematic weaknesses like per-dataset threshold tuning and undisclosed operating points. The framework uses cross-validation across 16 benchmarks, selects a single global operating point, and includes multiple diagnostics for generalization.
I built a proxy that prevents AI agents from taking actions based on hidden instructions. Here are the numbers.
Arc Gate is a proxy that blocks prompt injection attacks by treating all external content as non-instructional, achieving high detection rates (99-100%) with low false positives across multiple benchmarks.
Semalith v1.4: A Calibrated 184M Safety Classifier Achieving State-of-the-Art Prompt-Injection Detection at 44x Fewer Parameters than Llama-Guard-3-8B
Semalith v1.4 is a compact 184M parameter DeBERTa-v3 safety classifier that excels at prompt-injection detection with zero false positives on benign agentic prompts, while also handling general harm and financial compliance in a single pass, outperforming Llama-Guard-3-8B at 44x fewer parameters.
Send a SCOUT First: Pre-hoc Reasoning for Adaptive Detector Allocation in Prompt-Injection Defense
Introduces SCOUT, a framework that dynamically allocates prompt-injection detectors per request by predicting reliability and latency, improving safety and efficiency. Also presents SCOUT-450, a benchmark for complex agent-facing injections, showing a 46% reduction in attack-success rate and 40% latency reduction over a fixed GPT-4o judge.