stuntd is a local proxy server that learns from your traffic to serve typed LLM decisions locally using the Laya model, compatible with the Jev protocol and requiring no API key.
What it is. stuntd is a local proxy for typed LLM decisions: choice, yes/no, score. Your app keeps calling its provider through it. stuntd records the decisions, trains a small head on top of Laya (the open-weights System One model; the encoder stays frozen), serves it in shadow mode, and switches to live only once it agrees with the provider at your target. Below the confidence threshold the request still goes to the provider. If agreement drops, it demotes itself. It also speaks the Jev protocol. Point typesafe-sdk at http://127.0.0.1:8787 and Laya answers POST /v1/systemone locally, zero-shot, no key. Put any Jev-compatible server behind it (the paid API, kev, SemIf, LLM2Jev, laya-server) and it learns from that server's answers until it can answer itself. Why. Jev is fast and good, but it is a hosted API: about 380 ms round trip, paid per token, early access. Laya is open and answers in ~22 ms on a laptop GPU, but zero-shot it falls apart when the label set is large: on banking77 (77 intents) it scores 38% where Jev scores 76% (numbers from dhruvmehra/jevbench). A head trained on your own traffic closes that gap, and nobody had wired that into a proxy with a fallback loop. Quickstart, local Jev, no key: pip install "stuntd[train]" stuntd config init stuntd serve Client: TypeSafeClient(api_key="anything", base_url="http://127.0.0.1:8787"). Numbers (RTX 5060 laptop, everything reproducible from the repo; all teachers are rules or an oracle because I had no Jev key): Head answer p50: 22 ms CUDA, 60 ms CPU, 28 ms through the daemon over TCP. Snake, BFS oracle as teacher, same seed: zero-shot Laya scores 0.7 per game; after 30 teacher games and 138 s of training the head plays 99.6% of the moves itself and scores 11.4 (the oracle: 21). Side-by-side GIF in the README. Support triage (3 questions per request): 69/34/66% agreement with the teacher zero-shot, 99/86/98% trained, 90% of questions answered locally at the default 0.99 target. Banking intents over 12 labels: 89.5% zero-shot, 100% trained. A command gate for coding agents (allow/ask/deny): 45% zero-shot, 97% trained, 96% local. Training caches the frozen encoder's output once per site, so 24 epochs over 3000 rows take 2 to 6 minutes. What it is not. It only learns closed decisions, never free text. The encoder is frozen, so it learns what is in the words, not arithmetic over fields: a risk rule written as "amount > X" scored 0.42; the same rule written as "first transfer to this payee, larger than usual" scored 0.94. Anthropic, Responses and Gemini traffic pass through untouched for now. Every serve loads the 1.6 GB checkpoint. Also in the box: a 20-line PreToolUse hook that asks a local command_gate question before Claude Code runs a shell command. Repo: https://github.com/bladedevoff/stuntd, Apache-2.0. I would like to hear which decisions in your pipelines are closed-choice, and whether you would hand them to a 20 ms local model.
Stuntd is an open-source local proxy that records LLM decision calls, trains a lightweight model to handle them locally, reducing costs while maintaining high agreement with the teacher model.
Laya is an open-source, fast multilingual decision engine that offers non-autoregressive, calibrated probabilities for structured schemas, claiming to be 6-8 times faster than Jev with full openness.
The author shares a DIY Jev-like inference setup using open weight LLMs, demonstrating that simple prompting with logit-based verification achieves good accuracy without fine-tuning, and provides a rust web server for local deployment.
AgentJev-0.6B is a small open-source AI decision model that outperforms Laya on Typed Decisions with improved accuracy and significant computation reduction, enabling faster local inference.
OpenJev is an open-source server offering a free API compatible with TypeSafe's Jev for probabilistic predictions, with performance benchmarks showing competitive latency.