Describes a conversational AI system that uses Typesafe's Jev non-generative model to select from pre-written responses through parallel classification and scoring, providing a cost-effective and transparent alternative to generative LLMs.
I built a conversational AI that doesn't generate a single token — it selects from 400 pre-written responses using TypeSafe's Jev, a non-generative model that returns probabilistic judgments instead of text. The technical approach: Traditional LLMs generate responses token by token. I flipped this — I wrote 400 responses by hand and let the AI pick the best match. Here's the architecture: Step 1: Classification (400ms, 1 API call) I ask Jev 18 questions in parallel about the user's message: Questions asked simultaneously: - intent (Choice: greeting/complaint/question/etc — 14 options) - formality (Score: 0-4 scale, 5 levels) - emotional_intensity (Score: 0-4 scale) - needs_human (Noul: yes/no probability) - subcategory_greeting (Choice: if intent=greeting, which type?) - subcategory_complaint (Choice: if intent=complaint, which type?) ... [14 speculative subcategory questions, one per intent] All 18 run in one API call using speculative fan-out — code only reads the subcategory answer for whichever intent wins. This eliminates a round trip without extra cost. Step 2: Candidate Fetch (0ms, pure code) The response bank is structured as: { complaint: { empathetic: [ { id: "cmp_emp_1", text: "Three times is way too many..." }, { id: "cmp_emp_2", text: "I hear you — that's frustrating..." }, // ...10 responses total ], solution_oriented: [ ... ] }, greeting: { ... } } Code looks up bank[intent][subcategory] and pulls ~10 candidates. If there are >10 in a category, I run a cheap Noul shortlist (one yes/no question per candidate) to narrow it down. Step 3: Scoring (400ms, 1 API call) Each candidate gets scored on 6 dimensions in parallel: Per candidate (10 candidates × 6 dimensions = 60 questions): - relevance (Score: 0-4) - tone_match (Score: 0-4) - helpfulness (Score: 0-4) - answers_question (Noul: 0-1) - specificity (Score: 0-4) - natural_flow (Score: 0-4) Then I apply context-aware weights: // Default weights weights = { relevance: 0.25, tone: 0.15, helpfulness: 0.20, ... } // High emotion? Tone matters more if (emotional_intensity > 2.5) { weights.tone = 0.30; weights.relevance = 0.20; } // Small talk? Natural flow matters more if (intent === "small_talk") { weights.natural_flow = 0.35; weights.helpfulness = 0.05; } Winner = highest weighted score. Step 4: Output If final score < 0.4 or intent confidence < 0.3, optional LLM fallback kicks in (I use Claude Haiku). Otherwise, return the selected response. Example flow: User: "I'm frustrated, my order has been wrong three times" Classify (400ms): intent: complaint (91% confidence) subcategory: empathetic (speculative answer, used since complaint won) emotional_intensity: 3.8/4.0 formality: 1.2/4.0 needs_human: 0.34 (below 0.8 threshold, proceed) Fetch: 10 responses from complaint/empathetic Score (400ms): Each candidate × 6 dimensions Weights adjusted: tone=30% (emotion is high) Top 3: #1: "Three times is way too many..." → 0.82 #2: "I hear you — that's frustrating..." → 0.77 #3: "That sounds really frustrating..." → 0.70 Winner: Response #1 (highest weighted score) Total: ~800ms, ~$0.0001 per message Why this works: Philosophical questions nail it. Philosophy is about picking the right framing, not generating insights. "What's the purpose of life?" has many valid perspectives — selection feels natural. No hallucinations. Every word was pre-written. Perfect for customer support or anywhere brand safety matters. Full transparency. You see intent → scores → winner. No "the model decided" black box. 100x cheaper. ~$0.0001 vs ~$0.01-0.10 per message. Where it breaks: Can't generate anything. No coding, writing, creative tasks. If it's not in the bank, I got nothing. Limited to 400 responses. Works for bounded conversations (support, FAQs, onboarding). Scaling needs manual work or template responses with dynamic slots. Factual Q&A is rough unless you template responses with live data. I added an LLM fallback for low confidence, but 90% is pure selection. What Jev is NOT doing: It doesn't read the response bank — I fetch candidates in code. Jev only classifies the message and scores the candidates. No reasoning, no generation, just fast probabilistic judgments. Links: Live demo: https://jevllm.pages.dev (terminal mode + pipeline explorer) Source code: https://github.com/akash-kamat/jev-llm Would love feedback — especially if anyone's tried similar selection-based approaches or has ideas for scaling the response bank!
Jev is a novel AI model that outputs scores, choices, or binary decisions, praised for its speed, affordability, and accuracy when queried creatively, unlike traditional frontier models.
TypeSafe AI has released Jev, an AI model designed for machine-native interactions that returns typed probabilistic decisions instead of text, offering lower hallucination rates and faster response times compared to traditional LLMs.
Jev is TypeSafe AI's frontier model for fast, structured AI decisions, returning typed outputs with calibrated probabilities and now available to everyone.
LLM2Jev is an open-source tool that adapts local HuggingFace models to perform structured decisions with Choice, Score, and Noul frameworks, offering prefill-only inference and integration with Transformers and SGLang.
The article introduces Jev, an AI model designed to make decisions rather than generate answers, using structured outputs for applications like fraud detection and risk assessment, positioning it as a routing layer for larger reasoning models.