What Jev & Jev-like models are actually good for in phone voice bots

This is my personal take on where Jev and similar classifier models fit, based on what I've seen as Head of Product at a voice AI orchestration platform and what other people have published. I originally had links to every source in the text, but the sub doesn't allow links in posts, so they're all in the first comment. Some history first Back in 2022 to 2024, large language models weren't widely available yet, but a bot on a phone call already had to understand what the caller wanted. So we built our own classifier. It quickly and cheaply picked the caller's intent out of a phrase, using a catalog the customer could configure, and it flagged entities in the phrase, the structured data a bot needs, either built-in types like dates and numbers or custom ones defined for a project. It worked well enough, and some projects still run on it. Then LLMs appeared, and for a minute it seemed a classifier wasn't needed anymore. The model would grasp the meaning on its own and compose the answer on its own. In practice, as usual, the euphoria ended pretty fast. On live projects, medium-sized models broke down under the size of the initial prompt or after a few changes of subject, and started making things up. The large expensive models could carry the load, but they either wrecked the cost per conversation or were so slow that a live call couldn't accept it. When you build voice bots professionally, for a production system at a large customer, you have to optimize all of it: answer quality, the metrics you're measured against and the budget. Bot builders went around this with engineering, using multi-stage bots, several agents on one phone number with a router model, and scripted stretches wherever the dialog is known to follow a fixed scenario. We launch several hundred large commercial projects a year, and that's enough experience to say this: one single model misses picking a tool, overruns the context window or burns so many tokens that it ruins the unit economics of the service. In real cases you can't rely on one model, it will almost certainly fail to carry the whole conversation. By that point our old classifier was out of date, its architecture first of all, and we'd started replacing it with a similar scheme built on a cheap, fast small language model. And then Jev came out. Everything new is a well-forgotten old thing :) So I decided to look, against our own experience, at where and how classifier models are being used in voice bots. What Jev actually is Jev is a fast classifier model from TypeSafe. It's been in early access since September 2026. It takes a state (for example, the bot's last turn, the current step of the scenario) and a set of questions, and returns a typed structured answer: a choice among options you set, a score on a short scale or the probability of yes/no on one direct question. Jev doesn't write text and has no output tokens as such. It just returns a typed structure, and what to do with it next is the bot designer's job. It's quite fast. Latency measurements don't agree with each other, but taken together they sit under 500 ms. In one case a batch of 35 questions fit into about 170 ms on a warm connection, in another it was 332 to 892 ms. TypeSafe's own materials say 70 to 500 ms. For the full picture, Jev isn't alone. Here's what else has appeared lately: Laya, the main and most widely used free counterpart, open source under Apache 2.0, from a researcher at Convai Innovations. Clef from Cloudflare, an open model for very fast decisions inside software systems. CMF, a lightweight open model at only 304.5 MB. It answers in milliseconds, doesn't generate text at all and shows high accuracy on classification tasks. Bespoke Nimble, a recipe for an "open Jev" by Bespoke Labs, a LoRA on Qwen3.5-9B with constrained decoding so the model can only answer inside the allowed choices. Kev by Jared Palmer, which started as Kev-0.5B, an ultra-compact Jev-like model on Qwen2.5-0.5B made to run locally on an ordinary machine like a MacBook Pro, and is now a whole family of models. CLM-8B from Stanford and Nvidia, which caches agents' repeated actions and ran up to 9x faster than Jev in its authors' tests. There are quite a few of these by now. What I want to look at is how a classifier model can improve an LLM voice bot. A classifier at the entrance, a specialized agent per subject Typical voice menu situation: one phone number with several subjects inside it, like a booking, order status, a complaint or "connect me to a human". If all of that lives in one prompt, a medium-sized model swells, and after the second or third change of subject it starts talking rubbish. The fix is a fast classifier at the entrance. It picks a small agent tuned to a single subject and connects it to the conversation. A change of subject is its decision too, as a separate beat, before control goes to the large model and the model starts composing an answer. Piotr Zaczek showed this on a voice setup in a short video on LinkedIn. It's about the time the system needs to decide the dialog should change direction: 0.5 s against almost 5 s, plus savings on the large model's tokens. Piotr took away the large model's permission to draw conclusions and reason outside its own subject, and that job went to the classifier. Why do it this way? A specialized agent has a short prompt and a single catalog of tools. The classifier is cheap and fast, so it costs nothing to ask it at every change of subject. In the same post Zaczek names three uses he likes alongside routing: a review after the call, background limits that are always on and sit outside the spoken answer, and the decision of when to hand the call to another agent or a person. Worth keeping in mind. External control: the model proposes, Jev judges A bot built on an LLM can talk pleasantly, and it rarely carries the business through to the end. It found an open slot and never got to the booking. It wrote the appointment down under the wrong name. It didn't hang up on "goodbye". It asked the same thing twice. Giorgi Lomidze goes through how they moved decisions like these out of the speech model and into a Jev-based classifier in two weeks (JEV in Production on Medium, 24 September 2026). The product is Pocos, a phone assistant for salons and studios, and it runs on Voximplant in production. Nice to see, so thanks to the author for mentioning it. How it's put together: the call arrives over SIP, a scenario on the platform connects to the LLM through a realtime API client (Gemini Live in their production) and keeps the spoken dialog going, handling the greeting, interruptions, timers and hang-up. The speech model hears the caller and proposes a tool call. Every conclusion the model draws is moderated by the classifier before it can change anything in the system or the dialog. A short packet of facts is assembled, Jev gets several yes/no questions and returns probabilities. The questions are literal, things like "did the caller confirm this action?", "does the last remark change, confirm or cancel what we just said?" and "has the caller said goodbye?". What happens next depends on the probability Jev returns. The thresholds need fitting to your situation. In Giorgi's case strong agreement starts at 0.9, weak agreement goes down to 0.1, and a goodbye counts as strong from 0.75 and only if the caller hasn't brought up a new subject. The heart of the approach in his words: the LLM only proposes, Jev judges and the business makes the decision. End of turn: meaning on top of audio The caller suddenly goes quiet mid-word, and the silence timer is ready to give the turn to the bot. "Yes" after "shall we move it to Tuesday?" is a finished remark. "What I wanted to say was..." isn't, even though the pause is the same length. We have our own turn detection stack: one module with Silero VAD, the Pipecat Smart Turn end-of-turn detector and optional Hush noise suppression. Developers get turn events and can react to each of them. Smart Turn answers the pause question, whether the turn is over or not, and if not it holds the silence a bit longer. That's control of the sound and an acoustic model of the turn. On top of that we want to add a check on meaning: is the thought finished, or is the person searching for the next word? Technically it should work. A typical ASR delay is about 50 ms and Jev classification about 200 ms, so roughly 250 ms together. Not instant, but for most industries on a phone line with a full-cascade bot, where recognition, the LLM and TTS are separate stages, it should be enough. What we want to experiment with: The turn starts with VAD and the acoustic detector. On partial or final text, one question goes to Jev: "has the speaker finished what they wanted to say?" The bot's last remark goes into the state, because "yes" is a complete answer only because a question came before it. If turn detection says "done" and Jev sees an unfinished thought, we treat it as the caller wanting to continue and don't pass the turn to the bot. If both say "done", the bot can start answering. Supreet Tare from Taritas used the same mechanism in a live voice coaching project and wrote it up on the Taritas blog. In his setup the audio catches the silence, after 350 ms of silence Jev gets one question, and the answer takes about 0.4 s. Then the code decides how much longer to wait: 1.7 s if the thought is finished (probability from 0.75), 2.8 s if it's uncertain and 4.5 s if the phrase breaks off on "and", "but" or a comma. Looks very slow, but in his trainer not missing anything important matters more than answering quickly. Barge-in, "hold on a sec", backchannel and side talk Jay Albo covered an interesting case on Agora Conversational AI (Jev VAD, with code in the AgoraIO-Community/convoai-jev-vad repo). Skipping the audio side and speech-start detection by signal energy: the caller starts speaking, the text goes to Jev, and one call returns four probabilities. turn_complete: the caller finished the thought and the turn can be handed over. We've already been through this one. holding_floor: the caller explicitly asks for a pause, as in "wait, let me look it up...". Already more interesting. addressed_to_agent: was this actually said to the bot? "Hey, keep it down over there, I'm on the phone" wasn't. For me this is the most curious detail, and the first time I've come across it. taking_floor: while the bot is speaking, is this a real barge-in or a backchannel like "mm" and "uh-huh"? Some models can catch these scraps on their own, but the behavior isn't consistent, and a model change or update can break the bot right at this point. In his sessions a Jev call came back in about 300 ms. Filling in a form: Jev sorts the data Booking an appointment, moving it, cancelling a visit, a question about a bill. The domain is small and the variation limited, but the phrases are live, and in one remark the caller brings their own details, the specialist and the date and time of the visit. An LLM that also has to speak is overkill here. What you need is that same old classifier, one that picks the actionable data out of what the person said and hands it to the bot's code. Jason Stiles built this as a phone line for an imaginary clinic (Jev IVR, updated 24 September 2026, with code on GitHub). The stack is Node, Twilio ConversationRelay, Deepgram for recognition, ElevenLabs for the voice and Jev for understanding. The bot's code holds the form state, and each turn sends one request to Jev with about 30 to 35 questions. Intent is chosen from a description. "Move it", "let's take another day" and "I need to change my appointment" all land in reschedule, because the criteria describe what the person wants. Structured data works in a more involved way than entity spotting. The code finds the candidates in the transcript itself: every number-like stretch for the year of birth, every stretch of one to four words for the name. Jev picks which stretch is the year or whose name it is. One advantage of a classifier here is that hallucination just doesn't happen as a thing. Jev can't make up a date, so an error in the data is always the caller's own words. Confirmation is always a separate request: the caller agreed, refused or named the form field to change. It does sound old-fashioned, but it works quickly and cleanly. Does the job, literally. An expensive answer or a cheap one The subject router picks what to talk about. There's another choice too: what each turn's answer should cost. "Yes", an account number or a short status can be answered cheaply. Confusion, instructions, a long explanation or a refusal probably can't. Deepesh Jayal from Evalgent described this choice in Jev for Model Routing in Voice Agents. He splits the models into three classes: A prepared answer, where no model is called and the reply comes from plain TTS or a script. A fast model for a short factual answer. A heavy, expensive model for ambiguity and for a high cost of being wrong. On every turn Jev picks the class, effectively answering "is an expensive answer needed?". The code Deepesh relies on is ModelRouterMiddleware from the langchain-typesafe package. The criteria for choosing a model are written in plain language and the probabilities stay in the agent's state, so a post-call review can show why a turn went to the expensive model. I'd lock in three rules from his article: Low confidence lifts the choice one class higher and never leaves it on the cheap class. A question with a high cost of being wrong never lands on the cheap class, even when the classifier is sure it's simple. Mid-call, a raised class isn't lowered. The expensive model is already awake, and going back to the cheap one on the next "yes" saves little and can spoil the context the expensive model has gathered. The cascade I'd build To sum up, here's the order in which I think it makes sense to assemble a voice bot around a fast, cheap Jev-like classifier. Each step has its own confidence threshold that you get from experience and fit to the case. End of turn. On text, adds about 250 ms (ASR about 50 ms plus classification about 200 ms). The question is "is the thought finished?", with the bot's last line in the state, following Supreet Tare's setup. Barge-in and side talk. While the bot is speaking: "is this a backchannel?", "is the person asking to wait?" and "is the remark actually meant for us?", following Jay Albo's setup. About 300 ms in his tests. Subject. If the menu holds more than one subject, use a classifier the way Piotr Zaczek does and connect an optimized agent for each one. A change of subject mid-call goes through the same classifier and never relies on the agent's guess. Form and slot filling. Booking, moving, cancelling, status, billing. The code cuts out candidate data, Jev picks a stretch of the transcript, phrases come from prepared replies and tests keep a cassette of answers. I'd personally argue with this one, since there are more effective ways to pull out structured data than yes/no splits by a classifier, but as an experiment it has its place. As navigation through the menu and the fields a form needs, it really does deserve attention. Changing data and writing records. Create, move, cancel, charge, hang up. Giorgi Lomidze's approach: the LLM only proposes, Jev judges whether the proposal is correct and appropriate. If yes, we write the change. If the two disagree, we don't, and we think about what to do next. The price of the answer. Prepared reply, fast model or heavy model. Wake the heavy one when the classifier says the turn is heavy or when the cost of being wrong is already high, and don't lower the class mid-call. If you're running Jev or a Jev-like model in voice production, please share your experience and thoughts.