hearim is an open-source gateway that repurposes ordinary local LLMs into Jev-compatible decision APIs by leveraging their probability distributions, eliminating the need for specialized fine-tuned models.
What if you don’t need a new model to get Jev-style decisions? I built hearim(헤아림. hea-a-rim. not Here, I am.. lol), an open-source gateway that turns ordinary local or hosted LLMs into a Jev-compatible decision API. Give it some data and a set of choices, and it returns a decision with probabilities—using models you can already run through llama.cpp, vLLM, SGLang, Ollama, or an OpenAI-compatible server. No task-specific fine-tuning model. No custom decision-model weights. No parsing generated JSON and hoping the model followed the schema. only pre-fill (with prefix caching) + only 1 token decode (with logit) This kill most time-latency and no need to generate text. The basic idea is that a pretrained language model already learned a great deal before instruction tuning or RLHF taught it how to chat. If you only need it to choose between clearly defined alternatives, much of that capability may already be present in its probability distribution. The model may already know. We just need to stop asking it to write an essay. hearim turns a decision into a constrained set of alternatives, reads the model’s probabilities for them, and returns the result through the Jev Choice, Score, and Noul API format. I’ve been using this underlying technique for more than a year in a separate implementation—before Jev was announced. hearim itself is a new, independent reimplementation, built specifically around the Jev API contract. I’m not claiming that it reproduces Jev’s architecture, training, calibration, speed, or performance. The interesting claim is narrower: A useful Jev-style interface does not necessarily require a specialized Jev-style model. The implementation, supported backends, limitations, benchmarks, and technical details are here: GitHub: https://github.com/ziozzang/hearim Paper: https://github.com/ziozzang/hearim/blob/main/paper/paper.pdf I’d love to see what happens when people try it with different local models—especially smaller base models, not only instruction-tuned ones.The model may already know. We just need to stop asking it to write an essay.
LLM2Jev is an open-source tool that adapts local HuggingFace models to perform structured decisions with Choice, Score, and Noul frameworks, offering prefill-only inference and integration with Transformers and SGLang.
The article introduces Jev, an AI model designed to make decisions rather than generate answers, using structured outputs for applications like fraud detection and risk assessment, positioning it as a routing layer for larger reasoning models.
Jev is a novel AI model that outputs scores, choices, or binary decisions, praised for its speed, affordability, and accuracy when queried creatively, unlike traditional frontier models.
The author shares a DIY Jev-like inference setup using open weight LLMs, demonstrating that simple prompting with logit-based verification achieves good accuracy without fine-tuning, and provides a rust web server for local deployment.
Describes a conversational AI system that uses Typesafe's Jev non-generative model to select from pre-written responses through parallel classification and scoring, providing a cost-effective and transparent alternative to generative LLMs.