akhilaaa3/Jev-Omni · Hugging Face

Source: https://huggingface.co/akhilaaa3/Jev-Omni A multimodal decision classifier fortext, images, audio and video. Supply a question and options; receive a probability for each option—not a generated explanation.

Built on Gemma 4 12B IT, with a 30,000-question fine-tuning run.

https://huggingface.co/akhilaaa3/Jev-Omni#resultsResults

BenchmarkAccuracy¹Micro accuracyDecisionBench Medium · 80 scenarios / 293 questions**87.57%86.01%JevBench · matched 195 groups / 231 decisions86.15%**87.45%MMAU · 1,000 questions—**63.10%MVBench · 14 evaluated tasks / 2,786 questions53.10%**53.09% Merged-model results. ¹Equal-weight scenario/group average.

https://huggingface.co/akhilaaa3/Jev-Omni#open-weight-comparisonOpen-weight comparison

ModelParametersMMAUMVBenchModalitiesJev-Omni12B**63.10%****53.10%**Text, image, audio, videoInkling975B total / 41B active77.20%—Text, image, audioQwen3.5-397B-A17B397B total / 17B active—77.60%Text, image, video Reference scores are officially reported by their developers and may use different evaluation protocols.

Jev-Omni cost uses its recorded input tokens at OpenRouter’s Gemma 3 12B input rate ($0.05/M); its classifier generates no output tokens.

Both classifiers are pricedone call per question: a classifier answers one question at a time, so the state is re-sent for each of them and there is no discount for asking several at once. The three chat models are priced one call per state, with all of that state’s questions together, which is their own cheapest shape. Splitting a state into per-question calls multiplies input tokens by 2.82x on this set, so pricing the chat models the same way would move all three marks right by roughly that factor without changing the order.

DecisionBench dataset

https://huggingface.co/akhilaaa3/Jev-Omni#quick-startQuick start

CUDA GPU required. FP32 weights use about 50 GB before runtime overhead; inference uses BF16 autocast.

pip install -r https://huggingface.co/akhilaaa3/Jev-Omni/resolve/main/requirements.txt
# ffmpeg is also required for audio input

File:requirements\.txt

from huggingface_hub import snapshot_download

path = snapshot_download("akhilaaa3/Jev-Omni")
import sys; sys.path.insert(0, path)
from jev_omni import load_jev_omni

classifier = load_jev_omni()
result = classifier.predict(
    state="The meeting starts at 10 AM. It is now 9 AM.",
    question="Has the meeting started?",
    options=["Yes", "No"],
)
print(result)

For an image, audio file or video, addmedia="/path/to/file"andmodality="image","audio"or"video". The loader downloads the original Gemma 4 multimodal components automatically. Audio is capped at 30 seconds; video uses 16 frames.

https://huggingface.co/akhilaaa3/Jev-Omni#speedSpeed

Warm H200 inference:83 ms(~2k-token text),26 ms(image),31 ms(13-second audio),504 ms(16-frame video). Medians over 20 optimized-backend requests; preprocessing and network time are extra.

https://huggingface.co/akhilaaa3/Jev-Omni#limitsLimits

Best supported at**≤20 options**. The head accepts 256, but quality above 20 is not established.

https://huggingface.co/akhilaaa3/Jev-Omni#licenseLicense

Apache-2.0, followingGemma 4. Dataset rights remain separate.

https://huggingface.co/akhilaaa3/Jev-Omni#calibrationCalibration

Medium ECE:0.0400(10 bins; lower is better). The graph uses five bins for readability.

Note: Jev-Omni is an independent open model that implements the typed-decision interface — noul (yes/no), choice and score questions answered with calibrated probabilities. It is not affiliated with, endorsed by, sponsored by, or derived from TypeSafe AI or its Jev model, and nothing in it was trained on Jev output.