[P] Pecision models that score every allowed label from the logits: Jebadiah v2.1 (27B, 9B), open weights and self-run benchmark results [P]
Summary
作者发布 Jebadiah v2.1(27B 与 9B)开源模型,将决策建模为封闭集打分问题,直接从候选标签 logits 输出各选项概率,并公布在 Decision Index 0.3 上的自测成绩、污染检查与量化一致性结果。
Similar Articles
Perplexity Decider 27B: Open weights decision model fine tune of Qwen3.8 27B
Perplexity has released pplx-decider-v1-27b, an open-weights decision model fine-tuned from Qwen3-27B that scores 85.71% overall across 11 benchmarks, notably improving on tasks like RAGTruth (88.80%), FinancialPhraseBank (84.18%), and TabFact (90.60%). The model can be run locally via Hugging Face and the Perplexity API for structured choice and yes/no probability predictions.
Benchmarking small confidence scoring decision (Jev, Laya) models [P]
A self-run evaluation of two small confidence-scoring classification models — TypeSafe AI's hosted Jev and the open-source Laya — finds that label descriptions phrased around input features, consistent time windows, and calibration (ECE) matter more than model scale for downstream thresholding performance.
H2O-Lightning-4B: Apache-2.0 4B Decision model, official #1 open model on JevBench (above Jev)
H2O.ai 发布开源 Apache-2.0 4B 决策模型 H2O-Lightning-4B,基于 Qwen3.5-4B 微调,通过单次前向传播输出校准概率,JevBench 综合得分 72.5,超越 Jev 1.13 成为榜首开源模型,后续将推出 12B 和 31B 版本。
jevman: AI decision models play Pac-Man
作者将六个流行的决策模型(jev 1.13、Kev 4B、Clef、Clef Flash、GPT-6 Luna 和 Laya)放在《吃豆人》游戏中进行实时对决,jev 1.13 以平均 2,750 分领跑排行榜。团队已公开排行榜和开源仓库,任何人提交自己的模型即可参赛。
Jiwo - Small decision models topping decision index leaderboard
A developer released Jiwo, two Qwen3.5 fine-tuned 'decision models' (a sub-1B 0.8B model and a 4B model) that topped the Decision Index leaderboard for their size categories, achieving 45.7% improvement over the best other Qwen3.5-0.8B fine-tune.