Quartermaster 1.7 update now lets you run llama.cpp's new decision models (/v1/systemone) with a playground tab

Reddit r/LocalLLaMA Tools
local-inference llama.cpp gguf model-management decision-models self-hosted

Summary

Quartermaster 1.7 发布，新增对 llama.cpp 新决策模型（/v1/systemone）的支持，包括自动发现决策 GGUF 模型、带概率条的 Playground 测试标签页，以及通过调度器和 API 密钥代理 /v1/systemone 请求。

Quartermaster (my local inference manager: one server that swaps llama.cpp, image, audio and video models in and out of VRAM) can now help you server and manage decision models: Auto-discovered. Drop a decision GGUF (Kev, OpenJev, Laya, ...) in your models folder and it gets its own config entry, sized for VRAM like everything else. Playground tab. Write a prompt, ask one question, pick the answer shape, hit Run. Each option gets a probability bar. Useful for quick testing Model browser. A Decision tab finds them on Hugging Face. Same API, proxied. POST /v1/systemone goes through Quartermaster's scheduler and API keys, so it swaps in next to your chat model like any other request. Needs a llama.cpp new enough to have /v1/systemone (upstream b11361+; Quartermaster will warn about older builds). Project info: https://quartermaster-labs.github.io/Quartermaster/ Release: https://github.com/Quartermaster-Labs/Quartermaster/releases/tag/v1.7.0 Happy to hear what people are using these for.
Original Article

Similar Articles

New in llama.cpp: Decision Models

Reddit r/LocalLLaMA

llama.cpp server now supports 'decision models' via a new /v1/systemone endpoint, where a single forward pass scores typed options (choice/score/yes-no) with probabilities instead of generating text. Several ggml-org models (Julia-1, Laya, Kev-4B, lev, OpenJev) are available in GGUF, following TypeSafe's Jev System One format.

whats happening on llama.cpp

Reddit r/LocalLLaMA

A significant update to llama.cpp requires all previously generated GGUF files to be regenerated, indicating a major breaking change to the model format.