api-server

Quartermaster is an open source local AI platform that simplifies model management, VRAM allocation, and provides an OpenAI-compatible API with a built-in chat playground, supporting various backends like llama.cpp and stable-diffusion.cpp.

llama.cpp now supports MTP (Multi-Token Prediction) for GLM-Air models, enhancing inference capabilities for large language models in C/C++.

Nativ is a new macOS desktop app that wraps MLX to run AI models locally, offering a chat interface and API server.

llm_wiki v0.4.11 released, adds local API Server supporting Agent integration, unified search backend, new knowledge graph search feature, Star count exceeds 8k.