@no_stp_on_snek: turboquant+ is now a swappable backend in LocalAI alongside tinygrad and sglang. if you're running GGUF models and want…
Summary
turboquant+ backend added to LocalAI, enabling longer context for GGUF models without hardware upgrade.
Similar Articles
@no_stp_on_snek: got it here if ya want to try it out:
A fork of llama.cpp integrating TurboQuant+ for advanced KV-cache and weight quantization, with cross-backend kernel support (Apple Silicon, NVIDIA CUDA, AMD ROCm, Vulkan) and used in production by LocalAI, Chronara, and AtomicChat.
@coffeecup2020: TurboQuant - Qwopus3.6-27B-v2-TQ3_4S.gguf Confirmed with gpqa test this is something great. https://huggingface.co/YTan…
TurboQuant is a GGUF quantized version of the Qwopus3.6-27B-v2 model, confirmed with GPQA test results and shared on Hugging Face, with credits to Jackrong and KyleHessling.
@UnslothAI: GLM-5.2 can now be run locally! The 2-bit model retains ~82% accuracy after we shrunk it from 1.51TB to 238GB (-84% siz…
UnslothAI announces GLM-5.2, Z.ai's strongest open model with 744B parameters, now runnable locally via dynamic GGUF quantization reducing size by ~84% to 239GB while retaining ~82% accuracy. It fits on 256GB Macs and supports long-context, reasoning, and agentic tasks.
@malikwas1f: well well well, Beellama managed to merge Dflash+TurboQuant already. this unlocks Q5 quants. Things just keep getting b…
A GitHub repository called club-3090 provides recipes and configs for serving large language models locally on RTX 3090 GPUs, with support for multiple engines and quantization methods like Dflash and TurboQuant, including newly unlocked Q5 quants.
@aisearchio: GLM 5.2 GGUF is already here! 8-bit is ~half the size of the full model. Smaller versions coming soon https://huggingfa…
GLM 5.2 GGUF quantized model is released, with 8-bit version half the size of the full model; smaller versions are coming soon.