quantized

LiquidAI releases LFM 2.5, a language model optimized for quantized deployment, with GGUF format available on Hugging Face.

A model card for Huihui-DeepSeek-V4-Flash-0731-abliterated-GGUF, an abliterated (uncensored) GGUF quantized variant of DeepSeek-V4-Flash, designed for local use with llama.cpp and ds4.

Demonstrates running two 80B Qwen models simultaneously on a MacBook Pro using BBQ-FP4 quantization, claiming functionally lossless performance and speed.

Running a 4-bit quantized version of GLM-5.2 (753B MoE) on 4 DGX Spark machines achieves 70.8% on Terminal-Bench 2.1, compared to 81.0% from the full model.

Pre-converted int4 quantized weights for the GLM-5.2 744B MoE model, designed to run on consumer hardware with ~25 GB RAM using the colibrì engine.

Un modello linguistico locale e privato (Qwen 3 da 1.5B e 4B quantizzati) può girare offline su smartphone, con fine-tuning e LoRA distillato da un 32B.

Running the quantized Gemma-4-31B model on an RTX 5090 increases context length from 35k to 80k, showcasing significant performance improvement.

Bonsai-27B is a 1-bit binary transformer model that achieves full 27B-class reasoning on a phone (iPhone 17 Pro Max) with ~3.9 GB footprint and ~11 tok/s, retaining ~90% of FP16 intelligence.

Qwythos 9B is a new open-source, uncensored reasoning model based on Qwen3.5, offering GGUF quantizations, 1 million token context, vision, and function calling, with significant performance improvements over the base model.

A quantized GGUF version of the abliterated GLM-5.2 model is released on Hugging Face, enabling local inference with various tools like Transformers, llama.cpp, and vLLM.

A new uncensored GGUF quantized version of the Qwythos-9B-Claude-Mythos-5-1M model, created using abliteration, is released on Hugging Face.

Unsloth released a GGUF quantization of Qwen-AgentWorld-35B-A3B, a native language world model that simulates agentic environments across seven domains (MCP, Search, Terminal, SWE, Android, Web, OS) using long chain-of-thought reasoning and trained via CPT, SFT, and RL.

Antirez announces high probability of merging a branch implementing GLM 5.2 in DwarfStar, which could become the best model for 512GB Mac Studio and potentially run on distributed 128GB MacBooks with 2-bit quantization.

NVIDIA released GLM-5.2-NVFP4, a quantized version of ZAI's GLM-5.2 MoE language model optimized for inference on NVIDIA Blackwell GPUs using Model Optimizer.

NVIDIA released Qwen3.6-27B-NVFP4, a quantized version of Alibaba's Qwen3.6-27B model, optimized for deployment on NVIDIA GPUs with support for text, image, and video input.

unsloth has uploaded a GGUF version of GLM-5.2 to Hugging Face, providing ready-to-use model files for various inference engines like llama.cpp, vLLM, and SGLang.

GestaltLabs releases Ornstein-3.5-9B-V1.5 GGUF quantizations, a reasoning-focused fine-tune of Qwen 3.5 9B with an MTP head and vision projector for multimodal use.

A Hugging Face repository (kaitchup/Qwen3.6-27B-GGUF-MoQ) provides GGUF quantized weights for the Qwen3.6-27B MoQ model, enabling local inference with tools like llama.cpp and Ollama.

A GGUF quantized version of the Qwopus3.6-27B-Coder-MTP model is released on Hugging Face, optimized for local inference and compatible with Transformers, vLLM, SGLang, and Unsloth Studio.

Holo3.1 is an updated computer-use model family that improves robustness across web, desktop, and mobile environments, introduces quantized checkpoints for local execution, and adds native support for function-calling protocols.