gguf

A user extracted the pure DFlash draft part from Ling-3.0-tiny-UBTree (dropping the ubtree_selector tensors) so it runs on stock mainline llama.cpp, reporting +8% speedup for prose and +61% for Python code generation, with the converted GGUF and conversion procedure published on Hugging Face.

SparkLLM 釋出 Spark-X2.6-4B 端側模型，具備 4B 參數、Apache 2.0 授權、最高 1M context 與 3 種思考模式，並支援 GGUF 本地部署。

A LocalLLaMA user reports negative hands-on experiences with the quantized LFM2.5-24B-A2B (Q4_K_M GGUF) model, citing looping behavior, broken code generation on agentic tasks, and poor plugin development results despite using a corrected chat template and Hermes integration.

OrcaRouter 发布了 DeepSeek-V4.1-Flash-1017 的 abliterated（移除拒绝）GGUF 版本，可在单机 236GB 内存上通过 llama.cpp 本地运行这个 763B MoE 模型，其中 197B 参数的 Engram n-gram 记忆表按需从 SSD 流式读取。

Unsloth 发布了 Recursive 公司的 Flint 模型（一个面向编码智能体的 39.8B 总参数/2.5B 激活参数的 MoE 模型）的 GGUF 量化版本，附带在 Unsloth Desktop 中的运行指南与量化基准测试。

作者在 ARM (Neoverse-N1) 和 AVX2 x86 CPU 上对比了多个低比特（1-bit/三值/2-bit）GGUF 模型运行时的正确性与速度：bitnet.cpp main 在 CPU 上仍输出错误结果（需 #588/#642 补丁），Bonsai 1.7B 在所有能加载的运行时中均正确，而 ik_llama.cpp 配合 -rtr 通常最快。附带可复现脚本与 Kaggle/Colab 验证笔记本。

llama.cpp 新增对 Cohere 系列模型的支持，继续扩展其 C/C++ 本地 LLM 推理框架的模型兼容性。

ISTA-DASLab 发布/更新了 Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF 模型仓库，提供可在 llama.cpp、vLLM、Ollama、LM Studio 等本地环境中运行的量化编码模型及部署说明。

Spark-X2.6-4B is a compact 4B on-device language model released in GGUF format, featuring improved post-training for coding, agentic, long-context, and multilingual tasks along with a controllable thinking budget. It claims state-of-the-art performance among similarly sized open-source models, matching or surpassing ~9B parameter models.

A user loaded the 560B-parameter Ling 3.1 Flash (25B active) MoE model across four DGX Sparks using llama.cpp RPC with Atomic Chat's Q4 GGUF build, since no official FP4 vLLM build was available. Performance comparisons against Ling 3.0 Flash 124B are still pending.

nazeshinjite 发布了 DeepSeek V4.1 Flash 1017 的 3.91bpw GGUF 量化版本，面向 antirez 的 DwarfStar (ds4) 推理引擎，采用 SSD 流式加载，可在 128GB Mac 上以最高 20tps 解码。

mrfakename 发布的 DeepSeek V4.1 Flash 1017 Hugging Face 合集，包含 748B 图文到文本模型的 GGUF 量化版本以及 763B 的 abliterated 版本及其 GGUF 变体。

A community guide on running Qwen 3.8 Flash Next with a full FP8 n-gram table (PLE) instead of a heavily quantized one, using ple_fp8_pack.py to pack safetensors into a GGUF sidecar, yielding noticeably better output quality with no speed penalty.

A Hugging Face release packaging HauhauCS's uncensored Qwen3.8-27B weights with DavidAU's Turbo-Brilliance 'AI Construct' chat template modes into a single GGUF, enabling role-based uncensored responses in llama.cpp, LM Studio, and Ollama.

PS5LM is an open-source project that runs llama.cpp on a jailbroken PS5's RDNA 2 GPU, turning the console into a private model server that serves GGUF models via OpenAI and Anthropic compatible APIs to any device on the local network.

OrcaRouter 发布 OrcaCyber Nano 1.0，一个 27B 视觉语言推理模型，以 GGUF 格式打包，可在本地离线运行，面向逆向工程、漏洞分析、安全代码审查等防御性安全工作，并公布多项基准测试成绩。

A Reddit user reviews agentionai's GYRO quants of Qwen3.8-Flash-Next, claiming they match or beat ISTA-lab's q3-level quants in KLD and loop resistance while being significantly smaller — praising the work of a single contributor.

QUASAR 释放了针对 Qwen3.5-4B、Gemma 4 E4B/12B 和 Muse Glimmer 30B 的 4-bit QAT 检查点（NVFP4 与 Q4_0 GGUF 格式），通过量化感知蒸馏在 KL 散度和下游基准上超越 Google 与 NVIDIA 的官方 4-bit 版本。代码、论文与项目页现已公开。

AtomicChat 发布了 inclusionAI Ling 3.1 flash 的 GGUF 量化版本，采用其自研的 Atomic Dynamic 布局与公开校准语料，覆盖从 Q8_0 到实验性 1-bit 的多种量化档位，可在 llama.cpp 上本地运行。

User shares independent testing results showing that Strata and Swift quantization variants of Qwen3.8 models sacrifice significant intelligence for speed, with the 27B Q4 model delivering better real-world coding results despite being 3-5x slower.