Tag
A new EXL3 quant of Qwen3.8-27B by yeasah achieves the lowest KLD (0.0002) among competitors while requiring only 18.8 GiB of VRAM for weights, outperforming quants from Unsloth and turboderp.
A showcase of Mica v0.5 4B, a small local model that decomposes natural-language requests in Minecraft into chains of probabilistic multiple-choice decisions, enabling a bot to complete complex multi-goal tasks like gathering porkchops and crafting a netherite ingot from bare hands.
Opera launches WebAI, a feature that lets websites run AI models locally on the user's computer through the browser.
A user jokes about feeling like a 'local King' running two Strata engines at 70 tps and 12M tokens daily, contrasting it with Poteto, a former Cursor employee now at xAI, who reportedly uses 34M tokens per minute and ships 2,500 PRs a month to Cursor and GrokBot.
A user shows off a custom AI workstation built around 8x Radeon R9700 GPUs, a Threadripper 9955WX, and dual Seasonic PSUs, upgraded from 4 GPUs to run larger local models.
Liquid AI 发布 LFM3-1.7B 的 GGUF 版本，支持在 CPU/GPU 上本地推理，兼容 llama.cpp、Ollama、LM Studio 等工具，并提供经过量化感知蒸馏（QAD）的 Q4_0 检查点以恢复量化损失的精度。
作者分享如何用一台12年前的办公PC运行本地脚本（IMAP邮件轮询、Tesseract OCR、Piper TTS），把Telegram机器人例行任务从昂贵的LLM API调用转为零token的本地处理，显著降低成本。
HuggingFace Bio 发布 Carbon-X，一个 14B 的 DNA+自然语言联合模型，在五项 DNA 基准上超越所有开源模型并匹配 GPT-6.1 Sol，且成本低约 1000 倍，同时提供 GGUF 版本支持 llama.cpp 本地运行。
OrcaRouter 发布了 DeepSeek-V4.1-Flash-1017 的 abliterated（移除拒绝）GGUF 版本，可在单机 236GB 内存上通过 llama.cpp 本地运行这个 763B MoE 模型，其中 197B 参数的 Engram n-gram 记忆表按需从 SSD 流式读取。
Recursive released Flint, a free 40B-parameter mixture-of-experts coding model that runs on laptops, claiming it was built largely by its automated AI research system, the Eureka Machine. Flint achieves high token throughput and strong SWE-Bench Verified results for its size, and is available under the Apache 2.0 license on Hugging Face.
llama.cpp PR #29887 adds an LRU MoE expert cache that exploits locality of reference, boosting Qwen3.8-Flash-Next (94GB) runs to 67.8 tok/s on an RTX 5090 (2.2x), with SGLang merging a similar 'paged experts' feature days later.
Recursive 发布了 Flint，一款 40B 总参数、仅激活 2.5B 参数的 MoE 本地 agentic 编码模型，在 50B 以下 MoE 模型中于 SWE-Bench Pro v2、Terminal-Bench 2.1 等基准上领先，并借助自研 Eureka Machine 自动化研发系统在四个月内完成构建。
qev-0.8b is a 0.8B-parameter local decision model that takes a single forward pass to return calibrated probabilities over any input, ranking #1 of 27 similar-sized models on Benchmark Heaven (77.1%) at 3.5 ms p50 latency while running in 1.3 GB on a laptop. It ships in CUDA, MLX and GGUF formats and generalizes to games (Tetris, Pac-Man, 2048, Breakout, Whac-A-Mole) never seen in training.
Continuity Research 发布 Tess-4.5-27B，一个基于 Qwen3.8-27B 微调的 27B 开源权重模型，主打本地单卡运行的工程与智能体能力，BenchLocal 成绩与基座持平但推理 token 少 25%，并附带多 token 预测草稿头以提升本地推理速度。
A user shares their experience using the fast DiffusionGemma model locally for image analysis and offline image-generation prompting, and asks the community for other use cases where its speed tradeoff pays off.
TensorSharp, an open-source local inference engine, adds a Video Studio feature enabling local video generation (LTX 2.5 Distilled, MiniMax-H3, Wan 2.1/2.2) with an editable timeline, multi-track editing, and an optional local LLM assistant, running fully offline on a 48GB M5 Pro MacBook Pro.
The author benchmarks Nemotron 3.5 Super VL on obscure knowledge (Anno 1602 game rules, no websearch allowed) and finds it far more accurate than GPT-OSS, Qwen, DeepSeek, and Gemma models, making it a solid local model to keep.
一周内四个大型开放权重模型接连发布：Meta的Muse Spark Open、NVIDIA的Nemotron 3.5 Bolt、Reflection的Beam以及DeepSeek的V4.1 Flash，作者对比了它们在24GB显存门槛下的可运行性、许可证差异与硬件成本。
nazeshinjite 发布了 DeepSeek V4.1 Flash 1017 的 3.91bpw GGUF 量化版本，面向 antirez 的 DwarfStar (ds4) 推理引擎，采用 SSD 流式加载，可在 128GB Mac 上以最高 20tps 解码。
A community member reports that Qwen 3.8 Flash Next (UD-Q4_K_XL on 4x RTX 3090s) underperforms the Qwen 3.8 27B Q8 in a Tetris-building coding task, producing bugs and failing to deliver a playable .exe in Strata despite faster generation speeds, challenging the hype around the smaller model.