Tag
用户分享使用 Strata 替换 llama.cpp 运行 Qwen 3.8 Flash Next 后，解码速度从 70-80 tk/s 提升至 150 tk/s，并顺带测试了 DeepSeek v4.1flash 2.25bit 量化在多 GPU 上的表现。
A developer compares running Qwen3.8-Flash-Next locally (UD-Q4_K_XL on llama-server with ROCm) against Claude Opus in Claude Code for reviewing a real Swift commit: Qwen produced 8 findings with zero false positives and caught 2 issues Claude missed, but missed 5 others, making it a solid private first-pass reviewer rather than a final authority.
Strata's latest update significantly boosts inference performance for AMD R9700 users running Flash Next, roughly doubling token output from ~70 tok/s to 140-160 tok/s while still trailing Radiance. The user notes quantization limits (Q5 falling just short of VRAM for split-engine mode) and hopes future updates close the gap further.
The author reports that RTX 8000 48GB prices have fallen to around $2100 while RTX 3090 prices have roughly doubled, arguing the RTX 8000 may now be the best value 'do everything' option for running large local models in VRAM for home-lab setups.
A user shares using a local Qwen 3.8 27B model to analyze 16 years of city council meeting notes to trace the approval of Flock cameras, and asks for advice on scraping YouTube auto-transcriptions to feed into their LLM.
A near-zero-coding user shares how they built their own invoicing desktop app entirely via vibe coding, using Qwen 3.8 27B running locally in LM Studio through Open Code v2.0 on an RTX 5090, saving $100/year on subscription software.
A hobbyist benchmark pits locally run quantized models (Qwen3.8-Flash-Next, Qwen3.8-27B, Gemma 4) against Claude models on 9 real C++/CUDA bug-fixing tasks, finding 3-bit quantization costs little in score but makes models think longer, and that local Flash-Next roughly matches Claude Haiku 5.5 while Opus 5.5 leads overall.
A community proposal in r/LocalLLaMA urging members to stop fixating on tok/s speed metrics and instead create private quality benchmarks, especially to evaluate model quality across quantisation levels, and to share results with each other.
The author tested three local open-source models — DeepSeek-V4.1-Flash-1017, Qwen-3.8-Flash-Next, and Qwen-3.8-27b — on an RTX 5090, an RTX 4090, and dual V100s, comparing the real-world experience gap against cloud models like GPT-5.5 and Opus-4.8 through a Core-300 blind test and 200K+ long-context stress tests.
Kapitsa 0.2.26, an open-source Apple Silicon inference engine, publishes M5 Max benchmarks for Qwen, Nemotron, Flint and Clover models, featuring its KFlash best-first tree verification and SSD offloading that runs a 180B MoE at 61 tok/s on a 48GB MacBook Pro.
A user shares a detailed setup guide for running DeepSeek-V4.1-Flash-1017 (~35 tok/s) and Qwen3.8-Flash-Next (~130-140 tok/s) on older consumer hardware (RTX 3090 Ti, 128GB RAM, Threadripper 3990X), including an NVIDIA firmware-throttle unlock via an open-source kernel module and an Xolotl inference engine build.
A user asks whether Gemma 4 E4B remains the best small multimodal model (under 8B/6GB) for running a local, tool-calling personal assistant on a Mac via MLX, noting failures with Ling 3.0 tiny, LFM models, and Qwen 3.5 4B/9B.
A developer diagnoses why identical browser-agent runs on a local 27B model ranged from 40 to 261 seconds, eventually finding the culprit was a requirement-checking rule layer forcing unnecessary page opens rather than the model itself.
作者认为64GB内存不再是本地LLM的'死区'——在M4 Max上运行Qwen3.8 27B进行agentic coding，KV cache和prompt cache使64GB恰好成为sweet spot，打破了此前认为该容量尴尬的共识。
作者发布了 Betelgeuse（bel-cli / bel-server），一个 llama.cpp 分叉，通过 VRAM 槽位池、CUDA 门铃内核和 io_uring 分页器，在 RTX 3060 12GB + 16GB DDR4 上以 100% 位精确方式将 Qwen 3.8 Flash Next（68GB MoE）的推理速度从 1.47 tok/s 提升到 22 tok/s（峰值 48 tok/s）。
nullprogram 介绍并开源了 Prospero——一个用 C++ 和 Dear ImGui 编写的轻量级 AI 编码 harness，资源占用比现有方案低约两个数量级，支持子代理、无头部署与桌面控制能力。
A detailed benchmark shows QEngine 1.3.1 (compiled for RTX 5090s) is the fastest Qwen3.8-Flash-Next engine for single-stream decode, ~1.4-1.9x faster than Strata and ~3x faster than tuned vLLM, with a KV-in-RAM mode that dramatically improves warm returns to idle conversations.
A Reddit post sharing contrarian opinions against the /r/LocalLLaMa consensus, arguing that Artificial-Analysis rankings overcorrected toward open-weight models, praising Haiku 5.5, dismissing ThreeJS-style animation benchmarks, and defending the Strata Dev's approach.
QuixiDesk has been significantly improved into a cross-platform (Mac, Windows, Linux) open-source alternative to Claude Desktop that works with any AI model, whether local or cloud-based.
维护者分享了基于 2xRTX 3090 的本地 LLM Pareto 前沿基准图表，指出自 DeepSeek-V4-Flash-Vision-Exp 和 Qwen3.8-Flash-Next 发布两个月来排名未变，并邀请社区协作完善细粒度本地模型能力测绘。