Tag
作者测试了 Strata 0.2.7.1 的两项路由优化（STRATA_ROUTE_PRIOR 和 STRATA_ROUTE_TAIL_SKIP）在 Qwen 3.8 Flash Next IQ3_S 上对编码、数学和指令遵循任务的影响，发现 route prior 可大幅提升解码速度且质量损失在长推理下基本可忽略。
作者分享在Strata上从IQ2_XS迁移到Unsloth UD-IQ4_XS时的经验：0.1.45之前缺少sampling配置会默认贪心解码导致模型循环，以及默认STRATA_ROUTE_TAIL_SKIP=7会牺牲4-bit量化质量，并给出配置与校准建议。
A user shares benchmarks showing that raising the --adapt-every parameter from 1 to 6 in Strata boosts token generation speed by 9-55% across multiple Qwen 3.8 Flash Next prompts, with the largest gains on long-context code descriptions.
一位用户分享了对本地 LLM 推理工具 Strata 的深度性能调优经验，通过调整 --pcie-frac、--pool-workers、--trim-stage-weights、--pipeline-windows 等参数并在双 RTX 3060 上优化 GPU 优先级，实现了约 35 tps 到 60+ tps 的解码速度提升。
Second post in a three-part Percona series explaining how Profile-Guided Optimization (PGO) works: instrumented two-pass builds, per-edge counter collection, and how the compiler uses that data to drive inlining, branch layout, hot/cold function splitting, and instruction-cache locality.
ROCmFPX 更新了融合 HC norm 的匹配器，使其能识别 qwen4exp 将 REPEAT/MUL/ADD 合并为单一 op 的组合，从而将 HC 内核每 2048 token 耗时从 416 ms 降至 175 ms。相关修复还解决了导致 llama-perplexity 崩溃的 xn 别名问题，并在 BF16 拷贝路径上移除了变慢的 Q8_0 MMB down proj。
A dual-DGX-Spark user reports that a new GLM 5.3 Flash recipe delivers a 50-90% decode speed boost, making it faster than DeepSeek v4.0 Flash while offering better reasoning quality on local hardware.
The author proposes optimizing llama.cpp for specific models by stripping unnecessary code and using AI to automate the process, aiming for 2x+ performance improvements.
A user shares their experience switching to the QWEN3.6-27B-MLX-8bit model for local AI tasks, finding it performs comparably to larger models while saving significant RAM on a Mac Studio, improving workflow stability.
A user has built a dual R9700 rig and is seeking community advice on various AI inference, fine-tuning, and system optimization topics.
This article introduces an open-source GPU profiler that serves metrics in JSON format, allowing AI agents to automate performance tuning for CUDA-based AI engines like vLLM and SGLang without manual trace analysis.
Optimizations for GLM 5.3 Flash on Apple M3 Ultra achieve up to 550 t/s prefill and 38 t/s inference speed through kernel fusion and efficient memory use, without quality loss.
A discussion questioning the common interpretation of premature optimization in software development, referencing insights from performance expert Casey Muratori.
The article provides optimized settings for using the Qwen 3.8 model with llama.cpp, highlighting performance gains in token generation speed and context window management on an RTX 3090 GPU.
This article provides a detailed guide on running the Qwen3.8-Flash AI model on an RTX 3090 with 64GB RAM, discussing performance metrics, quantization settings, and deployment steps using tools like llamacpp.
Scott Hanselman debugs slow 10 Gigabit Ethernet performance by adjusting Intel NIC receive buffers and disabling Large Send Offload, identifying a Windows-specific issue.
Reverse-engineered the Axera AX8850 NPU's int8 weight format to enable direct GGUF inference in llama.cpp, achieving up to 24.5 t/s decode and 716 t/s prefill on a Raspberry Pi 5, outperforming the vendor's runtime by 1.5×.
The article details progress in achieving over 50 tokens per second inference speed for DeepSeek-V4-Flash on M3 Ultra using the DwarfStar inference engine, with improvements in performance and quality metrics.
A user shares how they achieved a 2x performance boost in AI inference by splitting a large MoE model between a Strix Halo APU and an R9700 GPU, detailing configurations and code modifications.
An article reports that enabling CPU affinity on a 24-core build machine unexpectedly slowed down builds, highlighting complexities in optimizing multi-core processing.