Tag
A user extracted the pure DFlash draft part from Ling-3.0-tiny-UBTree (dropping the ubtree_selector tensors) so it runs on stock mainline llama.cpp, reporting +8% speedup for prose and +61% for Python code generation, with the converted GGUF and conversion procedure published on Hugging Face.
Kapitsa, an open-source inference engine optimized for Apple Silicon, achieves 128 tok/s decode on a MacBook Pro M4 Pro 48GB running Recursive's new Flint 40B Q4 model, outperforming llama.cpp and Recursive's own engine in long-context decode and prefill benchmarks.
该论文研究了推测解码中目标模型概率的复用问题，提出在树分配前利用目标概率进行重排序和首词采样，可在小树预算下提升期望接受率并加快解码速度。
This paper shows that regenerating target model responses shrinks the gains that draft-training objectives like knowledge distillation provide in speculative decoding, replicating findings across vision-language (Qwen3-VL-8B) and text (Qwen3-4B) settings with multiple controls.
A hobbyist researcher proposes a speculative decoding scheme where a 2-bit model copy on GPU drafts tokens while the full residual (6-8 bit) in system RAM verifies them, only exchanging activations over PCIe. Early measurements suggest 2-bit + 8-bit residual outperforms Q8_0 quality and the 2-bit drafter beats Gemma 4's official drafter on acceptance rate.
OpenBMB 的工程实践展示了如何通过 Lookahead Expert Scheduling（LES）在 iPhone 上以 20.51 tok/s 运行 35B-A3B MoE 模型，无需跳过专家或改变路由，相关实现已在 llama.cpp-omni 开源。
Nace.AI's Kapitsa 0.2 ships KFlash, a best-first tree-based speculative decoding method for Apple Silicon, benchmarked against Splash 1.4.0's DFlash2, mlx-serve, Ollama, llama.cpp and others on Qwen models across M4 Pro and M1 Pro MacBooks, showing top decode throughput (~65-201 tok/s).
DStill 是一个两阶段轨迹一致性蒸馏框架，将块扩散投机解码中多步精修的 drafter 蒸馏为单步 drafting，在三个模型族上相较单步 DFlash 最高提升 28% 接受长度和 25% 加速，较自回归解码最高达 6.45 倍加速。
EAGLE-4 发布，作为投机解码加速组件在 Qwen3.8-27B 上达到 961.6 tokens/s，单请求下比 DFlash 2 快 1.28 倍、比 DSpark 快 1.40 倍，通过 draft-and-verify 循环实现约 3 倍加速且输出完全一致。
DeepSeek V4.1 Flash 的大规模推理部署文章，介绍 DeepInfer 结合分布式推理、缓存、负载均衡与投机解码提升效率，DeepSeek 单集群日处理 155 万亿 token，每 GPU 小时收入 38 美元，并开源了大部分推理框架。
EAGLE-4 发布：新一代投机解码草稿模型与 eagle-harness 工具包，可跨批量大小、硬件和工作负载自适应加速 LLM 推理，速度超过 DFlash2 与 DSpark，支持 SGLang、vLLM 和 TokenSpeed 引擎。
Continuity Research 发布 Tess-4.5-27B，一个基于 Qwen3.8-27B 微调的 27B 开源权重模型，主打本地单卡运行的工程与智能体能力，BenchLocal 成绩与基座持平但推理 token 少 25%，并附带多 token 预测草稿头以提升本地推理速度。
ProVe reverses speculative decoding roles by fine-tuning a large model to propose token blocks verified by a smaller or identical model, achieving up to 2.7x throughput with adjustable speed-accuracy tradeoffs across six benchmarks.
Specter 提出了一种通过量化实现的多步推测性专家卸载方法，在单 GPU 上加速 MoE 推理，相比最先进系统平均获得 1.50 倍（最高 1.88 倍）的 TPOT 加速。
Hugging Face 主播了一场名为 Local AI 202 的直播，介绍如何通过推测解码（Speculative Decoding）大幅提升本地 AI 推理速度，现提供回放。
incoai发布了DFlash 2草稿模型（Qwen3.8-Flash-Next-DFlash2），这是一种用于推测解码的块扩散式草稿器，可在单次前向中预测整个token块，并通过SGLang实现无损解码加速，在接受长度上优于内置MTP。
Nace.ai 发布开源（Apache 2.0）Apple Silicon 优化 Metal 推理引擎 Kapitsa，专为 Qwen 和 Nemotron 模型定制内核，并在 M1 Pro、M4 Pro、M5 Max 上公布对比 MLX、llama.cpp、Ollama 等引擎的解码基准。
提出 CADENCE（Context-Adaptive Decoding）方法，通过高斯预测头与学习式路由自适应选择每次解码的跨度长度，在 Llama-3.1-8B-Instruct 上实现最高 1.3× 的生成吞吐提升。
The paper introduces SPOT, a 0.62M-parameter drafter that predicts token commit order for diffusion language models, enabling lossless tree-based speculative decoding with coupled-noise match verification that cuts target passes by up to 6.8x and boosts throughput up to 3.2x across six DLMs.
A hobbyist built a distributed inference system chaining two gaming PCs, an M4 Pro MacBook, and two iPhone 17 Pro Maxes to run Qwen 3.8 Flash Next (a 48-layer MoE model with 512 experts), achieving 37-39 tok/s at 60k context via pipelined speculative decoding across the device chain.