Tag
Liquid AI releases DSpark draft models for their LFM series, incorporating speculative decoding to achieve up to 4x decode speedup on device while maintaining output quality.
Charlie Marsh shares that OpenAI's Codex autonomously found a 5-10% speed-up in a formatter while he was working on other tasks.
Charlie Marsh reports a 5% speed-up after asking an unnamed AI or tool to retry an optimization, likely related to Ruff or uv.
Charlie Marsh notes that Codex found a 5-10% speed-up in the formatter automatically, highlighting the power of AI-assisted tooling improvements.
quicktok is a fast and exact BPE tokenizer in C++ that is byte-identical with tiktoken, achieving 2–11x speedup over existing alternatives. It supports cl100k, o200k, GPT-OSS, Llama-3, and Qwen2.5/3 encoders.
llama.cpp natively supports Multi-Token Prediction (MTP) without requiring an extra draft model. By leveraging the model's built-in prediction head, local models like Qwen3.6-27B achieve 1.7x+ speedup, making 27B models run smoothly on consumer GPUs.
A tweet recommending --ddtree-budget 36 for Nvidia RTX 4090, claiming 2.5x speedup during decoding for Qwen3.6_27B.