Tag
Charlie Marsh shares that OpenAI's Codex autonomously found a 5-10% speed-up in a formatter while he was working on other tasks.
Charlie Marsh reports a 5% speed-up after asking an unnamed AI or tool to retry an optimization, likely related to Ruff or uv.
Charlie Marsh notes that Codex found a 5-10% speed-up in the formatter automatically, highlighting the power of AI-assisted tooling improvements.
quicktok is a fast and exact BPE tokenizer in C++ that is byte-identical with tiktoken, achieving 2–11x speedup over existing alternatives. It supports cl100k, o200k, GPT-OSS, Llama-3, and Qwen2.5/3 encoders.
llama.cpp natively supports Multi-Token Prediction (MTP) without requiring an extra draft model. By leveraging the model's built-in prediction head, local models like Qwen3.6-27B achieve 1.7x+ speedup, making 27B models run smoothly on consumer GPUs.
A tweet recommending --ddtree-budget 36 for Nvidia RTX 4090, claiming 2.5x speedup during decoding for Qwen3.6_27B.