speedup

HyperspaceDB v3.1.4 introduces advanced quantization techniques for significant speed improvements, a drop-in replacement for Mem0 and Zep with lower latency, and built-in tracking for agent trajectories and stability analysis.

This paper introduces ClusterAttention, a training-free method to speed up bidirectional attention in transformers by using recursive clustering for block-sparse attention, achieving 2-6x speedups on tabular data and 1.8x on video generation while maintaining high accuracy.

An early-preview LoRA for MiniMax-H3 enables 4-step audio-video generation instead of ~20 steps, yielding roughly 5x faster sampling, though quality is still immature.

A llama.cpp PR moves sampling from CPU to GPU, yielding 8% faster tokens on an RTX 5090 and ~4% on a Tesla P40 for Qwen3.6-35B inference.

SpecLA proposes a speculative decoding runtime tailored for stateful linear-attention models, achieving up to 1.70x end-to-end speedup over autoregressive decoding on an NVIDIA H100 with a GDN-1.3B target.

The FastVideo team releases new FP4 attention kernels for B300, achieving up to 1.69x speedup over FlashAttention 4.

Unsloth releases quantized Qwen3.6 models using NVFP4 format, achieving 2.5x faster inference speeds.

Two Hong Kong students achieved a 5x speedup by adding another loop outside the original automated research framework, without needing a better model or more compute. It is considered one of the most useful papers for ordinary Agent developers.

Blog post by Sankalp detailing how he used Codex to achieve a 232x faster GPU kernel for QR decomposition in GPU Mode's contest, outlining his auto-research methodology.

A roofline model from the LLM Engineer's Almanac estimates speedups from speculative decoding for different draft lengths across models and hardware, with a note that it may underestimate benefits when overhead is significant.

Fable achieves top rank on KernelBench-Mega by writing a highly efficient CUDA megakernel with an 18.71X speedup, signaling progress toward recursive self-improvement in AI R&D.

The author used Codex to optimize DeepSeek V4 Flash 8-bit MLX on oMLX, achieving approximately 1.6x prefill and 3x decode speedup.

Ollama announces that Gemma 4 is now nearly 90% faster on Apple Silicon using MLX, thanks to improved multi-token prediction enabled by default, with automatic tuning to avoid slowdown.

DeepSeek released a paper and MIT-licensed open-source implementation of speculative decoding (DSpark) that speeds up LLM responses by up to 80% by using a small 'guess' model and a large 'check' model, achieving both speed and accuracy without tradeoffs.

JetSpec introduces parallel tree drafting for speculative decoding, achieving up to 9.64x end-to-end speedup on LLM inference while maintaining lossless accuracy, with throughput reaching ~1000 TPS on a single B200 GPU.

Nvidia claims a 15x speedup in text generation using a diffusion model, generating entire blocks at once.

GLM 5.2 delivers major performance gains on Mac Studio with 512GB RAM, achieving prefill speeds above 100 t/s at high context lengths and enabling 4-bit quantization for contexts over 100k tokens, as detailed in a pull request by the oMLX creator.

Flash-KMeans is an IO-aware implementation of exact KMeans that redesigns the algorithm around modern GPU bottlenecks, achieving 33x speedup over cuML and 200x over FAISS by eliminating redundant memory reads and writes.

Anima Anandkumar highlights that neural operators, despite simple benchmarks, have achieved massive speedups (10,000–million times) in hard real-world problems like high-resolution AI weather modeling (FourCastNet) and nuclear fusion turbulence, referencing a new paper showing learned solvers become more cost-effective as PDE tasks get harder.

This paper accelerates the NeurASP neurosymbolic AI framework by implementing vectorization, batch processing, and caching, achieving multiple orders of magnitude speedup on larger tasks.