Tag
GPT-6 Sol and Claude Opus 5.5 achieve near-frontier performance at a fraction of the cost and speed of previous generations, highlighting major efficiency gains.
This article benchmarks the performance of subnormal floating-point numbers on Intel, AMD, and ARM processors, revealing that Intel processors experience significant slowdowns with subnormals while AMD and ARM are unaffected.
A user benchmarks exllamav3 against llama.cpp for CPU-offloaded inference, showing exllamav3 is faster for Qwen models but slower for others, depending on hardware and model architecture.
The user shares benchmark results showing vLLM's significantly faster prefill performance compared to llama.cpp and other engines, and questions the technical reasons behind this speed difference.
The author built FlashMLA for consumer-grade Blackwell sm_120, achieving 2-3x performance gains over PyTorch SDPA in attention-heavy workloads like long-context training and sparse prefill.
The author conducted experiments to run DeepSeek-V4-Flash-0731 with 4-bit quantizations on a 128GB RAM system, using optimizations like memory mlocking and prompt processing strategies to achieve acceptable inference speeds.
Google's new Gemma 4 12B model claims near-26B performance. In a local test on RTX 4090, the 26B-A4B model was faster and better but the 12B used less VRAM, making it suitable for laptops.