performance

GPT-5.6 Sol now runs at an impressive inference speed of about 750 tokens per second.

An educational thread explaining how GPUs work, focusing on the memory-compute asymmetry that dominates LLM serving performance, and demonstrating how techniques like quantization, speculative decoding, and continuous batching follow from that fundamental constraint.

OpenAI previews Ultrafast, a new service tier for GPT-5.6 Sol that runs up to 14× faster via Cerebras, generating up to 750 tokens per second in the OpenAI API.

fearless_simd v0.7 adds 64-bit integer support, SSE2 level, improved generics, and more operations, with v1.0 upcoming.

This technical blog post explains the performance impact of memory alignment in SIMD vectorization, covering architectures with strict alignment requirements, cacheline crossing, and the behavior of modern CPUs.

A user reports that in Llama-CPP with parallel sub-agents, decode performance is great but a single agent's prefill (e.g., processing a web search) stalls all other agents, and asks for tuning suggestions.

QuixiAI reports running DeepSeek v4 Flash 0731 on 4x A100 with SlimServe, achieving 175 tok/s for single requests and 1k tok/s for 64 concurrent requests.

A technical blog post explains how to perform integer division and remainder using floating-point division and fused multiply-add, with constraints on operand bit widths, and discusses practical considerations for SIMD and rounding modes.

Benchmarks unsloth's Muse Glimmer 30B on an RTX 5090 with speculative decoding, achieving up to 253 t/s using a DFlash draft model and a GPU-based argmax PR, though the PR is still a draft.

Glimmer reportedly hits 233.4 tps on an RTX 5090 using Dflash, with 256k context fitting on 24GB VRAM, sparking excitement among users.

Klaus Post discusses the process of rebalancing deflate compression levels in the Go compression library to make speed/compression trade-offs more linear and intuitive.

nuskey8 released lkv, a lightweight embedded key-value store in Rust optimized for point lookups, claiming ~165x faster than SQLite and ~15x faster than LMDB with a minimal feature set.

A LWN comment discusses the relatively recent implementation of tail-call optimization in C compilers, citing historical limitations and noting that modern GCC and Clang now support it, with potential benefits for interpreter implementations.

The article explains how to speed up running Haskell scripts with Magix on GitHub Actions by caching dependencies and compiled artifacts, reducing full build times from over 100 seconds to near-instant reruns.

Daniel Lemire benchmarks profile-guided optimization (PGO) in Go, showing modest speedups of 2-5% when parsing JSON, with the best gains when the training profile matches the workload.

A technical article explains the true performance costs of garbage collection, contrasting tracing GCs, reference counting, and compile-time memory management in languages like Go, Java, Rust, Swift, and Python.

Windows 11's built-in Weather app can consume over 1 GB of RAM, about five times more than macOS's Weather app, due to its WebView2-based non-native architecture. This could noticeably impact lower-end PCs with 8-16 GB of RAM.

httptap is a Python CLI that dissects HTTP requests into phases like DNS, TCP, TLS, and data transfer to generate detailed waterfall timelines for troubleshooting and performance analysis.

Enabling PCI-E peer-to-peer (P2P) for consumer Nvidia GPUs with patched drivers and vLLM environment variables yields roughly 25% prefill throughput improvement for free, as demonstrated by benchmarks.

Ahmad Osman argues that dense models like Qwen 27B perform poorly on unified-memory systems such as NVIDIA DGX Spark, suggesting MoE models are a better fit; he claims discrete GPUs like the RTX PRO 6000 deliver far better performance for agentic workloads.