Tag
NVIDIA shares debugging lessons from its Exemplar Cloud program, detailing how configuration issues in SMMU power management, NUMA placement, NCCL queue-pair concurrency, and hardware defects cause 8-12% training throughput gaps on AI clusters, and how to diagnose and fix them.
A guide on optimizing MTP (Multi-Token Prediction) performance by tuning n_max parameter, with benchmark results for various models like Gemma-31b and Qwen on P100 and V100 GPUs.
The author shares techniques that improved training pipeline performance from 36 to 47 steps per minute.
ClickHouse Managed Postgres scales PgBouncer to 4x throughput by running a fleet of processes with SO_REUSEPORT, enabling multi-core utilization and solving cancellation forwarding via peering.
Benchmarking shows that optimizing Java code (e.g., reducing SLF4J logging) has a far greater impact on latency than choosing a garbage collector, especially at high percentiles.
This article describes using the Codex AI agent to automatically migrate terminal shell configuration from Oh My Zsh to Zinit + Starship + Rust toolchain, demonstrating the AI's ability to perform engineering steps such as backup, key isolation, and performance analysis, ultimately achieving an order-of-magnitude improvement in startup speed.
This article provides a comprehensive overview of the complete technology stack for cloud deployment of Transformer inference, covering application scenarios, workload definition, models, inference engines, hardware, observability, and performance optimization, along with future trends.
This article describes how to use the SYCL backend with llama.cpp to achieve over 60 tokens per second on the Qwen 3.6-35B-A3B model using an Intel Arc Pro B70 GPU, with the entire model and KV cache in VRAM.