gpu

NVIDIA explains how attention architecture choices (group size, head dimension, KV-cache size, parallelism) set the ceiling for long-context inference performance, with guidelines for co-designing models for faster serving.

A PSA reminding owners of older GPUs like the RTX 3090 to replace dried-out thermal paste, which can lower temperatures by 10°C and reduce fan noise. It's a cheap, quick fix but requires care.

The author observes that Nvidia's desktop 70-class GPUs have stayed at 12GB VRAM across two generations, and suggests Nvidia may be intentionally limiting memory to preserve demand for higher-margin AI-focused hardware.

This open-source project enables Nix and NixOS on the NVIDIA DGX Spark, providing USB boot images and a NixOS module for configuring the hardware.

A tweet thread shares a curated list of resources for learning LLM inference, covering tokenization, GPU hardware, optimization techniques like quantization and speculative decoding, and inference engines such as vLLM and SGLang, followed by an Nvidia architect's talk summary.

The tweet shares evidence that NVIDIA intentionally shapes its software ecosystem to make newer workflows appear Blackwell-exclusive while leaving Ampere paths unsupported, advising users to wait before upgrading and suggesting NVIDIA's moat is weakening in favor of Intel and AMD.

A tweet highlights that DeepSeek V4 Flash's performance can vary by up to 7x on the same hardware solely due to kernel selection, underscoring the importance of configuration.

The Futhark compiler now supports full flattening of nested data parallelism, allowing any Futhark program to be compiled to parallel GPU code, a milestone achieved after years of work by students and researchers.

Astral is open sourcing its build pipelines for pre-built wheels of GPU-enabled Python packages like FlashAttention and DeepSpeed, making them available to all via standard Python indexes.

Best Buy has raised the price of the Asus ROG Astral RTX 5080 OC to $2,099, exceeding the RTX 5090's MSRP, reflecting ongoing component shortages and GPU price hikes.

This paper presents an analytically structured, empirically calibrated methodology for estimating LLM inference energy on NVIDIA H100 GPUs without direct measurement, separating prefill and decoding phases and decomposing energy into compute, parameter-access, KV-cache write, and attention-read components.

Escha-W2 is a 2-bit quantized build of the Qwen3.6-35B-A3B MoE model, packaged with runtimes for local serving via an OpenAI-compatible API. It requires a 24 GB GPU (or 16 GB with trade-offs) and is available on Hugging Face.

This paper presents a method for evaluating Bézier curves on the GPU using texture lookups, improving efficiency for graphics applications.

Explains how shaders work using an Excel formula that computes color per cell, illustrating GPU parallelism.

Kernel Forge is an open-source agent harness that uses LLMs and Monte Carlo Tree Search to automatically generate and optimize CUDA kernels for any unmodified PyTorch model, achieving up to 2.83× speedup on softmax in Gemma 4 E2B.

Nvidia is reportedly raising GPU prices by up to 30% due to rising graphics memory costs, affecting all RTX models. This marks the third price hike since January, likely passed to consumers as inventory depletes.

SK Hynix stock has dropped 40% in 30 days, potentially signaling lower RAM and GPU prices due to eased supply constraints.

ASRock announces the Radeon RX 9050 with 4GB VRAM, the first modern GPU in years with such low memory, likely targeting budget builds due to component shortages.

Kimi K3 weights are being released today. The model has 2.8T parameters, MoE with 896 experts, 1M context, vision, and MXFP4 quantization. Deployment requires multiple nodes for A100s and H200s, but fits in single B300 node. Benchmarks for tok/s, ttft, and cost per M token across GPU configs are expected by end of week.

LMCache is an open-source KV cache management layer that separates cache I/O from compute, plugging into vLLM, SGLang, and TensorRT-LLM to achieve up to 14x faster time-to-first-token and 4x faster decoding by parallelizing cache lookups and sharing GPU memory.