cuda

A GitHub repository providing a structured 100-day learning plan for LLM inference engineering, covering topics from CUDA kernels to autoscaling, with runnable scripts.

A developer forked NInfer, a C++20/CUDA inference engine, to add tensor-parallelism and YaRN rope scaling, enabling Qwen3.8-27B to run with a 1M token context on dual 5090 GPUs and outperforming vLLM in specific decode scenarios.

Chinese robots dominate global humanoid shipments at 86%, and Nvidia is expanding its CUDA-style ecosystem into the concentrated robotics market, as reported by WSJ.

An adaptive KV cache streaming fork of llama.cpp enables running the Qwen 3.8 27B model with 262K context on a 16GB RTX 5070 Ti GPU, achieving ~25 tok/s by efficiently managing memory between RAM and VRAM.

This paper introduces DataKernelBench, a benchmark for evaluating LLMs on optimizing GPU kernels for database queries, achieving speedups over baseline methods like torch.compile.

This is a fork of NVIDIA's open GPU kernel modules that enables PCIe peer-to-peer communication between consumer GeForce GPUs (3090, 4090, 5090), significantly improving multi-GPU performance by allowing direct data transfer over PCIe.

The discussion compares CUDA's maturity to MLX in the context of Apple's M5 Ultra Mac Studio, stressing the need for competition to optimize hardware for local AI, which could enable Apple to surpass NVIDIA.

This paper presents a feature-major codebook layout for memory-efficient sparse-binary self-organizing maps, enabling the training of large-scale maps up to 1.05 million neurons on a single GPU with significant speed improvements for MEDLINE data.

At Hot Chips 2026, Nvidia announced plans to extend CUDA support to RISC-V CPUs, outlining specific hardware requirements like server-grade CPUs, ACPI, and PCIe coherency to enable efficient GPU compute.

The user is comparing GPU options like R9700, Mi210, and 4080S to achieve 128GB VRAM for running multiple AI models in parallel, considering factors like cost, performance, and compatibility.

This blog post details the hardware path of a GPU memory read instruction on an NVIDIA RTX 4090, explaining how a CUDA kernel accesses memory through components like L1 cache and DRAM for performance insights.

NVIDIA has released an NVIDIA-hosted CUDA MCP Server and an open-source Nsight Copilot Blueprint to provide AI-assisted CUDA development, including documentation access, code generation, and performance analysis.

A detailed write-up on solving an OSINT challenge to geolocate an island from a photo using geometric analysis, CUDA programming, and OpenStreetMap data filtering.

PTXBench is introduced as a benchmark to evaluate and adapt large language models for optimizing GPU kernels using architecture-specific PTX, showing uneven performance and fine-tuning insights.

KernelArc is a multi-agent framework that uses strategy-specialized agents to autonomously optimize GPU kernels across heterogeneous workloads, achieving top rankings on NVIDIA GPU benchmarks.

The article explains CUDA shared memory swizzling techniques to optimize GPU memory access patterns, with code examples demonstrating performance improvements.

An educational tweet explaining the three levels of NVIDIA's software stack (CUDA C++, PTX, SASS) and how CUDA's abstraction creates a moat, while mentioning Luminal's automatic compiler search.

Cursor open-sources Mixture-of-Kittens (MoK), a deterministic MoE training megakernel for NVIDIA Blackwell GPUs that fuses computation and communication, delivering up to 2.37x speedup over baseline implementations.

A user reports that updating CUDA from 13.2 to 13.3 fixes a looping problem with DeepSeek V4 Flash 0731, making the model usable again for long coding tasks.

NVIDIA and PyTorch explain how kernel fusion in CUDA improves GPU memory bandwidth by combining multiple operations into a single kernel, reducing round-trips through global memory and kernel launch overhead.