CUDA 13.3 and PTX 9.3

CUDA 13.3 is not just a routine point release.

The obvious headline is “new toolkit release,” but the more interesting story is that NVIDIA is continuing to expose more of the machine: tile-level programming, fabric operations, richer mbarrier semantics, async multimem, better runtime compilation support, and a few real compiler/library fixes.

If you write kernels, build runtimes, tune collectives, or care about Blackwell-era programming models, this one is worth reading.

The shape of the release

CUDA 13.3 ships with updated compiler, runtime, library, and developer-tool components.

The important versions:

• CUDA NVCC: 13.3.33

• NVRTC: 13.3.33

• CUDA Runtime: 13.3.29

• cuBLAS: 13.5.1.27

• Nsight Compute: 2026.2.0.7

• Nsight Systems: 2026.1.3

CUDA 13.x minor compatibility still lists driver >= 580, but the CUDA 13.3 packaged Linux driver branch is 610.43.02. That matters because some of the more interesting platform features are tied to the newer r610 stack.

Tile C++ is getting more serious

CUDA Tile C++ now shows up as a first-class part of the story.

This is NVIDIA pushing toward a more explicit tile-oriented programming model: closer to the level where kernel engineers think about data movement, matrix tiles, layout, and scheduling, without dropping all the way into raw PTX/SASS for everything.

In CUDA 13.3, Tile C++ support is present in both nvcc and NVRTC.

That matters because it makes the model useful in both ahead-of-time and runtime compilation workflows.

There are also new tile operations and types, including support around scaled MMA, pack/unpack operations, strided and gather/scatter-style views, i4, and a 4-bit float type.

This is probably the “watch this space” part of CUDA 13.x.

C++23 support is official

CUDA 13.3 adds official C++23 support in nvcc and NVRTC.

Not the flashiest thing in the release, but for real CUDA codebases it matters. Runtime compilation, template-heavy kernels, CCCL usage, and modern host/device C++ all benefit from CUDA staying closer to contemporary C++.

NVRTC gets better header handling

NVRTC can now install and use curated CUDA C++ / CCCL headers without requiring the same amount of manual toolkit/header plumbing.

That is a practical improvement for systems that compile kernels dynamically: Python runtimes, autotuners, DSLs, model compilers, inference engines, and internal kernel-generation stacks.

CUDA Python crossed an important line

CUDA Python now has stable 1.0 APIs for cuda.core and cuda.compute.

cuda.compute is especially interesting because it exposes CUB-backed algorithms from Python:

• reduce

• scan

• sort

• histogram

• select

• lower / upper bound

This is not “Python replacing CUDA C++.” It is more like CUDA’s low-level ecosystem getting a better programmable surface for tools, prototyping, and generated workflows.

PTX 9.3: fabric, mbarriers, and async multimem

PTX 9.3 is where the kernel-engineer candy is.

The new stuff is mostly about synchronization, fabric-level operations, async multimem, and more precise control over memory semantics.

New fabric instructions

• fabric.try_get

• fabric.try_put

• fabric.try_red

• fabric.try_pullred

• fabric.wait

• fabric.submit

New async multimem operations

• multimem.st.async

• multimem.red.async

More control over memory semantics

• .sem / .scope qualifiers for async bulk copy and reduce

• fence.proxy.to_proxykind::from_proxykind_fabric

More mbarrier machinery

• .phase_type::*

• reportPredicate

• reportValue

• .layout

• mbarrier.check_layout

Other PTX additions

• mma_throughput pragma

• clmad instruction

There is also a semantic clarification: red.async supports sys scope, not cluster scope, with .release.

The big read: NVIDIA is putting more formal machinery around multi-GPU / fabric-visible behavior, async memory movement, and barrier state. This is the kind of PTX surface area that runtime authors and people writing serious collectives should pay attention to.

MPS gets partial error isolation

CUDA 13.3 adds partial error isolation for MPS using static SM partitions.

Historically, one nasty client fault under MPS could have broader blast radius than you wanted. This feature lets MPS isolate some SM-triggered faults to a partition/client.

It is not perfect isolation. NVIDIA is explicit about that.

But it is still a big operational improvement for shared-GPU systems: inference services, mixed online/offline workloads, small query workloads, robotics/embedded stacks, and anything trying to keep independent processes alive on the same GPU.

DMA-BUF mmap for discrete GPUs

CUDA 13.3 adds mmap() support for DMA-BUF file descriptors exported from CUDA device memory on discrete GPUs.

The practical translation: there is now a lower-latency CPU mapping path for exported GPU memory in some environments, without necessarily relying on GDRCopy kernel drivers.

If you work on data paths between GPU memory, CPU control code, I/O devices, or external memory abstractions, this is worth a closer look.

cuBLAS gets meaningful Blackwell wins

cuBLAS 13.3 has several notable improvements:

• Green context support

• FP64 emulated GEMM workspace capped so it no longer exceeds 8 GB

• Better FP4 matrix multiplication performance on Blackwell Ultra

• Better TF32 matrix multiplication performance on Blackwell and Blackwell Ultra

• Better TF32 TN matmul performance on Hopper

• TMA-based SYMV acceleration for Hopper, Blackwell, and Blackwell Ultra

The TF32 line is the one that jumps out. NVIDIA reports a 27% geometric mean improvement across Blackwell / Blackwell Ultra problems and layouts, with some small problems up to 3.5x faster.

As always, benchmark your actual shapes.

Correctness fixes worth noticing

Two correctness fixes stood out.

First: CUDA 13.3 fixes a compiler issue present since CUDA 12.8 where compiler-inserted thread reconvergence could fail in kernels with multiple nested levels of divergence. That could leave stale or corrupted register values and cause incorrect execution.

That is the kind of bug kernel engineers should care about immediately.

Second: CUDA Math fixed a silent data corruption issue involving __mul24() with compile-time constant inputs. That issue was introduced in CUDA 11.1 and resolved in CUDA 13.3.

Known caveats

A few caveats:

• Legacy Nsight Eclipse Edition plugins are no longer delivered.

• CUDA Tile C++ has a known issue where printf() from a tile function can produce an incorrect diagnostic when GLIBC fortification level 2+ is enabled. Workaround: add -U_FORTIFY_SOURCE.

• Compute Fabric Transport has some known issues. Fabric Manager 610+ is required for unicast logical endpoints, fabric.try_pullred is currently suboptimal, and some logical-endpoint memory bindings may not behave correctly for same-process multi-GPU pointer access.

Bottom line

CUDA 13.3 is interesting because it continues the CUDA 13.x trend: more explicit control over the machine, more tile/fabric-level programming surface, better runtime compilation ergonomics, and meaningful Blackwell-era library tuning.

For most application developers, this is a nice toolkit update.

For kernel engineers, compiler people, runtime authors, and anyone building GPU systems across multiple devices, PTX 9.3 is the part to read carefully.

This was written by GPT 5.5 Medium Fast