@wafer_ai: 6 NVIDIA competitors all found the same weakness in CUDA @gpuemi wrote deep dives on 6 nvidia competitors. different te…

6 NVIDIA competitors all found the same weakness in CUDA

@gpuemi wrote deep dives on 6 nvidia competitors. different teams, architectures, and even countries, but all 6 independently arrived at similar conclusions about what’s wrong with CUDA:

every single one killed hardware caches.

tenstorrent, cerebras, trainium, TPU, sambanova, furiosa all use software-managed SRAM with zero hardware caches. NVIDIA is the only major AI chip maker still running L1/L2. the bet is that AI workloads have predictable enough access patterns that speculative caching is wasted silicon. you never pay for cache misses if every byte is explicitly placed.

every single one abandoned threads. CUDA’s thread/warp/block model exists because GPUs were originally built for graphics, where you don’t know which pixels need more work. AI workloads are regular. matmul, attention, MoE routing, the access patterns are completely predictable.

tenstorrent uses producer-consumer pipelines. cerebras uses dataflow with wavelets. trainium writes sequential code and lets the compiler extract parallelism. TPU executes sequentially on a wide vector unit. sambanova maps spatially onto silicon. furiosa compiles tensor contractions directly. six solutions all agreeing threads are the wrong abstraction for AI.

most of them eliminated synchronization entirely. __syncthreads(), atomics, barriers, race conditions, which is arguably the hardest part of GPU programming.

cerebras doesn’t need synchronization because dataflow is inherently ordered.

TPU doesn’t need it because execution is sequential.

sambanova doesn’t need it because the pipeline is spatial.

tenstorrent replaced it with circular buffers.

all six keep data on-chip between layers. on a GPU, every transformer layer reads from HBM and writes back to HBM. on these chips, data stays in SRAM and flows directly to the next operation.

there’s also a clear spectrum forming. tenstorrent gives you a full C++ kernel SDK. trainium and TPU give you python DSLs. sambanova and furiosa currently give you no kernel API at all, but furiosa does plan to offer a low-level programming API (kernel-level) and a higher-level API (DSL-level) to the developer community in the coming months.

generally, more control = higher ceiling but harder to program. less control = at the mercy of the compiler.

stay tuned! will be posting the deep dives this coming week