GateGPT: 56k tokens per second Transformer (KV cache) on FPGA at 80 MHz

Hacker News Top 06/16/26, 04:12 PM Tools

fpga transformer kv-cache hardware-acceleration micro-gpt custom-chip high-throughput

Summary

A custom FPGA implementation of a Transformer with KV cache achieves 56,000 tokens per second at 80 MHz, running microGPT on a tiny LCD.

56,000+ tokens/sec at just 80 MHz. I burned a full Transformer with KV cache into a custom chip. Designed gate by gate as a 100% digital integrated circuit. Prototyped on a FPGA. (No GPU. No CPU) Just pure digital silicon running @karpathy microGPT, spelling out names on a tiny LCD. This is GateGPT

Original Article

View Cached Full Text

Cached at: 06/16/26, 05:36 PM

56,000+ tokens/sec at just 80 MHz.

I burned a full Transformer with KV cache into a custom chip. Designed gate by gate as a 100% digital integrated circuit. Prototyped on a FPGA. (No GPU. No CPU) Just pure digital silicon running @karpathy microGPT, spelling out names on a tiny LCD.

This is GateGPT

Similar Articles

@VedaAI00: When Transformers Are Hardwired Into Hardware, Inference Speed Changes Orders of Magnitude. Luthira demonstrates a complete implementation of MicroGPT on an FPGA array. No GPU, no PyTorch, not even a CPU inference loop, just a pure hardware-driven Tra…

X AI KOLs Timeline

Luthira fully implements MicroGPT on an FPGA array, without GPU, PyTorch, or CPU inference loop, achieving inference speeds of 50,000+ tokens per second, demonstrating the potential of hardware-based Transformers.

Microgpt in pure C hits 10M tps on Apple m5

Hacker News Top

The microGPT-C project provides a minimal, dependency-free implementation of a character-level transformer in pure C, achieving over 10 million tokens per second inference speed on Apple M5 hardware.

Show HN: TurboGPT: train 22KiB transformer in 13s

Hacker News Top

TurboGPT is an open-source CUDA C++ tool that enables rapid training of a tiny byte-level GPT model, achieving 2.5295 BPB on 1.5G tokens in 13 seconds.

@mweinbach: Who said TPUs can't be fast! This is roughly Groq speeds out of TPU 8i, but with Gemini Flash model so far more intelli…

X AI KOLs Timeline

Google demonstrated Gemini Flash model achieving 600-1400 tokens per second on TPU 8i, rivaling Groq's inference speeds.

Tiny Scale Is All I Can Spare To Play With Transformer

Reddit r/LocalLLaMA

A student introduces Silia, a novel transformer architecture that combines attention and FFN into a unified operation to save parameters at scales ≤10M, achieving comparable performance to GPT-2 with fewer parameters despite limited compute resources.