@fahdmirza: Luce Megakernel just proved the NVIDIA efficiency gap is a software problem not a hardware one a 2020 RTX 3090 at 220W …
Summary
Luce Megakernel demonstrates that NVIDIA's efficiency gap is a software problem, achieving 1.8x throughput on an RTX 3090 compared to llama.cpp and matching Apple M5 Max efficiency at lower cost.
View Cached Full Text
Cached at: 05/16/26, 07:22 PM
Luce Megakernel just proved the NVIDIA efficiency gap is a software problem not a hardware one
a 2020 RTX 3090 at 220W now matches Apple M5 Max efficiency and delivers 1.8x the throughput
413 tok/s decode vs 267 tok/s on llama.cpp — same GPU, different software 1.87 tok/J — matching Apple M5 Max at less than a third of the system cost All 24 layers of Qwen3.5-0.8B fused into a single CUDA kernel — zero CPU round trips 25x faster than PyTorch HuggingFace on the same hardware Hybrid DeltaNet and Attention architecture — the first megakernel ever built for this pattern
Full breakdown and live benchmark below
indeed.
Similar Articles
Luce Megakernal: Why nobody is taking about this?
Lucebox Hub provides optimized CUDA kernels (Megakernel, DFlash, PFlash) for local LLM inference, achieving significant speedups (2-10x) over llama.cpp on various models and GPUs.
@Snixtp: More efficiency tests on a single 3090 TL;DR: - I tested 8 local LLMs on a single RTX 3090, power limit from 100W to 45…
The article presents benchmark results for 8 local LLMs on an RTX 3090, showing that power efficiency peaks around 225W, with diminishing returns at maximum power.
@TheAhmadOsman: Why do I focus on Inference Engines/Software Stacks for your hardware? - 2x RTX 3090s: ~14.5 tok/s → ~64 tok/s moving t…
Comparison of inference engine performance on different hardware: moving from baseline to vLLM with TP=2 on 2x RTX 3090s improves from ~14.5 tok/s to ~64 tok/s, and on RTX PRO 6000 moving to Sglang improves from ~32 tok/s to ~110 tok/s. Recommends vLLM/Sglang for CUDA/multi-GPU and llama.cpp for edge devices.
Luce Spark: a 35B MoE on a 16 GB GPU, without the offload tax
Luce Spark is an open-source tool that enables running 35B MoE models on 16GB GPUs by intelligently caching hot experts on the GPU while keeping the rest in system RAM, using a calibrated placement and bounded async cache to maintain high throughput without the usual offload speed cliff.
@TeksEdge: A localmaxxer hit ~381 tok/s on a SINGLE RTX 3090 with Qwen3.8-27B. This developer has turned a 24GB RTX 3090 into a mo…
A developer achieved up to 381 tok/s inference speed on a single RTX 3090 with the Qwen3.8-27B model using optimized techniques like DFlash2 and prefix caching, particularly effective for document-based tasks like RAG and coding assistants.