@Oluwaphilemon1: Qwen3.8-27B at 56 tok/s on a 9-year-old GPU. Let that sink in. The GPU? NVIDIA V100 32GB. A card that launched at aroun…

Qwen3.8-27B at 56 tok/s on a 9-year-old GPU. Let that sink in. The GPU? NVIDIA V100 32GB. A card that launched at around $11,500 is now available used for roughly $650, and it’s still surprisingly capable for local AI. With DFlash2, Qwen3.8-27B can hit around: → 56–63 tok/s for code generation → Slightly lower speeds for thinking and prose → Even more headroom when ECC is disabled The interesting part is that speculative decoding behaves differently depending on the workload. MTP tends to win for prose, especially when the GPU is power-limited. But for sustained code generation, DFlash2 pulls ahead. That’s a pretty important distinction. You can’t just ask which speculative decoding method is “faster” in isolation. The answer depends heavily on what you’re generating, how predictable the next tokens are, and how much power the GPU has available. And all of this is happening on PCIe 3.0 and an architecture that’s several generations old. Those sound like major disadvantages on paper. In practice? The software stack is getting good enough that they aren’t nearly as crippling as you’d expect. That’s what makes the V100 interesting again. You don’t need a modern $2,000+ GPU to experiment with serious local models. A used datacenter accelerator from the previous decade can still deliver ridiculous inference performance when paired with the right runtime and speculative decoding. I’m also working on a repo to make the setup easier to reproduce, so you won’t have to spend hours figuring out the configuration yourself. The V100 was once an $11,500 flagship accelerator. Today, for around $650 used, it’s still pushing a 27B model at tens of tokens per second. https://x.com/KyleHessling1/status/2093509375352799338/video/1…