I benched quad 5060Tis for code generation with Qwen3.6-27B so you don't have to (it's really good)

WEIRD DISCLAIMER: none of this was written by an LLM until you get to the Github repo/site, which was obviously assembled by your friend and mine, Ser Claudric. so if this reads like an ADHD-ass brain wrote it, that's because -- wait for it -- it did. very strange doing a reverse "no really, I'm human, sorry" disclaimer in a post like this, but it's mid-2026 so here we are. Requirements for a while now I've suspected that for $2Kish in cards and $1Kish in rest-of-computer (unless you already have a machine to use) this is probably the best bang for buck you're going to get for code generation in this market and this particular inference landscape, where at "I have some small number of thousands of dollars to spend" level budgets you're trying to run the thing that is probably state of the art and will be for a while, Qwen3.6-27B. so if you're optimizing for that model (again, for code generation), you're looking at a few factors: as close to maximum context (256Kt native) as possible without degradation, which means: to be safe, running at Q8 (do tests say various flavors of Q6 and maybe even Q5_K_XL are basically the same? yeah. do I trust them? not particularly, not yet. with FP16 kv (do tests say Q8 kv is basically the same? yeah. do I trust them? not particularly, not yet.) and with MTP (which really shines for codegen in particular) for single-stream, single-user use, batch size = 1 this should give you nice, fast results with high accuracy and hopefully a minimum of toolcalling fuckups and endless thinking chains. (I could really use some help w/r/t figuring out what harness and sets of plugins or whatever gets you there, btw.) I do HPC hardware and architecture for a living, so to me, the hardware puzzle was, if not the easy part, at least the easier part. (Possible) Solution 4 5060Tis meets the above requirements with the following characteristics: Blackwell precision levels if you decide for whatever reason you DO want to do multi-stream/multi-batch things and want to go down to some flavor of Q4 idle power consumption of around 14-16W in total concomitantly little heat buildable piece by piece with the exception of going from 2 cards to 4 cards, a single $1K step (plus bifurcation hardware, probably) is a little rough no matter which way you slice it) Build Details this benchmark run was conducted on a Vast AI instance! I only own half the equation here -- two 5060Tis running on an X570 board, but I will be expanding it soon enough. that said, one thing you do need to do should you choose to build something like this is have PCI interconnect be nice and even. if you're building fresh, either an X570 or X870E (if you want to go whole-hog PCIe 5.0 for the minor latency drop) board with two physical x16 slots that can each be bifurcated is probably the way to go. the one I'm thinking of with PCIe 5.0 is the ASRock TaiChi Lite: https://www.amazon.com/dp/B0DFNPK2MX when you put a card into both x16 slots they auto-negotiate down to x8 each, which is still fine because you end up after bifurcation with x4 to each card, which gets you 16GB/s bidirectional per card. that is MORE than enough. even with an X570 running the slots at 4.0 instead that's still 8GB/s bidirectional and the inter-card traffic never rises that high during sustained inference. Alternatives other hardware possibilities I considered: pair of 3090s: gets you 48GB VRAM, way better membw and more CUDA cores, but for around $400 more, with no Blackwell, using much more power at idle, with cards that have been beaten to shit by crypto miners, and you can't quite fit the same quants with the same performance. you either sacrifice MTP or FP16 kv or running the model itself at Q8_0. turns out to only barely outperform the quad 5060Tis anyway pair of 20GB modded 3080s: same pros and cons, down to 40GB, but around $800-900 for a pair of cards from AliBaba which is admittedly a very good deal STXH at 128GB: good for what it is, Bosgame M5 is apparently down to $2800 these days, but is a performance dog with dense models like this one DGX Spark/GB10: $5K, great at prefill, trash at decode for dense models like STXH, lol, no M5 Max Macs: see above re: cost at a minimum of $6699 (although that's at least 128GB with actually reasonable prefill compared to older Macs and pretty good tg as well) if I overlooked more options, please lemme know. upshot: https://github.com/jdkruzr/qwen36-27b-5060ti-bench (click the "live report" link in there for prettier pictures) with MTP turned on, at Q8, with FP16 kv on Qwen3.6-27B: I got 608t/s cold prefill and 52.2t/s decode at 256Kt (255Kt, actually, weird borderline edge case I didn't bother troubleshooting) context. to me, these are excellent numbers for $2K of hardware and especially for this particular model which has become infamous for defeating lots of other combinations of hardware that are friendlier to MoE architectures. so, I'm sure I missed something. if so, what was it?