@no_stp_on_snek: @antirez Turbo3 BEATS fp8 by +5% decode tok/s at 32K context still tinkering but i've been cooking TQ+ in your kitchen
Summary
Turbo3 achieves 5% faster decode tokens per second compared to fp8 at 32K context, a performance improvement in quantization or model optimization.
View Cached Full Text
Cached at: 05/25/26, 10:45 PM
@antirez
🔥Turbo3 BEATS fp8 by +5% decode tok/s at 32K context
still tinkering but i’ve been cooking TQ+ in your kitchen
Similar Articles
@no_stp_on_snek: TurboQuant+ updates. 4.25→4.125 bpw, faster decode, lower KLD, crash fixed. NEED TESTERS: spent the last stretch hammer…
TurboQuant+ updates bring lower bitrate (4.125 bpw), faster decode, and reduced KLD (34-35% down). The developer seeks testers on various hardware backends.
Got MTP + TurboQuant running — Qwen3.6-27B -- 80+ t/s at 262K context on a single RTX 4090
Developer achieved 80+ t/s inference on Qwen3.6-27B with 262K context on a single RTX 4090 by combining MTP (Multi-Token Prediction) with TurboQuant's lossless KV cache compression, sharing their implementation fork and technical details.
Qwen3.6:35b UD Q4_K_M 80 tok/s on Nvidia P40
A user shares achieving 80 tok/s on a Qwen3.6 35B model with Q4_K_M quantization and 100k context on a single Nvidia P40 using TheTom's TurboQuant fork of llama.cpp, highlighting various optimizations.
@iotcoi: Qwen3.6-27B-FP8 + Dflash + DDTree, 256k context, 10 agents ~200 tokens/sec max decode 136t/s average on a single tiny G…
Quantized 27B Qwen3.6 model achieves 200 tok/s peak (136 avg) with 256k context and 10 agents on a single 49W GB10 GPU using Dflash+DDTree optimizations.
@sudoingX: i was running Ornith new 35b moe on llama.cpp with a Q4 quant, 4 bit, small, fast. it hit ~78 tok/s. then i swapped eng…
A 35B MoE agentic coding model called Ornith runs near lossless at FP8 on a single DGX Spark, achieving 3M token context and ~36 tok/s, with speculative decoding expected to boost speed further.