Ninfer Stahp

Reddit r/LocalLLaMA Tools
gpu-inference inference-backend blackwell nvfp4 windows high-performance llm-serving

Summary

A user benchmarks the new Ninfer inference backend on a Blackwell MaxQ GPU, reporting 130-180 t/s on qwen3.8-27b-nvfp4 with minimal slowdown under parallel decoding — claiming vLLM-level concurrency and far faster throughput than llama-server, even after installing it on Windows 10.

GPU: Blackwell MaxQ Model: qwen3.8-27b-nvfp4 (Orcarouter) - Dflash2 - MTP-7 Running this on my Blackwell MaxQ. I can't believe how fucking fast it is compared to llama-server. It easily reaches speeds between 130-180 t/s, even in parallel decoding I hardly see any slowdowns at all! Its concurrent capabilities are on par with vLLM in my experience. Spent this afternoon installing it on Windows 10, trying to migrate my daily driver from llama-server to something else. When I found out that I could actually run this backend on Windows 10 I gave it a try. Several hours later, this is the result. The hype is real for once. I love the kind of work put in here. This backend has such a niche level of specialization it becomes extremely powerful when used for agentic vibecoding. Props to the author for building this from scratch. I love llama-server for its versatility and level of control, but this backend is like a crotch rocket in comparison. Only thing I can say is that since its still very new, it is going to be initially missing a lot of features like advanced sampling params, broader model support, etc. but it already has all I need to do my work. Really mindblown by this thing, seriously.
Original Article

Similar Articles

Qwen3.8FN vLLM vs. llama.cpp vs Ninfer6000

Reddit r/LocalLLaMA

A community member benchmarks Ninfer6000 (a Ninfer fork tuned for RTX Pro 6000 Blackwell and Qwen3.8-Flash-Next) against vLLM and llama.cpp, finding ~1.6x decode speedup over vLLM and 2.2x over llama.cpp with near-identical accuracy, but reports crippling real-world tool-calling malformations that keep them on vLLM.

I built Ninfer 4080 for 16GB class GPUs

Reddit r/LocalLLaMA

A community developer released Ninfer 4080, an optimized local inference runtime that runs the Qwen 3.8 27B GSQ model at 100k context on a 16GB RTX 4080, achieving up to ~2720 tok/s prefill and 262 tok/s decode via DFlash2 speculative decoding — significantly faster than general-purpose engines like llama.cpp or vLLM.