Tag
The author successfully implemented a jev-like API on NInfer using Qwen3.8-27B, achieving 84.4% accuracy on JevBench with decent results, though it remains a proof-of-concept with some bugs.
NInfer adds day-0 support for the Qwen3.8-27B model, achieving around 200 tokens per second on a single RTX 5090 with speculative decoding, and includes engine improvements like concurrent requests and kernel optimizations.