As of version 0.2.7.1 Strata has two routing optimizations that can affect both performance and quality. I was curious what the actual impact was so I ran some tests on my hardware with Qwen 3.8 Flash Next IQ3_S. STRATA_ROUTE_PRIOR This is the new one - this optimization configures Strata to prefer experts already in VRAM by adding a bonus of 1.5 (about a 4.48x effective multiplier) to the scores of experts already cached in VRAM. This means that a slightly less preferred cached expert can replace an expert the model would normally select if its base score was close enough. STRATA_ROUTE_TAIL_SKIP This one is a lot simpler - with a value of STRATA_ROUTE_TAIL_SKIP=7 (the default value), Strata will skip computing experts 7 through 10 if they aren't already in VRAM. It does not change selected experts, it just drops contributions from less important selected experts. Benchmarks I had DS4-Flash run Qwen through IFEvalCode, AIME 2026, and IFEval; then worked with ChatGPT to analyze the results. Config Decode IFEvalCode (code) AIME 2026 (math) IFEval (instruction Following) Both off 72.3 tok/s 24/120 (20%) 23/30 (76.7%) 122/150 (81.3%) Tail skip 83.9 tok/s 24/120 (20%) 19/30 (63.3%) 123/150 (82%) Route prior 104.6 tok/s 20/120 (16.7%) 20/30 (66.7%) 118/150 (78.7%) Tail skip + Route prior 104.2 tok/s 23/120 (19.2%) 20/30 (66.7%) 117/150 (78%) Round 2 - re-ran a smaller subset of questions (mostly the ones whose answers differed in round 1) with much larger thinking budgets. Absolute percentages shouldn't be directly compared between rounds. Config Decode IFEvalCode AIME 2026 IFEval Both off 72.3 tok/s 25/57 (43.9%) 23/30 (76.7%) 59/78 (75.6%) Route prior 104.6 tok/s 27/57 (47.4%) 24/30 (80%) 61/78 (78.2%) Both enabled 104.2 tok/s 23/57 (40.4%) 24/30 (80%) 56/78 (71.8%) My interpretation Round 1 showed that STRATA_ROUTE_PRIOR might reduce quality slightly, but round 2 showed that many of those differences don't matter with longer reasoning. Route prior is such a huge boost to speeds that I think it's worth enabling for all but the hardest tasks. On the other hand, STRATA_ROUTE_TAIL_SKIP offers a slight boost to speeds (without route prior enabled) but results in significantly more degradation than just route prior alone. I plan to continue testing more extensively, but for now these give me a decent read on the tradeoffs between the two settings. Shoutout to Niko1221 for making an awesome inference engine.
A user shares benchmarks showing that raising the --adapt-every parameter from 1 to 6 in Strata boosts token generation speed by 9-55% across multiple Qwen 3.8 Flash Next prompts, with the largest gains on long-context code descriptions.
Strata 0.1.39 has been released with multi-GPU optimization. According to user feedback, inference speeds in some scenarios have doubled — benchmarks on a 4090 48G running the hundred-billion-parameter Qwen3.8-Flash-Next showed a significant boost in tokens/s.
A user reports strong single-request decode speeds running Qwen3.8-Flash-Next in Unsloth's UD-Q4_K_XL quant on Strata with an RTX PRO 6000, showing roughly 2x faster throughput than vLLM on the same GPU across prose, counting, coding, and reasoning tasks.