Tried TypeSafe AI’s Jev vs a regular LLM for model routing and the latency difference is pretty noticeable

I’ve been playing around with TypeSafe AI’s System One model (Jev) to see how it compares with a regular LLM for pure decision-making / model routing. I built a small smart model router where I run two completely independent decision pipelines in parallel for every message: one decision through Jev one through a normal LLM using structured output Both get the same signals and have to make the exact same routing decision. The main thing I wanted to test was latency. TypeSafe says Jev uses parallel sampling instead of autoregressive token-by-token generation, so I wanted to see whether that actually translates into a noticeable latency difference in a real routing loop. So far, it definitely does. Jev evaluates all the routing signals in a single call and usually gives me a decision in roughly ~1 second, whereas the regular LLM with structured output is taking anywhere from ~4-14 seconds for the same decision. Still pretty early experimentation, but Jev looks interesting for cases where you basically want a fast decision layer in front of larger models.