Ultra low temps make your model stupid at coding (a warning for beginners)

Many of you may know this, so this is probably worth skipping (except for the data to prove it). But if you're on the newer side for local, or have heard it but don't know why, I have some data to show you how running low temps like t=0 or t=0.2 make your model significantly worse (at coding in particular). You will see a lot of benchmarks for speculative decoding claiming crazy speeds, often with dflash2, running greedy decoding. Temp = 0, or greedy decoding causes your model to pick the most likely token every single time (with the caveat that parallel predictions on servers can make things still minutely random). They do this because speculative decoding is extremely easy on temperature 0, so the numbers they can claim for their post, product, etc. look amazing. The problem is, much of the "creative" thinking of models comes from the randomness that token selection allows. Greedy decoding forces the model to always pick a narrow path, which directly lowers its "intelligence" on non-recall tasks and significantly increases the risk of (usually catastrophic) looping. The testing parameters Model: Qwen3.8, compressed to IQ4_XS-M. That format stores each number in about 4 bits instead of 16, at ~4.6bpw. Engine: llamAmpere v0.5.2, run with its published settings for 24 GB graphics cards. Reasoning effort was set to "medium," which controls how long the model thinks before it answers. Fixed sampler settings: top_p 0.95 (only words that together make up 95% of the probability are candidates), top_k 20 (at most 20 candidates), min_p 0 (no extra cutoff). Only temperature changed. Temperatures: 0, 0.25, 0.5, 0.75 and 1.0. At 1.0 the model's probabilities are used unchanged. Coding test: LiveCodeBench v6, 100 problems labeled hard from the newest contests in that release. A program passes only if it passes every hidden test, meaning tests the model never sees. Results: Coding scores rose significantly with temperature. Pooled accuracy was 70.0% at 0, 76.0% at 0.25, 76.0% at 0.5, 79.7% at 0.75 and 82.7% at 1.0. The gaps from 1.0 at 0 and 0.25 are statistically significant. The gaps at 0.5 (−4.1 points) and 0.75 (−3.0 points) are not when split out separately, but as a pooled total they are; 0.5 was marginally insignificant but would have been under a 1 tailed test. Differences are in percentage points, so 82.7% minus 76.0% is 6.7 points. https://preview.redd.it/xo831qb1dmyh1.png?width=674&format=png&auto=webp&s=863fce9050de703371ba520951253095871da7f8 The frequency of looping rose significantly at the lower end, with a 9% loop rate at 0 (not all of these were catastrophic, but they all caused an incorrect answer) and 6% at 0.25, vs 0.3% at temp=1. The effects were statistically significant when pooling 0.5 and 0.75 together, but the test was not powered enough to confirm the individual rungs themselves. To put these #s into context, these t=0 scores are about what my older 2.5 bpw quants for qwen27b would achieve (~40% smaller than the quant used here).. It's worse than qwen3.6 on the same configurations Lower temperatures also make answers more verbose on average, with a 20% increase in answer length for temp=1. vs temp=1. For knowledge/recall questions (GPQA Diamaond), lower temperatures did not harm scores, but they did make the answers longer (~8%, marginal statistical significance). I am doing some follow up tests at 0.8 and 0.9 to see if they are viable, with the caveat that I am checking wall clock time, not TPS, since we dont want a model that makes so many more tokens that it overwhelms the TPS advantage. dataset, for those who are interested: https://huggingface.co/datasets/jakeatx/swift-1.5-temperature-sweep-gpqa-lcb