Try out this "high" reasoning mode for 27B (tested on VLLM)

After a lot of tweaking, I have come to the conclusion that 27B lacks a reasoning mode that is between low and xhigh. The "medium" mode isn't actually medium, it erases the explicit instructions to the model. When medium is enabled, the model acts very differently - to me it looks like it regresses to behaving more like 3.6, and loses some of the 3.8 gains. Low and high mode behavior in the model seem to be triggered almost exclusively by using certain keywords in the reasoning instructions, and act in a surprisingly binary manner. You can add all the additional instructions you want to the reasoning instructions, and tweak the prompt, but the model mostly ignores those changes. You can ask it for medium effort and it just won't do it. Because of this weird behavior, I experimented with blending the words in the low and xhigh prompts together, until the model would output reasoning that was a more reasonable length. It seemed to work pretty well, and the end result is a high reasoning mode, with a reasoning block that is typically around 1/5 the size of the xhigh reasoning. Output quality seems good, more like xhigh than low or medium, and the reasoning is definitely much shorter, which makes the model much more enjoyable to use. Play around with the chat template modification below, add "high" to your reasoning effort, and see if you can improve it further or if it makes the model worse -- add high to reasoning_effort -- {%- if resolved_reasoning_effort not in ('xhigh', 'high', 'medium', 'low') %} {{- raise_exception('Unexpected reasoning effort ' ~ reasoning_effort ~ '. Supported types are xhigh (default), high, medium, and low.') }} {%- endif %} -- add new blended prompt for high, which mixes the low & xhigh prompts -- {%- elif resolved_reasoning_effort == 'high' %} {%- set reasoning_instructions = 'Reasoning effort is set to halfway between low and xhigh. Please think careful but brief, validate key assumptions but keep it brief, and move quickly to the conclusion without unnecessary elaboration.' %}