@rasbt: Crazy model! It actually uses the old Qwen2.5-Coder-3B stack and got really great performance with their post-training …

X AI KOLs Following Models
model-release coding-benchmarks post-training reinforcement-learning qwen2.5-coder open-source

Summary

A 3B parameter model using the Qwen2.5-Coder-3B stack achieves coding benchmark scores comparable to Claude Opus 4.5, with detailed post-training techniques including synthetic data, filtering, two-stage SFT, and a novel RL method (MGPO).

Crazy model! It actually uses the old Qwen2.5-Coder-3B stack and got really great performance with their post-training stack. Need to use it in the next days to see if vibes of VibeCoder actually check out in practice. But impressive first impression! Based on the tech report, some of the important pieces of their post-training stack: 1. High-signal synthetic data (math problems with credible solutions, code with tests) 2. Multiple reasoning paths for each answer 3. Filtering, filtering, filtering 4. 2-stage SFT (start with broad training, then train on hard long-reasoning samples) 5. Use target (pass@k) accuracy over validation loss for checkpoint selection 6. MGPO (MaxEnt-Guided Policy Optimization) for RLVR: basically a GRPO-style RL method with an extra weighting that favors examples that are neither too easy nor too hard for the current policy 7. Single 64k long-context RL (they found that the usual progressive context expansion hurt this model because early truncation damaged long-thinking behavior) 8. Training data order: they do Math RL, then Code RL, then STEM RL in this particular oder which they found helped overall 9. After optimizing for accuracy, they add a stage that rewards shorter correct trajectories; basically making the model more efficient without accuracy degradation
Original Article
View Cached Full Text

Cached at: 06/17/26, 01:42 AM

Crazy model! It actually uses the old Qwen2.5-Coder-3B stack and got really great performance with their post-training stack. Need to use it in the next days to see if vibes of VibeCoder actually check out in practice. But impressive first impression!

Based on the tech report, some of the important pieces of their post-training stack:

  1. High-signal synthetic data (math problems with credible solutions, code with tests)

  2. Multiple reasoning paths for each answer

  3. Filtering, filtering, filtering

  4. 2-stage SFT (start with broad training, then train on hard long-reasoning samples)

  5. Use target (pass@k) accuracy over validation loss for checkpoint selection

  6. MGPO (MaxEnt-Guided Policy Optimization) for RLVR: basically a GRPO-style RL method with an extra weighting that favors examples that are neither too easy nor too hard for the current policy

  7. Single 64k long-context RL (they found that the usual progressive context expansion hurt this model because early truncation damaged long-thinking behavior)

  8. Training data order: they do Math RL, then Code RL, then STEM RL in this particular oder which they found helped overall

  9. After optimizing for accuracy, they add a stage that rewards shorter correct trajectories; basically making the model more efficient without accuracy degradation

orcus108 (@orcus108): WHAT THE HELL is happening in AI?

A 3B parameter model just put up coding benchmark scores in the same league as Claude Opus 4.5.

3 BILLION.

The weights are on Hugging Face, anyone can test it.

I genuinely don’t know if this is a breakthrough or if the benchmarks are broken.

Similar Articles

@f14bertolotti: Stellar performance from a 3B model. These results were achieved primarily through post-training refinements on Qwen2.5…

X AI KOLs Timeline

This technical report introduces VibeThinker-3B, a 3B parameter model that achieves frontier-level verifiable reasoning performance through post-training refinements on Qwen2.5-Coder, including curriculum-based supervised fine-tuning, multi-domain reinforcement learning, and offline self-distillation, matching or exceeding much larger models like DeepSeek V3.2.

Wow! Qwen 3.6:35b-a3b on a 3090... pretty amazing.

Reddit r/artificial

A user shares impressive results running a quantized Qwen 3.6:35b-a3b model on a used RTX 3090, achieving 160 tokens per second output after fitting the model into VRAM, and demonstrates vision capabilities with a 75-second video processing time.