Has anyone compared pre-training, SFT/LoRA and reinforcement post-training on Qwen3.6-27B?
Summary
A community member asks for direct comparisons of pre-training, SFT/LoRA, and reinforcement post-training on Qwen3.6-27B, citing recent research on catastrophic forgetting and mitigation strategies.
Similar Articles
I created an LLM post-training method called RPS. Preliminary results show that it improved Qwen3-8b's program synthesis reliability. [R]
RPS is a two-stage LLM post-training method inspired by neuroscience, combining curriculum learning with learning rate decay. Preliminary results show improved program synthesis reliability on Qwen3-8b compared to equal learning rate training.
Mechanistic origins of catastrophic forgetting: why RL preserves circuits better than SFT?
This paper investigates the mechanistic origins of catastrophic forgetting in LLMs, finding that reinforcement learning preserves internal computational circuits better than supervised fine-tuning, resulting in less forgetting of prior capabilities.
Are recent LLM gains mostly from pretraining or post-training?
A discussion question exploring whether recent LLM gains are driven more by pretraining or post-training techniques like RL and fine-tuning, given that both require significant compute.
Real Continual Learning Model (Prototype)
A developer claims to have built a real continual learning model prototype using LoRA to give Qwen4B instant, generalizable memory without retraining, and is inviting independent researchers to validate the mechanism.
I trained TIME: short context-triggered thinking on Qwen model instead of overthinking
A personal project led to an ACL 2026 paper introducing TIME, a method training Qwen3 models to engage in short, context-triggered thinking rather than excessive reasoning. The work uses QLoRA and a four-phase curriculum, with all data and code released open-source.