Tag
DynamicTune transfers reasoning ability from a larger teacher into an 0.8B model via closed-form linear algebra in ~12 minutes with zero backpropagation, achieving 42.15% acc_norm on ARC-Challenge and beating a stock 2B model (41.10%). The FP16 checkpoint, GGUF file, and GitHub repo are open source and independently benchmarked on an NVIDIA L4.
Researchers propose a backprop-free, closed-form trajectory-matching method to transfer 4B model capabilities into a 0.8B student, finding that restricting weight surgery to 4 anchor layers avoids NLL explosions, yielding 10.8% lower held-out NLL across 30 benchmarks while running entirely on an 8GB RX 580.
Author Alexey Fateev dissected the Qwen3.8-27B AI model to carve out a MoE structure through zero-training weight surgery, finding only two neurons active on over 90% of tokens.
This paper investigates whether repetition loops in long factual enumeration tasks by Gemma 4 models can be fixed by editing a single neuron. It finds that targeted weight edits on a small set of MLP neurons can significantly reduce loop failures, though not completely eliminate doom looping in larger models.