attention-modification

A developer is rebuilding Gemma 4 31b into a smaller 26b model by removing weak SWA layers, adding attention-based residual networks (from Moonshot), and using topK logit targets for retraining, aiming for better long context and performance.