Tag
A reproducibility study reveals asymmetric effects when removing residual connections in Pre-LN transformers: attention-skip removal leads to collapse, while FFN-skip removal allows partial recovery at smaller scales.