ablations

A reproducibility study reveals asymmetric effects when removing residual connections in Pre-LN transformers: attention-skip removal leads to collapse, while FFN-skip removal allows partial recovery at smaller scales.

Introduces a three-step recipe for identifying attention-head circuits in pretrained transformers using a spectral signal and task-pattern screen without requiring labels, validated across 51M to 1B parameter models and multiple architectures.