mlp-layers

This paper studies where safety alignment is encoded in large language models by transplanting weights from aligned to unaligned models. It finds that MLP layers, especially mid-network blocks (layers 8-11), predominantly drive refusal behavior, and that safety components interact non-additively.