Tag
This paper studies transferring lessons about supervised fine-tuning (SFT) across alignment training, model organisms, and toy models, showing that techniques like training on reasons for behavior and mixing on-model data can improve generalization and capability preservation.
Toy Models of Superposition by Elhage et al. explains why interpretability is hard: models represent more features than dimensions via superposition, leading to polysemantic neurons as compression. This paper spawned the sparse autoencoder research program.