Tag
ICA Lens revives independent component analysis as an efficient method for interpreting language model representations, offering a faster alternative to sparse autoencoder training while maintaining competitive performance.
This paper investigates when rank-1 activation steering is effective and cost-efficient, proposing geometry-guided search and the concept of granularity to explain variability, and introduces the GRACE framework for efficient LLM control.