Tag
The paper introduces SAE Additivity Residuals (SAR) to measure how much of model behavior cannot be explained by individual or low-order SAE features, showing SAE coordinates are more behaviorally additive than random rotations while important feature interactions remain, improving coalition and steering rank fidelity when pairwise interactions are modeled.
This paper introduces Product-Unit Residual Networks (PURe), which integrate multiplicative product units with residual connections to explicitly model nonlinear feature interactions, showing improved accuracy, robustness, and interpretability over standard MLPs.