gradient-based-methods

This survey examines LLM unlearning methods for cyber defense, introducing a three-level framework to distinguish behavioral suppression, representation-level attenuation, and true forgetting, and analyzing gradient-based, influence-based, and localized editing approaches.

This paper demonstrates that two-layer neural networks trained with gradient-based methods can achieve the optimal computational-statistical tradeoff for learning Gaussian single-index models, matching the SQ lower bound up to polylogarithmic factors for all generative exponents and extending to sparse settings with a novel weight perturbation technique.