Tag
The paper identifies 'Thinking Collapse' in on-policy self-distillation for large language models, characterized by a decline in intermediate reasoning steps, and proposes AD-OPSD, a control framework that mitigates this collapse by anchoring high-suppression-risk tokens to a reference prior. The method achieves up to +4.1% absolute average accuracy improvement on mathematical benchmarks.
This article examines adversarial attacks on machine learning models and demonstrates why gradient masking—a defensive technique that attempts to deny attackers access to useful gradients—is fundamentally ineffective. The paper shows that attackers can circumvent gradient masking by training substitute models that mimic the defended model's behavior, making the defense strategy ultimately futile.