标签
新预印本提出了 SoftServe，一种针对非凸目标设计的拟牛顿优化方法，可扩展到超大规模神经网络，旨在解决拟牛顿方法在现代深度学习中的实用性问题。
This paper proposes FedCoMuon, a federated compositional Muon optimizer for matrix-wise models, along with a variance-reduced variant (FedCoMuon-VR). The authors provide convergence analysis under non-i.i.d. and non-convex settings, showing improved sample complexity over existing FedMuon algorithms, and demonstrate competitive performance on robust federated learning and task-distributed risk-sensitive meta learning.
本文介绍了一种面向在线优化的曲率自适应跟随扰动的领导者（FTPL）算法，该算法采用时变扰动尺度，在非凸Lipschitz损失和强凸损失下均能实现最优遗憾界。