Tag
该论文提出了一种可扩展的在线熵正则最优传输（OT）近似方法，使 Flow Matching 能够使用显著更大的源-目标耦合小批量，同时在理论上证明了相对独立配对能严格降低随机梯度方差和积分误差，并通过单细胞基因插值和图像流训练实验验证了效果。
This paper introduces a trust-region framework for understanding adaptive moment estimation methods like Adam, deriving a family of learning-rate mechanisms based on p-th moment constraints (2≤p≤4), including a kurtosis-based variant (Gmake). Experiments on GPT2-124M show that second-moment versions become competitive when trust-region controls are stronger.
The paper proposes a parallel architecture that assembles static gradient methods to achieve adaptivity in stochastic gradient descent, simplifying convergence analysis while retaining parameter adaptivity.
This paper sharpens the convergence analysis of LoRA, improving deterministic oracle complexity from exponential to O(epsilon^{-4}), and proposes stochastic variants LoRA-NSGDM and LoRA-STORM with improved oracle complexities of O(epsilon^{-8}) and O(epsilon^{-6}) respectively.
This paper proposes new discrete-time approximations for stochastic gradient Langevin dynamics (SGLD) with and without momentum, enabling accurate predictions of stationary covariance, iterate average covariance, and integrated autocorrelation time. The method provides improved tuning guidance for large-sample uncertainty quantification, especially under model misspecification.
This paper proposes LMO-IGT, a new class of stochastic optimization methods that accelerates convergence using implicit gradient transport while maintaining a single-gradient-per-iteration structure. It introduces a unified theoretical framework and demonstrates improved performance over existing LMO-based optimizers like Muon.