Tag
This paper introduces a framework to reduce per-sample harm in stochastic optimization, where parameter updates from batch averaging and historical states increase individual sample loss. The method uses dimensionality reduction and focuses on the last linear layer for efficiency, showing improved generalization on image classification tasks.
This paper presents a unified theoretical framework for gradient aggregation in multi-objective optimization, establishing convergence rates to Pareto stationarity. The authors introduce a sufficient alignment condition and demonstrate its application to existing and new algorithms, such as capped MGDA.