adaptive-optimization

This paper examines the coupling between the preconditioning exponent and learning rate in adaptive optimizers, showing that the exponent maximizing cross-environment accuracy decreases with log learning rate, and identifies a conflict between source-domain validation and robustness to environmental change.