Tag
This paper introduces a trust-region framework for understanding adaptive moment estimation methods like Adam, deriving a family of learning-rate mechanisms based on p-th moment constraints (2≤p≤4), including a kurtosis-based variant (Gmake). Experiments on GPT2-124M show that second-moment versions become competitive when trust-region controls are stronger.