Tag
Introduces SymExpLin (SEL), a weight reparameterization that combines symmetric-exponential and linear pathways to improve optimization in neural networks, reducing training steps by up to 1.49x on transformers.