mlstm

This work proposes orthogonalizing the memory matrix of mLSTM recurrent models to improve their performance on noisy associative recall tasks. Experiments show that using Newton-Schulz iterations for read-only orthogonalization enhances validation accuracy compared to baseline mLSTM.