orthogonal-fine-tuning

Introduces AuroOFT, a method that enhances quantized orthogonal fine-tuning (QOFT) with a zero-start gated low-rank nonlinear residual, improving mathematical reasoning accuracy over QOFT and QLoRA while using fewer trainable parameters on low-bit language models.