capability-preserving

The paper introduces Fora (Function-space Orthogonal Residual Adaptation), a method to protect existing capabilities during fine-tuning by projecting updates onto function-space directions derived from activations rather than weight-space directions. Experiments on Qwen3-1.7B show it outperforms weight-space projection and standard regularization in preserving translation and math abilities.