adaptive-distillation

RAFT is a two-stage framework for domain-specific fine-tuning of LLMs that addresses catastrophic forgetting by refining supervision data and using on-policy distillation with adaptive loss balancing, achieving significant improvements on domain accuracy while recovering general capabilities.