merging

CT-Merging proposes a method to merge LoRA adapters by estimating consensus directions from task subspace projectors and assigning task-level RMS coefficient scales, achieving superior performance on the DC-Merge CLIP adapter benchmark.

SiGMA proposes sign-guided adaptive tuning during training and sign-guided merging at inference to mitigate negative interference in multimodal continual instruction tuning, achieving state-of-the-art results on UCIT and DCL benchmarks.