Tag
The paper introduces Net Utility, a data-free metric for budget-aware LoRA merging that optimizes rank allocation across tasks, achieving +2.1% improvement on vision tasks and +2.2% on language tasks over uniform methods.
CT-Merging proposes a method to merge LoRA adapters by estimating consensus directions from task subspace projectors and assigning task-level RMS coefficient scales, achieving superior performance on the DC-Merge CLIP adapter benchmark.
This paper introduces Preference Delta Aggregation (PDA) and Geometric Alignment Merging (GAM) to aggregate multiple 'weak' preference signals from weaker model pairs via LoRA merging, improving strong LLMs on knowledge reasoning and agentic search tasks by over 6% on average.