delta-learning

The paper demonstrates that residual scale alone is insufficient for learnability in delta-learning, and introduces a diagnostic metric and target design principles to improve generalization in scientific machine learning.

This paper introduces Preference Delta Aggregation (PDA) and Geometric Alignment Merging (GAM) to aggregate multiple 'weak' preference signals from weaker model pairs via LoRA merging, improving strong LLMs on knowledge reasoning and agentic search tasks by over 6% on average.