Tag
FedSEAM 提出一种联邦持续学习框架，通过子空间专家共识合并（SEAM）在客户端之间选择性共享稀疏专家知识，缓解路由漂移与合并擦除导致的灾难性遗忘，并在 TRACE 基准上使用 Qwen3-4B 和 Llama-3.2-3B 进行了评估。
This paper proposes Error-Subspace Calibration (ESCal), a post-merging calibration framework that identifies low-dimensional error subspaces from weight differences between expert and merged models, using lightweight Bridge modules to recover task-specific behavior. Experiments on vision and NLU benchmarks show ESCal improves merged-model performance across diverse architectures and merging methods.
The paper introduces 'All-in-One', a composable attack framework from ETH Zurich that trains LLMs to pass safety checks yet turn malicious after deployment-time transformations like quantization, pruning, or model merging, by approximating each transformation as weight perturbations jointly optimized in a single training objective.
OliviaRossi 发布了 Minimax-h3_Singularity-x-10Eros-Max_PRISM，使用自定义 PRISM 算法（非 mergekit）合并两个 Minimax-H3 Ref2VA 微调模型，包含谱去噪、极坐标测地线插值和符号冲突调解等步骤。
Orange Research presents 0.6B–14B open-weight Qwen models adapted to Wolof, a language spoken by 23 million people, using instruction tuning, preference alignment, and model merging. The models are competitive with much larger commercial systems on Wolof tasks and are being piloted in customer-care deployments, with data and weights publicly released.
论文提出 COMPASS（COMPatibility ASsessment Score），通过测量局部邻域重叠与范式上限，在对齐或合并之前低成本预测模型对的兼容性，在 72 个跨模态对齐实验和 392 个任务算术合并中与下游成功度高度相关（Pearson 相关分别达 0.91 和 0.70）。
DynamicTune transfers reasoning ability from a larger teacher into an 0.8B model via closed-form linear algebra in ~12 minutes with zero backpropagation, achieving 42.15% acc_norm on ARC-Challenge and beating a stock 2B model (41.10%). The FP16 checkpoint, GGUF file, and GitHub repo are open source and independently benchmarked on an NVIDIA L4.
论文提出 Adaptive Mutual Distillation (AMD) 框架，通过两个采用不同任务平衡策略的模型进行协同蒸馏，并利用短训练探针和任务级验证分数自适应调整各任务与传输方向的蒸馏权重，在六个基准和三个 LLM 骨干上取得优于 SFT 与现有任务平衡方法的效果，合并后平均提升 2.91 分。
SMAT introduces a simple and efficient merge-aware training method that jointly optimizes expert loss and expected loss at simulated merged parameters, improving merged model performance with minimal computational overhead.
This paper investigates whether routing drift in merged Mixture-of-Experts Large Language Models indicates failure, and proposes a routing analysis toolkit and Selective Router Repair method for assessment.
The paper investigates fine-tuning strategies for customer support LLMs, comparing multi-task training, sequential updates, and model merging across multiple model families. It concludes that multi-task full fine-tuning is the most robust default, while specialist models degrade off-task and require reliable routing.
The paper presents ACLArena, a framework for evaluating Agent Continual Learning in multi-stage post-training, analyzing forgetting and generalization mechanisms, and proposing an improved ACL recipe using offline replay and LoRA experts.
This survey paper defines model fusion and organizes research into three levels—parameter, representation, and behavior—for integrating capabilities in large language models.
CoMerge is a conflict-driven preference optimization framework for merging multi-task LLMs, using self-supervised strategies to mitigate parameter interference and achieve high performance on benchmarks like MergeBench.
The paper introduces Net Utility, a data-free metric for budget-aware LoRA merging that optimizes rank allocation across tasks, achieving +2.1% improvement on vision tasks and +2.2% on language tasks over uniform methods.
The paper investigates the geometric challenges in merging differentially private task models and introduces DP-Merging, a framework to enhance mergeability while maintaining privacy guarantees.
This paper reveals a jailbreak risk in model merging even when constituent models are safety-aligned, and proposes Basin-Aware Jailbreak (BAJ) to generate transferable adversarial suffixes across merged model families.
GRIP introduces a reward-guided parameter interpolation framework to merge reasoning and instruction models, improving accuracy-efficiency trade-off for LLM reasoning without retraining.
CORAM introduces a coherent orthogonal rotation method for model merging that partitions weight matrices into row slices, uses SVD in the base model's frame, and merges updates on manifolds to improve accuracy over existing techniques.
This paper proposes Activation-Prune-Merge (APM), a training-free framework for cross-scale fusion that improves smaller language models using larger donors without semantic alignment, achieving performance gains on multiple benchmarks.