Access Sets Matter: Budgeting Expert Reads for Scalable Weight-Space Model Merging
Summary
This paper introduces the concept of Access Sets to budget expert reads, enabling scalable weight-space model merging.
View Cached Full Text
Cached at: 06/04/26, 03:41 AM
Paper page - Access Sets Matter: Budgeting Expert Reads for Scalable Weight-Space Model Merging
Source: https://huggingface.co/papers/2605.29489 Get this paper in your agent:
hf papers read 2605\.29489
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.29489 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.29489 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.29489 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
CABS+: Efficient and Scalable Model Merging via Conflict-Aware Sparsification and Adaptive Weight Allocation
This paper presents CABS+, an enhanced model merging method that uses conflict-aware sparsification and adaptive weight allocation to improve multi-task model merging efficiency and performance, achieving significant gains over prior methods across 27 datasets.
Model Merging by Output-Space Projection
This paper presents a new framework for model merging that casts the problem as a convex quadratic program over residual updates, minimizing a squared-output calibration objective. It subsumes existing heuristic methods and provides a closed-form diagnostic to predict merge quality, showing consistent gains on language and vision benchmarks.
Model Merging Scaling Laws in Large Language Models
This paper establishes empirical scaling laws for language model merging, identifying power-law relationships between model size, expert count, and performance to enable predictive planning for optimal model composition.
Compute-Optimal Is Not Cluster-Optimal: Systems-Aware Scaling for Sparse Mixture-of-Experts
This paper introduces MOSAIC, a framework that jointly optimizes sparse Mixture-of-Experts model architecture and hardware systems for large-scale pretraining, showing that compute-optimal sparsity is not necessarily cluster-optimal when MFU, communication costs, and parallel layouts are considered.
PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference
PuzzleMoE introduces a pairwise dual-mask expert merging algorithm and bit-level packing technique to compress large Mixture-of-Experts models, maintaining performance while reducing storage and accelerating inference.