sparsification

EDiS 提出一个边缘不相交子图稀疏化框架，将一次性结构提取与逐轮图组合分离，从而在降低 GNN 稀疏训练开销的同时保持拓扑多样性，并在 19 个基准上取得最佳平均成绩。

The paper proposes TSS, a target-side sparsification framework for speculative decoding in domain-specific LLMs, which improves inference efficiency and performance by skipping selected target layers.

This paper introduces a layerwise decoupling approach for structured sparsification of fully connected layers in neural networks, demonstrating improved robustness and reduced over-pruning compared to joint methods.

This paper presents CABS+, an enhanced model merging method that uses conflict-aware sparsification and adaptive weight allocation to improve multi-task model merging efficiency and performance, achieving significant gains over prior methods across 27 datasets.

Presents DG-FedReuse, a federated learning mechanism that reuses age-decayed cached client updates under a proxy-gradient threshold to reduce uplink communication, achieving significant modeled savings with minimal accuracy loss on image classification benchmarks.

This paper challenges the common assumption that domain experts for model merging should be trained to their optimal validation loss, showing that the optimal training duration depends strongly on the merging method. Simple averaging degrades with overfitting while sparsification-based methods benefit from training past the optimum, suggesting that training duration and merging method should be chosen jointly.

The paper proposes a post-training acceleration framework for video diffusion models that integrates dynamic structural sparsification with few-step distillation, achieving significant speedup while maintaining quality.

SCAPE is a communication-efficient distributed optimizer that leverages first-moment statistics to enable extreme sparsification for LLM training, preserving accuracy while reducing wall-clock time by up to 43.3%.

Llama Surgery injects learned block-sparse attention topologies into pre-trained Llama 3.1 8B without retraining from scratch, using a Dynamic Topology Router with Gumbel-Softmax routing, temperature annealing, and a Straight-Through Estimator to avoid gradient collapse, achieving stable convergence and coherent output.