Tag
LILA is a calibration-free structured pruning method for large language models that uses latent spectral geometry to score neuron importance, achieving competitive performance without calibration data.
This paper proposes a damage-aware multi-armed bandit method for structured post-training pruning of vision and language transformers, showing reduced performance degradation compared to baseline approaches in experiments across various models and datasets.
The paper proposes COEC, a training-free compensation framework for structured pruning of large language models that applies orthogonal rotations and calibration to reduce output error and improve accuracy after column removal.
This paper introduces Sniper, a two-stage structured pruning framework for LLMs that uses binary knapsack optimization to unify depth and width pruning, achieving near-exact compression ratio adherence and improved performance retention across multiple architectures.
TriSP introduces a tri-signal importance metric combining weight magnitude, activation norm, and gradient sensitivity for structured pruning of LLMs, achieving lowest perplexity and high throughput improvements on LLaMA-7B.
Proposes a two-stage structured pruning framework for LLMs that jointly optimizes latency and model size using multi-objective depth pruning and parallel Bayesian optimization, achieving favorable trade-offs for edge deployment.
ShortOPD proposes a short-to-long on-policy distillation schedule that recovers pruned LLMs for free-form generation by focusing training on effective prefixes, achieving up to 9x improvement over unrecovered models and matching long-horizon distillation with a quarter of the training time.
This paper introduces Maestro, a structured pruning framework for Mixture-of-Experts language models that uses Markov chains to model expert activation trajectories, achieving globally aware pruning and outperforming baselines by up to 10.61% under 50% compression.
This paper proposes a structured pruning method for LLMs that addresses distribution mismatch, sign-information loss, and outlier influence when adapting unstructured pruning techniques, achieving comparable accuracy with 1.56-1.57x speedup on models like Llama-3-8B and Vicuna-v1.5-13B.
This paper presents a cascaded multi-granularity pruning framework for deploying LLMs on Industrial IoT edge devices, achieving up to 13.8x compression with minimal accuracy loss on MHA+GELU architectures while exposing a collapse on GQA+SwiGLU designs.
This paper proposes a novel structured neuron pruning framework for deep neural networks using multi-armed bandit algorithms, demonstrating effectiveness on various tasks.
Proposes KOFF, a framework that decomposes pretrained LLMs into a sparse shared backbone and domain-specific external memories using structured pruning and LoRA adapters, achieving 12% sparsity without significant performance loss.