Tag
This paper proposes a training-free adaptive pruning method for large reasoning models during batched inference, using periodic top-k selection and activation memory to improve accuracy and computational efficiency.
PALS adjusts per-layer sparsity ratios for LLM pruning based on the 99th percentile of activation magnitudes, achieving significant perplexity improvements on LLaMA-2-7B compared to uniform sparsity, with negligible added cost.
Introduces SigmaScale, a method that learns auxiliary scaling matrices for SVD-based LLM compression, showing competitive performance on Llama 3.1 8B and Qwen3-8B benchmarks.
Introduces QAM-W, a joint 2D codebook quantization method for LLM weights using Hadamard rotation and activation-aware scaling, achieving near BF16 perplexity at 5–6 bits per weight and matching SmoothQuant W8A8 quality with 32% fewer weight bits.