mlp-trimming

Tag

Cards List
#mlp-trimming

Unlocking Structured Pruning for Reasoning Language Models via Attention-Preserving MLP Trimming

arXiv cs.LG ↗ · 5d ago Cached

提出LayerTrim——一种无需训练的结构化剪枝方法,通过保留注意力机制和解码器深度、选择性修剪冗余层的MLP特征,显著缓解了推理语言模型在长链推理任务上的性能崩溃,在Qwen3-14B上实现1.24倍预填充加速和19.7%峰值显存降低。

0 favorites 0 likes
← Back to home

Submit Feedback