model-pruning

Tag

Cards List
#model-pruning

The Fragility of Reasoning-Importance Attribution for Pruning Language Models

arXiv cs.AI ↗ · 4d ago Cached

This paper shows that reasoning-importance attribution used to guide pruning of chain-of-thought language models is fragile: answer-token scoring saturates, single-neuron causal importance is noise that cancels, and reasoning-specific importance is not separable—yielding practical rules to score reasoning tokens, attribute at coarse (~100-neuron) granularity, and evaluate with accuracy rather than perplexity.

0 favorites 0 likes
#model-pruning

From Large Language Models to Efficient System One Models via Adaptive Computation

Hugging Face Daily Papers ↗ · 2026-11-12 Cached

The paper finds decision information in LLMs is concentrated in intermediate layers (final six layers removable), and proposes a method that jointly trains a full model with shared prefix sub-models plus a lightweight gate for adaptive computation, matching or beating independently trained models across seven decision benchmarks.

0 favorites 0 likes
#model-pruning

Experts Exhibit Structured Redundancy: Structure-Preserving Expert Pruning for Mixture-of-Experts Large Language Models

arXiv cs.AI ↗ · 2026-11-02 Cached

本文提出结构保持的专家剪枝框架,通过 PEEP 与 DCEP 两种方法利用专家输出的结构冗余性对稀疏 MoE 大模型进行剪枝,在 DeepSeek-V2-Lite 和 Qwen3-30B-A3B 上超越现有基线。

0 favorites 0 likes
#model-pruning

pruned NVIDIA's new 30B speech LLM from 67.5GB to 10GB with zero retraining, it still transcribes at 5.16% WER (teacher: 4.12%)

Reddit r/LocalLLaMA ↗ · 2026-10-29 Cached

A zero-retraining REAP-pruned derivative of NVIDIA's Nemotron-3.5-Transcribe-30B-A4B speech LLM shrinks it from 67.5GB to 10GB (4-bit) while keeping WER at 5.16% vs the teacher's 4.12%, and it runs 27% faster in single-stream throughput on an H200 with vLLM.

0 favorites 0 likes
#model-pruning

All-in-One: A Composable Attack on LLM Transformations

arXiv cs.LG ↗ · 2026-10-28 Cached

The paper introduces 'All-in-One', a composable attack framework from ETH Zurich that trains LLMs to pass safety checks yet turn malicious after deployment-time transformations like quantization, pruning, or model merging, by approximating each transformation as weight perturbations jointly optimized in a single training objective.

0 favorites 0 likes
#model-pruning

Reconstruction-Optimized Expert Pruning for Mixture-of-Experts Language Models

arXiv cs.LG ↗ · 2026-10-21

论文提出 OptEP,一种面向混合专家(MoE)语言模型的后训练专家剪枝方法,通过连续松弛优化专家选择变量并结合重构驱动的权重更新,在 Qwen3、GLM-4.5-Air、Mixtral 等模型上显著优于现有剪枝基线,激进剪枝下精度提升最高达 16.20%。

0 favorites 0 likes
#model-pruning

Your Transformer Is Secretly Elastic

Hugging Face Daily Papers ↗ · 2026-10-14 Cached

提出一种免重训练的方法,将预训练 Transformer 转化为可弹性伸缩的模型,通过联合排序并软剪枝单个 MLP 神经元和注意力头,在多种稀疏度目标下连续自适应计算预算,在 27 个视觉与文本 Transformer 编码器上实现最高 60% 稀疏度下的平滑性能衰减。

0 favorites 0 likes
#model-pruning

TAP: Efficient Long-Horizon Agent Pruning via Trajectory-Anchored Recovery

arXiv cs.LG ↗ · 2026-10-08 Cached

The paper introduces Trajectory-Anchored Pruning (TAP), the first structural pruning framework for RL-trained long-horizon agents, which couples channel pruning with trajectory-anchored on-policy recovery. With 60% of FFN channels removed, TAP retains 99.2% and 88.0% of dense 7B agents' task success on ALFWorld and WebShop while cutting GPU time per successful task by roughly 22% and 17%.

0 favorites 0 likes
#model-pruning

MoRA: MoE Pruning via Router Bias Learning and Expert Approximation

arXiv cs.LG ↗ · 2026-10-03 Cached

MoRA proposes a structured MoE expert pruning framework using learnable router biases optimized with LM loss and a routing-diversity regularizer, plus a post-pruning expert approximation mechanism, outperforming state-of-the-art pruning methods on Qwen3-30B-A3B, DeepSeek-V2-Lite, and Moonlight-16B-A3B with 25-50% expert removal.

0 favorites 0 likes
#model-pruning

Two open-weights releases: Victoria (Qwen3.8-Flash-Next with 44% of experts cut, 70% Terminal-Bench 2.1, GGUF included) and Maple (a Canada-first fine-tune)

Reddit r/LocalLLaMA ↗ · 2026-09-30

Two open-weight fine-tunes of Qwen3.8-Flash-Next are released: Victoria, a pruned coding/agent model (44% of experts removed via REAP, retrained in NVFP4, 70% on Terminal-Bench 2.1) and Maple, a Canada-focused fine-tune that dramatically improves citation of official Canadian sources.

0 favorites 0 likes
#model-pruning

OBC-Prune: Outcome-Based Calibration for Large Reasoning Model Pruning

arXiv cs.AI ↗ · 2026-09-17 Cached

The paper proposes OBC-Prune, a calibration method for pruning large reasoning models that identifies causally important reasoning circuits to improve accuracy and reduce inference overhead on benchmarks like MATH500 and LiveCodeBench.

0 favorites 0 likes
#model-pruning

[Paper] ToMoE: Converting Dense Large Language Models to Mixture-of-Experts through Dynamic Structural Pruning

Reddit r/LocalLLaMA ↗ · 2026-08-24 Cached

ToMoE proposes a method to convert dense large language models into Mixture-of-Experts models using dynamic structural pruning without weight updates, outperforming existing techniques.

0 favorites 0 likes
#model-pruning

Kimi K3 (Unsloth) IQ2-XXS from 711GB down to 478GB!!! Only Multi-language was removed to trim the size

Reddit r/LocalLLaMA ↗ · 2026-08-08

A trimmed English-only GGUF version of Kimi K3 (IQ2-XXS) reduces model size from 711GB to 478GB by removing multi-language components, with early tests suggesting it may match or outperform the standard 2-bit version on coding tasks.

0 favorites 0 likes
#model-pruning

It Takes a MAESTRO To Prune Bad Experts

arXiv cs.CL ↗ · 2026-07-10 Cached

This paper introduces Maestro, a structured pruning framework for Mixture-of-Experts language models that uses Markov chains to model expert activation trajectories, achieving globally aware pruning and outperforming baselines by up to 10.61% under 50% compression.

0 favorites 0 likes
#model-pruning

Entropy-Regularized Probabilistic Gates for Sparse Model Discovery in Scarce-Data Federated Learning

arXiv cs.LG ↗ · 2026-07-02 Cached

This paper proposes entropy-regularized probabilistic gates to maintain uncertainty in sparse federated optimization, improving sparsity recovery and test performance under data heterogeneity and scarce data.

0 favorites 0 likes
#model-pruning

@dealignai: MiniMax m3, made for 128gb Mac’s Thank you to @hornsby_andrew for preparing the pruning calibration dataset and doing e…

X AI KOLs Timeline ↗ · 2026-06-18 Cached

A pruned and quantized version of MiniMax-M3 (MiniMax-M3-Medium-JANG_2L) optimized to run on 128GB Macs using vMLX, featuring 32% expert pruning and JANG_2L mixed-precision quantization to fit within ~105 GB.

0 favorites 0 likes
#model-pruning

Rethinking Layer Relevance in Large Language Models Beyond Cosine Similarity

arXiv cs.LG ↗ · 2026-05-15 Cached

This paper demonstrates that cosine similarity is a poor proxy for assessing layer importance in LLMs, and proposes using the actual accuracy drop from layer removal as a more robust metric.

0 favorites 0 likes
#model-pruning

Pruning Unsafe Tickets: A Resource-Efficient Framework for Safer and More Robust LLMs

arXiv cs.CL ↗ · 2026-04-20 Cached

This paper introduces a resource-efficient pruning framework that identifies and removes parameters associated with unsafe behaviors in large language models while preserving utility. Using gradient-free attribution and the Lottery Ticket Hypothesis perspective, the method achieves significant reductions in unsafe generations and improved robustness against jailbreak attacks with minimal performance loss.

0 favorites 0 likes
← Back to home

Submit Feedback