pruning

This paper introduces a PyTorch-based framework for pruning binarized neural networks and proposes a novel global weighting pruning method that achieves superior accuracy and pruning rates, enabling efficient deployment on edge hardware like FPGAs.

The paper introduces Calibration-Preserving Pruning (CPP), a method to enhance pruning for maintaining conformal prediction efficiency, showing reduced prediction set sizes and improved accuracy on benchmarks like DBpedia-14 with models such as Qwen2.5-1.5B.

This paper systematically evaluates LLM compression methods and reveals asymmetric harms, such as disproportionate loss of head knowledge and increased confidence in incorrect answers, which are masked by standard aggregate metrics.

This paper proposes Activation-Prune-Merge (APM), a training-free framework for cross-scale fusion that improves smaller language models using larger donors without semantic alignment, achieving performance gains on multiple benchmarks.

This paper proposes a training-free adaptive pruning method for large reasoning models during batched inference, using periodic top-k selection and activation memory to improve accuracy and computational efficiency.

This paper evaluates the trustworthiness of small language models across fairness, robustness, privacy, and ethics, comparing pre-trained SLMs with compressed larger models, and finds that quantization preserves trustworthiness better than pruning and that distillation can further enhance reliability.

This paper introduces difference-informed pruning methods (Wisp, Wisp+, Whisper) for large language models, showing that preserving output differences improves sparsification across Llama 2 and 3.1 models up to 405B parameters.

This paper presents a systematic study of pruning strategies for reducing token usage and latency in long-horizon deep research agents, showing that early pruning yields the largest efficiency gains with little quality loss.

This paper empirically investigates how pruning, adversarial training, and hardware-induced weight faults jointly affect the reliability of convolutional neural networks, finding that adversarial training increases sensitivity to stuck-at-zero faults while pruning has little effect on fault sensitivity.

TaskPress introduces a query-agnostic KV cache compression framework that uses a task guide as a meta-query and quantization scale factors to prune irrelevant tokens, enabling reusable caches across diverse queries with negligible overhead.

SparseKAN is a unified compression method for Kolmogorov–Arnold Networks that prunes basis functions, neurons, and numerical precision under learnable gates, achieving up to 73% parameter reduction and significant latency improvements on software and FPGA hardware.

Discusses Apple's AFM3 20B model architecture, which uses Instruction-Following Pruning to activate only ~20% of MLP layers per prompt and keep the model in flash storage for efficient on-device inference.

This paper introduces two library combinators, prune and defrel/bank, that bring bottom-up enumeration with observational deduplication to miniKanren relational programming, improving program-by-example synthesis performance.

This Hugging Face repository provides community-compiled quantized and pruned weights for MiniMax H3 (Hailuo 3.0), enabling local text/image/audio-to-video generation on consumer GPUs with 16-24GB VRAM. It includes INT4, INT8, and NVFP4 variants with hardware-specific guides.

This paper tests entropy-based pruning for chain-of-thought compression across models and tasks, finding it offers no advantage over random pruning, and that low-entropy token retention only helps on math benchmarks due to numeric tokens. It provides causal evidence that reasoning information is distributed across the full chain rather than concentrated in a few identifiable tokens.

This paper presents Fairness Pruning, a structural intervention method that locates demographic bias in GLU-MLP layers of large language models by identifying differentially activated neurons. Zeroing a very small number of neurons disrupts bias processing while preserving reasoning and general knowledge, suggesting bias and capabilities rely on dissociable circuits.

This paper shows that gently compressed LLMs can pass standard data-free quality guards (perplexity, MMLU, output fidelity) yet still invent procedure steps when used as agents, and proposes a data-free two-axis screen to detect such failures before deployment.

This paper introduces a loss-aware feature-map pruning framework for convolutional neural networks using multi-armed bandits (UCB1 and Thompson Sampling) to selectively remove redundant feature maps, reducing computational cost while preserving accuracy.

Reveals that MoE routing in LLMs follows a Frequency-Diversity Law analogous to Huffman coding, and proposes Subset Difference Pruning to eliminate functional redundancy and achieve optimal compression.

This paper introduces double-scoring, an augmented score-space parameterization for extracting strong lottery tickets from neural networks. It improves upon edge-popup and pruning-at-initialization baselines and reduces sensitivity to sparsity hyperparameters.