pruning

This paper investigates how text template tokens in diffusion transformers (DiTs) act as implicit semantic registers that causally maintain object identity during denoising, and proposes a training-free pruning rule that removes 20% of attention FLOPs with minimal performance drop.

A researcher made a video about their first ML research attempt on pruning vision-language models, including negative results, and submitted a paper to arXiv.

A community of tinkerers successfully runs the 753B-parameter GLM-5.2 mixture-of-experts model at near lossless quality on a $15,000 budget using REAP pruning and 2-bit GGUF quantization, sacrificing throughput for affordability.

Discusses methods or research on reducing the size of model parameters, likely focusing on techniques like pruning or quantization to improve efficiency.

This paper proposes a hardware-software co-design framework for N:M sparse vision Transformer inference, achieving over 2.2× latency speedup on GPUs while maintaining accuracy through a novel CUDA kernel (MD-SpMM) and a deployment-aware sparsity search.

This paper proposes an efficient GPU inference method for LLMs with moderate unstructured sparsity, introducing a three-layer matrix storage format and a SpMM kernel that jointly utilizes sparse tensor cores and CUDA cores, achieving up to 1.64× kernel-level speedup over SpInfer and up to 1.41× end-to-end speedup over FlashLLM.

Hermes Agent adds session pruning and archiving with various filters, allowing users to clean up their session database without losing important data.

This paper presents CineMobile, a method for efficient on-device image-to-video generation that achieves a 40x speedup over the teacher model through distillation-guided pruning, diffusion distillation, and hybrid quantization, enabling cinematic camera motion effects on mobile devices.

Proposes Generic TB-Coverage, a coverage-aware expert pruning method for sparse Mixture-of-Experts language models that uses only generic text corpora for calibration and preserves cross-corpus expert coverage, improving accuracy and reducing perplexity degradation.

This paper investigates the effects of domain-specific expert pruning on both utility and factual reliability of Mixture-of-Experts (MoE) models in the biomedical domain. It finds that moderate pruning preserves in-domain utility without immediate reliability loss, but extreme pruning increases hallucination risks, and generalization degrades rapidly in cross-domain settings.

Self-GC introduces a self-governing context management system for long-horizon LLM agents that treats agent history as indexed objects with fold, mask, and prune actions, reducing token usage while preserving future dependencies.

FlexMoE proposes a one-for-all nested intra-expert pruning method for MoE language models, enabling multiple deployable subnetworks from a single training run with minimal performance loss.

This paper introduces CascadeFormer, a depth-tapered transformer architecture, and CascadeFlow pruning, both motivated by Gradient Fan-in Asymmetry (GFA), which explains why deeper layers contribute less. The methods improve efficiency by reducing latency and increasing throughput while maintaining competitive performance.

ReFreeKV introduces a threshold-free approach to KV cache compression that adaptively allocates budgets, eliminating the need for input-specific thresholds while maintaining full-cache performance across diverse datasets and model sizes.

The author found while developing Maka Agent that aggressively pruning the tool result of an AI Agent (keeping only key summaries) hardly affects reasoning quality, and proposed three explanations, while also introducing the open-source desktop AI workstation Maka Agent.

This paper studies how pruning attention layers in LLMs affects explanation faithfulness and confidence calibration, finding that accuracy often remains high but interpretability and reliability degrade, highlighting a misalignment between model confidence, interpretability, and accuracy.

A new paper investigates whether it's better to prune a larger LLM or train a small LLM from scratch, finding that pruning provides more than just a good initialization.

Proposes a structural pruning framework for MoE models that maximizes channel-score coverage via attribution-based approximation, achieving 50% or 25% pruning with 4-bit quantization and reducing memory footprint by 5.27x on Qwen3-30B-A3B.

This paper reveals a 'benchmark illusion' where pruned LLMs perform well on multiple-choice benchmarks but fail to answer the same questions in open generation, suggesting that compressed models should be tested on generative tasks rather than just recognition tasks.

This paper proposes an inter-layer perturbation-absorption perspective for layer-wise sparsity in LLMs, showing that layers exhibit heterogeneous responses to pruning perturbations and introducing an absorption-aware correction that improves existing pruning methods by reducing perplexity and boosting accuracy.