model-compression

Tag

Cards List
#model-compression

Structured Pruning of Large Language Models via Power Transformation and Sign-Preserving Score Aggregation with Adaptive Feature Retention

arXiv cs.CL ↗ · 2026-07-10 Cached

This paper proposes a structured pruning method for LLMs that addresses distribution mismatch, sign-information loss, and outlier influence when adapting unstructured pruning techniques, achieving comparable accuracy with 1.56-1.57x speedup on models like Llama-3-8B and Vicuna-v1.5-13B.

0 favorites 0 likes
#model-compression

@j_golebiowski: How to replace a 744B agent with a 1.7B model using your own traces, straight from dlt. Four steps:

X AI KOLs Timeline ↗ · 2026-07-09 Cached

This tweet explains how to use dlt to collect traces and replace a 744B parameter agent with a much smaller 1.7B model in four steps.

0 favorites 0 likes
#model-compression

Enhancing deep learning models for time series classification via knowledge distillation

arXiv cs.LG ↗ · 2026-07-09 Cached

This paper investigates knowledge distillation for time series classification using FCN, Inception, and ConvTran architectures on the UCR Archive, showing significant parameter reduction and performance maintenance.

0 favorites 0 likes
#model-compression

@SergioPaniego: Frontier models use distillation as a step of their post-training pipelines! In 2026 it has three jobs: - compress a bi…

X AI KOLs Timeline ↗ · 2026-07-08 Cached

Explains three uses of distillation in frontier model post-training pipelines: compressing large models into small ones, merging RL experts, and self-teaching. Includes a write-up detailing which models use each method.

0 favorites 0 likes
#model-compression

Empirical Minimal-Realisation Compression of Deep Neural Networks via Controllability-Observability Tests

arXiv cs.LG ↗ · 2026-07-08 Cached

This paper proposes a controllability–observability framework for compressing deep neural networks by reducing hidden-state redundancy, demonstrating significant compression with minimal accuracy loss on MNIST and CIFAR-10.

0 favorites 0 likes
#model-compression

@omarsar0: Banger compression paper from NVIDIA. (bookmark it) Bigger MoE models keep winning on quality, but serving them at inte…

X AI KOLs Following ↗ · 2026-07-07 Cached

NVIDIA's paper introduces Puzzle-75B-A9B, a compressed hybrid MoE model that doubles server throughput while preserving quality, enabling cost-effective deployment of large language models.

0 favorites 0 likes
#model-compression

nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16 · Hugging Face

Reddit r/LocalLLaMA ↗ · 2026-07-07 Cached

NVIDIA releases Nemotron-Labs-3-Puzzle-75B-A9B, a compressed hybrid MoE LLM derived from Nemotron-3-Super, achieving approximately 2× higher server throughput and improved concurrency while maintaining strong accuracy across reasoning, coding, and long-context benchmarks.

0 favorites 0 likes
#model-compression

Rethinking Neural Nonlinearity as Gating

arXiv cs.LG ↗ · 2026-07-07 Cached

This paper proposes Threshold Gating (TG) as a unified primitive for neural nonlinearity, showing that standard activation functions can be expressed as instances of TG. The authors validate their approach by converting pretrained models across various architectures without retraining and discuss hardware benefits.

0 favorites 0 likes
#model-compression

Variable Bit-width Quantization: Learning Per-Group Precision for "Bigger-but-Smaller" Language Models

arXiv cs.LG ↗ · 2026-07-07 Cached

Introduces Variable Bit-width Quantization (VBQ), a training-time method where each group of 64 weights learns its own bit-width (1,2,4,8) via Gumbel-Softmax relaxation. VBQ discovers a heterogeneous allocation that yields a 'bigger-but-smaller' regime, e.g., a 131M parameter model at 1.82 mean bits beats a 55M FP16 model while using less storage, and a 1.46B model matches a 593M FP16 with ~3.7x less storage.

0 favorites 0 likes
#model-compression

PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference

Reddit r/singularity ↗ · 2026-07-07 Cached

PuzzleMoE introduces a pairwise dual-mask expert merging algorithm and bit-level packing technique to compress large Mixture-of-Experts models, maintaining performance while reducing storage and accelerating inference.

0 favorites 0 likes
#model-compression

Generic Expert Coverage for Pruning SparseMixture-of-Experts Language Models

arXiv cs.AI ↗ · 2026-07-03 Cached

Proposes Generic TB-Coverage, a coverage-aware expert pruning method for sparse Mixture-of-Experts language models that uses only generic text corpora for calibration and preserves cross-corpus expert coverage, improving accuracy and reducing perplexity degradation.

0 favorites 0 likes
#model-compression

On the Utility and Factual Reliability of Pruned Mixture-of-Experts Models in the Biomedical Domain

arXiv cs.LG ↗ · 2026-07-03 Cached

This paper investigates the effects of domain-specific expert pruning on both utility and factual reliability of Mixture-of-Experts (MoE) models in the biomedical domain. It finds that moderate pruning preserves in-domain utility without immediate reliability loss, but extreme pruning increases hallucination risks, and generalization degrades rapidly in cross-domain settings.

0 favorites 0 likes
#model-compression

@no_stp_on_snek: ok folks you know the drill.. verdict up front: NVIDIA's 4-bit Qwen3.6-27B (NVFP4) is near-lossless. on my own held-out…

X AI KOLs Following ↗ · 2026-07-02 Cached

NVIDIA's 4-bit quantized Qwen3.6-27B (NVFP4) is found to be near-lossless compared to the full bf16 model, with behavioral differences being minor and random rather than systematic, making it a practical drop-in replacement.

0 favorites 0 likes
#model-compression

Rebuilding Gemma 4 31b... better... As 26b...

Reddit r/LocalLLaMA ↗ · 2026-07-02

A developer is rebuilding Gemma 4 31b into a smaller 26b model by removing weak SWA layers, adding attention-based residual networks (from Moonshot), and using topK logit targets for retraining, aiming for better long context and performance.

0 favorites 0 likes
#model-compression

Cross-Space Distillation: Teaching One-Step Students with Modern Diffusion Teachers

Hugging Face Daily Papers ↗ · 2026-06-30 Cached

Introduces Cross-Space Distillation, a method to transfer knowledge from modern high-capacity diffusion models to compact student models with different latent spaces using a lightweight latent interface called Bridge, enabling quality improvements without modifying the student backbone.

0 favorites 0 likes
#model-compression

Apparently you can skip entire transformer blocks at load time with minimal performance impact

Reddit r/LocalLLaMA ↗ · 2026-06-29

A developer implemented a --skip-layers flag for a llama.cpp fork, allowing users to skip entire transformer blocks at load time with minimal performance impact, based on recent papers. This provides another method for fitting large models on limited hardware, complementing quantization and bake-time pruning.

0 favorites 0 likes
#model-compression

FlexMoE: One-for-All Nested Intra-Expert Pruning for MoE Language Models

arXiv cs.LG ↗ · 2026-06-29 Cached

FlexMoE proposes a one-for-all nested intra-expert pruning method for MoE language models, enabling multiple deployable subnetworks from a single training run with minimal performance loss.

0 favorites 0 likes
#model-compression

Knowledge Distillation of Black-Box Large Language Models

Hacker News Top ↗ · 2026-06-28 Cached

Introduces Proxy-KD, a novel method for distilling knowledge from black-box large language models (like GPT-4) into smaller models using a proxy model, surpassing both traditional black-box and white-box KD techniques.

0 favorites 0 likes
#model-compression

clark-labs/clark-air-sana-1.6b-1.58bit · Hugging Face

Reddit r/LocalLLaMA ↗ · 2026-06-28 Cached

Clark Labs released Clark Air Sana 1.6B, a ternary-quantized version of the Sana 1.6B text-to-image transformer that is 8.6× smaller than FP16 while maintaining near-FP16 quality, enabling efficient deployment.

0 favorites 0 likes
#model-compression

We built a calibration-aware Q4_K_M quant of Qwen3.5 0.8B that recovers 96.5% of the BF16 gap vs pure llama.cpp Q4_K_M (SpectralQuant)

Reddit r/LocalLLaMA ↗ · 2026-06-27

A calibration-aware Q4_K_M quantization of Qwen3.5 0.8B using SpectralQuant recovers 96.5% of the BF16 performance gap compared to the standard llama.cpp Q4_K_M quant.

0 favorites 0 likes
← Previous
Next →
← Back to home

Submit Feedback