SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions

Hugging Face Daily Papers Papers
bayesian-deep-learning neural-network-compression quantization sparsity model-compression variational-inference pruning

Summary

The paper introduces a unified Bayesian variational framework combining spike-and-slab sparsity and Gaussian mixture quantization for high compression rates in large neural networks with minimal accuracy loss.

Compressing large-scale neural networks is essential for deploying models on resource-constrained devices. Most existing methods adopt weight pruning or low-bit quantization individually, often resulting in suboptimal compression rates to preserve acceptable performance drops. We introduce a unified framework for simultaneous pruning and low-bit quantization via Bayesian variational learning (\method), which achieves higher compression rates than prior baselines while maintaining comparable performance. The key idea is to employ a spike-and-slab prior to induce sparsity and model quantized weights using Gaussian Mixture Models (GMMs) to enable low-bit precision. Due to the intractability of the objective involving spike-and-slab priors with GMMs, we derive an efficient approximation that facilitates effective compression with minimal accuracy loss. In theory, we provide a consistent result for our proposed variational approach to a sparse and quantized deep neural network. Extensive experiments on compressing ResNet, BERT-base, Llama3.2, and Qwen2.5 models show that our method achieves higher compression rates than a line of existing methods with comparable performance drops. Project page: https://comeusr.github.io/SQS_Webpage.
Original Article
View Cached Full Text

Cached at: 09/09/26, 08:30 AM

Paper page - SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions

Source: https://huggingface.co/papers/2510.08999

Abstract

A unified Bayesian variational framework combining spike-and-slab sparsity and Gaussian mixture quantization achieves high compression rates for large neural networks with minimal accuracy loss.

Compressing large-scale neural networks is essential for deploying models on resource-constrained devices. Most existing methods adopt weightpruningorlow-bit quantizationindividually, often resulting in suboptimal compression rates to preserve acceptable performance drops. We introduce a unified framework for simultaneouspruningandlow-bit quantizationviaBayesian variational learning(\method), which achieves higher compression rates than prior baselines while maintaining comparable performance. The key idea is to employ aspike-and-slab priorto induce sparsity and model quantized weights usingGaussian Mixture Models(GMMs) to enable low-bit precision. Due to the intractability of the objective involvingspike-and-slab priors with GMMs, we derive an efficient approximation that facilitates effective compression with minimal accuracy loss. In theory, we provide a consistent result for our proposed variational approach to asparse and quantized deep neural network. Extensive experiments on compressing ResNet, BERT-base, Llama3.2, and Qwen2.5 models show that our method achieves higher compression rates than a line of existing methods with comparable performance drops. Project page: https://comeusr.github.io/SQS_Webpage.

View arXiv pageView PDFProject pageGitHub0Add to collection

Get this paper in your agent:

hf papers read 2510\.08999

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2510.08999 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2510.08999 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2510.08999 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Sequential sparse Gaussian process quantile regression

arXiv cs.LG

This paper presents a sparse Gaussian process framework for quantile regression that uses a Laplace approximation for posterior inference and variance-based mechanisms for adaptive inducing-input placement and data acquisition.

LLM Compression with Jointly Optimizing Architectural and Quantization choices

arXiv cs.LG

Researchers from UiT and University of Oslo propose a differentiable NAS framework that jointly optimizes architectural configurations and mixed-precision quantization for LLM compression, achieving up to 1.4× faster inference or 6% higher accuracy across seven reasoning tasks compared to sequential NAS-then-quantization baselines.

D-Quant: Driftable Entropy Coding for KV Cache Quantization

arXiv cs.CL

D-Quant is a flexible KV cache quantization framework using driftable entropy coding to convert variable-length representations into fixed-size bitstreams, enabling efficient parallel dequantization in attention kernels and achieving up to 7x memory reduction and 3.5x throughput improvement while preserving near BF16 performance.

@MinLiBuilds: Scary, quantization is about to change its game plan. Unsloth Dynamic is already smart enough: lightly compress sensiti…

X AI KOLs Timeline

The post discusses how quantization is shifting from fixed bit-widths to adaptive schemes: Unsloth Dynamic already compresses sensitive weights lightly and insensitive ones aggressively, while GSQ + RCO trains a network to use gradients to decide how and at what bit-width each weight is compressed, with IQ3_S performance nearing BF16 on Qwen3.8-Flash-Next-GSQ-RCO-GGUF.