model-compression

Tag

Cards List
#model-compression

Memory Efficient Tabular Foundation Models

arXiv cs.LG ↗ · 2026-07-31 Cached

This paper investigates memory requirements for tabular foundation models like TabPFN and shows that model compression (e.g., INT4 quantization) can reduce memory footprint up to 7.6x with minimal accuracy loss, improving practical deployment efficiency.

0 favorites 0 likes
#model-compression

TriSP: Tri-Signal Structured Pruning for Large Language Models

arXiv cs.AI ↗ · 2026-07-28 Cached

TriSP introduces a tri-signal importance metric combining weight magnitude, activation norm, and gradient sensitivity for structured pruning of LLMs, achieving lowest perplexity and high throughput improvements on LLaMA-7B.

0 favorites 0 likes
#model-compression

Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization

arXiv cs.AI ↗ · 2026-07-28 Cached

Proposes a two-stage structured pruning framework for LLMs that jointly optimizes latency and model size using multi-objective depth pruning and parallel Bayesian optimization, achieving favorable trade-offs for edge deployment.

0 favorites 0 likes
#model-compression

CausalGate: Causal Importance Distillation for Transformer Module Pruning

arXiv cs.LG ↗ · 2026-07-28 Cached

CausalGate introduces a method that uses causal interventions to measure the importance of transformer sub-layers and distills this into static scalar gates for efficient inference without runtime overhead, outperforming existing pruning and routing methods.

0 favorites 0 likes
#model-compression

I want to run Kimi K3 at home, so I’m trying to make 2.8T-scale experimentation cheaper

Reddit r/LocalLLaMA ↗ · 2026-07-27

A retired engineer proposes Blueprint Distillation (BPD) to separate expensive teacher analysis from student compression for large MoE models like Kimi K3, enabling reusable compression blueprints for multiple student sizes.

0 favorites 0 likes
#model-compression

Neural Feature Governance: Extending Atom Prevalence

arXiv cs.LG ↗ · 2026-07-27 Cached

This paper introduces Neural Atom Prevalence (NAP), a Bayesian framework for structured node-level model selection that achieves high sparsity, accuracy, and uncertainty quantification in neural networks.

0 favorites 0 likes
#model-compression

Break Through the Compression Bottleneck: From Theory to Practice

arXiv cs.CL ↗ · 2026-07-24 Cached

This paper provides the first mathematical proof that low-rank decomposition and quantization are non-orthogonal when combined for LLM compression, leading to performance degradation, and proposes a novel Diagonal Adhesive Method (DAM) to mitigate this loss.

0 favorites 0 likes
#model-compression

Session-Adaptive Orthogonal Distillation (SAOD)? Technology compresses 744B (1.5TB) to under 100GB?

Reddit r/LocalLLaMA ↗ · 2026-07-22

Session-Adaptive Orthogonal Distillation (SAOD) is a technique that compresses a 744-billion parameter model (1.5TB) to under 100GB, greatly reducing storage and inference costs.

0 favorites 0 likes
#model-compression

@akshay_pachaar: LLM quantization techniques I'd learn if I had to fit a 70B model on a single GPU: (bookmark this) A 70B model in FP16 …

X AI KOLs Following ↗ · 2026-07-22 Cached

A Twitter thread explains five key LLM quantization techniques (RTN, GPTQ, AWQ, LLM.int8(), QAT) for fitting large models on limited hardware, and references a comprehensive study paper.

0 favorites 0 likes
#model-compression

@rohanpaul_ai: Paper from Meta shows Quantized reasoning models often lose because they keep doubting a correct answer instead of fini…

X AI KOLs Following ↗ · 2026-07-22 Cached

Paper from Meta shows that quantized reasoning models often fail because compression makes them second-guess correct answers, but a small penalty on hesitation words like 'wait' or 'but' can cut reasoning length by 12-23% while maintaining or improving accuracy.

0 favorites 0 likes
#model-compression

Beyond Single-Dimensional Compression: The Compound Sparsity Frontier of Large Language Models

arXiv cs.LG ↗ · 2026-07-22 Cached

This paper introduces a compound sparsity framework for LLMs that combines static parameter pruning with dynamic token-level computation, showing that mixing both mechanisms outperforms single-dimension compression and delays performance degradation.

0 favorites 0 likes
#model-compression

Running a 13M ASR conformer on a microcontroller

Reddit r/LocalLLaMA ↗ · 2026-07-20

Details a method to run a 13 million parameter ASR Conformer model directly on a microcontroller, highlighting advances in edge AI deployment.

0 favorites 0 likes
#model-compression

Reducing the model parameter size?

Reddit r/LocalLLaMA ↗ · 2026-07-20

Discusses methods or research on reducing the size of model parameters, likely focusing on techniques like pruning or quantization to improve efficiency.

0 favorites 0 likes
#model-compression

KDFlow: A User-Friendly and Efficient Knowledge Distillation Framework for Large Language Models

arXiv cs.CL ↗ · 2026-07-20 Cached

KDFlow is a novel knowledge distillation framework for large language models that uses a decoupled architecture with SGLang for teacher inference and FSDP2 for student training, achieving 1.44x to 6.36x speedup over existing frameworks.

0 favorites 0 likes
#model-compression

qZACH-ViT: Quantization-Aware Intrinsic Explanations with Recursive Attribution-Stabilized Optimization

arXiv cs.LG ↗ · 2026-07-20 Cached

Introduces qZACH-ViT, a quantization-aware extension of ZACH-ViT with recursive intrinsic explanations, and Recursive Attribution-Stabilized Optimization (RASO) for stable attribution gradients. Achieves high prediction agreement and speedups on MedMNIST datasets after INT8 conversion.

0 favorites 0 likes
#model-compression

@sudoingX: this lab took qwen 3.6 27b, the model i've been calling king of the 24gb tier all month, and crushed it down to 3.9gb. …

X AI KOLs Timeline ↗ · 2026-07-16 Cached

PrismML announces Bonsai 27B, a binary-quantized version of Qwen3.6 27B that runs on a phone using only 1.125 bits per weight, claiming 89.5% intelligence retention. The model is being independently tested by @sudoingX to verify performance.

0 favorites 0 likes
#model-compression

i tried ternary decomposition instead of quantization. it works as good at q4km but takes slightly more vram. while being completly ternary. and completly PTQ (no QAT)

Reddit r/LocalLLaMA ↗ · 2026-07-16

The author claims that ternary decomposition performs as well as Q4_K_M quantization while using slightly more VRAM, and is fully ternary and PTQ without QAT.

0 favorites 0 likes
#model-compression

Hy3 1Bit 89-93 GB

Reddit r/LocalLLaMA ↗ · 2026-07-15

Announcement of Hy3 1-bit quantized model with 89-93 GB memory footprint.

0 favorites 0 likes
#model-compression

Prism ML in talks with Apple to shrink models for local use on phones

Reddit r/ArtificialInteligence ↗ · 2026-07-14

Prism ML is in talks with Apple to deploy model-shrinking technology that would allow powerful AI models to run directly on iPhones, improving privacy and reducing reliance on cloud inference.

0 favorites 0 likes
#model-compression

EvoLP: Self-Evolving Latency Predictor for Model Compression in Real-Time Edge Systems

arXiv cs.LG ↗ · 2026-07-13 Cached

EvoLP is a self-evolving latency predictor for neural network models on edge devices, designed to guide model compression while satisfying strict latency constraints. It outperforms prior methods across multiple edge devices and model variants.

0 favorites 0 likes
← Previous
Next →
← Back to home

Submit Feedback