Tag
A new dynamic quantization method called Voodoo Quant, now open-sourced under MIT license, uses gradient descent to optimize per-tensor quant levels for AI models, showing better performance at aggressive quant levels.
This paper proposes adaptive reciprocal knowledge distillation (AR-KD), a novel method that improves knowledge transfer from teacher to student models by simplifying the teacher's output distribution through relational alignment, achieving up to 7.13% accuracy gain on CIFAR-100 and ImageNet-1k datasets.
This paper describes six submissions to the WMT26 Model Compression Shared Task, using routing-informed expert pruning and MXFP4 quantization to compress GPT-OSS-20B into smaller translation models with parameters ranging from 4.186B to 7.770B.
LILA is a calibration-free structured pruning method for large language models that uses latent spectral geometry to score neuron importance, achieving competitive performance without calibration data.
This paper proposes a knowledge distillation framework to compress vision-language models for on-device fire detection, showing that compact models can retain most of their teacher's capability while being deployable on embedded hardware.
The paper presents a reasoning-aware compression framework that identifies and protects vulnerable reasoning circuits in large reasoning models to improve energy efficiency, achieving Pareto-optimal performance over uniform quantization methods.
RAPID introduces a reliability-aware pair importance distillation method to improve knowledge distillation efficiency and performance in text classification tasks.
This paper proposes a damage-aware multi-armed bandit method for structured post-training pruning of vision and language transformers, showing reduced performance degradation compared to baseline approaches in experiments across various models and datasets.
This paper argues that retraining-free MoE compression checkpoints should be seen as compressed initializations that benefit from a small post-compression adjustment. It shows that with only 3,000 examples and one epoch, LM fine-tuning recovers a substantial portion of performance loss and is more cost-effective than knowledge distillation.
The paper introduces a unified Bayesian variational framework combining spike-and-slab sparsity and Gaussian mixture quantization for high compression rates in large neural networks with minimal accuracy loss.
Cerebras has announced the availability of the Qwen 3.8 27B model on its inference platform with a speed of 1500 tokens per second, detailing its model compression techniques such as quantization and pruning.
XMerge introduces a post-training method for compressing transformer-based LLMs by merging layers through cross-axis selection and reconstructive reconstruction, outperforming existing methods at aggressive depth reduction while maintaining performance.
This paper proposes a method to close the MLP reachability gap in Low-Rank Clone distillation by training the full deployed matrix, resulting in significant improvements in token efficiency and model performance at no additional inference cost.
This paper introduces a reinforcement learning-based distillation framework for training compact instruction-following rerankers, using off-policy GRPO for teacher enhancement and on-policy distillation for student learning, demonstrating superior performance under distribution shift.
This paper presents a theoretical framework showing that learning in neural networks generates graph symmetries (fibrations and coverings), which enable drastic model compression and improve continual learning performance.
This research report evaluates post-training ternarization of the Qwen3-4B model, achieving a 1.641-bit effective weight representation with substantial storage compression, while noting a performance trade-off and unresolved deployment acceleration issues.
Debias-SparseGPT is a bias-aware pruning method for large language models that reduces bias amplification from compression techniques like SparseGPT while maintaining model performance and efficiency.
Tencent AI announces Sherry, a quantization method that reduces model size from 1.5TB to 214GB without quality loss, enabling efficient deployment across GPUs.
This paper proposes a sample-wise adaptive temperature scaling method for Transformed Teacher Matching in knowledge distillation, improving performance on image classification benchmarks by locally minimizing KL divergence between teacher and student distributions.
This research paper analyzes capability-stratified degradation in ternary quantized language models, showing that while factual knowledge deteriorates significantly, commonsense reasoning and downstream task adaptability are retained, making the models viable for efficient edge deployment.