Tag
Taco 提出一种跨平台 LLM 压缩方法，通过纯整数推理、精确的 float32 到整数转换与一致性测试向量，确保编码端与解码端在不同 GPU/CPU 硬件上产生完全相同的输出，并在 7 个后端上验证了哈希一致。
ReCal 是一种即插即用的校准方法，在结构化剪枝前通过教师-剪枝探针间的前向 KL 识别受损预测并重加权校准统计，从而提升剪枝模型在在线策略蒸馏（OPD）后的恢复效果，在 AIME 上最高提升 16.7 个百分点。
LittleBit 提出了一种通过低秩潜在矩阵分解与二值化实现超低比特（低至0.1 BPW）LLM 量化的框架，配合多尺度补偿机制，在 Llama2-7B 上以 0.1 BPW 超越了现有 0.7 BPW 方法，实现约 31 倍内存压缩与最高 11.6 倍推理加速。
Samsung Labs 释出 LittleBit 项目（NeurIPS 2025 / ICML 2026），通过潜在空间低秩分解与二值化实现大语言模型的亚 1-bit（低至 0.1 bpw）压缩，并在 LittleBit-2 中引入 Joint-ITQ 潜在几何对齐初始化以提升性能。
Nokia Applied Research and Florida State University researchers present capability scaling-down laws for LLM compression (pruning, quantization, distillation), showing that sharing density-response relations halves the measurements needed to fit pruning predictors while matching fully fitted regressions within 0.020 nats per token on Pythia and OLMo-2 models.
HyperZip proposes an efficient LLM-based lossless data compression framework that uses diffusion-based LLMs with Multi-Token Prediction and a hypernetwork to adapt models to target data without fine-tuning, achieving a superior compression-rate/throughput trade-off over autoregressive baselines.
The paper introduces a layer-wise curriculum learning method for efficient LLM compression, achieving state-of-the-art performance with significant reductions in GPU memory usage and training time.
PrismML, a Caltech-founded startup, released Bonsai 2, a highly compressed LLM that reduces memory usage by 9-10x with minimal performance loss, aiming to enable AI on consumer devices like PCs and smartphones.
The paper introduces the 'Compression Trinity' framework, jointly applying sparsity, quantization, and low-rank approximations to compress Large Language Models for improved efficiency and performance.
This paper systematically evaluates LLM compression methods and reveals asymmetric harms, such as disproportionate loss of head knowledge and increased confidence in incorrect answers, which are masked by standard aggregate metrics.
Quantization-Aware Healing is a method that recovers compressed 4-bit language models by distilling directly from the original uncompressed model, offering faster and more stable performance than Quantization-Aware Training.
This paper shows that gently compressed LLMs can pass standard data-free quality guards (perplexity, MMLU, output fidelity) yet still invent procedure steps when used as agents, and proposes a data-free two-axis screen to detect such failures before deployment.
This paper introduces a compound sparsity framework for LLMs that combines static parameter pruning with dynamic token-level computation, showing that mixing both mechanisms outperforms single-dimension compression and delays performance degradation.
AngelSlim/Hy3-GGUF is a toolkit for quantizing and deploying Hy3 large language models on llama.cpp, featuring MTP self-speculative decoding and a thinking/tool-call parser for efficient inference.
KronQ is a post-training quantization framework that incorporates gradient covariance using a Kronecker-factored Hessian approximation, enabling bidirectional incoherence processing and improved sensitivity metrics for mixed-precision allocation. It achieves low perplexity even at 2-bit weight-only quantization on large models like LLaMA-3-70B.
LACE-SVD is a novel low-rank compression method for large language models that uses a loss-aware rank allocation strategy and a propagation-aware correction technique to mitigate cumulative error propagation in the residual stream, achieving better perplexity than prior SVD-based methods at high compression ratios.
This paper examines how LLM-based compression of financial documents can distort investment decisions by losing contextual qualifiers and introducing model-dependent biases, proposing Agentic Context Compression to audit disagreements against the original source.
This paper empirically investigates whether aligning the allocation cost with the output-space objective improves compressed model fidelity in ROCKET, a training-free LLM compression method. Results show a trade-off between accuracy and perplexity, with effects more pronounced at higher compression ratios.
A novel end-to-end framework for LLM compression that jointly optimizes structural pruning and mixed-precision quantization, achieving significant perplexity reductions and speedups over state-of-the-art methods, especially at ultra-low bit precisions.
Introduces SigmaScale, a method that learns auxiliary scaling matrices for SVD-based LLM compression, showing competitive performance on Llama 3.1 8B and Qwen3-8B benchmarks.