Tag
A community post discusses the rivalry between ISTA's GSQ-RCO quantization method and Unsloth, noting that both teams' competition benefits users with better quants, while questioning why ISTA doesn't publish KLD metrics for its quants.
ISTA DASLab released a Coder build of DeepSeek-V4.1-Flash-1017, a 763B sparse MoE model compressed to roughly 1 bit per transformer parameter via GSQ quantization and RCO-based expert pruning (192 of 384 experts kept), retaining 98.7% of LiveCodeBench v6 and 94.8% of SWE-bench Verified performance in a 72.1 GB GGUF.
TinyAstral evolves a frozen teacher's agent harness via recursive self-improvement with statistical acceptance tests, then distills the gains into a 4B student that matches the teacher's RoboTwin success (62.8% vs 63.6%) using about one fifth of the accelerator time.
OpenBonsai 是一个 Apache-2.0 的后训练三值量化框架，可将 BF16 模型和微调压缩为约 2 GB 的三值 GGUF 文件，兼容 Prism 的 PTQ1_0 格式，并用 Qwen3.5-9B（18 GB → 2 GB）验证了效果。
提出一种层级低秩重参数化方法，将PEFT方法的可训练参数重组为近似方阵并递归进行低秩分解，可在压缩约一个数量级参数的同时保持甚至提升LoRA/DoRA在常识推理与数学推理任务上的性能。
论文提出 WISD（Weakening-Induced On-Policy Self-Distillation），通过将大推理模型与量化的弱化版本进行对比生成训练信号，在无需特权数据的情况下实现自蒸馏，在数学与代码推理基准上平均提升 4.0–5.5 个百分点并达到最先进性能。
kaitchup released statistically-lossless SLQ-quantized GGUF versions of Qwen3.8-27B in three accuracy-target variants (EAR98, EAR98.5, EAR99), built by mixing donor quantized weights across 498 matrices.
GS-MoE 介绍了一种免训练的 MoE 压缩框架，通过分组子空间共享（低秩）与基于 Nyström 的神经元剪枝两条压缩轴同时削减专家参数；在 DeepSeek-MoE-16B-Base 上以 60% 参数压缩保留约 90% 的推理性能，优于 D2-MoE 的 73%。
论文提出 S-SALAAD 框架，在训练中联合学习模型权重及其低秩与结构化稀疏分量，配合预算感知压缩和专用 CPU/GPU kernel，将单个检查点的弹性压缩性转化为更低的内存占用与推理延迟，适用于边缘设备部署。
The paper shows that MoE channel selectors need no offline calibration data when using top-k selection instead of calibrated thresholds, and introduces a weight-derived reader (ternary sketch + low-rank map) that retains 98.1% accuracy at 17.8% of routed-expert reads while speeding up MoE blocks 1.60-2.69x over optimized vLLM.
EDWARD-NMT 提出了一种弹性 MoE 编码器-解码器架构，通过 Matryoshka 式弹性维度、自适应 sigmoid 路由和双一致性目标，实现「一次训练、随处部署」，可在资源受限设备上提取子网络且无需重训练即优于同等规模的稠密 Transformer。
ThinkBit 是一个严格标量的两比特量化框架，通过分层比特分配、Gumbel-softmax 搜索码字、知识蒸馏重校准和量化感知训练来保留推理能力，在 Qwen3-1.7B 和 14B 模型上以远小于 bfloat16 的体积取得接近原模型的 MATH-500 成绩。
The paper introduces Layer-wise Error Modeling Search (LEMS) and KFAC-SVD to improve low-rank SVD compression of LLMs, refining global rank allocation and decomposition quality to deliver up to 4.8 p.p. zero-shot accuracy gains and 15% average perplexity improvements across Mistral, Qwen3, and Llama 3 families.
提出了一种基于Kolmogorov–Arnold Networks（KAN）的非线性投影方法来增强低秩分解，用于NLP嵌入层压缩，实验表明该方法在固定维度下能生成更忠实的压缩嵌入。
The paper introduces Common-Support Distillation (CSD), a unified logits-based framework for cross-tokenizer knowledge transfer that constructs a shared support between teacher and student and matches induced distributions with a single objective, outperforming existing cross-tokenizer distillation methods across multiple teacher-student settings.
The paper introduces LEAN, a loss-engineered regularization method that reshapes a trained model's weight-sensitivity distribution, shrinking the sensitive tail 5-13x and enabling models to survive hundreds of bit-flip attacks while storing ImageNet ResNet-50 at a 3.29-bit payload (2.4x smaller than INT8).
A Virginia Commonwealth University paper explores extreme low-state weight quantization (3, 5, 7, 16 states) for Transformer-based electricity-load forecasting, showing that quantization-aware training and accumulated-update thresholding can retain predictive performance while reducing weight updates by ~99.7%.
论文提出 LADDER（Layer-Wise Allocation via Damage-Driven Evaluation and Refinement），一种针对大语言模型的非均匀后训练剪枝方法，通过逐层评估最小化每移除参数的性能损失，生成嵌套掩码轨迹，在 70% 以上高稀疏度下表现显著优于基线。
A hobbyist researcher proposes a speculative decoding scheme where a 2-bit model copy on GPU drafts tokens while the full residual (6-8 bit) in system RAM verifies them, only exchanging activations over PCIe. Early measurements suggest 2-bit + 8-bit residual outperforms Q8_0 quality and the 2-bit drafter beats Gemma 4's official drafter on acceptance rate.
The paper introduces CERTA, a decision-level statistical certificate using conformal risk control to determine when a quantized LLM can safely substitute for its BF16 reference in clinical verification tasks, showing that 8-bit quantization certified nearly all decisions while 4-bit configurations cut deployment costs to ~26% for 72B models.