Tag
The paper proposes Muon-RT, a momentum-free variant of the Muon optimizer that reuses the EMA teacher-student weight displacement in LLM self-distillation, eliminating separate momentum buffers while improving accuracy and reducing persistent training memory on Qwen2.5-7B and Olmo-3-7B.
A new EXL3 quant of Qwen3.8-27B by yeasah achieves the lowest KLD (0.0002) among competitors while requiring only 18.8 GiB of VRAM for weights, outperforming quants from Unsloth and turboderp.
The paper proposes Ori, a drift-aware streaming k-PCA low-rank optimizer for LLM training inspired by Oja++, which tracks shifting principal gradient directions at low per-step cost, reducing the performance gap to full-rank optimizers by up to 69.3% on 1B LLaMA pretraining and achieving best performance in 10 of 16 RoBERTa-Base fine-tuning scenarios.
The paper introduces COUNT (Counting Offline Unlocks N-gram Transformers), a method that builds n-gram embeddings from precomputed counts stored on host memory without learned parameters, improving validation loss for 1B-parameter LMs and enabling training-free domain adaptation at inference time.
该推文总结了优化 LLM 推理的 12 种 KV 缓存技术，包括滑动窗口、跨层共享、MLA、量化、前缀复用、分页、驱逐和 GQA/MQA 等，帮助开发者根据具体瓶颈选择合适的组合。
curl 8.23.0 引入多项性能优化，包括 base64 编解码、URL 编解码、日期和数字解析提速，以及约 40% 的结构体内存缩减和更少的内存分配。
提出 FramePack-U，一种免训练的常量内存帧打包框架，可将任意长度视频压缩为固定大小的视觉表示，直接插入现成 VLM 推理，在 Long VideoBench、MLVU 和 Video-MME 上显著超越全分辨率基线及现有 token 压缩方法。
作者介绍 ALHR（Adaptive Learnable Hierarchical Routing），一种基于静态二叉树与可学习函数的稀疏注意力系统，在 MQAR 测试中每查询仅读取约 30 个键（相比稠密注意力的 512 个），实现约 35.3 倍 KV 压缩，同时保持接近的准确率，并将推理复杂度降至 NlogN、显存线性扩展。
论文提出 LadderEdit，通过将每个 LoRA 编辑适配器压缩为低秩草图，并在草图不满足重写、泛化和局部性要求时沿阶梯提升秩，实现 5.2 倍的存储节省，并在 50,000 次连续编辑上保持有效。
Phase-HDC提出用梯度阈值取代Adam优化器的历史状态来训练超维分类器，在几乎不存储额外状态的情况下匹配Adam的精度，相比float32 Adam节省16-23倍存储，相比8位Adam节省4-6倍。
腾讯研究人员在 Hugging Face 上发布了 STEPQuant，一个针对 Delta-rule 循环状态的空间-时间后训练量化框架，用 6-bit 状态即可匹配 FP32 精度，并将服务内存最多削减 68.7%。
audio.cpp 带来多项运行时性能优化：Higgs Audio TTS 显存占用减少 48%（低于 6GB），HTDemucs 提速 2.2 倍，PocketTTS 在 CPU 上提速 2.2 倍，同时 WebUI 新增实验性的生成历史功能，目前支持 110+ 音频模型家族。
EncBank extends CoMem's intermediate-state interface, treating an LLM's lower layers as a reusable document encoder whose compact low-bit outputs can be cached and reused across queries. Evaluated on Qwen backbones, 4-bit storage nearly matches native precision while cutting persistent storage to 28.1%, with notable prefill speedups at some accuracy cost.
A project demonstrates a 2.78-trillion-parameter model that can run on just 8GB of memory using a pure C99 inference engine with no BLAS, frameworks, or GPU reliance, trading speed for extreme memory efficiency (about 26.5 seconds/token at 8GB, 5.6 seconds/token with 128GB+).
A paper proposing Residual Quantization (RQ) as a representation layer for contextual bandits, enabling nonlinear reward modeling with strictly bounded O(1) memory via gradient-boosted pseudo-targets and online depth adaptation through a shadow mechanism, beating non-RQ counterparts on 11 of 13 datasets while matching XGBoost and neural baselines at up to 1000x less memory.
HLA-WM提出一种训练免费的混合线性注意力框架，通过几何引导的检索与递归线性状态重组合来缓解GatedDeltaNet在长时视频世界模型中的远距离遗忘问题，在60秒基准上以12倍内存节省同时提升所有一致性与相机控制指标。
ETH Zurich and Microsoft researchers propose UnRL, an uncertainty-aware adaptive 3D mapping framework that refines voxels using semantic entropy, geometric curvature, and texture richness, with a reinforcement learning agent controlling the memory-accuracy trade-off under a user-specified budget.
GEM-KMeans introduces a spectrally normalized, memory-efficient NLR formulation for K-means clustering that fuses updates into a single matrix-multiplication epilogue, reducing GPU HBM storage to one factor while maintaining clustering accuracy at massive scale.
The paper introduces Honeycomb, a video world model that uses HexMemory, a low-rank fixed-size scene memory represented as six spatial and spatiotemporal planes, enabling constant storage during long-horizon video generation with strong generation quality and revisit consistency on WorldScore and RealEstate10K benchmarks.
The paper proposes STEPQuant, a spatial-temporal post-training quantization framework for delta-rule recurrent states in linear attention models, achieving near-FP32 accuracy at 6-bit precision, over 5x state compression, and up to 68.7% serving memory reduction when integrated into SGLang.