样本自适应零阶后训练

arXiv cs.LG 论文
zeroth-order-optimization fine-tuning llm memory-efficient sample-adaptive convergence

摘要

这篇论文提出 SAZO（样本自适应零阶优化），通过在线学习样本重要性并自适应分配查询，来加速大语言模型的零阶优化微调，在保持低内存开销的同时实现最高 6 倍的收敛加速。

arXiv:2610.12566v1 Announce Type: new Abstract: Zeroth-order (ZO) optimization offers a memory-efficient alternative to backpropagation for fine-tuning large language models. However, existing ZO methods typically sample training examples uniformly, overlooking their evolving importance and leaving avoidable estimation variance. To address this limitation, we propose Sample-Adaptive Zeroth-Order Optimization (SAZO), which learns sample importance online and adaptively allocates queries to accelerate convergence. SAZO extracts gradient-norm information from existing directional differences to estimate sample importance without access to true gradients. Furthermore, to track evolving importance from sparse minibatch feedback, SAZO maintains per-example exponential moving averages that aggregate observations across updates. The method requires no additional model evaluations and only $O(N)$ additional scalar state compared with MeZO. We prove that SAZO asymptotically matches the leading convergence term of the minimum-variance sampling oracle, and achieves a strictly faster convergence rate than standard methods. Extensive experiments across four model families demonstrate improved average downstream performance and up to $6\times$ faster convergence to the same training loss in terms of query count.
查看原文
查看缓存全文

缓存时间: 2026/10/12 06:41 

# Sample-Adaptive Zeroth-Order Post-Training
Source: [https://arxiv.org/abs/2610.12566](https://arxiv.org/abs/2610.12566)
[View PDF](https://arxiv.org/pdf/2610.12566)

> Abstract:Zeroth\-order \(ZO\) optimization offers a memory\-efficient alternative to backpropagation for fine\-tuning large language models\. However, existing ZO methods typically sample training examples uniformly, overlooking their evolving importance and leaving avoidable estimation variance\. To address this limitation, we propose Sample\-Adaptive Zeroth\-Order Optimization \(SAZO\), which learns sample importance online and adaptively allocates queries to accelerate convergence\. SAZO extracts gradient\-norm information from existing directional differences to estimate sample importance without access to true gradients\. Furthermore, to track evolving importance from sparse minibatch feedback, SAZO maintains per\-example exponential moving averages that aggregate observations across updates\. The method requires no additional model evaluations and only $O\(N\)$ additional scalar state compared with MeZO\. We prove that SAZO asymptotically matches the leading convergence term of the minimum\-variance sampling oracle, and achieves a strictly faster convergence rate than standard methods\. Extensive experiments across four model families demonstrate improved average downstream performance and up to $6\\times$ faster convergence to the same training loss in terms of query count\.

## Submission history

From: Haiyuan Wan \[[view email](https://arxiv.org/show-email/ff136a92/2610.12566)\] **\[v1\]**Thu, 8 Oct 2026 18:00:37 UTC \(305 KB\)

相似文章

为大语言模型非结构化后训练剪枝构建嵌套 LADDER

arXiv cs.LG

论文提出 LADDER（Layer-Wise Allocation via Damage-Driven Evaluation and Refinement，即基于损伤评估与优化的逐层剪枝分配），一种面向大语言模型的非均匀后训练剪枝方法。该方法通过逐层评估来最小化每移除参数所带来的性能损失，并生成一条嵌套的掩码轨迹；在稀疏度超过 70% 的情况下，其表现显著优于基线方法。

软自适应策略优化

Papers with Code Trending

SAPO引入了一个平滑、温度控制的门控机制，以自适应地减弱强化学习中的离策略更新，与使用硬裁剪的方法相比，提升了训练稳定性和性能。

OnlineQAT：面向超低位大语言模型的在线策略蒸馏

arXiv cs.CL

OnlineQAT 采用两阶段框架：先通过块级 QAT 获得低位初始化，再在学生自生成的轨迹上，利用冻结的全精度教师进行反向 KL 的在线蒸馏，从而缓解固定补全训练与部署时学生自采样前缀之间的分布错配。在 Qwen3-1.7B 上，OnlineQAT 在 W3A16（57.28）和 W2A16（32.52）设置中取得对比方法中的最佳平均分，较 ReasoningQAT 分别提升 2.90 和 0.44 分。