学习何时退出：微调语言智能体中的课程覆盖与特定规则失效

arXiv cs.AI 论文
language-agents fine-tuning supervised-fine-tuning tool-use agent-safety curriculum-learning rule-regression

摘要

该论文研究了工具使用型语言智能体在监督微调下的「选择性撤回」行为，提出一种能发现聚合准确率所掩盖的规则级回归的诊断方法，并表明微调本身并不足以保证智能体安全。

arXiv:2610.21045v1 Announce Type: new Abstract: Tool-using language agents must revise prepared actions when evidence changes or authority is revoked. Selective withdrawal means suspending affected actions, preserving supported work, and resuming only after sufficient repair. We study supervised fine-tuning in NAQD-Env, a synthetic environment with explicit dependencies and a deterministic reference policy. Four LoRA checkpoints of Qwen2.5-3B-Instruct compare an original curriculum with an enriched one, the latter under three system-prompt framings with identical views and targets. Three instruction-tuned baselines and all four checkpoints are evaluated under three inference prompts on fresh validation cases, held-out dependency families, and held-out structural combinations. Under the naqd prompt, SFT checkpoints reach decision agreement of 0.77-0.92 and withdrawal recall of 0.46-0.86, versus 0.45-0.63 and at most 0.06 for the baselines. Rule-level diagnostics qualify these gains. The original-curriculum checkpoint wrongly withdraws 26 of 34 actions with surviving alternative support; enriched checkpoints withdraw 0-1. Conversely, all SFT checkpoints narrow prohibited combinations less often than the 7B and 8B baselines, a case absent from both curricula. Delegation results show more withdrawal but poorer preservation of independently authorized actions, and all four checkpoints omit valid event reports. Framing contrasts vary across metrics and splits and do not establish a reproducible advantage or equivalence. One training run per arm, jointly changed curriculum properties, prior evaluation exposure, and baseline-SFT quantization differences limit causal attribution. The contribution is a method for detecting rule-specific regressions that aggregate accuracy can conceal, not evidence that fine-tuning alone ensures agent safety.
查看原文
查看缓存全文

缓存时间: 2026/10/16 06:54 

# Learning When to Withdraw: Curriculum Coverage and Rule-Specific Failures in Fine-Tuned Language Agents
Source: [https://arxiv.org/abs/2610.21045](https://arxiv.org/abs/2610.21045)
[View PDF](https://arxiv.org/pdf/2610.21045)

> Abstract:Tool\-using language agents must revise prepared actions when evidence changes or authority is revoked\. Selective withdrawal means suspending affected actions, preserving supported work, and resuming only after sufficient repair\. We study supervised fine\-tuning in NAQD\-Env, a synthetic environment with explicit dependencies and a deterministic reference policy\. Four LoRA checkpoints of Qwen2\.5\-3B\-Instruct compare an original curriculum with an enriched one, the latter under three system\-prompt framings with identical views and targets\. Three instruction\-tuned baselines and all four checkpoints are evaluated under three inference prompts on fresh validation cases, held\-out dependency families, and held\-out structural combinations\. Under the naqd prompt, SFT checkpoints reach decision agreement of 0\.77\-0\.92 and withdrawal recall of 0\.46\-0\.86, versus 0\.45\-0\.63 and at most 0\.06 for the baselines\. Rule\-level diagnostics qualify these gains\. The original\-curriculum checkpoint wrongly withdraws 26 of 34 actions with surviving alternative support; enriched checkpoints withdraw 0\-1\. Conversely, all SFT checkpoints narrow prohibited combinations less often than the 7B and 8B baselines, a case absent from both curricula\. Delegation results show more withdrawal but poorer preservation of independently authorized actions, and all four checkpoints omit valid event reports\. Framing contrasts vary across metrics and splits and do not establish a reproducible advantage or equivalence\. One training run per arm, jointly changed curriculum properties, prior evaluation exposure, and baseline\-SFT quantization differences limit causal attribution\. The contribution is a method for detecting rule\-specific regressions that aggregate accuracy can conceal, not evidence that fine\-tuning alone ensures agent safety\.

## Submission history

From: Mohamed Abouzahra \[[view email](https://arxiv.org/show-email/fd65bd11/2610.21045)\] **\[v1\]**Thu, 15 Oct 2026 00:38:39 UTC \(853 KB\)

相似文章

何时遗忘：长程智能体的风险受控上下文自适应

arXiv cs.AI

该论文提出 RECAP 框架，通过匹配上下文干预、执行关系预测与独立策略认证，研究长程智能体何时应保留、压缩或删除历史失败记录，在 Hunyuan-A13B 上以匹配的续跑成功率将平均提示成本降低 2.13%。

AdaptEvo：具有演化监督的自适应智能体学习

arXiv cs.CL

AdaptEvo 提出了一种在不完美监督下进行学习的框架，通过置信度自适应 GRPO（CA-GRPO）以及演化式的决策知识与评估规则，提升规则驱动的上下文决策任务（如内容审核）中 LLM 智能体的准确性，以及规则发生变化后的鲁棒性。在 In-Period 数据集上，其表现较 GRPO 提升 7.5 个百分点。