何时介入?关于大语言模型智能体错误与自我恢复的实证研究
摘要
本文对 LLM 智能体在执行 Web 与计算机任务时的错误及自我恢复能力进行了实证研究,构建了包含 1,259 条轨迹、1,403 个错误事件的 ReInBench 数据集,并提出 ReCoF 方法来预测错误是否可自恢复,从而决定是否需要外部干预。
查看缓存全文
缓存时间: 2026/10/28 08:00
# When to Intervene? An Empirical Study of Errors and Self-Recovery in LLM Agents Source: [https://arxiv.org/abs/2610.36471](https://arxiv.org/abs/2610.36471) [View PDF](https://arxiv.org/pdf/2610.36471)[HTML \(experimental\)](https://arxiv.org/html/2610.36471v1) > Abstract:LLM agents interact with Web services to perform tasks on users' behalf\. Errors during these interactions can disrupt execution and prevent task completion, motivating external intervention\. However, agents may self\-recover through continued execution, raising the question of when to intervene and when to wait for self\-recovery\. We present an empirical study of errors and self\-recovery in LLM agents\. We construct ReInBench, a dataset containing 1,259 agent traces and 1,403 error events across computer\-use and Web tasks\. Our study examines how often and how quickly agents self\-recover, how self\-recovery relates to subsequent execution and task completion, and whether self\-recovery can be predicted when an error occurs\. We find that 725 error events self\-recover, often within a few steps, with self\-recovery performance differing across Execution LLMs\. Self\-recovery can also help agents recover from subsequent errors of the same type and support overall task completion\. Intervention should therefore prioritize errors unlikely to self\-recover, while allowing agents to continue when self\-recovery is likely\. Building on these empirical findings, we propose ReCoF, a method that predicts whether an error will self\-recover to inform whether additional intervention is needed\. ReCoF achieves an average relative improvement of 9\.02% in Macro\-F1 over the strongest baseline on ReInBench\. Our code and data can be found at[this https URL](https://method-notes.github.io/ReCoF/)\. ## Submission history From: Chiming Duan \[[view email](https://arxiv.org/show-email/45a1b654/2610.36471)\] **\[v1\]**Tue, 27 Oct 2026 01:26:53 UTC \(4,063 KB\)
相似文章
更多记忆,更差的智能体:LLM智能体中的错误复现与反持久化
该论文提出"错误复现"(error reproduction)机制,指出在LLM智能体外部记忆中持久化抽象规则会循环放大错误,并提出Anti-Persistence记忆设计:仅累积轨迹与结果、按需临时合成指导,从而在多个基准上取得更高成功率与更低LLM调用成本。
审计医疗大语言模型智能体的升级处理可靠性:工作流状态故障的扰动受控基准测试
该论文提出了一个受扰动控制的医疗LLM智能体基准测试,包含400个多轮episode,评测在缺失、过时、冲突和迟到信息下的升级/拒答决策能力。结果显示模型在干净环境下准确率达0.97,但在不确定性扰动下漏掉32-97%的必要升级,且基于词汇规则的控制器在未见过的同义改写下召回率降至0。
您好!您只提供了文章标题: **Beyond Perfect Teammates: Multi-Agent Collaboration with Unreliable Colleagues** 但没有提供需要翻译的正文内容。请将完整的 markdown 文章粘贴过来,我会为您翻译成简体中文。
您提供的内容已经是简体中文摘要,并非需要翻译的英文原文。请提供需要翻译的英文 Markdown 文章原文,我将按照要求将其翻译为自然流畅的简体中文,保留标题、列表、强调、行内代码和链接等格式,并将专有名词(如 UnCoBench、LLM 等)保持英文不变。
智能体能学会预测自身的失败吗?基于最终结果的 LLM 智能体置信度校准
该论文提出一种在多步 LLM 智能体任务中,利用最终结果(而非单步正确性)来校准智能体自报置信度的强化学习方法。在 ALFWorld、TextWorld 和 WebShop 环境中,步级置信度对最终结果的 AUROC 从接近随机提升至 0.68–0.85,校准误差显著下降,且不损害任务成功率。
基于 LLM 的智能体系统中自动失败归因的机制研究
该论文对基于 LLM 的智能体系统的自动失败归因(AFA)进行了机制研究,发现模型存在'知道但不说'的差距:内部激活包含正确归因的信息,但无法显式输出。作者据此提出 grafting 干预方法,在多个 LLM 和基准上优于直接提示方法。