何时介入?关于大语言模型智能体错误与自我恢复的实证研究

arXiv cs.AI 论文

摘要

本文对 LLM 智能体在执行 Web 与计算机任务时的错误及自我恢复能力进行了实证研究,构建了包含 1,259 条轨迹、1,403 个错误事件的 ReInBench 数据集,并提出 ReCoF 方法来预测错误是否可自恢复,从而决定是否需要外部干预。

arXiv:2610.36471v1 Announce Type: new Abstract: LLM agents interact with Web services to perform tasks on users' behalf. Errors during these interactions can disrupt execution and prevent task completion, motivating external intervention. However, agents may self-recover through continued execution, raising the question of when to intervene and when to wait for self-recovery. We present an empirical study of errors and self-recovery in LLM agents. We construct ReInBench, a dataset containing 1,259 agent traces and 1,403 error events across computer-use and Web tasks. Our study examines how often and how quickly agents self-recover, how self-recovery relates to subsequent execution and task completion, and whether self-recovery can be predicted when an error occurs. We find that 725 error events self-recover, often within a few steps, with self-recovery performance differing across Execution LLMs. Self-recovery can also help agents recover from subsequent errors of the same type and support overall task completion. Intervention should therefore prioritize errors unlikely to self-recover, while allowing agents to continue when self-recovery is likely. Building on these empirical findings, we propose ReCoF, a method that predicts whether an error will self-recover to inform whether additional intervention is needed. ReCoF achieves an average relative improvement of 9.02% in Macro-F1 over the strongest baseline on ReInBench. Our code and data can be found at https://method-notes.github.io/ReCoF/.
查看原文
查看缓存全文

缓存时间: 2026/10/28 08:00

# When to Intervene? An Empirical Study of Errors and Self-Recovery in LLM Agents
Source: [https://arxiv.org/abs/2610.36471](https://arxiv.org/abs/2610.36471)
[View PDF](https://arxiv.org/pdf/2610.36471)[HTML \(experimental\)](https://arxiv.org/html/2610.36471v1)

> Abstract:LLM agents interact with Web services to perform tasks on users' behalf\. Errors during these interactions can disrupt execution and prevent task completion, motivating external intervention\. However, agents may self\-recover through continued execution, raising the question of when to intervene and when to wait for self\-recovery\. We present an empirical study of errors and self\-recovery in LLM agents\. We construct ReInBench, a dataset containing 1,259 agent traces and 1,403 error events across computer\-use and Web tasks\. Our study examines how often and how quickly agents self\-recover, how self\-recovery relates to subsequent execution and task completion, and whether self\-recovery can be predicted when an error occurs\. We find that 725 error events self\-recover, often within a few steps, with self\-recovery performance differing across Execution LLMs\. Self\-recovery can also help agents recover from subsequent errors of the same type and support overall task completion\. Intervention should therefore prioritize errors unlikely to self\-recover, while allowing agents to continue when self\-recovery is likely\. Building on these empirical findings, we propose ReCoF, a method that predicts whether an error will self\-recover to inform whether additional intervention is needed\. ReCoF achieves an average relative improvement of 9\.02% in Macro\-F1 over the strongest baseline on ReInBench\. Our code and data can be found at[this https URL](https://method-notes.github.io/ReCoF/)\.

## Submission history

From: Chiming Duan \[[view email](https://arxiv.org/show-email/45a1b654/2610.36471)\] **\[v1\]**Tue, 27 Oct 2026 01:26:53 UTC \(4,063 KB\)

相似文章

更多记忆,更差的智能体:LLM智能体中的错误复现与反持久化

arXiv cs.AI

该论文提出"错误复现"(error reproduction)机制,指出在LLM智能体外部记忆中持久化抽象规则会循环放大错误,并提出Anti-Persistence记忆设计:仅累积轨迹与结果、按需临时合成指导,从而在多个基准上取得更高成功率与更低LLM调用成本。

审计医疗大语言模型智能体的升级处理可靠性:工作流状态故障的扰动受控基准测试

arXiv cs.AI

该论文提出了一个受扰动控制的医疗LLM智能体基准测试,包含400个多轮episode,评测在缺失、过时、冲突和迟到信息下的升级/拒答决策能力。结果显示模型在干净环境下准确率达0.97,但在不确定性扰动下漏掉32-97%的必要升级,且基于词汇规则的控制器在未见过的同义改写下召回率降至0。

您好!您只提供了文章标题: **Beyond Perfect Teammates: Multi-Agent Collaboration with Unreliable Colleagues** 但没有提供需要翻译的正文内容。请将完整的 markdown 文章粘贴过来,我会为您翻译成简体中文。

Hugging Face Daily Papers

您提供的内容已经是简体中文摘要,并非需要翻译的英文原文。请提供需要翻译的英文 Markdown 文章原文,我将按照要求将其翻译为自然流畅的简体中文,保留标题、列表、强调、行内代码和链接等格式,并将专有名词(如 UnCoBench、LLM 等)保持英文不变。

基于 LLM 的智能体系统中自动失败归因的机制研究

arXiv cs.LG

该论文对基于 LLM 的智能体系统的自动失败归因(AFA)进行了机制研究,发现模型存在'知道但不说'的差距:内部激活包含正确归因的信息,但无法显式输出。作者据此提出 grafting 干预方法,在多个 LLM 和基准上优于直接提示方法。