CHIME: 用于长时域代理规划的信用感知分层记忆演化

# 信用感知的层级记忆演化：面向长期智能体规划
来源：https://arxiv.org/html/2609.02074
Tian Lan, Feihu Jiang, Muyang Ye, Bin Zhu, Qianghuai Jia, Longyue Wang（通讯作者）, Zhao Xu, Weihua Luo, Xiaodong Shi（通讯作者）

###### 摘要

规划是智能体将复杂的长期任务分解为可管理步骤的核心能力。测试时搜索和基于训练的方法虽能提升规划能力，但会带来高昂的推理成本或需要昂贵的训练数据。自我演化记忆则通过从智能体交互结果中积累可重用的经验到外部记忆库中，使得规划能力能在推理时持续提升而无需参数更新。然而，现有的自我演化记忆方法存在一个固有的信用分配问题：它们依赖最终任务结果作为反馈，但这种结果混淆了计划质量与执行错误以及环境因素，因此积累的规划经验往往是有偏且嘈杂的。为解决此问题，我们提出了信用感知的层级记忆演化（CHIME）。这是一个自我演化记忆框架，它维护独立的规划库和执行库，并遵循“先归因后记忆”的原则：CHIME首先将每个任务结果归因于计划、执行、两者皆有或两者皆无，然后仅更新相应的记忆库。在四个长期智能体基准上的大量实验表明，CHIME一致地超越了基于训练和自我演化记忆的最先进的基线方法。进一步的分析揭示了几个有趣的发现。例如，CHIME能以更少的记忆条目积累有效的记忆。此外，学习到的记忆价值忠实地反映了下游效用：高质量的规划记忆比执行记忆更有价值。最后，积累的记忆能有效地跨骨干模型迁移。代码将在https://github.com/ATH-MaaS/Marco-DeepResearch发布。

1厦门大学人工智能研究所
2非物质文化遗产数字保护与智能处理重点实验室（厦门大学），福建省与台湾省，文化和旅游部
3浙江大学
4阿里巴巴集团

## 1引言

长期任务对智能体系统提出了一个决定性挑战：智能体必须在相互依赖的步骤之间协调决策，同时在整个执行过程中持续遵守任务约束（Xie等人，2024 (https://arxiv.org/html/2609.02074#bib.bib26)；Wang等人，2024 (https://arxiv.org/html/2609.02074#bib.bib23)；Erdogan等人，2025 (https://arxiv.org/html/2609.02074#bib.bib4)）。智能体规划通过将总体目标分解为模块化、可独立执行的子目标（Wang等人，2024 (https://arxiv.org/html/2609.02074#bib.bib23)；Shen等人，2023 (https://arxiv.org/html/2609.02074#bib.bib18)；Zhang等人，2026b (https://arxiv.org/html/2609.02074#bib.bib34)；Liu等人，2026 (https://arxiv.org/html/2609.02074#bib.bib13)）并建立它们之间的依赖关系和约束（Kim等人，2023 (https://arxiv.org/html/2609.02074#bib.bib9)；Guo, Kingston, and Kavraki, 2024 (https://arxiv.org/html/2609.02074#bib.bib5)；Lan等人，2026 (https://arxiv.org/html/2609.02074#bib.bib10)），从而满足这些要求，使其成为解决长期任务的核心能力。

这促使先前的工作致力于提升智能体的规划能力。测试时搜索（Yao等人，2023a (https://arxiv.org/html/2609.02074#bib.bib28)；Hao等人，2023 (https://arxiv.org/html/2609.02074#bib.bib6)；Zhou等人，2024 (https://arxiv.org/html/2609.02074#bib.bib36)；Yu等人，2026 (https://arxiv.org/html/2609.02074#bib.bib32)；Team等人，2026a (https://arxiv.org/html/2609.02074#bib.bib20)）通过探索多个候选计划或轨迹来提升单个任务的表现，但会带来高昂的推理成本，且不会保留可重用的经验。为了跨任务保留和重用规划经验，后续工作遵循了两个方向：（1）规划器模型训练（Erdogan等人，2025 (https://arxiv.org/html/2609.02074#bib.bib4)；Si等人，2026 (https://arxiv.org/html/2609.02074#bib.bib19)；Liu等人，2026 (https://arxiv.org/html/2609.02074#bib.bib13)；Team等人，2026b (https://arxiv.org/html/2609.02074#bib.bib21)）将规划能力内化到模型参数中，但这需要昂贵的数据收集和后训练，因为识别高质量计划依赖于大量的下游执行；（2）自我演化记忆（Chen等人，2026 (https://arxiv.org/html/2609.02074#bib.bib2)；Ouyang等人，2026 (https://arxiv.org/html/2609.02074#bib.bib14)）则在外部、无需训练的记忆库中积累经验，提供了一种更高效且可扩展的替代方案。因此，本文聚焦于在自我演化记忆范式内改进规划。

参照标题图1：朴素自我演化智能体与CHIME的对比。朴素自我演化智能体将经验直接写入共享记忆。CHIME在更新规划或执行记忆之前，通过一个信用归因门控对任务结果进行归因。
然而，现有的自我演化记忆方法在智能体规划方面存在一个根本局限：它们将下游任务结果等同于计划质量。相同的结果可能源于不同的原因：成功可能源于一个合理的计划，也可能源于执行器从一个有缺陷的计划中恢复；而失败可能源于有缺陷的计划、不正确的执行或环境问题。因此，直接使用这样的结果作为记忆反馈会导致根本的信用分配失败。一方面，一旦有偏的经验被写入记忆库，它会在任务中被反复检索和重用，使得局部的误判累积成系统性的规划错误。另一方面，执行层面或环境层面的经验可能被误认为是规划记忆；检索到这些低层级的细节反而会误导高层级的规划决策，而不是提供规划指导。

为解决此问题，我们提出了信用感知的层级记忆演化（CHIME），这是一个自我演化的智能体规划框架，它用一个“先归因后记忆”的过程取代了基于结果的记忆更新（Ye等人，2026 (https://arxiv.org/html/2609.02074#bib.bib30)）。具体而言，CHIME包含三个组件：（1）层级记忆库明确地将规划和执行记忆分离为一个“规划库”和一个“执行库”，通过相似性检索和基于价值的重排序检索阶段特定的记忆，分别指导智能体的规划和执行；（2）信用归因门控在写入任何记忆之前，对任务、计划、执行和检索到的记忆进行反思，以将可重用的反馈归因于规划、执行、两个阶段或两者皆无；它还生成阶段特定的经验并识别误导性记忆；以及（3）信用感知记忆演化使用置信度加权的反馈更新检索到的记忆的价值，同时仅将过滤、合并或插入新经验到被归因的记忆库中。通过这种方式，CHIME仅使用被归因到该阶段的记忆来演化每个库，防止有偏的信号持续存在，并防止执行层面的记忆污染规划库。

在两个骨干模型上对四个长期基准进行的实验表明，CHIME一致地超越了基于训练和自我演化记忆的基线方法，在两个骨干模型上分别将评估平均表现比最强基线提高了2.96%和3.68%。进一步的分析揭示了几个有趣的发现：（1）CHIME在仅保留129条记忆的情况下实现了最高准确率，而强基线方法保留了3,585条；（2）学习到的记忆价值忠实地反映了下游效用，规划记忆带来的准确率提升是执行记忆的两倍多（21.7%→50.8% 对比 23.7%→35.4%）；（3）信用归因门控是可靠的，重复归因在高达97.9%的情况下保持一致，超过一半的失败通过生成的经验被挽救；（4）用一个更强的信用模型替换模型自身的反思信用，可进一步获得高达3.12%的提升；以及（5）积累的记忆能够有效地跨骨干模型迁移，超越A-MapReduce的迁移记忆高达4.68%。这些发现证实了“先归因后记忆”是CHIME有效性的关键。

## 2相关工作

#### 自我演化记忆。

自我演化记忆方法将外部记忆库视为智能体可演化的参数，将交互轨迹提炼为可重用的经验或技能（Yu等人，2025 (https://arxiv.org/html/2609.02074#bib.bib31)；Wang等人，2025 (https://arxiv.org/html/2609.02074#bib.bib22)；Zhang等人，2026a (https://arxiv.org/html/2609.02074#bib.bib33)）。与基于训练的方法不同，它们保持模型参数冻结，仅更新记忆库，避免了昂贵的轨迹收集和后训练（Li等人，2025 (https://arxiv.org/html/2609.02074#bib.bib12)；Wu等人，2025 (https://arxiv.org/html/2609.02074#bib.bib25)）。例如，ReasoningBank（Ouyang等人，2026 (https://arxiv.org/html/2609.02074#bib.bib14)）从自我判断的成功和失败轨迹中蒸馏出可泛化的推理策略，而UMEM（Ye等人，2026 (https://arxiv.org/html/2609.02074#bib.bib30)）使用强化学习联合优化记忆提取和管理。

#### 智能体规划。

智能体规划将复杂目标分解为结构化的步骤并协调它们的执行。它支撑着广泛的智能体系统（Yao等人，2023b (https://arxiv.org/html/2609.02074#bib.bib29)；Erdogan等人，2025 (https://arxiv.org/html/2609.02074#bib.bib4)；Liu等人，2026 (https://arxiv.org/html/2609.02074#bib.bib13)；Lan等人，2026 (https://arxiv.org/html/2609.02074#bib.bib10)）。长期任务通常涉及许多相互依赖的子任务。如果没有明确的计划来组织它们，智能体很容易失去进度，这种失败被称为“迷失在中间”问题（Lan等人，2026 (https://arxiv.org/html/2609.02074#bib.bib10)；Liu等人，2026 (https://arxiv.org/html/2609.02074#bib.bib13)）。随着近期基准引入越来越具有挑战性的任务，提升规划能力已成为智能体系统的核心研究问题（Wong等人，2025 (https://arxiv.org/html/2609.02074#bib.bib24)；Lan等人，2025 (https://arxiv.org/html/2609.02074#bib.bib11)；Yang等人，2025 (https://arxiv.org/html/2609.02074#bib.bib27)）。

#### 改进智能体规划。

现有努力可分为三个范式：（1）测试时搜索在推理时探索多个候选计划或轨迹，并通过价值估计或环境反馈在它们之间进行选择（Yao等人，2023a (https://arxiv.org/html/2609.02074#bib.bib28)；Hao等人，2023 (https://arxiv.org/html/2609.02074#bib.bib6)；Zhou等人，2024 (https://arxiv.org/html/2609.02074#bib.bib36)）。例如，MiroThinker-H1（Team等人，2026b (https://arxiv.org/html/2609.02074#bib.bib21)）和WebAnchor（Yu等人，2026 (https://arxiv.org/html/2609.02074#bib.bib32)）通过拒绝采样选择高质量计划；（2）规划器模型训练将规划能力内化到模型参数中（Erdogan等人，2025 (https://arxiv.org/html/2609.02074#bib.bib4)；Si等人，2026 (https://arxiv.org/html/2609.02074#bib.bib19)；Yu等人，2026 (https://arxiv.org/html/2609.02074#bib.bib32)）。例如，TodoEvolve（Liu等人，2026 (https://arxiv.org/html/2609.02074#bib.bib13)）构建了规划架构的模块化设计空间，并使用强化学习训练一个元规划器以合成特定任务的规划系统。然而，此类方法需要昂贵的轨迹收集和后训练，并且生成的规划器持续更新代价高昂（Liu等人，2026 (https://arxiv.org/html/2609.02074#bib.bib13)）；以及（3）自我演化记忆提供了一种无需训练且可持续的替代方案，但很少有工作专门为智能体规划演化记忆（Kagaya等人，2024 (https://arxiv.org/html/2609.02074#bib.bib8)）。一个代表是A-MapReduce（Chen等人，2026 (https://arxiv.org/html/2609.02074#bib.bib2)），它从过去的执行中演化结构化的提示，以改进长期智能体搜索中的任务分解和结果聚合。然而，这三个范式都通过最终任务结果来评估智能体的计划，隐含地将结果等同于计划质量。这种等价关系并不成立，因为结果也受执行细节和环境因素的影响。

## 3CHIME方法论

### 3.1自我演化智能体的任务形式化

我们考虑一个解决连续任务流的自我演化智能体。智能体的策略参数保持冻结，适应由跨情节演化的外部记忆库Mt\mathcal{M}_t执行。在情节t，智能体接收一个任务xtx_t，并从Mt\mathcal{M}_t中检索任务相关经验Mtret\mathcal{M}_t^{\mathrm{ret}}。以xtx_t和Mtret\mathcal{M}_t^{\mathrm{ret}}为条件，冻结的策略生成轨迹τt\tau_t，并从环境和评估器获得结果st s_t。然后，记忆更新算子将该情节提炼为可重用的经验：

Mt+1=U⁡(Mt,xt,τt,st).\mathcal{M}_{t+1}=\mathcal{U}\left(\mathcal{M}_{t},x_{t},\tau_{t},s_{t}\right). (1)
在评估期间，记忆库被冻结，因此性能反映了评估前积累的经验。现有的自我演化记忆方法主要使用最终结果st s_t作为记忆更新的监督信号。对于智能体规划而言，这个信号是有偏的：结果取决于计划和执行过程，因此它不能忠实地反映计划质量。

### 3.2CHIME概述

参照标题图2：CHIME概述。左：层级记忆库维护规划和执行记忆库。对于每个任务，相似性检索和基于价值的重排序选择阶段特定的记忆来指导规划器和执行器。右：在每个情节，信用归因门控对任务、计划、执行和检索到的记忆进行反思，将可重用的反馈归因于规划、执行、两者或两者皆无，并生成阶段特定的经验。反馈循环：信用感知记忆演化更新记忆价值，并将过滤、合并或插入新经验到被归因的记忆库中。
我们提出CHIME（图2 (https://arxiv.org/html/2609.02074#S3.F2)），一个用于智能体规划的“先归因后记忆”的自我演化框架。它将每个任务结果归因于规划、执行或外部因素，并仅更新相应的记忆库，从而减少有偏的反馈和跨阶段的记忆污染。CHIME包含三个组件：一个分离规划和执行经验的层级记忆库，一个将每个结果归因于其负责阶段的信用归因门控，以及一个根据归因的信用更新记忆库的信用感知记忆演化。

#### 层级记忆库。

与现有的自我演化方法（Ye等人，2026 (https://arxiv.org/html/2609.02074#bib.bib30)）不同，CHIME维护层级记忆库

Mt=(Mplan,t,Mexec,t),\mathcal{M}_{t}=\left(\mathcal{M}_{\mathrm{plan},t},\mathcal{M}_{\mathrm{exec},t}\right), (2)
包括一个规划库和一个执行库，由l∈{plan,exec}\ell\in\{\mathrm{plan},\mathrm{exec}\}索引。规划库存储战略经验（例如，子任务分解、约束检查、计划修正），而执行库存储操作性经验（例如，API使用、错误恢复、工具链调整）。每个记忆条目m包含一个内容c用于相似性检索，以及一个标量值v用于基于价值的重排序。对于每个新任务xtx_t，规划器和执行器首先使用相似性检索从各自库中获取Top-K候选记忆。然后，通过一个轻量级的价值重排序器（例如LLM）对这些候选记忆进行重新排序，选择Top-k记忆作为阶段特定的指导。

#### 信用归因门控。

在接收到任务结果st s_t后，CHIME不会直接将整个轨迹τt\tau_t写入记忆。相反，它激活信用归因门控，该门控利用LLM反思整个交互过程：它评估任务xtx_t的完成情况、计划τplan_t的质量、执行τexec_t的有效性以及检索到的记忆Mtret\mathcal{M}_t^{\mathrm{ret}}的相关性。基于此反思，门控输出一个归因决策a∈{plan,exec,both,neither}，以确定结果st s_t（以及生成的经验）应归因于哪个阶段。此外，门控为规划和执行阶段分别生成简洁的总结（“经验”），并明确识别任何误导性或有冲突的记忆条目，以便进行过滤或修正。

#### 信用感知记忆演化。

根据归因决策a，CHIME执行有针对性的记忆更新：
1. **记忆值更新**：如果检索到的记忆对结果有贡献（正向或负向），则使用基于置信度的加权反馈更新其值v。如果结果是积极的，相关的正面记忆的v会增加，而误导性记忆的v会降低。对于消极结果则相反。
2. **记忆库更新**：
    * **情况a=plan**：将生成的规划经验插入规划库Mplan,t+1\mathcal{M}_{\mathrm{plan},t+1}，可能合并或替换旧条目。执行库保持不变。
    * **情况a=exec**：将生成的执行经验插入执行库Mexec,t+1\mathcal{M}_{\mathrm{exec},t+1}。
    * **情况a=both**：同时更新规划库和执行库。
    * **情况a=neither**：不添加新经验（可能是环境随机性或任务定义不清），但仍根据因果分析更新检索到的记忆的v值。
这种分离确保了执行级别的噪声不会污染规划库，反之亦然，从而维护了记忆的专业性和效用。いたる紛る紛紛����紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛紛