EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory
Summary
EngramEdit 提出一种通过条件记忆(conditional memory)实现 LLM 解耦知识更新的方法,联合更新共享 n-gram 嵌入以实现近乎完美的编辑成功率,在多跳推理中达到最强基线近三倍的准确率,同时保留无关知识与通用能力。
View Cached Full Text
Cached at: 10/08/26, 12:13 PM
Paper page - EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory
Source: https://huggingface.co/papers/2610.10533
Abstract
ConditionalmemoryarchitecturessuchasDeepSeekEngramuseinputn-gramstolookuplearnedembeddings,expandingthecapacityoflargelanguagemodels(LLMs)withlimitedadditionalcomputation.Beyondmodelscaling,thisarchitecturehasdemonstratedthepotentialtodecouplefactualknowledgestoragefromgeneral-purposecomputation,offeringapromisingroutetoupdatingfactualknowledgewhilekeepingtheTransformerbackbonefixed.Realizingthispotentialischallengingbecausedifferentexpressionsofafactmayactivatedifferentn-gramembeddings,whileupdatingsharedembeddingscanunintentionallychangethemodel’spredictionsaboutotherfacts.WeproposeEngramEditfordecoupledknowledgeupdatesthroughconditionalmemory.EngramEditfirstcomputestargetmemoryrepresentationsthatmakethemodelpredicttheupdatedfactacrossmultipleexpressions.Itthenjointlyupdatesthesharedn-gramembeddingstomatchthesetargetsacrossexpressionsandedits,penalizingupdatestofrequentlyreusedembeddingsmorestronglytopreserveunrelatedknowledge.ExperimentsshowthatEngramEditenablesindependentfactualknowledgeupdatesthroughconditionalmemory,achievingnear-perfecteditingsuccess.Revisedknowledgeisusableacrossunseenexpressionsandinmulti-hopreasoning,withnearlythreetimesthestrongestbaseline’saccuracyunderchain-of-thought(CoT)prompting.Unrelatedknowledgeandgeneralcapabilitiesarelargelypreservedevenasfactualupdatesaccumulate.ThesefindingsshowthatEngramEditturnsconditionalmemoryintoaneditableknowledgeinterface,extendingitsrolebeyondmodelscalingtosupportdecoupledknowledgeupdates.
Similar Articles
Less Context, More Accuracy: A Bi-Temporal Memory Engine for LLM Agents Where a Lean Retrieved Context Beats the Full History
This paper introduces Engram, an open-source bi-temporal memory engine for LLM agents that retrieves a compact context slice (∼9.6k tokens) to outperform the full-history baseline (79k tokens) by 10.4 accuracy points on LongMemEval, using a hybrid read path fusing dense, lexical, graph, and temporal signals.
LadderEdit: Edit-Level Residual Compression for Memory-Efficient Lifelong Editing of LLMs
论文提出 LadderEdit,通过将每个 LoRA 编辑适配器压缩为低秩草图,并在草图不满足重写、泛化和局部性要求时沿阶梯提升秩,实现 5.2 倍的存储节省,并在 50,000 次连续编辑上保持有效。
MemCo: Memory-Centric Collaboration for Generalizing LLM Agents to Unseen Environments
MemCo提出了一种以记忆为中心的协作框架,通过维护互补的局部与全局记忆空间,帮助LLM智能体在未见过的交互环境中实现更好的泛化,实验表明其在交互式决策基准上提升了任务成功率并减少了冗余探索。
MemComposer: Learning to Assemble Memories for LLM Agents via Reinforcement Learning
本文提出 MemComposer,一个通过强化学习(基于 GRPO 的 ExPO 框架)学习为 LLM 智能体组装记忆的模型,联合考虑查询与候选记忆决定选取内容、数量与顺序,在 τ²-bench 上超越基线并降低 16.9%/4.9% 的记忆 token 开销。
$\varepsilon$-MemEvo: Adaptive Cross-Task Memory Transfer for LLM Program Evolution
The paper introduces ε-MemEvo, a framework for cross-task knowledge transfer in LLM-based program evolution, storing tactic memories as natural-language summaries and using an adaptive injection gate. It reports consistent gains on 8 optimization benchmarks with less than 1% computational overhead.