MemHarness: Memory Is Reconstructed, Not Replayed

Hugging Face Daily Papers Papers
memory-augmented-agents llm-agents reinforcement-learning grpo reconstruction

Summary

MemHarness is a framework that enables LLM agents to reconstruct past experiences conditioned on the current context instead of replaying them verbatim, improving performance on ALFWorld and WebShop while reducing negative transfer.

Retrieving past experiences has become a common strategy to enhance large language model agents. However, most existing memory-augmented agents treat retrieved experiences as static records to be replayed verbatim, injecting them into the context regardless of whether they align with the agent's current situation. This ``replay'' paradigm ignores the gap between the abstract, general nature of stored experience and the concrete, ever-changing states encountered at decision time, frequently causing negative transfer. In contrast, humans rarely recall past experiences verbatim; instead, they reorganize and adapt retrieved memories to fit the present context. Inspired by this, we propose MemHarness, a framework that equips LLM agents to actively harness and reconstruct past experiences based on the present context. At each decision step, a unified policy model critiques and reconstructs the retrieved experience conditioned on the current state, producing context-grounded guidance before acting. This reconstructive ability emerges naturally through end-to-end training with GRPO. Experiments on ALFWorld and WebShop show that MemHarness substantially outperforms pure RL and static memory-augmented baselines, demonstrating strong robustness in out-of-distribution (OOD) scenarios. Furthermore, our analyses reveal that this reconstruction objective not only prevents negative transfer but also serves as latent guidance during training, fundamentally improving the agent's intrinsic reasoning capabilities.
Original Article
View Cached Full Text

Cached at: 07/31/26, 05:53 AM

Paper page - MemHarness: Memory Is Reconstructed, Not Replayed

Source: https://huggingface.co/papers/2607.28272 Authors:

,

,

,

,

,

,

,

,

,

,

,

Abstract

Retrievingpastexperienceshasbecomeacommonstrategytoenhancelargelanguagemodelagents.However,mostexistingmemory-augmentedagentstreatretrievedexperiencesasstaticrecordstobereplayedverbatim,injectingthemintothecontextregardlessofwhethertheyalignwiththeagent’scurrentsituation.This``replay’’paradigmignoresthegapbetweentheabstract,generalnatureofstoredexperienceandtheconcrete,ever-changingstatesencounteredatdecisiontime,frequentlycausingnegativetransfer.Incontrast,humansrarelyrecallpastexperiencesverbatim;instead,theyreorganizeandadaptretrievedmemoriestofitthepresentcontext.Inspiredbythis,weproposeMemHarness,aframeworkthatequipsLLMagentstoactivelyharnessandreconstructpastexperiencesbasedonthepresentcontext.Ateachdecisionstep,aunifiedpolicymodelcritiquesandreconstructstheretrievedexperienceconditionedonthecurrentstate,producingcontext-groundedguidancebeforeacting.Thisreconstructiveabilityemergesnaturallythroughend-to-endtrainingwithGRPO.ExperimentsonALFWorldandWebShopshowthatMemHarnesssubstantiallyoutperformspureRLandstaticmemory-augmentedbaselines,demonstratingstrongrobustnessinout-of-distribution(OOD)scenarios.Furthermore,ouranalysesrevealthatthisreconstructionobjectivenotonlypreventsnegativetransferbutalsoservesaslatentguidanceduringtraining,fundamentallyimprovingtheagent’sintrinsicreasoningcapabilities.

View arXiv pageView PDFProject pageGitHub1Add to collection

Get this paper in your agent:

hf papers read 2607\.28272

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.28272 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.28272 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.28272 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

@Xudong07452910: Agent memory is most dangerous when it trusts the past too much. Many Memory Agents stuff similar experiences directly into context after retrieval. But similar tasks do not mean the current state is the same; old experiences can sometimes steer decisions astray. This paper proposes MemHarness, turning Agent...

X AI KOLs Timeline

MemHarness proposes changing Agent memory from simple replay to reconstruction based on the current state, trained end-to-end with GRPO, significantly improving success rates on ALFWorld and WebShop.

State of Memory in Agent Harness (12 minute read)

TLDR AI

A survey of memory implementations across major AI agent harnesses (Claude Code, Codex, Copilot, etc.) reveals common boundary failures including bounded local storage, keyword retrieval, harness scoping, weak staleness handling, and 57-71% cross-user contamination rates, highlighting unsolved problems in agent memory infrastructure.

MemoryForge: Synthesize Lifelong Memory for Human-Like LLM Agents

arXiv cs.CL

This paper introduces MemoryForge, a framework for synthesizing lifelong autobiographical memory from brief target personas to enable frozen LLMs to exhibit more human-like behaviors in role-play and user-simulation, outperforming descriptive conditioning baselines.