PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents

Hugging Face Daily Papers Papers
recursive-self-improvement benchmarking personal-agents memory evaluation agent-frameworks

Summary

Introduces PAST-Bench, a benchmark for evaluating whether personal AI agents improve from retained experience across sessions, and Hermes+, an extension with targeted interventions. Finds improvement is real but uneven across capabilities and models.

Recursive self-improvement requires agents to turn accumulated experience into better future behavior. Personal AI agents offer a concrete setting for studying this capability because they retain preferences, task histories, tool routines, and learned skills across sessions. Yet whether retained experience actually improves them over time has not been systematically tested. We introduce PAST-Bench, a benchmark designed to isolate this question. Each agent runs through ordered sequences of fresh-session tasks under matched conditions that turn retained experience on and off. It spans 26 scenarios and 204 episodes across memory, procedural reuse, information gathering, and update. We report both later-task gains and whether those gains follow the intended save, retrieve, and update pathway. Across seven base models and four agent frameworks, improvement is real but uneven across capabilities. Agents with the same headline gain can differ markedly in whether that gain is supported by evidence of the intended pathway. Guided by these findings, we develop Hermes+, which extends Hermes with five targeted interventions across stages of the agent loop. Hermes+ raises the average gain from retained experience and provides clearer pathway evidence, with its strongest improvement on tasks requiring outdated state to be replaced, although the effect remains capability- and model-dependent. Together, PAST-Bench and Hermes+ provide an evaluation and diagnostic foundation for studying how persistent agents can progress from retaining experience to systematically improving through it. Code: https://github.com/Gen-Verse/PAST-Bench
Original Article
View Cached Full Text

Cached at: 08/05/26, 05:43 AM

Paper page - PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents

Source: https://huggingface.co/papers/2608.04003

Abstract

Recursiveself-improvementrequiresagentstoturnaccumulatedexperienceintobetterfuturebehavior.PersonalAIagentsofferaconcretesettingforstudyingthiscapabilitybecausetheyretainpreferences,taskhistories,toolroutines,andlearnedskillsacrosssessions.Yetwhetherretainedexperienceactuallyimprovesthemovertimehasnotbeensystematicallytested.WeintroducePAST-Bench,abenchmarkdesignedtoisolatethisquestion.Eachagentrunsthroughorderedsequencesoffresh-sessiontasksundermatchedconditionsthatturnretainedexperienceonandoff.Itspans26scenariosand204episodesacrossmemory,proceduralreuse,informationgathering,andupdate.Wereportbothlater-taskgainsandwhetherthosegainsfollowtheintendedsave,retrieve,andupdatepathway.Acrosssevenbasemodelsandfouragentframeworks,improvementisrealbutunevenacrosscapabilities.Agentswiththesameheadlinegaincandiffermarkedlyinwhetherthatgainissupportedbyevidenceoftheintendedpathway.Guidedbythesefindings,wedevelopHermes+,whichextendsHermeswithfivetargetedinterventionsacrossstagesoftheagentloop.Hermes+raisestheaveragegainfromretainedexperienceandprovidesclearerpathwayevidence,withitsstrongestimprovementontasksrequiringoutdatedstatetobereplaced,althoughtheeffectremainscapability-andmodel-dependent.Together,PAST-BenchandHermes+provideanevaluationanddiagnosticfoundationforstudyinghowpersistentagentscanprogressfromretainingexperiencetosystematicallyimprovingthroughit.Code:https://github.com/Gen-Verse/PAST-Bench

View arXiv pageView PDFGitHub2Add to collection

Get this paper in your agent:

hf papers read 2608\.04003

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.04003 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.04003 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.04003 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles