@omarsar0: Brilliant paper on scaling research agents from UC Berkeley. (bookmark it) They propose an unexpected and interesting w…
Summary
UC Berkeley researchers show that scaling LLM research agents stalls when agents anchor on their own history, and propose periodically discarding session state and restarting from the best candidate — a 'forgetting' approach that yields up to 15% gains over parallel agents and 39% over a single agent at the same budget.
View Cached Full Text
Cached at: 10/16/26, 08:46 PM
Brilliant paper on scaling research agents from UC Berkeley.
(bookmark it)
They propose an unexpected and interesting way to scale automated research with agents.
It’s common to see research agents keep their full history and iterate on it to solve a task. After a while, each agent anchors on its own past attempts, makes smaller edits, and extra budget stops improving things.
So this work induces forgetting in the agents.
Periodically, they discard each agent’s session and the candidate history, and then restart from the best candidate found so far.
Across heuristic programming, kernel optimization, and model training tasks, this gives up to 15% over parallel agents and up to 39% over a single agent with the same budget.
https://arxiv.org/abs/2610.20563
Chat with Paper: https://academy.dair.ai/papers/scaling-discovery-requires-forgetting-2610.20563…
Scaling Discovery Requires Forgetting
Source: https://arxiv.org/abs/2610.20563 Authors:Shangyin Tan,Ben Zhou,Lakshya A Agrawal,Jialin Zhang,Yilin Chen,Qiuyang Mang,Donghyun Lee,Dennis Jacob,Koushik Sen,Alex Dimakis,Matei Zaharia
Abstract:Automated research with LLM agents has shown promising results in various domains. Through iterative refinement and scored verifiers, LLM agents are capable of discovering new ideas. There are different ways to scale automated research: let one agent iterate for longer, run several agents in parallel, or further let the parallel agents communicate and build on one another’s candidates. However, we find that all three strategies eventually plateau as the agents accumulate history. Each agent becomes anchored to its own past attempts, making fewer and smaller edits. Beyond this point, additional budget brings little further improvement. In this paper, we show that the key to scaling discovery further is to forget. Periodically discarding the search state, including every agent’s session and the candidate history, and restarting from the best candidate found so far, unblocks the search and pushes the same budget further. We implement forgetting in a team of agents with communication and evaluate it on four tasks spanning heuristic programming, kernel optimization and model training. The results show up to 15% improvement compared to parallel agents, and up to 39% compared to a single agent. Our results suggest that, for open-ended discovery, what an agent is allowed to remember matters as much as how much compute it is given.
Submission history
From: Shangyin Tan [view email] **[v1]**Wed, 14 Oct 2026 17:49:37 UTC (114 KB)
Similar Articles
@omarsar0: // The Memory Curse in LLM Agents // (bookmark it) Long histories apparently degrades agents as they become increasingl…
This research paper identifies the 'memory curse' in LLM agents, demonstrating that expanded context windows systematically degrade cooperative behavior in multi-agent social dilemmas by eroding forward-looking intent. The authors show that targeted fine-tuning, synthetic memory sanitization, and reducing explicit Chain-of-Thought reasoning can effectively mitigate this behavioral decay.
@omarsar0: New research from Meta. (bookmark it) It's on how to fix agents that forget previously made decisions. It's well know t…
Meta researchers propose a proactive memory agent that runs alongside an unmodified action agent to surface relevant past decisions and prevent forgetting in long-horizon tasks. Their method improves pass@1 on Terminal-Bench 2.0 and tau-squared-Bench by 8.3 and 6.8 percentage points respectively.
@omarsar0: Banger paper from NVIDIA on shared memory for research agents. (bookmark it) If you run several coding agents on the sa…
Agora is a Git-based shared memory system for collective auto-research that allows multiple AI agents to collaborate without duplication, demonstrated by initializing a model without training data using LLM workers.
@omarsar0: Impressive work from Alibaba. (bookmark it) If you build long-running agents and keep rewriting your memory schema, tak…
The article introduces Scroll, a context management method for long-running AI agents that treats context as a programming task using an append-only event log and persistent Python kernel, achieving state-of-the-art results on benchmarks like LongMemEval_S and LOCA_256K.
@dair_ai: Good work on improving memory for long-horizon agents. They separate two things that agent memory papers usually collap…
The paper introduces RSM-full, an online clustered-memory pipeline for LLM agents that separates memory merge and retrieval assembly, achieving 83% of full-context quality at 32% of token cost under tight prompt budgets.