The author argues that recursive self-improvement is not yet a true loop because most learned improvements are discarded, positioning retention as the binding constraint alongside discrimination and promotion. The post discloses MemT 1.0, a 293B model attempting memory state inside the model's own computation, releasing October 20th.
Opening Recursive self improvement is usually described as a loop. I do not think it is a loop yet, because most of what a system improves at gets discarded before the next iteration. The loop needs three separate things Strip it down and there are three functions that have to work independently: Something retains the improvement. Something decides which changes were actually improvements. Something promotes the verified changes into the next attempt. Retention, discrimination, promotion. The second one is where most of the engineering effort goes today and it works reasonably well. The third one mostly means a human edits a file. The first one we largely skip. Why retention is the binding constraint If nothing carries forward, every iteration starts from the same point. That is repetition, not recursion. You can run a million tasks and the millionth is no better than the first. And retention is not a longer context. A context window is supplied input. It is not something the model has, it is text you hand it again every time. That distinction matters, because it decides whether the ability is inside the capability or outside it. But retention alone makes things worse Keep everything and you keep the mistakes too. A system that retains without discriminating does not become more capable, it becomes more confidently wrong, faster. So the claim is not that memory solves recursive self improvement. It is narrower: without retention, discrimination and promotion have nothing to operate on. So where should it live Four candidates, each with a real cost: In the weights. Durable and always on. But every update is training, and attributing a behavior back to a change is hard. In an external store. retrieval, or notes files. Cheap, inspectable, deletable. But it is input, not capability, and it is not optimized together with the model. In the scaffolding around the model. This is where a large share of measured gains come from today, and those gains are real. But the harness holds the memory, not the model. Switch harnesses and you start over. In a state inside the model's own computation. Updates without a training run. But you now own state lifetime, isolation, versioning, and drift. These are not mutually exclusive, and most current progress is the third. What I am unsure about is whether the fourth is a genuinely different axis, or just a more convenient interface over the third. What would change my mind If a system with no persistent state, no fine tuning and no human maintained notes file can show monotonic improvement across repeated runs on the same task family, the retention argument is wrong. I would like someone to show me that, and I do not think it exists yet. The part I think gets skipped Retention is also a safety surface, and it cuts both ways. It makes behavior auditable — you can look at what a system kept and why. It also creates failure modes that are hard to reason about: memory that is poisoned, memory that is wrong and gets promoted anyway, and drift that nobody can reproduce because no two users hold the same state. Retention makes the recursive loop possible and makes the loop harder to inspect at the same time. What we are working on Disclosure: this is the question behind MemT 1.0, which we are releasing on 20th oct. It is an attempt at the fourth option: memory state that lives inside the model's computation rather than in a store beside it, at 293B. Weights are not public yet. I am not claiming the question is settled — we are testing one answer to it and would rather argue about the question than the release. Question Where do you think retention should live: in the weights, in the scaffolding around the model, or in a state the model updates itself — and what test would tell you it is actually doing the work?
The article questions whether a system with persistent memory but fixed model weights, like AQuA, qualifies as recursive self-improvement, referencing a paper that uses a narrower definition.
A comprehensive survey of 1,250 papers (2024–2026) on recursive self-improvement in AI, proposing a taxonomy distinguishing bounded self-refinement from open-ended recursive self-improvement, and analyzing the evaluator design space and failure modes.
Researchers present AIDE², a system with recursive auto-research loops that improved its own code over 100 iterations, discovering seven improvements and beating a hand-tuned agent on held-out benchmarks.
The paper introduces a model for recursive AI self-improvement, defining a recursive reproduction number to determine when incremental improvements in AI research become self-amplifying or dampening across development cycles.
The article examines the feasibility of recursive self-improvement in AI given computational constraints and criticizes AI companies for prioritizing profit over societal benefits.