2 Ways towards Self-Evolving Agents

(every word is generated by brain and hand-typed)

**TLDR: **Self-evolving agents can be achieved via 2 ways: Model or Harness. The model approach includes architecture and training methods. The harness approach includes memory and meta harness.

2 Ways: Model or Harness

2 Ways: Model or Harness

Model

Enabling the self-evolving capability from the foundation model is probably the ultimate approach. There are two aspects: Model Architecture and Model Training.

Architecture

The first possible architecture is State-Space Models (SSM), like Mamba series, DeltaNet series. Such models can theoretically ingest infinitely long context when it interact with the environment, and transfer it to its own experience. It does this via iteratively updating its parameters to get familiar with (continuously learn) the external knowledge.

The advantage of this approach is that it is totally model-native and can theoretically have a very high upper bound. The disadvantage of this approach is that it needs pre-training from scratch, which is expensive; and SSM typically has poor scaling law compared with transformers. When the model size grows to frontier size (trillion-parameter level), SSM is simply not working. So far, I won’t say SSM is not working, but I expect better scientific understanding of SSM scaling.

The second possible architecture is Transformer-based. It relies on either compact context encoder or fast-weight updates. The compact context encoder can compress very long context experience into much fewer tokens, so that the model can see more and achieving self-evolution via context-efficient in-context learning. The fast-weight updates approach makes a small part of parameters in the transformer updatable as it ingest new knowledge.

The advantage of this approach is that it can build on today’s success of transformer and is more realistic to achieve. The disadvantage of this approach is that it may face under-fitting issues.

Training Methods

I would say the current frontier model training is already half self-evolving (half means human-in-the-loop). For example, Anthropic has good models to power Claude Code. Anthropic engineers use Claude Code to help develop better data recipe and training code. Then this better data recipe and training code gives Anthropic better model, which gives them better Claude Code that powers the next iteration of model development.

I won’t be surprised if claude is training itself without human in the loop in the next 2 years. Other frontier labs will catch up too.

Harness

Enabling the self-evolving capability from harness is the most practical approach. You can implement in 1 hours. And it works well. This approach include Memory Harness and Meta Harness.

Memory

Harness with Memory is probably the easiest way to implement the self-evolving capability. You simply provide the model with the write memory tool and the read memory tool, and maintain the memory via a dictionary. When solving a problem, the model is able to ingest new knowledge by writing text to the memory, and retrieve it when it needs to solve a similar problem.

This approach is the most widely implemented approach. Anthropic, OpenAI and Google Gemini all implemented the memory feature in their chatbots and agents, which can provide personalized experience (because it remembers the past interactions with you).

Meta Harness

Meta harness is a type of harness that can rebuild itself. The harness allows the model to change the harness code or harness logic to adapt to the new environment. This approach is partially implemented in OpenClaw. The reliability and robustness of this approach demands better engineering science.

Ultimate Recommendation

• For GPU-rich researchers, go with Self-evolving Model Training.

• For GPU-poor researchers, go with Meta Harness.

• For startup builders, go with Memory Harness.

• For people who don’t care failure and curious to explore, go with Model Architecture.

Disclaimer. All information discussed in this article is based on publicly available and indexable sources. No confidential, proprietary, or non-public information has been used. The views expressed are solely those of the author and do not reflect the opinions, positions, or activities of any organization, employer, institution, or affiliation associated with Zengyi Qin, past or present.