Just in time for ICML week, we are sharing our take on a key question for recursive self-improving AI. How can AI keep improving if the evaluator stays fixed?

Our answer is the Red Queen Gödel Machine (RQGM), developed as a collaboration between the @CaMLSys, @nvidia, @flwrlabs, @mbzuai and @Inria, and led by @Alex__Iacob (@Cambridge_Uni). It is still early, but the potential is high. One headline result is that, in a few narrow cases, we show an agent using largely Nemotron 3 Ultra can reach similar performance to GPT 5.5.

Self-improvement is one of the more plausible bets for the next major breakthrough in AI. But there is a core issue limiting a lot of this work. The evaluator. Current approaches often assume something like a benchmark or reward model that sits outside the self-improvement loop. That works for a while. But evaluators saturate, they get gamed, and as the system improves, they can stop being useful.

Our Red Queen Gödel Machine lets the agent and evaluator improve together. The agent gets better at doing the task. The evaluator gets better at judging it. On coding, it already beats the prior best search method for self-improving AI while using 1.35–1.72× fewer tokens, and the same idea carries over to paper writing and proof grading.

Paper: https://arxiv.org/abs/2606.26294

Full RQGM team: @Alex__Iacob, @itsmaddox_j, @williamfshen, @DBBurkhardt, @meghdadkurmanji, Nurbek Tastan, @lorenzosani97, @NickVenanzi, @AmbroiseOdonnat, Jamie Cao, Bill Marino, @xinchiqiu, and @niclane7.

The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators

Source: https://arxiv.org/abs/2606.26294 Authors:Alex Iacob,Andrej Jovanović,William F. Shen,Daniel Burkhardt,Meghdad Kurmanji,Nurbek Tastan,Lorenzo Sani,Niccolò Alberto Elia Venanzi,Ambroise Odonnat,Zeyu Cao,Bill Marino,Xinchi Qiu,Nicholas D. Lane

View PDF

Abstract:Self-improving agents are state-of-the-art (SOTA) on agentic coding benchmarks and have recently been extended to general domains. However, their search methods generally assume a stationary evaluation criterion: a fixed verifier, benchmark, or labeled dataset that remains valid as the agent improves. This ignores a central feature of evolution: species adapt as their environments change with them. We aim to bring the same principle to recursive self-improvement, making evaluation part of the improvement loop and opening search to evolving evaluators, adversarial objectives, and dynamic utilities that may surpass static benchmarks. We introduce the Red Queen Godel Machine (RQGM), an evolutionary framework for recursive self-improvement under non-stationary utilities. The RQGM makes this possible through controlled utility evolution: search is organized into epochs with a fixed within-epoch evaluation criterion, while the utility can be updated at epoch boundaries, so self-improvement guarantees hold per epoch as the objective evolves across them. We begin by showing that even on verifiable coding tasks, the RQGM improves test pass rate over the prior SOTA by adding a complementary agent-as-a-judge code-review signal. This signal is cheaper and the RQGM uses 1.35x-1.72x fewer tokens. We then turn to scientific paper writing and reviewing, and Olympiad-level proof writing and grading, where the RQGM improves performance over prior self-improving agents: co-evolved writers reach 1.78x-1.86x higher acceptance rates under a diverse agent-as-a-judge panel, while co-evolved graders reach 9% higher ground-truth accuracy. In paper reviewing, the strongest baseline reviewer over-accepts AI-generated papers at up to 1.91x the human rate. The RQGM corrects this by introducing an adversarial objective that discovers reviewers equally stringent on AI and human work.

Submission history

From: Alex Iacob [view email] **[v1]**Wed, 24 Jun 2026 18:38:26 UTC (1,058 KB) **[v2]**Mon, 29 Jun 2026 17:13:25 UTC (1,116 KB)