I made a superhuman Generals.io agent with self-play RL [P]

Reddit r/MachineLearning 06/24/26, 04:18 PM Models

self-play reinforcement-learning jax vision-transformer game-ai open-source generals-io

Summary

Trained a superhuman Generals.io agent using self-play reinforcement learning with a JAX-based pipeline and Vision Transformer. Achieved #1 on human 1v1 leaderboard; all code and a fast JAX simulator open-sourced.

Hi everyone, I trained a self-play RL agent for Generals.io that reached superhuman-level and ranked #1 on the human 1v1 leaderboard. It began as my master's thesis where the goal was to beat a prior algorithm based agent. We succeeded using behavior cloning, RL fine-tuning and reward shaping, but the agent was still consistently beaten by the top players. So I gave it a round two and fixed the largest bottlenecks: Reimplemented the whole pipeline in JAX (from NumPy/Torch) Used Vision Transformer instead of the CNN Both are a result of the same idea: to invest in scaling rather than human priors and ad-hoc patches. The blog is written as a guide for anyone building something similar — the dead ends, the decisions, and the intuitions and tricks I picked up along the way. It's all open source, including the fast JAX simulator — handy on its own if you want an imperfect-information RTS env to play with. Links - Guide: https://kam.mff.cuni.cz/~straka/blog/generals.html - Simulator (JAX): https://github.com/strakam/generals-bots - Agent: https://github.com/strakam/AverageJoe I hope you find the blogpost entertaining! Feedback and questions welcome 🤗.

Original Article

Similar Articles

@reach_vb: https://x.com/reach_vb/status/2057880274348695995

X AI KOLs Following

A user demonstrates using OpenAI's Codex to automatically generate a Colab notebook that trains a ~10 million parameter transformer in JAX/Flax/Optax on addition, achieving high accuracy after 4000 steps on a T4 GPU.

Show HN: I RL-trained an agent that trains models with RL (for –$1.3k)

Hacker News Top

A developer built a pipeline where an RL-trained AI agent creates and submits RL training jobs for small models, rewarding the agent for better performance. The project is fully open-sourced and demonstrates transfer to held-out tasks.

Self-play helped AI achieve superhuman performance in Go, so why hasn’t it done the same for LLMs? Researchers have found a solution.

Reddit r/singularity

Researchers introduce Self-Guided Self-Play (SGS), a self-play algorithm for LLMs that prevents reward hacking by using a Guide role to score synthetic problems. Applied to theorem proving in Lean4, SGS surpasses RL baselines and allows a 7B model to outperform a 671B model.

@dair_ai: // Self-play with a pinch of human data // Really cool paper combining human demonstrations and self-play RL. 30 minute…

X AI KOLs Following

A research paper that combines a small amount of human demonstrations as a regularization objective with self-play reinforcement learning, enabling human-compatible driving policies using far less human data (30 minutes vs thousands of hours) and training in 15 hours on a single consumer GPU.

I built a custom multi-agent framework (GenOS) to autonomously evolve algorithms. I pitted the 3 fundamental AI paradigms against an NP-Hard problem. Here is what happened.

Reddit r/AI_Agents

The article describes GenOS, a custom multi-agent framework that autonomously evolved Rust algorithms to solve the NP-Hard Reverse Game of Life problem, discovering three optimization paradigms and proving a mathematical limit of 378/400.