Beating Warcraft III’s insane AI with self-play reinforcement learning on one desktop [P]

Reddit r/MachineLearning Papers

Summary

An agent trained via self-play reinforcement learning on a single desktop (RTX 3090) achieves an 89% win rate against Warcraft III's hardest built-in AI, running the real unmodified game under Wine with a custom shim and harness, in 840,000 games over three weeks.

No content available
Original Article
View Cached Full Text

Cached at: 10/18/26, 08:17 PM

# Beating Warcraft III's insane AI with self-play reinforcement learning on one desktop Source: [https://warlock.ai/blog/warcraftsim-1/](https://warlock.ai/blog/warcraftsim-1/) In three weeks, an agent that plays the real Warcraft III went from losing to the easy AI to winning about nine games in ten against the hardest one, insane, trained by self\-play on one desktop\. Reinforcement learningSelf\-playBehavioral cloningCurriculum learningReal\-time strategyGame AIWarcraft III [github\.com/RobinKa/warcraftsim](https://github.com/RobinKa/warcraftsim) The agent \(player A\) playing the built\-in AI on insane, in real time\. The panel on the right shows how likely the agent thinks it is to win, its reward, and each side’s army and resources\.**89%**win rate against the insane built\-in AI **13 ms**per step, small enough to play on an ordinary CPU **11 years**of Warcraft III played in training, in 840,000 games **1 GPU**trained on an RTX 3090 and 16 CPU cores [Warcraft III](https://en.wikipedia.org/wiki/Warcraft_III:_Reign_of_Chaos)is a real\-time strategy game from 2002\. You gather gold and lumber, build a base, research technology, raise heroes and an army, and destroy your opponent while they do the same\. More than twenty years on, people still play it competitively, and streamers such as former pro player[Grubby](https://www.twitch.tv/grubby)and the casters of[Back2Warcraft](https://www.twitch.tv/back2warcraft)bring it to large audiences\. This post is about an agent that plays the whole game of Warcraft III on the classic ladder map[Echo Isles](https://liquipedia.net/warcraft/Echo_Isles), under the game’s own rules, against the strongest opponent the game ships with, the built\-in AI on*insane*, the hardest of its levels \(easy, normal and insane\)\. It now wins about nine games in ten\. Here is how it got there\. ## Why this is hard Most games last 10 to 25 minutes, with thousands of decisions for dozens of units at once\. There are four races, human, orc, undead and night elf, each with its own units, buildings and spells, and a fog of war hides the opponent\. The one real reward, a win or a loss, comes at the very end\. DeepMind’s[AlphaStar](https://doi.org/10.1038/s41586-019-1724-z)\(StarCraft II\) and[OpenAI Five](https://arxiv.org/abs/1912.06680)\(Dota 2\) mastered games like this in 2019 and beat top human players, with the game makers’ help and a datacenter\. AlphaStar trained 12 agents for 44 days on 32 TPUs each\. I had one desktop and a game that was never built for any of this\. ## Running the real game, fast Warcraft III has no API for bots, no headless mode and no faster\-than\-real\-time option\. It is a 32\-bit Windows program, and its rules, pathfinding, spells and AI hold so many details that reimplementing them faithfully would be very hard\. So I run the real thing, under[Wine](https://www.winehq.org/)on Linux, about 50 copies at once, and drive it from Python\. **Python**the learner, the inference loop, ~50 games at once one Wine process per game **w3shim\.dll**a virtual clock; stops the game at every step until Python answers **Warcraft III 1\.29**the unmodified game **The harness**JASS in the map that reports what the player sees and carries out its orders Three pieces let Python play the game\.**The harness**, written in the game’s own map\-scripting language, reports what the player can see every half second of game time and carries out the agent’s orders\.**The shim**, a small DLL inside the game process, replaces the game’s clocks with a virtual one, so the game runs as fast as the CPU allows and waits at every step for Python’s orders\. Games are deterministic, so a replay reproduces a game step for step\. That is how every video here was made\. Self\-play speed, agent steps per second An agent step is one decision by one player, made every half second of game time\. Each bar adds one optimization to the ones above it\. the starting pointunits read in C, not in JASSfive games per map loadthe learner as CUDA graphsgames draw nothingWine with fsyncan inference loop in C\+\+native Linux, not WSLthe game’s code, faster5607198221,0001,1101,2801,4901,8502,280 Most of the speed came from removing waste, not from adding hardware\. The games skip drawing, five games share one map load, the per\-unit bookkeeping moved from the game’s script into C, and the network runs as CUDA graphs from a C\+\+ loop\. Today the desktop plays about 50 games at once, each about 15 times faster than real time, which adds up to roughly**730 hours of Warcraft III every hour**\. ## The agent WHAT IT SEESTHE COREWHAT IT DECIDESUnits in viewup to 160 tokens: type, position,health, current orderThe playergold, lumber, supply, time,races, upgradesTarget strategyoptional: a build order, as 128 numbersThe whole gamehidden from the player: value onlyTransformer5 layers · every token attends to allplayer tokenunitunitunit…out: a token per unit,and the player tokenMemorya state carried from step to step\(minGRU\)next stepcontextOrder headeach own unit: one of 399 orders, or nonemasked: its type, what it can affordTarget unita pointer to atoken in viewTarget pointx, then y given x352 bins a sideorder →Commands for every own unitValue headpredict game outcome: for training only Swipe sideways to see the whole diagram → One network plays one side\. Hover over or tap a part to see what it does\.The agent sees only what its player sees, and every half second it decides for every one of its units at once\. The network has four main parts: - **The order head**picks each unit’s order, such as train, build, attack, move, cast or nothing\. - **The target heads**pick what that order is aimed at, either a unit in view or a point on the map\. A point is picked as x and then y, so both coordinates belong to the same spot\. - **The memory**carries a state from step to step, so the agent can remember what it saw before it went out of view\. - **The value head**predicts how the game will end\. It reads hidden information and is only used in training\. The whole network is small, about 3\.2 million weights\. One decision for 60 units, with 120 in view, takes about 13 milliseconds on four cores of an ordinary CPU, far less than the half second it has\. **A head start from the built\-in AI\.**Learning by playing from scratch is too hard: an agent that gives random orders almost never wins, so it has nothing to learn from\. To give it a good start, I recorded thousands of games of the built\-in AI playing itself and trained the agent to copy the orders the AI gave\. This is called[behavioral cloning](https://en.wikipedia.org/wiki/Imitation_learning), and it leaves the agent playing roughly like the AI\. **Learning from playing\.**Once the copying stops improving, the agent switches to self\-play\. It plays against a*league*of opponents \(itself, its past selves and the built\-in AI\) and learns from the reward\. The reward is \+1 for a win and −1 for a loss, plus small rewards along the way for gaining material \(more on those later\)\. After each batch of games, every order the agent gave is judged by the reward that followed it, using[policy gradients](https://spinningup.openai.com/en/latest/spinningup/rl_intro3.html)\. If the reward turned out better than the value head expected at that moment, the order becomes a little more likely in that situation; if worse, a little less\. The gap between the reward and what was expected is called the*advantage*\. **Keeping it steady\.**A second term, a*KL divergence*, pulls the agent back towards an earlier version of itself, the*anchor*, so one noisy batch cannot undo what already works\. **Play**thousands of games against the league **Judge each order**what followed against what the value head expected \(the advantage\) **Update**better than expected becomes more likely, worse less likely **KL to the anchor**a pull back towards an earlier version of itself One round of learning from play\. Training repeats it every few seconds on the newest games\.## Measuring progress Training numbers can’t tell whether a change made the agent better\. The reward includes the small material rewards, not just wins and losses, and the training opponents change all the time, from past versions of the agent to a built\-in AI that is made easier or harder as the agent improves\. So every training run also plays*test games*that it never learns from\. They are games against the built\-in AI at the hardest level each stage aims at, without any tax or other help for the agent\. On the real map that is insane\. Each experiment is a new run that starts from the best agent so far, with one change\. The new version and the one it started from play exactly the same matchups, with the same races and sides, so their win rates can be compared directly\. A few hundred games each are enough to see clear differences\. Every experiment, with its numbers, is in the[experiment log](https://github.com/RobinKa/warcraftsim/blob/master/docs/reference/experiment-log.md)\. ![Matchup tables: win rates by the agent's race and its opponent's](https://warlock.ai/blog/warcraftsim-1/media/matchups.png)The dashboard’s matchup tables show the current best agent \(left\) and the version before it \(right\) against the insane AI, race by race, over the same 1,500 test games each\.## How it learned **The problem\.**A real game often lasts 25 minutes, and the agent decides twice a second, for all of its units at once\. That is 25 × 60 × 2, about 3,000 decisions per side before the one reward, a win or a loss, arrives\. With a single result at the end it is hard to tell which of those decisions mattered, and an agent that loses every game gets nothing to learn from\. Even an agent trained by behavioral cloning on the built\-in AI’s games on the real map lost all 40 of its test games\. **A curriculum\.**So the training follows a curriculum: the agent starts with easier problems and moves on to harder ones as it gets better\. It began with small fights, then whole games on a small map with sped\-up rules, then longer games, then the real map, first with fast rules and then with the game’s own\. At each stage the test games showed a problem, experiments went after it, and most of them failed\. These are the ones that worked, in the order they came\. **Micro fights**a few units a side, seconds long · 91% against a scripted opponent **2\-minute games**a small duel map, everything 7× faster · 79% against the built\-in AI **6\-minute games**half costs, production 3× faster · 80% against insane **Echo Isles, fast rules**the real map · ~55–60% against insane **Echo Isles, real rules**the real game · 49% at once, now 89% The curriculum\. The two small maps each started again from behavioral cloning of the built\-in AI; the real map started from the six\-minute map’s best agent\.The first stage was small fights: a hero and a few units on each side, over in seconds\. This is where the basics were worked out, such as units as tokens, targets as pointers, starting from behavioral cloning of a scripted player, and a league of past selves\. A micro fight at 1\.5× speed between a league\-trained agent \(A, green rings\) and its predecessor \(B, orange rings\), with a Lich and four units a side\. A focuses B’s Lich first and casts Frost Nova and Frost Armor until its mana runs out\. Over 120 games it won 81, lost 31 and drew 8\. The panel shows each unit’s action probabilities\.### 27 September – 2 OctoberTwo\-minute games The first whole games were played on a small flat*duel map*with two bases laid out like Echo Isles’ main bases, with the rules sped up seven times, so a game took two minutes\. The agent began with behavioral cloning of the built\-in AI’s orders\. On its first day of self\-play it lost game after game, even against the*easy*AI\. 28 September #### A fair opponent **What was wrong\.**The first self\-play run lost all of its last 75 games against the AI, so it had nothing to learn from\. Two ways of making the AI easier failed, because the agent exploited both\. With extra hit points for its units, it never learned to spend its money\. Against an AI that started late, it simply rushed the AI before it could do anything\. **27 September\.**On its first day of self\-play, the agent cloned from the built\-in AI \(orc, blue\) loses to the*easy*AI in 84 seconds\. 3× speed\.**The change\.**A tax on the AI\. It plays normally, but loses a share of the gold and lumber it gathers\. The share grows after each of the agent’s losses and shrinks after each win, so the agent wins about half of those games\. The test games never have a tax\. **The outcome\.**As the agent improved, the tax fell from about 75% to almost zero, averaged over all matchups\. By 2 October it won 79% of its test games against the AI\. **2 October\.**The agent \(night elf, red\) beats the normal AI \(orc, blue\) in two minutes\. 4× speed\. ### 2 – 14 OctoberSix\-minute games, against insane The next stage had slower rules and a harder opponent, the built\-in AI on insane\. Against the untaxed AI, games took about six minutes\. The agent again started from behavioral cloning, this time of the AI’s games under these rules\. The tax kept the training games balanced, but they didn’t teach much\. Most of the taxed training games dragged on and ended in a tie at the 20\-minute time limit\. The agent built more than the taxed AI, but sat on about 4,000 gold and lumber it never spent, and hardly attacked\. The tax also hid how slowly the agent started\. One minute in, the taxed AI had 7 supply against the agent’s 15, but the untaxed AI had 21 and won the test games in five to seven minutes\. 3 October #### On\-policy distillation from the built\-in AI **What was wrong\.**After hours of self\-play the agent had won only 1 of 98 test games\. Behavioral cloning only shows the agent situations that the built\-in AI itself gets into\. The built\-in AI always keeps its buildings busy, so its games almost never show an idle barracks with money to spend, and the agent never learned what to do there\. **3 October\.**The agent \(undead, blue\) loses its base to an insane AI \(night elf, red\) that even pays a 54% tax\. The last two minutes at 4×\.**The change\.**Let the built\-in AI coach the agent in half of its training games\. The built\-in AI watches the agent’s game\. In every situation the agent gets into, the built\-in AI notes the order it would give, and training nudges the agent’s orders towards those notes\. This is[on\-policy distillation](https://arxiv.org/abs/2306.13649)\. the agent’s gameagent’s ordercarried outbuilt\-in AI’s orderonly a labelstate 1state 2state 3state 4train a workerattackbuild a farmattack=≠=≠train a workerbuild a farmbuild a farmtrain a footmantraining: reward for winning, plus a loss that moves the agent’s orders towards the built\-in AI’s Swipe sideways to see the whole diagram → On\-policy distillation in one of the agent’s games\. Every half second there are two orders for each unit\. The agent’s order is carried out\. The order the built\-in AI would give is only recorded\. Where the two differ, training nudges the agent towards the built\-in AI’s choice\.**The outcome\.**Within fifty minutes of training the agent went from 1 win in 98 test games to 13 in 50, and the training games stopped ending in ties\. Over the next week and a half, most experiments made no difference\. Two did\. 13 October #### Credit that skips mistakes **What was wrong\.**A game’s result is spread back over every action before it, so a good move followed by a lost fight was blamed for the fight\. **The change\.**Switch to AlphaStar’s UPGO\. With it, a move gets credit from what actually happened next only until the first later move that went worse than expected\. After that point, it is judged by what the value head expected instead\. **The outcome\.**Over two runs, this was the largest gain on the six\-minute map\. 14 October #### Stop on\-policy distillation **What was wrong\.**The agent now won about 60% of its games against insane, so it had become better than the built\-in AI coaching it\. Yet copying the built\-in AI’s orders still made up most of each training update\. **The change\.**Stop on\-policy distillation\. **The outcome\.**It won more often against insane, and training ran faster without the built\-in AI coaching in the games\. **14 October\.**The agent \(orc, blue\) breaks the insane AI \(human, red\) at full strength\. 4× speed\. ### 15 – 17 OctoberThe real map By 14 October the agent won 80% of its games against insane on the six\-minute map, so it moved on to the real map, Echo Isles\. The real map is far bigger and busier than the duel map, and a few things had to change for it: - **Where things are\.**Positions are now measured from the agent’s own base, at the scale it knew from the duel map, and the stretch between the bases is squeezed so that the enemy base is where it expects it\. Without that, its buildings landed in the trees and its attacks went to the middle of the map, where the enemy base used to be\. - **Where it can point\.**The grid of map points it can target grew from 128 to 352 cells a side, to cover the bigger map\. - **The rules, in two steps\.**Training first used Echo Isles with the fast rules of the six\-minute games, then the game’s own rules\. The duel map48 × 48 tiles, flatEcho Isles116 × 84 tiles: islands, trees, cliffs, shallow water4,600 apartone mine a base, 4 small camps,no items, shops or expansionsTGM$$9,856 apartthe agent’s basethe enemy’sgold minecreep camp: neutral monsters that guard itemstavern, goblin merchant, marketplace, mercenaries Swipe sideways to see the whole diagram → Both maps at the same scale, Echo Isles drawn from its own minimap\. Near its own base the agent sees Echo Isles at the duel map’s scale, and the distance between the bases is squeezed, so the enemy base appears where the agent expects it\.The agent was trained on Echo Isles with the fast rules only\. Tested under the game’s real rules, which it had never played, it won 49% straight away\. From there it kept improving: Games won against the insane AI on Echo Isles Each bar is the best agent after one change, measured in test games\. The outcomes below use the same numbers\. 100%75%50%25%049%59%62%72%82%89%first real gameslook furtherfind roomhalve materiala looser anchorgive it a planaheadto buildrewards 15 October #### Look further ahead **What was wrong\.**Real games often last 25 minutes or more, against six on the small map, but the agent only looked about eight minutes ahead\. Anything that paid off later hardly counted\. **15 October, the first day under the real rules\.**The agent \(orc, blue\) loses a 22\-minute game to an insane AI \(undead, red\) that pays a 27% tax, while the agent sits on 9,600 gold and 3,500 lumber it never spent\. The last two minutes at 4×\.**The change\.**The agent values a reward less the further in the future it comes\. At every step, half a second, a future reward is multiplied by a number just below 1, the*discount*\. With a discount of 0\.999, a win eight minutes away \(about 1,000 steps\) counts only about a third as much as a win right now, since 0\.9991000≈ 0\.37\. Raising the discount to 0\.9997 stretches that to about 28 minutes, so the agent also cares about what happens late in a long game\. The chart shows how much a win counts, depending on how far ahead it is\. **The outcome\.**Its win rate rose from 49% to 59%, and the games got shorter\. Half of the test games were over within 13 minutes, against 18 before, and fewer reached the 40\-minute limit\. How long the test games against insane lasted, before and after the change\. 16 October #### Find room to build **What was wrong\.**Building spots learned on the flat duel map often landed on Echo Isles’ trees, cliffs or blight \(the undead build only on their own corrupted ground\), and the game refused three of four undead buildings\. **The change\.**Make the harness search three times farther for a free spot whenever the one the agent asked for is blocked\. **The outcome\.**Once the agent had learned to use it, its win rate rose from 59% to 62%\. 16 October #### Halve the material rewards **What was wrong\.**Winning or losing is the only reward that really matters, but it comes only at the end of a long game\. To give the agent feedback sooner, it also gets small rewards for*material*, the value of its units and buildings compared with the opponent’s\. Rewards like this have to be designed carefully\. If the agent were rewarded for every unit it trains, it could train a footman, let it die and train another, and collect reward forever without getting any closer to a win\. So the material reward is paid only for changes in the material lead \(this is called[potential\-based shaping](https://people.eecs.berkeley.edu/~pabbeel/cs287-fa09/readings/NgHaradaRussell-shaping-ICML1999.pdf)\)\. Training a footman worth 135 gold adds 135 to the lead, and losing it takes the 135 away again\. Going round that loop earns nothing, so the only way to gain is to come out ahead, and the best way to play stays the same\. Even so, the material rewards could pull the agent towards trading armies instead of winning\. And now that the agent looked further ahead, the win itself already counted from early in the game, so it needed less help\. **The change\.**Halve the weight of the material rewards\. **The outcome\.**Its win rate rose from 62% to 72%, most of all for the weakest races\. 17 October #### A looser anchor **What was wrong\.**Three experiments in a row made no difference\. Two forces train the agent\. The reward pushes it towards better play, and a KL term pulls it back towards an earlier version of itself, the anchor, for stability\. Measured per update, the push was about 1% signal and 99% noise, and the pull was clean\. **The change\.**Cut the pull to a quarter\. **The outcome\.**No difference for the first ninety minutes of training, then it pulled ahead, from 72% to 82%\. 17 October #### Give it a plan **What was wrong\.**The agent played one strategy, an early rush, and its losses were long games in which it never moved up to stronger units\. **The change\.**Give the agent a plan, as in[AlphaStar](https://doi.org/10.1038/s41586-019-1724-z)\. Half of its training games now come with a target build order: which buildings, units, heroes and upgrades to make, and when\. Each one is the build order of one player in a real game, one of 8,072 taken from the built\-in AI’s own games and from replays of games between people\. The target is an input to the network, and the agent gets a small reward for following it\. **The outcome\.**Its win rate rose from 82% to 89%, even though it follows the targets only loosely\. **18 October\.**The agent \(night elf, red\) destroys the base of the insane AI \(human, blue\) at full strength, eleven minutes in\. The last two minutes at 4×\. Many more experiments didn’t help, and each one is in the log\. They include rewarding hero levels, twice the learning rate, a bigger batch, more games for the weakest races, less of the built\-in AI among the opponents, mixing behavioral cloning back in, an exploiter trained to beat the agent, more reward for following the target, phasing out the material reward, an anchor that moves along with the agent, and capping workers in the material reward\. Here is the agent today, playing each of the four races\. **Human**\(red\) against night elf \(blue\)\. The agent’s army burns down the night elf base\.**Orc**\(red\) against human \(blue\)\. A level\-5 Far Seer leads the army through the human base\.**Undead**\(blue\) against night elf \(red\)\. The undead army kills the Priestess of the Moon in her own base\.**Night elf**\(blue\) against undead \(red\)\. The agent’s army takes the undead base apart\.One won game with each race against the insane AI on Echo Isles, the last two minutes at 4×\. As night elf it wins almost every game; human and undead are its weakest races\.## What it can’t do yet **It plays one strategy\.**It wins with an early rush of cheap units, half of its wins within ten minutes\. When the rush fails the game goes long, and almost all of its losses are long games\. Across the best agent’s 183 long losses, which lasted 27 minutes on average, it built a big base with 35 workers against the AI’s 16\. But it stuck to two or three kinds of cheap unit against the AI’s nine, and researched three upgrades against the AI’s seventeen\. A long loss by its predecessor, from minute 23 at 4× speed\. The agent \(human, blue\) has 67 workers to the AI’s 9, five altars, five barracks and a field of farms, and it trained 97 footmen over the game as its only unit\. The insane AI \(undead, red\) made ten kinds of unit, among them gargoyles, abominations and frost wyrms, and 18 upgrades to the agent’s 2\. It wipes out the army, then the base\.**It neglects heroes\.**It raises about one a game, where the built\-in AI raises one and a half or more\. **So far, the target was the built\-in AI\.**Apart from itself and its past selves, the insane AI is the only opponent it has played, and beating it was the whole goal\. Good human players beat the insane AI reliably; they are far stronger than anything the agent has faced\. A human would likely see the rush coming and hold it off, and the long game that follows is where the agent is weakest\. ## What’s next The agent plays one way, and so far it has only learned to beat a script\. Next, I want it to: - **Play more diverse strategies**, not just one early rush, and use more of the game\. It hardly creeps, fighting the neutral camps for experience and items\. It rarely expands to a second gold mine or builds a shop, and its heroes don’t buy or use items\. It never hires heroes at the tavern or mercenaries at their camps\. Giving it a target build order was a first step, but it still follows those targets only loosely\. Even when the target is enforced, so that it can only make the target’s items, it makes only about a third of the target’s buildings, units and upgrades\. - **Keep getting better through self\-play**, well past the built\-in AI\. Playing against itself and its past selves can go in circles, with the agent forgetting how to beat strategies it no longer sees\.[AlphaStar](https://doi.org/10.1038/s41586-019-1724-z)countered this with a league that also has*exploiters*: agents trained only to find and punish the main agent’s weaknesses, so it has to fix them\. - **Play more maps\.**Echo Isles is only one of the game’s ladder maps\. Patch 1\.29 alone ships with many more two\-player maps, such as Terenas Stand, Tirisfal Glades, Melting Valley, Secret Valley and The Two Rivers, plus four\-player classics like Lost Temple, Twisted Meadows and Turtle Rock\. Beyond the ladder maps, Warcraft III has thousands of custom maps, the birthplace of whole genres, such as[DotA](https://en.wikipedia.org/wiki/Defense_of_the_Ancients), which became[Dota 2](https://en.wikipedia.org/wiki/Dota_2)and inspired[League of Legends](https://en.wikipedia.org/wiki/League_of_Legends), or the tower\-defense map Legion TD, now the standalone[Legion TD 2](https://legiontd2.com/)\. - **Scale up compute\.**Everything here ran on one desktop, which played about 11 years of Warcraft III in all, two years a day at full speed\. OpenAI Five played about 45,000 years of Dota 2 in ten months, some 250 years a day, on[800 petaflop/s\-days](https://openai.com/index/openai-five-defeats-dota-2-world-champions)of compute\. AlphaStar trained 12 agents for 44 days on 32 TPUs each\. And reinforcement learning, like language models, tends to improve predictably with more experience and bigger networks \([scaling laws](https://arxiv.org/abs/2301.13442)\)\. All of this runs on patch 1\.29 from 2018\. Newer versions of the game, from[Warcraft III: Reforged](https://en.wikipedia.org/wiki/Warcraft_III:_Reforged)in 2020 to today’s 3\.0 with its Definitive Edition graphics, run only through Battle\.net, Blizzard’s online launcher and copy protection, so I didn’t attempt them\. Beating the insane AI was the first milestone\. The next ones are more strategic variety, and beating good human players\. The research continues, and I will write follow\-up posts as it goes\. The[project’s documentation](https://github.com/RobinKa/warcraftsim/blob/master/docs/overview.md)has the details, and the[experiment log](https://github.com/RobinKa/warcraftsim/blob/master/docs/reference/experiment-log.md)has every run\. ## Citation If you use this work, please cite the project: ``` @software{kahlow2026warcraftsim, author = {Kahlow, Robin}, title = {{warcraftsim} - {Self-play} reinforcement learning in {Warcraft III}}, year = {2026}, url = {https://github.com/RobinKa/warcraftsim} } ``` ## Contact Have questions or ideas, or want to work together? Get in touch, or join the Warcraft 3 AI server on Discord\. [tora@warlock\.ai](mailto:[email protected])[LinkedIn](https://www.linkedin.com/in/robin-kahlow)[github\.com/RobinKa](https://github.com/RobinKa)[Discord: Warcraft 3 AI](https://discord.gg/4F9KVRDVhx)

Similar Articles

Dota 2

OpenAI Blog

OpenAI created a bot that defeats world-class Dota 2 professionals in 1v1 matches using only self-play learning, without imitation learning or tree search. The achievement demonstrates progress toward AI systems that can accomplish complex goals in dynamic, multi-agent environments.

Dota 2 with large scale deep reinforcement learning

OpenAI Blog

OpenAI Five became the first AI system to defeat Dota 2 world champions using large-scale deep reinforcement learning with self-play, demonstrating superhuman performance on a complex game with long time horizons and imperfect information.