OpenAI describes iterative improvements to their Dota 2 bot during The International tournament, combining coaching with self-play to enhance agent performance through rapid training cycles and strategic refinements discovered during professional matches.
Our Dota 2 result shows that self-play can catapult the performance of machine learning systems from far below human level to superhuman, given sufficient compute. In the span of a month, our system went from barely matching a high-ranked player to beating the top pros and has continued to improve since then. Supervised deep learning systems can only be as good as their training datasets, but in self-play systems, the available data improves automatically as the agent gets better.
# More on Dota 2
Source: [https://openai.com/index/more-on-dota-2/](https://openai.com/index/more-on-dota-2/)
Our approach, combining small amounts of “coaching” with self\-play, allowed us to massively improve our agent between the Monday and Thursday of The International\. On Monday evening, Pajkatt won using an unusual item build \(buying an early magic wand\)\. We added this item build to the training whitelist\.
Around 1pm on Wednesday, we tested the latest bot\. The bot would lose a bunch of health in the first wave\. We thought perhaps we needed to roll back, but noticed further gameplay was amazing, and the first wave behavior was baiting the other bots to be aggressive towards it\. Further self\-play fixed the issue, as the bot learned to counter the baiting strategy\. In the meanwhile, we stitched it together with Monday’s bot for the first wave only, and completed the process twenty minutes before Arteezy showed up at 4pm\.
After the Arteezy matches, we updated the creep block model, which increased TrueSkill by one point\. Further training before Sumail’s match on Thursday increased TrueSkill by two points\. Sumail pointed out that the bot had learned to cast razes out of the enemy’s vision\. This was due to a mechanic we hadn’t known about: abilities cast outside of the enemy’s vision prevent the enemy from gaining a wand charge\.
Arteezy also played a match against our 7\.5k semi\-pro tester\. Arteezy was winning the whole game, but our tester still managed to surprise him with a strategy he’d learned from the bot\. Arteezy remarked afterwards that this was a strategy that Paparazi had used against him once and was not commonly practiced\.
OpenAI created a bot that defeats world-class Dota 2 professionals in 1v1 matches using only self-play learning, without imitation learning or tree search. The achievement demonstrates progress toward AI systems that can accomplish complex goals in dynamic, multi-agent environments.
OpenAI Five competed against top professional Dota 2 teams at The International 2018, losing both matches against elite human players while demonstrating competitive gameplay and strategic depth developed through self-taught learning.
OpenAI Five became the first AI system to defeat Dota 2 world champions using large-scale deep reinforcement learning with self-play, demonstrating superhuman performance on a complex game with long time horizons and imperfect information.
OpenAI Five is a reinforcement learning agent that masters Dota 2 through self-play training with curriculum learning and strategic randomization, progressing from random behavior to executing complex human-level strategies.
OpenAI Five completed a benchmark match against humans in Dota 2, demonstrating improved capabilities including expanded hero pool (18 heroes), Roshan pit mechanics, and wards. The system shows general training flexibility in acquiring complex game skills.