The International 2018: Results

In contrast to our Benchmark⁠ 17 days ago, these games:

• Were played against significantly better human players

• Used hero lineups provided by a third party rather than by Five drafting against humans

• Removed our last major restriction from what most pros consider “Real Dota”⁠ gameplay.

Remarkably, the games were exciting and close—in contrast, all three Benchmark games were very one-sided—showing that even though Five teaches itself Dota from scratch, its playstyle results in incredible gameplay versus the best professionals. Winning is good, but losing shows the amazing skill of top professionals, and helps us compare Five’s play to the best of the best.

We’re incredibly grateful to everyone in the Dota community for helping to create such a great training ground for AI progress: from motivating Valve to create and evolve the incredibly complex game, to supporting the analysts and professionals who can help us measure our progress, to the excitement we’ve seen from so many viewers which makes the project so much more fun to work on.

Five’s progress from extremely restricted Dota against low-ranked players in May to competing at Dota with the latest restriction set.

The games

The purpose of the games were to showcase the capabilities of Five against the world’s best humans, playing games of “Real Dota”. Going into The International, we weren’t sure exactly who we would get to play, as it depended on the availability of people willing to play us on the mainstage. We were grateful to play against teams far stronger than the one at the Benchmark.

Rules change

The Benchmark games⁠ contained a very impactful restriction which we have now removed: each hero was given its own invulnerable courier (a unit which delivers items to your hero) rather than having a single mortal team courier.

Next steps

We don’t believe that the courier change was responsible for the losses. We think we need more training, bugfixes, and to remove the last pieces of scripted logic in our model.