Retro Contest

OpenAI

We’re launching a transfer learning contest that measures a reinforcement learning algorithm’s ability to generalize from previous experience.

Why it matters

In typical RL research, algorithms are tested in the same environment where they were trained, which favors algorithms which are good at memorization and have many hyperparameters. Instead, our contest tests an algorithm on previously unseen video game levels. This contest uses Gym Retro, a new platform integrating classic games into Gym, starting with 30 SEGA Genesis games.

The OpenAI Retro Contest⁠(opens in a new window) gives you a training set of levels from the Sonic The Hedgehog™ series of games, and we evaluate your algorithm on a test set of custom levels that we have created for this contest. The contest will run from April 5 to June 5. To get people started we’re releasing retro-baselines⁠(opens in a new window), which shows how to run several RL algorithms on the contest tasks.

Baseline results on the Retro Contest (test set) show that RL algorithms fall far below human performance, even when using transfer learning. Human performance is shown as a dashed horizontal line. The humans only played for one hour, versus eighteen for the algorithms.

You can use any environments or datasets you want at training time, but at test time you only get about 18 hours (1 million timesteps) on each never-before-seen level. 18 hours may sound like a long time to play a single game level, but existing RL algorithms perform far worse than humans given this training budget.

Sonic Benchmark

But we have a long way to go before our algorithms can rival human performance. As shown above, after two hours of practice on the training levels and one hour of play on each test level, humans are able to attain scores that are significantly higher than those attained by RL algorithms, including ones that perform transfer learning.

Sonic Recordings

Gym Retro Beta

To get started with Gym Retro check out the Getting Started section on GitHub.