Spinning Up in Deep RL: Workshop review

We hosted ~90 people at our office and engaged nearly 300 more through our livestream. Participants came from a wide range of backgrounds, including academia, software engineering, data science, ML engineering, medicine, and education. This workshop built off our Spinning Up in Deep RL⁠ resource package and took a deeper dive into RL algorithm design, robotics, and building safe AI systems.

Building educational tools

One of the goals for education at OpenAI is to help people develop the skills needed to participate in research and development in AI—especially in deep RL, a core area of research at OpenAI. From our experience working with Scholars⁠(opens in a new window) and Fellows⁠(opens in a new window), we’ve found that the key ingredients for skill development are:

1. a flexible curriculum that includes core material and a review of research frontiers,

2. mentorship and discussions with experts, and

3. having the students work on projects that are at the right level to help them grow.

The challenge for education at OpenAI is to figure out how to deliver these at scale. While sharing a curriculum at scale is relatively easy, it isn’t obvious how to scale up mentorship and guidance on projects. Our working theory is that workshops might help us do just that. Our first Spinning Up workshop has given us several positive signs that this is a useful direction, and we’re excited to share what we learned.

The crowd

We hosted around 90 people at our office and involved nearly 300 more through our livestream. Our guests came from a wide range of backgrounds, including academic research, software engineering, data science, ML engineering, medicine, and education. The level of ML experience varied quite significantly across the group, from “almost none” to “built their own Dota bot!”

The talks

The afternoon

Our takeaways

What worked: We asked our participants what their highlights were, and these responses are a fairly representative sample:

“Extend the workshop to two days.”