Learning dexterity

We’ve trained a human-like robot hand to manipulate physical objects with unprecedented dexterity.

Our system, called Dactyl, is trained entirely in simulation and transfers its knowledge to reality, adapting to real-world physics using techniques we’ve been working on for the past⁠ year⁠. Dactyl learns from scratch using the same general-purpose reinforcement learning algorithm and code as OpenAI Five⁠. Our results⁠ show that it’s possible to train agents in simulation and have them solve real-world tasks, without physically-accurate modeling of the world.

The task

Dactyl is a system for manipulating objects using a Shadow Dexterous Hand⁠(opens in a new window). We place an object such as a block or a prism in the palm of the hand and ask Dactyl to reposition it into a different orientation; for example, rotating the block to put a new face on top. The network observes only the coordinates of the fingertips and the images from three regular RGB cameras.

Although the first humanoid hands were developed decades ago, using them to manipulate objects effectively has been a long-standing challenge in robotic control. Unlike other problems such as locomotion⁠(opens in a new window), progress on dextrous manipulation using traditional robotics approaches has been slow, and current techniques⁠(opens in a new window) remain limited in their ability to manipulate objects in the real world.

Reorienting an object in the hand requires the following problems to be solved:

Our approach

Dactyl learns to solve the object reorientation task entirely in simulation without any human input. After this training phase, the learned policy works on the real robot without any fine-tuning.

We built a simulated version⁠ of our robotics setup using the MuJoCo⁠(opens in a new window) physics engine. This simulation is only a coarse approximation of the real robot:

Learning to control

Learning to see

Example training images used for learning to estimate the pose of the block.