Post-Training Isaac GR00T N1.5 for LeRobot SO-101 Arm

Introduction

NVIDIA Isaac GR00T (Generalist Robot 00 Technology) is a research and development platform for building robot foundation models and data pipelines, designed to accelerate the creation of intelligent, adaptable robots.

Today, we announced the availability of Isaac GR00T N1.5, the first major update to Isaac GR00T N1, the world’s first open foundation model for generalized humanoid robot reasoning and skills. This cross-embodiment model processes multimodal inputs, including language and images, to perform manipulation tasks across diverse environments. It is adaptable through post-training for specific embodiments, tasks, and environments.

In this blog, we’ll demonstrate how to post-train (fine-tune) GR00T N1.5 using teleoperation data from a single SO-101 robot arm.

Technical Blog for GR00T N1.5: https://research.nvidia.com/labs/gear/gr00t-n1_5/

Step-by-step tutorial

Now accessible to developers working with a wide range of robot form factors, GR00T N1.5 can be easily fine-tuned and adapted using the affordable, open-source LeRobot SO-101 arm.

This flexibility is enabled by the EmbodimentTag system, which allows seamless customization of the model for different robotic platforms, empowering hobbyists, researchers, and engineers to tailor advanced humanoid reasoning and manipulation capabilities to their own hardware.

Step 0: Installation

Before proceeding to installation, please check if you satisfy the prerequisites.

0.1 Clone the Isaac-GR00T Repo

git clone https://github.com/NVIDIA/Isaac-GR00T
cd Isaac-GR00T

0.2 Create the environment

conda create -n gr00t python=3.10
conda activate gr00t
pip install --upgrade setuptools
pip install -e .[base]
pip install --no-build-isolation flash-attn==2.7.1.post4 

Step 1: Dataset Preparation

Users can fine-tune GROOT N1.5 with any LeRobot dataset. For this tutorial, we will be using the table cleanup task as an example for fine-tuning.