NVIDIA's GTC 2025 Announcement for Physical AI Developers: New Open Models and Datasets

NVIDIA Isaac GR00T N1 used in object manipulation.

At its annual GTC conference, NVIDIA has unveiled a trio of groundbreaking open-source releases aimed at accelerating physical AI development. Release of a new suite of world foundation models (WFMs) with multicontrols called Cosmos Transfer, a highly curated Physical AI Dataset, and the first open model for general humanoid reasoning called NVIDIA Isaac GR00T N1 - represent a significant leap forward in physical AI technology, offering developers powerful tools and resources to advance robotics systems, and enhance autonomous vehicle technology.

New World Foundation Model - Cosmos Transfer

Cosmos Transfer, the latest addition to NVIDIA's Cosmos™ world foundation models (WFMs), introduces a new level of control and accuracy in generating virtual world scenes.

Available in 7 billion parameter size, the model utilizes multicontrols to guide the generation of high-fidelity world scenes from structural inputs, ensuring precise spatial alignment and scene composition.

How it works

The model is built by training individual ControlNets separately for each sensor modality used to capture the simulated world.

Input types include 3D bounding box map, Trajectory map, Depth map, Segmentation map.

• At inference time, developers can use various input types, including structured visual or geometric data such as segmentation maps, depth maps, edge maps, human motion keypoints, LiDAR scans, trajectories, HD maps, and 3D bounding boxes to guide the output.
• The control signals from each control branch are multiplied by their corresponding adaptive spatiotemporal control maps and then summed before being added to the transformer blocks of the base model.
• The generated output is photorealistic video sequences with controlled layout, object placement, and motion. Developers can control the output in multiple ways, such as preserving structure and appearance or allowing appearance variations while maintaining structure.