3701/40 175794-01 Jen-hsun Huang, founder and CEO of NVIDIA, said: “The next wave of AI is robotics, and one of the most exciting developments is humanoid robots. “We are advancing the entire NVIDIA robotics stack with open access for humanoid robot developers and companies around the world, giving them access to the platforms, accelerators and AI models that best meet their needs.”

Accelerate development with NVIDIA NIM and OSMO

NIM microservices provides pre-built containers powered by NVIDIA Inference software, enabling developers to reduce deployment time from weeks to minutes. Roboticists will be able to enhance generative physical AI simulation workflows in NVIDIA Isaac Sim, a robotics simulation reference application built on the NVIDIA Omniverse platform, through two new AI microservices.

The MimicGen NIM microservice generates synthetic motion data from remote operation data recorded by spatial computing devices such as the Apple Vision Pro. Robocasa NIM microservices generate robotic task and simulation ready environments in OpenUSD, a common framework for development and collaboration3701/40 175794-01 in a 3D world.

Now available, NVIDIA OSMO is a cloud-native hosted service that allows users to coordinate and scale complex robot development workflows across distributed computing resources, whether on-premises or in the cloud.

OSMO greatly simplifies robot training and simulation workflows, reducing deployment and development cycles from months to a week. Users can visually manage a variety of tasks, including synthetic data generation, model training, reinforcement learning, and software in-the-loop testing of large-scale humanoid robots, autonomous mobile robots, and industrial robotic arms.

Advanced data capture workflows for humanoid robot developers

Training the basic model of a humanoid robot requires a lot of data. Remote opera3701/40 175794-01 tion is one of the ways to obtain human demonstration data, but the process is becoming increasingly expensive and lengthy.

With the NVIDIA AI and Omniverse remotely operated reference workflows demonstrated at the SIGGRAPH computer Graphics Conference, researchers and AI developers are able to generate large amounts of synthetic motion and perception data from a very small number of remotely captured human demonstrations.

First, the developers captured a small number of remote operation demonstrations using the Apple Vision Pro, then emulated the footage in the NVIDIA Isaac Sim, and used the MimicGen NIM microservice to generate a synthetic dataset based on the footage.

Developers can use real and synthetic data to train the Project GR00T humanoid robot base model to save time and reduce costs. They can then use the Robocasa NIM microservice in Isaac Lab, a robot learning framework, to generate experiences and retrain robot models. Throughout the workflow, NVIDIA OSMO seamlessly distributes compute tasks to different resources, saving developers weeks of administrative effort.

General Robotics Platform Company Fourier saw the advantages of using simulation techniques to synthesize training data. Alex Gu, CEO of Fourier, said: “The development of humanoid robots is extremely complex, and the work requires tedious access to large amounts of real data from the real world. NVIDIA’s new simulation and generative AI developer tools will help guide and accelerate our model development workflow.”