Technology company Nvidia’s latest suite of services, models, and computing platforms is designed to advance the development of humanoid robotics on a global scale. This initiative aims to support leading robot manufacturers, artificial intelligence (AI) model developers, and software makers in creating the next generation of humanoid robots.
Among Nvidia’s new offerings are the NIM microservices and frameworks for robot simulation and learning, the Osmo orchestration service for managing multi-stage robotics workloads, and a teleoperation workflow enabled by AI and simulation. These tools are designed to facilitate the training of robots using minimal amounts of human demonstration data.
“The next wave of AI is robotics and one of the most exciting developments is humanoid robots,” said Jensen Huang, founder and CEO of Nvidia. “We’re advancing the entire Nvidia robotics stack, opening access for worldwide humanoid developers and companies to use the platforms, acceleration libraries, and AI models best suited to their needs.”
Key components
NIM Microservices are pre-built containers powered by Nvidia inference software, to reduce deployment times from weeks to minutes. Two new AI microservices, MimicGen and Robocasa, are introduced to enhance simulation workflows for generative physical AI in Nvidia Isaac Sim.
The MimicGen generates synthetic motion data based on teleoperated data from devices like Apple Vision Pro.
Robocasa creates robot tasks and simulation-ready environments in OpenUSD, a universal framework for 3D world development.
The Nvidia Osmo, a cloud-native managed service, facilitates the orchestration and scaling of complex robotics development workflows across distributed computing resources. Osmo simplifies robot training and simulation workflows, significantly reducing deployment and development cycle times.
Simulation innovations
Nvidia’s AI- and Omniverse-enabled teleoperation workflow, demonstrated at the SIGGRAPH computer graphics conference, allows developers to generate large amounts of synthetic motion and perception data from minimal human demonstrations. The process involves capturing a small number of teleoperated demonstrations using Apple Vision Pro, simulating these recordings in Nvidia Isaac Sim, and using MimicGen to generate synthetic datasets. These datasets train the Project GR00T humanoid foundation model, combining real and synthetic data to save time and reduce costs.
Comprehensive Computing Platforms
Nvidia provides three primary computing platforms to support humanoid robotics development:
- Nvidia AI Supercomputers: For training models.
- Nvidia Isaac Sim: Built on Omniverse, for robots to learn and refine skills in simulated environments.
- Nvidia Jetson Thor: Humanoid robot computers for running models.
- Developers can access these platforms through the Nvidia Humanoid Robot Developer Program, which also offers early access to the latest releases of Nvidia Isaac Sim, Nvidia Isaac Lab, Jetson Thor, and Project GR00T models.
Developers interested in leveraging these advanced tools and resources can join the Nvidia Humanoid Robot Developer Program, gaining access to Nvidia Osmo, Isaac Lab, and soon, the NIM microservices.