At CES 2025, NVIDIA’s next step in finding new uses for generative AI applications finally has an answer with a full commitment to physical AI to transform yet another layer of industry.
Building on the foundation of Omniverse, new tools are being used to build 3D worlds that allow AI simulations with physical attributes and/or photorealistic output which can ultimately help AI models to learn about the real world. Examples of this include USD Code and USDSearch NVIDIA NIM microservices that accelerate developments by providing search or generate options for OpenUSD assets via text prompts. Additionally, the new NVIDIA Edify SimReady generative AI model automatically labels 3D assets with attributes like physics and materials, enabling developers to process 1,000 objects in minutes instead of over 40 hours manually.
Team Green also unveiled Cosmos, a platform featuring open, pretrained world foundation models specifically designed for generating physics-aware videos and world states for physical AI development. Trained on 18 quadrillion tokens, including 2 million hours of data from robotics, drones, autonomous driving, and synthetic sources, Cosmos reduces the simulation-to-reality gap by enhancing 3D simulations to closely resemble real-world scenarios. This is achieved through its integration with Omniverse, a platform that offers APIs and microservices for creating 3D applications. The combination of Cosmos and Omniverse ensures accurate simulations and minimizes errors, such as hallucinations often associated with world models.
In terms of blueprints, they also announced a total of 4 that can help simplify the creation of OpenUSD-based digital twins for physical AI which are the following:
- Mega Blueprint: Designed for simulating and testing robot fleets in digital twin environments before deployment.
- Autonomous Vehicle Simulation Blueprint: Enables closed-loop testing, replaying driving data, and generating ground-truth data to streamline AV development.
- Omniverse Spatial Streaming Blueprint: Integrates with Apple Vision Pro for immersive streaming of large-scale industrial digital twins.
- Real-Time Digital Twins for CAE: Provides workflows powered by NVIDIA CUDA-X™ and physics AI for real-time computational fluid dynamics and physics visualization.
Particularly on the subject of Omniverse Spatial Streaming Blueprint, Katana Studio, and Innoactive have embraced it in production where the former created custom car configurators for Nissan and Volkswagen, improving customer decision-making processes while Innoactive used the workflow to enable Volkswagen Group and Syntegon to conduct high-resolution project reviews of digital twins for car models and pharmaceutical production installations.
Other industry leaders are also getting more and more into Omniverse applications, including Accenture, Altair, Ansys, Foretellix, Cadence, MITRE, and Neural Concept.
On the other hand, NVIDIA’s consistent progress in the humanoid robot segment has been rather “hidden” from the sights of the public but now, they have announced a suite of tools, including robot foundation models, data pipelines, and simulation frameworks, designed to accelerate the development of next-generation humanoid robots.
The main highlight of this news is the Isaac GR00T Blueprint which is a system for synthetic motion generation that enables developers to produce vast amounts of synthetic motion data for training humanoid robots using imitation learning, mainly targeting the reduction of data collection in the real world alongside the associated cost.
Its underlying tech comes from imitation learning which is a subset of the general robot learning field where robots acquire new skills by observing and imitating expert human demonstrations. Through Isaac GR00T blueprint, developers can generate extensive synthetic datasets from just a handful of human demonstrations.
The GR00T-Teleop workflow utilizes the Apple Vision Pro to record human actions in a digital twin environment. These actions are then mimicked by robots in simulation and used as ground truth. The GR00T-Mimic workflow expands the dataset further, and the GR00T-Gen workflow, built on NVIDIA Omniverse and NVIDIA Cosmos platforms, exponentially increases the dataset through domain randomization and 3D upscaling. The resulting data trains robots to move and interact with their environments effectively and safely using NVIDIA Isaac Lab, a modular and open-source robot learning framework.
Industry leaders like Boston Dynamics and Figure have already adopted the Isaac GR00T system, demonstrating promising results. Meanwhile, developers and manufacturers can apply for early access here.
