Into the Omniverse: OpenUSD Workflows Advance Physical AI for Robotics, Autonomous Vehicles
22/01/2025
The next frontier of AI is physical AI. Physical AI models can understand instructions and perceive, interact and perform complex actions in the real world to power autonomous machines like robots and self-driving cars.
Similar to how large language models can process and generate text, physical AI models can understand the world and generate actions. To do this, these models must be trained in simulation environments to comprehend physical dynamics, like gravity, friction or inertia - and understand geometric and spatial relationships, as well as the principles of cause and effect.
Global leaders in software development and professional services are using NVIDIA Omniverse, powered by OpenUSD, to build new products and services that will accelerate the development of AI and controllable simulations to enable the creation of true-to-reality virtual worlds, known as digital twins, that can be used to train physical AI with unprecedented accuracy and detail.
Generate Exponentially More Synthetic Data With Omniverse and NVIDIA Cosmos At CES, NVIDIA announced generative AI models and blueprints that expand Omniverse integration further into physical AI applications such as robotics, autonomous vehicles and vision AI.
Among these announcements was NVIDIA Cosmos, a platform of state-of-the-art generative world foundation models, advanced tokenizers, guardrails and an accelerated video processing pipeline - all designed to accelerate physical AI development.
Developing physical AI models is a costly, resource- and time-intensive process that requires vast amounts of real-world data and testing. Cosmos' world foundation models (WFM), which predict future world states as videos based on multimodal inputs, provide an easy way for developers to generate massive amounts of photoreal, physics-based synthetic data to train and evaluate AI for robotics, autonomous vehicles and machines. Developers can also fine-tune Cosmos WFMs to build downstream world models or improve quality and efficiency for specific physical AI use cases.
When paired with Omniverse, Cosmos creates a powerful synthetic data multiplication engine. Developers can use Omniverse to create 3D scenarios, then feed the outputs into Cosmos to generate controlled videos and variations. This can drastically accelerate the development of physical AI systems such as autonomous vehicles and robots by rapidly generating exponentially more training data covering a variety of environments and interactions.
OpenUSD ensures the data in these scenarios is seamlessly integrated and consistently represented, enhancing the realism and effectiveness of the simulations.
Leading robotics and automotive companies, including 1X, Agile Robots, Agility Robotics, Figure AI, Foretellix, Fourier, Galbot, Hillbot, IntBot, Neura Robotics, Skild AI, Virtual Incision, Waabi and XPENG, along with ridesharing giant Uber, are among the first to adopt Cosmos.
Learn more about how world foundation models will advance physical AI by listening to the NVIDIA AI Podcast episode with Ming-Yu Liu, vice president of research at NVIDIA.
See Cosmos in Action for Physical AI Use Cases Cosmos WFMs are revolutionizing industries by providing a unified framework for developing, training and deploying large-scale AI models across various applications. Enterprises in the automotive, industrial and robotics sectors can harness the power of generative physical AI and simulation to accelerate innovation and operational efficiency.
Humanoid robots: The NVIDIA Isaac GR00T Blueprint for synthetic motion generation helps developers generate massive synthetic motion datasets to train humanoid robots using imitation learning. With GR00T workflows, users can capture human actions and use Cosmos to exponentially increase the size and variety of the dataset, making it more robust for training physical AI systems.
Autonomous vehicles: Autonomous vehicle (AV) simulation powered by Omniverse Sensor RTX application programming interfaces lets AV developers replay driving data, generate new ground-truth data and perform closed-loop testing to accelerate their pipelines. With Cosmos, developers can generate synthetic driving scenarios to amplify training data by orders of magnitude, accelerating physical AI model development for autonomous vehicles. Global ridesharing giant Uber is partnering with NVIDIA to accelerate autonomous mobility. Rich driving datasets from Uber, combined with Cosmos and NVIDIA DGX Cloud, can help AV partners build stronger AI models more efficiently.
Industrial settings: Mega is an Omniverse Blueprint for developing, testing and optimizing physical AI and robot fleets at scale in a USD-based digital twin before deployment in factories and warehouses. The blueprint uses Omniverse Cloud Sensor RTX APIs to simultaneously render multisensor data from any type of intelligent machine, enabling high-fidelity sensor simulation at scale. Cosmos can enhance Mega by generating synthetic edge case scenarios to amplify training data, significantly improving the robustness and efficiency of training robots in simulation. KION Group, a supply chain solutions company, is among the first to adopt Mega to drive warehouse automation in retail, consumer packaged goods, parcel services and more.
Get Plugged Into the World of OpenUSD For more on Cosmos, watch the replay of NVIDIA CEO Jensen Huang's CES keynote, and get started with Cosmos WFMs available now under an open model license on Hugging Face and the NVIDIA NGC catalog. Join the upcoming livestream on Wednesday, February 5 for a deep dive into Cosmos WFMs and physical AI work
Most recent headlines
22/01/2025
4 Tips To Supercharge Your Discovery on Spotify in 2025
Personalized listening is key to unlocking your perfect soundtrack for 2025. It's about more than just hearing your favorites, and at Spotify, we want to he...
22/01/2025
The boys are back! Smash-hit series Rogue Heroes returns for an adrenaline-fuelled second season
The boys are back! Smash-hit series Rogue Heroes returns for an adrenaline-fuell...
22/01/2025
CES 2025: Kicking Off the Year by Diving Into The Latest Innovations
In our latest blog, Tim Pearson reflects on CES 2025, the latest trends, and the key conversations held with visitors to the NAGRAVISION booth. Connect the wor...
22/01/2025
The Gauge: Poland | December 2024
December is the perfect month for spending time together in front of the TV, with the winter weather and festive period combining to create the ideal conditions...
22/01/2025
24.6 Million Viewers Watch President Donald Trump's Inauguration as 47th U.S. President
NEW YORK - January 21, 2025 - An estimated 24.6 million viewers tuned in on Mond...
22/01/2025
PBS Reno, Sinclair Launch ATSC 3.0 Virtual Channel
RENO, Nev. PBS Reno and Sinclair Broadcast Group have launched KNPB-VC, a new virtual ATSC 3.0 channel, broadcasting on virtual channel 5-11....
22/01/2025
TASCAM Releases Cost-Effective IF-ST2110 Expansion Card a...
TASCAM, renowned for its versatile and adaptable audio solutions, announced the availability of the new IF-ST2110 Expansion card for its TASCAM Sonicview 16XP/2...
22/01/2025
Cobalt Digital Announces Plans to Exhibit Award-Winning S...
Cobalt Digital, the leading designer and manufacturer of award-winning signal processing products, and a founding partner in the openGear initiative has reveal...
22/01/2025
Panasonic introduces four new 4K professional camcorders
Featuring IP connectivity, the cameras aim to provide an end-to-end solution for on-location filming, production and distribution By Matthew Corrigan Publish...
22/01/2025
AV and broadcast convergence ebook now available
ISE 2025: Technology Convergence is now free to access ahead of ISE 2025, and is jam-packed with features, interviews and product news By Jenny Priestley Pub...
22/01/2025
Ateliere Taps Media Visionary Michael Sid to Spearhead Gr...
Ateliere Creative Technologies, a leading GenAI media software solutions company, today announced the appointment of Michael Sid as Strategic Business Accelerat...
22/01/2025
Hiltron Reports 45th Year of Satcom Technology Innovation
Hiltron Communications reports its 45th year of success. Based near Stuttgart, the company is a globally active systems integrator, manufacturer and distributor...
22/01/2025
WestCom Broadcast Services GmbH Embraces the Future with...
WestCom Broadcast Services GmbH, renowned for producing German news show SAT.1 NRW, has made a forward-looking investment in its production infrastructure with ...
22/01/2025
BBright selects Leader LV7600W for ST 2110 IP validation
Test & measurement innovator, Leader Electronics of Europe, has announced that BBright, a prominent provider of cutting-edge media and broadcast solutions, has ...
22/01/2025
ASL CN1 Cuneo selects Cubbit geo-distributed cloud to ach...
Cubbit, the first geo-distributed cloud storage enabler, today announced that ASL CN1 Cuneo, a North Italian healthcare public service organisation, has reduced...
22/01/2025
Flexible programming for Radio Botswana with IP native Ty...
Aiming to create more flexibility, encourage growth, and future-proof its infrastructure, Radio Botswana has upgraded its 60-year-old analogue system to IP with...
22/01/2025
Grande Halle de la Villette Transforms Event Production W...
Riedel Communications today announced that the Grande Halle de la Villette (EPPGHV) in Paris has deployed Riedel's state-of-the-art intercom and signal dist...
22/01/2025
Guntermann and Drunck and VuWall and Tritec Unite to Rede...
Guntermann & Drunck GmbH (G&D), a leading provider of high-performance KVM solutions, is pleased to welcome VuWall as a new member of Naxicap's portfolio of...
22/01/2025
Blackmagic URSA Cine 12K LF Added to Netflix's Approved Camera List
FREMONT, Calif. Blackmagic Design has announced that its URSA Cine 12K LF digital film camera was recently added to the Netflix Approved Camera List....
22/01/2025
H.I.G. Capital To Buy Kantar Media for $1 Billion
Bain Capitals Kantar Group has announced it plans to sell the media measurement and analytics firm Kantar Media for about $1 billion to H.I.G. Capital, a major ...
22/01/2025
Cleveland Cavaliers to Simulcast 5 NBA Games on Gray's WUAB
ATLANTA The Cleveland Cavaliers, Rock Entertainment Sports Network, Gray Media, and FanDuel Sports Network have announced an agreement will simulcast five upcom...
22/01/2025
FCC Chair Carr Ends FCC's Promotion of DEI
WASHINGTON, D.C. In one of his first notable acts as the head of the Federal Communications Commission, FCC chair Brendan Carr has announced that he is ending ...
22/01/2025
Netflix Adds 18.9M Subs, Passes 300M Users, Announces Price Hikes
Netflix stock jumped in after hours trading after announcing Q4 results that easily beat Wall Street expectations for sub counts, revenue and profits as it adde...
22/01/2025
Gerald Peter performs Distant Variations with Ivory 3 American Concert D
We are thrilled to present the premiere of new music by Viennese composer and keyboardist, Gerald Peter. Known for his innovative and highly creative style, Ger...
22/01/2025
Miami Heat's Kurt Doster on the Immediate Impact of Their New Control Room, Flame Ball Centerhung
Miami Heat's Kurt Doster on the Immediate Impact of Their New Control Room, ...
22/01/2025
The Shared Fan Experience: How Cosm's Immersive Venues Change What It Means to Attend a Live Event
The Shared Fan Experience: How Cosm's Immersive Venues Change What It Means ...
22/01/2025
EMG/Gravity Media U.S. Names Carston Bell Specialist Systems Manager
EMG/Gravity Media U.S. Names Carston Bell Specialist Systems Manager He is an expert in RF-transmission technology and specialty cameras By Ken Kerschbaumer, E...
22/01/2025
Larger-Than-Life Live Viewing: How Sports Is Re-Imagining the Location-Based Experience
Larger-Than-Life Live Viewing: How Sports Is Re-Imagining the Location-Based Exp...
22/01/2025
Is Your Corporation Looking to Support Victims of L.A. Fires? Donate to Sports Broadcasting Fund Today
Is Your Corporation Looking to Support Victims of L.A. Fires? Donate to Sports B...
22/01/2025
Adirondack Bank Center Is First North American Venue To Install Permanent Digital Dasherboards
Adirondack Bank Center Is First North American Venue To Install Permanent Digita...
22/01/2025
Official trailer revealed for Chris O'Dowd's Small Town, Big Story, a very dramatic comedy starring Christina Hendricks and Paddy Considine
Official trailer revealed for Chris O'Dowd's Small Town, Big Story, a ve...
22/01/2025
Strap yourself in: Production begins on the farewell series of critically acclaimed Sky Original comedy, Brassic
Strap yourself in: Production begins on the farewell series of critically acclai...
22/01/2025
Grande Halle de la Villette Transforms Event Production With Riedel Solutions
Wuppertal January 22, 2025 Grande Halle de la Villette Transforms Event Production With Riedel SolutionsRiedel Communications today announced that the Grande ...
22/01/2025
Colour Talks. Brussels. Thursday January 23
Thursday January 23 Cinema Palace / 85 Boulevard Anspach, 1000 Brussels Brought to you by FilmLight & Sondr 6:00pm / Doors open 6:30pm / Presentations follo...
22/01/2025
Micro Salon. Paris. Friday February 7
Friday February 7, 11am Parc Floral de Paris Register for Micro Salon Rencontre cr ative : Dans les coulisses de Saint-Ex FilmLight sera au Micro Salon Pa...
22/01/2025
Comscore Appoints Industry Change-Maker Frank Friedman to Lead Next Era of Measurement Transformation
Comscore Appoints Industry Change-Maker Frank Friedman to Lead Next Era of Measu...
22/01/2025
Feature Highlight: New Macro Wizards in Broadcast Pix Version 10.2 Software
Effortlessly Create and Control PiP Effects with Macro Wizards...
22/01/2025
Into the Omniverse: OpenUSD Workflows Advance Physical AI for Robotics, Autonomous Vehicles
Editor's note: This post is part of Into the Omniverse, a series focused on ...
22/01/2025
The Future of Marketing: How AI Agents Can Enhance Customer Journeys in Retail
AI agents - which can understand, adapt to and support each user's unique journey - are making online shopping and digital marketing more efficient and pers...
22/01/2025
The Italian Space Agency entrusts Thales Alenia Space which will contribute to NASA's Surface Biology and Geology Earth Observation mission
Facebook Twitter LinkedIn A joint cooperation between ASI, NASA and NASA...
22/01/2025
Third Act for Eurosong Announced
Adgy hoping to represent Ireland at Eurovision 2025 First listen and interview on today's Ray D'Arcy Show on RT Radio 1 Donegal born Adgy is the late...
21/01/2025
Leading journalists offer unique insights in war and conflict training course
Two senior war reporters from the BBC and Sky News have joined an outstanding array of experts in a unique series of training courses for journalists covering p...
21/01/2025
Set a Course of Self-Improvement With Spotify; Plus, a Special Offer From adidas
If you're looking to introduce better habits in the year ahead, Spotify is the perfect place to start. Beginning with a special offer for new Spotify Premiu...
21/01/2025
L3Harris Names New Aerojet Rocketdyne President, Expands Leadership Roles
MELBOURNE, Fla., Jan. 21, 2025 - L3Harris Technologies (NYSE: LHX) today announced Ken Bedingfield, 52, has been appointed President, Aerojet Rocketdyne, effect...
21/01/2025
US Army Awards L3Harris Second Full-Scale ENVG-B Production Order
LONDONDERRY, N.H., Jan. 21, 2025 - L3Harris Technologies (NYSE: LHX) has received a $263 million order from the U.S. Army for continued production of the Enhanc...
21/01/2025
L3Harris Adds Electronic Warfare Capabilities to T7 Robot at Vanguard
T7 robot with electronic warfare effects enables signals to be sensed and detected a few kilometres ahead of troops, keeping them out of harm's way....
21/01/2025
Holidays, Football and Blockbusters Drive Record Highs for YouTube, Netflix and Amazon Prime Video in December, according to The Gauge
Four days in December interval exceed 100 billion viewing minutes of TV. Netfli...
21/01/2025
Brendan Carr Thanks Trump for FCC Chair Designation
WASHINGTON Brendan Carr expressed gratitude and eagerness to move forward on a range of issues in a statement Jan. 20 upon President Donald Trump's order na...
21/01/2025
Nexstar, Altice USA End Blackout
IRVING, Texas, and NEW YORK After exchanging increasingly pointed barbs in recent weeks, Nexstar Media Group, Inc. and Altice USA (NYSE: ATUS) reached a new dis...