NVIDIA Brings Generative AI Tools, Simulation and Perception Workflows to ROS Developer Ecosystem
22/10/2024
Among the reveals were new generative AI nodes and workflows for ROS developers deploying to the NVIDIA Jetson platform for edge AI and robotics. Generative AI enables robots to perceive and understand the context of their surroundings, communicate naturally with humans and make adaptive decisions autonomously.
Generative AI Comes to ROS Community ReMEmbR, built on ROS 2, uses generative AI to enhance robotic reasoning and action. It combines large language models (LLMs), vision language models (VLMs) and retrieval-augmented generation to allow robots to build and query long-term semantic memories and improve their ability to navigate and interact with their environments.
The speech recognition capability is powered by the WhisperTRT ROS 2 node. This node uses NVIDIA TensorRT to optimize OpenAI's Whisper model to enable low-latency inference on NVIDIA Jetson, resulting in responsive human-robot interaction.
The ROS 2 robots with voice control project uses the NVIDIA Riva ASR-TTS service to make robots understand and respond to spoken commands. The NASA Jet Propulsion Laboratory independently demonstrated ROSA, an AI-powered agent for ROS, operating on its Nebula-SPOT robot and the NVIDIA Nova Carter robot in NVIDIA Isaac Sim.
At ROSCon, Canonical is demonstrating NanoOWL, a zero-shot object detection model running on the NVIDIA Jetson Orin Nano system-on-module. It allows robots to identify a broad range of objects in real time, without relying on predefined categories.
Developers can get started today with ROS 2 Nodes for Generative AI, which brings NVIDIA Jetson-optimized LLMs and VLMs to enhance robot capabilities.
Enhancing ROS Workflows With a Sim-First' Approach
Simulation is critical to safely test and validate AI-enabled robots before deployment. NVIDIA Isaac Sim, a robotics simulation platform built on OpenUSD, provides ROS developers a virtual environment to test robots by easily connecting them to their ROS packages. A new Beginner's Guide to ROS 2 Workflows With Isaac Sim, which illustrates the end-to-end workflow for robot simulation and testing, is now available.
Foxglove, a member of the NVIDIA Inception program for startups, demonstrated an integration that helps developers visualize and debug simulation data in real time using Foxglove's custom extension, built on Isaac Sim.
New Capabilities for Isaac ROS 3.2 NVIDIA Isaac ROS, built on the open-source ROS 2 software framework, is a suite of accelerated computing packages and AI models for robotics development. The upcoming 3.2 release enhances robot perception, manipulation and environment mapping.
Key improvements to NVIDIA Isaac Manipulator include new reference workflows that integrate FoundationPose and cuMotion to accelerate development of pick-and-place and object-following pipelines in robotics.
Another is to NVIDIA Isaac Perceptor, which features a new visual SLAM reference workflow, enhanced multi-camera detection and 3D reconstruction to improve an autonomous mobile robot's (AMR) environmental awareness and performance in dynamic settings like warehouses.
Partners Adopting NVIDIA Isaac
Robotics companies are integrating NVIDIA Isaac accelerated libraries and AI models into their platforms.
Universal Robots, a Teradyne Robotics company, launched a new AI Accelerator toolkit to enable the development of AI-powered cobot applications.
Miso Robotics is using Isaac ROS to speed up its AI-powered robotic french fry-making Flippy Fry Station and drive advances in efficiency and accuracy in food service automation.
Wheel.me is partnering with RGo Robotics and NVIDIA to create a production-ready AMR using Isaac Perceptor.
Main Street Autonomy is using Isaac Perceptor to streamline sensor calibration.
Orbbec announced its Perceptor Developer Kit, an out-of-the-box AMR solution for Isaac Perceptor.
LIPS Corporation has introduced a multi-camera perception devkit for improved AMR navigation.
Canonical highlighted a fully certified Ubuntu environment for ROS developers, offering long-term support out of the box.
Connecting With Partners at ROSCon ROS community members and partners, including Canonical, Ekumen, Foxglove, Intrinsic, Open Navigation, Siemens and Teradyne Robotics, will be in Denmark presenting workshops, talks, booth demos and sessions. Highlights include:
Nav2 User Meetup Birds of a Feather session with Steve Macenski from Open Navigation LLC
ROS in Large-Scale Factory Automation with Michael Gentner from BMW AG and Carsten Braunroth from Siemens AG
Integrating AI in Robot Manipulation Workflows Birds of a Feather session with Kalyan Vadrevu from NVIDIA
Accelerating Robot Learning at Scale in Simulation Birds of a Feather session with Markus Wuensch from NVIDIA
On Use of Nav2 Docking with Open Navigation's Macenski
Additionally, Teradyne Robotics and NVIDIA are co-hosting a lunch and evening reception on Tuesday, Oct. 22, in Odense, Denmark.
The Open Source Robotics Foundation (OSRF) organizes ROSCon. NVIDIA is a supporter of Open Robotics, the umbrella organization for OSRF and all its initiatives.
For the latest updates, visit the ROSCon page.
LINK: | https://blogs.nvidia.com/blog/generative-ai-simulation-roscon/... |
See more stories from nvidia |
More from Nvidia
30/10/2024
Spooks Await at the Haunted Sanctuary,' Built With RTX and AI
Among the artists using AI to enhance and accelerate their creative endeavors is Sabour Amirazodi, a creator and tech marketing and workflow specialist at NVIDI...
29/10/2024
A New ERA of AI Factories: NVIDIA Unveils Enterprise Reference Architectures
As the world transitions from general-purpose to accelerated computing, finding a path to building data center infrastructure at scale is becoming more importan...
28/10/2024
Bring Receipts: New NVIDIA AI Workflow Detects Fraudulent Credit Card Transactions
Financial losses from worldwide credit card transaction fraud are expected to re...
28/10/2024
Fintech Leaders Tap Generative AI for Safer, Faster, More Accurate Financial Services
An overwhelming 91% of financial services industry (FSI) companies are either as...
24/10/2024
India Should Manufacture Its Own AI,' Declares NVIDIA CEO
Artificial intelligence will be the driving force behind India's digital transformation, fueling innovation, economic growth, and global leadership, NVIDIA ...
24/10/2024
Zoom's AI-First Transformation to Boost Business Productivity, Collaboration
Zoom, a company that helped change the way people work during the COVID-19 pandemic, is continuing to reimagine the future of work by transforming itself into a...
24/10/2024
Call of Duty: Black Ops 6' Storms Into the Cloud With GeForce NOW
Attention, recruits! It's time to test combat skills and strategic prowess. Drop into the heart of the action this GFN Thursday with the launch of the highl...
23/10/2024
Healthcare Leaders Across India Bring NVIDIA NIM for Hindi Language to LLM Applications
Life sciences and healthcare organizations across India are using generative AI ...
23/10/2024
India Manufacturers Build Factory Digital Twins With NVIDIA AI and Omniverse
Manufacturers and service providers in India are adopting NVIDIA Omniverse to tap into simulation, digital twins and generative AI to accelerate their factory p...
23/10/2024
India's Robotics Ecosystem Adopts NVIDIA Isaac and Omniverse to Build Next Wave of Physical AI
In vast warehouses, Addverb's robots work tirelessly, picking, sorting and d...
23/10/2024
Open for AI: India Tech Leaders Build AI Factories for Economic Transformation
India's leading cloud infrastructure providers and server manufacturers are ramping up accelerated data center capacity. By year's end, they'll have...
23/10/2024
World's Greatest Upskill: Consulting Giants Team With NVIDIA to Transform India Into Front Office for AI Era
Information technology giants including Infosys, TCS, Tech Mahindra and Wipro ar...
23/10/2024
Start Local, Go Global: India's Startups Spur Growth and Innovation With NVIDIA Technology
India is becoming a key producer of AI for virtually every industry - powered by...
23/10/2024
NVIDIA, F5 Turbocharge Sovereign AI Cloud Security, Efficiency
To improve AI efficiency and security in sovereign cloud environments, NVIDIA and F5 are integrating NVIDIA BlueField-3 DPUs with the F5 BIG-IP Next for Kuberne...
23/10/2024
The Three Computer Solution: Powering the Next Wave of AI Robotics
ChatGPT marked the big bang moment of generative AI. Answers can be generated in response to nearly any query, helping transform digital work such as content cr...
23/10/2024
Denmark Launches Leading Sovereign AI Supercomputer to Solve Scientific Challenges With Social Impact
NVIDIA founder and CEO Jensen Huang joined the king of Denmark to launch the cou...
23/10/2024
How to Accelerate Larger LLMs Locally on RTX With LM Studio
Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and showcases new hardware, softwa...
22/10/2024
What Is Agentic AI?
AI chatbots use generative AI to provide responses based on a single interaction. A person makes a query and the chatbot uses natural language processing to rep...
22/10/2024
NVIDIA Brings Generative AI Tools, Simulation and Perception Workflows to ROS Developer Ecosystem
At ROSCon in Odense, one of Denmark's oldest cities and a hub of automation,...
21/10/2024
NVIDIA CEO Jensen Huang to Spotlight Innovation at India's AI Summit
The NVIDIA AI Summit India, taking place Oct. 23-25 at the Jio World Convention Centre in Mumbai, will bring together the brightest minds to explore how India i...
21/10/2024
NVIDIA and Microsoft Give AI Startups a Double Dose of Acceleration
NVIDIA is expanding its collaboration with Microsoft to support global AI startups across industries - with an initial focus on healthcare and life sciences com...
21/10/2024
NVIDIA Works With Deloitte to Deploy Digital AI Agents for Healthcare
Ahead of a visit to the hospital for a surgical procedure, patients often have plenty of questions about what to expect - and can be plenty nervous. To help mi...
17/10/2024
How Digital Twins Are Driving Efficiency and Cutting Emissions in Manufacturing
Improving the sustainability of manufacturing involves optimizing entire product lifecycles - from material sourcing and transportation to design, production, d...
17/10/2024
Waterways Wonder: Clearbot Autonomously Cleans Waters With Energy-Efficient AI
What started as two classmates seeking a free graduation trip to Bali subsidized by a university project ended up as an AI-driven sea-cleaning boat prototype bu...
17/10/2024
Sustainable Manufacturing and Design: How Digital Twins Are Driving Efficiency and Cutting Emissions
Improving the sustainability of manufacturing involves optimizing entire product...
17/10/2024
Get Ready to Slay: Dragon Age: The Veilguard' to Soar Into GeForce NOW at Launch
Bundle up this fall with GeForce NOW and Dragon Age: The Veilguard with a specia...
15/10/2024
We Would Like to Achieve Superhuman Productivity,' NVIDIA CEO Says as Lenovo Brings Smarter AI to Enterprises
Moving to accelerate enterprise AI innovation, NVIDIA founder and CEO Jensen Hua...
14/10/2024
MAXimum AI: RTX-Accelerated Adobe AI-Powered Features Speed Up Content Creation
At the Adobe MAX creativity conference this week, Adobe announced updates to its Adobe Creative Cloud products, including Premiere Pro and After Effects, as wel...
11/10/2024
NVIDIA AI Summit Panel Outlines Autonomous Driving Safety
The autonomous driving industry is shaped by rapid technological advancements and the need for standardization of guidelines to ensure the safety of both autono...
11/10/2024
Game-Changer: How the World's First GPU Leveled Up Gaming and Ignited the AI Era
In 1999, fans lined up at Blockbuster to rent chunky VHS tapes of The Matrix. Y2...
10/10/2024
The Next Chapter Awaits: Dive Into Diablo IV's' Latest Adventure Vessel of Hatred' on GeForce NOW
Prepare for a devilishly good time this GFN Thursday as the critically acclaimed...
10/10/2024
AI'll Be by Your Side: Mental Health Startup Enhances Therapist-Client Connections
Half of the world's population will experience a mental health disorder - bu...
09/10/2024
AI Summit: US Energy Secretary Highlights AI's Role in Science, Energy and Security
AI can help solve some of the world's biggest challenges - whether climate c...
09/10/2024
Flux and Furious: New Image Generation Model Runs Fastest on RTX AI PCs and Workstations
Editor's note: This post is part of the AI Decoded series, which demystifies...
09/10/2024
What's the ROI? Getting the Most Out of LLM Inference
Large language models and the applications they power enable unprecedented opportunities for organizations to get deeper insights from their data reservoirs and...
08/10/2024
NVIDIA AI Summit Highlights Game-Changing Energy Efficiency and AI-Driven Innovation
Accelerated computing is sustainable computing, Bob Pette, NVIDIA's vice pre...
08/10/2024
Accelerated Computing Key to Quantum Research
A recently released joint research paper by NVIDIA, Moderna and Yale reviews how techniques from quantum machine learning (QML) may enhance drug discovery metho...
08/10/2024
Pittsburgh Steels Itself for Innovation With Launch of NVIDIA AI Tech Community
Serving as a bridge for academia, industry and public-sector groups to partner on artificial intelligence innovation, NVIDIA is launching its inaugural AI Tech ...
08/10/2024
TSMC and NVIDIA Transform Semiconductor Manufacturing With Accelerated Computing
TSMC, the world leader in semiconductor manufacturing, is moving to production with NVIDIA's computational lithography platform, called cuLitho, to accelera...
08/10/2024
SETI Institute Researchers Engage in World's First Real-Time AI Search for Fast Radio Bursts
This summer, scientists supercharged their tools in the hunt for signs of life b...
08/10/2024
From Concept to Compliance, MITRE Digital Proving Ground Will Accelerate Validation of Autonomous Vehicles
The path to safe, widespread autonomous vehicles is going digital. MITRE - a go...
08/10/2024
A Not-So-Secret Agent: NVIDIA Unveils NIM Blueprint for Cybersecurity
Artificial intelligence is transforming cybersecurity with new generative AI tools and capabilities that were once the stuff of science fiction. And like many o...
08/10/2024
US Healthcare System Deploys AI Agents, From Research to Rounds
The U.S. healthcare system is adopting digital health agents to harness AI across the board, from research laboratories to clinical settings. The latest AI-acc...
07/10/2024
Foxconn to Build Taiwan's Fastest AI Supercomputer With NVIDIA Blackwell
NVIDIA and Foxconn are building Taiwan's largest supercomputer, marking a milestone in the island's AI advancement. The project, Hon Hai Kaohsiung Supe...
03/10/2024
No Tricks, Just Games: GeForce NOW Thrills With 22 Games in October
The air is crisp, the pumpkins are waiting to be carved, and GFN Thursday is ready to deliver some gaming thrills. GeForce NOW is unleashing a monster mash of ...
03/10/2024
How AI and Accelerated Computing Drive Energy Efficiency
AI isn't just about building smarter machines. It's about building a greener world. From optimizing energy use to reducing emissions, AI and accelerate...
02/10/2024
Brave New World: Leo AI and Ollama Bring RTX-Accelerated Local LLMs to Brave Browser Users
Editor's note: This post is part of the AI Decoded series, which demystifies...
01/10/2024
NVIDIA AI Summit DC: Industry Leaders Gather to Showcase AI's Real-World Impact
Washington, D.C., is where possibility has always met policy, and AI presents un...
27/09/2024
Bon Voyage: NIO Unveils ONVO L60 Smart Electric SUV, Built on NVIDIA DRIVE Orin
NIO's smart EV brand, ONVO, has unveiled the L60 flagship mid-size family SUV, built on the NVIDIA DRIVE Orin system-on-a-chip. Earlier this year, the auto...
26/09/2024
A Whole New World: GreedFall II: The Dying World' Joins GeForce NOW
Whether looking for a time-traveling adventure, strategic roleplay or epic action, anyone can find something to play on GeForce NOW, with over 2,000 games in th...