The pace of technology innovation has accelerated in the past year, most dramatically in AI. And in 2024, there was no better place to be a part of creating those breakthroughs than NVIDIA Research.
NVIDIA Research is comprised of hundreds of extremely bright people pushing the frontiers of knowledge, not just in AI, but across many areas of technology.
In the past year, NVIDIA Research laid the groundwork for future improvements in GPU performance with major research discoveries in circuits, memory architecture and sparse arithmetic. The team's invention of novel graphics techniques continues to raise the bar for real-time rendering. And we developed new methods for improving the efficiency of AI - requiring less energy, taking fewer GPU cycles and delivering even better results.
But the most exciting developments of the year have been in generative AI.
We're now able to generate, not just images and text, but 3D models, music and sounds. We're also developing better control over what is generated: to generate realistic humanoid motion and to generate sequences of images with consistent subjects.
The application of generative AI to science has resulted in high-resolution weather forecasts that are more accurate than conventional numerical weather models. AI models have given us the ability to accurately predict how blood glucose levels respond to different foods. Embodied generative AI is being used to develop autonomous vehicles and robots.
And that was just this year. What follows is a deeper dive into some of NVIDIA Research's greatest generative AI work in 2024. Of course, we continue to develop new models and methods for AI, and expect even more exciting results next year.
ConsiStory: AI-Generated Images With Main Character Energy ConsiStory, a collaboration between researchers at NVIDIA and Tel Aviv University, makes it easier to generate multiple images with a consistent main character - an essential capability for storytelling use cases such as illustrating a comic strip or developing a storyboard.
The researchers' approach introduced a technique called subject-driven shared attention, which reduces the time it takes to generate consistent imagery from 13 minutes to around 30 seconds.
Read the ConsiStory paper.
ConsiStory is capable of generating a series of images featuring the same character. Edify 3D: Generative AI Enters a New Dimension NVIDIA Edify 3D is a foundation model that enables developers and content creators to quickly generate 3D objects that can be used to prototype ideas and populate virtual worlds.
Edify 3D helps creators quickly ideate, lay out and conceptualize immersive environments with AI-generated assets. Novice and experienced content creators can use text and image prompts to harness the model, which is now part of the NVIDIA Edify multimodal architecture for developing visual generative AI.
Read the Edify 3D paper and watch the video on YouTube.
Fugatto: Flexible AI Sound Machine for Music, Voices and More A team of NVIDIA researchers recently unveiled Fugatto, a foundational generative AI model that can create or transform any mix of music, voices and sounds based on text or audio prompts.
The model can, for example, create music snippets based on text prompts, add or remove instruments from existing songs, modify the accent or emotion in a voice recording, or generate completely novel sounds. It could be used by music producers, ad agencies, video game developers or creators of language learning tools.
Read the Fugatto paper.
GluFormer: AI Predicts Blood Sugar Levels Four Years Out Researchers from the Weizmann Institute of Science, Tel Aviv-based startup Pheno.AI and NVIDIA led the development of GluFormer, an AI model that can predict an individual's future glucose levels and other health metrics based on past glucose monitoring data.
The researchers showed that, after adding dietary intake data into the model, GluFormer can also predict how a person's glucose levels will respond to specific foods and dietary changes, enabling precision nutrition. The research team validated GluFormer across 15 other datasets and found it generalizes well to predict health outcomes for other groups, including those with prediabetes, type 1 and type 2 diabetes, gestational diabetes and obesity.
Read the GluFormer paper.
LATTE3D: Enabling Near-Instant Generation, From Text to 3D Shape Another 3D generator released by NVIDIA Research this year is LATTE3D, which converts text prompts into 3D representations within a second - like a speedy, virtual 3D printer. Crafted in a popular format used for standard rendering applications, the generated shapes can be easily served up in virtual environments for developing video games, ad campaigns, design projects or virtual training grounds for robotics.
Read the LATTE3D paper.
MaskedMimic: Reconstructing Realistic Movement for Humanoid Robots To advance the development of humanoid robots, NVIDIA researchers introduced MaskedMimic, an AI framework that applies inpainting - the process of reconstructing complete data from an incomplete, or masked, view - to descriptions of motion.
Given partial information, such as a text description of movement, or head and hand position data from a virtual reality headset, MaskedMimic can fill in the blanks to infer full-body motion. It's become part of NVIDIA Project GR00T, a research initiative to accelerate humanoid robot development.
Read the MaskedMimic paper.
StormCast: Boosting Weather Prediction, Climate Simulation In the field of climate science, NVIDIA Research announced StormCast, a generative AI model for emulating atmospheric dynamics. While other machine learning models trained on global data have a spatial resolution of about 30 kilometers and a temporal resolution of six hours, StormCast achieves a 3-kilometer, hourly scale.
Most recent headlines
02/01/2025
Creator Economy Isn't Really Like Film/Show Entertainment
Andy Marken January 1, 2025
0 Comments
Come on Barry, you're good at this sort of s...
02/01/2025
Blackmagic Design Announces New Lower Price for Blackmagic Pocket Cinema Camera ...
02/01/2025
Hollywood Lighting Hacks: Big Looks, Tiny Budget
Sean Alami January 1, 2025
0 Comments
Discover how to create stunning Hollywood-style lighting using ...
01/01/2025
Marcy Lefkovitz, Dalet SVP product innovation, details the findings of the companys report into the future of newsrooms, including leveraging AI, optimising mul...
31/12/2024
Rashad Frett shares a moment with Michelle Satter at the 2023 Directors Lab. Pho...
31/12/2024
What's Ahead in 2025? OWC's Larry O'Connor Offers Some Bold Predicti...
31/12/2024
Blackmagic Design Announces New Features for Blackmagic Cloud
Brie Clayton December 31, 2024
0 Comments
New thumbnail icon view, clip metadata inspect...
31/12/2024
5 Features in DaVinci Resolve You're Not Using (But You Should Be)
Kasia Jarco December 31, 2024
0 Comments
In this video we are going to talk abo...
31/12/2024
Back to All News
NFL Christmas Day Games on Netflix Average Over 30 Million Global Viewers
Entertainment
31 December 2024
Global
Link copied to clipboard
...
31/12/2024
Back to All News
Squid Game' Season 2 Smashes Top 10 Records as Millions W...
30/12/2024
Leaving Thomson after 21 years, former Managing Director David Quin reflects on a career spent navigating the shifting sands of global media development. From r...
30/12/2024
[Pictured: a still from Love Lies Bleeding, one of the 100 Sundance-supported titles that opened to wider audiences in 2024]...
30/12/2024
Digital Domain Creates Groundbreaking VFX for Venom: The Last Dance, Bringing ...
30/12/2024
Internet Giant GMP Builds Tokyo Virtual Event Space with URSA Broadcast G2
Brie Clayton December 30, 2024
0 Comments
Blackmagic URSA Broadcast G2 and ...
30/12/2024
Renderboxes launches Photon, a no-compromise transportable high-performance work...
30/12/2024
Shinobi II Monitor Supports Canon, Sony, Nikon - Touch to Focus with New Feature...
30/12/2024
30 Dec 2024
In a Groundbreaking Agreement, VEON and Starlink to Bring Starlink ...
30/12/2024
The pace of technology innovation has accelerated in the past year, most dramati...
29/12/2024
Gyeongnam Culture and Arts Foundation (GCAF), an active promotor of cultural and artistic diversity, has chosen Ikegami UHK-X700 camera chains for integration i...
27/12/2024
After starting 2024 with hopes that the transition to streaming might finally produce a more stable, profitable businesses, the media and entertainment industry...
27/12/2024
Broadcasters ended the year with hopes and fears about how the federal government will regulate the TV industry in 2025. While the incoming Trump administration...
27/12/2024
NVIDIA's AI Podcast gives listeners the inside scoop on the ways AI is transforming nearly every industry. Since the show's debut in 2016, it's gar...
26/12/2024
Talking about artificial intelligence in 2024 is akin to talking about the internet in 1994 everyone's excited about it and thinks it will have an enormous ...
26/12/2024
Back to All News
Netflix NFL Christmas Gameday Reaches 65 Million US Viewers
Entertainment
26 December 2024
GlobalUnited States
Link copied to clipboard
U...
26/12/2024
Back to All News
Check Out All the Must-See Trailers From Netflix's NFL Chr...
26/12/2024
Back to All News
Beyonc Delivers Epic Gift of a Halftime Performance "Beyonc...
26/12/2024
Back to All News
Get in the Game With Squid Game' Season 2 Complete Coverage Guide
Entertainment
26 December 2024
Global
Link copied to clipboard
Are...
26/12/2024
This GFN Thursday wraps up another incredible year for cloud gaming. Take a look back at the top games and new features that made 2024 a standout for GeForce NO...
24/12/2024
The National Film and Video Foundation (NFVF) is calling for applications from South African film and television filmmakers to be part of the NFVF's delegat...
24/12/2024
WASHINGTON, D.C. The Federal Communications Commission Enforcement Bureau said Paramount Global is being fined $244,952 for violations of emergency alerting rul...
24/12/2024
NEW YORK Carnegie Hall's video-streaming channel, Carnegie Hall+, said it will celebrate the holiday season by offering a wide selection of holiday-themed o...
24/12/2024
LOS ANGELES PlayersTV, an athlete and fan-owned media company, has announced the acquisition of Cloud Media Center, an AI-driven sports adtech and media distrib...
24/12/2024
The Best Playlists of 2024 The top playlists we created in 2024, from dad rock to dembow.
By
Tara Bellucci
December 23, 2024
Image by Kelly Davidson
Tak...
24/12/2024
Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and showcases new hardware, softwa...
23/12/2024
At IBC2024, Grass Valley CTO Ian Fletcher told attendees at GV Forum that broadcast is now an IT industry. During our discussions at the show, TVBEurope heard v...
23/12/2024
According to co-director Nick Park, the production team embraced technology to help create the iconic duos latest adventure
By Jenny Priestley
Published: Dec...
23/12/2024
VFX house Lux Aeterna used wave height data from the National Oceanic & Atmosphe...
23/12/2024
We are committed to supporting and investing in Foxtel's television and streaming services, across both sports and entertainment, using our world-leading te...
23/12/2024
From AI to IP, the cloud to virtual production, key industry executives take a look back at some of the biggest trends that impacted the media and entertainment...
23/12/2024
Boost Graphics, the specialist international graphics and virtual production subsidiary of EMG / Gravity Media, the leading force in production and content & me...
23/12/2024
LucidLink, the leading storage collaboration platform revolutionizing how global teams work, proudly announces its achievement of the prestigious Trusted Partne...
23/12/2024
Mediagenix, a leader in software solutions for content strategy, content value management, and content scheduling, announces the appointment of Bruno Langlais a...
23/12/2024
WCTEPublic Media Venture Group (PMVG) has announced that it is now providing real-time translation of closed captioning from English to Spanish on PMVG's Ne...
23/12/2024
Deltatre, the tech company behind the media and sporting moments that matter, has appointed Charlie Mitchell as Head of Sports for the Americas. Charlie will sp...
23/12/2024
Media Excel, a global leader in real-time video solutions, proudly announces the appointment of Matthew Goldman to its Technical Advisory Board. A distinguished...
23/12/2024
Disguise the company behind the virtual production technology used on commercials for Apple Music and Lenovo, as well as feature film Daddio starring Sean Pen...
23/12/2024
Leading AI and video search company Moments Lab is pleased to announce its integration with Bitcentral. The integration enables Bitcentral Oasis MAM users to en...
23/12/2024
Amagi, the global leader in cloud-based SaaS technology for broadcast and connected TV (CTV), today announced that ABC Commercial has partnered with Amagi to la...
23/12/2024
Digital Alert Systems, the global leader in emergency communications solutions for media providers, today announced that Ed Czarnecki, the company's vice pr...
The pace of technology innovation has accelerated in the past year, most dramatically in AI. And in 2024, there was no better place to be a part of creating those breakthroughs than NVIDIA Research.
