NVIDIA Research Achieves AI Training Breakthrough Using Limited Datasets
08/12/2020
By applying a breakthrough neural network training technique to the popular NVIDIA StyleGAN2 model, NVIDIA researchers reimagined artwork based on fewer than 1,500 images from the Metropolitan Museum of Art. Using NVIDIA DGX systems to accelerate training, they generated new AI art inspired by the historical portraits.
The technique - called adaptive discriminator augmentation, or ADA - reduces the number of training images by 10-20x while still getting great results. The same method could someday have a significant impact in healthcare, for example by creating cancer histology images to help train other AI models.
These results mean people can use GANs to tackle problems where vast quantities of data are too time-consuming or difficult to obtain, said David Luebke, vice president of graphics research at NVIDIA. I can't wait to see what artists, medical experts and researchers use it for.
The research paper behind this project is being presented this week at the annual Conference on Neural Information Processing Systems, known as NeurIPS. It's one of a record 28 NVIDIA Research papers accepted to the prestigious conference.
This new method is the latest in a legacy of GAN innovation by NVIDIA researchers, who've developed groundbreaking GAN-based models for the AI painting app GauGAN, the game engine mimicker GameGAN, and the pet photo transformer GANimal. All are available on the NVIDIA AI Playground.
The Training Data Dilemma Like most neural networks, GANs have long followed a basic principle: the more training data, the better the model. That's because each GAN consists of two cooperating networks - a generator, which creates synthetic images, and a discriminator, which learns what realistic images should look like based on training data.
The discriminator coaches the generator, giving pixel-by-pixel feedback to help it improve the realism of its synthetic images. But with limited training data to learn from, a discriminator won't be able to help the generator reach its full potential - like a rookie coach who's experienced far fewer games than a seasoned expert.
It typically takes 50,000 to 100,000 training images to train a high-quality GAN. But in many cases, researchers simply don't have tens or hundreds of thousands of sample images at their disposal.
With just a couple thousand images for training, many GANs would falter at producing realistic results. This problem, called overfitting, occurs when the discriminator simply memorizes the training images and fails to provide useful feedback to the generator.
In image classification tasks, researchers get around overfitting with data augmentation, a technique that expands smaller datasets using copies of existing images that are randomly distorted by processes like rotating, cropping or flipping - forcing the model to generalize better.
But previous attempts to apply augmentation to GAN training images resulted in a generator that learned to mimic those distortions, rather than creating believable synthetic images.
A GAN on a Mission NVIDIA Research's ADA method applies data augmentations adaptively, meaning the amount of data augmentation is adjusted at different points in the training process to avoid overfitting. This enables models like StyleGAN2 to achieve equally amazing results using an order of magnitude fewer training images.
As a result, researchers can apply GANs to previously impractical applications where examples are too scarce, too hard to obtain or too time-consuming to gather into a large dataset.
Different editions of StyleGAN have been used by artists to create stunning exhibits and produce a new manga based on the style of legendary illustrator Osamu Tezuka. It's even been adopted by Adobe to power Photoshop's new AI tool, Neural Filters.
With less training data required to get started, StyleGAN2 with ADA could be applied to rare art, such as the work by Paris-based AI art collective Obvious on African Kota masks.
Another promising application lies in healthcare, where medical images of rare diseases can be few and far between because most tests come back normal. Amassing a useful dataset of abnormal pathology slides would require many hours of painstaking labeling by medical experts.
Synthetic images created with a GAN using ADA could fill that gap, generating training data for another AI model that helps pathologists or radiologists spot rare conditions on pathology images or MRI studies. An added bonus: With AI-generated data, there are no patient data or privacy concerns, making it easier for healthcare institutions to share datasets.
NVIDIA Research at NeurIPS The NVIDIA Research team consists of more than 200 scientists around the globe, focusing on areas including AI, computer vision, self-driving cars, robotics and graphics. Over two dozen papers authored by NVIDIA researchers will be highlighted at NeurIPS, the year's largest AI research conference, taking place virtually from Dec. 6-12.
Check out the full lineup of NVIDIA Research papers at NeurIPS.
Main images generated by StyleGAN2 with ADA, trained on a dataset of fewer than 1,500 images from the Metropolitan Museum of Art Collection API.
LINK: | https://blogs.nvidia.com/blog/2020/12/07/neurips-research-limited-data... |
See more stories from nvidia |
Most recent headlines
09/12/2024
Dalet Named an IDC Innovator in Media and Entertainment
Dalet, a leading technology and service provider for media-rich organizations, today announced that it has been named an IDC Innovator in the IDC Innovators: ...
09/11/2024
Dalet Expands Leadership Team to Fuel Next Stage of Growth
Dalet, a leading technology and service provider for media-rich organizations, today announced three new members of its executive team. Tara Bryant joins as Chi...
30/10/2024
Drone detection just got a whole lot smarter with ARDRONIS Detect from Rohde & Schwarz
Drone detection just got a whole lot smarter with ARDRONIS Detect from Rohde & S...
26/10/2024
Kelsea Ballerini and Spotify Host Intimate Listening Party To Celebrate Her New Album, PATTERNS'
I love New York City, but I can't stay at the Carlyle anymore, sings Kelsea...
26/10/2024
USA Cheer Adopts NBC Sports Next's SportsEngine Motion
MINNEAPOLIS NBC Sports Next has announced a multiyear technology agreement with USA Cheer, the governing body for sport cheering in the U.S. The organization an...
26/10/2024
Canada's MSCTV Chooses Witbe Virtual NOC for Monitoring
NEW YORK Witbe has announced that Mitchell Seaforth Cable TV (MSCTV), a Dublin, Ontario, Canada-based provider of solutions to telecommunications companies, is ...
26/10/2024
IAB Tech Lab Launches Ad Format Idol' Initiative
NEW YORK The IAB Tech Lab has launched Ad Format Idol, an initiative to standardize emerging connected TV ad formats....
26/10/2024
T-Mobile Pledges Up to $1 Million for Ongoing Hurricane Recovery Efforts
BELLEVUE, Wash. T Mobile has announced a Rally for Relief effort to help those impacted by recent hurricanes. As part of the campaign, the cellphone carrier sai...
26/10/2024
Comercio TV Taps Broadpeak for OTT Video Delivery and Monetization
CESSON-SEVIGNE, France Broadpeak said Comercio TV, the Spanish-language OTT financial-news channel, has adopted its's software-as-a-service (SaaS) platform ...
26/10/2024
Africana Studies at Berklee to Induct Kid 'n Play into Berklee's Hip-Hop Hall of Fame
Africana Studies at Berklee to Induct Kid 'n Play into Berklee's Hip-Hop...
25/10/2024
What to Watch: These 7 DFP-Supported Films From 2024 Will Inspire You to Organize for Change
It's easy to become overwhelmed when thinking about the current state of the...
25/10/2024
Fall in Love With Your Monster
PARK CITY, UTAH - JANUARY 18: (L-R) Tommy Dewey, Melissa Barrera, Kayla Foster, Meghann Fahy, and Edmund Donovan attend the 2024 Sundance Film Festival Your Mo...
25/10/2024
How Matthew Belloni Preps for a Live Recording of His Hit Ringer Podcast, The Town'
Curious about what's going on behind the scenes in Hollywood? Odds are Matth...
25/10/2024
SBS celebrates the Festivals of Lights with distinctive content to entertain all Australians
SBS celebrates the Festivals of Lights with distinctive content to entertain all...
25/10/2024
L3Harris Technologies Reports Strong Third Quarter 2024 Results, Increases 2024 Guidance
Highlights* Orders of $7.2 billion; book-to-bill of 1.4x Revenue of $5.3 billi...
25/10/2024
L3Harris, Palantir CEOs Discuss Strategic Partnership on CNBC's "Closing Bell: Overtime"
On Oct. 24, Chris Kubasik and Alex Karp sat down with CNBC's Morgan Brennan ...
25/10/2024
Meta Quest to Offer 52 NBA Games in VR for 2024-2025 Season
Meta Quest, a division of Facebook owner Meta, has announced it will continue its virtual reality broadcasts of regular-season NBA games for the fourth consecut...
25/10/2024
Disney Expected to Announce Major AI Initiative'
Disney one of the world's largest owners of intellectual property is getting ready to unveil a major AI initiative that will focus primarily on post-produ...
25/10/2024
Broadcast Solutions establishes CTO Office to drive innov...
Broadcast Solutions, one of Europe's largest media systems integration groups, has extended its technical leadership with two key appointments. Pierre Mestr...
25/10/2024
Unified Streaming fortifies customer ties with two new hi...
Unified Streaming, a leading streaming software company, has hired Valentijn Siebrands, an industry veteran, as its new streaming solutions manager. The company...
25/10/2024
Imagine Communications John Mailhot Receives 2024 David S...
Imagine Communications' John Mailhot was awarded the prestigious 2024 David Sarnoff Medal during SMPTE's annual Awards Gala, held on October 24 at the R...
25/10/2024
Independent Midwest Broadcaster selects Rohde & Schwarz TE1 liquid-cooled transmitter to expand audience reach
Independent Midwest Broadcaster selects Rohde & Schwarz TE1 liquid-cooled transm...
25/10/2024
Mexico's First Virtual Production Game Show Produced with Blackmagic Design
Mexico's First Virtual Production Game Show Produced with Blackmagic Design Brie Clayton October 25, 2024 0 Comments Club Cin polis Desaf o India...
25/10/2024
Using invisible effects to hide a stage light in Adobe After Effects
Using invisible effects to hide a stage light in Adobe After Effects Graham Quince October 25, 2024 0 Comments Lasers, dinosaurs and space VFX are coo...
25/10/2024
Improve your Color Grading using these simple steps in DaVinci Resolve 19
Improve your Color Grading using these simple steps in DaVinci Resolve 19 Kasia Jarco October 25, 2024 0 Comments Do you truly understand how to color...
25/10/2024
Brighter Eyes
Jenny Priestley talks to the BFI's Douglas Weir who oversaw the restoration of a much-loved animated classic, which returns to UK and Ireland cinemas today ...
25/10/2024
Comercio TV Chooses Broadpeak to Optimize OTT Video Deliv...
Broadpeak , a leading provider of content delivery network (CDN) and video streaming solutions for content providers and pay-TV operators worldwide, announced t...
25/10/2024
Amagi Partners With Allrites to Expand the Global Distrib...
Amagi, the global leader in cloud-based SaaS technology for broadcast and connected TV (CTV), has announced a strategic partnership with Allrites, a premium con...
25/10/2024
Studio Technologies Enhances ST 2110 Integration with Mer...
Studio Technologies, Inc., a pioneer in high-performance audio, video, and fibre-optic solutions, has long been at the forefront of the professional broadcast a...
25/10/2024
QuickLink Improves Remote Production for Saturday Night T...
QuickLink, empowering creatives to exceed modern-day standards of production, has been instrumental in easing the remote production challenges of ITV Studios...
25/10/2024
stYpe AI Glue and StypeCentral Advance the Art of TV Tale...
stYpe, a globally active innovator of virtual studio production resources, reports strong interest in the latest additions to its wide range of broadcast-qualit...
25/10/2024
Top Argentine Music Facility Casa Frida Estudio Chooses P...
Casa Frida Estudio, one of the most prestigious music facilities in Argentina, has upgraded its main control room to Dolby Atmos specifications and installed a ...
25/10/2024
Frequency expands Managed Channel Services to Europe at M...
Frequency, the engine behind many of the world's best-known streaming television channels, announces the European expansion of its Managed Channel Services ...
25/10/2024
InnovateUK and EU Approve Funding for Immersive Technolog...
Disguise, an HPA-Award-winning immersive experience company, has received £1.2 Million in funding from Innovate UK and the EU as part of a larger investment int...
25/10/2024
MASORANGE AND AGILE CONTENT RENEW THEIR COLLABORATION AGR...
Through this agreement, MASORANGE will incorporate new AGILE CONTENT technologies into its television service, enhancing customer experience with optimum servic...
25/10/2024
Swiss Indoors Basel 2024 delivered with Appear X Platform
Appear, a global leader in live production technology, is proud to announce its role as an official provider of cutting-edge broadcast transmission solutions fo...
25/10/2024
MSCTV Chooses Witbes Virtual NOC for Proactive Live TV Mo...
Witbe (Euronext Growth FR0013143872 ALWIT), a global leader in test automation and monitoring technology for video service providers, today announced that M...
25/10/2024
Unified Streaming fortifies customer ties with two hires
Unified Streaming, a leading streaming software company, has hired Valentijn Siebrands, an industry veteran, as its new streaming solutions manager. The company...
25/10/2024
Independent Midwest Broadcaster selects Rohde and Schwarz...
Competitive search leads Toledo's WLMB-TV 40 to TE1 based on energy efficiency and durability As part of its effort to reach a wider audience, the Toledo-b...
25/10/2024
IBC Accelerator Programme 2025 - Call For Industry Challe...
IBC announces the Call for Challenges for the 2025 Accelerator Media Innovation Programme, inviting proposals for transformative, fast-track innovation projects...
25/10/2024
TVN Deploys Lawo AoIP Technology in New OB Truck
HANNOVER, Germany TVN Live Production is using a variety of solutions and technologies from Lawo in its new TVN- 8UHD OB truck, which it unveiled at this year...
25/10/2024
LG Ad Solutions Joins Innovid's Harmony Initiative
NEW YORK Innovid, a software platform for the creation, delivery, measurement and optimization of connected TV (CTV), linear and digital advertising, said LG Ad...
25/10/2024
Hightouch Launches New CTV Advertising Offering
SAN FRANCISCO Consumer data platform provider Hightouch has announced a new solution it said will streamline and accelerate connected TV (CTV) advertising....
25/10/2024
Max Adds DAZN's Sports Offering in Spain
MADRID Warner Bros. Discovery's streaming service Max has reached a new agreement with DAZN that will give Max customers in Spain access to the premium spor...
25/10/2024
Beloved Genre-Benders Karate Make a Fitting Comeback
Beloved Genre-Benders Karate Make a Fitting Comeback Its one of the decades feel-good music stories: How the legendary alumni acts rediscovery led to rereleas...
25/10/2024
Berklee Presents Holiday Extravaganza: Duke Ellingtons Nutcracker Suite and Christmas Vibes Featuring Warren Wolf
Berklee Presents Holiday Extravaganza: Duke Ellingtons Nutcracker Suite and Chri...
25/10/2024
Canadian engineer Elias Ghannoum wins 2024 Lord Kelvin Award
The IEC has awarded its annual Lord Kelvin Award to the Canadian engineer, Elias Ghannoum. The award recognizes exceptional and long-term contributions to globa...
25/10/2024
Dolby and Jin from BTS Celebrate His New Single I'll Be There in Dolby Atmos
25 October 2024, 16:00 (PDT) Dolby and Jin from BTS Celebrate His New Single I'll Be There in Dolby Atmos Love Jin More In Dolby | #LoveMoreInDolby F...
25/10/2024
JazzCash and Dgpays Join Forces to Enhance Marketplace Payments in Pakistan
25 Oct 2024 JazzCash and Dgpays Join Forces to Enhance Marketplace Payments in Pakistan Amsterdam and Dubai, 25 October 2024: VEON Ltd. (Nasdaq: VEON, Euronext...
25/10/2024
Monthly Rewind: Groundbreaking Colour Innovation, Unreal Engine, The Academy Museum, and more.
October has been a wicked month across Technicolor Group! So much so, we introdu...