Digital Renaissance: NVIDIA Neuralangelo Research Reconstructs 3D Scenes
01/06/2023
Like Michelangelo sculpting stunning, life-like visions from blocks of marble, Neuralangelo generates 3D structures with intricate details and textures. Creative professionals can then import these 3D objects into design applications, editing them further for use in art, video game development, robotics and industrial digital twins.
Neuralangelo's ability to translate the textures of complex materials - including roof shingles, panes of glass and smooth marble - from 2D videos to 3D assets significantly surpasses prior methods. The high fidelity makes its 3D reconstructions easier for developers and creative professionals to rapidly create usable virtual objects for their projects using footage captured by smartphones.
The 3D reconstruction capabilities Neuralangelo offers will be a huge benefit to creators, helping them recreate the real world in the digital world, said Ming-Yu Liu, senior director of research and co-author on the paper. This tool will eventually enable developers to import detailed objects - whether small statues or massive buildings - into virtual environments for video games or industrial digital twins.
In a demo, NVIDIA researchers showcased how the model could recreate objects as iconic as Michelangelo's David and as commonplace as a flatbed truck. Neuralangelo can also reconstruct building interiors and exteriors - demonstrated with a detailed 3D model of the park at NVIDIA's Bay Area campus.
Neural Rendering Model Sees in 3D Prior AI models to reconstruct 3D scenes have struggled to accurately capture repetitive texture patterns, homogenous colors and strong color variations. Neuralangelo adopts instant neural graphics primitives, the technology behind NVIDIA Instant NeRF, to help capture these finer details.
Using a 2D video of an object or scene filmed from various angles, the model selects several frames that capture different viewpoints - like an artist considering a subject from multiple sides to get a sense of depth, size and shape.
Once it's determined the camera position of each frame, Neuralangelo's AI creates a rough 3D representation of the scene, like a sculptor starting to chisel the subject's shape.
The model then optimizes the render to sharpen the details, just as a sculptor painstakingly hews stone to mimic the texture of fabric or a human figure.
The final result is a 3D object or large-scale scene that can be used in virtual reality applications, digital twins or robotics development.
Find NVIDIA Research at CVPR, June 18-22 Neuralangelo is one of nearly 30 projects by NVIDIA Research to be presented at the Conference on Computer Vision and Pattern Recognition (CVPR), taking place June 18-22 in Vancouver. The papers span topics including pose estimation, 3D reconstruction and video generation.
One of these projects, DiffCollage, is a diffusion method that creates large-scale content - including long landscape orientation, 360-degree panorama and looped-motion images. When fed a training dataset of images with a standard aspect ratio, DiffCollage treats these smaller images as sections of a larger visual - like pieces of a collage. This enables diffusion models to generate cohesive-looking large content without being trained on images of the same scale.
The technique can also transform text prompts into video sequences, demonstrated using a pretrained diffusion model that captures human motion:
Learn more about NVIDIA Research at CVPR.
LINK: | https://blogs.nvidia.com/blog/2023/06/01/neuralangelo-ai-research-3d-r... |
See more stories from nvidia |
Most recent headlines
04/08/2024
Dalet Appoints Santiago Solanas as CEO to Lead Next Era of Growth and Innovation
Dalet, a leading technology and service provider for media-rich organizations, is excited to announce Santiago Solanas as its new Chief Executive Officer (CEO)....
03/06/2024
Dalet and Veritone Reach Agreement to Distribute, Transact and Monetize Media Archives
Dalet, a leading technology and service provider for media-rich organizations, a...
02/06/2024
AE add 3D motion to still photographs
AE add 3D motion to still photographs Brie Clayton June 2, 2024 0 Comments AE add 3D motion to still photographs May 30, 2024COW Featured Resume: Pa...
01/06/2024
How Spotify is Protecting Election Integrity in 2024
Dustee Jenkins, Chief Public Affairs Officer, Spotify With billions of people from over 50 countries heading to the polls to cast their vote, 2024 is shaping u...
01/06/2024
TV Tech Weekly Product News Wrap-Up
Busy this week catching up from the Memorial Day holiday? Our TV Tech weekly wrap-up of all our coverage of new products, services and deployments will get you ...
01/06/2024
Guy Ritchie Behind Young Sherlock' Series on Prime Video
Young Sherlock, with Guy Ritchie directing and executive producing, is a go at Prime Video. Hero Fiennes Tiffin plays Sherlock. Prime Video calls it the origin...
01/06/2024
Caitlin Clark-Cameron Brink WNBA Matchup Draws 724,000 Viewers
The Los Angeles Sparks-Indiana Fever game on Friday featuring the top two WNBA draft picks, Caitlin Clark and Cameron Brink, was the most watched WNBA game on I...
01/06/2024
Lionsgate Names Brian Weinstein Co-CEO of 3 Arts Entertainment
Lionsgate, which increased its majority stake in 3 Arts Entertainment in January, said it named Brian Weinstein co-CEO of 3 Arts....
01/06/2024
Netflix to Launch Video Games For Perfect Match', Too Hot to Handle', Other Unscripted Hits
Netflix is launching four video games based on unscripted series this year. They...
01/06/2024
Broadcast Networks Go Live for Donald Trump Verdict
ABC, CBS, NBC and Fox affiliates switched from their regular programming to carry live coverage of the verdicts in the porn star hush-money trial of former Pres...
31/05/2024
Orange County Expansion Helps Aerojet Rocketdyne Boost Solid Rocket Motor Production
A U.S. Marine fires an FIM-92 Stinger missile during a training exercise in Yuma...
31/05/2024
IAB Tech Lab Expands Open Measurement SDK Capabilities to Support Samsung and LG TVs
NEW YORK The global body that sets technical standards for digital advertising h...
31/05/2024
Study: Max, Apple TV+ Have Fewest 'Committed Customers'
In a new analysis of what services might make up a good bundle and what services might benefit the most from bundling, Antenna has released new data that highli...
31/05/2024
Report: Older SVOD Subscribers Unfazed by Price Hikes Will Drive Revenue Growth
Global SVOD revenues will increase by $116.6 billion by 2031, a growth rate of 13.3% per year over the next eight years, according to a new report from MIDiA Re...
31/05/2024
FCC Media Bureau Incorporates 2020 Census Data into TVStudy Software
WASHINGTON, D.C. The FCC Media Bureau has announced that effective August 1, 2024 the Commission's TVStudy software will incorporate the most recent U.S. Ce...
31/05/2024
NBCU's FAST Channel TNBC to Rebrand as NBC Comedy Vault
NBCUniversal has announced that its FAST channel TNBC will be rebranding to NBC Comedy Vault on June 1 and that the free streaming service will officially launc...
31/05/2024
TVRI Taps Ateme for Transition to 4K UHD OTT Streaming
PARIS Ateme has announced that the Indonesian public broadcaster TVRI is making the transition to 4K UHD OTT streaming using Ateme technologies....
31/05/2024
Marlow Film Studios considering next steps' as council blocks plans
The proposal had received support from James Cameron, Andy Serkis, Paul Greengrass, Richard Curtis, and Jeremy Irons By Matthew Corrigan Published: May 31, 2...
31/05/2024
Skydance submits new Paramount Global bid, Sony talks still progressing'
According to multiple reports, the new bid from Skydance offers a better deal to Paramounts Class B shareholders, who were said to be unhappy with the companys ...
31/05/2024
2024 Rise Awards open for entries
The awards recognise women and companies who are stand-out performers, innovators and leaders in the broadcast media technology industry By Matthew Corrigan ...
31/05/2024
Samsung, LG Adopt IAB Software Kit for CTV Ad Measurement
NEW YORK The global body that sets technical standards for digital advertising has expanded the reach of its measurement software development kit to include Sam...
31/05/2024
The Old Investment Cycles of Broadcast Technology are Being Run Over'
The rise of digital platforms is fundamentally changing the broadcast newsroom and its investment in technology according to Jon Roberts, director of technology...
31/05/2024
Study: Max, Apple TV Have Fewest 'Committed Customers'
In a new analysis of what services might make up a good bundle and what services might benefit the most from bundling, Antenna has released new data that highli...
31/05/2024
State Broadcasters Urge Passage of Broadcast VOICES Act
WASHINGTON, D.C. Fifty state broadcasters associations, including the District of Columbia and the Commonwealth of Puerto Rico, have sent a letter to Congressio...
31/05/2024
EVS Names Richard Katz SVP Operations for NALA region
EVS has announced the appointment of Richard Katz as senior vice president of operations for the North and Latin American (NALA) region. This appointment is par...
31/05/2024
CommScope to Buy Cable Business Assets of Casa Systems for $45.1M
CommScope has announced that it was selected by Casa Systems, Inc. as the highest to acquire Casa's cable business assets and that the two companies have en...
31/05/2024
NAB Withdraws De Niro's Service Award
Robert De Niro won't be receiving the Service to America Leadership Award from the NAB Leadership Foundation after all....
31/05/2024
Viant Integrates With Google Cloud's BigQuery Clean Rooms
Viant Technology said it formed an integration with Google Cloud's BigQuery data clean rooms...
31/05/2024
The CW To Broadcast Snoop Dogg Arizona Bowl
The CW said it will broadcast the 2024 Snoop Dogg Arizona Bowl presented by Gin & Juice by Dre and Snoop Last year, The CW aired the game when it was called Bar...
31/05/2024
Allen Media Makes Carriage Deal With Amazon Fire Channels
Allen Media Group said it made a deal with Amazon's Fire TV Channel that will make content from several of Allen's outlet available via Fire TV and Echo...
31/05/2024
Robert De Niro Will Not Receive Service to America Award
Robert De Niro, announced as the winner of the NAB's Leadership Foundation's Service to America Leadership Award on May 28, will not get the award. The ...
31/05/2024
Comcast Technology Solutions To Manage Channels for AccuWeather
Comcast Technology Solutions was selected by AccuWeather to create, manage and distribute linear and over-the-top video channels....
31/05/2024
Attention to Connected TV Ads Grew in Q1, TVision Study Finds
Connected TV is becoming a bigger deal for consumers and advertisers, according to a new report from TVision....
31/05/2024
A Guitar Solo Trick You've Never Heard Before
By Craig Anderton Let's get right to what this sounds like. It's not quite feedback or tape reverse, it's well, listen to what it does in this blue...
31/05/2024
VEON discloses April YTD trading update at its AGM
31 May 2024 VEON discloses April YTD trading update at its AGM Amsterdam, 31 May 2024 07:00 CEST - VEON April YTD 2024 Highlights Total revenue of USD 1,278 ...
31/05/2024
VEON Announces Its New Board, Names Founder Augie Fabela as Chairman
31 May 2024 VEON Announces Its New Board, Names Founder Augie Fabela as Chairman Amsterdam, 31 May 2024, 16.00 CET: VEON Ltd. (NASDAQ: VEON, Euronext Amsterdam...
31/05/2024
Tribeca Festival and Open AI Announce Sora Shorts
May 31st, 2024 TRIBECA FESTIVAL AND OPEN AI ANNOUNCE SORA SHORTS' Five Commissioned Short Films Created Using Sora Debut at 2024 Tribeca Festival New ...
31/05/2024
In-Venue Technology Wrap-Up: How 4K, IP, and More are Changing the Game Pres Game
In-Venue Technology Wrap-Up: How 4K, IP, and More are Changing the Game Pres Gam...
31/05/2024
UEFA Champions League Final 2024: The Facts and Figures Behind This Weekend's Knockout Between Real Madrid CF and Borussia Dortmund
UEFA Champions League Final 2024: The host broadcast facts and figures for Real ...
31/05/2024
UEFA Europa Conference League Final 2024: Bringing the Broadcast to Global TV Screens With the Production Facts
UEFA Europa Conference League Final 2024: Bringing the broadcast to global TV sc...
31/05/2024
Willow Offers Free 2024 ICC T20 Cricket World Cup Preview via DirecTV, Verizon, Optimum
Willow Offers Free 2024 ICC T20 Cricket World Cup Preview via DirecTV, Verizon, ...
31/05/2024
Women's College World Series: ESPN Brings Live Drone, Two-Point Cabled, and Cinematic On-Field Cameras to Oklahoma City
Women's College World Series: ESPN Brings Live Drone, Two-Point Cabled, and ...
31/05/2024
SVG College Summit Brings Video-Production Industry Together in Atlanta in the Midst of Unprecedented Change
SVG College Summit Brings Video-Production Industry Together in Atlanta in the M...
31/05/2024
UEFA Champions League Final 2024: CBS Sports' UCL Today Studio Show Is Pitchside at Wembley Stadium
UEFA Champions League Final 2024: CBS Sports' UCL Today Studio Show Is Pitch...
31/05/2024
Master of the House' Unveils a Scandalous Tale of Power and Greed on July 18
Back to All News Master of the House' Unveils a Scandalous Tale of Power and Greed on July 18 Entertainment 31 May 2024 GlobalThailand Link copied to ...
31/05/2024
JEEtu Bhaiya is Back: Kota Factory' Season 3 Arrives on Netflix June 20
Back to All News JEEtu Bhaiya is Back: Kota Factory' Season 3 Arrives on Netflix June 20 Entertainment 31 May 2024 GlobalIndia Link copied to clipboar...
31/05/2024
Thales has completed the sale of its Ground Transportation Systems business
Facebook Twitter LinkedIn With around 9,000 employees, Ground Transportation Systems is a global leader in Rail Signaling and Train Control Systems, Telec...
31/05/2024
RT 2FM is the home of music this summer
RT 2FM is the home of music this summer, entertaining listeners across the day and night, with the best music, chat and interviews providing the soundtrack to ...
31/05/2024
NVIDIA Grace Hopper Superchip Accelerates Murex MX.3 Analytics Performance, Reduces Power Consumption
After the 2008 financial crisis and increased risk-management regulations that f...