Nvidia reveals a new AI model capable of turning 2D video clips to 3D structures, generating life-like virtual replicas of buildings, sculptures or any object in the video clip.
Nvidia, the latest firm to join Trillion-dollar-club of market valuation, announces its recent digital renaissance, an AI model that can create detailed 3D structures from 2D video clips. Called as Neuralangelo, the AI generates 3D structures with intricate details and textures, using the firm’s own NeRF technology, which stands for ‘neural radiance fields’.
The high-fidelity technology of Neuralangelo makes the 3D reconstructions easy for game-developers, metaverse engineers or creativity workers by enabling them to create 3D models using footage captured by smartphones. Like, you can shoot in 2D and convert it to 3D.
“The 3D reconstruction capabilities Neuralangelo offers will be a huge benefit to creators, helping them recreate the real world in the digital world,” said Ming-Yu Liu, senior director of research and co-author on the paper. “This tool will eventually enable developers to import detailed objects – whether small statues or massive buildings – into virtual environments for video games or industrial digital twins.”
Earlier, Stanford University researchers led a study that lets users capture images in 3D with smartphones. And this Nvidia’s AI model might bring the capability to shoot in 3D with your smartphones.
How it works?
Neuralangelo adopts instant neural graphics primitives, the technology behind Nvidia Instant NeRF, to help capture finer details in video clips.
The AI model analyzes an object or scene filmed in a 2D video from various angles and selects several frames that capture different viewpoints of that object or scene. Like engineering pupils sketching a 3D diagram with top, front and side view, AI creates 3D models in a similar way but with dozens of views.
After selection of frames, the AI creates a rough 3D representation of the scene and then optimizes the render to sharpen the details. The final result is a 3D object or large-scale scene that can be used in virtual reality applications, robotics development or digital twins.
Related Posts
Applications of Neuralangelo
- Neuralangelo AI will be most benefiting in creation of 3D renders for virtual world, game development and metaverse.
- As the AI can pull off even a building in 3D space, it will help cities to create a digital twin for online world, likely as this island building its exact replica in metaverse.
- Neuralangelo could power smartphones the ability to capture videos or images and turn it into 3D models.
- This might allow you to relive the past memories with help of virtual reality technology, similar to what this startup’s app plans for.
Neuralangelo is one of nearly 30 projects by NVIDIA Research to be presented at the conference on Computer Vision and Pattern Recognition (CVPR), taking place on June 18-22 in Vancouver, Canada.
Hope you find the page useful!
(For more such interesting informational, technology and innovation stuffs, keep reading The Inner Detail).