This year’s NVIDIA SIGGRAPH graphics conference will be held in Denver, Colorado on July 29 where Team Green as well as leading industry partners will be showcasing some new graphics tech for the first time so in this post, we’ll summarize what will be shown.
Expanded Capabilities in Diffusion Models
Diffusion models that take text prompts and push out images have been around for years but with each day, they are getting better. One of the new NVIDIA-authored papers extends to ConsiStory which is a collaboration between internal researchers and Tel Aviv University that refines the consistency and quality of generating multiple images of the same character which can be extremely useful for use cases like storyboard development.
The improvement in consistency through this new ‘Subject-Driven Shared Attention’ technique has led to the results of getting a collection of consistent imageries from 13 minutes down to just around 30 seconds.
On the other hand, there will be a new AI model that applies 2D generative diffusion models to interactive texture painting on 3D meshes based on reference images, essentially turning last year’s “text to custom texture” static model into an active one that reflects changes in real-time.
Development Kickstarting in Physics-Based Simulation
Even though NVIDIA has and still is at the forefront when it comes to physics-based simulation, several new research papers from them discovered breakthroughs in the field, and one of them is called SuperPADL which takes text prompts into generating complex human motions.
Utilizing the power of reinforced learning and supervised learning, the prototype model could demonstrate the generation of more than 5,000 skills, and most importantly, can be scaled down to be run on consumer-class NVIDIA GPUs.
Another paper suggests a new neural physics method for AI to learn about the movement of a wide variety of objects from 3D mesh, NeRF, or a solid block generated by text-to-3D model, would behave in a given environment.
And we also have a new renderer that can do thermal analysis, electrostatics and fluid mechanic which will definitely be extremely useful in the engineering world.
Diffraction Simulation and Rendering Realism at the Next Level
As for lighting-based effects, several NVIDIA-authored papers revolve around improvements to visible light modeling that has up to 25x faster performance while diffraction could achieve a whopping 1000x uplift.
Beyond integrating free-space diffraction into path-tracing workflows, said model can also be used to simulate lights with longer wavelengths like radars, sounds, and radiowaves.
One specific approach to path-tracing technique comes from the collaboration with the University of Utah where they reuse calculated paths to increase effect sample count by up to 25x that leads to higher image quality with fewer visual artifacts.
Teaching AIs to Think in 3D
Last but not least and perhaps the most interesting, NVIDIA researchers have found a way to incorporate 3D representation and designs into a multipurpose AI tool with one paper introducing fVDB, a GPU-optimized framework for 3D deep learning based on the scales of the real world by providing large spatial scale and high-resolution 3D models, NeRFs, segmentation and reconstruction of large-scale point clouds.
Other collaborations also include one Best Technical Paper award winner written in collaboration with Dartmouth College researchers that tackles the way of representation in how 3D objects interact with light while another combines the efforts of the University of Tokyo, the University of Toronto, and Adobe Research to develop an algorithm that generates smooth, space-filling curves on 3D meshes in real-time, essentially cutting hours of hard work down in mere seconds.
More at NVIDIA SIGGRAPH
Aside from all the tech demonstrations, there will be other attractive sessions to attend including a special fireside chat between NVIDIA CEO Jensen Huang and Meta founder and CEO Mark Zuckerberg, a fireside chat with Huang and one of WIRED’s senior writers Lauren Goode, and OpenUSD Day that dedicates to the whole industry adoption and constant evolution of OpenUSD to build AI-enabled 3D pipelines.
