Physically-Based Lighting for 3D Generative Models of Cars

Simple item page
dc.contributor.authorViolante, Nicolasen_US
dc.contributor.authorGauthier, Albanen_US
dc.contributor.authorDiolatzis, Stavrosen_US
dc.contributor.authorLeimkühler, Thomasen_US
dc.contributor.authorDrettakis, Georgeen_US
dc.contributor.editorBermano, Amit H.en_US
dc.contributor.editorKalogerakis, Evangelosen_US
dc.date.accessioned2024-04-30T09:07:30Z
dc.date.available2024-04-30T09:07:30Z
dc.date.issued2024
dc.description.abstractRecent work has demonstrated that Generative Adversarial Networks (GANs) can be trained to generate 3D content from 2D image collections, by synthesizing features for neural radiance field rendering. However, most such solutions generate radiance, with lighting entangled with materials. This results in unrealistic appearance, since lighting cannot be changed and view-dependent effects such as reflections do not move correctly with the viewpoint. In addition, many methods have difficulty for full, 360? rotations, since they are often designed for mainly front-facing scenes such as faces. We introduce a new 3D GAN framework that addresses these shortcomings, allowing multi-view coherent 360? viewing and at the same time relighting for objects with shiny reflections, which we exemplify using a car dataset. The success of our solution stems from three main contributions. First, we estimate initial camera poses for a dataset of car images, and then learn to refine the distribution of camera parameters while training the GAN. Second, we propose an efficient Image-Based Lighting model, that we use in a 3D GAN to generate disentangled reflectance, as opposed to the radiance synthesized in most previous work. The material is used for physically-based rendering with a dataset of environment maps. Third, we improve the 3D GAN architecture compared to previous work and design a careful training strategy that allows effective disentanglement. Our model is the first that generate a variety of 3D cars that are multi-view consistent and that can be relit interactively with any environment map.en_US
dc.description.number2
dc.description.sectionheadersNeural 3D Shape Synthesis
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume43
dc.identifier.doi10.1111/cgf.15011
dc.identifier.issn1467-8659
dc.identifier.pages16 pages
dc.identifier.urihttps://doi.org/10.1111/cgf.15011
dc.identifier.urihttps://diglib.eg.org/handle/10.1111/cgf15011
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.titlePhysically-Based Lighting for 3D Generative Models of Carsen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
v43i2_13_15011.pdf
Size:
31.03 MB
Format:
Adobe Portable Document Format
Download
Collections
43-Issue 2