Practical Face Reconstruction via Differentiable Ray Tracing

Simple item page
dc.contributor.authorDib, Abdallahen_US
dc.contributor.authorBharaj, Gauraven_US
dc.contributor.authorAhn, Junghyunen_US
dc.contributor.authorThébault, Cédricen_US
dc.contributor.authorGosselin, Philippeen_US
dc.contributor.authorRomeo, Marcoen_US
dc.contributor.authorChevallier, Louisen_US
dc.contributor.editorMitra, Niloy and Viola, Ivanen_US
dc.date.accessioned2021-04-09T08:00:02Z
dc.date.available2021-04-09T08:00:02Z
dc.date.issued2021
dc.description.abstractWe present a differentiable ray-tracing based novel face reconstruction approach where scene attributes - 3D geometry, reflectance (diffuse, specular and roughness), pose, camera parameters, and scene illumination - are estimated from unconstrained monocular images. The proposed method models scene illumination via a novel, parameterized virtual light stage, which in-conjunction with differentiable ray-tracing, introduces a coarse-to-fine optimization formulation for face reconstruction. Our method can not only handle unconstrained illumination and self-shadows conditions, but also estimates diffuse and specular albedos. To estimate the face attributes consistently and with practical semantics, a two-stage optimization strategy systematically uses a subset of parametric attributes, where subsequent attribute estimations factor those previously estimated. For example, self-shadows estimated during the first stage, later prevent its baking into the personalized diffuse and specular albedos in the second stage. We show the efficacy of our approach in several real-world scenarios, where face attributes can be estimated even under extreme illumination conditions. Ablation studies, analyses and comparisons against several recent state-of-the-art methods show improved accuracy and versatility of our approach. With consistent face attributes reconstruction, our method leads to several style - illumination, albedo, self-shadow - edit and transfer applications, as discussed in the paper.en_US
dc.description.number2
dc.description.sectionheadersDeep Rendering
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume40
dc.identifier.doi10.1111/cgf.142622
dc.identifier.issn1467-8659
dc.identifier.pages153-164
dc.identifier.urihttps://doi.org/10.1111/cgf.142622
dc.identifier.urihttps://diglib.eg.org:443/handle/10.1111/cgf142622
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.subjectComputing methodologies
dc.subjectMesh geometry models
dc.subjectReflectance modeling
dc.subjectRay tracing
dc.titlePractical Face Reconstruction via Differentiable Ray Tracingen_US
Files
Original bundle
Now showing 1 - 3 of 3
Thumbnail Image
Name:
v40i2pp153-164.pdf
Size:
16.2 MB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
paper1113_practical_face_reconstruction_via_differentiable_ray_tracing_supplementary.mp4
Size:
50.85 MB
Format:
Unknown data format
Download
Thumbnail Image
Name:
suppmateiral-practical_face_reconstruction_via_differentiable_ray_tracing.pdf
Size:
24.04 MB
Format:
Adobe Portable Document Format
Download
Collections
40-Issue 2