Learning to Stabilize Faces

Simple item page
dc.contributor.authorBednarik, Janen_US
dc.contributor.authorWood, Errollen_US
dc.contributor.authorChoutas, Vassilisen_US
dc.contributor.authorBolkart, Timoen_US
dc.contributor.authorWang, Daoyeen_US
dc.contributor.authorWu, Chengleien_US
dc.contributor.authorBeeler, Thaboen_US
dc.contributor.editorBermano, Amit H.en_US
dc.contributor.editorKalogerakis, Evangelosen_US
dc.date.accessioned2024-04-30T09:09:22Z
dc.date.available2024-04-30T09:09:22Z
dc.date.issued2024
dc.description.abstractNowadays, it is possible to scan faces and automatically register them with high quality. However, the resulting face meshes often need further processing: we need to stabilize them to remove unwanted head movement. Stabilization is important for tasks like game development or movie making which require facial expressions to be cleanly separated from rigid head motion. Since manual stabilization is labor-intensive, there have been attempts to automate it. However, previous methods remain impractical: they either still require some manual input, produce imprecise alignments, rely on dubious heuristics and slow optimization, or assume a temporally ordered input. Instead, we present a new learning-based approach that is simple and fully automatic. We treat stabilization as a regression problem: given two face meshes, our network directly predicts the rigid transform between them that brings their skulls into alignment. We generate synthetic training data using a 3D Morphable Model (3DMM), exploiting the fact that 3DMM parameters separate skull motion from facial skin motion. Through extensive experiments we show that our approach outperforms the state-of-the-art both quantitatively and qualitatively on the tasks of stabilizing discrete sets of facial expressions as well as dynamic facial performances. Furthermore, we provide an ablation study detailing the design choices and best practices to help others adopt our approach for their own uses.en_US
dc.description.number2
dc.description.sectionheadersFace Modeling and Reconstruction
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume43
dc.identifier.doi10.1111/cgf.15038
dc.identifier.issn1467-8659
dc.identifier.pages10 pages
dc.identifier.urihttps://doi.org/10.1111/cgf.15038
dc.identifier.urihttps://diglib.eg.org/handle/10.1111/cgf15038
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.titleLearning to Stabilize Facesen_US
Files
Original bundle
Now showing 1 - 3 of 3
No Thumbnail Available
Name:
v43i2_42_15038.pdf
Size:
35.7 MB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
learning_to_stabilize_faces_appendix.pdf
Size:
4.78 MB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
supplementary_website.zip
Size:
22.38 MB
Format:
Zip file
Download
Collections
43-Issue 2