Generalizable Dynamic Radiance Fields For Talking Head Synthesis With Few-shot

Simple item page
dc.contributor.authorDang, Rujingen_US
dc.contributor.authorWang, Shaohuien_US
dc.contributor.authorWang, Haoqianen_US
dc.contributor.editorChaine, Raphaëlleen_US
dc.contributor.editorDeng, Zhigangen_US
dc.contributor.editorKim, Min H.en_US
dc.date.accessioned2023-10-09T07:42:51Z
dc.date.available2023-10-09T07:42:51Z
dc.date.issued2023
dc.description.abstractAudio-driven talking head generation has wide applications in virtual games, hosts, online meetings, etc. Recently, great achievements have been made in synthesizing talking heads based on neural radiance fields. However, the existing few-shot talking head synthesis methods still suffer from inaccurate deformation and lack of visual consistency. Therefore, we propose a Generalizable Dynamic Radiance Field (GDRF), which can rapidly generalize to unseen identities with few-shot. We introduce a warping module with 3D constraints to act in feature volume space, which is identity adaptive and exhibits excellent shape-shifting abilities. Our method can generate more accurately deformed and view consistent target images compared to previous methods. Furthermore, we map the audio signal to 3DMM parameters by applying an LSTM network, which helps get long-term context and generate more continuous and natural video. Extensive experiments demonstrate the superiority of our proposed method.en_US
dc.description.sectionheadersRadiance and Appearance
dc.description.seriesinformationPacific Graphics Short Papers and Posters
dc.identifier.doi10.2312/pg.20231274
dc.identifier.isbn978-3-03868-234-9
dc.identifier.pages81-88
dc.identifier.pages8 pages
dc.identifier.urihttps://doi.org/10.2312/pg.20231274
dc.identifier.urihttps://diglib.eg.org:443/handle/10.2312/pg20231274
dc.publisherThe Eurographics Associationen_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectCCS Concepts: Computing methodologies -> Reconstruction; Animation; Shape representations
dc.subjectComputing methodologies
dc.subjectReconstruction
dc.subjectAnimation
dc.subjectShape representations
dc.titleGeneralizable Dynamic Radiance Fields For Talking Head Synthesis With Few-shoten_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
081-088.pdf
Size:
2.11 MB
Format:
Adobe Portable Document Format
Download
Collections
PG2023 Short Papers and Posters