MoNeRF: Deformable Neural Rendering for Talking Heads via Latent Motion Navigation

Simple item page
dc.contributor.authorLi, X.en_US
dc.contributor.authorDing, Y.en_US
dc.contributor.authorLi, R.en_US
dc.contributor.authorTang, Z.en_US
dc.contributor.authorLi, K.en_US
dc.date.accessioned2025-03-07T16:48:51Z
dc.date.available2025-03-07T16:48:51Z
dc.date.issued2024
dc.description.abstractNovel view synthesis for talking heads presents significant challenges due to the complex and diverse motion transformations involved. Conventional methods often resort to reliance on structure priors, like facial templates, to warp observed images into a canonical space conducive to rendering. However, the incorporation of such priors introduces a trade‐off‐while aiding in synthesis, they concurrently amplify model complexity, limiting generalizability to other deformable scenes. Departing from this paradigm, we introduce a pioneering solution: the motion‐conditioned neural radiance field, MoNeRF, designed to model talking heads through latent motion navigation. At the core of MoNeRF lies a novel approach utilizing a compact set of latent codes to represent orthogonal motion directions. This innovative strategy empowers MoNeRF to efficiently capture and depict intricate scene motion by linearly combining these latent codes. In an extended capability, MoNeRF facilitates motion control through latent code adjustments, supports view transfer based on reference videos, and seamlessly extends its applicability to model human bodies without necessitating structural modifications. Rigorous quantitative and qualitative experiments unequivocally demonstrate MoNeRF's superior performance compared to state‐of‐the‐art methods in talking head synthesis. We will release the source code upon publication.en_US
dc.description.number1
dc.description.sectionheadersOriginal Article
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume44
dc.identifier.doi10.1111/cgf.15274
dc.identifier.issn1467-8659
dc.identifier.pages13
dc.identifier.urihttps://doi.org/10.1111/cgf.15274
dc.identifier.urihttps://diglib.eg.org/handle/10.1111/cgf15274
dc.publisherEurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd.en_US
dc.subjectimage and video processing
dc.subjectrendering
dc.subjectimage‐based rendering
dc.subject• Computing methodologies → Computer graphics; Image manipulation; Image‐based rendering
dc.titleMoNeRF: Deformable Neural Rendering for Talking Heads via Latent Motion Navigationen_US
Files
Original bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
14_cgf15274.pdf
Size:
12.19 MB
Format:
Adobe Portable Document Format
Download
Collections
44-Issue 1