THGS: Lifelike Talking Human Avatar Synthesis From Monocular Video Via 3D Gaussian Splatting

Simple item page
dc.contributor.authorChen, Chuangen_US
dc.contributor.authorYu, Lingyunen_US
dc.contributor.authorYang, Quanweien_US
dc.contributor.authorZheng, Aihuaen_US
dc.contributor.authorXie, Hongtaoen_US
dc.date.accessioned2025-03-07T16:49:41Z
dc.date.available2025-03-07T16:49:41Z
dc.date.issued2025
dc.description.abstractDespite the remarkable progress in 3D talking head generation, directly generating 3D talking human avatars still suffers from rigid facial expressions, distorted hand textures and out‐of‐sync lip movements. In this paper, we extend speaker‐specific talking head generation task to and propose a novel pipeline, , that animates lifelike Talking Human avatars using 3D Gaussian Splatting (3DGS). Given speech audio, expression and body poses as input, effectively overcomes the limitations of 3DGS human re‐construction methods in capturing expressive dynamics, such as , from a short monocular video. Firstly, we introduce a simple yet effective for facial dynamics re‐construction, where subtle facial dynamics can be generated by linearly combining the static head model and expression blendshapes. Secondly, a is proposed for lip‐synced mouth movement animation, building connections between speech audio and mouth Gaussian movements. Thirdly, we employ a to optimize these parameters on the fly, which aligns hand movements and expressions better with video input. Experimental results demonstrate that can achieve high‐fidelity 3D talking human avatar animation at 150+ fps on a web‐based rendering system, improving the requirements of real‐time applications. Our project page is at .en_US
dc.description.number1
dc.description.sectionheadersMajor Revision from Pacific Graphics
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume44
dc.identifier.doi10.1111/cgf.15282
dc.identifier.issn1467-8659
dc.identifier.pages17
dc.identifier.urihttps://doi.org/10.1111/cgf.15282
dc.identifier.urihttps://diglib.eg.org/handle/10.1111/cgf15282
dc.publisherEurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd.en_US
dc.subjectimage and video processing
dc.subjectreal‐time rendering
dc.subjectmonocular re‐construction
dc.subject3D Gaussian Splatting
dc.subject• Computing methodologies → Re‐construction; Appearance and texture representations
dc.titleTHGS: Lifelike Talking Human Avatar Synthesis From Monocular Video Via 3D Gaussian Splattingen_US
Files
Original bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
32_cgf15282.pdf
Size:
9.29 MB
Format:
Adobe Portable Document Format
Download
Collections
44-Issue 1