THGS: Lifelike Talking Human Avatar Synthesis From Monocular Video Via 3D Gaussian Splatting
Loading...
Date
2025
Journal Title
Journal ISSN
Volume Title
Publisher
Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd.
Abstract
Despite the remarkable progress in 3D talking head generation, directly generating 3D talking human avatars still suffers from rigid facial expressions, distorted hand textures and out‐of‐sync lip movements. In this paper, we extend speaker‐specific talking head generation task to and propose a novel pipeline, , that animates lifelike Talking Human avatars using 3D Gaussian Splatting (3DGS). Given speech audio, expression and body poses as input, effectively overcomes the limitations of 3DGS human re‐construction methods in capturing expressive dynamics, such as , from a short monocular video. Firstly, we introduce a simple yet effective for facial dynamics re‐construction, where subtle facial dynamics can be generated by linearly combining the static head model and expression blendshapes. Secondly, a is proposed for lip‐synced mouth movement animation, building connections between speech audio and mouth Gaussian movements. Thirdly, we employ a to optimize these parameters on the fly, which aligns hand movements and expressions better with video input. Experimental results demonstrate that can achieve high‐fidelity 3D talking human avatar animation at 150+ fps on a web‐based rendering system, improving the requirements of real‐time applications. Our project page is at .
Description
@article{10.1111:cgf.15282,
journal = {Computer Graphics Forum},
title = {{THGS: Lifelike Talking Human Avatar Synthesis From Monocular Video Via 3D Gaussian Splatting}},
author = {Chen, Chuang and Yu, Lingyun and Yang, Quanwei and Zheng, Aihua and Xie, Hongtao},
year = {2025},
publisher = {Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {10.1111/cgf.15282}
}