A Data Collection Protocol, Tool and Analysis for the Mapping of Speech Volume to Avatar Facial Animation
Loading...
Date
2022
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
Knowing the relationship between speech-related facial movement and speech is important for avatar animation. Accurate facial displays are necessary to convey perceptual speech characteristics fully. Recently, an effort has been made to infer the relationship between facial movement and speech with data-driven methodologies using computer vision. To this aim, we propose to use blendshape-based facial movement tracking, because it can be easily translated to avatar movement. Furthermore, we present a protocol for audio-visual and behavioral data collection and a tool running on WEB that aids in collecting and synchronizing data. As a start, we provide a database of six Japanese participants reading emotion-related scripts at different volume levels. Using this methodology, we found a relationship between speech volume and facial movement around the nose, cheek, mouth, and head pitch. We hope that our protocols, WEB-based tool, and collected data will be useful for other scientists to derive models for avatar animation.
Description
CCS Concepts: Human-centered computing -> Visualization toolkits
@inproceedings{10.2312:egve.20221273,
booktitle = {ICAT-EGVE 2022 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments},
editor = {Hideaki Uchiyama and Jean-Marie Normand},
title = {{A Data Collection Protocol, Tool and Analysis for the Mapping of Speech Volume to Avatar Facial Animation}},
author = {Miyawaki, Ryosuke and Perusquia-Hernandez, Monica and Isoyama, Naoya and Uchiyama, Hideaki and Kiyokawa, Kiyoshi},
year = {2022},
publisher = {The Eurographics Association},
ISSN = {1727-530X},
ISBN = {978-3-03868-179-3},
DOI = {10.2312/egve.20221273}
}