Real-Time Multi-View Facial Capture with Synthetic Training

dc.contributor.authorKlaudiny, Martinen_US
dc.contributor.authorMcDonagh, Stevenen_US
dc.contributor.authorBradley, Dereken_US
dc.contributor.authorBeeler, Thaboen_US
dc.contributor.authorMitchell, Kennyen_US
dc.contributor.editorLoic Barthe and Bedrich Benesen_US
dc.date.accessioned2017-04-22T16:26:49Z
dc.date.available2017-04-22T16:26:49Z
dc.date.issued2017
dc.description.abstractWe present a real-time multi-view facial capture system facilitated by synthetic training imagery. Our method is able to achieve high-quality markerless facial performance capture in real-time from multi-view helmet camera data, employing an actor specific regressor. The regressor training is tailored to specified actor appearance and we further condition it for the expected illumination conditions and the physical capture rig by generating the training data synthetically. In order to leverage the information present in live imagery, which is typically provided by multiple cameras, we propose a novel multi-view regression algorithm that uses multi-dimensional random ferns. We show that higher quality can be achieved by regressing on multiple video streams than previous approaches that were designed to operate on only a single view. Furthermore, we evaluate possible camera placements and propose a novel camera configuration that allows to mount cameras outside the field of view of the actor, which is very beneficial as the cameras are then less of a distraction for the actor and allow for an unobstructed line of sight to the director and other actors. Our new real-time facial capture approach has immediate application in on-set virtual production, in particular with the ever-growing demand for motion-captured facial animation in visual effects and video games.en_US
dc.description.number2
dc.description.sectionheadersCapturing Faces
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume36
dc.identifier.doi10.1111/cgf.13129
dc.identifier.issn1467-8659
dc.identifier.pages325-336
dc.identifier.urihttps://doi.org/10.1111/cgf.13129
dc.identifier.urihttps://diglib.eg.org:443/handle/10.1111/cgf13129
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.subjectI.3.7 [Computer Graphics]
dc.subjectThree Dimensional Graphics and Realism
dc.subjectAnimation
dc.subjectI.4.8 [Computer Graphics]
dc.subjectScene Analysis
dc.subjectTracking
dc.titleReal-Time Multi-View Facial Capture with Synthetic Trainingen_US
Files
Collections