Garment Animation NeRF with Color Editing

Wang, Renke; Zhang, Meng; Li, Jun; Yang, Jian

Garment Animation NeRF with Color Editing

Files

v43i8_cgf15178.pdf (2.48 MB)

paper1006.mp4 (88.29 MB)

Date

2024

Authors

Wang, Renke
Zhang, Meng
Li, Jun
Yang, Jian

Publisher

The Eurographics Association and John Wiley & Sons Ltd.

Abstract

Generating high-fidelity garment animations through traditional workflows, from modeling to rendering, is both tedious and expensive. These workflows often require repetitive steps in response to updates in character motion, rendering viewpoint changes, or appearance edits. Although recent neural rendering offers an efficient solution for computationally intensive processes, it struggles with rendering complex garment animations containing fine wrinkle details and realistic garment-and-body occlusions, while maintaining structural consistency across frames and dense view rendering. In this paper, we propose a novel approach to directly synthesize garment animations from body motion sequences without the need for an explicit garment proxy. Our approach infers garment dynamic features from body motion, providing a preliminary overview of garment structure. Simultaneously, we capture detailed features from synthesized reference images of the garment's front and back, generated by a pre-trained image model. These features are then used to construct a neural radiance field that renders the garment animation video. Additionally, our technique enables garment recoloring by decomposing its visual elements. We demonstrate the generalizability of our method across unseen body motions and camera views, ensuring detailed structural consistency. Furthermore, we showcase its applicability to color editing on both real and synthetic garment data. Compared to existing neural rendering techniques, our method exhibits qualitative and quantitative improvements in garment dynamics and wrinkle detail modeling. Code is available at https://github.com/wrk226/GarmentAnimationNeRF.

CCS Concepts: Computing methodologies → Rendering; Animation; Neural networks

        @article{10.1111:cgf.15178
,
journal = {Computer Graphics Forum},
title = {{Garment Animation NeRF with Color Editing
}},
author = {Wang, Renke and 
Zhang, Meng and 
Li, Jun and 
Yang, Jian
},
year = {2024
},
publisher = {The Eurographics Association and John Wiley & Sons Ltd.
},
ISSN = {1467-8659
},
DOI = {10.1111/cgf.15178
}
}