SketchBodyNet: A Sketch-Driven Multi-faceted Decoder Network for 3D Human Reconstruction

Simple item page
dc.contributor.authorWang, Feien_US
dc.contributor.authorTang, Kongzhangen_US
dc.contributor.authorWu, Hefengen_US
dc.contributor.authorZhao, Baoquanen_US
dc.contributor.authorCai, Haoen_US
dc.contributor.authorZhou, Tengen_US
dc.contributor.editorChaine, Raphaëlleen_US
dc.contributor.editorDeng, Zhigangen_US
dc.contributor.editorKim, Min H.en_US
dc.date.accessioned2023-10-09T07:42:35Z
dc.date.available2023-10-09T07:42:35Z
dc.date.issued2023
dc.description.abstractReconstructing 3D human shapes from 2D images has received increasing attention recently due to its fundamental support for many high-level 3D applications. Compared with natural images, freehand sketches are much more flexible to depict various shapes, providing a high potential and valuable way for 3D human reconstruction. However, such a task is highly challenging. The sparse abstract characteristics of sketches add severe difficulties, such as arbitrariness, inaccuracy, and lacking image details, to the already badly ill-posed problem of 2D-to-3D reconstruction. Although current methods have achieved great success in reconstructing 3D human bodies from a single-view image, they do not work well on freehand sketches. In this paper, we propose a novel sketch-driven multi-faceted decoder network termed SketchBodyNet to address this task. Specifically, the network consists of a backbone and three separate attention decoder branches, where a multi-head self-attention module is exploited in each decoder to obtain enhanced features, followed by a multi-layer perceptron. The multi-faceted decoders aim to predict the camera, shape, and pose parameters, respectively, which are then associated with the SMPL model to reconstruct the corresponding 3D human mesh. In learning, existing 3D meshes are projected via the camera parameters into 2D synthetic sketches with joints, which are combined with the freehand sketches to optimize the model. To verify our method, we collect a large-scale dataset of about 26k freehand sketches and their corresponding 3D meshes containing various poses of human bodies from 14 different angles. Extensive experimental results demonstrate our SketchBodyNet achieves superior performance in reconstructing 3D human meshes from freehand sketches.en_US
dc.description.sectionheadersSketch-based Modeling
dc.description.seriesinformationPacific Graphics Short Papers and Posters
dc.identifier.doi10.2312/pg.20231266
dc.identifier.isbn978-3-03868-234-9
dc.identifier.pages11-19
dc.identifier.pages9 pages
dc.identifier.urihttps://doi.org/10.2312/pg.20231266
dc.identifier.urihttps://diglib.eg.org:443/handle/10.2312/pg20231266
dc.publisherThe Eurographics Associationen_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectCCS Concepts: Computing methodologies -> 3D Reconstruction
dc.subjectComputing methodologies
dc.subject3D Reconstruction
dc.titleSketchBodyNet: A Sketch-Driven Multi-faceted Decoder Network for 3D Human Reconstructionen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
011-019.pdf
Size:
1.57 MB
Format:
Adobe Portable Document Format
Download
Collections
PG2023 Short Papers and Posters