SMFS‐GAN: Style‐Guided Multi‐class Freehand Sketch‐to‐Image Synthesis

Simple item page
dc.contributor.authorCheng, Zhenweien_US
dc.contributor.authorWu, Leien_US
dc.contributor.authorLi, Xiangen_US
dc.contributor.authorMeng, Xiangxuen_US
dc.contributor.editorAlliez, Pierreen_US
dc.contributor.editorWimmer, Michaelen_US
dc.date.accessioned2024-12-19T11:15:59Z
dc.date.available2024-12-19T11:15:59Z
dc.date.issued2024
dc.description.abstractFreehand sketch‐to‐image (S2I) is a challenging task due to the individualized lines and the random shape of freehand sketches. The multi‐class freehand sketch‐to‐image synthesis task, in turn, presents new challenges for this research area. This task requires not only the consideration of the problems posed by freehand sketches but also the analysis of multi‐class domain differences in the conditions of a single model. However, existing methods often have difficulty learning domain differences between multiple classes, and cannot generate controllable and appropriate textures while maintaining shape stability. In this paper, we propose a style‐guided multi‐class freehand sketch‐to‐image synthesis model, SMFS‐GAN, which can be trained using only unpaired data. To this end, we introduce a contrast‐based style encoder that optimizes the network's perception of domain disparities by explicitly modelling the differences between classes and thus extracting style information across domains. Further, to optimize the fine‐grained texture of the generated results and the shape consistency with freehand sketches, we propose a local texture refinement discriminator and a Shape Constraint Module, respectively. In addition, to address the imbalance of data classes in the QMUL‐Sketch dataset, we add 6K images by drawing manually and obtain QMUL‐Sketch+ dataset. Extensive experiments on SketchyCOCO Object dataset, QMUL‐Sketch+ dataset and Pseudosketches dataset demonstrate the effectiveness as well as the superiority of our proposed method.en_US
dc.description.number6
dc.description.sectionheadersMajor Revision from Pacific Graphics
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume43
dc.identifier.doi10.1111/cgf.15190
dc.identifier.pages13 pages
dc.identifier.urihttps://doi.org/10.1111/cgf.15190
dc.identifier.urihttps://diglib.eg.org/handle/10.1111/cgf15190
dc.publisher© 2024 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd.en_US
dc.subjectimage and video processing
dc.subjectimage generation
dc.titleSMFS‐GAN: Style‐Guided Multi‐class Freehand Sketch‐to‐Image Synthesisen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
23_cgf15190.pdf
Size:
3.95 MB
Format:
Adobe Portable Document Format
Download
Collections
43-Issue 6