Browsing by Author "LIAO, Jing"

Now showing 1 - 2 of 2
  • No Thumbnail Available
    Item
    Continuous Layout Editing of Single Images with Diffusion Models
    (The Eurographics Association and John Wiley & Sons Ltd., 2023) Zhang, Zhiyuan; Huang, Zhitong; Liao, Jing; Chaine, Raphaƫlle; Deng, Zhigang; Kim, Min H.
    Recent advancements in large-scale text-to-image diffusion models have enabled many applications in image editing. However, none of these methods have been able to edit the layout of single existing images. To address this gap, we propose the first framework for layout editing of a single image while preserving its visual properties, thus allowing for continuous editing on a single image. Our approach is achieved through two key modules. First, to preserve the characteristics of multiple objects within an image, we disentangle the concepts of different objects and embed them into separate textual tokens using a novel method called masked textual inversion. Next, we propose a training-free optimization method to perform layout control for a pre-trained diffusion model, which allows us to regenerate images with learned concepts and align them with user-specified layouts. As the first framework to edit the layout of existing images, we demonstrate that our method is effective and outperforms other baselines that were modified to support this task. Code is available at our project page.
  • Thumbnail Image
    Item
    Style Mixer: Semantic-aware Multi-Style Transfer Network
    (The Eurographics Association and John Wiley & Sons Ltd., 2019) HUANG, Zixuan; ZHANG, Jinghuai; LIAO, Jing; Lee, Jehee and Theobalt, Christian and Wetzstein, Gordon
    Recent neural style transfer frameworks have obtained astonishing visual quality and flexibility in Single-style Transfer (SST), but little attention has been paid to Multi-style Transfer (MST) which refers to simultaneously transferring multiple styles to the same image. Compared to SST, MST has the potential to create more diverse and visually pleasing stylization results. In this paper, we propose the first MST framework to automatically incorporate multiple styles into one result based on regional semantics. We first improve the existing SST backbone network by introducing a novel multi-level feature fusion module and a patch attention module to achieve better semantic correspondences and preserve richer style details. For MST, we designed a conceptually simple yet effective region-based style fusion module to insert into the backbone. It assigns corresponding styles to content regions based on semantic matching, and then seamlessly combines multiple styles together. Comprehensive evaluations demonstrate that our framework outperforms existing works of SST and MST.