40-Issue 7

Permanent URI for this collection

Pacific Graphics 2021 - Symposium Proceedings
Wellington, New Zealand

(for Short Papers, Posters, and Work-in-Progress Papers see PG 2021 - Short Papers, Posters, and Work-in-Progress Papers)
Stylization and Layout
Line Art Colorization Based on Explicit Region Segmentation
Ruizhi Cao, Haoran Mo, and Chengying Gao
Global Illumination-Aware Stylised Shading
Kohei Doi, Yuki Morimoto, and Reiji Tsuruno
Fluidymation: Stylizing Animations Using Natural Dynamics of Artistic Media
Adam Platkevic, Cassidy Curtis, and Daniel Sýkora
Modeling Visual Containment for Web Page Layout Optimization
Kotaro Kikuchi, Mayu Otani, Kota Yamaguchi, and Edgar Simo-Serra
Animation
Deep Learning-Based Unsupervised Human Facial Retargeting
Seonghyeon Kim, Sunjin Jung, Kwanggyoon Seo, Roger Blanco i Ribera, and Junyong Noh
Conservative Meshlet Bounds for Robust Culling of Skinned Meshes
Johannes Unterguggenberger, Bernhard Kerbl, Jakob Pernsteiner, and Michael Wimmer
Diverse Dance Synthesis via Keyframes with Transformer Controllers
Junjun Pan, Siyuan Wang, Junxuan Bai, and Ju Dai
A Dynamic Mixture Model for Non-equilibrium Multiphase Fluids
Yuntao Jiang and Yingjie Lan
A Lagrangian Particle-based Formulation for Coupled Simulation of Fracture and Diffusion in Thin Membranes
Chengguizi Han, Tao Xue, and Mridul Aanjaneya
Global Illumination
Geometric Sample Reweighting for Monte Carlo Integration
Jerry Jinfeng Guo and Elmar Eisemann
Consistent Post-Reconstruction for Progressive Photon Mapping
Hajin Choi and Bochang Moon
Neural Sequence Transformation
Sabyasachi Mukherjee, Sayan Mukherjee, Binh-Son Hua, Nobuyuki Umetani, and Daniel Meister
A Multi-pass Method for Accelerated Spectral Sampling
Mark van de Ruit and Elmar Eisemann
Real-Time Rendering
Z-Thickness Blending: Effective Fragment Merging for Multi-Fragment Rendering
Dongjoon Kim and Heewon Kye
Optimizing Ray Tracing of Trimmed NURBS Surfaces on the GPU
Jaroslav Sloup and Vlastimil Havran
Real-time Denoising Using BRDF Pre-integration Factorization
Tao Zhuang, Pengfei Shen, Beibei Wang, and Ligang Liu
Image Synthesis and Enhancement
Luminance Attentive Networks for HDR Image and Panorama Reconstruction
Hanning Yu, Wentao Liu, Chengjiang Long, Bo Dong, Qin Zou, and Chunxia Xiao
Seamless Satellite-image Synthesis
Jialin Zhu and Tom Kelly
Relighting Humans in the Wild: Monocular Full-Body Human Relighting with Domain Adaptation
Daichi Tajima, Yoshihiro Kanamori, and Yuki Endo
Interaction and Interfaces
Write-An-Animation: High-level Text-based Animation Editing with Character-Scene Interaction
Jia-Qi Zhang, Xiang Xu, Zhi-Meng Shen, Ze-Huan Huang, Yang Zhao, Yan-Pei Cao, Pengfei Wan, and Miao Wang
A Motion-guided Interface for Modeling 3D Multi-functional Furniture
Minchan Chen and Manfred Lau
Egocentric Network Exploration for Immersive Analytics
Johannes Sorger, Alessio Arleo, Peter Kán, Wolfgang Knecht, and Manuela Waldner
Interactive Analysis of CNN Robustness
Stefan Sietzen, Mathias Lechner, Judy Borowski, Ramin Hasani, and Manuela Waldner
Modelling
UprightRL: Upright Orientation Estimation of 3D Shapes via Reinforcement Learning
Luanmin Chen, Juzhan Xu, Chuan Wang, Haibin Huang, Hui Huang, and Ruizhen Hu
Synthesizing Geologically Coherent Cave Networks
Axel Paris, Eric Guérin, Adrien Peytavie, Pauline Collon, and Eric Galin
Manhattan-world Urban Building Reconstruction by Fitting Cubes
Zhenbang He, Yunhai Wang, and Zhanglin Cheng

BibTeX (40-Issue 7)
                
@article{
10.1111:cgf.14396,
journal = {Computer Graphics Forum}, title = {{
Line Art Colorization Based on Explicit Region Segmentation}},
author = {
Cao, Ruizhi
 and
Mo, Haoran
 and
Gao, Chengying
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14396}
}
                
@article{
10.1111:cgf.14397,
journal = {Computer Graphics Forum}, title = {{
Global Illumination-Aware Stylised Shading}},
author = {
Doi, Kohei
 and
Morimoto, Yuki
 and
Tsuruno, Reiji
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14397}
}
                
@article{
10.1111:cgf.14399,
journal = {Computer Graphics Forum}, title = {{
Modeling Visual Containment for Web Page Layout Optimization}},
author = {
Kikuchi, Kotaro
 and
Otani, Mayu
 and
Yamaguchi, Kota
 and
Simo-Serra, Edgar
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14399}
}
                
@article{
10.1111:cgf.14398,
journal = {Computer Graphics Forum}, title = {{
Fluidymation: Stylizing Animations Using Natural Dynamics of Artistic Media}},
author = {
Platkevic, Adam
 and
Curtis, Cassidy
 and
Sýkora, Daniel
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14398}
}
                
@article{
10.1111:cgf.14400,
journal = {Computer Graphics Forum}, title = {{
Deep Learning-Based Unsupervised Human Facial Retargeting}},
author = {
Kim, Seonghyeon
 and
Jung, Sunjin
 and
Seo, Kwanggyoon
 and
Ribera, Roger Blanco i
 and
Noh, Junyong
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14400}
}
                
@article{
10.1111:cgf.14401,
journal = {Computer Graphics Forum}, title = {{
Conservative Meshlet Bounds for Robust Culling of Skinned Meshes}},
author = {
Unterguggenberger, Johannes
 and
Kerbl, Bernhard
 and
Pernsteiner, Jakob
 and
Wimmer, Michael
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14401}
}
                
@article{
10.1111:cgf.14402,
journal = {Computer Graphics Forum}, title = {{
Diverse Dance Synthesis via Keyframes with Transformer Controllers}},
author = {
Pan, Junjun
 and
Wang, Siyuan
 and
Bai, Junxuan
 and
Dai, Ju
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14402}
}
                
@article{
10.1111:cgf.14403,
journal = {Computer Graphics Forum}, title = {{
A Dynamic Mixture Model for Non-equilibrium Multiphase Fluids}},
author = {
Jiang, Yuntao
 and
Lan, Yingjie
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14403}
}
                
@article{
10.1111:cgf.14404,
journal = {Computer Graphics Forum}, title = {{
A Lagrangian Particle-based Formulation for Coupled Simulation of Fracture and Diffusion in Thin Membranes}},
author = {
Han, Chengguizi
 and
Xue, Tao
 and
Aanjaneya, Mridul
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14404}
}
                
@article{
10.1111:cgf.14405,
journal = {Computer Graphics Forum}, title = {{
Geometric Sample Reweighting for Monte Carlo Integration}},
author = {
Guo, Jerry Jinfeng
 and
Eisemann, Elmar
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14405}
}
                
@article{
10.1111:cgf.14406,
journal = {Computer Graphics Forum}, title = {{
Consistent Post-Reconstruction for Progressive Photon Mapping}},
author = {
Choi, Hajin
 and
Moon, Bochang
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14406}
}
                
@article{
10.1111:cgf.14407,
journal = {Computer Graphics Forum}, title = {{
Neural Sequence Transformation}},
author = {
Mukherjee, Sabyasachi
 and
Mukherjee, Sayan
 and
Hua, Binh-Son
 and
Umetani, Nobuyuki
 and
Meister, Daniel
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14407}
}
                
@article{
10.1111:cgf.14408,
journal = {Computer Graphics Forum}, title = {{
A Multi-pass Method for Accelerated Spectral Sampling}},
author = {
Ruit, Mark van de
 and
Eisemann, Elmar
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14408}
}
                
@article{
10.1111:cgf.14409,
journal = {Computer Graphics Forum}, title = {{
Z-Thickness Blending: Effective Fragment Merging for Multi-Fragment Rendering}},
author = {
Kim, Dongjoon
 and
Kye, Heewon
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14409}
}
                
@article{
10.1111:cgf.14410,
journal = {Computer Graphics Forum}, title = {{
Optimizing Ray Tracing of Trimmed NURBS Surfaces on the GPU}},
author = {
Sloup, Jaroslav
 and
Havran, Vlastimil
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14410}
}
                
@article{
10.1111:cgf.14411,
journal = {Computer Graphics Forum}, title = {{
Real-time Denoising Using BRDF Pre-integration Factorization}},
author = {
Zhuang, Tao
 and
Shen, Pengfei
 and
Wang, Beibei
 and
Liu, Ligang
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14411}
}
                
@article{
10.1111:cgf.14412,
journal = {Computer Graphics Forum}, title = {{
Luminance Attentive Networks for HDR Image and Panorama Reconstruction}},
author = {
Yu, Hanning
 and
Liu, Wentao
 and
Long, Chengjiang
 and
Dong, Bo
 and
Zou, Qin
 and
Xiao, Chunxia
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14412}
}
                
@article{
10.1111:cgf.14413,
journal = {Computer Graphics Forum}, title = {{
Seamless Satellite-image Synthesis}},
author = {
Zhu, Jialin
 and
Kelly, Tom
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14413}
}
                
@article{
10.1111:cgf.14414,
journal = {Computer Graphics Forum}, title = {{
Relighting Humans in the Wild: Monocular Full-Body Human Relighting with Domain Adaptation}},
author = {
Tajima, Daichi
 and
Kanamori, Yoshihiro
 and
Endo, Yuki
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14414}
}
                
@article{
10.1111:cgf.14415,
journal = {Computer Graphics Forum}, title = {{
Write-An-Animation: High-level Text-based Animation Editing with Character-Scene Interaction}},
author = {
Zhang, Jia-Qi
 and
Xu, Xiang
 and
Shen, Zhi-Meng
 and
Huang, Ze-Huan
 and
Zhao, Yang
 and
Cao, Yan-Pei
 and
Wan, Pengfei
 and
Wang, Miao
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14415}
}
                
@article{
10.1111:cgf.14416,
journal = {Computer Graphics Forum}, title = {{
A Motion-guided Interface for Modeling 3D Multi-functional Furniture}},
author = {
Chen, Minchan
 and
Lau, Manfred
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14416}
}
                
@article{
10.1111:cgf.14417,
journal = {Computer Graphics Forum}, title = {{
Egocentric Network Exploration for Immersive Analytics}},
author = {
Sorger, Johannes
 and
Arleo, Alessio
 and
Kán, Peter
 and
Knecht, Wolfgang
 and
Waldner, Manuela
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14417}
}
                
@article{
10.1111:cgf.14418,
journal = {Computer Graphics Forum}, title = {{
Interactive Analysis of CNN Robustness}},
author = {
Sietzen, Stefan
 and
Lechner, Mathias
 and
Borowski, Judy
 and
Hasani, Ramin
 and
Waldner, Manuela
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14418}
}
                
@article{
10.1111:cgf.14419,
journal = {Computer Graphics Forum}, title = {{
UprightRL: Upright Orientation Estimation of 3D Shapes via Reinforcement Learning}},
author = {
Chen, Luanmin
 and
Xu, Juzhan
 and
Wang, Chuan
 and
Huang, Haibin
 and
Huang, Hui
 and
Hu, Ruizhen
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14419}
}
                
@article{
10.1111:cgf.14420,
journal = {Computer Graphics Forum}, title = {{
Synthesizing Geologically Coherent Cave Networks}},
author = {
Paris, Axel
 and
Guérin, Eric
 and
Peytavie, Adrien
 and
Collon, Pauline
 and
Galin, Eric
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14420}
}
                
@article{
10.1111:cgf.14421,
journal = {Computer Graphics Forum}, title = {{
Manhattan-world Urban Building Reconstruction by Fitting Cubes}},
author = {
He, Zhenbang
 and
Wang, Yunhai
 and
Cheng, Zhanglin
}, year = {
2021},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14421}
}

Browse

Recent Submissions

Now showing 1 - 27 of 27
  • Item
    Pacific Graphics 2021 - CGF 40-7: Frontmatter
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Zhang, Fang-Lue; Eisemann, Elmar; Singh, Karan; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
  • Item
    Line Art Colorization Based on Explicit Region Segmentation
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Cao, Ruizhi; Mo, Haoran; Gao, Chengying; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    Automatic line art colorization plays an important role in anime and comic industry. While existing methods for line art colorization are able to generate plausible colorized results, they tend to suffer from the color bleeding issue. We introduce an explicit segmentation fusion mechanism to aid colorization frameworks in avoiding color bleeding artifacts. This mechanism is able to provide region segmentation information for the colorization process explicitly so that the colorization model can learn to avoid assigning the same color across regions with different semantics or inconsistent colors inside an individual region. The proposed mechanism is designed in a plug-and-play manner, so it can be applied to a diversity of line art colorization frameworks with various kinds of user guidances. We evaluate this mechanism in tag-based and referencebased line art colorization tasks by incorporating it into the state-of-the-art models. Comparisons with these existing models corroborate the effectiveness of our method which largely alleviates the color bleeding artifacts. The code is available at https://github.com/Ricardo-L-C/ColorizationWithRegion.
  • Item
    Global Illumination-Aware Stylised Shading
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Doi, Kohei; Morimoto, Yuki; Tsuruno, Reiji; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    Our aim is to convert an object's appearance to an arbitrary colour considering the light scattering in the entire scene, which is often called the global illumination. Existing stylisation methods convert the colour of an object with a 1-dimensional texture for 3-dimensional computer graphics to reproduce a typical style used in illustrations and cel animations. However, they cannot express global illumination effects. We propose two individual methods to compute the global illumination and convert the shading to an arbitrary colour. The methods reproduce reflections in other objects with the converted colour. As a result, we can convert the colour of illumination effects that have not yet been reproduced, such as soft shadows and refractions
  • Item
    Modeling Visual Containment for Web Page Layout Optimization
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Kikuchi, Kotaro; Otani, Mayu; Yamaguchi, Kota; Simo-Serra, Edgar; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    Web pages have become fundamental in conveying information for companies and individuals, yet designing web page layouts remains a challenging task for inexperienced individuals despite web builders and templates. Visual containment, in which elements are grouped together and placed inside container elements, is an efficient design strategy for organizing elements in a limited display, and is widely implemented in most web page designs. Yet, visual containment has not been explicitly addressed in the research on generating layouts from scratch, which may be due to the lack of hierarchical structure. In this work, we represent such visual containment as a layout tree, and formulate the layout design task as a hierarchical optimization problem. We first estimate the layout tree from a given a set of elements, which is then used to compute tree-aware energies corresponding to various desirable design properties such as alignment or spacing. Using an optimization approach also allows our method to naturally incorporate user intentions and create an interactive web design application. We obtain a dataset of diverse and popular real-world web designs to optimize and evaluate various aspects of our method. Experimental results show that our method generates better quality layouts compared to the baseline method.
  • Item
    Fluidymation: Stylizing Animations Using Natural Dynamics of Artistic Media
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Platkevic, Adam; Curtis, Cassidy; Sýkora, Daniel; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    We present Fluidymation-a new example-based approach to stylizing animation that employs the natural dynamics of artistic media to convey a prescribed motion. In contrast to previous stylization techniques that transfer the hand-painted appearance of a static style exemplar and then try to enforce temporal coherence, we use moving exemplars that capture the artistic medium's inherent dynamic properties, and transfer both movement and appearance to reproduce natural-looking transitions between individual animation frames. Our approach can synthetically generate stylized sequences that look as if actual paint is diffusing across a canvas in the direction and speed of the target motion.
  • Item
    Deep Learning-Based Unsupervised Human Facial Retargeting
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Kim, Seonghyeon; Jung, Sunjin; Seo, Kwanggyoon; Ribera, Roger Blanco i; Noh, Junyong; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    Traditional approaches to retarget existing facial blendshape animations to other characters rely heavily on manually paired data including corresponding anchors, expressions, or semantic parametrizations to preserve the characteristics of the original performance. In this paper, inspired by recent developments in face swapping and reenactment, we propose a novel unsupervised learning method that reformulates the retargeting of 3D facial blendshape-based animations in the image domain. The expressions of a source model is transferred to a target model via the rendered images of the source animation. For this purpose, a reenactment network is trained with the rendered images of various expressions created by the source and target models in a shared latent space. The use of shared latent space enable an automatic cross-mapping obviating the need for manual pairing. Next, a blendshape prediction network is used to extract the blendshape weights from the translated image to complete the retargeting of the animation onto a 3D target model. Our method allows for fully unsupervised retargeting of facial expressions between models of different configurations, and once trained, is suitable for automatic real-time applications.
  • Item
    Conservative Meshlet Bounds for Robust Culling of Skinned Meshes
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Unterguggenberger, Johannes; Kerbl, Bernhard; Pernsteiner, Jakob; Wimmer, Michael; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    Following recent advances in GPU hardware development and newly introduced rendering pipeline extensions, the segmentation of input geometry into small geometry clusters-so-called meshlets-has emerged as an important practice for efficient rendering of complex 3D models. Meshlets can be processed efficiently using mesh shaders on modern graphics processing units, in order to achieve streamlined geometry processing in just two tightly coupled shader stages that allow for dynamic workload manipulation in-between. The additional granularity layer between entire models and individual triangles enables new opportunities for fine-grained visibility culling methods. However, in contrast to static models, view frustum and backface culling on a per-meshlet basis for skinned, animated models are difficult to achieve while respecting the conservative spatio-temporal bounds that are required for robust rendering results. In this paper, we describe a solution for computing and exploiting relevant conservative bounds for culling meshlets of models that are animated using linear blend skinning. By enabling visibility culling for animated meshlets, our approach can help to improve rendering performance and alleviate bottlenecks in the notoriously performanceand memory-intensive skeletal animation pipelines of modern real-time graphics applications.
  • Item
    Diverse Dance Synthesis via Keyframes with Transformer Controllers
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Pan, Junjun; Wang, Siyuan; Bai, Junxuan; Dai, Ju; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    Existing keyframe-based motion synthesis mainly focuses on the generation of cyclic actions or short-term motion, such as walking, running, and transitions between close postures. However, these methods will significantly degrade the naturalness and diversity of the synthesized motion when dealing with complex and impromptu movements, e.g., dance performance and martial arts. In addition, current research lacks fine-grained control over the generated motion, which is essential for intelligent human-computer interaction and animation creation. In this paper, we propose a novel keyframe-based motion generation network based on multiple constraints, which can achieve diverse dance synthesis via learned knowledge. Specifically, the algorithm is mainly formulated based on the recurrent neural network (RNN) and the Transformer architecture. The backbone of our network is a hierarchical RNN module composed of two long short-term memory (LSTM) units, in which the first LSTM is utilized to embed the posture information of the historical frames into a latent space, and the second one is employed to predict the human posture for the next frame. Moreover, our framework contains two Transformer-based controllers, which are used to model the constraints of the root trajectory and the velocity factor respectively, so as to better utilize the temporal context of the frames and achieve fine-grained motion control. We verify the proposed approach on a dance dataset containing a wide range of contemporary dance. The results of three quantitative analyses validate the superiority of our algorithm. The video and qualitative experimental results demonstrate that the complex motion sequences generated by our algorithm can achieve diverse and smooth motion transitions between keyframes, even for long-term synthesis.
  • Item
    A Dynamic Mixture Model for Non-equilibrium Multiphase Fluids
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Jiang, Yuntao; Lan, Yingjie; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    We present a dynamic mixture model for simulating multiphase fluids with highly dynamic relative motions. The previous mixture models assume that the multiphase fluids are under a local equilibrium condition such that the drift velocity and the phase transport can be computed analytically. By doing so, it avoids solving multiple sets of Navier-Stokes equations and improves the simulation efficiency and stability. However, due to the local equilibrium assumption, these approaches can only deal with tightly coupled multiphase systems, where the relative speed between phases are assumed stable. In this work we abandon the local equilibrium assumption, and redesign the computation workflow of the mixture model to explicitly track and decouple the velocities of all phases. The phases still share the same pressure, with which we enforce the incompressibility for the mixture. The phase transport is calculated with drift velocities, and we propose a novel correction scheme to handle the transport at fluid boundaries to ensure mass conservation. Compared with previous mixture models, the proposed approach enables the simulation of much more dynamic scenarios with negligible extra overheads. In addition, it allows fluid control techniques to be applied to individual phases to generate locally dynamic and visually interesting effects.
  • Item
    A Lagrangian Particle-based Formulation for Coupled Simulation of Fracture and Diffusion in Thin Membranes
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Han, Chengguizi; Xue, Tao; Aanjaneya, Mridul; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    We propose a Lagrangian particle-based formulation for simulating deformation, fracture, and diffusion in thin membranelike structures, such as aluminium foil, rubbery films, and seaweed flakes. We integrate our model with diffusion processes and derive a unified framework for simulating deformation-diffusion coupled phenomena, which is applied to provide realistic heterogeneity induced by the diffusion process to fracture patterns. To the best of our knowledge, our work is the first to simulate the complex fracture patterns of single-layered membranes in computer graphics and introduce heterogeneity induced by the diffusion process, which generates more geometrically rich fracture patterns. Our end-to-end 3D simulations show that our deformation-diffusion coupling framework captures detailed fracture growth patterns in thin membranes due to both in-plane and out-of-plane motions, producing realistically wrinkled slit edges, and heterogeneity introduced due to diffusion.
  • Item
    Geometric Sample Reweighting for Monte Carlo Integration
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Guo, Jerry Jinfeng; Eisemann, Elmar; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    Numerical integration is fundamental in multiple Monte Carlo rendering problems. We present a sample reweighting scheme, including underlying theory, and analysis of numerical performance for the integration of an unknown one-dimensional function. Our method is simple to implement and builds upon the insight to link the weights to a function reconstruction process during integration. We provide proof that our solution is unbiased in one-dimensional cases and consistent in multi-dimensional cases. We illustrate its effectiveness in several use cases.
  • Item
    Consistent Post-Reconstruction for Progressive Photon Mapping
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Choi, Hajin; Moon, Bochang; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    Photon mapping is a light transport algorithm that simulates various rendering effects (e.g., caustics) robustly, and its progressive variants, progressive photon mapping (PPM) methods, can produce a biased but consistent rendering output. PPM estimates radiance using a kernel density estimation whose parameters (bandwidths) are adjusted progressively, and this refinement enables to reduce its estimation bias. Nonetheless, many iterations (and thus a large number of photons) are often required until PPM produces nearly converged estimates. This paper proposes a post-reconstruction that improves the performance of PPM by reducing residual errors in PPM estimates. Our key idea is to take multiple PPM estimates with multi-level correlation structures, and fuse the input images using a weight function trained by supervised learning with maintaining the consistency of PPM. We demonstrate that our technique boosts an existing PPM technique for various rendering scenes.
  • Item
    Neural Sequence Transformation
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Mukherjee, Sabyasachi; Mukherjee, Sayan; Hua, Binh-Son; Umetani, Nobuyuki; Meister, Daniel; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    Monte Carlo integration is a technique for numerically estimating a definite integral by stochastically sampling its integrand. These samples can be averaged to make an improved estimate, and the progressive estimates form a sequence that converges to the integral value on the limit. Unfortunately, the sequence of Monte Carlo estimates converges at a rate of O(pn), where n denotes the sample count, effectively slowing down as more samples are drawn. To overcome this, we can apply sequence transformation, which transforms one converging sequence into another with the goal of accelerating the rate of convergence. However, analytically finding such a transformation for Monte Carlo estimates can be challenging, due to both the stochastic nature of the sequence, and the complexity of the integrand. In this paper, we propose to leverage neural networks to learn sequence transformations that improve the convergence of the progressive estimates of Monte Carlo integration. We demonstrate the effectiveness of our method on several canonical 1D integration problems as well as applications in light transport simulation.
  • Item
    A Multi-pass Method for Accelerated Spectral Sampling
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Ruit, Mark van de; Eisemann, Elmar; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    Spectral Monte Carlo rendering can simulate advanced light phenomena, such as chromatic dispersion, but typically shows a slow convergence behavior. Properly sampling the spectral domain can be challenging in scenes with many complex spectral distributions. To this end, we propose a multi-pass approach. We build and store coarse screen-space estimates of incident spectral radiance and use these to then importance sample the spectral domain. Hereby, we lower variance and reduce noise with little overhead. Our method handles challenging scenarios with difficult spectral distributions, many different emitters, and participating media. Finally, it can be integrated into existing spectral rendering methods for an additional acceleration.
  • Item
    Z-Thickness Blending: Effective Fragment Merging for Multi-Fragment Rendering
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Kim, Dongjoon; Kye, Heewon; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    An effective fragment merging technique is presented in this study that addresses multi-fragment problems, including fragment overflow and z-fighting, and provides visual effects that are beneficial for various screen-space rendering algorithms. The proposed method merges locally adjacent fragments along the viewing direction to resolve the aforementioned problems based on cost-effective multi-layer representation and coplanar blending. We introduce a z-thickness model based on the radiosity spreading from the viewing z-direction. Moreover, we present the fragment-merging schemes and rules for determining the visibility of the merged fragments based on the proposed z-thickness model. The proposed method is targeted at multi-fragment rendering that handles individual fragments (e.g., k-buffer) instead of representing the fragments as an approximated transmittance function. In addition, our method provides a smooth visibility transition across overlapping fragments, resulting in visual advantages in various visualization applications. In this paper, we demonstrate the advantages of the proposed method through several screen-space rendering applications.
  • Item
    Optimizing Ray Tracing of Trimmed NURBS Surfaces on the GPU
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Sloup, Jaroslav; Havran, Vlastimil; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    The representation of geometric models by trimmed NURBS surfaces has become a standard in the CAD industry. In CAD applications, the rendering of surfaces is usually solved by tessellation followed up by z-buffer rendering. Ray tracing of NURBS surfaces has not been widely used in industry due to its computational complexity that hinders achieving real-time performance in practice. We propose novel methods achieving faster point location search needed by trimming in the context of ray tracing trimmed NURBS surfaces. The proposed 2D data structure based on kd-trees allows for faster ray tracing while it requires less memory for its representation and less preprocessing time than previously published methods. Further, we show the current state of the art for ray tracing trimmed NURBS surfaces on a GPU. With careful design and implementation, the number of rays cast on a GPU may reach real-time performance in the order of tens to hundreds of million rays per second for moderately to large complex scenes containing hundreds of thousands of NURBS surfaces and trimming curves.
  • Item
    Real-time Denoising Using BRDF Pre-integration Factorization
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Zhuang, Tao; Shen, Pengfei; Wang, Beibei; Liu, Ligang; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    Path tracing has been used for real-time renderings, thanks to the powerful GPU device. Unfortunately, path tracing produces noisy rendered results, thus, filtering or denoising is often applied as a post-process to remove the noise. Previous works produce high-quality denoised results, by accumulating the temporal samples. However, they cannot handle the details from bidirectional reflectance distribution function (BRDF) maps (e.g. roughness map). In this paper, we introduce the BRDF preintegration factorization for denoising to better preserve the details from BRDF maps. More specifically, we reformulate the rendering equation into two components: the BRDF pre-integration component and the weighted-lighting component. The BRDF pre-integration component is noise-free, since it does not depend on the lighting. Another key observation is that the weighted-lighting component tends to be smooth and low-frequency, which indicates that it is more suitable for denoising than the final rendered image. Hence, the weighted-lighting component is denoised individually. Our BRDF pre-integration demodulation approach is flexible for many real-time filtering methods. We have implemented it in spatio-temporal varianceguided filtering (SVGF), ReLAX and ReBLUR. Compared to the original methods, our method manages to better preserve the details from BRDF maps, while both the memory and time cost are negligible.
  • Item
    Luminance Attentive Networks for HDR Image and Panorama Reconstruction
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Yu, Hanning; Liu, Wentao; Long, Chengjiang; Dong, Bo; Zou, Qin; Xiao, Chunxia; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    It is very challenging to reconstruct a high dynamic range (HDR) from a low dynamic range (LDR) image as an ill-posed problem. This paper proposes a luminance attentive network named LANet for HDR reconstruction from a single LDR image. Our method is based on two fundamental observations: (1) HDR images stored in relative luminance are scale-invariant, which means the HDR images will hold the same information when multiplied by any positive real number. Based on this observation, we propose a novel normalization method called " HDR calibration " for HDR images stored in relative luminance, calibrating HDR images into a similar luminance scale according to the LDR images. (2) The main difference between HDR images and LDR images is in under-/over-exposed areas, especially those highlighted. Following this observation, we propose a luminance attention module with a two-stream structure for LANet to pay more attention to the under-/over-exposed areas. In addition, we propose an extended network called panoLANet for HDR panorama reconstruction from an LDR panorama and build a dualnet structure for panoLANet to solve the distortion problem caused by the equirectangular panorama. Extensive experiments show that our proposed approach LANet can reconstruct visually convincing HDR images and demonstrate its superiority over state-of-the-art approaches in terms of all metrics in inverse tone mapping. The image-based lighting application with our proposed panoLANet also demonstrates that our method can simulate natural scene lighting using only LDR panorama. Our source code is available at https://github.com/LWT3437/LANet.
  • Item
    Seamless Satellite-image Synthesis
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Zhu, Jialin; Kelly, Tom; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    We introduce Seamless Satellite-image Synthesis (SSS), a novel neural architecture to create scale-and-space continuous satellite textures from cartographic data. While 2D map data is cheap and easily synthesized, accurate satellite imagery is expensive and often unavailable or out of date. Our approach generates seamless textures over arbitrarily large spatial extents which are consistent through scale-space. To overcome tile size limitations in image-to-image translation approaches, SSS learns to remove seams between tiled images in a semantically meaningful manner. Scale-space continuity is achieved by a hierarchy of networks conditioned on style and cartographic data. Our qualitative and quantitative evaluations show that our system improves over the state-of-the-art in several key areas. We show applications to texturing procedurally generation maps and interactive satellite image manipulation.
  • Item
    Relighting Humans in the Wild: Monocular Full-Body Human Relighting with Domain Adaptation
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Tajima, Daichi; Kanamori, Yoshihiro; Endo, Yuki; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    The modern supervised approaches for human image relighting rely on training data generated from 3D human models. However, such datasets are often small (e.g., Light Stage data with a small number of individuals) or limited to diffuse materials (e.g., commercial 3D scanned human models). Thus, the human relighting techniques suffer from the poor generalization capability and synthetic-to-real domain gap. In this paper, we propose a two-stage method for single-image human relighting with domain adaptation. In the first stage, we train a neural network for diffuse-only relighting. In the second stage, we train another network for enhancing non-diffuse reflection by learning residuals between real photos and images reconstructed by the diffuse-only network. Thanks to the second stage, we can achieve higher generalization capability against various cloth textures, while reducing the domain gap. Furthermore, to handle input videos, we integrate illumination-aware deep video prior to greatly reduce flickering artifacts even with challenging settings under dynamic illuminations.
  • Item
    Write-An-Animation: High-level Text-based Animation Editing with Character-Scene Interaction
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Zhang, Jia-Qi; Xu, Xiang; Shen, Zhi-Meng; Huang, Ze-Huan; Zhao, Yang; Cao, Yan-Pei; Wan, Pengfei; Wang, Miao; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    3D animation production for storytelling requires essential manual processes of virtual scene composition, character creation, and motion editing, etc. Although professional artists can favorably create 3D animations using software, it remains a complex and challenging task for novice users to handle and learn such tools for content creation. In this paper, we present Write-An- Animation, a 3D animation system that allows novice users to create, edit, preview, and render animations, all through text editing. Based on the input texts describing virtual scenes and human motions in natural languages, our system first parses the texts as semantic scene graphs, then retrieves 3D object models for virtual scene composition and motion clips for character animation. Character motion is synthesized with the combination of generative locomotions using neural state machine as well as template action motions retrieved from the dataset. Moreover, to make the virtual scene layout compatible with character motion, we propose an iterative scene layout and character motion optimization algorithm that jointly considers characterobject collision and interaction. We demonstrate the effectiveness of our system with customized texts and public film scripts. Experimental results indicate that our system can generate satisfactory animations from texts.
  • Item
    A Motion-guided Interface for Modeling 3D Multi-functional Furniture
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Chen, Minchan; Lau, Manfred; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    While non-expert 3D design systems are helpful for performing conceptual design, most existing works focused on modeling static objects. However, the 3D modeling interfaces can include more interactions between the user and the models that are dynamic (and can be interacted with). In this paper, we propose a 3D modeling system for the conceptual design of interactable multi-functional furniture. Our contribution is in the design and development of a motion-guided interface. The key idea is that users should create interactable furniture components as if they are interacting with them with their hands. We conducted a preliminary user study to explore users' preferred hand gestures for creating various dynamic furniture components, implemented a 3D modeling system with the preferred gestures as a basis for the motion-guided user interface, and conducted an evaluation user study to demonstrate that our user interface is user-friendly and efficient for novice designers to perform conceptual furniture designs.
  • Item
    Egocentric Network Exploration for Immersive Analytics
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Sorger, Johannes; Arleo, Alessio; Kán, Peter; Knecht, Wolfgang; Waldner, Manuela; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    To exploit the potential of immersive network analytics for engaging and effective exploration, we promote the metaphor of ''egocentrism'', where data depiction and interaction are adapted to the perspective of the user within a 3D network. Egocentrism has the potential to overcome some of the inherent downsides of virtual environments, e.g., visual clutter and cyber-sickness. To investigate the effect of this metaphor on immersive network exploration, we designed and evaluated interfaces of varying degrees of egocentrism. In a user study, we evaluated the effect of these interfaces on visual search tasks, efficiency of network traversal, spatial orientation, as well as cyber-sickness. Results show that a simple egocentric interface considerably improves visual search efficiency and navigation performance, yet does not decrease spatial orientation or increase cyber-sickness. An occlusion-free Ego-Bubble view of the neighborhood only marginally improves the user's performance. We tie our findings together in an open online tool for egocentric network exploration, providing actionable insights on the benefits of the egocentric network exploration metaphor
  • Item
    Interactive Analysis of CNN Robustness
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Sietzen, Stefan; Lechner, Mathias; Borowski, Judy; Hasani, Ramin; Waldner, Manuela; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    While convolutional neural networks (CNNs) have found wide adoption as state-of-the-art models for image-related tasks, their predictions are often highly sensitive to small input perturbations, which the human vision is robust against. This paper presents Perturber, a web-based application that allows users to instantaneously explore how CNN activations and predictions evolve when a 3D input scene is interactively perturbed. Perturber offers a large variety of scene modifications, such as camera controls, lighting and shading effects, background modifications, object morphing, as well as adversarial attacks, to facilitate the discovery of potential vulnerabilities. Fine-tuned model versions can be directly compared for qualitative evaluation of their robustness. Case studies with machine learning experts have shown that Perturber helps users to quickly generate hypotheses about model vulnerabilities and to qualitatively compare model behavior. Using quantitative analyses, we could replicate users' insights with other CNN architectures and input images, yielding new insights about the vulnerability of adversarially trained models.
  • Item
    UprightRL: Upright Orientation Estimation of 3D Shapes via Reinforcement Learning
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Chen, Luanmin; Xu, Juzhan; Wang, Chuan; Huang, Haibin; Huang, Hui; Hu, Ruizhen; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    In this paper, we study the problem of 3D shape upright orientation estimation from the perspective of reinforcement learning, i.e. we teach a machine (agent) to orientate 3D shapes step by step to upright given its current observation. Unlike previous methods, we take this problem as a sequential decision-making process instead of a strong supervised learning problem. To achieve this, we propose UprightRL, a deep network architecture designed for upright orientation estimation. UprightRL mainly consists of two submodules: an Actor module and a Critic module which can be learned with a reinforcement learning manner. Specifically, the Actor module selects an action from the action space to perform a point cloud transformation and obtain the new point cloud for the next environment state, while the Critic module evaluates the strategy and guides the Actor to choose the next stage action. Moreover, we design a reward function that encourages the agent to select action which is conducive to orient model towards upright orientation with a positive reward and negative otherwise. We conducted extensive experiments to demonstrate the effectiveness of the proposed model, and experimental results show that our network outperforms the stateof- the-art. We also apply our method to the robot grasping-and-placing experiment, to reveal the practicability of our method.
  • Item
    Synthesizing Geologically Coherent Cave Networks
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Paris, Axel; Guérin, Eric; Peytavie, Adrien; Collon, Pauline; Galin, Eric; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    We present a geologically-based method to generate complex karstic networks. Karsts are a type of landscape formed by the dissolution of highly soluble rocks (generally limestones). In particular, they are characterized by complex underground networks made of varieties of tunnels and breakout chambers with stalagmites and stalactites. Our method computes skeletons of karstic networks by using a gridless anisotropic shortest path algorithm according to field data of the underground system (such as inlets and outlets), geomorphological features and parameters such as faults, inception horizons, fractures, and permeability contrasts. From this skeleton, we define the geometry of the conduits as a signed distance function construction tree combining primitives with blending and warping operators. Our framework provides multiple levels of control, allowing us to author both the structure of the karstic network and the geometric cross-section shapes and details of the generated conduits.
  • Item
    Manhattan-world Urban Building Reconstruction by Fitting Cubes
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) He, Zhenbang; Wang, Yunhai; Cheng, Zhanglin; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    The Manhattan-world building is a kind of dominant scene in urban areas. Many existing methods for reconstructing such scenes are either vulnerable to noisy and incomplete data or suffer from high computational complexity. In this paper, we present a novel approach to quickly reconstruct lightweight Manhattan-world urban building models from images. Our key idea is to reconstruct buildings through the salient feature - corners. Given a set of urban building images, Structure-from- Motion and 3D line reconstruction operations are applied first to recover camera poses, sparse point clouds, and line clouds. Then we use orthogonal planes detected from the line cloud to generate corners, which indicate a part of possible buildings. Starting from the corners, we fit cubes to point clouds by optimizing corner parameters and obtain cube representations of corresponding buildings. Finally, a registration step is performed on cube representations to generate more accurate models. Experiment results show that our approach can handle some nasty cases containing noisy and incomplete data, meanwhile, output lightweight polygonal building models with a low time-consuming.