43-Issue 7

Permanent URI for this collection

Pacific Graphics 2024 - Symposium Proceedings
Huangshan (Yellow Mountain), China || October 13 – 16, 2024

(for Conference Papers and Posters see PG 2024 - Conference Papers and Posters)
3D Reconstruction and Novel View Synthesis I
Seamless and Aligned Texture Optimization for 3D Reconstruction
Lei Wang, Linlin Ge, Qitong Zhang, and Jieqing Feng
GS-Octree: Octree-based 3D Gaussian Splatting for Robust Object-level 3D Reconstruction Under Strong Lighting
Jiaze Li, Zhengyu Wen, Luo Zhang, Jiangbei Hu, Fei Hou, Zhebin Zhang, and Ying He
TaNSR: Efficient 3D Reconstruction with Tetrahedral Difference and Feature Aggregation
Zhaohan Lv, Xingcan Bao, Yong Tang, and Jing Zhao
CoupNeRF: Property-aware Neural Radiance Fields for Multi-Material Coupled Scenario Reconstruction
Jin Li, Yang Gao, Wenfeng Song, Yacong Li, Shuai Li, Aimin Hao, and Hong Qin
Image and Video Enhancement I
A TransISP Based Image Enhancement Method for Visual Disbalance in Low-light Images
Jiaqi Wu, Jing Guo, Rui Jing, Shihao Zhang, Zijian Tian, Wei Chen, and Zehua Wang
Exploring Fast and Flexible Zero-Shot Low-Light Image/Video Enhancement
Xianjun Han, Taoli Bao, and Hongyu Yang
TempDiff: Enhancing Temporal-awareness in Latent Diffusion for Real-World Video Super-Resolution
Qin Jiang, Qing Lin Wang, Li Hua Chi, Xin Hai Chen, Qing Yang Zhang, Richard Zhou, Zheng Qiu Deng, Jin Sheng Deng, Bin Bing Tang, Shao He Lv, and Jie Liu
Distinguishing Structures from Textures by Patch-based Contrasts around Pixels for High-quality and Efficient Texture filtering
Shengchun Wang, Panpan Xu, Fei Hou, Wencheng Wang, and Chong Zhao
3D Reconstruction and Novel View Synthesis II
Gaussian in the Dark: Real-Time View Synthesis From Inconsistent Dark Images Using Gaussian Splatting
Sheng Ye, Zhen-Hui Dong, Yubin Hu, Yu-Hui Wen, and Yong-Jin Liu
Cinematic Gaussians: Real-Time HDR Radiance Fields with Depth of Field
Chao Wang, Krzysztof Wolski, Bernhard Kerbl, Ana Serrano, Mojtaba Bemamai, Hans-Peter Seidel, Karol Myszkowsk, and Thomas Leimkühler
GSEditPro: 3D Gaussian Splatting Editing with Attention-based Progressive Localization
Yanhao Sun, Runze Tian, Xiao Han, Xinyao Liu, Yan Zhang, and Kai Xu
Point Cloud Processing and Analysis II
GETr: A Geometric Equivariant Transformer for Point Cloud Registration
Chang Yu, Sanguo Zhang, and Li-Yong Shen
PCLC-Net: Point Cloud Completion in Arbitrary Poses using Learnable Canonical Space
Hanmo Xu, Qingyao Shuai, and Xuejin Chen
DSGI-Net: Density-based Selective Grouping Point Cloud Learning Network for Indoor Scene
Xin Wen, Yao Duan, Kai Xu, and Chenyang Zhu
Point-AGM : Attention Guided Masked Auto-Encoder for Joint Self-supervised Learning on Point Clouds
Jie Liu, Mengna Yang, Yu Tian, Yancui Li, Da Song, Kang Li, and Xin Cao
Image and Video Enhancement II
Frequency-Aware Facial Image Shadow Removal through Skin Color and Texture Learning
Ling Zhang, Wenyang Xie, and Chunxia Xiao
Density-Aware Diffusion Model for Efficient Image Dehazing
Ling Zhang, Wenxu Bai, and Chunxia Xiao
MISNeR: Medical Implicit Shape Neural Representation for Image Volume Visualisation
Ge Jin, Younhyun Jung, Lei Bi, and Jinman Kim
Geometric Processing I
Surface Cutting and Flattening to Target Shapes
Yuanhao Li, Wenzheng Wu, and Ligang Liu
Variable Offsets and Processing of Implicit Forms Toward the Adaptive Synthesis and Analysis of Heterogeneous Conforming Microstructure
Q. Youn Hong, Pablo Antolin, Gershon Elber, and Myung-Soo Kim
Rendering and Lighting I
Ray Tracing Animated Displaced Micro-Meshes
Holger Gruen, Carsten Benthin, Andrew Kensler, Joshua Barczak, and David McAllister
Faster Ray Tracing through Hierarchy Cut Code
Weilai Xiang, Fengqi Liu, Zaonan Tan, Dan Li, PengZhan Xu, MeiZhi Liu, and QiLong Kou
CrystalNet: Texture-Aware Neural Refraction Baking for Global Illumination
Ziyang Zhang and Edgar Simo-Serra
Human and Character Animation
Spatially and Temporally Optimized Audio-Driven Talking Face Generation
Biao Dong, Bo-Yao Ma, and Lei Zhang
Disentangled Lifespan Synthesis via Transformer-Based Nonlinear Regression
Mingyuan Li and Yingchun Guo
Geometric Processing II
Multiscale Spectral Manifold Wavelet Regularizer for Unsupervised Deep Functional Maps
Haibo Wang, Jing Meng, Qinsong Li, Ling Hu, Yueyu Guo, Xinru Liu, Xiaoxia Yang, and Shengjun Liu
FSH3D: 3D Representation via Fibonacci Spherical Harmonics
Zikuan Li, Anyi Huang, Wenru Jia, Qiaoyun Wu, Mingqiang Wei, and Jun Wang
Curved Image Triangulation Based on Differentiable Rendering
Wanyi Wang, Zhonggui Chen, Lincong Fang, and Juan Cao
Rendering and Lighting II
Anisotropic Specular Image-Based Lighting Based on BRDF Major Axis Sampling
Giovanni Cocco, Cédric Zanni, and Xavier Chermain
NeuPreSS: Compact Neural Precomputed Subsurface Scattering for Distant Lighting of Heterogeneous Translucent Objects
Thomson Tg, Jeppe Revall Frisvad, Ravi Ramamoorthi, and Henrik W. Jensen
A Surface-based Appearance Model for Pennaceous Feathers
Juan Raúl Padrón-Griffe, Dario Lanza, Adrian Jarabo, and Adolfo Muñoz
Crowd and Scene Analysis
GLTScene: Global-to-Local Transformers for Indoor Scene Synthesis with General Room Boundaries
Yijie Li, Pengfei Xu, Junquan Ren, Zefan Shao, and Hui Huang
Evolutive 3D Urban Data Representation through Timeline Design Space
Corentin Le Bihan Gautier, Johanna Delanoy, and Gilles Gesquière
LightUrban: Similarity Based Fine-grained Instancing for Lightweighting Complex Urban Point Clouds
Zi Ang Lu, Wei Dan Xiong, Peng Ren, and Jin Yuan Jia
Curve and Surface Modeling
Disk B-spline on S2: A Skeleton-based Region Representation on 2-Sphere
Chunhao Zheng, Yuming Zhao, Zhongke Wu, and Xingce Wang
A Hybrid Parametrization Method for B-Spline Curve Interpolation via Supervised Learning
Tianyu Song, Tong Shen, Linlin Ge, and Jieqing Feng
Strictly Conservative Neural Implicits
Ingmar Ludwig and Marcel Campen
Symmetric Piecewise Developable Approximations
Ying He, Qing Fang, Zheng Zhang, Tielin Dai, Kang Wu, Ligang Liu, and Xiao-Ming Fu
Simulation
FastFlow: GPU Acceleration of Flow and Depression Routing for Landscape Simulation
Aryamaan Jain, Bernhard Kerbl, James Gain, Brandon Finley, and Guillaume Cordonnier
Image Processing and Filtering I
P-Hologen: An End-to-End Generative Framework for Phase-Only Holograms
JooHyun Park, YuJin Jeon, HuiYong Kim, SeungHwan Baek, and HyeongYeop Kang
Image Synthesis
Controllable Anime Image Editing Based on the Probability of Attribute Tags
Zhenghao Song, Haoran Mo, and Chengying Gao
DiffPop: Plausibility-Guided Object Placement Diffusion for Image Composition
Jiacheng Liu, Hang Zhou, Shida Wei, and Rui Ma
CustomSketching: Sketch Concept Extraction for Sketch-based Image Synthesis and Editing
Chufeng Xiao and Hongbo Fu
Garment Modeling and Simulation
Digital Garment Alteration
Anna Maria Eggler, Raphael Falque, Mark Liu, Teresa Vidal-Calleja, Olga Sorkine-Hornung, and Nico Pietroni
Inverse Garment and Pattern Modeling with a Differentiable Simulator
Boyang Yu, Frederic Cordier, and Hyewon Seo
Image Processing and Filtering II
Adversarial Unsupervised Domain Adaptation for 3D Semantic Segmentation with 2D Image Fusion of Dense Depth
Xindan Zhang, Ying Li, Huankun Sheng, and Xinnian Zhang
SOD-diffusion: Salient Object Detection via Diffusion-Based Image Generators
Shuo Zhang, Jiaming Huang, Shizhe Chen, Yan Wu, Tao Hu, and Jing Liu
Color-Accurate Camera Capture with Multispectral Illumination and Multiple Exposures
Hongyun Gao, Rafal K. Mantiuk, and Graham D. Finlayson
3D Modeling and Editing
iShapEditing: Intelligent Shape Editing with Diffusion Models
Jing Li, Juyong Zhang, and Falai Chen
VRTree: Example-Based 3D Interactive Tree Modeling in Virtual Reality
Di Wu, Mingxin Yang, Zhihao Liu, Fangyuan Tu, Fang Liu, and Zhanglin Cheng
Neural Radiance Fields and Gaussian Splatting
SCARF: Scalable Continual Learning Framework for Memory-efficiency Multiple Neural Radiance Fields
Yuze Wang, Junyi Wang, Chen Wang, Wantong Duan, Yongtang Bao, and Yue Qi
GauLoc: 3D Gaussian Splatting-based Camera Relocalization
Zhe Xin, Chengkai Dai, Ying Li, and Chenming Wu
LGSur-Net: A Local Gaussian Surface Representation Network for Upsampling Highly Sparse Point Cloud
Zijian Xiao, Tianchen Zhou, and Li Yao
Advanced 3D Synthesis and Stylization
Palette-Based Recolouring of Gradient Meshes
Willard A. Verschoore de la Houssaije, Jose Echevarria, and Jirí Kosinka
G-Style: Stylized Gaussian Splatting
Áron Samuel Kovács, Pedro Hermosilla, and Renata Georgia Raidou
Human II
Robust Diffusion-based Motion In-betweening
Jia Qin, Peng Yan, and Bo An

BibTeX (43-Issue 7)
                
@article{
10.1111:cgf.14853,
journal = {Computer Graphics Forum}, title = {{
Pacific Graphics 2024 - CGF 43-7: Frontmatter}},
author = {
Chen, Renjie
and
Ritschel, Tobias
and
Whiting, Emily
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.14853}
}
                
@article{
10.1111:cgf.15205,
journal = {Computer Graphics Forum}, title = {{
Seamless and Aligned Texture Optimization for 3D Reconstruction}},
author = {
Wang, Lei
and
Ge, Linlin
and
Zhang, Qitong
and
Feng, Jieqing
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15205}
}
                
@article{
10.1111:cgf.15206,
journal = {Computer Graphics Forum}, title = {{
GS-Octree: Octree-based 3D Gaussian Splatting for Robust Object-level 3D Reconstruction Under Strong Lighting}},
author = {
Li, Jiaze
and
Wen, Zhengyu
and
Zhang, Luo
and
Hu, Jiangbei
and
Hou, Fei
and
Zhang, Zhebin
and
He, Ying
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15206}
}
                
@article{
10.1111:cgf.15207,
journal = {Computer Graphics Forum}, title = {{
TaNSR: Efficient 3D Reconstruction with Tetrahedral Difference and Feature Aggregation}},
author = {
Lv, Zhaohan
and
Bao, Xingcan
and
Tang, Yong
and
Zhao, Jing
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15207}
}
                
@article{
10.1111:cgf.15208,
journal = {Computer Graphics Forum}, title = {{
CoupNeRF: Property-aware Neural Radiance Fields for Multi-Material Coupled Scenario Reconstruction}},
author = {
Li, Jin
and
Gao, Yang
and
Song, Wenfeng
and
Li, Yacong
and
Li, Shuai
and
Hao, Aimin
and
Qin, Hong
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15208}
}
                
@article{
10.1111:cgf.15209,
journal = {Computer Graphics Forum}, title = {{
A TransISP Based Image Enhancement Method for Visual Disbalance in Low-light Images}},
author = {
Wu, Jiaqi
and
Guo, Jing
and
Jing, Rui
and
Zhang, Shihao
and
Tian, Zijian
and
Chen, Wei
and
Wang, Zehua
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15209}
}
                
@article{
10.1111:cgf.15210,
journal = {Computer Graphics Forum}, title = {{
Exploring Fast and Flexible Zero-Shot Low-Light Image/Video Enhancement}},
author = {
Han, Xianjun
and
Bao, Taoli
and
Yang, Hongyu
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15210}
}
                
@article{
10.1111:cgf.15211,
journal = {Computer Graphics Forum}, title = {{
TempDiff: Enhancing Temporal-awareness in Latent Diffusion for Real-World Video Super-Resolution}},
author = {
Jiang, Qin
and
Wang, Qing Lin
and
Liu, Jie
and
Chi, Li Hua
and
Chen, Xin Hai
and
Zhang, Qing Yang
and
Zhou, Richard
and
Deng, Zheng Qiu
and
Deng, Jin Sheng
and
Tang, Bin Bing
and
Lv, Shao He
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15211}
}
                
@article{
10.1111:cgf.15212,
journal = {Computer Graphics Forum}, title = {{
Distinguishing Structures from Textures by Patch-based Contrasts around Pixels for High-quality and Efficient Texture filtering}},
author = {
Wang, Shengchun
and
Xu, Panpan
and
Hou, Fei
and
Wang, Wencheng
and
Zhao, Chong
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15212}
}
                
@article{
10.1111:cgf.15213,
journal = {Computer Graphics Forum}, title = {{
Gaussian in the Dark: Real-Time View Synthesis From Inconsistent Dark Images Using Gaussian Splatting}},
author = {
Ye, Sheng
and
Dong, Zhen-Hui
and
Hu, Yubin
and
Wen, Yu-Hui
and
Liu, Yong-Jin
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15213}
}
                
@article{
10.1111:cgf.15214,
journal = {Computer Graphics Forum}, title = {{
Cinematic Gaussians: Real-Time HDR Radiance Fields with Depth of Field}},
author = {
Wang, Chao
and
Wolski, Krzysztof
and
Kerbl, Bernhard
and
Serrano, Ana
and
Bemama, Mojtaba
and
Seidel, Hans-Peter
and
Myszkowski, Karol
and
Leimkühler, Thomas
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15214}
}
                
@article{
10.1111:cgf.15215,
journal = {Computer Graphics Forum}, title = {{
GSEditPro: 3D Gaussian Splatting Editing with Attention-based Progressive Localization}},
author = {
Sun, Yanhao
and
Tian, Runze
and
Han, Xiao
and
Liu, Xinyao
and
Zhang, Yan
and
Xu, Kai
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15215}
}
                
@article{
10.1111:cgf.15216,
journal = {Computer Graphics Forum}, title = {{
GETr: A Geometric Equivariant Transformer for Point Cloud Registration}},
author = {
Yu, Chang
and
Zhang, Sanguo
and
Shen, Li-Yong
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15216}
}
                
@article{
10.1111:cgf.15217,
journal = {Computer Graphics Forum}, title = {{
PCLC-Net: Point Cloud Completion in Arbitrary Poses using Learnable Canonical Space}},
author = {
Xu, Hanmo
and
Shuai, Qingyao
and
Chen, Xuejin
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15217}
}
                
@article{
10.1111:cgf.15218,
journal = {Computer Graphics Forum}, title = {{
DSGI-Net: Density-based Selective Grouping Point Cloud Learning Network for Indoor Scene}},
author = {
Wen, Xin
and
Duan, Yao
and
Xu, Kai
and
Zhu, Chenyang
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15218}
}
                
@article{
10.1111:cgf.15219,
journal = {Computer Graphics Forum}, title = {{
Point-AGM : Attention Guided Masked Auto-Encoder for Joint Self-supervised Learning on Point Clouds}},
author = {
Liu, Jie
and
Yang, Mengna
and
Tian, Yu
and
Li, Yancui
and
Song, Da
and
Li, Kang
and
Cao, Xin
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15219}
}
                
@article{
10.1111:cgf.15220,
journal = {Computer Graphics Forum}, title = {{
Frequency-Aware Facial Image Shadow Removal through Skin Color and Texture Learning}},
author = {
Zhang, Ling
and
Xie, Wenyang
and
Xiao, Chunxia
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15220}
}
                
@article{
10.1111:cgf.15221,
journal = {Computer Graphics Forum}, title = {{
Density-Aware Diffusion Model for Efficient Image Dehazing}},
author = {
Zhang, Ling
and
Bai, Wenxu
and
Xiao, Chunxia
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15221}
}
                
@article{
10.1111:cgf.15222,
journal = {Computer Graphics Forum}, title = {{
MISNeR: Medical Implicit Shape Neural Representation for Image Volume Visualisation}},
author = {
Jin, Ge
and
Jung, Younhyun
and
Bi, Lei
and
Kim, Jinman
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15222}
}
                
@article{
10.1111:cgf.15223,
journal = {Computer Graphics Forum}, title = {{
Surface Cutting and Flattening to Target Shapes}},
author = {
Li, Yuanhao
and
Wu, Wenzheng
and
Liu, Ligang
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15223}
}
                
@article{
10.1111:cgf.15224,
journal = {Computer Graphics Forum}, title = {{
Variable Offsets and Processing of Implicit Forms Toward the Adaptive Synthesis and Analysis of Heterogeneous Conforming Microstructure}},
author = {
Hong, Q. Youn
and
Antolin, Pablo
and
Elber, Gershon
and
Kim, Myung-Soo
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15224}
}
                
@article{
10.1111:cgf.15225,
journal = {Computer Graphics Forum}, title = {{
Ray Tracing Animated Displaced Micro-Meshes}},
author = {
Gruen, Holger
and
Benthin, Carsten
and
Kensler, Andrew
and
Barczak, Joshua
and
McAllister, David
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15225}
}
                
@article{
10.1111:cgf.15226,
journal = {Computer Graphics Forum}, title = {{
Faster Ray Tracing through Hierarchy Cut Code}},
author = {
Xiang, WeiLai
and
Liu, FengQi
and
Tan, Zaonan
and
Li, Dan
and
Xu, PengZhan
and
Liu, MeiZhi
and
Kou, QiLong
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15226}
}
                
@article{
10.1111:cgf.15227,
journal = {Computer Graphics Forum}, title = {{
CrystalNet: Texture-Aware Neural Refraction Baking for Global Illumination}},
author = {
Zhang, Ziyang
and
Simo-Serra, Edgar
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15227}
}
                
@article{
10.1111:cgf.15228,
journal = {Computer Graphics Forum}, title = {{
Spatially and Temporally Optimized Audio-Driven Talking Face Generation}},
author = {
Dong, Biao
and
Ma, Bo-Yao
and
Zhang, Lei
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15228}
}
                
@article{
10.1111:cgf.15229,
journal = {Computer Graphics Forum}, title = {{
Disentangled Lifespan Synthesis via Transformer-Based Nonlinear Regression}},
author = {
Li, Mingyuan
and
Guo, Yingchun
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15229}
}
                
@article{
10.1111:cgf.15230,
journal = {Computer Graphics Forum}, title = {{
Multiscale Spectral Manifold Wavelet Regularizer for Unsupervised Deep Functional Maps}},
author = {
Wang, Haibo
and
Meng, Jing
and
Li, Qinsong
and
Hu, Ling
and
Guo, Yueyu
and
Liu, Xinru
and
Yang, Xiaoxia
and
Liu, Shengjun
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15230}
}
                
@article{
10.1111:cgf.15231,
journal = {Computer Graphics Forum}, title = {{
FSH3D: 3D Representation via Fibonacci Spherical Harmonics}},
author = {
Li, Zikuan
and
Huang, Anyi
and
Jia, Wenru
and
Wu, Qiaoyun
and
Wei, Mingqiang
and
Wang, Jun
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15231}
}
                
@article{
10.1111:cgf.15232,
journal = {Computer Graphics Forum}, title = {{
Curved Image Triangulation Based on Differentiable Rendering}},
author = {
Wang, Wanyi
and
Chen, Zhonggui
and
Fang, Lincong
and
Cao, Juan
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15232}
}
                
@article{
10.1111:cgf.15233,
journal = {Computer Graphics Forum}, title = {{
Anisotropic Specular Image-Based Lighting Based on BRDF Major Axis Sampling}},
author = {
Cocco, Giovanni
and
Zanni, Cédric
and
Chermain, Xavier
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15233}
}
                
@article{
10.1111:cgf.15234,
journal = {Computer Graphics Forum}, title = {{
NeuPreSS: Compact Neural Precomputed Subsurface Scattering for Distant Lighting of Heterogeneous Translucent Objects}},
author = {
Tg, Thomson
and
Frisvad, Jeppe Revall
and
Ramamoorthi, Ravi
and
Jensen, Henrik W.
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15234}
}
                
@article{
10.1111:cgf.15235,
journal = {Computer Graphics Forum}, title = {{
A Surface-based Appearance Model for Pennaceous Feathers}},
author = {
Padrón-Griffe, Juan Raúl
and
Lanza, Dario
and
Jarabo, Adrian
and
Muñoz, Adolfo
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15235}
}
                
@article{
10.1111:cgf.15236,
journal = {Computer Graphics Forum}, title = {{
GLTScene: Global-to-Local Transformers for Indoor Scene Synthesis with General Room Boundaries}},
author = {
Li, Yijie
and
Xu, Pengfei
and
Ren, Junquan
and
Shao, Zefan
and
Huang, Hui
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15236}
}
                
@article{
10.1111:cgf.15237,
journal = {Computer Graphics Forum}, title = {{
Evolutive 3D Urban Data Representation through Timeline Design Space}},
author = {
Gautier, Corentin Le Bihan
and
Delanoy, Johanna
and
Gesquière, Gilles
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15237}
}
                
@article{
10.1111:cgf.15238,
journal = {Computer Graphics Forum}, title = {{
LightUrban: Similarity Based Fine-grained Instancing for Lightweighting Complex Urban Point Clouds}},
author = {
Lu, Zi Ang
and
Xiong, Wei Dan
and
Ren, Peng
and
Jia, Jin Yuan
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15238}
}
                
@article{
10.1111:cgf.15239,
journal = {Computer Graphics Forum}, title = {{
Disk B-spline on S2: A Skeleton-based Region Representation on 2-Sphere}},
author = {
Zheng, Chunhao
and
Zhao, Yuming
and
Wu, Zhongke
and
Wang, Xingce
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15239}
}
                
@article{
10.1111:cgf.15240,
journal = {Computer Graphics Forum}, title = {{
A Hybrid Parametrization Method for B-Spline Curve Interpolation via Supervised Learning}},
author = {
Song, Tianyu
and
Shen, Tong
and
Ge, Linlin
and
Feng, Jieqing
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15240}
}
                
@article{
10.1111:cgf.15241,
journal = {Computer Graphics Forum}, title = {{
Strictly Conservative Neural Implicits}},
author = {
Ludwig, Ingmar
and
Campen, Marcel
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15241}
}
                
@article{
10.1111:cgf.15242,
journal = {Computer Graphics Forum}, title = {{
Symmetric Piecewise Developable Approximations}},
author = {
He, Ying
and
Fang, Qing
and
Zhang, Zheng
and
Dai, Tielin
and
Wu, Kang
and
Liu, Ligang
and
Fu, Xiao-Ming
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15242}
}
                
@article{
10.1111:cgf.15243,
journal = {Computer Graphics Forum}, title = {{
FastFlow: GPU Acceleration of Flow and Depression Routing for Landscape Simulation}},
author = {
Jain, Aryamaan
and
Kerbl, Bernhard
and
Gain, James
and
Finley, Brandon
and
Cordonnier, Guillaume
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15243}
}
                
@article{
10.1111:cgf.15244,
journal = {Computer Graphics Forum}, title = {{
P-Hologen: An End-to-End Generative Framework for Phase-Only Holograms}},
author = {
Park, JooHyun
and
Jeon, YuJin
and
Kim, HuiYong
and
Baek, SeungHwan
and
Kang, HyeongYeop
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15244}
}
                
@article{
10.1111:cgf.15245,
journal = {Computer Graphics Forum}, title = {{
Controllable Anime Image Editing Based on the Probability of Attribute Tags}},
author = {
Song, Zhenghao
and
Mo, Haoran
and
Gao, Chengying
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15245}
}
                
@article{
10.1111:cgf.15246,
journal = {Computer Graphics Forum}, title = {{
DiffPop: Plausibility-Guided Object Placement Diffusion for Image Composition}},
author = {
Liu, Jiacheng
and
Zhou, Hang
and
Wei, Shida
and
Ma, Rui
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15246}
}
                
@article{
10.1111:cgf.15247,
journal = {Computer Graphics Forum}, title = {{
CustomSketching: Sketch Concept Extraction for Sketch-based Image Synthesis and Editing}},
author = {
Xiao, Chufeng
and
Fu, Hongbo
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15247}
}
                
@article{
10.1111:cgf.15248,
journal = {Computer Graphics Forum}, title = {{
Digital Garment Alteration}},
author = {
Eggler, Anna Maria
and
Falque, Raphael
and
Liu, Mark
and
Vidal-Calleja, Teresa
and
Sorkine-Hornung, Olga
and
Pietroni, Nico
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15248}
}
                
@article{
10.1111:cgf.15249,
journal = {Computer Graphics Forum}, title = {{
Inverse Garment and Pattern Modeling with a Differentiable Simulator}},
author = {
Yu, Boyang
and
Cordier, Frederic
and
Seo, Hyewon
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15249}
}
                
@article{
10.1111:cgf.15250,
journal = {Computer Graphics Forum}, title = {{
Adversarial Unsupervised Domain Adaptation for 3D Semantic Segmentation with 2D Image Fusion of Dense Depth}},
author = {
Zhang, Xindan
and
Li, Ying
and
Sheng, Huankun
and
Zhang, Xinnian
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15250}
}
                
@article{
10.1111:cgf.15251,
journal = {Computer Graphics Forum}, title = {{
SOD-diffusion: Salient Object Detection via Diffusion-Based Image Generators}},
author = {
Zhang, Shuo
and
Huang, Jiaming
and
Chen, Shizhe
and
Wu, Yan
and
Hu, Tao
and
Liu, Jing
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15251}
}
                
@article{
10.1111:cgf.15252,
journal = {Computer Graphics Forum}, title = {{
Color-Accurate Camera Capture with Multispectral Illumination and Multiple Exposures}},
author = {
Gao, Hongyun
and
Mantiuk, Rafal K.
and
Finlayson, Graham D.
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15252}
}
                
@article{
10.1111:cgf.15253,
journal = {Computer Graphics Forum}, title = {{
iShapEditing: Intelligent Shape Editing with Diffusion Models}},
author = {
Li, Jing
and
Zhang, Juyong
and
Chen, Falai
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15253}
}
                
@article{
10.1111:cgf.15254,
journal = {Computer Graphics Forum}, title = {{
VRTree: Example-Based 3D Interactive Tree Modeling in Virtual Reality}},
author = {
Wu, Di
and
Yang, Mingxin
and
Liu, Zhihao
and
Tu, Fangyuan
and
Liu, Fang
and
Cheng, Zhanglin
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15254}
}
                
@article{
10.1111:cgf.15255,
journal = {Computer Graphics Forum}, title = {{
SCARF: Scalable Continual Learning Framework for Memory-efficiency Multiple Neural Radiance Fields}},
author = {
Wang, Yuze
and
Wang, Junyi
and
Wang, Chen
and
Duan, Wantong
and
Bao, Yongtang
and
Qi, Yue
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15255}
}
                
@article{
10.1111:cgf.15256,
journal = {Computer Graphics Forum}, title = {{
GauLoc: 3D Gaussian Splatting-based Camera Relocalization}},
author = {
Xin, Zhe
and
Dai, Chengkai
and
Li, Ying
and
Wu, Chenming
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15256}
}
                
@article{
10.1111:cgf.15257,
journal = {Computer Graphics Forum}, title = {{
LGSur-Net: A Local Gaussian Surface Representation Network for Upsampling Highly Sparse Point Cloud}},
author = {
Xiao, Zijian
and
Zhou, Tianchen
and
Yao, Li
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15257}
}
                
@article{
10.1111:cgf.15258,
journal = {Computer Graphics Forum}, title = {{
Palette-Based Recolouring of Gradient Meshes}},
author = {
Houssaije, Willard A. Verschoore de la
and
Echevarria, Jose
and
Kosinka, Jirí
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15258}
}
                
@article{
10.1111:cgf.15259,
journal = {Computer Graphics Forum}, title = {{
G-Style: Stylized Gaussian Splatting}},
author = {
Kovács, Áron Samuel
and
Hermosilla, Pedro
and
Raidou, Renata Georgia
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15259}
}
                
@article{
10.1111:cgf.15260,
journal = {Computer Graphics Forum}, title = {{
Robust Diffusion-based Motion In-betweening}},
author = {
Qin, Jia
and
Yan, Peng
and
An, Bo
}, year = {
2024},
publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {
10.1111/cgf.15260}
}

Browse

Recent Submissions

Now showing 1 - 57 of 57
  • Item
    Pacific Graphics 2024 - CGF 43-7: Frontmatter
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Chen, Renjie; Ritschel, Tobias; Whiting, Emily; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
  • Item
    Seamless and Aligned Texture Optimization for 3D Reconstruction
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Wang, Lei; Ge, Linlin; Zhang, Qitong; Feng, Jieqing; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Restoring the appearance of the model is a crucial step for achieving realistic 3D reconstruction. High-fidelity textures can also conceal some geometric defects. Since the estimated camera parameters and reconstructed geometry usually contain errors, subsequent texture mapping often suffers from undesirable visual artifacts such as blurring, ghosting, and visual seams. In particular, significant misalignment between the reconstructed model and the registered images will lead to texturing the mesh with inconsistent image regions. However, eliminating various artifacts to generate high-quality textures remains a challenge. In this paper, we address this issue by designing a texture optimization method to generate seamless and aligned textures for 3D reconstruction. The main idea is to detect misalignment regions between images and geometry and exclude them from texture mapping. To handle the texture holes caused by these excluded regions, a cross-patch texture hole-filling method is proposed, which can also synthesize plausible textures for invisible faces. Moreover, for better stitching of the textures from different views, an improved camera pose optimization is present by introducing color adjustment and boundary point sampling. Experimental results show that the proposed method can eliminate the artifacts caused by inaccurate input data robustly and produce highquality texture results compared with state-of-the-art methods.
  • Item
    GS-Octree: Octree-based 3D Gaussian Splatting for Robust Object-level 3D Reconstruction Under Strong Lighting
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Li, Jiaze; Wen, Zhengyu; Zhang, Luo; Hu, Jiangbei; Hou, Fei; Zhang, Zhebin; He, Ying; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    The 3D Gaussian Splatting technique has significantly advanced the construction of radiance fields from multi-view images, enabling real-time rendering. While point-based rasterization effectively reduces computational demands for rendering, it often struggles to accurately reconstruct the geometry of the target object, especially under strong lighting conditions. Strong lighting can cause significant color variations on the object's surface when viewed from different directions, complicating the reconstruction process. To address this challenge, we introduce an approach that combines octree-based implicit surface representations with Gaussian Splatting. Initially, it reconstructs a signed distance field (SDF) and a radiance field through volume rendering, encoding them in a low-resolution octree. This initial SDF represents the coarse geometry of the target object. Subsequently, it introduces 3D Gaussians as additional degrees of freedom, which are guided by the initial SDF. In the third stage, the optimized Gaussians enhance the accuracy of the SDF, enabling the recovery of finer geometric details compared to the initial SDF. Finally, the refined SDF is used to further optimize the 3D Gaussians via splatting, eliminating those that contribute little to the visual appearance. Experimental results show that our method, which leverages the distribution of 3D Gaussians with SDFs, reconstructs more accurate geometry, particularly in images with specular highlights caused by strong lighting. The source code can be downloaded from https://github.com/LaoChui999/GS-Octree.
  • Item
    TaNSR: Efficient 3D Reconstruction with Tetrahedral Difference and Feature Aggregation
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Lv, Zhaohan; Bao, Xingcan; Tang, Yong; Zhao, Jing; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Neural surface reconstruction methods have demonstrated their ability to recover 3D surfaces from multiple images. However, current approaches struggle to rapidly achieve high-fidelity surface reconstructions. In this work, we propose TaNSR, which inherits the speed advantages of multi-resolution hash encodings and extends its representation capabilities. To reduce training time, we propose an efficient numerical gradient computation method that significantly reduces additional memory access overhead. To further improve reconstruction quality and expedite training, we propose a feature aggregation strategy in volume rendering. Building on this, we introduce an adaptively weighted aggregation function to ensure the network can accurately reconstruct the surface of objects and recover more geometric details. Experiments on multiple datasets indicate that TaNSR significantly reduces training time while achieving better reconstruction accuracy compared to state-of-the-art nerual implicit methods.
  • Item
    CoupNeRF: Property-aware Neural Radiance Fields for Multi-Material Coupled Scenario Reconstruction
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Li, Jin; Gao, Yang; Song, Wenfeng; Li, Yacong; Li, Shuai; Hao, Aimin; Qin, Hong; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Neural Radiance Fields (NeRFs) have achieved significant recognition for their proficiency in scene reconstruction and rendering by utilizing neural networks to depict intricate volumetric environments. Despite considerable research dedicated to reconstructing physical scenes, rare works succeed in challenging scenarios involving dynamic, multi-material objects. To alleviate, we introduce CoupNeRF, an efficient neural network architecture that is aware of multiple material properties. This architecture combines physically grounded continuum mechanics with NeRF, facilitating the identification of motion systems across a wide range of physical coupling scenarios. We first reconstruct specific-material of objects within 3D physical fields to learn material parameters. Then, we develop a method to model the neighbouring particles, enhancing the learning process specifically in regions where material transitions occur. The effectiveness of CoupNeRF is demonstrated through extensive experiments, showcasing its proficiency in accurately coupling and identifying the behavior of complex physical scenes that span multiple physics domains.
  • Item
    A TransISP Based Image Enhancement Method for Visual Disbalance in Low-light Images
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Wu, Jiaqi; Guo, Jing; Jing, Rui; Zhang, Shihao; Tian, Zijian; Chen, Wei; Wang, Zehua; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Existing image enhancement algorithms often fail to effectively address issues of visual disbalance, such as brightness unevenness and color distortion, in low-light images. To overcome these challenges, we propose a TransISP-based image enhancement method specifically designed for low-light images. To mitigate color distortion, we design dual encoders based on decoupled representation learning, which enable complete decoupling of the reflection and illumination components, thereby preventing mutual interference during the image enhancement process. To address brightness unevenness, we introduce CNNformer, a hybrid model combining CNN and Transformer. This model efficiently captures local details and long-distance dependencies between pixels, contributing to the enhancement of brightness features across various local regions. Additionally, we integrate traditional image signal processing algorithms to achieve efficient color correction and denoising of the reflection component. Furthermore, we employ a generative adversarial network (GAN) as the overarching framework to facilitate unsupervised learning. The experimental results show that, compared with six SOTA image enhancement algorithms, our method obtains significant improvement in evaluation indexes (e.g., on LOL, PSNR: 15.59%, SSIM: 9.77%, VIF: 9.65%), and it can improve visual disbalance defects in low-light images captured from real-world coal mine underground scenarios.
  • Item
    Exploring Fast and Flexible Zero-Shot Low-Light Image/Video Enhancement
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Han, Xianjun; Bao, Taoli; Yang, Hongyu; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Low-light image/video enhancement is a challenging task when images or video are captured under harsh lighting conditions. Existing methods mostly formulate this task as an image-to-image conversion task via supervised or unsupervised learning. However, such conversion methods require an extremely large amount of data for training, whether paired or unpaired. In addition, these methods are restricted to specific training data, making it difficult for the trained model to enhance other types of images or video. In this paper, we explore a novel, fast and flexible, zero-shot, low-light image or video enhancement framework. Without relying on prior training or relationships among neighboring frames, we are committed to estimating the illumination of the input image/frame by a well-designed network. The proposed zero-shot, low-light image/video enhancement architecture includes illumination estimation and residual correction modules. The network architecture is very concise and does not require any paired or unpaired data during training, which allows low-light enhancement to be performed with several simple iterations. Despite its simplicity, we show that the method is fast and generalizes well to diverse lighting conditions. Many experiments on various images and videos qualitatively and quantitatively demonstrate the advantages of our method over state-of-the-art methods.
  • Item
    TempDiff: Enhancing Temporal-awareness in Latent Diffusion for Real-World Video Super-Resolution
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Jiang, Qin; Wang, Qing Lin; Chi, Li Hua; Chen, Xin Hai; Zhang, Qing Yang; Zhou, Richard; Deng, Zheng Qiu; Deng, Jin Sheng; Tang, Bin Bing; Lv, Shao He; Liu, Jie; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Latent diffusion models (LDMs) have demonstrated remarkable success in generative modeling. It is promising to leverage the potential of diffusion priors to enhance performance in image and video tasks. However, applying LDMs to video superresolution (VSR) presents significant challenges due to the high demands for realistic details and temporal consistency in generated videos, exacerbated by the inherent stochasticity in the diffusion process. In this work, we propose a novel diffusionbased framework, Temporal-awareness Latent Diffusion Model (TempDiff), specifically designed for real-world video superresolution, where degradations are diverse and complex. TempDiff harnesses the powerful generative prior of a pre-trained diffusion model and enhances temporal awareness through the following mechanisms: 1) Incorporating temporal layers into the denoising U-Net and VAE-Decoder, and fine-tuning these added modules to maintain temporal coherency; 2) Estimating optical flow guidance using a pre-trained flow net for latent optimization and propagation across video sequences, ensuring overall stability in the generated high-quality video. Extensive experiments demonstrate that TempDiff achieves compelling results, outperforming state-of-the-art methods on both synthetic and real-world VSR benchmark datasets. Code will be available at https://github.com/jiangqin567/TempDiff
  • Item
    Distinguishing Structures from Textures by Patch-based Contrasts around Pixels for High-quality and Efficient Texture filtering
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Wang, Shengchun; Xu, Panpan; Hou, Fei; Wang, Wencheng; Zhao, Chong; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    It is still challenging with existing methods to distinguish structures from texture details, and so preventing texture filtering. Considering that the textures on both sides of a structural edge always differ much from each other in appearances, we determine whether a pixel is on a structure edge by exploiting the appearance contrast between patches around the pixel, and further propose an efficient implementation method. We demonstrate that our proposed method is more effective than existing methods to distinguish structures from texture details, and our required patches for texture measurement can be smaller than the used patches in existing methods by at least half. Thus, we can improve texture filtering on both quality and efficiency, as shown by the experimental results, e.g., we can handle the textured images with a resolution of 800 × 600 pixels in real-time. (The code is available at https://github.com/hefengxiyulu/MLPC)
  • Item
    Gaussian in the Dark: Real-Time View Synthesis From Inconsistent Dark Images Using Gaussian Splatting
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Ye, Sheng; Dong, Zhen-Hui; Hu, Yubin; Wen, Yu-Hui; Liu, Yong-Jin; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    3D Gaussian Splatting has recently emerged as a powerful representation that can synthesize remarkable novel views using consistent multi-view images as input. However, we notice that images captured in dark environments where the scenes are not fully illuminated can exhibit considerable brightness variations and multi-view inconsistency, which poses great challenges to 3D Gaussian Splatting and severely degrades its performance. To tackle this problem, we propose Gaussian-DK. Observing that inconsistencies are mainly caused by camera imaging, we represent a consistent radiance field of the physical world using a set of anisotropic 3D Gaussians, and design a camera response module to compensate for multi-view inconsistencies. We also introduce a step-based gradient scaling strategy to constrain Gaussians near the camera, which turn out to be floaters, from splitting and cloning. Experiments on our proposed benchmark dataset demonstrate that Gaussian-DK produces high-quality renderings without ghosting and floater artifacts and significantly outperforms existing methods. Furthermore, we can also synthesize light-up images by controlling exposure levels that clearly show details in shadow areas.
  • Item
    Cinematic Gaussians: Real-Time HDR Radiance Fields with Depth of Field
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Wang, Chao; Wolski, Krzysztof; Kerbl, Bernhard; Serrano, Ana; Bemama, Mojtaba; Seidel, Hans-Peter; Myszkowski, Karol; Leimkühler, Thomas; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Radiance field methods represent the state of the art in reconstructing complex scenes from multi-view photos. However, these reconstructions often suffer from one or both of the following limitations: First, they typically represent scenes in low dynamic range (LDR), which restricts their use to evenly lit environments and hinders immersive viewing experiences. Secondly, their reliance on a pinhole camera model, assuming all scene elements are in focus in the input images, presents practical challenges and complicates refocusing during novel-view synthesis. Addressing these limitations, we present a lightweight method based on 3D Gaussian Splatting that utilizes multi-view LDR images of a scene with varying exposure times, apertures, and focus distances as input to reconstruct a high-dynamic-range (HDR) radiance field. By incorporating analytical convolutions of Gaussians based on a thin-lens camera model as well as a tonemapping module, our reconstructions enable the rendering of HDR content with flexible refocusing capabilities. We demonstrate that our combined treatment of HDR and depth of field facilitates real-time cinematic rendering, outperforming the state of the art.
  • Item
    GSEditPro: 3D Gaussian Splatting Editing with Attention-based Progressive Localization
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Sun, Yanhao; Tian, Runze; Han, Xiao; Liu, Xinyao; Zhang, Yan; Xu, Kai; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    With the emergence of large-scale Text-to-Image(T2I) models and implicit 3D representations like Neural Radiance Fields (NeRF), many text-driven generative editing methods based on NeRF have appeared. However, the implicit encoding of geometric and textural information poses challenges in accurately locating and controlling objects during editing. Recently, significant advancements have been made in the editing methods of 3D Gaussian Splatting, a real-time rendering technology that relies on explicit representation. However, these methods still suffer from issues including inaccurate localization and limited manipulation over editing. To tackle these challenges, we propose GSEditPro, a novel 3D scene editing framework which allows users to perform various creative and precise editing using text prompts only. Leveraging the explicit nature of the 3D Gaussian distribution, we introduce an attention-based progressive localization module to add semantic labels to each Gaussian during rendering. This enables precise localization on editing areas by classifying Gaussians based on their relevance to the editing prompts derived from cross-attention layers of the T2I model. Furthermore, we present an innovative editing optimization method based on 3D Gaussian Splatting, obtaining stable and refined editing results through the guidance of Score Distillation Sampling and pseudo ground truth. We prove the efficacy of our method through extensive experiments.
  • Item
    GETr: A Geometric Equivariant Transformer for Point Cloud Registration
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Yu, Chang; Zhang, Sanguo; Shen, Li-Yong; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    As a fundamental problem in computer vision, 3D point cloud registration (PCR) aims to seek the optimal transformation to align point cloud pairs. Meanwhile, the equivariance lies at the core of matching point clouds at arbitrary pose. In this paper, we propose GETr, a geometric equivariant transformer for PCR. By learning the point-wise orientations, we decouple the coordinate to the pose of the point clouds, which is the key to achieve equivariance in our framework. Then we utilize attention mechanism to learn the geometric features for superpoints matching, the proposed novel self-attention mechanism encodes the geometric information of point clouds. Finally, the coarse-to-fine manner is used to obtain high-quality correspondence for registration. Extensive experiments on both indoor and outdoor benchmarks demonstrate that our method outperforms various existing state-of-the-art methods.
  • Item
    PCLC-Net: Point Cloud Completion in Arbitrary Poses using Learnable Canonical Space
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Xu, Hanmo; Shuai, Qingyao; Chen, Xuejin; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Recovering the complete structure from partial point clouds in arbitrary poses is challenging. Recently, many efforts have been made to address this problem by developing SO(3)-equivariant completion networks or aligning the partial point clouds with a predefined canonical space before completion. However, these approaches are limited to random rotations only or demand costly pose annotation for model training. In this paper, we present a novel Network for Point cloud Completion with Learnable Canonical space (PCLC-Net) to reduce the need for pose annotations and extract SE(3)-invariant geometry features to improve the completion quality in arbitrary poses. Without pose annotations, our PCLC-Net utilizes self-supervised pose estimation to align the input partial point clouds to a canonical space that is learnable for an object category and subsequently performs shape completion in the learned canonical space. Our PCLC-Net can complete partial point clouds with arbitrary SE(3) poses without requiring pose annotations for supervision. Our PCLC-Net achieves state-of-the-art results on shape completion with arbitrary SE(3) poses on both synthetic and real scanned data. To the best of our knowledge, our method is the first to achieve shape completion in arbitrary poses without pose annotations during network training.
  • Item
    DSGI-Net: Density-based Selective Grouping Point Cloud Learning Network for Indoor Scene
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Wen, Xin; Duan, Yao; Xu, Kai; Zhu, Chenyang; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Indoor scene point clouds exhibit diverse distributions and varying levels of sparsity, characterized by more intricate geometry and occlusion compared to outdoor scenes or individual objects. Despite recent advancements in 3D point cloud analysis introducing various network architectures, there remains a lack of frameworks tailored to the unique attributes of indoor scenarios. To address this, we propose DSGI-Net, a novel indoor scene point cloud learning network that can be integrated into existing models. The key innovation of this work is selectively grouping more informative neighbor points in sparse regions and promoting semantic consistency of the local area where different instances are in proximity but belong to distinct categories. Furthermore, our method encodes both semantic and spatial relationships between points in local regions to reduce the loss of local geometric details. Extensive experiments on the ScanNetv2, SUN RGB-D, and S3DIS indoor scene benchmarks demonstrate that our method is straightforward yet effective.
  • Item
    Point-AGM : Attention Guided Masked Auto-Encoder for Joint Self-supervised Learning on Point Clouds
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Liu, Jie; Yang, Mengna; Tian, Yu; Li, Yancui; Song, Da; Li, Kang; Cao, Xin; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Masked point modeling (MPM) has gained considerable attention in self-supervised learning for 3D point clouds. While existing self-supervised methods have progressed in learning from point clouds, we aim to address their limitation of capturing high-level semantics through our novel attention-guided masking framework, Point-AGM. Our approach introduces an attention-guided masking mechanism that selectively masks low-attended regions, enabling the model to concentrate on reconstructing more critical areas and addressing the limitations of random and block masking strategies. Furthermore, we exploit the inherent advantages of the teacher-student network to enable cross-view contrastive learning on augmented dual-view point clouds, enforcing consistency between complete and partially masked views of the same 3D shape in the feature space. This unified framework leverages the complementary strengths of masked point modeling, attention-guided masking, and contrastive learning for robust representation learning. Extensive experiments have shown the effectiveness of our approach and its well-transferable performance across various downstream tasks. Specifically, our model achieves an accuracy of 94.12% on ModelNet40 and 87.16% on the PB-T50-RS setting of ScanObjectNN, outperforming other self-supervised learning methods.
  • Item
    Frequency-Aware Facial Image Shadow Removal through Skin Color and Texture Learning
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Zhang, Ling; Xie, Wenyang; Xiao, Chunxia; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Existing facial image shadow removal methods predominantly rely on pre-extracted facial features. However, these methods often fail to capitalize on the full potential of these features, resorting to simplified utilization. Furthermore, they tend to overlook the importance of low-frequency information during the extraction of prior features, which can be easily compromised by noises. In our work, we propose a frequency-aware shadow removal network (FSRNet) for facial image shadow removal, which utilizes the skin color and texture information in the face to help recover illumination in shadow regions. Our FSRNet uses a frequencydomain image decomposition network to extract the low-frequency skin color map and high-frequency texture map from the face images, and applies a color-texture guided shadow removal network to produce final shadow removal result. Concretely, the designed fourier sparse attention block (FSABlock) can transform images from the spatial domain to the frequency domain and help the network focus on the key information. We also introduce a skin color fusion module (CFModule) and a texture fusion module (TFModule) to enhance the understanding and utilization of color and texture features, promoting high-quality result without color distortion and detail blurring. Extensive experiments demonstrate the superiority of the proposed method. The code is available at https://github.com/laoxie521/FSRNet.
  • Item
    Density-Aware Diffusion Model for Efficient Image Dehazing
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Zhang, Ling; Bai, Wenxu; Xiao, Chunxia; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Existing image dehazing methods have made remarkable progress. However, they generally perform poorly on images with dense haze, and often suffer from unsatisfactory results with detail degradation or color distortion. In this paper, we propose a density-aware diffusion model (DADM) for image dehazing. Guided by the haze density, our DADM can handle images with dense haze and complex environments. Specifically, we introduce a density-aware dehazing network (DADNet) in the reverse diffusion process, which can help DADM gradually recover a clear haze-free image from a haze image. To improve the performance of the network, we design a cross-feature density extraction module (CDEModule) to extract the haze density for the image and a density-guided feature fusion block (DFFBlock) to learn the effective contextual features. Furthermore, we introduce an indirect sampling strategy in the test sampling process, which not only suppresses the accumulation of errors but also ensures the stability of the results. Extensive experiments on popular benchmarks validate the superior performance of the proposed method. The code is released in https://github.com/benchacha/DADM.
  • Item
    MISNeR: Medical Implicit Shape Neural Representation for Image Volume Visualisation
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Jin, Ge; Jung, Younhyun; Bi, Lei; Kim, Jinman; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Three-dimensional visualisation of mesh reconstruction of the medical images is commonly used for various clinical applications including pre / post-surgical planning. Such meshes are conventionally generated by extracting the surface from volumetric segmentation masks. Therefore, they have inherent limitations of staircase artefacts due to their anisotropic voxel dimensions. The time-consuming process for manual refinement to remove artefacts and/or the isolated regions further adds to these limitations. Methods for directly generating meshes from volumetric data by template deformation are often limited to simple topological structures, and methods that use implicit functions for continuous surfaces, do not achieve the level of mesh reconstruction accuracy when compared to segmentation-based methods. In this study, we address these limitations by combining the implicit function representation with a multi-level deep learning architecture. We introduce a novel multi-level local feature sampling component which leverages the spatial features for the implicit function regression to enhance the segmentation result. We further introduce a shape boundary estimator that accelerates the explicit mesh reconstruction by minimising the number of the signed distance queries during model inference. The result is a multi-level deep learning network that directly regresses the implicit function from medical image volumes to a continuous surface model, which can be used for mesh reconstruction from arbitrary high volume resolution to minimise staircase artefacts. We evaluated our method using pelvic computed tomography (CT) dataset from two public sources with varying z-axis resolutions. We show that our method minimised the staircase artefacts while achieving comparable results in surface accuracy when compared to the state-of-the-art segmentation algorithms. Furthermore, our method was 9 times faster in volume reconstruction than comparable implicit shape representation networks.
  • Item
    Surface Cutting and Flattening to Target Shapes
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Li, Yuanhao; Wu, Wenzheng; Liu, Ligang; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    We introduce a novel framework for surface cutting and flattening, aiming to align the boundary of planar parameterization with a target shape. Diverging from traditional methods focused on minimizing distortion, we intend to also achieve shape similarity between the parameterized mesh and a specific planar target, which is important in some applications of art design and texture mapping. However, with existing methods commonly limited to ellipsoidal surfaces, it still remains a challenge to solve this problem on general surfaces. Our framework models the general case as a joint optimization of cuts and parameterization, guided by a novel metric assessing shape similarity. To circumvent the common issue of local minima, we introduce an extra global seam updating strategy which is guided by the target shape. Experimental results show that our framework not only aligns with previous approaches on ellipsoidal surfaces but also achieves satisfactory results on more complex ones.
  • Item
    Variable Offsets and Processing of Implicit Forms Toward the Adaptive Synthesis and Analysis of Heterogeneous Conforming Microstructure
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Hong, Q. Youn; Antolin, Pablo; Elber, Gershon; Kim, Myung-Soo; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    The synthesis of porous, lattice, or microstructure geometries has captured the attention of many researchers in recent years. Implicit forms, such as triply periodic minimal surfaces (TPMS) has captured a significant attention, recently, as tiles in lattices, partially because implicit forms have the potential for synthesizing with ease more complex topologies of tiles, compared to parametric forms. In this work, we show how variable offsets of implicit forms could be used in lattice design as well as lattice analysis, while graded wall and edge thicknesses could be fully controlled in the lattice and even vary within a single tile. As a result, (geometrically) heterogeneous lattices could be created and adapted to follow analysis results while maintaining continuity between adjacent tiles. We demonstrate this ability on several 3D models, including TPMS.
  • Item
    Ray Tracing Animated Displaced Micro-Meshes
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Gruen, Holger; Benthin, Carsten; Kensler, Andrew; Barczak, Joshua; McAllister, David; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    We present a new method that allows efficient ray tracing of virtually artefact-free animated displaced micro-meshes (DMMs) [MMT23] and preserves their low memory footprint and low BVH build and update cost. DMMs allow for compact representation of micro-triangle geometry through hierarchical encoding of displacements. Displacements are computed with respect to a coarse base mesh and are used to displace new vertices introduced during 1 : 4 subdivision of the base mesh. Applying non-rigid transformation to the base mesh can result in silhouette and normal artefacts (see Figure 1) during animation. We propose an approach which prevents these artefacts by interpolating transformation matrices before applying them to the DMM representation. Our interpolation-based algorithm does not change DMM data structures and it allows for efficient bounding of animated micro-triangle geometry which is essential for fast tessellation-free ray tracing of animated DMMs.
  • Item
    Faster Ray Tracing through Hierarchy Cut Code
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Xiang, WeiLai; Liu, FengQi; Tan, Zaonan; Li, Dan; Xu, PengZhan; Liu, MeiZhi; Kou, QiLong; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    We propose a novel ray reordering technique designed to accelerate the ray tracing process by encoding and sorting rays prior to traversal. Our method, called ''hierarchy cut code'', involves encoding rays based on the cuts of the hierarchical acceleration structure, rather than relying solely on spatial coordinates. This approach allows for a more effective adaptation to the acceleration structure, resulting in a more reliable and efficient encoding outcome. Furthermore, our research identifies ''bounding drift'' as a major obstacle in achieving better acceleration effects using longer sorting keys in existing reordering methods. Fortunately, our hierarchy cut code successfully overcomes this issue, providing improved performance in ray tracing. Experimental results demonstrate the effectiveness of our approach, showing up to a 1.81 times faster secondary ray tracing compared to existing methods. These promising results highlight the potential for further enhancement in the acceleration effect of reordering techniques, warranting further exploration and research in this exciting field.
  • Item
    CrystalNet: Texture-Aware Neural Refraction Baking for Global Illumination
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Zhang, Ziyang; Simo-Serra, Edgar; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Neural rendering bakes global illumination and other computationally costly effects into the weights of a neural network, allowing to efficiently synthesize photorealistic images without relying on path tracing. In neural rendering approaches, G-buffers obtained from rasterization through direct rendering provide information regarding the scene such as position, normal, and textures to the neural network, achieving accurate and stable rendering quality in real-time. However, due to the use of G-buffers, existing methods struggle to accurately render transparency and refraction effects, as G-buffers do not capture any ray information from multiple light ray bounces. This limitation results in blurriness, distortions, and loss of detail in rendered images that contain transparency and refraction, and is particularly notable in scenes with refracted objects that have high-frequency textures. In this work, we propose a neural network architecture to encode critical rendering information, including texture coordinates from refracted rays, and enable reconstruction of high-frequency textures in areas with refraction. Our approach is able to achieve accurate refraction rendering in challenging scenes with a diversity of overlapping transparent objects. Experimental results demonstrate that our method can interactively render high quality refraction effects with global illumination, unlike existing neural rendering approaches. Our code can be found at https://github.com/ziyangz5/CrystalNet
  • Item
    Spatially and Temporally Optimized Audio-Driven Talking Face Generation
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Dong, Biao; Ma, Bo-Yao; Zhang, Lei; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Audio-driven talking face generation is essentially a cross-modal mapping from audio to video frames. The main challenge lies in the intricate one-to-many mapping, which affects lip sync accuracy. And the loss of facial details during image reconstruction often results in visual artifacts in the generated video. To overcome these challenges, this paper proposes to enhance the quality of generated talking faces with a new spatio-temporal consistency. Specifically, the temporal consistency is achieved through consecutive frames of the each phoneme, which form temporal modules that exhibit similar lip appearance changes. This allows for adaptive adjustment in the lip movement for accurate sync. The spatial consistency pertains to the uniform distribution of textures within local regions, which form spatial modules and regulate the texture distribution in the generator. This yields fine details in the reconstructed facial images. Extensive experiments show that our method can generate more natural talking faces than previous state-of-the-art methods in both accurate lip sync and realistic facial details.
  • Item
    Disentangled Lifespan Synthesis via Transformer-Based Nonlinear Regression
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Li, Mingyuan; Guo, Yingchun; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Lifespan face age transformation aims to generate facial images that accurately depict an individual's appearance at different age stages. This task is highly challenging due to the need for reasonable changes in facial features while preserving identity characteristics. Existing methods tend to synthesize unsatisfactory results, such as entangled facial attributes and low identity preservation, especially when dealing with large age gaps. Furthermore, over-manipulating the style vector may deviate it from the latent space and damage image quality. To address these issues, this paper introduces a novel nonlinear regression model- Disentangled Lifespan face Aging (DL-Aging) to achieve high-quality age transformation images. Specifically, we propose an age modulation encoder to extract age-related multi-scale facial features as key and value, and use the reconstructed style vector of the image as the query. The multi-head cross-attention in the W+ space is utilized to update the query for aging image reconstruction iteratively. This nonlinear transformation enables the model to learn a more disentangled mode of transformation, which is crucial for alleviating facial attribute entanglement. Additionally, we introduce a W+ space age regularization term to prevent excessive manipulation of the style vector and ensure it remains within theW+ space during transformation, thereby improving generation quality and aging accuracy. Extensive qualitative and quantitative experiments demonstrate that the proposed DL-Aging outperforms state-of-the-art methods regarding aging accuracy, image quality, attribute disentanglement, and identity preservation, especially for large age gaps.
  • Item
    Multiscale Spectral Manifold Wavelet Regularizer for Unsupervised Deep Functional Maps
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Wang, Haibo; Meng, Jing; Li, Qinsong; Hu, Ling; Guo, Yueyu; Liu, Xinru; Yang, Xiaoxia; Liu, Shengjun; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    In deep functional maps, the regularizer computing the functional map is especially crucial for ensuring the global consistency of the computed pointwise map. As the regularizers integrated into deep learning should be differentiable, it is not trivial to incorporate informative axiomatic structural constraints into the deep functional map, such as the orientation-preserving term. Although commonly used regularizers include the Laplacian-commutativity term and the resolvent Laplacian commutativity term, these are limited to single-scale analysis for capturing geometric information. To this end, we propose a novel and theoretically well-justified regularizer commuting the functional map with the multiscale spectral manifold wavelet operator. This regularizer enhances the isometric constraints of the functional map and is conducive to providing it with better structural properties with multiscale analysis. Furthermore, we design an unsupervised deep functional map with the regularizer in a fully differentiable way. The quantitative and qualitative comparisons with several existing techniques on the (near-)isometric and non-isometric datasets show our method's superior accuracy and generalization capabilities. Additionally, we illustrate that our regularizer can be easily inserted into other functional map methods and improve their accuracy.
  • Item
    FSH3D: 3D Representation via Fibonacci Spherical Harmonics
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Li, Zikuan; Huang, Anyi; Jia, Wenru; Wu, Qiaoyun; Wei, Mingqiang; Wang, Jun; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Spherical harmonics are a favorable technique for 3D representation, employing a frequency-based approach through the spherical harmonic transform (SHT). Typically, SHT is performed using equiangular sampling grids. However, these grids are non-uniform on spherical surfaces and exhibit local anisotropy, a common limitation in existing spherical harmonic decomposition methods. This paper proposes a 3D representation method using Fibonacci Spherical Harmonics (FSH3D). We introduce a spherical Fibonacci grid (SFG), which is more uniform than equiangular grids for SHT in the frequency domain. Our method employs analytical weights for SHT on SFG, effectively assigning sampling errors to spherical harmonic degrees higher than the recovered band-limited function. This provides a novel solution for spherical harmonic transformation on non-equiangular grids. The key advantages of our FSH3D method include: 1) With the same number of sampling points, SFG captures more features without bias compared to equiangular grids; 2) The root mean square error of 32-degree spherical harmonic coefficients is reduced by approximately 34.6% for SFG compared to equiangular grids; and 3) FSH3D offers more stable frequency domain representations, especially for rotating functions. FSH3D enhances the stability of frequency domain representations under rotational transformations. Its application in 3D shape reconstruction and 3D shape classification results in more accurate and robust representations. Our code is publicly available at https://github.com/Miraclelzk/Fibonacci-Spherical-Harmonics.
  • Item
    Curved Image Triangulation Based on Differentiable Rendering
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Wang, Wanyi; Chen, Zhonggui; Fang, Lincong; Cao, Juan; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Image triangulation methods, which decompose an image into a series of triangles, are fundamental in artistic creation and image processing. This paper introduces a novel framework that integrates cubic Bézier curves into image triangulation, enabling the precise reconstruction of curved image features. Our developed framework constructs a well-structured curved triangle mesh, effectively preventing overlaps between curves. A refined energy function, grounded in differentiable rendering, establishes a direct link between mesh geometry and rendering effects and is instrumental in guiding the curved mesh generation. Additionally, we derive an explicit gradient formula with respect to mesh parameters, facilitating the adaptive and efficient optimization of these parameters to fully leverage the capabilities of cubic Bézier curves. Through experimental and comparative analyses with state-of-the-art methods, our approach demonstrates a significant enhancement in both numerical accuracy and visual quality.
  • Item
    Anisotropic Specular Image-Based Lighting Based on BRDF Major Axis Sampling
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Cocco, Giovanni; Zanni, Cédric; Chermain, Xavier; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Anisotropic specular appearances are ubiquitous in the environment: brushed stainless steel pans, kettles, elevator walls, fur, or scratched plastics. Real-time rendering of these materials with image-based lighting is challenging due to the complex shape of the bidirectional reflectance distribution function (BRDF). We propose an anisotropic specular image-based lighting method that can serve as a drop-in replacement for the standard bent normal technique [Rev11]. Our method yields more realistic results with a 50% increase in computation time of the previous technique, using the same high dynamic range (HDR) preintegrated environment image. We use several environment samples positioned along the major axis of the specular microfacet BRDF. We derive an analytic formula to determine the two closest and two farthest points from the reflected direction on an approximation of the BRDF confidence region boundary. The two farthest points define the BRDF major axis, while the two closest points are used to approximate the BRDF width. The environment level of detail is derived from the BRDF width and the distance between the samples. We extensively compare our method with the bent normal technique and the ground truth using the GGX specular BRDF.
  • Item
    NeuPreSS: Compact Neural Precomputed Subsurface Scattering for Distant Lighting of Heterogeneous Translucent Objects
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Tg, Thomson; Frisvad, Jeppe Revall; Ramamoorthi, Ravi; Jensen, Henrik W.; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Monte Carlo rendering of translucent objects with heterogeneous scattering properties is often expensive both in terms of memory and computation. If the scattering properties are described by a 3D texture, memory consumption is high. If we do path tracing and use a high dynamic range lighting environment, the computational cost of the rendering can easily become significant. We propose a compact and efficient neural method for representing and rendering the appearance of heterogeneous translucent objects. Instead of assuming only surface variation of optical properties, our method represents the appearance of a full object taking its geometry and volumetric heterogeneities into account. This is similar to a neural radiance field, but our representation works for an arbitrary distant lighting environment. In a sense, we present a version of neural precomputed radiance transfer that captures relighting of heterogeneous translucent objects. We use a multi-layer perceptron (MLP) with skip connections to represent the appearance of an object as a function of spatial position, direction of observation, and direction of incidence. The latter is considered a directional light incident across the entire non-self-shadowed part of the object. We demonstrate the ability of our method to compactly store highly complex materials while having high accuracy when comparing to reference images of the represented object in unseen lighting environments. As compared with path tracing of a heterogeneous light scattering volume behind a refractive interface, our method more easily enables importance sampling of the directions of incidence and can be integrated into existing rendering frameworks while achieving interactive frame rates.
  • Item
    A Surface-based Appearance Model for Pennaceous Feathers
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Padrón-Griffe, Juan Raúl; Lanza, Dario; Jarabo, Adrian; Muñoz, Adolfo; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    The appearance of a real-world feather results from the complex interaction of light with its multi-scale biological structure, including the central shaft, branching barbs, and interlocking barbules on those barbs. In this work, we propose a practical surface-based appearance model for feathers. We represent the far-field appearance of feathers using a BSDF that implicitly represents the light scattering from the main biological structures of a feather, such as the shaft, barb and barbules. Our model accounts for the particular characteristics of feather barbs such as the non-cylindrical cross-sections and the scattering media via a numerically-based BCSDF. To model the relative visibility between barbs and barbules, we derive a masking term for the differential projected areas of the different components of the feather's microgeometry, which allows us to analytically compute the masking between barbs and barbules. As opposed to previous works, our model uses a lightweight representation of the geometry based on a 2D texture, and does not require explicitly representing the barbs as curves. We show the flexibility and potential of our appearance model approach to represent the most important visual features of several pennaceous feathers.
  • Item
    GLTScene: Global-to-Local Transformers for Indoor Scene Synthesis with General Room Boundaries
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Li, Yijie; Xu, Pengfei; Ren, Junquan; Shao, Zefan; Huang, Hui; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    We present GLTScene, a novel data-driven method for high-quality furniture layout synthesis with general room boundaries as conditions. This task is challenging since the existing indoor scene datasets do not cover the variety of general room boundaries. We incorporate the interior design principles with learning techniques and adopt a global-to-local strategy for this task. Globally, we learn the placement of furniture objects from the datasets without considering their alignment. Locally, we learn the alignment of furniture objects relative to their nearest walls, according to the alignment principle in interior design. The global placement and local alignment of furniture objects are achieved by two transformers respectively. We compare our method with several baselines in the task of furniture layout synthesis with general room boundaries as conditions. Our method outperforms these baselines both quantitatively and qualitatively. We also demonstrate that our method can achieve other conditional layout synthesis tasks, including object-level conditional generation and attribute-level conditional generation. The code is publicly available at https://github.com/WWalter-Lee/GLTScene.
  • Item
    Evolutive 3D Urban Data Representation through Timeline Design Space
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Gautier, Corentin Le Bihan; Delanoy, Johanna; Gesquière, Gilles; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Cities are constantly changing to adapt to new societal and environmental challenges. Understanding their evolution is thus essential to make informed decisions about their future. To capture these changes, cities are increasingly offering digital 3D snapshots of their territory over time. However, existing tools to visualise these data typically represent the city at a specific point in time, limiting a comprehensive analysis of its evolution. In this paper, we propose a new method for simultaneously visualising different versions of the city in a 3D space. We integrate the different versions of the city along a new way of 3D timeline that can take different shapes depending on the needs of the user and the dataset being visualised. We propose four different shapes of timelines and three ways to place the versions along it. Our method places the versions such that there is no visual overlap for the user by varying the parameters of the timelines, and offer options to ease the understanding of the scene by changing the orientation or scale of the versions. We evaluate our method on different datasets to demonstrate the advantages and limitations of the different shapes of timeline and provide recommendations so as to which shape to chose.
  • Item
    LightUrban: Similarity Based Fine-grained Instancing for Lightweighting Complex Urban Point Clouds
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Lu, Zi Ang; Xiong, Wei Dan; Ren, Peng; Jia, Jin Yuan; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Large-scale urban point clouds play a vital role in various applications, while rendering and transmitting such data remains challenging due to its large volume, complicated structures, and significant redundancy. In this paper, we present LightUrban, the first point cloud instancing framework for efficient rendering and transmission of fine-grained complex urban scenes.We first introduce a segmentation method to organize the point clouds into individual buildings and vegetation instances from coarse to fine. Next, we propose an unsupervised similarity detection approach to accurately group instances with similar shapes. Furthermore, a fast pose and size estimation component is applied to calculate the transformations between the representative instance and the corresponding similar instances in each group. By replacing individual instances with their group's representative instances, the data volume and redundancy can be dramatically reduced. Experimental results on large-scale urban scenes demonstrate the effectiveness of our algorithm. To sum up, our method not only structures the urban point clouds but also significantly reduces data volume and redundancy, filling the gap in lightweighting urban landscapes through instancing.
  • Item
    Disk B-spline on S2: A Skeleton-based Region Representation on 2-Sphere
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Zheng, Chunhao; Zhao, Yuming; Wu, Zhongke; Wang, Xingce; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Due to the widespread applications of 2-dimensional spherical designs, there has been an increasing requirement of modeling on the S2 manifold in recent years. Due to the non-Euclidean nature of the sphere, it has some challenges to find a method to represent 2D regions on S2 manifold. In this paper, a skeleton-based representation method of regions on S2, disk B-spline(DBSC) on S2 is proposed. Firstly, we give the definition and basic algorithms of DBSC on S2. Then we provide the calculation method of DBSC on S2, which includes calculating the boundary points, internal points and their corresponding derivatives. Based on that, we give some modeling methods of DBSC on S2, including approximation, deformation. In the end, some stunning application examples of DBSC on S2 are shown. This work lays a theoretical foundation for further applications of DBSC on S2.
  • Item
    A Hybrid Parametrization Method for B-Spline Curve Interpolation via Supervised Learning
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Song, Tianyu; Shen, Tong; Ge, Linlin; Feng, Jieqing; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    B-spline curve interpolation is a fundamental algorithm in computer-aided geometric design. Determining suitable parameters based on data points distribution has always been an important issue for high-quality interpolation curves generation. Various parameterization methods have been proposed. However, there is no universally satisfactory method that is applicable to data points with diverse distributions. In this work, a hybrid parametrization method is proposed to overcome the problem. For a given set of data points, a classifier via supervised learning identifies an optimal local parameterization method based on the local geometric distribution of four adjacent data points, and the optimal local parameters are computed using the selected optimal local parameterization method for the four adjacent data points. Then a merging method is employed to calculate global parameters which align closely with the local parameters. Experiments demonstrate that the proposed hybrid parameterization method well adapts the different distributions of data points statistically. The proposed method has a flexible and scalable framework, which can includes current and potential new parameterization methods as its components.
  • Item
    Strictly Conservative Neural Implicits
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Ludwig, Ingmar; Campen, Marcel; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    We describe a method to convert 3D shapes into neural implicit form such that the shape is approximated in a guaranteed conservative manner. This means the input shape is strictly contained inside the neural implicit or, alternatively, vice versa. Such conservative approximations are of interest in a variety of applications, including collision detection, occlusion culling, or intersection testing. Our approach is the first to guarantee conservativeness in this context of neural implicits. We support input given as mesh, voxel set, or implicit function. Adaptive affine arithmetic is employed in the neural network fitting process, enabling the reasoning over infinite sets of points despite using a finite set of training data. Combined with an interior point style optimization approach this yields the desired guarantee.
  • Item
    Symmetric Piecewise Developable Approximations
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) He, Ying; Fang, Qing; Zhang, Zheng; Dai, Tielin; Wu, Kang; Liu, Ligang; Fu, Xiao-Ming; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    We propose a novel method for generating symmetric piecewise developable approximations for shapes in approximately global reflectional or rotational symmetry. Given a shape and its symmetry constraint, the algorithm contains two crucial steps: (i) a symmetric deformation to achieve a nearly developable model and (ii) a symmetric segmentation aided by the deformed shape. The key to the deformation step is the use of the symmetric implicit neural representations of the shape and the deformation field. A new mesh extraction from the implicit function is introduced to construct a strictly symmetric mesh for the subsequent segmentation. The symmetry constraint is carefully integrated into the partition to achieve the symmetric piecewise developable approximation. We demonstrate the effectiveness of our algorithm over various meshes.
  • Item
    FastFlow: GPU Acceleration of Flow and Depression Routing for Landscape Simulation
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Jain, Aryamaan; Kerbl, Bernhard; Gain, James; Finley, Brandon; Cordonnier, Guillaume; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Terrain analysis plays an important role in computer graphics, hydrology and geomorphology. In particular, analyzing the path of material flow over a terrain with consideration of local depressions is a precursor to many further tasks in erosion, river formation, and plant ecosystem simulation. For example, fluvial erosion simulation used in terrain modeling computes water discharge to repeatedly locate erosion channels for soil removal and transport. Despite its significance, traditional methods face performance constraints, limiting their broader applicability. In this paper, we propose a novel GPU flow routing algorithm that computes the water discharge in O(logn) iterations for a terrain with n vertices (assuming n processors). We also provide a depression routing algorithm to route the water out of local minima formed by depressions in the terrain, which converges in O(log2 n) iterations. Our implementation of these algorithms leads to a 5× speedup for flow routing and 34× to 52× speedup for depression routing compared to previous work on a 10242 terrain, enabling interactive control of terrain simulation.
  • Item
    P-Hologen: An End-to-End Generative Framework for Phase-Only Holograms
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Park, JooHyun; Jeon, YuJin; Kim, HuiYong; Baek, SeungHwan; Kang, HyeongYeop; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Holography stands at the forefront of visual technology, offering immersive, three-dimensional visualizations through the manipulation of light wave amplitude and phase. Although generative models have been extensively explored in the image domain, their application to holograms remains relatively underexplored due to the inherent complexity of phase learning. Exploiting generative models for holograms offers exciting opportunities for advancing innovation and creativity, such as semantic-aware hologram generation and editing. Currently, the most viable approach for utilizing generative models in the hologram domain involves integrating an image-based generative model with an image-to-hologram conversion model, which comes at the cost of increased computational complexity and inefficiency. To tackle this problem, we introduce P-Hologen, the first endto- end generative framework designed for phase-only holograms (POHs). P-Hologen employs vector quantized variational autoencoders to capture the complex distributions of POHs. It also integrates the angular spectrum method into the training process, constructing latent spaces for complex phase data using strategies from the image processing domain. Extensive experiments demonstrate that P-Hologen achieves superior quality and computational efficiency compared to the existing methods. Furthermore, our model generates high-quality unseen, diverse holographic content from its learned latent space without requiring pre-existing images. Our work paves the way for new applications and methodologies in holographic content creation, opening a new era in the exploration of generative holographic content. The code for our paper is publicly available on https://github.com/james0223/P-Hologen.
  • Item
    Controllable Anime Image Editing Based on the Probability of Attribute Tags
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Song, Zhenghao; Mo, Haoran; Gao, Chengying; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Editing anime images via probabilities of attribute tags allows controlling the degree of the manipulation in an intuitive and convenient manner. Existing methods fall short in the progressive modification and preservation of unintended regions in the input image. We propose a controllable anime image editing framework based on adjusting the tag probabilities, in which a probability encoding network (PEN) is developed to encode the probabilities into features that capture continuous characteristic of the probabilities. Thus, the encoded features are able to direct the generative process of a pre-trained diffusion model and facilitate the linear manipulation.We also introduce a local editing module that automatically identifies the intended regions and constrains the edits to be applied to those regions only, which preserves the others unchanged. Comprehensive comparisons with existing methods indicate the effectiveness of our framework in both one-shot and linear editing modes. Results in additional applications further demonstrate the generalization ability of our approach.
  • Item
    DiffPop: Plausibility-Guided Object Placement Diffusion for Image Composition
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Liu, Jiacheng; Zhou, Hang; Wei, Shida; Ma, Rui; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    In this paper, we address the problem of plausible object placement for the challenging task of realistic image composition. We propose DiffPop, the first framework that utilizes plausibility-guided denoising diffusion probabilistic model to learn the scale and spatial relations among multiple objects and the corresponding scene image. First, we train an unguided diffusion model to directly learn the object placement parameters in a self-supervised manner. Then, we develop a human-in-the-loop pipeline which exploits human labeling on the diffusion-generated composite images to provide the weak supervision for training a structural plausibility classifier. The classifier is further used to guide the diffusion sampling process towards generating the plausible object placement. Experimental results verify the superiority of our method for producing plausible and diverse composite images on the new Cityscapes-OP dataset and the public OPA dataset, as well as demonstrate its potential in applications such as data augmentation and multi-object placement tasks. Our dataset and code will be released.
  • Item
    CustomSketching: Sketch Concept Extraction for Sketch-based Image Synthesis and Editing
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Xiao, Chufeng; Fu, Hongbo; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Personalization techniques for large text-to-image (T2I) models allow users to incorporate new concepts from reference images. However, existing methods primarily rely on textual descriptions, leading to limited control over customized images and failing to support fine-grained and local editing (e.g., shape, pose, and details). In this paper, we identify sketches as an intuitive and versatile representation that can facilitate such control, e.g., contour lines capturing shape information and flow lines representing texture. This motivates us to explore a novel task of sketch concept extraction: given one or more sketch-image pairs, we aim to extract a special sketch concept that bridges the correspondence between the images and sketches, thus enabling sketch-based image synthesis and editing at a fine-grained level. To accomplish this, we introduce CustomSketching, a two-stage framework for extracting novel sketch concepts via few-shot learning. Considering that an object can often be depicted by a contour for general shapes and additional strokes for internal details, we introduce a dual-sketch representation to reduce the inherent ambiguity in sketch depiction. We employ a shape loss and a regularization loss to balance fidelity and editability during optimization. Through extensive experiments, a user study, and several applications, we show our method is effective and superior to the adapted baselines.
  • Item
    Digital Garment Alteration
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Eggler, Anna Maria; Falque, Raphael; Liu, Mark; Vidal-Calleja, Teresa; Sorkine-Hornung, Olga; Pietroni, Nico; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Garment alteration is a practical technique to adapt an existing garment to fit a target body shape. Typically executed by skilled tailors, this process involves a series of strategic fabric operations-removing or adding material-to achieve the desired fit on a target body. We propose an innovative approach to automate this process by computing a set of practically feasible modifications that adapt an existing garment to fit a different body shape. We first assess the garment's fit on a reference body; then, we replicate this fit on the target by deriving a set of pattern modifications via a linear program. We compute these alterations by employing an iterative process that alternates between global geometric optimization and physical simulation. Our method utilizes geometry-based simulation of woven fabric's anisotropic behavior, accounts for tailoring details like seam matching, and incorporates elements such as darts or gussets. We validate our technique by producing digital and physical garments, demonstrating practical and achievable alterations.
  • Item
    Inverse Garment and Pattern Modeling with a Differentiable Simulator
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Yu, Boyang; Cordier, Frederic; Seo, Hyewon; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    The capability to generate simulation-ready garment models from 3D shapes of clothed people will significantly enhance the interpretability of captured geometry of real garments, as well as their faithful reproduction in the digital world. This will have notable impact on fields like shape capture in social VR, and virtual try-on in the fashion industry. To align with the garment modeling process standardized by the fashion industry and cloth simulation software, it is required to recover 2D patterns, which are then placed around the wearer's body model and seamed prior to the draping simulation. This involves an inverse garment design problem, which is the focus of our work here: Starting with an arbitrary target garment geometry, our system estimates its animatable replica along with its corresponding 2D pattern. Built upon a differentiable cloth simulator, it runs an optimization process that is directed towards minimizing the deviation of the simulated garment shape from the target geometry, while maintaining desirable properties such as left-to-right symmetry. Experimental results on various real-world and synthetic data show that our method outperforms state-of-the-art methods in producing both high-quality garment models and accurate 2D patterns.
  • Item
    Adversarial Unsupervised Domain Adaptation for 3D Semantic Segmentation with 2D Image Fusion of Dense Depth
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Zhang, Xindan; Li, Ying; Sheng, Huankun; Zhang, Xinnian; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Unsupervised domain adaptation (UDA) is increasingly used for 3D point cloud semantic segmentation tasks due to its ability to address the issue of missing labels for new domains. However, most existing unsupervised domain adaptation methods focus only on uni-modal data and are rarely applied to multi-modal data. Therefore, we propose a cross-modal UDA on multimodal datasets that contain 3D point clouds and 2D images for 3D Semantic Segmentation. Specifically, we first propose a Dual discriminator-based Domain Adaptation (Dd-bDA) module to enhance the adaptability of different domains. Second, given that the robustness of depth information to domain shifts can provide more details for semantic segmentation, we further employ a Dense depth Feature Fusion (DdFF) module to extract image features with rich depth cues. We evaluate our model in four unsupervised domain adaptation scenarios, i.e., dataset-to-dataset (A2D2→SemanticKITTI), Day-to-Night, country-tocountry (USA→Singapore), and synthetic-to-real (VirtualKITTI→SemanticKITTI). In all settings, the experimental results achieve significant improvements and surpass state-of-the-art models.
  • Item
    SOD-diffusion: Salient Object Detection via Diffusion-Based Image Generators
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Zhang, Shuo; Huang, Jiaming; Chen, Shizhe; Wu, Yan; Hu, Tao; Liu, Jing; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Salient Object Detection (SOD) is a challenging task that aims to precisely identify and segment the salient objects. However, existing SOD methods still face challenges in making explicit predictions near the edges and often lack end-to-end training capabilities. To alleviate these problems, we propose SOD-diffusion, a novel framework that formulates salient object detection as a denoising diffusion process from noisy masks to object masks. Specifically, object masks diffuse from ground-truth masks to random distribution in latent space, and the model learns to reverse this noising process to reconstruct object masks. To enhance the denoising learning process, we design an attention feature interaction module (AFIM) and a specific fine-tuning protocol to integrate conditional semantic features from the input image with diffusion noise embedding. Extensive experiments on five widely used SOD benchmark datasets demonstrate that our proposed SOD-diffusion achieves favorable performance compared to previous well-established methods. Furthermore, leveraging the outstanding generalization capability of SOD-diffusion, we applied it to publicly available images, generating high-quality masks that serve as an additional SOD benchmark testset.
  • Item
    Color-Accurate Camera Capture with Multispectral Illumination and Multiple Exposures
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Gao, Hongyun; Mantiuk, Rafal K.; Finlayson, Graham D.; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Cameras cannot capture the same colors as those seen by the human eye because the eye and the cameras' sensors differ in their spectral sensitivity. To obtain a plausible approximation of perceived colors, the camera's Image Signal Processor (ISP) employs a color correction step. However, even advanced color correction methods cannot solve this underdetermined problem, and visible color inaccuracies are always present. Here, we explore an approach in which we can capture accurate colors with a regular camera by optimizing the spectral composition of the illuminant and capturing one or more exposures. We jointly optimize for the signal-to-noise ratio and for the color accuracy irrespective of the spectral composition of the scene. One or more images captured under controlled multispectral illuminants are then converted into a color-accurate image as seen under the standard illuminant of D65. Our optimization allows us to reduce the color error by 20-60% (in terms of CIEDE 2000), depending on the number of exposures and camera type. The method can be used in applications in which illumination can be controlled, and high colour accuracy is required, such as product photography or with a multispectral camera flash. The code is available at https://github.com/gfxdisp/multispectral_color_correction.
  • Item
    iShapEditing: Intelligent Shape Editing with Diffusion Models
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Li, Jing; Zhang, Juyong; Chen, Falai; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Recent advancements in generative models have enabled image editing very effective with impressive results. By extending this progress to 3D geometry models, we introduce iShapEditing, a novel framework for 3D shape editing which is applicable to both generated and real shapes. Users manipulate shapes by dragging handle points to corresponding targets, offering an intuitive and intelligent editing interface. Leveraging the Triplane Diffusion model and robust intermediate feature correspondence, our framework utilizes classifier guidance to adjust noise representations during sampling process, ensuring alignment with user expectations while preserving plausibility. For real shapes, we employ shape predictions at each time step alongside a DDPM-based inversion algorithm to derive their latent codes, facilitating seamless editing. iShapEditing provides effective and intelligent control over shapes without the need for additional model training or fine-tuning. Experimental examples demonstrate the effectiveness and superiority of our method in terms of editing accuracy and plausibility.
  • Item
    VRTree: Example-Based 3D Interactive Tree Modeling in Virtual Reality
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Wu, Di; Yang, Mingxin; Liu, Zhihao; Tu, Fangyuan; Liu, Fang; Cheng, Zhanglin; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    We present VRTree, an example-based interactive virtual reality (VR) system designed to efficiently create diverse 3D tree models while faithfully preserving botanical characteristics of real-world references. Our method employs a novel representation called Hierarchical Branch Lobe (HBL), which captures the hierarchical features of trees and serves as a versatile intermediary for intuitive VR interaction. The HBL representation decomposes a 3D tree into a series of concise examples, each consisting of a small set of main branches, secondary branches, and lobe-bounded twigs. The core of our system involves two key components: (1) We design an automatic algorithm to extract an initial library of HBL examples from real tree point clouds. These HBL examples can be optionally refined according to user intentions through an interactive editing process. (2) Users can interact with the extracted HBL examples to assemble new tree structures, ensuring the local features align with the target tree species. A shape-guided procedural growth algorithm then transforms these assembled HBL structures into highly realistic, finegrained 3D tree models. Extensive experiments and user studies demonstrate that VRTree outperforms current state-of-the-art approaches, offering a highly effective and easy-to-use VR tool for tree modeling.
  • Item
    SCARF: Scalable Continual Learning Framework for Memory-efficiency Multiple Neural Radiance Fields
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Wang, Yuze; Wang, Junyi; Wang, Chen; Duan, Wantong; Bao, Yongtang; Qi, Yue; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    This paper introduces a novel continual learning framework for synthesising novel views of multiple scenes, learning multiple 3D scenes incrementally, and updating the network parameters only with the training data of the upcoming new scene. We build on Neural Radiance Fields (NeRF), which uses multi-layer perceptron to model the density and radiance field of a scene as the implicit function. While NeRF and its extensions have shown a powerful capability of rendering photo-realistic novel views in a single 3D scene, managing these growing 3D NeRF assets efficiently is a new scientific problem. Very few works focus on the efficient representation or continuous learning capability of multiple scenes, which is crucial for the practical applications of NeRF. To achieve these goals, our key idea is to represent multiple scenes as the linear combination of a cross-scene weight matrix and a set of scene-specific weight matrices generated from a global parameter generator. Furthermore, we propose an uncertain surface knowledge distillation strategy to transfer the radiance field knowledge of previous scenes to the new model. Representing multiple 3D scenes with such weight matrices significantly reduces memory requirements. At the same time, the uncertain surface distillation strategy greatly overcomes the catastrophic forgetting problem and maintains the photo-realistic rendering quality of previous scenes. Experiments show that the proposed approach achieves state-of-the-art rendering quality of continual learning NeRF on NeRF-Synthetic, LLFF, and TanksAndTemples datasets while preserving extra low storage cost.
  • Item
    GauLoc: 3D Gaussian Splatting-based Camera Relocalization
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Xin, Zhe; Dai, Chengkai; Li, Ying; Wu, Chenming; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    3D Gaussian Splatting (3DGS) has emerged as a promising representation for scene reconstruction and novel view synthesis for its explicit representation and real-time capabilities. This technique thus holds immense potential for use in mapping applications. Consequently, there is a growing need for an efficient and effective camera relocalization method to complement the advantages of 3DGS. This paper presents a camera relocalization method, namely GauLoc, in a scene represented by 3DGS. Unlike previous methods that rely on pose regression or photometric alignment, our proposed method leverages the differential rendering capability provided by 3DGS. The key insight of our work is the proposed implicit featuremetric alignment, which effectively optimizes the alignment between rendered keyframes and the query frames, and leverages the epipolar geometry to facilitate the convergence of camera poses conditioned explicit 3DGS representation. The proposed method significantly improves the relocalization accuracy even in complex scenarios with large initial camera rotation and translation deviations. Extensive experiments validate the effectiveness of our proposed method, showcasing its potential to be applied in many realworld applications. Source code will be released at https://github.com/xinzhe11/GauLoc.
  • Item
    LGSur-Net: A Local Gaussian Surface Representation Network for Upsampling Highly Sparse Point Cloud
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Xiao, Zijian; Zhou, Tianchen; Yao, Li; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    We introduce LGSur-Net, an end-to-end deep learning architecture, engineered for the upsampling of sparse point clouds. LGSur-Net harnesses a trainable Gaussian local representation by positioning a series of Gaussian functions on an oriented plane, complemented by the optimization of individual covariance matrices. The integration of parametric factors allows for the encoding of the plane's rotational dynamics and Gaussian weightings into a linear transformation matrix. Then we extract the feature maps from the point cloud and its adjoining edges and learn the local Gaussian depictions to accurately model the shape's local geometry through an attention-based network. The Gaussian representation's inherent high-order continuity endows LGSur-Net with the natural ability to predict surface normals and support upsampling to any specified resolution. Comprehensive experiments validate that LGSur-Net efficiently learns from sparse data inputs, surpassing the performance of existing state-of-the-art upsampling methods. Our code is publicly available at https://github.com/Rangiant5b72/LGSur-Net.
  • Item
    Palette-Based Recolouring of Gradient Meshes
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Houssaije, Willard A. Verschoore de la; Echevarria, Jose; Kosinka, Jirí; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    Gradient meshes are a vector graphics primitive formed by a regular grid of bicubic quad patches. They allow for the creation of complex geometries and colour gradients, with recent extensions supporting features such as local refinement and sharp colour transitions. While many methods exist for recolouring raster images, often achieved by modifying an automatically detected palette of the image, gradient meshes have not received the same amount of attention when it comes to global colour editing. We present a novel method that allows for real-time palette-based recolouring of gradient meshes, including gradient meshes constructed using local refinement and containing sharp colour transitions. We demonstrate the utility of our method on synthetic illustrative examples as well as on complex gradient meshes.
  • Item
    G-Style: Stylized Gaussian Splatting
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Kovács, Áron Samuel; Hermosilla, Pedro; Raidou, Renata Georgia; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    We introduce G -Style, a novel algorithm designed to transfer the style of an image onto a 3D scene represented using Gaussian Splatting. Gaussian Splatting is a powerful 3D representation for novel view synthesis, as-compared to other approaches based on Neural Radiance Fields-it provides fast scene renderings and user control over the scene. Recent pre-prints have demonstrated that the style of Gaussian Splatting scenes can be modified using an image exemplar. However, since the scene geometry remains fixed during the stylization process, current solutions fall short of producing satisfactory results. Our algorithm aims to address these limitations by following a three-step process: In a pre-processing step, we remove undesirable Gaussians with large projection areas or highly elongated shapes. Subsequently, we combine several losses carefully designed to preserve different scales of the style in the image, while maintaining as much as possible the integrity of the original scene content. During the stylization process and following the original design of Gaussian Splatting, we split Gaussians where additional detail is necessary within our scene by tracking the gradient of the stylized color. Our experiments demonstrate that G -Style generates high-quality stylizations within just a few minutes, outperforming existing methods both qualitatively and quantitatively
  • Item
    Robust Diffusion-based Motion In-betweening
    (The Eurographics Association and John Wiley & Sons Ltd., 2024) Qin, Jia; Yan, Peng; An, Bo; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
    The emergence of learning-based motion in-betweening techniques offers animators a more efficient way to animate characters. However, existing non-generative methods either struggle to support long transition generation or produce results that lack diversity. Meanwhile, diffusion models have shown promising results in synthesizing diverse and high-quality motions driven by text and keyframes. However, in these methods, keyframes often serve as a guide rather than a strict constraint and can sometimes be ignored when keyframes are sparse. To address these issues, we propose a lightweight yet effective diffusionbased motion in-betweening framework that generates animations conforming to keyframe constraints.We incorporate keyframe constraints into the training phase to enhance robustness in handling various constraint densities. Moreover, we employ relative positional encoding to improve the model's generalization on long range in-betweening tasks. This approach enables the model to learn from short animations while generating realistic in-betweening motions spanning thousands of frames. We conduct extensive experiments to validate our framework using the newly proposed metrics K-FID, K-Diversity, and K-Error, designed to evaluate generative in-betweening methods. Results demonstrate that our method outperforms existing diffusion-based methods across various lengths and keyframe densities. We also show that our method can be applied to text-driven motion synthesis, offering fine-grained control over the generated results.