37-Issue 7

Permanent URI for this collection

https://diglib.eg.org/handle/10.2312/2632664

Browse

Now showing 1 - 20 of 43

Biorthogonal Wavelet Surface Reconstruction Using Partial Integrations
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Ren, Xiaohua; Lyu, Luan; He, Xiaowei; Cao, Wei; Yang, Zhixin; Sheng, Bin; Zhang, Yanci; Wu, Enhua; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
We introduce a new biorthogonal wavelet approach to creating a water-tight surface defined by an implicit function, from a finite set of oriented points. Our approach aims at addressing problems with previous wavelet methods which are not resilient to missing or nonuniformly sampled data. To address the problems, our approach has two key elements. First, by applying a three-dimensional partial integration, we derive a new integral formula to compute the wavelet coefficients without requiring the implicit function to be an indicator function. It can be shown that the previously used formula is a special case of our formula when the integrated function is an indicator function. Second, a simple yet general method is proposed to construct smooth wavelets with small support. With our method, a family of wavelets can be constructed with the same support size as previously used wavelets while having one more degree of continuity. Experiments show that our approach can robustly produce results comparable to those produced by the Fourier and Poisson methods, regardless of the input data being noisy, missing or nonuniform. Moreover, our approach does not need to compute global integrals or solve large linear systems.
Dynamic Deep Octree for High-resolution Volumetric Painting in Virtual Reality
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Kim, Yeojin; Kim, Byungmoon; Kim, Young J.; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
With virtual reality, digital painting on 2D canvas is now being extended to 3D space. In this paper, we generalize the 2D pixel canvas to a 3D voxel canvas to allow artists to synthesize volumetric color fields. We develop a deep and dynamic octree-based painting and rendering system using both CPU and GPU to take advantage of the characteristics of both processors (CPU for octree modeling and GPU for volume rendering). On the CPU-side, we dynamically adjust an octree and incrementally update the octree to GPU with low latency without compromising the frame rates of the rendering. Our octree is balanced and uses a novel 3-neighbor connectivity for format simplicity and efficient storage, while allowing constant neighbor access time in ray casting. To further reduce the GPU-side 3-neighbor computations, we precompute a culling mask in CPU and upload it to GPU. Finally, we analyze the numerical error-propagation in ray casting through high resolution octree and present a theoretical error bound.
Generative Adversarial Image Super-Resolution Through Deep Dense Skip Connections
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Zhu, Xiaobin; Li, Zhuangzi; Zhang, Xiaoyu; Li, Haisheng; Xue, Ziyu; Wang, Lei; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Recently, image super-resolution works based on Convolutional Neural Networks (CNNs) and Generative Adversarial Nets (GANs) have shown promising performance. However, these methods tend to generate blurry and over-smoothed super-resolved (SR) images, due to the incomplete loss function and powerless architectures of networks. In this paper, a novel generative adversarial image super-resolution through deep dense skip connections (GSR-DDNet), is proposed to solve the above-mentioned problems. It aims to take advantage of GAN's ability of modeling data distributions, so that GSR-DDNet can select informative feature representation and model the mapping across the low-quality and high-quality images in an adversarial way. The pipeline of the proposed method consists of three main components: 1) The generator of a novel dense skip connection network with the deep structure for learning robust mapping function is proposed to generate SR images from low-resolution images; 2) The feature extraction network based on VGG-19 is adopted to capture high frequency feature maps for content loss; and 3) The discriminator with Wasserstein distance is adopted to identify the overall style of SR and ground-truth images. Experiments conducted on four publicly available datasets demonstrate the superiority against the state-of-the-art methods.
Mumford-Shah Mesh Processing using the Ambrosio-Tortorelli Functional
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Bonneel, Nicolas; Coeurjolly, David; Gueth, Pierre; Lachaud, Jacques-Olivier; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
The Mumford-Shah functional approximates a function by a piecewise smooth function. Its versatility makes it ideal for tasks such as image segmentation or restoration, and it is now a widespread tool of image processing. Recent work has started to investigate its use for mesh segmentation and feature lines detection, but we take the stance that the power of this functional could reach far beyond these tasks and integrate the everyday mesh processing toolbox. In this paper, we discretize an Ambrosio-Tortorelli approximation via a Discrete Exterior Calculus formulation. We show that, combined with a new shape optimization routine, several mesh processing problems can be readily tackled within the same framework. In particular, we illustrate applications in mesh denoising, normal map embossing, mesh inpainting and mesh segmentation.
Deep Video Stabilization Using Adversarial Networks
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Xu, Sen-Zhe; Hu, Jun; Wang, Miao; Mu, Tai-Jiang; Hu, Shi-Min; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Video stabilization is necessary for many hand-held shot videos. In the past decades, although various video stabilization methods were proposed based on the smoothing of 2D, 2.5D or 3D camera paths, hardly have there been any deep learning methods to solve this problem. Instead of explicitly estimating and smoothing the camera path, we present a novel online deep learning framework to learn the stabilization transformation for each unsteady frame, given historical steady frames. Our network is composed of a generative network with spatial transformer networks embedded in different layers, and generates a stable frame for the incoming unstable frame by computing an appropriate affine transformation. We also introduce an adversarial network to determine the stability of a piece of video. The network is trained directly using the pair of steady and unsteady videos. Experiments show that our method can produce similar results as traditional methods, moreover, it is capable of handling challenging unsteady video of low quality, where traditional methods fail, such as video with heavy noise or multiple exposures. Our method runs in real time, which is much faster than traditional methods.
Curvature Continuity Conditions Between Adjacent Toric Surface Patches
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Sun, Lanyin; Zhu, Chungang; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Toric surface patch is the multi-sided generalization of classical Bézier surface patch. Geometric continuity of the parametric surface patches plays a crucial role in geometric modeling. In this paper, the necessary and sufficient conditions of curvature continuity between toric surface patches are illustrated with the theory of toric degeneration. Furthermore, some practical sufficient conditions of curvature continuity of toric surface patches are also developed.
FashionGAN: Display your fashion design using Conditional Generative Adversarial Nets
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Cui, Yi Rui; Liu, Qi; Gao, Cheng Ying; Su, Zhuo; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Virtual garment display plays an important role in fashion design for it can directly show the design effect of the garment without having to make a sample garment like traditional clothing industry. In this paper, we propose an end-to-end virtual garment display method based on Conditional Generative Adversarial Networks. Different from existing 3D virtual garment methods which need complex interactions and domain-specific user knowledge, our method only need users to input a desired fashion sketch and a specified fabric image then the image of the virtual garment whose shape and texture are consistent with the input fashion sketch and fabric image can be shown out quickly and automatically. Moreover, it can also be extended to contour images and garment images, which further improves the reuse rate of fashion design. Compared with the existing image-to-image methods, the quality of images generated by our method is better in terms of color and shape.
Sit & Relax: Interactive Design of Body-Supporting Surfaces
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Leimer, Kurt; Birsak, Michael; Rist, Florian; Musialski, Przemyslaw; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
We propose a novel method for interactive design of well-fitting body-supporting surfaces that is driven by the pressure distribution on the body's surface. Our main contribution is an interactive modeling system that utilizes captured body poses and computes an importance field that is proportional to the pressure distribution on the body for a given pose. This distribution indicates where the body should be supported in order to easily hold a particular pose, which is one of the measures of comfortable sitting. Using our approximation, we propose the entire workflow for interactive design of C2 smooth surfaces which serve as seats, or generally, as body supporting furniture for comfortable sitting. Finally, we also provide a design tool for RHINOCEROS/GRASSHOPPER that allows for interactive creation of single designs or entire multi-person sitting scenarios. We also test the tool with design students and present several results. Our method aims at interactive design in order to help designers to create appropriate surfaces digitally without additional empirical design passes.
Light Optimization for Detail Highlighting
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Gkaravelis, Anastasios; Papaioannou, Georgios; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
In this paper we propose an effective technique for the automatic arrangement of spot lights and other luminaires on or near user-provided arbitrary mounting surfaces in order to highlight the geometric details of complex objects. Since potential applications include the lighting design for exhibitions and similar installations, the method takes into account obstructing geometry and potential occlusion from visitors and other non-permanent blocking geometry. Our technique generates the most appropriate position and orientation for light sources based on a local contrast maximization near salient geometric features and a clustering mechanism, producing consistent and view-independent results, with minimal user intervention. We validate our method with realistic test cases including multiple and disjoint exhibits as well as high occlusion scenarios.
Parallel Multigrid for Nonlinear Cloth Simulation
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Wang, Zhendong; Wu, Longhua; Fratarcangeli, Marco; Tang, Min; Wang, Huamin; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Accurate high-resolution simulation of cloth is a highly desired computational tool in graphics applications. As singleresolution simulation starts to reach the limit of computational power, we believe the future of cloth simulation is in multi-resolution simulation. In this paper, we explore nonlinearity, adaptive smoothing, and parallelization under a full multigrid (FMG) framework. The foundation of this research is a novel nonlinear FMG method for unstructured meshes. To introduce nonlinearity into FMG, we propose to formulate the smoothing process at each resolution level as the computation of a search direction for the original high-resolution nonlinear optimization problem. We prove that our nonlinear FMG is guaranteed to converge under various conditions and we investigate the improvements to its performance. We present an adaptive smoother which is used to reduce the computational cost in the regions with low residuals already. Compared to normal iterative solvers, our nonlinear FMG method provides faster convergence and better performance for both Newton's method and Projective Dynamics. Our experiment shows our method is efficient, accurate, stable against large time steps, and friendly with GPU parallelization. The performance of the method has a good scalability to the mesh resolution, and the method has good potential to be combined with multi-resolution collision handling for real-time simulation in the future.
Controlling Stroke Size in Fast Style Transfer with Recurrent Convolutional Neural Network
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Yang, Lingchen; Yang, Lumin; Zhao, Mingbo; Zheng, Youyi; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Controlling stroke size in Fast Style Transfer remains a difficult task. So far, only a few attempts have been made towards it, and they still exhibit several deficiencies regarding efficiency, flexibility, and diversity. In this paper, we aim to tackle these problems and propose a recurrent convolutional neural subnetwork, which we call recurrent stroke-pyramid, to control the stroke size in Fast Style Transfer. Compared to the state-of-the-art methods, our method not only achieves competitive results with much fewer parameters but provides more flexibility and efficiency for generalizing to unseen larger stroke size and being able to produce a wide range of stroke sizes with only one residual unit. We further embed the recurrent stroke-pyramid into the Multi-Styles and the Arbitrary-Style models, achieving both style and stroke-size control in an entirely feed-forward manner with two novel run-time control strategies.
Binocular Tone Mapping with Improved Overall Contrast and Local Details
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Zhang, Zhuming; Hu, Xinghong; Liu, Xueting; Wong, Tien-Tsin; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Tone mapping is a commonly used technique that maps the set of colors in high-dynamic-range (HDR) images to another set of colors in low-dynamic-range (LDR) images, to fit the need for print-outs, LCD monitors and projectors. Unfortunately, during the compression of dynamic range, the overall contrast and local details generally cannot be preserved simultaneously. Recently, with the increased use of stereoscopic devices, the notion of binocular tone mapping has been proposed in the existing research study. However, the existing research lacks the binocular perception study and is unable to generate the optimal binocular pair that presents the most visual content. In this paper, we propose a novel perception-based binocular tone mapping method, that can generate an optimal binocular image pair (generating left and right images simultaneously) from an HDR image that presents the most visual content by designing a binocular perception metric. Our method outperforms the existing method in terms of both visual and time performance.
Semantic Reconstruction: Reconstruction of Semantically Segmented 3D Meshes via Volumetric Semantic Fusion
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Jeon, Junho; Jung, Jinwoong; Kim, Jungeon; Lee, Seungyong; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Semantic segmentation partitions a given image or 3D model of a scene into semantically meaning parts and assigns predetermined labels to the parts. With well-established datasets, deep networks have been successfully used for semantic segmentation of RGB and RGB-D images. On the other hand, due to the lack of annotated large-scale 3D datasets, semantic segmentation for 3D scenes has not yet been much addressed with deep learning. In this paper, we present a novel framework for generating semantically segmented triangular meshes of reconstructed 3D indoor scenes using volumetric semantic fusion in the reconstruction process. Our method integrates the results of CNN-based 2D semantic segmentation that is applied to the RGB-D stream used for dense surface reconstruction. To reduce the artifacts from noise and uncertainty of single-view semantic segmentation, we introduce adaptive integration for the volumetric semantic fusion and CRF-based semantic label regularization. With these methods, our framework can easily generate a high-quality triangular mesh of the reconstructed 3D scene with dense (i.e., per-vertex) semantic labels. Extensive experiments demonstrate that our semantic segmentation results of 3D scenes achieves the state-of-the-art performance compared to the previous voxel-based and point cloud-based methods.
Automatic Mechanism Modeling from a Single Image with CNNs
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Lin, Minmin; Shao, Tianjia; Zheng, Youyi; Ren, Zhong; Weng, Yanlin; Yang, Yin; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
This paper presents a novel system that enables a fully automatic modeling of both 3D geometry and functionality of a mechanism assembly from a single RGB image. The resulting 3D mechanism model highly resembles the one in the input image with the geometry, mechanical attributes, connectivity, and functionality of all the mechanical parts prescribed in a physically valid way. This challenging task is realized by combining various deep convolutional neural networks to provide high-quality and automatic part detection, segmentation, camera pose estimation and mechanical attributes retrieval for each individual part component. On the top of this, we use a local/global optimization algorithm to establish geometric interdependencies among all the parts while retaining their desired spatial arrangement. We use an interaction graph to abstract the inter-part connection in the resulting mechanism system. If an isolated component is identified in the graph, our system enumerates all the possible solutions to restore the graph connectivity, and outputs the one with the smallest residual error. We have extensively tested our system with a wide range of classic mechanism photos, and experimental results show that the proposed system is able to build high-quality 3D mechanism models without user guidance.
Improved Use of LOP for Curve Skeleton Extraction
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Li, Lei; Wang, Wencheng; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
It remains a challenge to robustly and rapidly extract high quality curve skeletons from 3D models of closed surfaces, especially when there are nearby surface sheets. In this paper, we address this challenge by improving the use of LOP (Locally Optimal Projection) to adaptively contract medial surfaces of 3D models. LOP was originally designed to optimize a raw scanned point cloud to its corresponding geometry surface. It has the effect of contraction, and the contraction amplitude is controlled by a support radius. Our improvements are twofold. First, we constrain the LOP operator applied in the 2D medial surface instead of in the 3D space and take a local region growing strategy to find neighborhoods for implementing LOP. Thus, we avoid interference between disconnected surface parts and accelerate the process due to the reduced search space. Second, we adaptively adjust the support radii to have different parts of the medial surface contracted adaptively and synchronously for generating connected skeletal curves. In this paper, we demonstrate that our method allows for each part of the medial surface to be contracted symmetrically to its center line and is insensitive to surface noises. Thus, with our method, centered and connected high quality curve skeletons can be extracted robustly and rapidly, even for models with nearby surface sheets. Experimental results highlight the effectiveness and high efficiency of the method, even for noisy and topologically complex models, making it superior to other state-of-the-art methods.
Reconstructing Flowers from Sketches
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Bobenrieth, Cédric; Seo, Hyewon; Cordier, Frédéric; Habibi, Arash; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
As the symbol of beauty, floral objects have been one of the most popular subjects of artistic drawing. However, designing 3D floral models is generally time- and resource-consuming, because of their structural and geometrical complexity. In this paper, we address the problem of reconstructing floral objects from sketch input. The user draws a relatively clean sketch of a flower and a few additional guide markings from an arbitrary view to rapidly create quality geometric models of flowers. Our system offers a novel modeling scheme compared to several existing flower modelers accepting sketch as input, where the user is required to work with different views, providing step-by-step sketch input. Given the silhouette and the guide strokes, an assumed, common botanical structure is estimated, i.e. a cone for each ring of petals. The cones and the silhouette sketch that we segment into elementary curves are used to retrieve model elements from the pre-constructed shape database. These elements are then placed together around the cone, where an additional, per-element deformation is performed so as to maximize the silhouette similarity between the user sketch and the 3D flower model from the chosen view. Our system has shown to robustly create a variety of flowers in various configurations, including flower models with several petal layers and various blooming degrees, drawn from different views.
Non-Local Low-Rank Normal Filtering for Mesh Denoising
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Li, Xianzhi; Zhu, Lei; Fu, Chi-Wing; Heng, Pheng-Ann; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
This paper presents a non-local low-rank normal filtering method for mesh denoising. By exploring the geometric similarity between local surface patches on 3D meshes in the form of normal fields, we devise a low-rank recovery model that filters normal vectors by means of patch groups. In summary, our method has the following key contributions. First, we present the guided normal patch covariance descriptor to analyze the similarity between patches. Second, we pack normal vectors on similar patches into the normal-field patch-group (NPG) matrix for rank analysis. Third, we formulate mesh denoising as a low-rank matrix recovery problem based on the prior that the rank of the NPG matrix is high for raw meshes with noise, but can be significantly reduced for denoised meshes, whose normal vectors across similar patches should be more strongly correlated. Furthermore, we devise an objective function based on an improved truncated 'gamma' norm, and derive an optimization procedure using the alternative direction method of multipliers and iteratively re-weighted least squares techniques.We conducted several experiments to evaluate our method using various 3D models, and compared our results against several state-of-the-art methods. Experimental results show that our method consistently outperforms other methods and better preserves the fine details.
Translucent Image Recoloring through Homography Estimation
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Huang, Yifei; Wang, Changbo; Li, Chenhui; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Image color editing techniques are of great significance for users who wish to adjust the image color. However, previous works paid less attention to the translucent images. In this paper, we propose a new method to recolor the translucent images while preserving detailed information and color relationships of the source image. We consider the recolor problem as a location transformation problem and solve it in two steps: automatic palette extraction and homography estimation. First, we propose the Hmeans method to extract the dominant colors of the source image based on histogram statistics and clustering. Then, we propose homography estimation to map the source colors to desired colors in the CIE-LAB color space. Further, we adopt a non-linear optimization approach to improve the result generated by the last step. The proposed method maintains high fidelity of the source image. Experiments have shown that our method generates a state-of-the-art visual result, in particular in the shadow areas. The source images with ground truth generated by a ray tracer further verify the effectiveness of our method.
Learning Scene Illumination by Pairwise Photos from Rear and Front Mobile Cameras
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Cheng, Dachuan; Shi, Jian; Chen, Yanyun; Deng, Xiaoming; Zhang, Xiaopeng; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Illumination estimation is an essential problem in computer vision, graphics and augmented reality. In this paper, we propose a learning based method to recover low-frequency scene illumination represented as spherical harmonic (SH) functions by pairwise photos from rear and front cameras on mobile devices. An end-to-end deep convolutional neural network (CNN) structure is designed to process images on symmetric views and predict SH coefficients. We introduce a novel Render Loss to improve the rendering quality of the predicted illumination. A high quality high dynamic range (HDR) panoramic image dataset was developed for training and evaluation. Experiments show that our model produces visually and quantitatively superior results compared to the state-of-the-arts. Moreover, our method is practical for mobile-based applications.
A Practical Approach to Physically-Based Reproduction of Diffusive Cosmetics
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Kim, Goanghun; Ko, Hyeong-Seok; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
In this paper, we introduce so-called the bSX method as a new way to utilize the Kubelka-Munk (K-M) model. Assuming the material is completely diffusive, the K-M model gives the reflectance and transmittance of the material from the observation of the material applied on a backing, where the observation includes the thickness of the material application. By rearranging the original K-M equation, we propose that the reflectance and transmittance can be calculated without knowing the thickness. This is a practically useful contribution. Based on the above finding, we develop the bSX method which can (1) capture the material specific parameters from the two photos - taken before and after the material application, and (2) reproduce its effect on a novel backing. We experimented the proposed method in various cases related to virtual cosmetic try-on, which include (1) capture from a single color backing, (2) capture from human skin backing, (3) reproduction of varying thickness effect, (4) reproduction of multi-layer cosmetic application effect, (5) applying the proposed method to makeup transfer. Compared to previous image-based makeup transfer methods, the bSX method reproduces the feel of the cosmetics more accurately.

Browse

Browsing 37-Issue 7 by Issue Date

Results Per Page

Sort Options