Volume 38 (2019)

Permanent URI for this community

https://diglib.eg.org/handle/10.2312/2632736

Browse

Now showing 1 - 20 of 267

A Low-Dimensional Function Space for Efficient Spectral Upsampling
(The Eurographics Association and John Wiley & Sons Ltd., 2019) Jakob, Wenzel; Hanika, Johannes; Alliez, Pierre and Pellacini, Fabio
We present a versatile technique to convert textures with tristimulus colors into the spectral domain, allowing such content to be used in modern rendering systems. Our method is based on the observation that suitable reflectance spectra can be represented using a low-dimensional parametric model that is intrinsically smooth and energy-conserving, which leads to significant simplifications compared to prior work. The resulting spectral textures are compact and efficient: storage requirements are identical to standard RGB textures, and as few as six floating point instructions are required to evaluate them at any wavelength. Our model is the first spectral upsampling method to achieve zero error on the full sRGB gamut. The technique also supports large-gamut color spaces, and can be vectorized effectively for use in rendering systems that handle many wavelengths at once.
Flexible SVBRDF Capture with a Multi-Image Deep Network
(The Eurographics Association and John Wiley & Sons Ltd., 2019) Deschaintre, Valentin; Aittala, Miika; Durand, Fredo; Drettakis, George; Bousseau, Adrien; Boubekeur, Tamy and Sen, Pradeep
Empowered by deep learning, recent methods for material capture can estimate a spatially-varying reflectance from a single photograph. Such lightweight capture is in stark contrast with the tens or hundreds of pictures required by traditional optimization-based approaches. However, a single image is often simply not enough to observe the rich appearance of realworld materials. We present a deep-learning method capable of estimating material appearance from a variable number of uncalibrated and unordered pictures captured with a handheld camera and flash. Thanks to an order-independent fusing layer, this architecture extracts the most useful information from each picture, while benefiting from strong priors learned from data. The method can handle both view and light direction variation without calibration. We show how our method improves its prediction with the number of input pictures, and reaches high quality reconstructions with as little as 1 to 10 images - a sweet spot between existing single-image and complex multi-image approaches.
Real‐Time Facial Expression Transformation for Monocular RGB Video
(© 2019 The Eurographics Association and John Wiley & Sons Ltd., 2019) Ma, L.; Deng, Z.; Chen, Min and Benes, Bedrich
This paper describes a novel real‐time end‐to‐end system for facial expression transformation, without the need of any driving source. Its core idea is to directly generate desired and photo‐realistic facial expressions on top of input monocular RGB video. Specifically, an unpaired learning framework is developed to learn the mapping between any two facial expressions in the facial blendshape space. Then, it automatically transforms the source expression in an input video clip to a specified target expression through the combination of automated 3D face construction, the learned bi‐directional expression mapping and automated lip correction. It can be applied to new users without additional training. Its effectiveness is demonstrated through many experiments on faces from live and online video, with different identities, ages, speeches and expressions.This paper describes a novel real‐time end‐to‐end system for facial expression transformation, without the need of any driving source. Its core idea is to directly generate desired and photo‐realistic facial expressions on top of input monocular RGB video. Specifically, an unpaired learning framework is developed to learn the mapping between any two facial expressions in the facial blendshape space. Then, it automatically transforms the source expression in an input video clip to a specified target expression through the combination of automated 3D face construction, the learned bi‐directional expression mapping and automated lip correction. It can be applied to new users without additional training. Its effectiveness is demonstrated through many experiments on faces from live and online video, with different identities, ages, speeches and expressions.
Light Field Video Compression and Real Time Rendering
(The Eurographics Association and John Wiley & Sons Ltd., 2019) Hajisharif, Saghi; Miandji, Ehsan; Larsson, Per; Tran, Kiet; Unger, Jonas; Lee, Jehee and Theobalt, Christian and Wetzstein, Gordon
Light field imaging is rapidly becoming an established method for generating flexible image based description of scene appearances. Compared to classical 2D imaging techniques, the angular information included in light fields enables effects such as post-capture refocusing and the exploration of the scene from different vantage points. In this paper, we describe a novel GPU pipeline for compression and real-time rendering of light field videos with full parallax. To achieve this, we employ a dictionary learning approach and train an ensemble of dictionaries capable of efficiently representing light field video data using highly sparse coefficient sets. A novel, key element in our representation is that we simultaneously compress both image data (pixel colors) and the auxiliary information (depth, disparity, or optical flow) required for view interpolation. During playback, the coefficients are streamed to the GPU where the light field and the auxiliary information are reconstructed using the dictionary ensemble and view interpolation is performed. In order to realize the pipeline we present several technical contributions including a denoising scheme enhancing the sparsity in the dataset which enables higher compression ratios, and a novel pruning strategy which reduces the size of the dictionary ensemble and leads to significant reductions in computational complexity during the encoding of a light field. Our approach is independent of the light field parameterization and can be used with data from any light field video capture system. To demonstrate the usefulness of our pipeline, we utilize various publicly available light field video datasets and discuss the medical application of documenting heart surgery.
Learning Style Compatibility Between Objects in a Real-World 3D Asset Database
(The Eurographics Association and John Wiley & Sons Ltd., 2019) Liu, Yifan; Tang, Ruolan; Ritchie, Daniel; Lee, Jehee and Theobalt, Christian and Wetzstein, Gordon
Large 3D asset databases are critical for designing virtual worlds, and using them effectively requires techniques for efficient querying and navigation. One important form of query is search by style compatibility: given a query object, find others that would be visually compatible if used in the same scene. In this paper, we present a scalable, learning-based approach for solving this problem which is designed for use with real-world 3D asset databases; we conduct experiments on 121 3D asset packages containing around 4000 3D objects from the Unity Asset Store. By leveraging the structure of the object packages, we introduce a technique to synthesize training labels for metric learning that work as well as human labels. These labels can grow exponentially with the number of objects, allowing our approach to scale to large real-world 3D asset databases without the need for expensive human training labels. We use these synthetic training labels in a metric learning model that analyzes the in-engine rendered appearance of an object—-combining geometry, material, and texture-whereas prior work considers only object geometry, or disjoint geometry and texture features. Through an ablation experiment, we find that using this representation yields better results than using renders which lack texture, materiality, or both.
Object Partitioning for Support-Free 3D-Printing
(The Eurographics Association and John Wiley & Sons Ltd., 2019) Karasik, Eli; Fattal, Raanan; Werman, Michael; Alliez, Pierre and Pellacini, Fabio
Fused deposition modeling based 3D-printing is becoming increasingly popular due to it's low-cost and simple operation and maintenance. While it produces rugged prints made from a wide range of materials, it suffers from an inherent printing limitation where it cannot produce overhanging surfaces of non-trivial size. This limitation can be handled by constructing temporary support-structures, however this solution involves additional material costs, longer print time, and often a fair amount of labor in removing it. In this paper we present a new method for partitioning general solid objects into a small number of parts that can be printed with no support. The partitioning is computed by applying a sequence of cutting-planes that split the object recursively. Unlike existing algorithms, the planes are not chosen at random, rather they are derived from shape analysis routines that identify and resolve various commonly-found geometric configurations. In addition, we guide this search by a revised set of conditions that both ensure the objects' printability as well as realistically model the printing capabilities of the printer at hand. Evaluation of the new method demonstrates its ability to efficiently obtain support-free partitionings typically containing fewer parts compared to existing methods that rely on support-structures.
Examining Implicit Discretization in Spectral Schemes
(The Eurographics Association and John Wiley & Sons Ltd., 2019) Quinan, P. Samuel; Padilla, Lace M. K.; Creem-Regehr, Sarah H.; Meyer, Miriah; Gleicher, Michael and Viola, Ivan and Leitte, Heike
Two of the primary reasons rainbow color maps are considered ineffective trace back to the idea that they implicitly discretize encoded data into hue-based bands, yet no research addresses what this discretization looks like or how consistent it is across individuals. This paper presents an exploratory study designed to empirically investigate the implicit discretization of common spectral schemes and explore whether the phenomenon can be modeled by variations in lightness, chroma, and hue. Our results suggest that three commonly used rainbow color maps are implicitly discretized with consistency across individuals. The results also indicate, however, that this implicit discretization varies across different datasets, in a way that suggests the visualization community's understanding of both rainbow color maps, and more generally effective color usage, remains incomplete.
Dense Point-to-Point Correspondences Between Genus-Zero Shapes
(The Eurographics Association and John Wiley & Sons Ltd., 2019) Lee, Sing Chun; Kazhdan, Misha; Bommes, David and Huang, Hui
We describe a novel approach that addresses the problem of establishing correspondences between non-rigidly deformed shapes by performing the registration over the unit sphere. In a pre-processing step, each shape is conformally parametrized over the sphere, centered to remove Möbius inversion ambiguity, and authalically evolved to expand regions that are excessively compressed by the conformal parametrization. Then, for each pair of shapes, we perform fast SO(3) correlation to find the optimal rotational alignment and refine the registration using optical flow. We evaluate our approach on the TOSCA dataset, demonstrating that our approach compares favorably to state-of-the-art methods.
User‐Guided Facial Animation through an Evolutionary Interface
(© 2019 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2019) Reed, K.; Cosker, D.; Chen, Min and Benes, Bedrich
We propose a design framework to assist with user‐generated content in facial animation — without requiring any animation experience or ground truth reference. Where conventional prototyping methods rely on handcrafting by experienced animators, our approach looks to encode the role of the animator as an Evolutionary Algorithm acting on animation controls, driven by visual feedback from a user. Presented as a simple interface, users sample control combinations and select favourable results to influence later sampling. Over multiple iterations of disregarding unfavourable control values, parameters converge towards the user's ideal. We demonstrate our framework through two non‐trivial applications: creating highly nuanced expressions by evolving control values of a face rig and non‐linear motion through evolving control point positions of animation curves.We propose a design framework to assist with user‐generated content in facial animation — without requiring any animation experience or ground truth reference. Where conventional prototyping methods rely on handcrafting by experienced animators, our approach looks to encode the role of the animator as an Evolutionary Algorithm acting on animation controls, driven by visual feedback from a user. Presented as a simple interface, users sample control combinations and select favourable results to influence later sampling. Over multiple iterations of disregarding unfavourable control values, parameters converge towards the user's ideal. We demonstrate our framework through two non‐trivial applications: creating highly nuanced expressions by evolving control values of a face rig and non‐linear motion through evolving control point positions of animation curves.
Solid Geometry Processing on Deconstructed Domains
(© 2019 The Eurographics Association and John Wiley & Sons Ltd., 2019) Sellán, Silvia; Cheng, Herng Yi; Ma, Yuming; Dembowski, Mitchell; Jacobson, Alec; Chen, Min and Benes, Bedrich
Many tasks in geometry processing are modelled as variational problems solved numerically using the finite element method. For solid shapes, this requires a volumetric discretization, such as a boundary conforming tetrahedral mesh. Unfortunately, tetrahedral meshing remains an open challenge and existing methods either struggle to conform to complex boundary surfaces or require manual intervention to prevent failure. Rather than create a single volumetric mesh for the entire shape, we advocate for solid geometry processing on , where a large and complex shape is composed of overlapping solid subdomains. As each smaller and simpler part is now easier to tetrahedralize, the question becomes how to account for overlaps during problem modelling and how to couple solutions on each subdomain together . We explore how and why previous coupling methods fail, and propose a method that couples solid domains only along their boundary surfaces. We demonstrate the superiority of this method through empirical convergence tests and qualitative applications to solid geometry processing on a variety of popular second‐order and fourth‐order partial differential equations.Many tasks in geometry processing are modelled as variational problems solved numerically using the finite element method. For solid shapes, this requires a volumetric discretization, such as a boundary conforming tetrahedral mesh. Unfortunately, tetrahedral meshing remains an open challenge and existing methods either struggle to conform to complex boundary surfaces or require manual intervention to prevent failure. Rather than create a single volumetric mesh for the entire shape, we advocate for solid geometry processing on , where a large and complex shape is composed of overlapping solid subdomains. As each smaller and simpler part is now easier to tetrahedralize, the question becomes how to account for overlaps during problem modelling and how to couple solutions on each subdomain together . We explore how and why previous coupling methods fail, and propose a method that couples solid domains only along their boundary surfaces. We demonstrate the superiority of this method through empirical convergence tests and qualitative applications to solid geometry processing on a variety of popular second‐order and fourth‐order partial differential equations.
Divergence-Free Shape Correspondence by Deformation
(The Eurographics Association and John Wiley & Sons Ltd., 2019) Eisenberger, Marvin; Lähner, Zorah; Cremers, Daniel; Bommes, David and Huang, Hui
We present a novel approach for solving the correspondence problem between a given pair of input shapes with non-rigid, nearly isometric pose difference. Our method alternates between calculating a deformation field and a sparse correspondence. The deformation field is constructed with a low rank Fourier basis which allows for a compact representation. Furthermore, we restrict the deformation fields to be divergence-free which makes our morphings volume preserving. This can be used to extract a correspondence between the inputs by deforming one of them along the deformation field using a second order Runge-Kutta method and resulting in an alignment of the inputs. The advantages of using our basis are that there is no need to discretize the embedding space and the deformation is volume preserving. The optimization of the deformation field is done efficiently using only a subsampling of the orginal shapes but the correspondence can be extracted for any mesh resolution with close to linear increase in runtime. We show 3D correspondence results on several known data sets and examples of natural intermediate shape sequences that appear as a by-product of our method.
MyEvents: A Personal Visual Analytics Approach for Mining Key Events and Knowledge Discovery in Support of Personal Reminiscence
(© 2019 The Eurographics Association and John Wiley & Sons Ltd., 2019) Parvinzamir, F.; Zhao, Y.; Deng, Z.; Dong, F.; Chen, Min and Benes, Bedrich
Reminiscence is an important aspect in our life. It preserves precious memories, allows us to form our own identities and encourages us to accept the past. Our work takes the advantage of modern sensor technologies to support reminiscence, enabling self‐monitoring of personal activities and individual movement in space and time on a daily basis. This paper presents MyEvents, a web‐based personal visual analytics platform designed for non‐computing experts, that allows for the collection of long‐term location and movement data and the generation of event mementos. Our research is focused on two prominent goals in event reminiscence: (1) selection subjectivity and human involvement in the process of self‐knowledge discovery and memento creation; and (2) the enhancement of event familiarity by presenting target events and their related information for optimal memory recall and reminiscence. A novel multi‐significance event ranking model is proposed to determine significant events in the personal history according to user preferences for event category, frequency and regularity. The evaluation results show that MyEvents effectively fulfils the reminiscence goals and tasks.Reminiscence is an important aspect in our life. It preserves precious memories, allows us to form our own identities and encourages us to accept the past. Our work takes the advantage of modern sensor technologies to support reminiscence, enabling self‐monitoring of personal activities and individual movement in space and time on a daily basis. This paper presents MyEvents, a web‐based personal visual analytics platform designed for non‐computing experts, that allows for the collection of long‐term location and movement data and the generation of event mementos. Our research is focused on two prominent goals in event reminiscence: (1) selection subjectivity and human involvement in the process of self‐knowledge discovery and memento creation; and (2) the enhancement of event familiarity by presenting target events and their related information for optimal memory recall and reminiscence. A novel multi‐significance event ranking model is proposed to determine significant events in the personal history according to user preferences for event category, frequency and regularity. The evaluation results show that MyEvents effectively fulfils the reminiscence goals and tasks.
Wavelet Flow: Optical Flow Guided Wavelet Facial Image Fusion
(The Eurographics Association and John Wiley & Sons Ltd., 2019) Ding, Hong; Yan, Qingan; Fu, Gang; Xiao, Chunxia; Lee, Jehee and Theobalt, Christian and Wetzstein, Gordon
Estimating the correspondence between the images using optical flow is the key component for image fusion, however, computing optical flow between a pair of facial images including backgrounds is challenging due to large differences in illumination, texture, color and background in the images. To improve optical flow results for image fusion, we propose a novel flow estimation method, wavelet flow, which can handle both the face and background in the input images. The key idea is that instead of computing flow directly between the input image pair, we estimate the image flow by incorporating multi-scale image transfer and optical flow guided wavelet fusion. Multi-scale image transfer helps to preserve the background and lighting detail of input, while optical flow guided wavelet fusion produces a series of intermediate images for further fusion quality optimizing. Our approach can significantly improve the performance of the optical flow algorithm and provide more natural fusion results for both faces and backgrounds in the images. We evaluate our method on a variety of datasets to show its high outperformance.
Generic Interactive Pixel-level Image Editing
(The Eurographics Association and John Wiley & Sons Ltd., 2019) Liang, Yun; Gan, Yibo; Chen, Mingqin; Gutierrez, Diego; Muñoz Orbañanos, Adolfo; Lee, Jehee and Theobalt, Christian and Wetzstein, Gordon
Several image editing methods have been proposed in the past decades, achieving brilliant results. The most sophisticated of them, however, require additional information per-pixel. For instance, dehazing requires a specific transmittance value per pixel, or depth of field blurring requires depth or disparity values per pixel. This additional per-pixel value is obtained either through elaborated heuristics or through additional control over the capture hardware, which is very often tailored for the specific editing application. In contrast, however, we propose a generic editing paradigm that can become the base of several different applications. This paradigm generates both the needed per-pixel values and the resulting edit at interactive rates, with minimal user input that can be iteratively refined. Our key insight for getting per-pixel values at such speed is to cluster them into superpixels, but, instead of a constant value per superpixel (which yields accuracy problems), we have a mathematical expression for pixel values at each superpixel: in our case, an order two multinomial per superpixel. This leads to a linear leastsquares system, effectively enabling specific per-pixel values at fast speeds. We illustrate this approach in three applications: depth of field blurring (from depth values), dehazing (from transmittance values) and tone mapping (from brightness and contrast local values), and our approach proves both favorably interactive and accurate in all three. Our technique is also evaluated with a common dataset and compared favorably.
Visualization Support for Developing a Matrix Calculus Algorithm: A Case Study
(The Eurographics Association and John Wiley & Sons Ltd., 2019) Giesen, Joachim; Klaus, Julien; Laue, Sören; Schreck, Ferdinand; Gleicher, Michael and Viola, Ivan and Leitte, Heike
The development of custom interactive visualization tools for specific domains and applications has been made much simpler recently by a surge of visualization tools, libraries and frameworks. Most of these tools are developed for classical data science applications, where a user is supported in analyzing measured or simulated data. But recently, there has also been an increasing interest in visual support for understanding machine learning algorithms and frameworks, especially for deep learning. Many, if not most, of the visualization support for (deep) learning addresses the developer of the learning system and not the end user (data scientist). Here we show on a specific example, namely the development of a matrix calculus algorithm, that supporting visualizations can also greatly benefit the development of algorithms in classical domains like in our case computer algebra. The idea is similar to visually supporting the understanding of learning algorithms, namely provide the developer with an interactive, visual tool that provides insights into the workings and, importantly, also into the failures of the algorithm under development. Developing visualization support for matrix calculus development went similar as the development of more traditional visual support systems for data analysts. First, we had to acquaint ourselves with the problem, its language and challenges by talking to the core developer of the matrix calculus algorithm. Once we understood the challenge, it was fairly easy to develop visual support that streamlined the development of the matrix calculus algorithm significantly.
Augmenting Tactile 3D Data Navigation With Pressure Sensing
(The Eurographics Association and John Wiley & Sons Ltd., 2019) Wang, Xiyao; Besançon, Lonni; Ammi, Mehdi; Isenberg, Tobias; Gleicher, Michael and Viola, Ivan and Leitte, Heike
We present a pressure-augmented tactile 3D data navigation technique, specifically designed for small devices, motivated by the need to support the interactive visualization beyond traditional workstations. While touch input has been studied extensively on large screens, current techniques do not scale to small and portable devices. We use phone-based pressure sensing with a binary mapping to separate interaction degrees of freedom (DOF) and thus allow users to easily select different manipulation schemes (e. g., users first perform only rotation and then with a simple pressure input to switch to translation). We compare our technique to traditional 3D-RST (rotation, scaling, translation) using a docking task in a controlled experiment. The results show that our technique increases the accuracy of interaction, with limited impact on speed. We discuss the implications for 3D interaction design and verify that our results extend to older devices with pseudo pressure and are valid in realistic phone usage scenarios.
High Quality Refinable G-splines for Locally Quad-dominant Meshes With T-gons
(The Eurographics Association and John Wiley & Sons Ltd., 2019) Karciauskas, Kestutis; Peters, Jorg; Bommes, David and Huang, Hui
Polyhedral modeling and re-meshing algorithms use T-junctions to add or remove feature lines in a quadrilateral mesh. In many ways this is akin to adaptive knot insertion in a tensor-product spline, but differs in that the designer or meshing algorithm does not necessarily protect the consistent combinatorial structure that is required to interpret the resulting quad-dominant mesh as the control net of a hierarchical spline - and so associate a smooth surface with the mesh as in the popular tensor-product spline paradigm. While G-splines for multi-sided holes or generalized subdivision can, in principle, convert quad-dominant meshes with T-junctions into smooth surfaces, they do not preserve the two preferred directions and so cause visible shape artifacts. Only recently have n-gons with T-junctions (T-gons) in unstructured quad-dominant meshes been recognized as a distinct challenge for generalized splines. This paper makes precise the notion of locally quad-dominant mesh as quad-meshes including t-nets, i.e. T-gons surrounded by quads; and presents the first high-quality G-spline construction that can use t-nets as control nets for spline surfaces suitable, e.g., for automobile outer surfaces. Remarkably, T-gons can be neighbors, separated by only one quad, both of T-gons and of points where many quads meet. A t-net surface cap consists of 16 polynomial pieces of degree (3,5) and is refinable in a way that is consistent with the surrounding surface. An alternative, everywhere bi-3 cap is not formally smooth, but achieves the same high-quality highlight line distribution.
Microfacet Model Regularization for Robust Light Transport
(The Eurographics Association and John Wiley & Sons Ltd., 2019) Jendersie, Johannes; Grosch, Thorsten; Boubekeur, Tamy and Sen, Pradeep
Today, Monte Carlo light transport algorithms are used in many applications to render realistic images. Depending on the complexity of the used methods, several light effects can or cannot be found by the sampling process. Especially, specular and smooth glossy surfaces often lead to high noise and missing light effects. Path space regularization provides a solution, improving any sampling algorithm, by modifying the material evaluation code. Previously, Kaplanyan and Dachsbacher [KD13] introduced the concept for pure specular interactions. We extend this idea to the commonly used microfacet models by manipulating the roughness parameter prior to the evaluation. We also show that this kind of regularization requires a change in the MIS weight computation and provide the solution. Finally, we propose two heuristics to adaptively reduce the introduced bias. Using our method, many complex light effects are reproduced and the fidelity of smooth objects is increased. Additionally, if a path was sampleable before, the variance is partially reduced.
Neural BTF Compression and Interpolation
(The Eurographics Association and John Wiley & Sons Ltd., 2019) Rainer, Gilles; Jakob, Wenzel; Ghosh, Abhijeet; Weyrich, Tim; Alliez, Pierre and Pellacini, Fabio
The Bidirectional Texture Function (BTF) is a data-driven solution to render materials with complex appearance. A typical capture contains tens of thousands of images of a material sample under varying viewing and lighting conditions.While capable of faithfully recording complex light interactions in the material, the main drawback is the massive memory requirement, both for storing and rendering, making effective compression of BTF data a critical component in practical applications. Common compression schemes used in practice are based on matrix factorization techniques, which preserve the discrete format of the original dataset. While this approach generalizes well to different materials, rendering with the compressed dataset still relies on interpolating between the closest samples. Depending on the material and the angular resolution of the BTF, this can lead to blurring and ghosting artefacts. An alternative approach uses analytic model fitting to approximate the BTF data, using continuous functions that naturally interpolate well, but whose expressive range is often not wide enough to faithfully recreate materials with complex non-local lighting effects (subsurface scattering, inter-reflections, shadowing and masking...). In light of these observations, we propose a neural network-based BTF representation inspired by autoencoders: our encoder compresses each texel to a small set of latent coefficients, while our decoder additionally takes in a light and view direction and outputs a single RGB vector at a time. This allows us to continuously query reflectance values in the light and view hemispheres, eliminating the need for linear interpolation between discrete samples. We train our architecture on fabric BTFs with a challenging appearance and compare to standard PCA as a baseline. We achieve competitive compression ratios and high-quality interpolation/extrapolation without blurring or ghosting artifacts.
EUROGRAPHICS 2019: CGF 38-2 Frontmatter
(The Eurographics Association and John Wiley & Sons Ltd., 2019) Alliez, Pierre; Pellacini, Fabio; Alliez, Pierre and Pellacini, Fabio
-

Browse

Browsing Volume 38 (2019) by Issue Date

Results Per Page

Sort Options