Volume 36 (2017)

Permanent URI for this community

https://diglib.eg.org/handle/10.2312/2631199

Browse

Now showing 1 - 20 of 248

Photometric Stabilization for Fast-forward Videos
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Zhang, Xuaner; Lee, Joon-Young; Sunkavalli, Kalyan; Wang, Zhaowen; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
Videos captured by consumer cameras often exhibit temporal variations in color and tone that are caused by camera autoadjustments like white-balance and exposure. When such videos are sub-sampled to play fast-forward, as in the increasingly popular forms of timelapse and hyperlapse videos, these temporal variations are exacerbated and appear as visually disturbing high frequency flickering. Previous techniques to photometrically stabilize videos typically rely on computing dense correspondences between video frames, and use these correspondences to remove all color changes in the video sequences. However, this approach is limited in fast-forward videos that often have large content changes and also might exhibit changes in scene illumination that should be preserved. In this work, we propose a novel photometric stabilization algorithm for fast-forward videos that is robust to large content-variation across frames. We compute pairwise color and tone transformations between neighboring frames and smooth these pair-wise transformations while taking in account the possibility of scene/content variations. This allows us to eliminate high-frequency fluctuations, while still adapting to real variations in scene characteristics. We evaluate our technique on a new dataset consisting of controlled synthetic and real videos, and demonstrate that our techniques outperforms the state-of-the-art.
Efficient Gradient-Domain Compositing Using an Approximate Curl-free Wavelet Projection
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Ren, Xiaohua; Luan, Lyu; He, Xiaowei; Zhang, Yanci; Wu, Enhua; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
Gradient-domain compositing has been widely used to create a seamless composite with gradient close to a composite gradient field generated from one or more registered images. The key to this problem is to solve a Poisson equation, whose unknown variables can reach the size of the composite if no region of interest is drawn explicitly, thus making both the time and memory cost expensive in processing multi-megapixel images. In this paper, we propose an approximate projection method based on biorthogonal Multiresolution Analyses (MRA) to solve the Poisson equation. Unlike previous Poisson equation solvers which try to converge to the accurate solution with iterative algorithms, we use biorthogonal compactly supported curl-free wavelets as the fundamental bases to approximately project the composite gradient field onto a curl-free vector space. Then, the composite can be efficiently recovered by applying a fast inverse wavelet transform. Considering an n-pixel composite, our method only requires 2n of memory for all vector fields and is more efficient than state-of-the-art methods while achieving almost identical results. Specifically, experiments show that our method gains a 5x speedup over the streaming multigrid in certain cases.
Split-Depth Image Generation and Optimization
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Liao, Jingtang; Eisemann, Martin; Eisemann, Elmar; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
Split-depth images use an optical illusion, which can enhance the 3D impression of a 2D animation. In split-depth images (also often called split-depth GIFs due to the commonly used file format), static virtual occluders in form of vertical or horizontal bars are added to a video clip, which leads to occlusions that are interpreted by the observer as a depth cue. In this paper, we study different factors that contribute to the illusion and propose a solution to generate split-depth images for a given RGB + depth image sequence. The presented solution builds upon a motion summarization of the object of interest (OOI) through space and time. It allows us to formulate the bar positioning as an energy-minimization problem, which we solve efficiently. We take a variety of important features into account, such as the changes of the 3D effect due to changes in the motion topology, occlusion, the proximity of bars or the OOI, and scene saliency. We conducted a number of psycho-visual experiments to derive an appropriate energy formulation. Our method helps in finding optimal positions for the bars and, thus, improves the 3D perception of the original animation. We demonstrate the effectiveness of our approach on a variety of examples. Our study with novice users shows that our approach allows them to quickly create satisfying results even for complex animations.
A Probabilistic Framework for Component-based Vector Graphics
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Lieng, Henrik; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
We propose a framework for data-driven manipulation and synthesis of component-based vector graphics. Using labelled vector graphical images of a given type of object as input, our processing pipeline produces training data, learns a probabilistic Bayesian network from that training data, and offer various data-driven vector-related tools using synthesis functions. The tools ranges from data-driven vector design to automatic synthesis of vector graphics. Our tools were well received by designers, our model provides good generalisation performance, also from small data sets, and our method for synthesis produces vector graphics deemed significantly more plausible compared with alternative methods.
Modeling, Evaluation and Optimization of Interlocking Shell Pieces
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Yao, Miaojun; Chen, Zhili; Xu, Weiwei; Wang, Huamin; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
While the 3D printing technology has become increasingly popular in recent years, it suffers from two critical limitations: expensive printing material and long printing time. An effective solution is to hollow the 3D model into a shell and print the shell by parts. Unfortunately, making shell pieces tightly assembled and easy to disassemble seem to be two contradictory conditions, and there exists no easy way to satisfy them at the same time yet. In this paper, we present a computational system to design an interlocking structure of a partitioned shell model, which uses only male and female connectors to lock shell pieces in the assembled configuration. Given a mesh segmentation input, our system automatically finds an optimal installation plan specifying both the installation order and the installation directions of the pieces, and then builds the models of the shell pieces using optimized shell thickness and connector sizes. To find the optimal installation plan, we develop simulation-based and data-driven metrics, and we incorporate them into an optimal plan search algorithm with fast pruning and local optimization strategies. The whole system is automatic, except for the shape design of the key piece. The interlocking structure does not introduce new gaps on the outer surface, which would become noticeable inevitably due to limited printer precision. Our experiment shows that the assembled object is strong against separation, yet still easy to disassemble.
Saliency-aware Real-time Volumetric Fusion for Object Reconstruction
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Yang, Sheng; Chen, Kang; Liu, Minghua; Fu, Hongbo; Hu, Shi-Min; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
We present a real-time approach for acquiring 3D objects with high fidelity using hand-held consumer-level RGB-D scanning devices. Existing real-time reconstruction methods typically do not take the point of interest into account, and thus might fail to produce clean reconstruction results of desired objects due to distracting objects or backgrounds. In addition, any changes in background during scanning, which can often occur in real scenarios, can easily break up the whole reconstruction process. To address these issues, we incorporate visual saliency into a traditional real-time volumetric fusion pipeline. Salient regions detected from RGB-D frames suggest user-intended objects, and by understanding user intentions our approach can put more emphasis on important targets, and meanwhile, eliminate disturbance of non-important objects. Experimental results on realworld scans demonstrate that our system is capable of effectively acquiring geometric information of salient objects in cluttered real-world scenes, even if the backgrounds are changing.
Patch2Vec: Globally Consistent Image Patch Representation
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Fried, Ohad; Avidan, Shai; Cohen-Or, Daniel; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
Many image editing applications rely on the analysis of image patches. In this paper, we present a method to analyze patches by embedding them to a vector space, in which the Euclidean distance reflects patch similarity. Inspired by Word2Vec, we term our approach Patch2Vec. However, there is a significant difference between words and patches. Words have a fairly small and well defined dictionary. Image patches, on the other hand, have no such dictionary and the number of different patch types is not well defined. The problem is aggravated by the fact that each patch might contain several objects and textures. Moreover, Patch2Vec should be universal because it must be able to map never-seen-before texture to the vector space. The mapping is learned by analyzing the distribution of all natural patches. We use Convolutional Neural Networks (CNN) to learn Patch2Vec. In particular, we train a CNN on labeled images with a triplet-loss objective function. The trained network encodes a given patch to a 128D vector. Patch2Vec is evaluated visually, qualitatively, and quantitatively. We then use several variants of an interactive single-click image segmentation algorithm to demonstrate the power of our method.
Modeling Cumulus Cloud Scenes from High-resolution Satellite Images
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Zhang, Zili; Liang, Xiaohui; Yuan, Chunqiang; Li, Frederick W. B.; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
We present a reconstruction framework, which fits physically-based constraints to model large-scale cloud scenes from satellite images. Applications include weather phenomena visualization, flight simulation, and weather spotter training. In our method, the cloud shape is assumed to be composed of a cloud top surface and a nearly flat cloud base surface. Based on this, an effective method of multi-spectral data processing is developed to obtain relevant information for calculating the cloud base height and the cloud top height, including ground temperature, cloud top temperature and cloud shadow. A lapse rate model is proposed to formulate cloud shape as an implicit function of temperature lapse rate and cloud base temperature. After obtaining initial cloud shapes, we enrich the shapes by a fractal method and represent reconstructed clouds by a particle system. Experiment results demonstrate the capability of our method in generating physically sound large-scale cloud scenes from high-resolution satellite images.
High-resolution 360 Video Foveated Stitching for Real-time VR
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Lee, Wei-Tse; Chen, Hsin-I; Chen, Ming-Shiuan; Shen, I-Chao; Chen, Bing-Yu; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
In virtual reality (VR) applications, the contents are usually generated by creating a 360 video panorama of a real-world scene. Although many capture devices are being released, getting high-resolution panoramas and displaying a virtual world in realtime remains challenging due to its computationally demanding nature. In this paper, we propose a real-time 360 video foveated stitching framework, that renders the entire scene in different level of detail, aiming to create a high-resolution panoramic video in real-time that can be streamed directly to the client. Our foveated stitching algorithm takes videos from multiple cameras as input, combined with measurements of human visual attention (i.e. the acuity map and the saliency map), can greatly reduce the number of pixels to be processed. We further parallelize the algorithm using GPU to achieve a responsive interface and validate our results via a user study. Our system accelerates graphics computation by a factor of 6 on a Google Cardboard display.
Data-Driven Sparse Priors of 3D Shapes
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Remil, Oussama; Xie, Qian; Xie, Xingyu; Xu, Kai; Wang, Jun; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
We present a sparse optimization framework for extracting sparse shape priors from a collection of 3D models. Shape priors are defined as point-set neighborhoods sampled from shape surfaces which convey important information encompassing normals and local shape characterization. A 3D shape model can be considered to be formed with a set of 3D local shape priors, while most of them are likely to have similar geometry. Our key observation is that the local priors extracted from a family of 3D shapes lie in a very low-dimensional manifold. Consequently, a compact and informative subset of priors can be learned to efficiently encode all shapes of the same family. A comprehensive library of local shape priors is first built with the given collection of 3D models of the same family. We then formulate a global, sparse optimization problem which enforces selecting representative priors while minimizing the reconstruction error. To solve the optimization problem, we design an efficient solver based on the Augmented Lagrangian Multipliers method (ALM). Extensive experiments exhibit the power of our data-driven sparse priors in elegantly solving several high-level shape analysis applications and geometry processing tasks, such as shape retrieval, style analysis and symmetry detection.
A Data-Driven Approach for Sketch-Based 3D Shape Retrieval via Similar Drawing-Style Recommendation
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Wang, Fei; Lin, Shujin; Luo, Xiaonan; Wu, Hefeng; Wang, Ruomei; Zhou, Fan; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
Sketching is a simple and natural way of expression and communication for humans. For this reason, it gains increasing popularity in human computer interaction, with the emergence of multitouch tablets and styluses. In recent years, sketch-based interactive methods are widely used in many retrieval systems. In particular, a variety of sketch-based 3D model retrieval works have been presented. However, almost all of these works focus on directly matching sketches with the projection views of 3D models, and they suffer from the large differences between the sketch drawing and the views of 3D models, leading to unsatisfying retrieval results. Therefore, in this paper, during the matching procedure in the retrieval, we propose to match the sketch with each 3D model from historical users instead of projection views. Yet since the sketches between the current user and the historical users can have big difference, we also aim to handle users' personalized deviations and differences. To this end, we leverage recommendation algorithms to estimate the drawing style characteristic similarity between the current user and historical users. Experimental results on the Large Scale Sketch Track Benchmark(SHREC14LSSTB) demonstrate that our method outperforms several state-of-the-art methods.
Printable 3D Trees
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Bo, Zhitao; Lu, Lin; Sharf, Andrei; Xia, Yang; Deussen, Oliver; Chen, Baoquan; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
With the growing popularity of 3D printing, different shape classes such as fibers and hair have been shown, driving research toward class-specific solutions. Among them, 3D trees are an important class, consisting of unique structures, characteristics and botanical features. Nevertheless, trees are an especially challenging case for 3D manufacturing. They typically consist of non-volumetric patch leaves, an extreme amount of small detail often below printable resolution and are often physically weak to be self-sustainable. We introduce a novel 3D tree printability method which optimizes trees through a set of geometry modifications for manufacturing purposes. Our key idea is to formulate tree modifications as a minimal constrained set which accounts for the visual appearance of the model and its structural soundness. To handle non-printable fine details, our method modifies the tree shape by gradually abstracting details of visible parts while reducing details of non-visible parts. To guarantee structural soundness and to increase strength and stability, our algorithm incorporates a physical analysis and adjusts the tree topology and geometry accordingly while adhering to allometric rules. Our results show a variety of tree species with different complexity that are physically sound and correctly printed within reasonable time. The printed trees are correct in terms of their allometry and of high visual quality, which makes them suitable for various applications in the realm of outdoor design, modeling and manufacturing.
Albero: A Visual Analytics Approach for Probabilistic Weather Forecasting
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Diehl, Alexandra; Pelorosso, Leandro; Delrieux, Claudio; Matkovic, Kresimir; Ruiz, Juan; Gröller, M. Eduard; Bruckner, Stefan; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
Probabilistic weather forecasts are amongst the most popular ways to quantify numerical forecast uncertainties. The analog regression method can quantify uncertainties and express them as probabilities. The method comprises the analysis of errors from a large database of past forecasts generated with a specific numerical model and observational data. Current visualization tools based on this method are essentially automated and provide limited analysis capabilities. In this paper, we propose a novel approach that breaks down the automatic process using the experience and knowledge of the users and creates a new interactive visual workflow. Our approach allows forecasters to study probabilistic forecasts, their inner analogs and observations, their associated spatial errors, and additional statistical information by means of coordinated and linked views. We designed the presented solution following a participatory methodology together with domain experts. Several meteorologists with different backgrounds validated the approach. Two case studies illustrate the capabilities of our solution. It successfully facilitates the analysis of uncertainty and systematic model biases for improved decision-making and process-quality measurements.
Rib-reinforced Shell Structure
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Li, Wei; Zheng, Anzong; You, Lihua; Yang, Xiaosong; Zhang, Jianjun; Liu, Ligang; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
Shell structures are extensively used in engineering due to their efficient load-carrying capacity relative to material volume. However, large-span shells require additional supporting structures to strengthen fragile regions. The problem of designing optimal stiffeners is therefore becoming a major challenge for shell applications. To address it, we propose a computational framework to design and optimize rib layout on arbitrary shell to improve the overall structural stiffness and mechanical performance. The essential of our method is to place ribs along the principal stress lines which reflect paths of material continuity and indicates trajectories of internal forces. Given a surface and user-specified external loads, we perform a Finite Element Analysis. Using the resulting principal stress field, we generate a quad-mesh whose edges align with this cross field. Then we extract an initial rib network from the quad-mesh. After simplifying rib network by removing ribs with little contribution, we perform a rib flow optimization which allows ribs to swing on surface to further adjust rib distribution. Finally, we optimize rib cross-section to maximally reduce material usage while achieving certain structural stiffness requirements. We demonstrate that our rib-reinforced shell structures achieve good static performances. And experimental results by 3D printed objects show the effectiveness of our method.
Group-Theme Recoloring for Multi-Image Color Consistency
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Nguyen, Rang M. H.; Price, Brian; Cohen, Scott; Brown, Michael S.; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
Modifying the colors of an image is a fundamental editing task with a wide range of methods available. Manipulating multiple images to share similar colors is more challenging, with limited tools available. Methods such as color transfer are effective in making an image share similar colors with a target image; however, color transfer is not suitable for modifying multiple images. Approaches for color consistency for photo collections give good results when the photo collection contains similar scene content, but are not applicable for general input images. To address these gaps, we propose an application framework for achieving color consistency for multi-image input. Our framework derives a group color theme from the input images' individual color palettes and uses this group color theme to recolor the image collection. This group-theme recoloring provides an effective way to ensure color consistency among multiple images and naturally lends itself to the inclusion of an additional external color theme. We detail our group-theme recoloring approach and demonstrate its effectiveness on a number of examples.
A Unified Cloth Untangling Framework Through Discrete Collision Detection
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Ye, Juntao; Ma, Guanghui; Jiang, Liguo; Chen, Lan; Li, Jituo; Xiong, Gang; Zhang, Xiaopeng; Tang, Min; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
We present an efficient and stable framework, called Unified Intersection Resolver (UIR), for cloth simulation systems where not only impending collisions but also pre-existing penetrations often arise. These two types of collisions are handled in a unified manner, by detecting edge-face intersections first and then forming penetration stencils to be resolved iteratively. A stencil is a quadruple of vertices and it reveals either a vertex-face or an edge-edge collision event happened. Each quadruple also implicitly defines a collision normal, through which the four stencil vertices can be relocated, so that the corresponding edge-face intersection disappear.We deduce three different ways, i.e., from predefined surface orientation, from history data and from global intersection analysis, to determine the collision normals of these stencils robustly. Multiple stencils that constitute a penetration region are processed simultaneously to eliminate penetrations. Cloth trapped in pinched environmental objects can be handled easily within our framework. We highlight its robustness by a number of challenging experiments involving collisions.
Exploring Online Learners' Interactive Dynamics by Visually Analyzing Their Time-anchored Comments
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Sung, Ching-Ying; Huang, Xun-Yi; Shen, Yicong; Cherng, Fu-Yin; Lin, Wen-Chieh; Wang, Hao-Chuan; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
MOOCs (Massive Open Online Courses) are increasingly prevalent as an online educational resource open to everyone and have attracted hundreds of thousands learners enrolling these online courses. At such scale, there is potentially rich information of learners' behaviors embedded in the interactions between learners and videos that may help instructors and content producers adjust the instructions and refine the online courses. However, the lack of tools to visualize information from interactive data, including messages left to the videos at particular timestamps as well as the temporal variations of learners' online participation and perceived experience, has prevented people from gaining more insights from video-watching logs. In this paper, we focus on extracting and visualizing useful information from time-anchored comments that learners left to specific time points of the videos when watching them. Timestamps as a kind of metadata of messages can be useful to recover the interactive dynamics of learners occurring around the videos. Therefore, we present a visualization system to analyze and categorize time-anchored comments based on topics and content types. Our system integrates visualization methods of temporal text data, namely ToPIN and ThemeRiver, which can help people understand the quality and quantity of online learners' feedback and their states of learning. To evaluate the proposed system, we visualized time-anchored commenting data from two online course videos, and conducted two user studies participated by course instructors and third-party educational evaluators. The results validate the usefulness of the approach and show how the quantitative and qualitative visualizations can be used to gain interesting insights around learners' online learning behaviors.
Semi-Automatic Conversion of 3D Shape into Flat-Foldable Polygonal Model
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Miyamoto, Emi; Endo, Yuki; Kanamori, Yoshihiro; Mitani, Jun; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
This paper presents a method that can convert a given 3D mesh into a flat-foldable model consisting of rigid panels. A previous work proposed a method to assist manual design of a single component of such flat-foldable model, consisting of verticallyconnected side panels as well as horizontal top and bottom panels. Our method semi-automatically generates a more complicated model that approximates the input mesh with multiple convex components. The user specifies the folding direction of each convex component and the fidelity of shape approximation. Given the user inputs, our method optimizes shapes and positions of panels of each convex component in order to make the whole model flat-foldable. The user can check a folding animation of the output model. We demonstrate the e ectiveness of our method by fabricating physical paper prototypes of flat-foldable models.
Regression-Based Landmark Detection on Dynamic Human Models
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Jang, Deok-Kyeong; Lee, Sung-Hee; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
Detecting anatomical landmarks on various human models with dynamic poses remains an important and challenging problem in computer graphics research. We present a novel framework that consists of two-level regressors for finding correlations between human shapes and landmark positions in both body part and holistic scales. To this end, we first develop pose invariant coordinates of landmarks that represent both local and global shape features by using the pose invariant local shape descriptors and their spatial relationships. Our body part-level regression deals with the shape features from only those body parts that correspond to a certain landmark. In order to do this, we develop a method that identifies such body parts per landmark, by using geometric shape dictionary obtained through the bag of features method. Our method is nearly automatic, as it requires human assistance only once to differentiate the left and right sides. The method also shows the prediction accuracy comparable to or better than those of existing methods, with a test data set containing a large variation of human shapes and poses.
Video Shadow Removal Using Spatio-temporal Illumination Transfer
(The Eurographics Association and John Wiley & Sons Ltd., 2016) Zhang, Ling; Zhu, Yao; Liao, Bin; Xiao, Chunxia; Jernej Barbic and Wen-Chieh Lin and Olga Sorkine-Hornung
Shadow removal for videos is an important and challenging vision task. In this paper, we present a novel shadow removal approach for videos captured by free moving cameras using illumination transfer optimization. We first detect the shadows of the input video using interactive fast video matting. Then, based on the shadow detection results, we decompose the input video into overlapped 2D patches, and find the coherent correspondences between the shadow and non-shadow patches via discrete optimization technique built on the patch similarity metric. We finally remove the shadows of the input video sequences using an optimized illumination transfer method, which reasonably recovers the illumination information of the shadow regions and produces spatio-temporal shadow-free videos. We also process the shadow boundaries to make the transition between shadow and non-shadow regions smooth. Compared with previous works, our method can handle videos captured by free moving cameras and achieve better shadow removal results. We validate the effectiveness of the proposed algorithm via a variety of experiments.

Browse

Browsing Volume 36 (2017) by Issue Date

Results Per Page

Sort Options