From motion capture to interactive virtual worlds: towards unconstrained motion-capture algorithms for real-time performance-driven character animation

[meta data] [files:

Rhodin.pdf

]

Rhodin, Helge

Tracking Hands in Action for Gesture-based Computer Input

[meta data] [files:

Sridhar.pdf

]

Srinath, Sridhar

Position-based Skin Deformations for Interactive Character Animation

[meta data] [files:

AburummanNadine_Thesis.pdf

]

Abu Rumman, Nadine

Interactive, Example-driven Synthesis and Manipulation of Visual Media

[meta data] [files:

thesis_neu_compressed.pdf

]

Reinert, Bernhard

Computational Methods for Capture and Reproduction of Photorealistic Surface Appearance

[meta data] [files:

isbn9789526070476.pdf

]

Aittala, Miika

Exploring Appearance and Style in Heterogeneous Visual Content

[meta data] [files:

thesis_garces.pdf

]

Garces, Elena

Browse

Now showing 1 - 21 of 21

Accurate 3D-reconstruction and -navigation for high-precision minimal-invasive interventions
(2016-02-03) El Hakimi, Wissam
The current lateral skull base surgery is largely invasive since it requires wide exposure and direct visualization of anatomical landmarks to avoid damaging critical structures. A multi-port approach aiming to reduce such invasiveness has been recently investigated. Thereby three canals are drilled from the skull surface to the surgical region of interest: the first canal for the instrument, the second for the endoscope, and the third for material removal or an additional instrument. The transition to minimal invasive approaches in the lateral skull base surgery requires sub-millimeter accuracy and high outcome predictability, which results in high requirements for the image acquisition as well as for the navigation. Computed tomography (CT) is a non-invasive imaging technique allowing the visualization of the internal patient organs. Planning optimal drill channels based on patient-specific models requires high-accurate three-dimensional (3D) CT images. This thesis focuses on the reconstruction of high quality CT volumes. Therefore, two conventional imaging systems are investigated: spiral CT scanners and C-arm cone-beam CT (CBCT) systems. Spiral CT scanners acquire volumes with typically anisotropic resolution, i.e. the voxel spacing in the slice-selection-direction is larger than the in-the-plane spacing. A new super-resolution reconstruction approach is proposed to recover images with high isotropic resolution from two orthogonal low-resolution CT volumes. C-arm CBCT systems offers CT-like 3D imaging capabilities while being appropriate for interventional suites. A main drawback of these systems is the commonly encountered CT artifacts due to several limitations in the imaging system, such as the mechanical inaccuracies. This thesis contributes new methods to enhance the CBCT reconstruction quality by addressing two main reconstruction artifacts: the misalignment artifacts caused by mechanical inaccuracies, and the metal-artifacts caused by the presence of metal objects in the scanned region. CBCT scanners are appropriate for intra-operative image-guided navigation. For instance, they can be used to control the drill process based on intra-operatively acquired 2D fluoroscopic images. For a successful navigation, accurate estimate of C-arm pose relative to the patient anatomy and the associated surgical plan is required. A new algorithm has been developed to fulfill this task with high-precision. The performance of the introduced methods is demonstrated on simulated and real data.
Scalable Comparative Visualization
(2016-06-28) Schmidt, Johanna
The comparison of two or more objects is getting an increasingly important task in data analysis. Visualization systems successively have to move from representing one phenomenon to allowing users to analyze several datasets at once. Visualization systems can support the users in several ways. Firstly, comparison tasks can be supported in a very intuitive way by allowing users to place objects that should be compared in an appropriate context. Secondly, visualization systems can explicitly compute differences among the datasets and present the results to the user. In comparative visualization, researchers are working on new approaches for computer-supported techniques that provide data comparison functionality. Techniques from this research field can be used to compare two objects with each other, but often reach their limits if a multitude of objects (i.e., 100 or more) have to be compared. Large data collections that contain a lot of individual, but related, datasets with slightly different characteristics can be called ensembles. The individual datasets being part of an ensemble are called the ensemble members. Ensembles have been created in the simulation domain, especially for weather and climate research, for already quite some time. These domains were greatly driving the development of ensemble visualization techniques. Due to the availability of affordable computing resources and the multitude of different analysis algorithms (e.g., for segmentation), other domains nowadays also face similar problems. All together, this shows a great need for ensemble visualization techniques in various domains. Ensembles can either be analyzed in a feature-based or in a location-based way. In the case of a location-based analysis, the ensemble members are compared based on certain spatial data positions of interest. For such an analysis, local selection and analysis techniques for ensembles are needed. In the course of this thesis different visual analytics techniques for the comparative visualization of datasets have been researched. A special focus has been set on providing scalable techniques, which makes them also suitable for ensemble datasets. The proposed techniques operate on different dataset types in 2D and 3D. In the first part of the thesis, a visual analytics approach for the analysis of 2D image datasets is introduced. The technique analyzes localized differences in 2D images. The approach not only identifies differences in the data, but also provides a technique to quickly find out what the differences are, and judge upon the underlying data. This way patterns can be found in the data, and outliers can be identified very quickly. As a second part of the thesis, a scalable application for the comparison of several similar 3D mesh datasets is described. Such meshes may be, for example, created by point-cloud reconstruction algorithms, using different parameter settings. Similar to the proposed technique for the comparison of 2D images, this application is also scalable to a large number of individual datasets. The application enables the automatic comparison of the meshes, searches interesting regions in the data, and allows users to also concentrate on local regions of interest. The analysis of the local regions is in this case done in 3D. The application provides the possibility to arrange local regions in a parallel coordinates plot. The regions are represented by the axes in the plot, and the input meshes are depicted as polylines. This way it can be very quickly spotted whether meshes produce good/bad results in a certain local region. In the third and last part of the thesis, a technique for the interactive analysis of local regions in a volume ensemble dataset is introduced. Users can pick regions of interest, and these regions can be arranged in a graph according to their similarity. The graph can then be used to detect similar regions with a similar data distribution within the ensemble, and to compare individual ensemble members against the rest of the ensemble. All proposed techniques and applications have been tested with real-world datasets from different domains. The results clearly show the usefulness of the techniques for the comparative analysis of ensembles.
Guided Interactive Volume Editing in Medicine
(2016-06-28) Karimov, Alexey
Various medical imaging techniques, such as Computed Tomography, Magnetic Resonance Imaging, Ultrasonic Imaging, are now gold standards in the diagnosis of different diseases. The diagnostic process can be greatly improved with the aid of automatic and interactive analysis tools, which, however, require certain prerequisites in order to operate. Such analysis tools can, for example, be used for pathology assessment, various standardized measurements, treatment and operation planning. One of the major requirements of such tools is the segmentation mask of an object-of-interest. However, the segmentation of medical data remains subject to errors and mistakes. Often, physicians have to manually inspect and correct the segmentation results, as (semi-)automatic techniques do not immediately satisfy the required quality. To this end, interactive segmentation editing is an integral part of medical image processing and visualization. In this thesis, we present three advanced segmentation-editing techniques. They are focused on simple interaction operations that allow the user to edit segmentation masks quickly and effectively. These operations are based on a topology-aware representation that captures structural features of the segmentation mask of the object-of-interest. Firstly, in order to streamline the correction process, we classify segmentation defects according to underlying structural features and propose a correction procedure for each type of defect. This alleviates users from manually applying the proper editing operations, but the segmentation defects still have to be located by users. Secondly, we extend the basic editing process by detecting regions that potentially contain defects. With subsequently suggested correction scenarios, users are hereby immediately able to correct a specific defect, instead of manually searching for defects beforehand. For each suggested correction scenario, we automatically determine the corresponding region of the respective defect in the segmentation mask and propose a suitable correction operation. In order to create the correction scenarios, we detect dissimilarities within the data values of the mask and then classify them according to the characteristics of a certain type of defect. Potential findings are presented with a glyph-based visualization that facilitates users to interactively explore the suggested correction scenarios on different levels-of-detail. As a consequence, our approach even offers users the possibility to fine-tune the chosen correction scenario instead of directly manipulating the segmentation mask, which is a time-consuming and cumbersome task. Third and finally, we guide users through the multitude of suggested correction scenarios of the entire correction process. After statistically evaluating all suggested correction scenarios, we rank them according to their significance of dissimilarities, offering fine-grained editing capabilities at a user-specified level-of-detail. As we visually convey this ranking in a radial layout, users can easily spot and select the most (or the least) dissimilar correction scenario, which improves the segmentation mask mostly towards the desired result. All techniques proposed within this thesis have been evaluated by collaborating radiologists. We assessed the usability, interaction aspects, the accuracy of the results and the expenditure of time of the entire correction process. The outcome of the assessment showed that our guided volume editing not only leads to acceptable segmentation results with only a few interaction steps, but also is applicable to various application scenarios.
Opacity Optimization and Inertial Particles in Flow Visualization
(2016-06-30) Günther, Tobias
Vector field visualization is a major discipline of scientific visualization that helps to push the frontiers of research in fluid mechanics, medicine, biology, astrophysics and many more. In particular, vector field visualization is concerned with the discovery of relationships in possibly large and complex vector fields, which serve as general descriptors of air and fluid flows, magnetic fields and dynamical systems. The visualization community found a number of different ways to assist in the analysis and exploration of these fields. Two major classes of approaches are the so-called geometry-based and feature-based / topology-based techniques. The first and second part of this thesis introduce techniques that reside in these two classes, respectively. The third part of the thesis addresses the analysis of inertial particles, i.e., finite-sized objects carried by fluid flows. When it comes to 3D flow visualization, we often encounter occlusion problems when displaying dense sets of lines or multiple surfaces. A vital aspect is the careful selection of the primitives that best communicate the relevant features in a data set. In the first part of the thesis, we present optimization-based approaches that adjust the opacity of lines and surfaces to strive for a balance between the presentation of relevant information and occlusion avoidance. The second part of the thesis is dedicated to novel rendering techniques for the visualization of unsteady flows. For this, we will apply techniques from light transport in heterogeneous participating media to the unbiased rendering of Lagrangian scalar fields, namely finite-time Lyapunov exponents. Further, we propose a new class of vortex definitions for flows that are induced by rotating mechanical parts, such as stirring devices, hydrocyclones, centrifugal pumps or ventilators. In the third part of this thesis, we introduce inertial particles as a new application domain to the flow visualization community. Recent research in flow visualization focused on the analysis of massless particles. However, in many application scenarios, the mass of particles and their resulting inertia are essential, such as when sand particles interact with aircraft. The governing ODE of even simple inertial flow models is up to seven dimensional, which makes feature extraction a challenging task. We abstract the description of mass-dependent particle trajectories and apply existing flow visualization methods to the mass-dependent case. In particular, we extract and visualize integral geometry, study the vortical motion and separation behavior of inertial particles, extend traditional vector field topology to the inertial case and present a new approach to the source inversion problem, i.e., the recovery of the source of dispersed pollutants. We demonstrate the usefulness of our methods by applying them to a variety of synthetic and real-world data sets.
Efficient methods for physically-based rendering of participating media
(2016-02-02) Elek, Oskar
This thesis proposes several novel methods for realistic synthesis of images containing participating media. This is a challenging problem, due to the multitude and complexity of ways how light interacts with participating media, but also an important one, since such media are ubiquitous in our environment and therefore are one of the main constituents of its appearance. The main paradigm we follow is designing efficient methods that provide their user with an interactive feedback, but are still physically plausible. The presented contributions have varying degrees of specialisation and, in a loose connection to that, their resulting efficiency. First, the screen-space scattering algorithm simulates scattering in homogeneous media, such as fog and water, as a fast image filtering process. Next, the amortised photon mapping method focuses on rendering clouds as arguably one of the most difficult media due to their high scattering anisotropy. Here, interactivity is achieved through adapting to certain conditions specific to clouds. A generalisation of this approach is principal-ordinates propagation, which tackles a much wider class of heterogeneous media. The resulting method can handle almost arbitrary optical properties in such media, thanks to a custom finite-element propagation scheme. Finally, spectral ray differentials aim at an efficient reconstruction of chromatic dispersion phenomena, which occur in transparent media such as water, glass and gemstones. This method is based on analytical ray differentiation and as such can be incorporated to any ray-based rendering framework, increasing the efficiency of reproducing dispersion by about an order of magnitude. All four proposed methods achieve efficiency primarily by utilising high-level mathematical abstractions, building on the understanding of the underlying physical principles that guide light transport. The methods have also been designed around simple data structures, allowing high execution parallelism and removing the need to rely on any sort of preprocessing. Thanks to these properties, the presented work is not only suitable for interactively computing light transport in participating media, but also allows dynamic changes to the simulated environment, all while maintaining high levels of visual realism.
Depth, Shading, and Stylization in Stereoscopic Cinematography
(2016-02-26) Templin, Krzysztof
Due to the constantly increasing focus of the entertainment industry on stereoscopic imaging, techniques and tools that enable precise control over the depth impression and help to overcome limitations of the current stereoscopic hardware are gaining in importance. In this dissertation, we address selected problems encountered during stereoscopic content production, with a particular focus on stereoscopic cinema. First, we consider abrupt changes of depth, such as those induced by cuts in films. We derive a model predicting the time the visual system needs to adapt to such changes and propose how to employ this model for film cut optimization. Second, we tackle the issue of discrepancies between the two views of a stereoscopic image due to view-dependent shading of glossy materials. The suggested solution eliminates discomfort caused by non-matching specular highlights while preserving the perception of gloss. Last, we deal with the problem of filmgrainmanagement in stereoscopic productions and propose a new method for film grain application that reconciles visual comfort with the idea of medium-scene separation.
Guided qualitative and quantitative analysis of cardiac 4D PC-MRI blood flow data
(2016) Köhler, Benjamin
The genesis and progression of cardiovascular diseases (CVDs) depend on various factors. A better comprehension of patient-specific blood flow hemodynamics has great potential to increase their diagnosis, support treatment decision-making and provide a realistic forecast of such pathologies, facilitating a better implementation of preventative measures. Four-dimensional phase-contrast magnetic resonance imaging (4D PC-MRI) gained increasing importance and clinical attention in recent years. It is a non-invasive imaging modality that allows for time-resolved, three-dimensional measurement of blood flow information. The resulting 4D grid data, which contain vectors that represent the blood flow direction and velocity, are of limited spatio-temporal resolution and suffer from multiple artifacts, making complex image processing methods a prerequisite. Qualitative data analysis aims to depict the course of the blood flow with emphasis on specific flow patterns, such as vortex flow, which can be an indicator for different cardiovascular diseases. For this purpose, flow visualization techniques can be adapted to the cardiac context. Quantitative data analysis facilitates assessment of, e.g., the cardiac function by evaluating stroke volumes, heart valve performances by evaluating percentaged back flows, and fluid-vessel wall interactions by evaluating wall shear stress. This thesis proposes both qualitative and quantitative data evaluation methods, embedded in a developed software prototype with a guided workflow. A semi-automatic extraction of vortex flow is presented that is based on the line predicates methodology and preserves visually appealing path lines with long and continuous courses. It was tailored towards our targeted user group: Radiologists focused on the cardiovascular system and cardiologists. The extracted path lines were used to establish an overview visualization of aortic vortex flow and to adapt the speed of videos so that the display vortical flow behavior is enhanced. Vortices were grouped into single entities (clustering) and subsequently analyzed according to different criteria that describe properties, such as their rotation direction and elongation. Based on this classification, a simplifying glyph visualization was established. Moreover, this thesis addresses an improved quantification of the flow rate-based measures, such as stroke volumes, which are prone to errors especially in case of pathologic vortex flow. A robust procedure is presented that analyzes multiple, systematically generated configurations of required measuring planes and evaluates the resulting sample distributions. Additionally, the flow rate calculation is influenced by the dynamic morphology. Therefore, a semi-automatic extraction of corresponding motion information was established and incorporated in an adapted quantification.
From Atoms to Cells: Interactive and Illustrative Visualization of Digitally Reproduced Lifeforms
(2016-11-30) Le Muzic, Mathieu
Macromolecules, such as proteins, are the building blocks of the machinery of life, and therefore are essential to the comprehension of physiological processes. In physiology, illustrations and animations are often utilized as a mean of communication because they can easily be understood with little background knowledge. However, their realization requires numerous months of manual work, which is both expensive and time consuming. Computational biology experts produce everyday large amount of data that is publicly available and that contains valuable information about the structure and also the function of these macromolecules. Instead of relying on manual work to generate illustrative visualizations of the cell biology, we envision a solution that would utilize all the data already available in order to streamline the creation process. In this thesis are presented several contributions that aim at enabling our vision. First, a novel GPU-based rendering pipeline that allows interactive visualization of realistic molecular datasets comprising up to hundreds of millions of macromolecules. The rendering pipeline is embedded into a popular game engine and well known computer graphics optimizations were adapted to support this type of data, such as level-of-detail, instancing and occlusion queries. Secondly, a new method for authoring cutaway views and improving spatial exploration of crowded molecular landscapes. The system relies on the use of clipping objects that are manually placed in the scene and on visibility equalizers that allows fine tuning of the visibility of each species present in the scene. Agent-based modeling produces trajectory data that can also be combined with structural information in order to animate these landscapes. The snapshots of the trajectories are often played in fast-forward to shorten the length of the visualized sequences, which also renders potentially interesting events occurring at a higher temporal resolution invisible. The third contribution is a solution to visualize time-lapse of agent-based simulations that also reveals hidden information that is only observable at higher temporal resolutions. And finally, a new type of particle-system that utilize quantitative models as input and generate missing spatial information to enable the visualization of molecular trajectories and interactions. The particle-system produces a similar visual output as traditional agent-based modeling tools for a much lower computational footprint and allows interactive changing of the simulation parameters, which was not achievable with previous methods.
Tracking, Correcting and Absorbing Water Surface Waves
(2016-07-15) Bojsen-Hansen, Morten
Computer graphics is an extremely exciting field for two reasons. On the one hand, there is a healthy injection of pragmatism coming from the visual effects industry that want robust algorithms that work so they can produce results at an increasingly frantic pace. On the other hand, they must always try to push the envelope and achieve the impossible to wow their audiences in the next blockbuster, which means that the industry has not succumb to conservatism, and there is plenty of room to try out new and crazy ideas if there is a chance that it will pan into something useful. Water simulation has been in visual effects for decades, however it still remains extremely challenging because of its high computational cost and difficult art-directability. The work in this thesis tries to address some of these difficulties. Specifically, we make the following three novel contributions to the state-of-the-art in water simulation for visual effects.
A computational appearance fabrication framework and derived applications
(ETH Zurich, 2015) Papas, Marios
Traditionally, control over the appearance of objects in the real world was performed manually. Understanding how some physical property of an object would affect its appearance was achieved primarily through trial and error. This procedure could be lengthy and cumbersome, depending on the complexity of the effect of physical properties on appearance and the duration of each fabrication cycle. Precise control of how light interacts with materials has many applications in arts, architecture, industrial design, and engineering. With the recent achievements in geometry retrieval and computational fabrication we are now able to precisely control and replicate the geometry of real-world objects. On the other hand, computational appearance fabrication is still in its infancy. In this thesis we lay he foundation for a general computational appearance fabrication framework, and we demonstrate a range of applications that benefit from it. We present various instances of our framework and detail the design of the corresponding components, such as: forward and backward appearance models, measurement, and fabrication. These framework instances help in understanding and controlling the appearance of three general classes of materials: homogeneous participating media (such as wax and milk), specular surfaces (such as lenses), and granular media (such as sugar and snow). More specifically we show how we can precisely measure, control, and fabricate the real-world appearance of homogeneous translucent materials, how to computationally design and fabricate steganographic lenses, and finally we present a fast appearance model for accurately simulating the appearance of granular media.
Online Surface Reconstruction From Unorganized Point Clouds With Integrated Texture Mapping
(2015-11-11) Vierjahn, Tom
Digital representations of the real world are becoming more and more important for different application domains. Individual objects, excavation sites or even complete cities can be digitized with today’s technology so that they can, for instance, be preserved as digital cultural heritage, be used as a basis for map creation, or be integrated into virtual environments for mission planning during emergency or disaster response tasks. Robust and efficient surface reconstruction algorithms are inevitable for these applications. Surface-reconstructing growing neural gas (sgng) presented in this dissertation constitutes an artificial neural network that takes a set of sample points lying on an object’s surface as an input and iteratively constructs a triangle mesh representing the original object’s surface. It starts with an initial approximation that gets continuously refined. At any time during execution, sgng instantly incorporates any modifications of the input data into the reconstruction. If images are available that are registered to the input points, sgng assigns suitable textures to the constructed triangles. The number of noticeable occlusion artifacts is reduced to a minimum by learning the required visibility information from the input data. Sgng is based on a family of closely related artificial neural networks. These are presented in detail and illustrated by pseudocode and examples. Sgng is derived according to a careful analysis of these prior approaches. Results of an extensive evaluation indicate that sgng improves significantly upon its predecessors and that it can compete with other state-of-the-art reconstruction algorithms.
Biomechanical Models for Human-Computer Interaction
(2016-11-04) Bachynskyi, Myroslav
Post-desktop user interfaces, such as smartphones, tablets, interactive tabletops, public displays and mid-air interfaces, already are a ubiquitous part of everyday human life, or have the potential to be. One of the key features of these interfaces is the reduced number or even absence of input movement constraints imposed by a device form-factor. This freedom is advantageous for users, allowing them to interact with computers using more natural limb movements; however, it is a source of 4 issues for research and design of post-desktop interfaces which make traditional analysis methods inefficient: the new movement space is orders of magnitude larger than the one analyzed for traditional desktops; the existing knowledge on post-desktop input methods is sparse and sporadic; the movement space is non-uniform with respect to performance; and traditional methods are ineffective or inefficient in tackling physical ergonomics pitfalls in post-desktop interfaces. These issues lead to the research problem of efficient assessment, analysis and design methods for high-throughput ergonomic post-desktop interfaces. To solve this research problem and support researchers and designers, this thesis proposes efficient experiment- and model-based assessment methods for post-desktop user interfaces. We achieve this through the following contributions: - adopt optical motion capture and biomechanical simulation for HCI experiments as a versatile source of both performance and ergonomics data describing an input method; - identify applicability limits of the method for a range of HCI tasks; - validate the method outputs against ground truth recordings in typical HCI setting; - demonstrate the added value of the method in analysis of performance and ergonomics of touchscreen devices; and - summarize performance and ergonomics of a movement space through a clustering of physiological data. The proposed method successfully deals with the 4 above-mentioned issues of post-desktop input. The efficiency of the methods makes it possible to effectively tackle the issue of large post-desktop movement spaces both at early design stages (through a generic model of a movement space) as well as at later design stages (through user studies). The method provides rich data on physical ergonomics (joint angles and moments, muscle forces and activations, energy expenditure and fatigue), making it possible to solve the issue of ergonomics pitfalls. Additionally, the method provides performance data (speed, accuracy and throughput) which can be related to the physiological data to solve the issue of non-uniformity of movement space. In our adaptation the method does not require experimenters to have specialized expertise, thus making it accessible to a wide range of researchers and designers and contributing towards the solution of the issue of post-desktop knowledge sparsity.
Perceptual modeling for stereoscopic 3D
(2016-11-04) Kellnhofer, Petr
Virtual and Augmented Reality applications typically rely on both stereoscopic presentation and involve intensive object and observer motion. A combination of high dynamic range and stereoscopic capabilities become popular for consumer displays, and is a desirable functionality of head mounted displays to come. The thesis is focused on complex interactions between all these visual cues on digital displays. The first part investigates challenges of the stereoscopic 3D and motion combination. We consider an interaction between the continuous motion presented as discrete frames. Then, we discuss a disparity processing for accurate reproduction of objects moving in the depth direction. Finally, we investigate the depth perception as a function of motion parallax and eye fixation changes by means of saccadic motion. The second part focuses on the role of high dynamic range imaging for stereoscopic displays. We go beyond the current display capabilities by considering the full perceivable luminance range and we simulate the real world experience in such adaptation conditions. In particular, we address the problems of disparity retargeting across such wide luminance ranges and reflective/refractive surface rendering.
Interactive Deformation of Virtual Paper
(2016) Schreck, Camille
Although paper is a very common material in our every-day life, it can hardly be found in 3D virtual environments. Indeed, due to its fibrous structure, paper material exhibits complex deformations and sound behavior which are hard to reproduce efficiently using standard methods. Most notably, the deforming surface of a sheet of paper is constantly isometric to its 2D pattern, and may be crumpled or torn leading to sharp and fine geometrical features. During deformation, paper material also has very characteristic sound, which highly depends on its complex shape. In this thesis, we propose to combine usual physics-based simulation with new procedu- ral, geometric methods in order to take advantage of prior knowledge to efficiently model the geometry and the sound of a deforming sheet of paper. Our goals are to reproduce a plausible behavior of paper rather than an entirely physically accurate one in order to enable a user to interactively deform and create animation of virtual paper.
Spectral Methods for Multimodal Data Analysis
(2016-11-01) Kovnatsky, Artiom
Spectral methods have proven themselves as an important and versatile tool in a wide range of problems in the fields of computer graphics, machine learning, pattern recognition, and computer vision, where many important problems boil down to constructing a Laplacian operator and finding a few of its eigenvalues and eigenfunctions. Classical examples include the computation of diffusion distances on manifolds in computer graphics, Laplacian eigenmaps, and spectral clustering in machine learning. In many cases, one has to deal with multiple data spaces simultaneously. For example, clustering multimedia data in machine learning applications involves various modalities or “views” (e.g., text and images), and finding correspondence between shapes in computer graphics problems is an operation performed between two or more modalities. In this thesis, we develop a generalization of spectral methods to deal with multiple data spaces and apply them to problems from the domains of computer graphics, machine learning, and image processing. Our main construction is based on simultaneous diagonalization of Laplacian operators. We present an efficient numerical technique for computing joint approximate eigenvectors of two or more Laplacians in challenging noisy scenarios, which also appears to be the first general non-smooth manifold optimization method. Finally, we use the relation between joint approximate diagonalizability and approximate commutativity of operators to define a structural similarity measure for images. We use this measure to perform structure-preserving color manipulations of a given image. To the best of our knowledge, the original contributions of this work are the following: 1 Introduction of joint diagonalization methods to the fields of machine learning, computer vision, pattern recognition, image processing, and graphics; 2 Formulation of the coupled approximate diagonalization problem that extends the joint diagonalization to cases with no bijective correspondence between the domains, and its application in a wide range of problems in the above fields; 3 Introduction of a new structural similarity measure of images based on the approximate commutativity of their respective Laplacians, and its application in image processing problems such as color-to-gray conversion, colors adaptation for color-blind viewers, gamut mapping, and multispectral image fusion; 4 Development of Manifold Alternating Direction Method of Multipliers (MADMM), the first general method for non-smooth optimization with manifold constraints, and its applications to several problems.
From motion capture to interactive virtual worlds: towards unconstrained motion-capture algorithms for real-time performance-driven character animation
(2016-12-15) Rhodin, Helge
This dissertation takes performance-driven character animation as a representative application and advances motion capture algorithms and animation methods to meet its high demands. Existing approaches have either coarse resolution and restricted capture volume, require expensive and complex multi-camera systems, or use intrusive suits and controllers. For motion capture, set-up time is reduced using fewer cameras, accuracy is increased despite occlusions and general environments, initialization is automated, and free roaming is enabled by egocentric cameras. For animation, increased robustness enables the use of low-cost sensors input, custom control gesture definition is guided to support novice users, and animation expressiveness is increased. The important contributions are: 1) an analytic and differentiable visibility model for pose optimization under strong occlusions, 2) a volumetric contour model for automatic actor initialization in general scenes, 3) a method to annotate and augment image-pose databases automatically, 4) the utilization of unlabeled examples for character control, and 5) the generalization and disambiguation of cyclical gestures for faithful character animation. In summary, the whole process of human motion capture, processing, and application to animation is advanced. These advances on the state of the art have the potential to improve many interactive applications, within and outside virtual reality.
Tracking Hands in Action for Gesture-based Computer Input
(2016-12-16) Srinath, Sridhar
This thesis introduces new methods for markerless tracking of the full articulated motion of hands and for informing the design of gesture-based computer input. Emerging devices such as smartwatches or virtual/augmented reality glasses are in need of new input devices for interaction on the move. The highly dexterous human hands could provide an always-on input capability without the actual need to carry a physical device. First, we present novel methods to address the hard computer vision-based hand tracking problem under varying number of cameras, viewpoints, and run-time requirements. Second, we contribute to the design of gesture-based interaction techniques by presenting heuristic and computational approaches. The contributions of this thesis allow users to effectively interact with computers through markerless tracking of hands and objects in desktop, mobile, and egocentric scenarios.
Position-based Skin Deformations for Interactive Character Animation
(2016-03-03) Abu Rumman, Nadine
Skeletal animation is a widely used technique for animating articulated characters, such as humans and animals. In skeleton-based animation, skinning is the process of defining how the geometric surface of the character deforms according to a function of the skeletal poses. One of the fundamental aspects when animating articulated character is the production of flesh-like deformations for the soft tissues when the character is moving. Creating believable and compelling skin deformations is the central challenge of animated feature films, computer games, and interactive applications. Traditionally, the skin deformations are driven by an underlying skeleton. The idea can be formulated with a simple expression that binds the character’s skin mesh with its underlying skeleton, whose bones can be transformed in order to obtain a smooth non-rigid deformation of the surrounding mesh. The deformation of each mesh vertex is computed as a weighted blend of the bones transformations. This technique does not generate realistic deformations and it usually suffers from unsightly artefacts. Moreover, skeleton-based deformation methods are incapable of capturing secondary motion effects, such as volume preservation, skin contact effects and the jiggling behaviors of soft tissues when the character is moving. In contrast, by employing a physically based method into the skinning process, the believability and realism of character motions are highly enhanced. Physics-based simulations manage to bring skeleton-driven deformations beyond the purely kinematic approach by simulating secondary motions. Despite offering such interesting effects, physics-based simulation requires complex and intensive computations, and thus it is usually avoided in interactive applications such as computer games. Furthermore, once the deformation parameters are specified in the simulation, it is difficult to control the actual resulting shape of the character in every animation frame. In this dissertation, we address the problem of creating believable mesh-based skin deformation for soft articulated characters. We present a novel two-layered deformation framework, which is able to mimic the macro-behaviors of the skin and capture secondary effects, such as volume conservation and jiggling. While minimizing the manual post-processing time, our system provides the artist with some level of control over the secondary effects. Our system is practical, relatively easy to implement and fast enough for real-time applications. We also introduce an efficient method for detecting collisions and self-collisions on articulated models, in which we exploit the skeletal nature of the deformation to achieve a good real-time performance. The output of the collision detection algorithm is used to enhance our layered skin deformation with responsive contact handling, and supports contact skin deformation between skin parts.
Interactive, Example-driven Synthesis and Manipulation of Visual Media
(2016-12-02) Reinert, Bernhard
This thesis proposes several novel techniques for interactive, example-driven synthesis and manipulation of visual media. The numerous display devices in our everyday lives make visual media, such as images, videos, or three-dimensional models, easily accessible to a large group of people. Consequently, there is a rising demand for efficient generation of syn- thetic visual content and its manipulation, especially by casual users operating on low-end, mobile devices. Off-the-shelf software supporting such tasks typically requires extensive training and in-depth understanding of the underlying concepts of content acquisition, on the one hand, and runs only on powerful desktop machines, on the other hand, limiting the possibility of artistic media generation to a small group of trained experts with appropriate hardware. Our proposed techniques aim to alleviate these requirements by allowing casual users to synthesize complex, high-quality content in real-time as well as to manipulate it by means of simple, example-driven interactions. First, this thesis discusses a manipulation technique that visualizes an additional level of information, such as importance, on images and three-dimensional surface models by local, non-uniform, and self-intersection-free size manipulations. Second, we propose a technique to automatically arrange and sort collections of images based on the images’ shape and a sparse set of exemplar images that builds on a novel distribution algorithm. Along this line, an extension for higher dimensions such as three-dimensional models is presented and the implications of distributions for lower-dimensional projections are discussed. Further, the spectral properties of the distributions are analyzed and the results are applied for efficient, high-quality image synthesis. Finally, we suggest an algorithm to extract deformable, three- dimensional content from a two-dimensional video leveraging a simple limb representation that the user sketches onto a sparse set of key frames. All methods build on the availability of massively parallel execution hardware, such as graphics processing units (GPUs), nowadays built also into cheap mobile devices. By mathematical abstraction, parallelization, and task distribution our algorithms achieve a high efficiency that allows running our methods in real-time on low-end devices.
Computational Methods for Capture and Reproduction of Photorealistic Surface Appearance
(2016-10-28) Aittala, Miika
This thesis addresses the problem of capturing and reproducing surface material appearance from real-world examples for use in computer graphics applications. Detailed variation of color, shininess and small-scale shape is a critically important factor in visual plausibility of objects in synthetic images. Capturing these properties relies on measuring reflected light under various viewing and illumination conditions. Existing methods typically employ either complex mechanical devices, or heuristics that sacrifice fidelity for simplicity. Consequently, computer graphics practitioners continue to use manual authoring tools. The thesis introduces three methods for capturing visually rich surface appearance descriptors using simple hardware setups and relatively little measurement data. The specific focus is on capturing detailed spatial variation of the reflectance properties, as opposed to angular variation, which is the primary focus of most previous work. We apply tools from modern data science — in particular, principled optimization-based approaches — to disentangle and explain the various reflectance effects in the scarce measurement data. The first method uses a flat panel monitor as a programmable light source, and an SLR camera to observe reflections off the captured surface. The monitor is used to emit Fourier basis function patterns, which are well suited for isolating the reflectance properties of interest, and also exhibit a rich set of mathematical properties that enable computationally efficient interpretation of the data. The other two methods rely on the observation that the spatial variation of many real-world materials is stationary, in the sense that it consists of small elements repeating across the surface. By taking advantage of this redundancy, the methods demonstrate high-quality appearance capture from two photographs, and only a single photograph, respectively. The photographs are acquired using a mobile phone camera. The resulting reflectance descriptors faithfully reproduce the appearance of the surface under novel viewing and illumination conditions. We demonstrate state of the art results among approaches with similar hardware complexity. The descriptors captured by the methods are directly usable in computer graphics applications, including games, film, and virtual and augmented reality.
Exploring Appearance and Style in Heterogeneous Visual Content
(2017-10-19) Garces, Elena
There are multiple ways to capture and represent the visual world; a drawing, a photograph, or a video are a few examples of visual data that are very frequent nowadays. Despite the different nature of each domain, there is a common need to process and edit these data after its production for different purposes. For example, we might want to modify the materials and the illumination of an object in a photograph, or we might want to explore a huge collection of non labeled images. The solutions to these problems mainly depend on the amount of information we have as input: it is not the same to process a plain set of colored pixels, like a photograph, than a scene captured with a 3D laser scan and multiple cameras. Thus, the nature of the visual data will also determine the complexity of the model we can use for processing. In this thesis, we focus on creating alternative representations of the visual content which will facilitate posterior editing and exploration tasks. In particular, we will focus on conventional visual data like pictures, video sequences , and light fields; and we will explore two different aspects or these data, the appearance in real scenes and the style in artistic scenes.

Browse

Recent Submissions

Results Per Page

Sort Options