Browsing by Author "Bharaj, Gaurav"

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item
    Generative Landmarks
    (The Eurographics Association, 2021) Ferman, David; Bharaj, Gaurav; Bittner, Jirí and Waldner, Manuela
    We propose a general purpose approach to detect landmarks with improved temporal consistency, and personalization. Most sparse landmark detection methods rely on laborious, manually labelled landmarks, where inconsistency in annotations over a temporal volume leads to sub-optimal landmark learning. Further, high-quality landmarks with personalization is often hard to achieve. We pose landmark detection as an image translation problem. We capture two sets of unpaired marked (with paint) and unmarked videos. We then use a generative adversarial network and cyclic consistency to predict deformations of landmark templates that simulate markers on unmarked images until these images are indistinguishable from ground-truth marked images. Our novel method does not rely on manually labelled priors, is temporally consistent, and image class agnostic - face, and hand landmarks detection examples are shown.
  • Thumbnail Image
    Item
    Practical Face Reconstruction via Differentiable Ray Tracing
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Dib, Abdallah; Bharaj, Gaurav; Ahn, Junghyun; Thébault, Cédric; Gosselin, Philippe; Romeo, Marco; Chevallier, Louis; Mitra, Niloy and Viola, Ivan
    We present a differentiable ray-tracing based novel face reconstruction approach where scene attributes - 3D geometry, reflectance (diffuse, specular and roughness), pose, camera parameters, and scene illumination - are estimated from unconstrained monocular images. The proposed method models scene illumination via a novel, parameterized virtual light stage, which in-conjunction with differentiable ray-tracing, introduces a coarse-to-fine optimization formulation for face reconstruction. Our method can not only handle unconstrained illumination and self-shadows conditions, but also estimates diffuse and specular albedos. To estimate the face attributes consistently and with practical semantics, a two-stage optimization strategy systematically uses a subset of parametric attributes, where subsequent attribute estimations factor those previously estimated. For example, self-shadows estimated during the first stage, later prevent its baking into the personalized diffuse and specular albedos in the second stage. We show the efficacy of our approach in several real-world scenarios, where face attributes can be estimated even under extreme illumination conditions. Ablation studies, analyses and comparisons against several recent state-of-the-art methods show improved accuracy and versatility of our approach. With consistent face attributes reconstruction, our method leads to several style - illumination, albedo, self-shadow - edit and transfer applications, as discussed in the paper.