Browsing by Author "Santos, Pedro"

Now showing 1 - 10 of 10
  • No Thumbnail Available
    Item
    AI Based Image Segmentation of Cultural Heritage Objects used for Multi-View Stereo 3D Reconstructions
    (The Eurographics Association, 2023) Kutlu, Hasan; Brucker, Felix; Kallendrusch, Ben; Santos, Pedro; Fellner, Dieter W.; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
    Image segmentation (or masking) finds a very useful use case within 3D reconstruction of cultural heritage objects. The 3D reconstructions can be accelerated, reconstructing the object without any background noise. Conventional segmentation methods can calculate erroneous masks for certain objects and environments, which can lead to errors within the reconstruction: Parts of the 3D reconstruction may be missing or are incorrectly reconstructed, which contradicts adequate archiving. The automated iterative Multi-View Stereo (MVS) scanning process makes it necessary to obtain masks that reconstruct the object in the best possible way, regardless of the environment, the stabilizing mount, the color of the background and the object. In addition, it should not be necessary to tweak the best possible parameters for conventional masking procedures and to create masks manually. State-of-the-art artificial intelligence (AI) segmentation networks will be trained and applied to the MVS scans to verify the behavior of the associated 3D reconstructions and the automated iterative scanning process. In addition, a comparison between different AI segmentation networks and a comparison between conventional masking methods and AI segmentation networks is performed.
  • No Thumbnail Available
    Item
    The Arena: An lndoor Mixed Reality Space
    (The Eurographics Association, 2021) Dias, José Miguel Salles; Bastos, Rafael; Santos, Pedro; Monteiro, Luís; Canhoto, Joaquim; Carriço, Luís and Correia, Nuno and Antunes, Pedro and Jorge, Joaquim
    ln this paper we introduce the Arena, an indoor space for mobile mixed reality interaction. The Arena, includes a new user tracking system appropriate for ARJMR applications anda new Too/kit oriented to the augmented and mixed reality applications developer, the MX Too/kit. This too/kit is defined at a somewhat higher abstraction levei, by hiding from the programmer low levei implementation details and facilitating ARJMR objectoriented programming. The system handles in a uniform way, video input, video output (for head-sefs and monitors), sound aurelisation and Multimodal Human-Computer Interaction in ARJMR, including, tangible interfaces, speech recognition and gesture recognition.
  • Thumbnail Image
    Item
    Automated Classification of Crests on Pottery Sherds Using Pattern Recognition on 2D Images
    (The Eurographics Association, 2022) Ritz, Martin; Santos, Pedro; Fellner, Dieter W.; Ponchio, Federico; Pintus, Ruggero
    Manual classification of artefacts is a labor intensive process. Based on 2D images and 3D scans of - for example - ceramic shards, we developed a pattern recognition algorithm which automatically extracts relief features for each newly recorded object and tries to automate the classification process. Based on characteristics found, previously unknown objects are automatically corelated to already classified objects of a collection exhibiting the greatest similarity. As a result, classes of artefacts form iteratively, which ultimately also corresponds to the overall goal which is the automated classification of entire collections. The greatest challenge in developing our software approach was the heterogeneity of reliefs, and in particular the fact that current machine learning approaches were out of question due to the very limited number of objects per class. This led to the implementation of an analytical approach that is capable of performing a classification based on very few artefacts.
  • Thumbnail Image
    Item
    CultArc3D_mini: Fully Automatic Zero-Button 3D Replicator
    (The Eurographics Association, 2018) Ritz, Martin; Knuth, Martin; Santos, Pedro; Fellner, Dieter W.; Sablatnig, Robert and Wimmer, Michael
    3D scanning and 3D printing are two rapidly evolving domains, both generating results with a huge and growing spectrum of applications. Especially in Cultural Heritage, a massive and increasing amount of objects awaits digitization for various purposes, one of them being replication. Yet, current approaches to optical 3D digitization are semi-automatic at best and require great user effort whenever high quality is desired. With our solution we provide the missing link between both domains, and present a fully automatic 3D object replicator which does not require user interaction. The system consists of our photogrammetric 3D scanner CultArc3D_mini that captures an optimal image set for 3D geometry and texture reconstruction and even optical material properties of objects in only minutes, a conveyor system for automatic object feed-in and -out, a 3D printer, and our sensor-based process flow software that handles every single process step of the complex sequence from image acquisition, sensor-based object transportation, 3D reconstruction involving different kinds of calibrations, to 3D printing of the resulting virtual replica immediately after 3D reconstruction. Typically, one-button machines require the user to start the process by interacting over a user interface. Since positioning and pickup of objects is automatically registered, the only thing left for the user to do is placing an object at the entry and retrieving it from the exit after scanning. Shortly after, the 3D replica can be picked up from the 3D printer. Technically, we created a zero-button 3D replicator that provides high throughput digitization in 3D, requiring only minutes per object, and it is publicly showcased in action at 3IT Berlin.
  • Thumbnail Image
    Item
    End-to-end Color 3D Reproduction of Cultural Heritage Artifacts: Roseninsel Replicas
    (The Eurographics Association, 2019) Domajnko, Matevz; Tanksale, Tejas; Tausch, Reimar; Ritz, Martin; Knuth, Martin; Santos, Pedro; Fellner, Dieter W.; Rizvic, Selma and Rodriguez Echavarria, Karina
    Planning exhibitions of cultural artifacts is always challenging. Artifacts can be very sensitive to the environment and therefore their display can be risky. One way to circumvent this is to build replicas of these artifacts. Here, 3D digitization and reproduction, either physical via 3D printing or virtual, using computer graphics, can be the method of choice. For this use case we present a workflow, from photogrammetric acquisition in challenging environments to representation of the acquired 3D models in different ways, such as online visualization and color 3D printed replicas. This work can also be seen as a first step towards establishing a workflow for full color end-to-end reproduction of artifacts. Our workflow was applied on cultural artifacts found around the ''Roseninsel'' (Rose Island), an island in Lake Starnberg (Bavaria), in collaboration with the Bavarian State Archaeological Collection in Munich. We demonstrate the results of the end-to-end reproduction workflow leading to virtual replicas (online 3D visualization, virtual and augmented reality) and physical replicas (3D printed objects). In addition, we discuss potential optimizations and briefly present an improved state-of-the-art 3D digitization system for fully autonomous acquisition of geometry and colors of cultural heritage objects.
  • Thumbnail Image
    Item
    Fully Automatic Mechanical Scan Range Extension and Signal to Noise Optimization of a Lens-Shifted Structured Light System
    (The Eurographics Association, 2021) Kutlu, Hasan; Ritz, Martin; Santos, Pedro; Fellner, Dieter W.; Hulusic, Vedad and Chalmers, Alan
    Digitization of cultural heritage is of growing importance, both for its preservation for coming generations in the face of looming dangers of natural decay or intentional destruction, and current generations, that increasingly have access to virtual cultural heritage for interactive exploring or scientific analysis. These goals can only be achieved by 3D replicas at reasonable quality and resolution, to come as close as possible to the original. This brings about several challenges to overcome. The challenge of digitizing huge numbers of artefacts is addressed by CultLab3D, the first fully automatic 3D digitization system. Another challenge is the size of objects, as each digitization system is designed for a certain optimum measurement range, leaving which results in loss of quality. Due to optical and mechanical constraints, most systems are not able to faithfully reconstruct objects under a certain size limit in their full geometric detail. Historic coins are one good example, where the deterioration of the surface structure in most cases has progressed to a degree that it not even is perceptible through the fingernail. This challenge is addressed by a modular extension of CultLab3D, the MesoScanner, which is a structured light system that breaks limits in depth resolution through a mechanical lens-shifting extension, allowing physically shifting of fringe patterns on top of the well-known multi-period phase shift method. This is where this work adds two major improvements: First, the signal to noise ratio and thus reconstruction quality has been improved significantly through several algorithmic processing steps. Second, the physical limitation of the measurement range was removed using a 2D actuator steering the object mount, thus allowing for a measurement range at theoretically arbitrary size. This opens up the fully automatic handling of two scenarios: Complete digitization of objects exceeding the measurement range, and unsupervised digitization of large collections of small objects in one run.
  • No Thumbnail Available
    Item
    Image Manipulation through Gestures
    (The Eurographics Association, 2021) Dias, José; Nande, Pedro; Santos, Pedro; Barata, Nuno; Correia, André; Marcos, Adérito and Mendonça, Ana and Leitão, Miguel and Costa, António and Jorge, Joaquim
    In this work we present a novel free-hand gesture user interface based on detecting the trajectory of fiducial markers attached to the user's fingers and pulse, able to interact a sequence of images of a digital video piece. The model adopted for the video representation, is based in its decomposition in a sequence of frames, or filmstrip. Totally sensor-less and cable-less interfaces, provide the means for a user to intuitively interact through gestures with the filmstrip, within the framework of an Augmented Virtuality usage scenario. By simply gesturing, users are able to select at random, drag, release, delete or zoom image frames, browse the filmstrip at a controlled user-defined rate and issue start, end, stop and play commands to better control the digital video sequence. A fixed video camera monitors the user interaction through gesturing of the mentioned fiducial markers. This scheme enables the system to simplify the more complex problem of marker-less free-hand gesture tracking. Once the markers are detected and recognized in real- time by the computer vision layer, the system obtains the 3D pose (position and orientation) of the marker centres in relation to a virtual camera reference frame, whose mathematical model matches the real video camera. We are specifically interested in obtaining the pose of the left and right hand pulses, left and right thumb, and left and right hand index. By projecting the positions of these poses in the 2D visualization window, simple topological analysis based in the study of the kinematics evolution of distances and angles, can be implemented, enabling gesture recognition and the activation of system functions and, subsequently, of specific gesture-based user interaction for a given active functionality. This interaction will affect the shape, scale factor, position and visualisation of scene objects, that is, of filmstrip frames. For the computer vision layer, our system adopts AR Toolkit, a C/Open GL-based open source library that uses accurate vision based tracking methods to determine the virtual camera pose information through the detection in real-time of fiducial markers. The graphical output is implemented with C++/Open GL. Our proposed system is general, in the sense that it can interact with any filmstrip, obtained ''a priori'' from a digital video source.
  • Thumbnail Image
    Item
    Lossless Compression of Multi-View Cultural Heritage Image Data
    (The Eurographics Association, 2019) von Buelow, Max; Guthe, Stefan; Ritz, Martin; Santos, Pedro; Fellner, Dieter W.; Rizvic, Selma and Rodriguez Echavarria, Karina
    Photometric multi-view 3D geometry reconstruction and material capture are important techniques for cultural heritage digitalization. Capturing images of artifacts with high resolution and high dynamic range and the possibility to store them losslessly enables future proof application of this data. As the images tend to consume immense amounts of storage, compression is essential for long time archiving. In this paper, we present a lossless image compression approach for multi-view and material reconstruction datasets with a strong focus on data created from cultural heritage digitalization. Our approach achieves compression rates of 2:1 compared against an uncompressed representation and 1.24:1 when compared against Gzip.
  • No Thumbnail Available
    Item
    MIXDesign, Tangible Mixed Reality for Architectural Design
    (The Eurographics Association, 2022) Salles Dias, J. Miguel; Santos, Pedro; Bastos, Rafael; Monteiro, Luis; Silvestre, Rui; Diniz, Nancy; Xavier Pueyo; Manuel Próspero dos Santos; Luiz Velho
    MIXDesign, provides a tangible Mixed-Reality system oriented towards tasks in Architectural Design, in severa! usage scenarios, such as Conceptual Design, Client Brief or even Architectural Design education. With MIXDesign, an architect can intuitively interact with a real scale model of the design, in normal working settings, where he can observe an enhanced version of the scale model, with JD virtual objects registered to the real ones. The architect is then able to use intuitive tangible interfaces, such as a paddle, to choose menu options, select a JD virtual object, transport a virtual object within the scale model surroundings and geometrically transform an object (by rotation or scaling). MIXDesign provides a platform for testing new design concepts while seamlessly transporting the Architect from Reality (RE) to Augmented Reality (AR) and then through Augmented Virtuality (AV), towards afull Virtual Environment (VE), and back, where he can perceive andjudge both the virtual and the real models, interactively and in real-time.
  • Thumbnail Image
    Item
    Segmentation-Based Near-Lossless Compression of Multi-View Cultural Heritage Image Data
    (The Eurographics Association, 2020) Buelow, Max von; Tausch, Reimar; Knauthe, Volker; Wirth, Tristan; Guthe, Stefan; Santos, Pedro; Fellner, Dieter W.; Spagnuolo, Michela and Melero, Francisco Javier
    Cultural heritage preservation using photometric approaches received increasing significance in the past years. Capturing of these datasets is usually done with high-end cameras at maximum image resolution enabling high quality reconstruction results while leading to immense storage consumptions. In order to maintain archives of these datasets, compression is mandatory for storing them at reasonable cost. In this paper, we make use of the mostly static background of the capturing environment that does not directly contribute information to 3d reconstruction algorithms and therefore may be approximated using lossy techniques. We use a superpixel and figure-ground segmentation based near-lossless image compression algorithm that transparently decides if regions are relevant for later photometric reconstructions. This makes sure that the actual artifact or structured background parts are compressed with lossless techniques. Our algorithm achieves compression rates compared to the PNG image compression standard ranging from 1:2 to 1:4 depending on the artifact size.