Neural Semantic Surface Maps
| dc.contributor.author | Morreale, Luca | en_US |
| dc.contributor.author | Aigerman, Noam | en_US |
| dc.contributor.author | Kim, Vladimir G. | en_US |
| dc.contributor.author | Mitra, Niloy J. | en_US |
| dc.contributor.editor | Bermano, Amit H. | en_US |
| dc.contributor.editor | Kalogerakis, Evangelos | en_US |
| dc.date.accessioned | 2024-04-30T09:06:45Z | |
| dc.date.available | 2024-04-30T09:06:45Z | |
| dc.date.issued | 2024 | |
| dc.description.abstract | We present an automated technique for computing a map between two genus-zero shapes, which matches semantically corresponding regions to one another. Lack of annotated data prohibits direct inference of 3D semantic priors; instead, current state-of-the-art methods predominantly optimize geometric properties or require varying amounts of manual annotation. To overcome the lack of annotated training data, we distill semantic matches from pre-trained vision models: our method renders the pair of untextured 3D shapes from multiple viewpoints; the resulting renders are then fed into an off-the-shelf imagematching strategy that leverages a pre-trained visual model to produce feature points. This yields semantic correspondences, which are projected back to the 3D shapes, producing a raw matching that is inaccurate and inconsistent across different viewpoints. These correspondences are refined and distilled into an inter-surface map by a dedicated optimization scheme, which promotes bijectivity and continuity of the output map. We illustrate that our approach can generate semantic surface-to-surface maps, eliminating manual annotations or any 3D training data requirement. Furthermore, it proves effective in scenarios with high semantic complexity, where objects are non-isometrically related, as well as in situations where they are nearly isometric. | en_US |
| dc.description.number | 2 | |
| dc.description.sectionheaders | Shape and Scene Understanding | |
| dc.description.seriesinformation | Computer Graphics Forum | |
| dc.description.volume | 43 | |
| dc.identifier.doi | 10.1111/cgf.15005 | |
| dc.identifier.issn | 1467-8659 | |
| dc.identifier.pages | 13 pages | |
| dc.identifier.uri | https://doi.org/10.1111/cgf.15005 | |
| dc.identifier.uri | https://diglib.eg.org/handle/10.1111/cgf15005 | |
| dc.publisher | The Eurographics Association and John Wiley & Sons Ltd. | en_US |
| dc.subject | CCS Concepts: Computing methodologies -> Shape analysis; Mesh geometry models; Feature selection | |
| dc.subject | Computing methodologies | |
| dc.subject | Shape analysis | |
| dc.subject | Mesh geometry models | |
| dc.subject | Feature selection | |
| dc.title | Neural Semantic Surface Maps | en_US |