Learning Style Compatibility Between Objects in a Real-World 3D Asset Database

dc.contributor.authorLiu, Yifanen_US
dc.contributor.authorTang, Ruolanen_US
dc.contributor.authorRitchie, Danielen_US
dc.contributor.editorLee, Jehee and Theobalt, Christian and Wetzstein, Gordonen_US
dc.date.accessioned2019-10-14T05:10:34Z
dc.date.available2019-10-14T05:10:34Z
dc.date.issued2019
dc.description.abstractLarge 3D asset databases are critical for designing virtual worlds, and using them effectively requires techniques for efficient querying and navigation. One important form of query is search by style compatibility: given a query object, find others that would be visually compatible if used in the same scene. In this paper, we present a scalable, learning-based approach for solving this problem which is designed for use with real-world 3D asset databases; we conduct experiments on 121 3D asset packages containing around 4000 3D objects from the Unity Asset Store. By leveraging the structure of the object packages, we introduce a technique to synthesize training labels for metric learning that work as well as human labels. These labels can grow exponentially with the number of objects, allowing our approach to scale to large real-world 3D asset databases without the need for expensive human training labels. We use these synthetic training labels in a metric learning model that analyzes the in-engine rendered appearance of an object—-combining geometry, material, and texture-whereas prior work considers only object geometry, or disjoint geometry and texture features. Through an ablation experiment, we find that using this representation yields better results than using renders which lack texture, materiality, or both.en_US
dc.description.number7
dc.description.sectionheadersModeling Interfaces
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume38
dc.identifier.doi10.1111/cgf.13879
dc.identifier.issn1467-8659
dc.identifier.pages775-784
dc.identifier.urihttps://doi.org/10.1111/cgf.13879
dc.identifier.urihttps://diglib.eg.org:443/handle/10.1111/cgf13879
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.subjectI.3.8 [Computer Graphics]
dc.subjectApplications
dc.subject3D Model Search
dc.subjectI.5.1 [Pattern Recognition]
dc.subjectModels
dc.subjectNeural Networks
dc.subjectI.2.10 [Artificial Intelligence]
dc.subjectVision and Scene Understanding
dc.subjectPerceptual Reasoning
dc.titleLearning Style Compatibility Between Objects in a Real-World 3D Asset Databaseen_US
Files
Collections