Real-Time Neural Materials using Block-Compressed Features

Simple item page
dc.contributor.authorWeinreich, Clémenten_US
dc.contributor.authorOliveira, Louis Deen_US
dc.contributor.authorHoudard, Antoineen_US
dc.contributor.authorNader, Georgesen_US
dc.contributor.editorBermano, Amit H.en_US
dc.contributor.editorKalogerakis, Evangelosen_US
dc.date.accessioned2024-04-30T09:07:23Z
dc.date.available2024-04-30T09:07:23Z
dc.date.issued2024
dc.description.abstractNeural materials typically consist of a collection of neural features along with a decoder network. The main challenge in integrating such models in real-time rendering pipelines lies in the large size required to store their features in GPU memory and the complexity of evaluating the network efficiently. We present a neural material model whose features and decoder are specifically designed to be used in real-time rendering pipelines. Our framework leverages hardware-based block compression (BC) texture formats to store the learned features and trains the model to output the material information continuously in space and scale. To achieve this, we organize the features in a block-based manner and emulate BC6 decompression during training, making it possible to export them as regular BC6 textures. This structure allows us to use high resolution features while maintaining a low memory footprint. Consequently, this enhances our model's overall capability, enabling the use of a lightweight and simple decoder architecture that can be evaluated directly in a shader. Furthermore, since the learned features can be decoded continuously, it allows for random uv sampling and smooth transition between scales without needing any subsequent filtering. As a result, our neural material has a small memory footprint, can be decoded extremely fast adding a minimal computational overhead to the rendering pipeline.en_US
dc.description.number2
dc.description.sectionheadersReal-time Neural Rendering
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume43
dc.identifier.doi10.1111/cgf.15013
dc.identifier.issn1467-8659
dc.identifier.pages13 pages
dc.identifier.urihttps://doi.org/10.1111/cgf.15013
dc.identifier.urihttps://diglib.eg.org/handle/10.1111/cgf15013
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.titleReal-Time Neural Materials using Block-Compressed Featuresen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
v43i2_11_15013.pdf
Size:
48.02 MB
Format:
Adobe Portable Document Format
Download
Collections
43-Issue 2