TRIPS: Trilinear Point Splatting for Real-Time Radiance Field Rendering

Simple item page
dc.contributor.authorFranke, Linusen_US
dc.contributor.authorRückert, Dariusen_US
dc.contributor.authorFink, Lauraen_US
dc.contributor.authorStamminger, Marcen_US
dc.contributor.editorBermano, Amit H.en_US
dc.contributor.editorKalogerakis, Evangelosen_US
dc.date.accessioned2024-04-30T09:07:15Z
dc.date.available2024-04-30T09:07:15Z
dc.date.issued2024
dc.description.abstractPoint-based radiance field rendering has demonstrated impressive results for novel view synthesis, offering a compelling blend of rendering quality and computational efficiency. However, also latest approaches in this domain are not without their shortcomings. 3D Gaussian Splatting [KKLD23] struggles when tasked with rendering highly detailed scenes, due to blurring and cloudy artifacts. On the other hand, ADOP [RFS22] can accommodate crisper images, but the neural reconstruction network decreases performance, it grapples with temporal instability and it is unable to effectively address large gaps in the point cloud. In this paper, we present TRIPS (Trilinear Point Splatting), an approach that combines ideas from both Gaussian Splatting and ADOP. The fundamental concept behind our novel technique involves rasterizing points into a screen-space image pyramid, with the selection of the pyramid layer determined by the projected point size. This approach allows rendering arbitrarily large points using a single trilinear write. A lightweight neural network is then used to reconstruct a hole-free image including detail beyond splat resolution. Importantly, our render pipeline is entirely differentiable, allowing for automatic optimization of both point sizes and positions. Our evaluation demonstrate that TRIPS surpasses existing state-of-the-art methods in terms of rendering quality while maintaining a real-time frame rate of 60 frames per second on readily available hardware. This performance extends to challenging scenarios, such as scenes featuring intricate geometry, expansive landscapes, and auto-exposed footage. The project page is located at: https://lfranke.github.io/tripsen_US
dc.description.number2
dc.description.sectionheadersReal-time Neural Rendering
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume43
dc.identifier.doi10.1111/cgf.15012
dc.identifier.issn1467-8659
dc.identifier.pages12 pages
dc.identifier.urihttps://doi.org/10.1111/cgf.15012
dc.identifier.urihttps://diglib.eg.org/handle/10.1111/cgf15012
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectCCS Concepts: Computing methodologies -> Rendering; Image-based rendering; Reconstruction
dc.subjectComputing methodologies
dc.subjectRendering
dc.subjectImage
dc.subjectbased rendering
dc.subjectReconstruction
dc.titleTRIPS: Trilinear Point Splatting for Real-Time Radiance Field Renderingen_US
Files
Original bundle
Now showing 1 - 3 of 3
No Thumbnail Available
Name:
v43i2_09_15012.pdf
Size:
26.29 MB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
trips_supplementalpdf.pdf
Size:
103.86 KB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
trips_supplementalvideo.mp4
Size:
147.72 MB
Format:
Video MP4
Download
Collections
43-Issue 2