Learning to Predict Image-based Rendering Artifacts with Respect to a Hidden Reference Image

Simple item page
dc.contributor.authorBemana, Mojtabaen_US
dc.contributor.authorKeinert, Joachimen_US
dc.contributor.authorMyszkowski, Karolen_US
dc.contributor.authorBätz, Michelen_US
dc.contributor.authorZiegler, Matthiasen_US
dc.contributor.authorSeidel, Hans-Peteren_US
dc.contributor.authorRitschel, Tobiasen_US
dc.contributor.editorLee, Jehee and Theobalt, Christian and Wetzstein, Gordonen_US
dc.date.accessioned2019-10-14T05:09:42Z
dc.date.available2019-10-14T05:09:42Z
dc.date.issued2019
dc.description.abstractImage metrics predict the perceived per-pixel difference between a reference image and its degraded (e. g., re-rendered) version. In several important applications, the reference image is not available and image metrics cannot be applied. We devise a neural network architecture and training procedure that allows predicting the MSE, SSIM or VGG16 image difference from the distorted image alone while the reference is not observed. This is enabled by two insights: The first is to inject sufficiently many un-distorted natural image patches, which can be found in arbitrary amounts and are known to have no perceivable difference to themselves. This avoids false positives. The second is to balance the learning, where it is carefully made sure that all image errors are equally likely, avoiding false negatives. Surprisingly, we observe that the resulting no-reference metric, subjectively, can even perform better than the reference-based one, as it had to become robust against mis-alignments. We evaluate the effectiveness of our approach in an image-based rendering context, both quantitatively and qualitatively. Finally, we demonstrate two applications which reduce light field capture time and provide guidance for interactive depth adjustment.en_US
dc.description.number7
dc.description.sectionheadersImage Based Rendering
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume38
dc.identifier.doi10.1111/cgf.13862
dc.identifier.issn1467-8659
dc.identifier.pages579-589
dc.identifier.urihttps://doi.org/10.1111/cgf.13862
dc.identifier.urihttps://diglib.eg.org:443/handle/10.1111/cgf13862
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.titleLearning to Predict Image-based Rendering Artifacts with Respect to a Hidden Reference Imageen_US
Files
Collections
38-Issue 7