PVSeRF: joint pixel-, voxel- and surface-aligned radiance field for single-image novel view synthesis

Yu, Xianggang, Tang, Jiapeng, Qin, Yipeng ORCID: https://orcid.org/0000-0002-1551-9126, Li, Chenghong, Han, Xiaoguang, Bao, Linchao and Cui, Shuguang 2022. PVSeRF: joint pixel-, voxel- and surface-aligned radiance field for single-image novel view synthesis. Presented at: 30th ACM International Conference on Multimedia (ACMMM 2022), Lisbon, Portugal, 10 - 14 October 2022. Proceedings ACMMM 2022 : 30th ACM International Conference on Multimedia. New York: ACM, 10.1145/3503161.3547893

Preview

PDF - Published Version Download (1MB) | Preview

Abstract

We present PVSeRF, a learning framework that reconstructs neural radiance fields from single-view RGB images, for novel view synthesis. Previous solutions, such as pixelNeRF [68], rely only on pixel-aligned features and suffer from feature ambiguity issues. As a result, they struggle with the disentanglement of geometry and appearance, leading to implausible geometries and blurry results. To address this challenge, we propose to incorporate explicit geometry reasoning and combine it with pixel-aligned features for radiance field prediction. Specifically, in addition to pixel-aligned features, we further constrain the radiance field learning to be conditioned on i) voxel-aligned features learned from a coarse volumetric grid and ii) fine surface-aligned features extracted from a regressed point cloud. We show that the introduction of such geometry-aware features helps to achieve a better disentanglement between appearance and geometry, i.e. recovering more accurate geometries and synthesizing higher quality images of novel views. Extensive experiments against state-of-the-art methods on ShapeNet benchmarks demonstrate the superiority of our approach for single-image novel view synthesis.

Item Type:	Conference or Workshop Item (Paper)
Status:	In Press
Schools:	Computer Science & Informatics
Publisher:	ACM
ISBN:	978-1-4503-9203-7/22/1
Date of First Compliant Deposit:	25 July 2022
Last Modified:	31 Mar 2023 18:46
URI:	https://orca.cardiff.ac.uk/id/eprint/151284

Citation Data

Cited 1 time in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item