Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

PVSeRF: joint pixel-, voxel- and surface-aligned radiance field for single-image novel view synthesis

Yu, Xianggang, Tang, Jiapeng, Qin, Yipeng ORCID: https://orcid.org/0000-0002-1551-9126, Li, Chenghong, Han, Xiaoguang, Bao, Linchao and Cui, Shuguang 2022. PVSeRF: joint pixel-, voxel- and surface-aligned radiance field for single-image novel view synthesis. Presented at: 30th ACM International Conference on Multimedia (ACMMM 2022), Lisbon, Portugal, 10 - 14 October 2022. Proceedings ACMMM 2022 : 30th ACM International Conference on Multimedia. New York: ACM, 10.1145/3503161.3547893

[thumbnail of mm2022_PVSeRF_ready.pdf]
Preview
PDF - Published Version
Download (1MB) | Preview

Abstract

We present PVSeRF, a learning framework that reconstructs neural radiance fields from single-view RGB images, for novel view synthesis. Previous solutions, such as pixelNeRF [68], rely only on pixel-aligned features and suffer from feature ambiguity issues. As a result, they struggle with the disentanglement of geometry and appearance, leading to implausible geometries and blurry results. To address this challenge, we propose to incorporate explicit geometry reasoning and combine it with pixel-aligned features for radiance field prediction. Specifically, in addition to pixel-aligned features, we further constrain the radiance field learning to be conditioned on i) voxel-aligned features learned from a coarse volumetric grid and ii) fine surface-aligned features extracted from a regressed point cloud. We show that the introduction of such geometry-aware features helps to achieve a better disentanglement between appearance and geometry, i.e. recovering more accurate geometries and synthesizing higher quality images of novel views. Extensive experiments against state-of-the-art methods on ShapeNet benchmarks demonstrate the superiority of our approach for single-image novel view synthesis.

Item Type: Conference or Workshop Item (Paper)
Status: In Press
Schools: Computer Science & Informatics
Publisher: ACM
ISBN: 978-1-4503-9203-7/22/1
Date of First Compliant Deposit: 25 July 2022
Last Modified: 10 Nov 2022 11:37
URI: https://orca.cardiff.ac.uk/id/eprint/151284

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics