NeRF-editing: geometry editing of neural radiance fields

Yuan, Yu-Jie, Sun, Yang-Tian, Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680, Ma, Yuewen, Jai, Rongfei and Gao, Lin 2022. NeRF-editing: geometry editing of neural radiance fields. Presented at: 2022 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022), New Orleans, LA, USA, 19-24 June 2022. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 10.1109/CVPR52688.2022.01781

Preview

PDF - Accepted Post-Print Version Download (14MB) | Preview

Abstract

Implicit neural rendering, especially Neural Radiance Field (NeRF), has shown great potential in novel view synthesis of a scene. However, current NeRF-based methods cannot enable users to perform user-controlled shape deformation in the scene. While existing works have proposed some approaches to modify the radiance field according to the user's constraints, the modification is limited to color editing or object translation and rotation. In this paper, we propose a method that allows users to perform controllable shape deformation on the implicit representation of the scene, and synthesizes the novel view images of the edited scene without re-training the network. Specifically, we establish a correspondence between the extracted explicit mesh representation and the implicit neural representation of the target scene. Users can first utilize well-developed mesh-based deformation methods to deform the mesh representation of the scene. Our method then utilizes user edits from the mesh representation to bend the camera rays by introducing a tetrahedra mesh as a proxy, obtaining the rendering results of the edited scene. Extensive experiments demonstrate that our framework can achieve ideal editing results not only on synthetic data, but also on real scenes captured by users.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Published Online
Status:	In Press
Schools:	Computer Science & Informatics
Publisher:	IEEE
ISSN:	9781665469470
Funders:	The Royal Society
Date of First Compliant Deposit:	2 April 2022
Date of Acceptance:	2 March 2022
Last Modified:	30 Nov 2022 13:09
URI:	https://orca.cardiff.ac.uk/id/eprint/149030

Actions (repository staff only)

Edit Item