Efficient multi-view inverse rendering using a hybrid differentiable rendering method

Zhu, Xiangyang, Pan, Yiling, Deng, Bailin ORCID: https://orcid.org/0000-0002-0158-7670 and Wang, Bin 2023. Efficient multi-view inverse rendering using a hybrid differentiable rendering method. Presented at: International Joint Conferences on Artificial Intelligence, Macao, 19-25 August 2023. Published in: Elkind, E. ed. Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence, pp. 1849-1857. 10.24963/ijcai.2023/205

Preview	PDF - Accepted Post-Print Version Download (14MB) | Preview
	PDF - Supplemental Material Download (806kB)

Abstract

Recovering the shape and appearance of real-world objects from natural 2D images is a long-standing and challenging inverse rendering problem. In this paper, we introduce a novel hybrid differentiable rendering method to efficiently reconstruct the 3D geometry and reflectance of a scene from multi-view images captured by conventional hand-held cameras. Our method follows an analysis-by-synthesis approach and consists of two phases. In the initialization phase, we use traditional SfM and MVS methods to reconstruct a virtual scene roughly matching the real scene. Then in the optimization phase, we adopt a hybrid approach to refine the geometry and reflectance, where the geometry is first optimized using an approximate differentiable rendering method, and the reflectance is optimized afterward using a physically-based differentiable rendering method. Our hybrid approach combines the efficiency of approximate methods with the high-quality results of physically-based methods. Extensive experiments on synthetic and real data demonstrate that our method can produce reconstructions with similar or higher quality than state-of-the-art methods while being more efficient.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Publication
Status:	Published
Schools:	Schools > Computer Science & Informatics
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science Q Science > QA Mathematics > QA76 Computer software
Publisher:	International Joint Conferences on Artificial Intelligence
Date of First Compliant Deposit:	3 June 2023
Date of Acceptance:	19 April 2023
Last Modified:	19 Mar 2025 13:01
URI:	https://orca.cardiff.ac.uk/id/eprint/160152

Actions (repository staff only)

Edit Item