Lightweight photometric stereo for facial details recovery

Wang, Xueying, Guo, Yudong, Deng, Bailin ORCID: https://orcid.org/0000-0002-0158-7670 and Zhang, Juyong 2020. Lightweight photometric stereo for facial details recovery. Presented at: Conference on Computer Vision and Pattern Recognition (CVPR 2020), Seattle, Washington, USA, 16-18 June 2020. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp. 737-746. 10.1109/CVPR42600.2020.00082

Preview	PDF - Supplemental Material Download (7MB) | Preview
Preview	PDF - Accepted Post-Print Version Download (8MB) | Preview

Abstract

Recently, 3D face reconstruction from a single image has achieved great success with the help of deep learning and shape prior knowledge, but they often fail to produce accurate geometry details. On the other hand, photometric stereo methods can recover reliable geometry details, but require dense inputs and need to solve a complex optimization problem. In this paper, we present a lightweight strategy that only requires sparse inputs or even a single image to recover high-fidelity face shapes with images captured under near-field lights. To this end, we construct a dataset containing 84 different subjects with 29 expressions under 3 different lights. Data augmentation is applied to enrich the data in terms of diversity in identity, lighting, expression, etc. With this constructed dataset, we propose a novel neural network specially designed for photometric stereo based 3D face reconstruction. Extensive experiments and comparisons demonstrate that our method can generate high-quality reconstruction results with one to three facial images captured under near-field lights. Our full framework is available at https://github.com/Juyong/FacePSNet.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Publication
Status:	Published
Schools:	Computer Science & Informatics
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science Q Science > QA Mathematics > QA76 Computer software
Publisher:	IEEE
ISBN:	978-1-7281-7168-5
Date of First Compliant Deposit:	30 March 2020
Date of Acceptance:	27 February 2020
Last Modified:	07 Nov 2022 09:55
URI:	https://orca.cardiff.ac.uk/id/eprint/130641

Citation Data

Cited 11 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item