Guo, Yudong, Zhang, Juyong, Chen, Yihua, Cai, Hongrui, Huang, Zhangjin and Deng, Bailin ORCID: https://orcid.org/0000-0002-0158-7670 2021. Real-time face view correction for front-facing cameras. Computational Visual Media 7 , pp. 437-452. 10.1007/s41095-021-0215-y |
Preview |
PDF
- Published Version
Available under License Creative Commons Attribution. Download (3MB) | Preview |
Abstract
Face view is particularly important in person-to-person communication. Disparity between the camera location and the face orientation can result in undesirable facial appearances of the participants during video conferencing. This phenomenon becomes particularly notable on devices where the front-facing camera is placed at unconventional locations such as below the display or within the keyboard. In this paper, we takes the video stream from a single RGB camera as input, and generates a video stream that emulates the view from a virtual camera at a designated location. The most challenging issue of this problem is that the corrected view often needs out-of-plane head rotations. To address this challenge, we reconstruct 3D face shape and re-render it into synthesized frames according to the virtual camera location. To output the corrected video stream with natural appearance in real-time, we propose several novel techniques including accurate eyebrow reconstruction, high-quality blending between corrected face image and background, and a template-based 3D reconstruction of glasses. Our system works well for different lighting conditions and skin tones, and is able to handle users wearing glasses. Extensive experiments and user studies demonstrate that our proposed method can achieve high-quality results.
Item Type: | Article |
---|---|
Date Type: | Publication |
Status: | Published |
Schools: | Computer Science & Informatics |
Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science Q Science > QA Mathematics > QA76 Computer software |
Publisher: | Springer |
ISSN: | 2096-0433 |
Date of First Compliant Deposit: | 6 April 2021 |
Date of Acceptance: | 25 February 2021 |
Last Modified: | 05 May 2023 08:40 |
URI: | https://orca.cardiff.ac.uk/id/eprint/139613 |
Actions (repository staff only)
Edit Item |