Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

Real-time face view correction for front-facing cameras

Guo, Yudong, Zhang, Juyong, Chen, Yihua, Cai, Hongrui, Huang, Zhangjin and Deng, Bailin ORCID: https://orcid.org/0000-0002-0158-7670 2021. Real-time face view correction for front-facing cameras. Computational Visual Media 7 , pp. 437-452. 10.1007/s41095-021-0215-y

[thumbnail of Guo2021_Article_Real-timeFaceViewCorrectionFor.pdf]
Preview
PDF - Published Version
Available under License Creative Commons Attribution.

Download (3MB) | Preview

Abstract

Face view is particularly important in person-to-person communication. Disparity between the camera location and the face orientation can result in undesirable facial appearances of the participants during video conferencing. This phenomenon becomes particularly notable on devices where the front-facing camera is placed at unconventional locations such as below the display or within the keyboard. In this paper, we takes the video stream from a single RGB camera as input, and generates a video stream that emulates the view from a virtual camera at a designated location. The most challenging issue of this problem is that the corrected view often needs out-of-plane head rotations. To address this challenge, we reconstruct 3D face shape and re-render it into synthesized frames according to the virtual camera location. To output the corrected video stream with natural appearance in real-time, we propose several novel techniques including accurate eyebrow reconstruction, high-quality blending between corrected face image and background, and a template-based 3D reconstruction of glasses. Our system works well for different lighting conditions and skin tones, and is able to handle users wearing glasses. Extensive experiments and user studies demonstrate that our proposed method can achieve high-quality results.

Item Type: Article
Date Type: Publication
Status: Published
Schools: Computer Science & Informatics
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Q Science > QA Mathematics > QA76 Computer software
Publisher: Springer
ISSN: 2096-0433
Date of First Compliant Deposit: 6 April 2021
Date of Acceptance: 25 February 2021
Last Modified: 05 May 2023 08:40
URI: https://orca.cardiff.ac.uk/id/eprint/139613

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics