Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

3D-CariGAN: an end-to-end solution to 3D caricature generation from normal face photos

Ye, Zipeng, Xia, Mengfei, Sun, Yanan, Yi, Ran, Yu, Minjing, Zhang, Juyong, Lai, Yu-Kun ORCID: https://orcid.org/0000-0002-2094-5680 and Liu, Yong-Jin 2021. 3D-CariGAN: an end-to-end solution to 3D caricature generation from normal face photos. IEEE Transactions on Visualization and Computer Graphics 10.1109/TVCG.2021.3126659

[thumbnail of 3DCariGAN_TVCG.pdf]
Preview
PDF - Accepted Post-Print Version
Download (6MB) | Preview

Abstract

Caricature is a kind of artistic style of human faces that attracts considerable attention in entertainment industry. So far a few 3D caricature generation methods exist and all of them require some caricature information (e.g., a caricature sketch or 2D caricature) as input. This kind of input, however, is difficult to provide by non-professional users. In this paper, we propose an end-to-end deep neural network model that generates high-quality 3D caricature directly from a simple normal face photo. The most challenging issue in our system is that the source domain of face photos (characterized by 2D normal faces) is significantly different from the target domain of 3D caricatures (characterized by 3D exaggerated face shapes and texture). To address this challenge, we (1) build a large dataset of 6,100 3D caricature meshes and use it to establish a PCA model in the 3D caricature shape space, (2) reconstruct a 3D normal full head from the input face photo and use its PCA representation in the 3D caricature shape space to set up correspondence between the input photo and 3D caricature shape, and (3) propose a novel character loss and a novel caricature loss based on previous psychological studies on caricatures. Experiments including a novel two-level user study show that our system can generate high-quality 3D caricatures directly from normal face photos.

Item Type: Article
Date Type: Published Online
Status: In Press
Schools: Computer Science & Informatics
Publisher: Institute of Electrical and Electronics Engineers
ISSN: 1077-2626
Date of First Compliant Deposit: 13 November 2021
Date of Acceptance: 6 November 2021
Last Modified: 10 Nov 2022 18:01
URI: https://orca.cardiff.ac.uk/id/eprint/145483

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics