3D-CariGAN: an end-to-end solution to 3D caricature generation from normal face photos

Ye, Zipeng, Xia, Mengfei, Sun, Yanan, Yi, Ran, Yu, Minjing, Zhang, Juyong, Lai, Yu-Kun ORCID: https://orcid.org/0000-0002-2094-5680 and Liu, Yong-Jin 2023. 3D-CariGAN: an end-to-end solution to 3D caricature generation from normal face photos. IEEE Transactions on Visualization and Computer Graphics 29 (4) , pp. 2203-2210. 10.1109/TVCG.2021.3126659

Preview

PDF - Accepted Post-Print Version Download (6MB) | Preview

Abstract

Caricature is a kind of artistic style of human faces that attracts considerable attention in entertainment industry. So far a few 3D caricature generation methods exist and all of them require some caricature information (e.g., a caricature sketch or 2D caricature) as input. This kind of input, however, is difficult to provide by non-professional users. In this paper, we propose an end-to-end deep neural network model that generates high-quality 3D caricature directly from a simple normal face photo. The most challenging issue in our system is that the source domain of face photos (characterized by 2D normal faces) is significantly different from the target domain of 3D caricatures (characterized by 3D exaggerated face shapes and texture). To address this challenge, we (1) build a large dataset of 6,100 3D caricature meshes and use it to establish a PCA model in the 3D caricature shape space, (2) reconstruct a 3D normal full head from the input face photo and use its PCA representation in the 3D caricature shape space to set up correspondence between the input photo and 3D caricature shape, and (3) propose a novel character loss and a novel caricature loss based on previous psychological studies on caricatures. Experiments including a novel two-level user study show that our system can generate high-quality 3D caricatures directly from normal face photos.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	Institute of Electrical and Electronics Engineers
ISSN:	1077-2626
Date of First Compliant Deposit:	13 November 2021
Date of Acceptance:	6 November 2021
Last Modified:	24 Nov 2024 08:30
URI:	https://orca.cardiff.ac.uk/id/eprint/145483

Actions (repository staff only)

Edit Item