Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

High-resolution face swapping via latent semantics disentanglement

Xu, Yangyang, Deng, Bailin ORCID: https://orcid.org/0000-0002-0158-7670, Wang, Junle, Jing, Yanqing, Pan, Jia and He, Shengfeng 2022. High-resolution face swapping via latent semantics disentanglement. Presented at: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, Louisiana, United States, 21 - 24 June 2022. Proceedings IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE,

[thumbnail of FaceSwapping.pdf]
Preview
PDF - Accepted Post-Print Version
Download (7MB) | Preview
[thumbnail of Supplementary.pdf]
Preview
PDF - Supplemental Material
Download (3MB) | Preview

Abstract

We present a novel high-resolution face swapping method using the inherent prior knowledge of a pre-trained GAN model. Although previous research can leverage generative priors to produce high-resolution results, their quality can suffer from the entangled semantics of the latent space. We explicitly disentangle the latent semantics by utilizing the progressive nature of the generator, deriving structure attributes from the shallow layers and appearance attributes from the deeper ones. Identity and pose information within the structure attributes are further separated by introducing a landmark-driven structure transfer latent direction. The disentangled latent code produces rich generative features that incorporate feature blending to produce a plausible swapping result. We further extend our method to video face swapping by enforcing two spatio-temporal constraints on the latent space and the image space. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art image/video face swapping methods in terms of hallucination quality and consistency. Code can be found at: https://github.com/cnnlstm/FSLSD_HiRes.

Item Type: Conference or Workshop Item (Paper)
Status: In Press
Schools: Computer Science & Informatics
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Q Science > QA Mathematics > QA76 Computer software
Additional Information: © 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works
Publisher: IEEE
Date of First Compliant Deposit: 26 March 2022
Date of Acceptance: 2 March 2022
Last Modified: 10 Nov 2022 10:55
URI: https://orca.cardiff.ac.uk/id/eprint/148646

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics