Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

PISE: person image synthesis and editing with decoupled GAN

Zhang, Jinsong, Li, Kun, Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680 and Yang, Jingyu 2021. PISE: person image synthesis and editing with decoupled GAN. Presented at: IEEE/CVF Conference on Computer Vision and Pattern Recognition, Virtual, 19-25 June 2021. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[thumbnail of PISE_CVPR2021.pdf]
Preview
PDF - Accepted Post-Print Version
Download (1MB) | Preview

Abstract

Person image synthesis, e.g., pose transfer, is a challenging problem due to large variation and occlusion. Existing methods have difficulties predicting reasonable invisible regions and fail to decouple the shape and style of clothing, which limits their applications on person image editing. In this paper, we propose PISE, a novel two-stage generative model for Person Image Synthesis and Editing, which is able to generate realistic person images with desired poses, textures, or semantic layouts. For human pose transfer, we first synthesize a human parsing map aligned with the target pose to represent the shape of clothing by a parsing generator, and then generate the final image by an image generator. To decouple the shape and style of clothing, we propose joint global and local per-region encoding and normalization to predict the reasonable style of clothing for invisible regions. We also propose spatial-aware normalization to retain the spatial context relationship in the source image. The results of qualitative and quantitative experiments demonstrate the superiority of our model on human pose transfer. Besides, the results of texture transfer and region editing show that our model can be applied to person image editing. The code is available for research purposes at https://github.com/Zhangjinso/PISE.

Item Type: Conference or Workshop Item (Paper)
Date Type: Published Online
Status: In Press
Schools: Computer Science & Informatics
ISBN: 978-1-6654-4510-8
ISSN: 2575-7075
Related URLs:
Date of First Compliant Deposit: 20 April 2021
Date of Acceptance: 3 March 2021
Last Modified: 09 Nov 2022 10:45
URI: https://orca.cardiff.ac.uk/id/eprint/140565

Citation Data

Cited 16 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics