Learning semantic-aware disentangled representation for flexible 3D human body editing

Sun, Xiaokun, Feng, Qiao, Li, Xiongzheng, Zhang, Jinsong, Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680, Yang, Jingyu and Li, Kun 2023. Learning semantic-aware disentangled representation for flexible 3D human body editing. Presented at: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, Canada, 18-22 June 2023. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, pp. 16985-16994. 10.1109/CVPR52729.2023.01629

Preview

PDF - Accepted Post-Print Version Download (4MB) | Preview

Abstract

3D human body representation learning has received increasing attention in recent years. However, existing works cannot flexibly, controllably and accurately represent human bodies, limited by coarse semantics and unsatisfactory representation capability, particularly in the absence of supervised data. In this paper, we propose a human body representation with fine-grained semantics and high reconstruction-accuracy in an unsupervised setting. Specifically, we establish a correspondence between latent vectors and geometric measures of body parts by designing a part-aware skeleton-separated decoupling strategy, which facilitates controllable editing of human bodies by modifying the corresponding latent codes. With the help of a bone-guided auto-encoder and an orientation-adaptive weighting strategy, our representation can be trained in an unsupervised manner. With the geometrically meaningful latent space, it can be applied to a wide range of applications, from human body editing to latent code interpolation and shape style transfer. Experimental results on public datasets demonstrate the accurate reconstruction and flexible editing abilities of the proposed method. The code will be available at http://cic.tju.edu.cn/faculty/likun/projects/SemanticHuman.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Published Online
Status:	Published
Schools:	Schools > Computer Science & Informatics
Publisher:	IEEE
ISBN:	979-8-3503-0130-4
ISSN:	1063-6919
Date of First Compliant Deposit:	20 April 2023
Date of Acceptance:	27 February 2023
Last Modified:	19 Mar 2025 14:06
URI:	https://orca.cardiff.ac.uk/id/eprint/158969

Actions (repository staff only)

Edit Item