Learning to infer inner-body under clothing from monocular video

Li, Xiongzheng, Huang, Jing, Zhang, Jinsong, Sun, Xiaokun, Xuan, Haibiao, Lai, Yu-Kun ORCID: https://orcid.org/0000-0002-2094-5680, Xie, Yingdi, Yang, Jingyu and Li, Kun 2023. Learning to infer inner-body under clothing from monocular video. IEEE Transactions on Visualization and Computer Graphics 29 (12) , pp. 5083-5096. 10.1109/TVCG.2022.3202240

Preview

PDF - Accepted Post-Print Version Download (9MB) | Preview

Abstract

Accurately estimating the human inner-body under clothing is very important for body measurement, virtual try-on and VR/AR applications. In this paper, we propose the first method to allow everyone to easily reconstruct their own 3D inner-body under daily clothing from a self-captured video with the mean reconstruction error of 0.73 cm within 15 s. This avoids privacy concerns arising from nudity or minimal clothing. Specifically, we propose a novel two-stage framework with a Semantic-guided Undressing Network (SUNet) and an Intra-Inter Transformer Network (IITNet). SUNet learns semantically related body features to alleviate the complexity and uncertainty of directly estimating 3D inner-bodies under clothing. IITNet reconstructs the 3D inner-body model by making full use of intra-frame and inter-frame information, which addresses the misalignment of inconsistent poses in different frames. Experimental results on both public datasets and our collected dataset demonstrate the effectiveness of the proposed method. The code and dataset is available for research purposes at http://cic.tju.edu.cn/faculty/likun/projects/Inner-Body .

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	Institute of Electrical and Electronics Engineers
ISSN:	1077-2626
Date of First Compliant Deposit:	12 October 2022
Date of Acceptance:	17 August 2022
Last Modified:	18 Dec 2023 18:06
URI:	https://orca.cardiff.ac.uk/id/eprint/153316

Actions (repository staff only)

Edit Item