Image-guided human reconstruction via multi-scale graph transformation networks

Li, Kun, Wen, Hao, Feng, Qiao, Zhang, Yuxiang, Lia, Xiongzheng, Huang, Jing, Yuan, Cunkuan, Lai, Yu-Kun ORCID: https://orcid.org/0000-0002-2094-5680 and Liu, Yebin 2021. Image-guided human reconstruction via multi-scale graph transformation networks. IEEE Transactions on Image Processing 30 , pp. 5239-5251. 10.1109/TIP.2021.3080177

Preview

PDF - Accepted Post-Print Version Download (12MB) | Preview

Abstract

3D human reconstruction from a single image is a challenging problem. Existing methods have difficulties to infer 3D clothed human models with consistent topologies for various poses. In this paper, we propose an efficient and effective method using a hierarchical graph transformation network. To deal with large deformations and avoid distorted geometries, rather than using Euclidean coordinates directly, 3D human shapes are represented by a vertex-based deformation representation that effectively encodes the deformation and copes well with large deformations. To infer a 3D human mesh consistent with the input real image, we also use a perspective projection layer to incorporate perceptual image features into the deformation representation. Our model is easy to train and fast to converge with short test time. Besides, we present the D2Human (Dynamic Detailed Human) dataset, including variously posed 3D human meshes with consistent topologies and rich geometry details, together with the captured color images and SMPL models, which is useful for training and evaluation of deep frameworks, particularly for graph neural networks. Experimental results demonstrate that our method achieves more plausible and complete 3D human reconstruction from a single image, compared with several state-of-the-art methods. The code and dataset are available for research purposes at http://cic.tju.edu.cn/faculty/likun/projects/MGTnet .

Item Type:	Article
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	Institute of Electrical and Electronics Engineers
ISSN:	1057-7149
Date of First Compliant Deposit:	19 June 2021
Date of Acceptance:	29 April 2021
Last Modified:	22 Nov 2024 07:00
URI:	https://orca.cardiff.ac.uk/id/eprint/141998

Citation Data

Cited 2 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item