SATPose: Improving monocular 3D pose estimation with spatial-aware ground tactility

Zhan, Lishuang, Ying, Enting, Gan, Jiabao, Guo, Shihui, Gao, Boyu and Qin, Yipeng ORCID: https://orcid.org/0000-0002-1551-9126 2024. SATPose: Improving monocular 3D pose estimation with spatial-aware ground tactility. Presented at: ACM Multimedia 2024, Melbourne, Australia, 28 October - 1 November 2024. MM '24: Proceedings of the 32nd ACM International Conference on Multimedia. ACM, pp. 6192-6201. 10.1145/3664647.3681654

Preview

PDF - Accepted Post-Print Version Download (9MB) | Preview

Abstract

Estimating 3D human poses from monocular images is an important research area with many practical applications. However, the depth ambiguity of 2D solutions limits their accuracy in actions where occlusion exits or where slight centroid shifts can result in significant 3D pose variations. In this paper, we introduce a novel multimodal approach to mitigate the depth ambiguity inherent in monocular solutions by integrating spatial-aware pressure information. We first establish a data collection system with a pressure mat and a monocular camera, and construct a large-scale multimodal human activity dataset comprising over 600,000 frames of motion data. Utilizing this dataset, we propose a pressure image reconstruction network to extract pressure priors from monocular images. Subsequently, we introduce a Transformer-based multimodal pose estimation network to combine pressure priors with monocular images, achieving a world mean per joint position error of 51.6mm, outperforming state-of-the-art methods. Extensive experiments demonstrate the effectiveness of our multimodal 3D human pose estimation method across various actions and joints, highlighting the significance of spatial-aware pressure in improving the accuracy of monocular-vision-based methods. Our dataset is available at: https://github.com/LishuangZhan/SATPose.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Published Online
Status:	In Press
Schools:	Computer Science & Informatics
Publisher:	ACM
Date of First Compliant Deposit:	6 September 2024
Date of Acceptance:	16 July 2024
Last Modified:	15 Nov 2024 10:20
URI:	https://orca.cardiff.ac.uk/id/eprint/170912

Actions (repository staff only)

Edit Item