AHRNet: Attention and heatmap-based regressor for hand pose estimation and mesh recovery

Zhou, Feng, Shen, Pei, Dai, Ju, Jiang, Na, Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680 and Rosin, Paul ORCID: https://orcid.org/0000-0002-4965-3884 2024. AHRNet: Attention and heatmap-based regressor for hand pose estimation and mesh recovery. Presented at: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Korea, 14-19 April 2024. Proceedings International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 10.1109/ICASSP48485.2024.10446600

Preview

PDF - Accepted Post-Print Version Download (1MB) | Preview

Abstract

Estimating 3D hand pose and recovering the full hand surface mesh from a single RGB image is a challenging task due to self-occlusions, viewpoint changes, and the complexity of hand articulations. In this paper, we propose a novel framework that combines an attention mechanism with heatmap regression to accurately and efficiently predict 3D joint locations and reconstruct the hand mesh. We adopt a pooling attention module that learns to focus on relevant regions in the input image to extract better features for handling occlusions, while greatly reducing the computational cost. The multi-scale 2D heatmaps provide spatial constraints to guide the 3D vertex predictions. By exploiting the complementary strengths of sparse 2D supervision and dense mesh regression, our method accurately reconstructs hand meshes with realistic details. Extensive experiments on standard benchmarks demonstrate that the proposed method efficiently improves the performance of 3D hand pose estimation and mesh recovery. The reproducible recipes are available at https://github.com/SDiannn/AHRNET-Heatmap.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Published Online
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	IEEE
ISBN:	979-8-3503-4485-1
ISSN:	2379-190X
Date of First Compliant Deposit:	21 March 2024
Date of Acceptance:	13 December 2023
Last Modified:	14 Aug 2024 15:27
URI:	https://orca.cardiff.ac.uk/id/eprint/167444

Actions (repository staff only)

Edit Item