Chat-driven 3D human pose and shape editing with Large Language Models

Zhou, Feng, Li, Chi, Dai, Ju, Zhu, Mengxiao, Zhang, Yongmei, Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680 and Rosin, Paul ORCID: https://orcid.org/0000-0002-4965-3884 2025. Chat-driven 3D human pose and shape editing with Large Language Models. Presented at: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Hyderabad, India, 6-11 April 2025. Published in: Rao, B. D., Trancoso, I., Sharma, G. and Mehta, N. B. eds. Proceedings of the International Conference on Acoustics, Speech and Signal Processing. IEEE, pp. 1-5. 10.1109/ICASSP49660.2025.10887921

Preview

PDF - Accepted Post-Print Version Download (2MB) | Preview

Abstract

Generating and creating humanoid 3D models has received increasing attention recently due to its fundamental support for many high-level 3D applications. Although automatic 3D pose and shape reconstruction methods have achieved promising results, there are still some failure cases due to self-occlusions, viewpoint changes, and the complexity of human pose articulations. In this paper, we propose a novel way to leverage Large Language Models (LLMs) to interactively reconstruct human pose and shape based on a Skinned Multi-Person Linear (SMPL) model. We construct a mapping table to fine-tune an LLM, enabling it to understand user inputs better and output the positional information of joint points. Additionally, a simple neural network is adopted to regress the shape cues of the SMPL. We demonstrate a gallery of results of numerous poses and shapes. We validate our method via numerical evaluations, user studies, and comparisons to manually posed characters and previous work.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Published Online
Status:	Published
Schools:	Schools > Computer Science & Informatics
Publisher:	IEEE
ISBN:	979-8-3503-6875-8
ISSN:	1520-6149
Related URLs:	https://2025.ieeeicassp.org/
Date of First Compliant Deposit:	8 February 2025
Date of Acceptance:	18 December 2024
Last Modified:	26 Mar 2025 15:15
URI:	https://orca.cardiff.ac.uk/id/eprint/176050

Actions (repository staff only)

Edit Item