Zhou, Feng, Li, Chi, Dai, Ju, Zhu, Mengxiao, Zhang, Yongmei, Lai, Yukun  ORCID: https://orcid.org/0000-0002-2094-5680 and Rosin, Paul ORCID: https://orcid.org/0000-0002-4965-3884
2025.
Chat-driven 3D human pose and shape editing with Large Language Models.
Presented at: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP),
Hyderabad, India,
6-11 April 2025.
Published in: Rao, B. D., Trancoso, I., Sharma, G. and Mehta, N. B. eds.
Proceedings of the International Conference on Acoustics, Speech and Signal Processing.
IEEE,
pp. 1-5.
10.1109/ICASSP49660.2025.10887921
|
Preview |
PDF
- Accepted Post-Print Version
Download (2MB) | Preview |
Abstract
Generating and creating humanoid 3D models has received increasing attention recently due to its fundamental support for many high-level 3D applications. Although automatic 3D pose and shape reconstruction methods have achieved promising results, there are still some failure cases due to self-occlusions, viewpoint changes, and the complexity of human pose articulations. In this paper, we propose a novel way to leverage Large Language Models (LLMs) to interactively reconstruct human pose and shape based on a Skinned Multi-Person Linear (SMPL) model. We construct a mapping table to fine-tune an LLM, enabling it to understand user inputs better and output the positional information of joint points. Additionally, a simple neural network is adopted to regress the shape cues of the SMPL. We demonstrate a gallery of results of numerous poses and shapes. We validate our method via numerical evaluations, user studies, and comparisons to manually posed characters and previous work.
| Item Type: | Conference or Workshop Item (Paper) |
|---|---|
| Date Type: | Published Online |
| Status: | Published |
| Schools: | Schools > Computer Science & Informatics |
| Publisher: | IEEE |
| ISBN: | 979-8-3503-6875-8 |
| ISSN: | 1520-6149 |
| Related URLs: | |
| Date of First Compliant Deposit: | 8 February 2025 |
| Date of Acceptance: | 18 December 2024 |
| Last Modified: | 26 Mar 2025 15:15 |
| URI: | https://orca.cardiff.ac.uk/id/eprint/176050 |
Actions (repository staff only)
 |
Edit Item |
Dimensions
Dimensions