Crowd4D: Scene-aware monocular 4D crowd reconstruction

Kang, Hongbo, Zhou, Tianyi, Yang, Qingyang, Wen, Hongwei, Huang, Jing, Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680 and Li, Kun 2026. Crowd4D: Scene-aware monocular 4D crowd reconstruction. Presented at: ICML 2026, Seoul, South Korea, 6-11 July 2026. Proceedings of Machine Learning Research. Item availability restricted.

	PDF (Provisional file) - Accepted Post-Print Version Download (17kB)
	PDF - Accepted Post-Print Version Restricted to Repository staff only Download (11MB)

Abstract

Recovering scene-consistent 4D crowd motion from monocular video in large-scale scenes remains challenging due to severe depth ambiguity and complex scene geometry. Existing monocular crowd reconstruction methods typically rely on single-plane assumptions, leading to unreliable metric scale and spatial drift under complex terrain. We propose Crowd4D, the first scene-aware 4D crowd reconstruction framework that jointly optimizes the crowd and scene from a monocular RGB video in large-scale scenes. Crowd4D explicitly incorporates scene geometry and ensures consistency across image and scene spaces via a multi-stage optimization strategy. A key bottleneck of this task lies in accurate human–scene alignment, particularly in scale and position. However, human and scene reconstructions are typically decoupled. To address this, we introduce the Human–Scene Interaction Proxy (HSIP) as an intermediate representation, derived from Scene Interaction Point Clouds and a Scene Interaction Surface (SIPC&SIS), which encode explicit scene-aware geometric priors and redefine the optimization space for large-scale monocular 4D crowd reconstruction. To further improve temporal stability under occlusions, we introduce Crowd Structural Coherence Regularization (CSCR), which leverages HSIP-based spatial priors to impose soft temporal consistency on pairwise relative displacements and directions within local crowd neighborhoods. Extensive experiments demonstrate that Crowd4D consistently outperforms existing state-of-the-art methods and enables robust monocular 4D crowd reconstruction in complex, large-scale real-world scenes.

Item Type:	Conference or Workshop Item - published (Paper)
Status:	In Press
Schools:	Schools > Computer Science & Informatics
ISSN:	2640-3498
Date of First Compliant Deposit:	1 July 2026
Date of Acceptance:	30 April 2026
Last Modified:	09 Sep 2026 13:08
URI:	https://orca.cardiff.ac.uk/id/eprint/187862

Actions (repository staff only)

Edit Item