Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

Towards grouping in large scenes with occlusion-aware spatio-temporal transformers

Zhang, Jinsong, Gu, Lingfeng, Lai, Yu-Kun ORCID:, Wang, Xueyang and Li, Kun 2024. Towards grouping in large scenes with occlusion-aware spatio-temporal transformers. IEEE Transactions on Circuits and Systems for Video Technology 34 (5) , pp. 3919-3929. 10.1109/TCSVT.2023.3324868

[thumbnail of GroupTransformerTCSVT.pdf]
PDF - Accepted Post-Print Version
Download (7MB) | Preview


Group detection, especially for large-scale scenes, has many potential applications for public safety and smart cities. Existing methods fail to cope with frequent occlusions in large-scale scenes with multiple people, and are difficult to effectively utilize spatio-temporal information. In this paper, we propose an end-to-end framework, GroupTransformer , for group detection in large-scale scenes. To deal with the frequent occlusions caused by multiple people, we design an occlusion encoder to detect and suppress severely occluded person crops. To explore the potential spatio-temporal relationship, we propose spatio-temporal transformers to simultaneously extract trajectory information and fuse inter-person features in a hierarchical manner. Experimental results on both large-scale and small-scale scenes demonstrate that our method achieves better performance compared with state-of-the-art methods. On large-scale scenes, our method significantly boosts the performance in terms of precision and F1 score by more than 10%. On small-scale scenes, our method still improves the performance of F1 score by more than 5%. We will release the code for research purposes .

Item Type: Article
Date Type: Publication
Status: Published
Schools: Computer Science & Informatics
Publisher: Institute of Electrical and Electronics Engineers
ISSN: 1051-8215
Date of First Compliant Deposit: 21 March 2024
Date of Acceptance: 8 October 2023
Last Modified: 06 Jun 2024 02:28

Actions (repository staff only)

Edit Item Edit Item


Downloads per month over past year

View more statistics