RSMPNet: Relationship guided semantic map prediction

Sun, Jingwen, Wu, Jing ORCID: https://orcid.org/0000-0001-5123-9861, Ji, Ze ORCID: https://orcid.org/0000-0002-8968-9902 and Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680 2024. RSMPNet: Relationship guided semantic map prediction. Presented at: IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Hawaii, USA, 4-8 January 2024. Proceedings of the 2024 IEEE/CVF Winter Conference on Applications of Computer Vision. IEEE, pp. 302-311. 10.1109/WACV57701.2024.00037

Preview

PDF - Accepted Post-Print Version Download (843kB) | Preview

Abstract

In semantic navigation, a top-down map with accurate and complete semantic information is vital to subsequent decision-making. However, due to occlusions and limitations of the robot’s field of view (FOV), there are often unobserved areas in the top-down maps. To address this problem, recent works have studied semantic map prediction to complete the top-down maps. In this work, we propose to improve map prediction by integrating relational information. We propose RSMPNet, a relationship-guided semantic map prediction network, which makes use of semantic and spatial relationships to predict unobserved areas from accumulated semantic maps. Specifically, we propose a Relationship Reasoning Layer that includes two modules, namely 1) the Semantic Relationship Graph Reasoning Module (SeGRM) to capture the semantic relationship and 2) the Spatial Relationship Graph Reasoning Module (SpGRM) to utilize the spatial relationship. We also design a semantic relationship enhanced loss to enhance our model to learn semantic relationship information. Experiments show the effectiveness of our proposed network which achieves state-of-the-art performance in semantic map prediction. Our code and dataset are publicly available at https://github.com/jws39/semantic-map-prediction

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Published Online
Status:	Published
Schools:	Professional Services > Advanced Research Computing @ Cardiff (ARCCA) Schools > Computer Science & Informatics Schools > Engineering
Publisher:	IEEE
ISBN:	979-8-3503-1893-7
Date of First Compliant Deposit:	9 November 2023
Date of Acceptance:	24 October 2023
Last Modified:	12 Mar 2025 14:11
URI:	https://orca.cardiff.ac.uk/id/eprint/163772

Actions (repository staff only)

Edit Item