Risk-aware deep reinforcement learning for mapless navigation of unmanned surface vehicles in uncertain and congested environments

Wu, Xiangyu, Wei, Changyun, Guan, Dawei and Ji, Ze ORCID: https://orcid.org/0000-0002-8968-9902 2025. Risk-aware deep reinforcement learning for mapless navigation of unmanned surface vehicles in uncertain and congested environments. Ocean Engineering 322 , 120446. 10.1016/j.oceaneng.2025.120446 Item availability restricted.

PDF - Accepted Post-Print Version Restricted to Repository staff only until 30 January 2026 due to copyright restrictions. Download (23MB)

Abstract

This paper addresses the navigation problem of Unmanned Surface Vehicles (USVs) in uncertain and congested environments. While previous research has extensively explored USV navigation, most approaches assume that the environmental maps and obstacle locations are pre-known to the USVs. In this paper, we focus on a sensor-level navigation approach that utilizes real-time LiDAR data integrated with deep reinforcement learning (DRL) for decision-making. To tackle sparse reward challenges, we propose a potential-based reward-shaping (PRS) module to regulate navigation behavior, and this module helps to improve the training efficiency of the twin delayed deep deterministic policy gradient (TD3) algorithm. Moreover, we introduce a risk evaluation and correction (REC) module to mitigate potential risks. This module employs a risk evaluation network to enhance the agent’s risk awareness and an action-level correction mechanism to avoid unsafe behavior. The proposed approach is validated through ablation studies and comparative experiments in OpenAI Gym-based environments and simulated island regions of Zhoushan. The results indicate that the proposed approach significantly improves training efficiency while maintaining consistency and robustness in unknown and congested marine environments.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Engineering
Additional Information:	License information from Publisher: LICENSE 1: URL: http://creativecommons.org/licenses/by-nc-nd/4.0/, Start Date: 2027-01-30
Publisher:	Elsevier
ISSN:	0029-8018
Date of First Compliant Deposit:	3 February 2025
Date of Acceptance:	18 January 2025
Last Modified:	03 Feb 2025 12:45
URI:	https://orca.cardiff.ac.uk/id/eprint/175842

Actions (repository staff only)

Edit Item