FGRL: Federated growing reinforcement learning for resilient mapless navigation in unfamiliar environments

Tian, Shunyu, Wei, Changyun, Li, Yajun and Ji, Ze ORCID: https://orcid.org/0000-0002-8968-9902 2024. FGRL: Federated growing reinforcement learning for resilient mapless navigation in unfamiliar environments. Applied Sciences 14 (23) , 11336. 10.3390/app142311336

PDF - Published Version Download (7MB)

Abstract

In this paper, we propose a federated growing reinforcement learning (FGRL) approach for solving the mapless navigation problem of unmanned ground vehicles (UGVs) facing cluttered unfamiliar obstacles. Deep reinforcement learning (DRL) has the potential to provide adaptive behaviors for autonomous agents through interactive learning, but standard episodic DRL algorithms often struggle with out-of-distribution observations. For navigation tasks, UGVs often encounter unfamiliar situations where novel obstacles differ from prior experience. To address this problem, the proposed FGRL approach enables multiple agents to obtain their individual navigation models in diverse scenarios, and achieves online knowledge aggregation to obtain an adaptive and resilient model that copes with unfamiliar uncertain obstacles. Specifically, during the learning process of navigation tasks, we introduce the growth rate of each agent’s local model based on the performance of consecutive learning rounds. Then, we weight the local model of each agent based on the growth rate to achieve knowledge aggregation in a shared model. We also consider a growth threshold to eliminate the interference of low-quality local models. We carry out extensive simulations to validate the proposed solution, and the results show that our approach can learn resilient behaviors of collision avoidance for UGVs to cope with never encountered and cluttered unfamiliar obstacles.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Engineering
Additional Information:	License information from Publisher: LICENSE 1: URL: https://creativecommons.org/licenses/by/4.0/, Start Date: 2024-12-05
Publisher:	MDPI
Date of First Compliant Deposit:	19 December 2024
Date of Acceptance:	4 December 2024
Last Modified:	19 Dec 2024 10:30
URI:	https://orca.cardiff.ac.uk/id/eprint/174835

Actions (repository staff only)

Edit Item