Wang, C, Zhang, X, Gao, H, Bashir, M, Li, H and Yang, Z (2024) COLERGs-constrained safe reinforcement learning for realising MASS's risk-informed collision avoidance decision making. Knowledge-Based Systems, 300. ISSN 0950-7051

Text KBS-Revised Manuscript (Clean Version).pdf - Accepted Version Restricted to Repository staff only until 16 July 2025. Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (2MB)

Abstract

Maritime autonomous surface ship (MASS) represents a significant advancement in maritime technology, offering the potential for increased efficiency, reduced operational costs, and enhanced maritime traffic safety. However, MASS navigation in complex maritime traffic and congested water areas presents challenges, especially in Collision Avoidance Decision Making (CADM) during multi-ship encounter scenarios. Through a robust risk assessment design for time-sequential and joint-target ships (TSs) encounter scenarios, a novel risk and reliability critic-enhanced safe hierarchical reinforcement learning (RA-SHRL), constrained by the International Regulations for Preventing Collisions at Sea (COLREGs), is proposed to realize the autonomous navigation and CADM of MASS. Finally, experimental simulations are conducted against a time-sequenced obstacle avoidance scenario and a swarm obstacle avoidance scenario. The experimental results demonstrate that RA-SHRL generates safe, efficient, and reliable collision avoidance strategies in both time-sequential dynamic obstacles and mixed joint-TSs environments. Additionally, the RA-SHRL is capable of assessing risk and avoiding multiple joint-TSs. Compared with Deep Q-network (DQN) and Constrained Policy Optimization (CPO), the search efficiency of the algorithm proposed in this paper is improved by 40% and 12%, respectively. Moreover, it achieved a 91.3% success rate of collision avoidance during training. The methodology could also benefit other autonomous systems in dynamic environments.

Item Type:	Article
Uncontrolled Keywords:	08 Information and Computing Sciences; 15 Commerce, Management, Tourism and Services; 17 Psychology and Cognitive Sciences; Artificial Intelligence & Image Processing
Subjects:	V Naval Science > VM Naval architecture. Shipbuilding. Marine engineering
Divisions:	Engineering
Publisher:	Elsevier BV
SWORD Depositor:	A Symplectic
Date Deposited:	19 Aug 2024 08:24
Last Modified:	19 Aug 2024 08:30
DOI or ID number:	10.1016/j.knosys.2024.112205
URI:	https://researchonline.ljmu.ac.uk/id/eprint/23970

View Item