Deep reinforcement learning based collision avoidance system for autonomous ships.

Autonomous ships is a key to avoid accidents caused by human errors and improve maritime safety. However, unlike the autonomous vehicles counterpart, collision avoidance for autonomous ships faces many challenges due to the hash driving environments, difficult ship control and large stopping distance. In this paper, we investigate a collision avoidance system for autonomous ships under complex encounter scenarios, such as busy ports. In the system various sensors are used to detect objects and perceive the maritime environments. To help the autonomous ships handle the complex and dynamic scenarios that may be encountered, a collision map used to describe the ships encounter scenarios is generated and utilized as the input of a deep reinforcement learning (DRL) model. The DRL model is applied to make collision avoidance and safe driving decisions. New reward functions are proposed to train the DRL model to generate safe ship maneuver actions to reduce collisions and ensure compliance with the Convention on the International Regulations for Preventing Collisions at Sea (COLREGs). Furthermore, a self-adaptive parameters sharing approach is designed for fast convergence and collision avoidance performance of the DRL model, where the parameters of the fully connected layers are shared and the correlation layers are self adapted for the DRL critic and actor networks. Simulation results show that the proposed system has high DRL convergence speed and excellent collision avoidance. • A novel sharing mechanism with hybrid structure for policy network and value network in DRL model is proposed to accelerate training speed. • According to the characteristics of ship navigation, the concept of collision map with different layers is proposed. • Both the dynamic futures and regional feature of ship navigation are considered in collision map to have comprehensive understanding of ship navigation for DRL model. • Several reward functions considering risk, COLREGs, navigation characteristics are designed to ensure safety of ship navigation. [ABSTRACT FROM AUTHOR]