Search

In the research of robot systems, path planning and obstacle avoidance are important research directions, especially in unknown dynamic environments where flexibility and rapid decision makings are required. In this paper, a state attention network (SAN) was developed to extract features to represent the interaction between an intelligent robot and its obstacles. An auxiliary actor discriminator (AAD) was developed to calculate the probability of a collision. Goal-directed and gap-based navigation strategies were proposed to guide robotic exploration. The proposed policy was trained through simulated scenarios and updated by the Soft Actor-Critic (SAC) algorithm. The robot executed the action depending on the AAD output. Heuristic knowledge (HK) was developed to prevent blind exploration of the robot. Compared to other methods, adopting our approach in robot systems can help robots converge towards an optimal action strategy. Furthermore, it enables them to explore paths in unknown environments with fewer moving steps (showing a decrease of 33.9%) and achieve higher average rewards (showning an increase of 29.15%). [ABSTRACT FROM AUTHOR]

Limitations are identified in the expressive capabilities of the deep feature extraction network employed in deep reinforcement learning (DRL), particularly in complex scenarios. Additionally, learning performance is negatively impacted by compound errors, with consideration given to the potential critical role of sample quality in learning outcomes. Therefore, how to improve the collision avoidance decision-making adaptive and effectiveness for autonomous ships navigating in various scenarios is crucial. To address these issues and overcome data acquisition difficulties, a method called rule-guided vision supervised learning (RGVSL) is proposed in this paper. Through a static collision avoidance decision-making task, a comparison is drawn between a deep feature extraction network and the Nature CNN in DQN, revealing shortcomings in the feature extraction of DRL. Additionally, a collision avoidance decision-making environment with the capability of generalization for maritime encounters is proposed, and the advantages of the method in terms of adaptability and learning cost are validated. Finally, it is demonstrated that the RGVSL method is equivalent to a DQN with γ set to 0, indicating a significant performance improvement without compound errors. Achieving an adaptive decision accuracy of over 90% in various encounter scenarios without retraining, this research substantially reduces learning costs. It can provide innovative and practical solutions for the technological development in the field of autonomous ship collision avoidance decision-making. • Revolutionizes DRL data by using explicit rules for diverse scenarios, boosting model performance. • Exposes Nature CNN's DQN inadequacy, emphasizing the need for effective feature expression in DRL. • RGVSL achieves 90% accuracy in diverse maritime scenarios, cutting retraining costs, enhancing performance. [ABSTRACT FROM AUTHOR]