Search

In the research of robot systems, path planning and obstacle avoidance are important research directions, especially in unknown dynamic environments where flexibility and rapid decision makings are required. In this paper, a state attention network (SAN) was developed to extract features to represent the interaction between an intelligent robot and its obstacles. An auxiliary actor discriminator (AAD) was developed to calculate the probability of a collision. Goal-directed and gap-based navigation strategies were proposed to guide robotic exploration. The proposed policy was trained through simulated scenarios and updated by the Soft Actor-Critic (SAC) algorithm. The robot executed the action depending on the AAD output. Heuristic knowledge (HK) was developed to prevent blind exploration of the robot. Compared to other methods, adopting our approach in robot systems can help robots converge towards an optimal action strategy. Furthermore, it enables them to explore paths in unknown environments with fewer moving steps (showing a decrease of 33.9%) and achieve higher average rewards (showning an increase of 29.15%). [ABSTRACT FROM AUTHOR]

Limitations are identified in the expressive capabilities of the deep feature extraction network employed in deep reinforcement learning (DRL), particularly in complex scenarios. Additionally, learning performance is negatively impacted by compound errors, with consideration given to the potential critical role of sample quality in learning outcomes. Therefore, how to improve the collision avoidance decision-making adaptive and effectiveness for autonomous ships navigating in various scenarios is crucial. To address these issues and overcome data acquisition difficulties, a method called rule-guided vision supervised learning (RGVSL) is proposed in this paper. Through a static collision avoidance decision-making task, a comparison is drawn between a deep feature extraction network and the Nature CNN in DQN, revealing shortcomings in the feature extraction of DRL. Additionally, a collision avoidance decision-making environment with the capability of generalization for maritime encounters is proposed, and the advantages of the method in terms of adaptability and learning cost are validated. Finally, it is demonstrated that the RGVSL method is equivalent to a DQN with γ set to 0, indicating a significant performance improvement without compound errors. Achieving an adaptive decision accuracy of over 90% in various encounter scenarios without retraining, this research substantially reduces learning costs. It can provide innovative and practical solutions for the technological development in the field of autonomous ship collision avoidance decision-making. • Revolutionizes DRL data by using explicit rules for diverse scenarios, boosting model performance. • Exposes Nature CNN's DQN inadequacy, emphasizing the need for effective feature expression in DRL. • RGVSL achieves 90% accuracy in diverse maritime scenarios, cutting retraining costs, enhancing performance. [ABSTRACT FROM AUTHOR]

Autonomous vehicles can reduce labor power during cargo transportation, and then improve transportation efficiency, for example, the automated guided vehicle (AGV) in the warehouse can improve the operation efficiency. To overcome the limitations of traditional path planning algorithms in unknown environments, such as reliance on high-precision maps, lack of generalization ability, and obstacle avoidance capability, this study focuses on investigating the Deep Q-Network and its derivative algorithm to enhance network and algorithm structures. A new algorithm named APF-D3QNPER is proposed, which combines the action output method of artificial potential field (APF) with the Dueling Double Deep Q Network algorithm, and experience sample rewards are considered in the experience playback portion of the traditional Deep Reinforcement Learning (DRL) algorithm, which enhances the convergence ability of the traditional DRL algorithm. A long short-term memory (LSTM) network is added to the state feature extraction network part to improve its adaptability in unknown environments and enhance its spatiotemporal sensitivity to the environment. The APF-D3QNPER algorithm is compared with mainstream deep reinforcement learning algorithms and traditional path planning algorithms using a robot operating system and the Gazebo simulation platform by conducting experiments. The results demonstrate that the APF-D3QNPER algorithm exhibits excellent generalization abilities in the simulation environment, and the convergence speed, the loss value, the path planning time, and the path planning length of the APF-D3QNPER algorithm are all less than for other algorithms in diverse scenarios. [ABSTRACT FROM AUTHOR]