Search

For the vertical landing phase of the reusable rocket, in order to improve the landing accuracy with consideration of multiple uncertainties, a novel strategy to calculate the ignition height online is proposed based on polynomial guidance law (PGL), particle swarm optimization (PSO), and deep reinforcement learning (DRL). Firstly, a deep neural network (DNN) is designed to describe the relationship between the state of the reusable rocket and the ignition height. To accomplish the guidance task of the vertical landing phase, PGL is modified by introducing the estimated aerodynamic acceleration. Through simulation, the output range of the DNN is estimated by the modified PSO. Then, the reward function is shaped and the parameters of the DNN are trained on a training set of simulation scenarios by the DRL algorithm. Finally, to demonstrate the effectiveness of the proposed strategy, the trained DNN is used to calculate the ignition height of 1500 unlearned simulation scenarios online. The numerical simulation results show that the proposed strategy has higher landing accuracy and lower fuel consumption than the offline strategy of fixed ignition height based on the modified PSO. [ABSTRACT FROM AUTHOR]

In this paper, a variable-structure multimodel (VSMM) filtering algorithm based on the long short-term memory (LSTM) regression-deep Q network (L-DQN) is proposed to accurately track strong maneuvering targets. The algorithm can map the selection of the model set to the selection of the action label and realize the purpose of a deep reinforcement-learning agent to replace the model switching in the traditional VSMM algorithm by reasonably designing a reward function, state space, and network structure. At the same time, the algorithm introduces a LSTM algorithm, which can compensate the error of tracking results based on model history information. The simulation results show that compared with the traditional VSMM algorithm, the proposed algorithm can quickly capture the maneuvering of the target, the response time is short, the calculation accuracy is significantly improved, and the range of adaptation is wider. Precise tracking of maneuvering targets was achieved. [ABSTRACT FROM AUTHOR]

A moving mass–actuated unmanned aerial vehicle (MAUAV) is controlled by mass sliders installed inside the airframe and has the advantages of high aerodynamic efficiency and good stealth performance. However, designing a controller for it faces severe challenges due to the strong nonlinearity and coupling of its dynamics. To this end, we proposed an attitude controller based on deep reinforcement learning for the MAUAV. It directly maps the states to the needed deflection of the actuators and is an end-to-end controller. For the sparse reward problem, the reward function required for training is reasonably designed through reward shaping to hasten the algorithm's training speed. In training, random initialization and parameter perturbation are used to strengthen the final policy's robustness further. The simulation results tentatively demonstrate that the proposed controller is not only robust but suboptimal. Compared with an active disturbance rejection controller (ADRC) optimized by the particle swarm algorithm, our controller still guarantees a 100% success rate in multiple unlearned scenarios, meaning it has good generalization ability. [ABSTRACT FROM AUTHOR]

This paper studies a novel guidance framework of the vehicle against a high-speed and maneuvering target based on deep reinforcement learning (DRL) considering the energy consumption, autopilot lag dynamics, and input saturation, which can effectively cope with the high flight-path angle error flight phase and various uncertainties. The guidance framework proposes an end-to-end mapping transformation between the guidance command and observation states consisting of line-of-sight (LOS) angle, relative distance, and their rate measured by the seeker. At the same time, the observability of the LOS angle and relative distance is included in constructing the reward function. Besides, the relative engagement kinematic model of the interceptor-target is established and combined with the PPO guidance algorithm, jointly described as a Markov decision process (MDP). Notably, the guidance framework is optimized using the improved proximal policy optimization (PPO) algorithm and demonstrated in a simulated terminal phase in the near-space. Specifically, the PPO guidance algorithm is structured by the policy (actor) neural network and the critic neural network, and both are standard fully-connected neural networks. Subsequently, observation states and rewards are fully collected and applied by introducing the experience replay method. Also, the exponential decay learning rate method, mini-batch stochastic gradient ascent (SGA) method, zero-score standardization, and Adam optimizer are proposed to train the reinforcement learning algorithm more efficiently. Moreover, the proposed guidance framework has an excellent generalization capability and guarantees the implementation of fixed and stochastic engagement scenarios, which means that the interceptor can realize the unlearned practical combat scenarios. The robust capacity is indicated and validated using Monte Carlo simulation under various uncertainties. Moreover, the DRL guidance framework can satisfy the onboard application requirement. [ABSTRACT FROM AUTHOR]