Search

Over the past decades, unmanned aerial vehicles (UAVs) have been widely used in both military and civilian fields. In these applications, flocking motion is a fundamental but crucial operation of multi-UAV systems. Traditional flocking motion methods usually designed for a specific environment. However, the real environment is mostly unknown and stochastic, which greatly reduces the practicality of these methods. In this paper, deep reinforcement learning (DRL) is used to realize the flocking motion of multi-UAV systems. Considering that the sim-to-real problem restricts the application of DRL to the flocking motion scenario, a digital twin (DT) enabled DRL training framework is proposed to solve this problem. The DRL model can learn from DT and be quickly deployed on the real-world UAV with the help of DT. Under this training framework, this paper proposes an actor-critic DRL algorithm named behavior-coupling deep deterministic policy gradient (BCDDPG) for the flocking motion problem, which is inspired by the flocking behavior of animals. Extensive simulations are conducted to evaluate the performance of BCDDPG. Simulation results show that BCDDPG achieves a higher average reward and performs better in terms of arrival rate and collision rate compared with existing methods.

With high mobility, low cost and outstanding maneuverability properties, unmanned aerial vehicle (UAV) swarm has attracted worldwide attentions in both academia and industry. Nevertheless, the complex and coherent characteristics of the intelligent cooperation of UAV swarm greatly restrict its wide application. The recent development of artificial intelligence provides new methodologies for intelligent cooperation of UAV swarm. However, these methods are resource-in-tensive that cannot be directly applied in the computation and storage constrained UAVs. In this article, we propose a novel digital twin (DT)-based intelligent cooperation framework of UAV swarm. In the framework, a digital twin model is established to reflect the physical entity (i.e., UAV swarm) with high-fidelity and monitors its whole life cycle. Next, the decision model that integrates a machine learning algorithm is built to explore the global optimal solution and controls the behaviors of UAV swarm. To demonstrate the effectiveness of our proposed framework, a case study on intelligent network reconstruction is introduced, and simulation results are presented. Finally, a representative application provided by the framework is discussed.