Search

Unmanned Aerial Vehicles (UAVs) have attacted much attention in the field of wireless communication due to its agility and altitude. UAVs can be used as low-altitude aerial base stations (UAV-BSs) to provide communication services for ground devices (GDs) in various scenarios, such as emergency communication and traffic offloading in hotspots. However, due to the limited communication ranges and high prices of commercial UAV-BSs, covering a target area all the time with sufficient UAVs is quite challenging, especially under dynamic environment. We need to design the trajectory of the UAV-BSs to optimize system performance. Most existing works focus on the energy-efficient coverage and throughput maximization but ignore the fairness of communication service, especially the fairness at user-level. Besides, reinforcement learning is suitable for solving decision problems in dynamic environments. However, most existing works use centralized deep reinforcement learning (DRL) approaches. Due to the scalability and low time complexity, a distributed DRL approach is more suitable for multiple UAV-BSs communication system in dynamic environment. Unlike previous works, we characterize the fairness at user-level based on proportional fairness scheduling and formulate a weighted-throughput maximization problem via designing UAV-BSs’ trajectory. Then we model the dynamic deploymentproblem of UAV-BSs as a Markov game and propose a multi-agent deep reinforcement learning-based distributed UAV-BSs control approach named MAUC. MAUC approach adopts the framework of centralized training with distributed execution. Simulation results show that the MAUC can improve fairness of communication service by sacrificing a small amount of throughput. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the problem of wireless service provisioning through a rotary-wing UAV which can serve as an aerial base station (BS) to communicate with multiple ground terminals (GTs) in a boost demand area. Our objective is to optimize the UAV control for maximizing the UAV.s energy efficiency, in which both aerodynamic energy and communication energy are considered while ensuring the communication requirements for each GT and backhaul link between the UAV and the terrestrial BS. The mobility of the UAV and GTs lead to time-varying channel conditions that make the environment dynamic. We formulate a nonconvex optimization for controlling the UAV considering the practical angle-dependent Rician fading channels between the UAV and GTs, and between the UAV and the terrestrial BS. Traditional optimization approaches are not able to handle the dynamic environment and high complexity of the problem in real-time. We propose to use a deep reinforcement learning-based approach namely Deep Deterministic Policy Gradient (DDPG) to solve the formulated nonconvex problem of UAV control with continuous action space that takes into account the real-time of the environment including time-varying UAV-ground channel conditions, available onboard energy of the UAV, and the communication requirement of the GTs. However, the DDPG method may not achieve good performance in an unstable environment and will face a large number of hyperparameters. We extend our approach to use the Trust Region Policy Optimization (TRPO) method that can improve the performance of the UAV compared to the DDPG method in such a dynamic environment. [ABSTRACT FROM AUTHOR]

In this paper, we aim to design a fully-distributed control solution to navigate a group of unmanned aerial vehicles (UAVs), as the mobile Base Stations (BSs) to fly around a target area, to provide long-term communication coverage for the ground mobile users. Different from existing solutions that mainly solve the problem from optimization perspectives, we proposed a decentralized deep reinforcement learning (DRL) based framework to control each UAV in a distributed manner. Our goal is to maximize the temporal average coverage score achieved by all UAVs in a task, maximize the geographical fairness of all considered point-of-interests (PoIs), and minimize the total energy consumptions, while keeping them connected and not flying out of the area border. We designed the state, observation, action space, and reward in an explicit manner, and model each UAV by deep neural networks (DNNs). We conducted extensive simulations and found the appropriate set of hyperparameters, including experience replay buffer size, number of neural units for two fully-connected hidden layers of actor, critic, and their target networks, and the discount factor for remembering the future reward. The simulation results justified the superiority of the proposed model over the state-of-the-art DRL-EC $^3$ 3 approach based on deep deterministic policy gradient (DDPG), and three other baselines. [ABSTRACT FROM AUTHOR]

Unmanned aerial vehicles (UAVs) can be used to serve as aerial base stations to enhance both the coverage and performance of communication networks in various scenarios, such as emergency communications and network access for remote areas. Mobile UAVs can establish communication links for ground users to deliver packets. However, UAVs have limited communication ranges and energy resources. Particularly, for a large region, they cannot cover the entire area all the time or keep flying for a long time. It is thus challenging to control a group of UAVs to achieve certain communication coverage in a long run, while preserving their connectivity and minimizing their energy consumption. Toward this end, we propose to leverage emerging deep reinforcement learning (DRL) for UAV control and present a novel and highly energy-efficient DRL-based method, which we call DRL-based energy-efficient control for coverage and connectivity (DRL-EC3). The proposed method 1) maximizes a novel energy efficiency function with joint consideration for communications coverage, fairness, energy consumption and connectivity; 2) learns the environment and its dynamics; and 3) makes decisions under the guidance of two powerful deep neural networks. We conduct extensive simulations for performance evaluation. Simulation results have shown that DRL-EC3significantly and consistently outperform two commonly used baseline methods in terms of coverage, fairness, and energy consumption. [ABSTRACT FROM AUTHOR]