Search

In this paper, we investigate the problem of wireless service provisioning through a rotary-wing UAV which can serve as an aerial base station (BS) to communicate with multiple ground terminals (GTs) in a boost demand area. Our objective is to optimize the UAV control for maximizing the UAV.s energy efficiency, in which both aerodynamic energy and communication energy are considered while ensuring the communication requirements for each GT and backhaul link between the UAV and the terrestrial BS. The mobility of the UAV and GTs lead to time-varying channel conditions that make the environment dynamic. We formulate a nonconvex optimization for controlling the UAV considering the practical angle-dependent Rician fading channels between the UAV and GTs, and between the UAV and the terrestrial BS. Traditional optimization approaches are not able to handle the dynamic environment and high complexity of the problem in real-time. We propose to use a deep reinforcement learning-based approach namely Deep Deterministic Policy Gradient (DDPG) to solve the formulated nonconvex problem of UAV control with continuous action space that takes into account the real-time of the environment including time-varying UAV-ground channel conditions, available onboard energy of the UAV, and the communication requirement of the GTs. However, the DDPG method may not achieve good performance in an unstable environment and will face a large number of hyperparameters. We extend our approach to use the Trust Region Policy Optimization (TRPO) method that can improve the performance of the UAV compared to the DDPG method in such a dynamic environment. [ABSTRACT FROM AUTHOR]

The renewable energy source (RES)-powered small cell base station (SBS) is a promising technology for the next-generation networks because RESs provide sustainable energy without being depleted. In RES-assisted networks, joint optimization of user association (UA) and power control (PC) is required to enhance the sum-rate performance by reducing inter-cell interference between SBSs. This paper tackles the UA and PC optimization for the sum-rate maximization under quality-of-service (QoS) and backhaul constraints. We formulate the problem as mixed-integer non-linear programming, in which UA and PC variables of different time slots are tightly coupled due to the RES energy dynamics model. Hence, designing a dynamic policy-based UA and PC with consideration of the future environments such as the quantity of channel gain and energy harvesting remains a challenge. First, we propose a deep unsupervised learning (DUL)-based UA scheme for a fixed PC variable. To lower the computational complexity and accelerate the convergence of the proposed learning-based optimization, we relax the UA variable, which originally has an extremely high dimension, into a low-dimensional continuous variable inspired by the Lagrangian method. Next, we propose a deep reinforcement learning (DRL)-based PC scheme, in which the stringent QoS and backhaul constraints are considered penalty terms on the reward design. The proposed DRL-based PC scheme facilitates dynamic PC inferring the relationship between the future environment and current PC. Simulation results demonstrate that the proposed scheme enhances the sum-rate by 10%, accommodates 3.3 percent point (%p) more QoS-qualified users, and reduces the computation time by 20 times compared to the conventional optimization-based method. [ABSTRACT FROM AUTHOR]

In the multi-server multi-access edge computing (MEC) system, the computing capacity could be insufficient and the computing resource could be over-utilized when an excessive number of computing tasks are offloaded for execution. To alleviate the excessive burden in the multi-server MEC system and tap the underutilized resources of wired edge devices, which are called edge computers in this paper, we investigate the computation offloading and shunting scheme in the wireless wireline internetwork. We formulate an optimization problem to minimize the average total time delay and use the deep reinforcement learning method to obtain the optimal shunting policy. By sensing the state of the wireless wireline internetwork, the proposed scheme can adaptively adjust the shunting ratio and utilize the resources of MEC servers and edge computers intelligently and jointly. Finally, we evaluate the computational complexity and validate the convergence property of this scheme. The extensive simulation results demonstrate that this scheme can improve the resource productivity and reduce the average total time delay. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the adaptive video delivery for multiple users over time-varying and mutually interfering multi-cell wireless networks. The key research challenge is to jointly design the physical-layer resource allocation scheme and application-layer rate adaptation logic, such that the users’ long-term fair quality of experience (QoE) can be maximized. Due to the timescale mismatch between these two layers and the asynchrony of user requests, however, it is difficult to directly model the cross-layer stochastic control problem by using a reinforcement learning framework. To address this difficulty, we propose a novel two-level decision framework where an optimization-based beamforming scheme (performed at the base stations) and a deep reinforcement learning (DRL)-based rate adaptation scheme (performed at the user terminals) are, respectively, developed, such that a highly complex long-term multi-user QoE fairness problem is decomposed into some relatively simple problems and solved effectively. Our strategy represents a significant departure from the existing schemes with consideration of either a short-term multi-user QoE maximization or a long-term single-user point-to-point QoE maximization. Extensive simulations demonstrate that the proposed cross-layer design is effective and promising. [ABSTRACT FROM AUTHOR]

This paper studies multi-agent deep reinforcement learning (MADRL) based resource allocation methods for multi-cell wireless powered communication networks (WPCNs) where multiple hybrid access points (H-APs) wirelessly charge energy-limited users to collect data from them. We design a distributed reinforcement learning strategy where H-APs individually determine time and power allocation variables. Unlike traditional centralized optimization algorithms which require global information collected at a central unit, the proposed MADRL technique models an H-AP as an agent producing its action based only on its own locally observable states. Numerical results verify that the proposed approach can achieve comparable performance of the centralized algorithms. [ABSTRACT FROM AUTHOR]

We propose a learning-based scheme to investigate the dynamic multi-channel access (DMCA) problem in the fifth generation (5G) and beyond networks with fast time-varying channels wherein the channel parameters are unknown. The proposed learning-based scheme can maintain near-optimal performance for a long time, even in the sharp changing channels. This scheme greatly reduces processing delay, and effectively alleviates the error due to decision lag, which is cased by the non-immediacy of the information acquisition and processing. We first propose a psychology-based personalized quality of service model after introducing the network model with unknown channel parameters and the streaming model. Then, two access criteria are presented for the living streaming model and the buffered streaming model. Their corresponding optimization problems are also formulated. The optimization problems are solved by learning-based DMCA scheme, which combines the recurrent neural network with deep reinforcement learning. In the learning-based DMCA scheme, the agent mainly invokes the proposed prediction-based deep deterministic policy gradient algorithm as the learning algorithm. As a novel technical paradigm, our scheme has strong universality, since it can be easily extended to solve other problems in wireless communications. The real channel data-based simulation results validate that the performance of the learning-based scheme approaches that derived from the exhaustive search when making a decision at each time-slot, and is superior to the exhaustive search method when making a decision at every few time-slots. [ABSTRACT FROM AUTHOR]