Dynamic scheduling for ultra reliable and low latency communications

Ultra-reliable and low-latency communications (URLLC) is one of the main services offered by the current and next generation of wireless communication networks. It usually supports mission-critical applications, such as wireless control and automation in industrial environments, tactile internet, and autonomous driving. Balancing the internal trade-off between low-latency and high-reliability requirements is a challenge for such applications. The limitation in radio resource brings the scheduling of URLLC more challenging when it coexists with enhanced mobile broadband (eMBB) services. Besides, under the assumption of channel distribution uncertainty, the current reliability measures such as average and outage block-error rate are random due to random fluctuations in the estimation errors of channel distribution as a result of limited data availability. Thus, new deterministic measures are required, and the URLLC packet design, i.e., the blocklength optimization, needs to be investigated to minimize statistical measures of URLLC reliability. These are the motivations behind this thesis, which focuses on 1) the dynamic scheduling of URLLC when coexisting with eMBB services and 2) the URLLC blocklength optimization under channel distribution uncertainty. The contribution of this work is fourfold. First, we propose a queuing-based dynamic scheduling for the joint eMBB/URLLC system to address the shortcomings of the instant scheduling policy. In more detail, when there is a demand for URLLC service, the base station suspends the ongoing eMBB traffic with a probability and prioritizes the URLLC transmission. Instead of assuming that the incoming URLLC packets could only either be served or dropped immediately at the beginning of each mini-slot, in the proposed queuing-based scheduling policy, the reserved URLLC packets can wait in the queue until the URLLC latency boundary. The URLLC queue mechanism monitors and updates the latency of each URLLC packet in real-time to ensure the extreme URLLC requirements. Then, a mathematical problem is proposed and solved to optimize the eMBB allocation factor and the URLLC puncturing weights on eMBB channels. Compared to the instant scheduling policy, our algorithm significantly reduces the URLLC loss rate, ensuring that eMBB throughput is not affected. Second, we design a new queuing scheme with a single queue for the joint eMBB/URLLC system. The aim is to flexibly schedule URLLC traffic to enhance the total eMBB throughput and the reliability of URLLC packets (i.e., the probability of not dropping URLLC packets in each mini-slot) while maintaining a satisfactory transmission latency as per the 3GPP requirements. Precisely, by deriving the steady-state probabilities of URLLC queue backlog analytically, we formulate a stochastic optimization problem to maximize the total normalized eMBB throughput and the URLLC utility. Due to the stochastic nature of the objective function, it is expensive to evaluate it for any set of inputs, and thus the Bayesian optimization is applied to obtain the optimal results of such a black-box objective function. Besides, the new queuing scheme with a single queue enhances the flexibility of the previous proposed queueing-based scheduling policy. In more detail, in the first work, we consider that each transmission channel has a particular URLLC queue, and the packet arrivals are evenly distributed between different queues. The base station makes the puncturing decision of each eMBB channel separately in turns at the beginning of each time slot. However, the even distribution of packet arrivals and separate decision making of different channels would lead to more drops depending on the channel conditions. The numerical results demonstrate that the dynamic scheduling of the multi-channel system under the new queuing mechanism could be managed jointly and hence more effectively and flexibly. Third, a risk-sensitive deep reinforcement learning (DRL) is studied to learn the optimal joint scheduling policy in a dynamic multiplexing scenario of eMBB and URLLC. The existing risk-neutral DRL based eMBB/URLLC scheduling algorithms only optimize the expected return and are not risk-sensitive. Robust handling of uncertainties and risks is a must for developing URLLC mission-critical applications in the real world to explicitly avoid the occurrences of catastrophic scheduling policies. We takes a step toward in achieving this goal by proposing a risk-sensitive DRL based algorithm that considers the conditional Value-at-Risk (CVaR) as the risk criterion to optimize. The reward is defined as the normalized eMBB throughput and the probability of not dropping URLLC packets given the state and action. A URLLC queuing mechanism is considered to improve the URLLC reliability and eMBB throughput compared to the instant scheduling policy. Our architecture is based on the actor-critic model but considering a transfer function to obtain a feasible solution of the unconstrained actor neural network, and the critic predicts the entire distribution over future returns instead of simply the expectation. Numerical results show that, under different levels of risk tolerance, the proposed risk-sensitive DRL based algorithm approaches a similar expected return, but the predicted uncertainty is halved compared to the risk-neutral DRL approach. Fourth, in the case of limited knowledge of the channel distribution, achieving the URLLC reliability target requires precise channel modeling. We apply the non-parametric statistical learning approach to estimate the probability density function of the channel distribution. We also propose and optimize a statistical measure as a way to surely assess reliability in finite-block communications regime. In particular, the confidence level in guaranteeing average block-error rate lower than a specific target is introduced and maximized to find the optimal blocklength, aiming to meet the strict requirements of URLLC.