Deep Reinforcement Learning-based Beam Tracking from mmWave Antennas Installed on Overhead Messenger Wires

To achieve reliable small cell millimeter-wave wireless backhauls, this study installs small cell base stations (SBSs) on overhead messenger wires to gain flexibility in physical deployments of SBSs ensuring in the line-of-sight connections between SBSs and gateway BSs. These installations pose challenges in aligning directional beams, whereby complicated wind-forced dynamics in on-wire SBSs require frequent beam training, and consequently, a large signaling overhead. To address this, this study aims at demonstrating the feasibility of learning-based beam tracking where a beam tracking policy is learned a priori to fix beam misalignment caused by the wind-forced dynamics. Because wind-forced dynamics in SBSs can be three-dimensional (3D), the proposed beam tracking newly exploits the 3D position/velocity of the SBS as state information. As a solution to fix beam misalignment, the beam tracking policy is learned via deep reinforcement learning wherein the 3D information and beam direction are regarded as a state and an action, respectively, and the received signal power at a gateway BS is maximized. The simulation results depict the feasibility of learning an appropriate beam tracking policy to prevent beam misalignment induced by wind-forced 3D dynamics in on-wire SBSs.