Search

This paper proposes a robust adversarial reinforcement learning (RARL)-based multi-access point (AP) coordination method that is robust even against unexpected decentralized operations of uncoordinated APs. Multi-AP coordination is a promising technique towards IEEE 802.11be, and there are studies that use RL for multi-AP coordination. Indeed, a simple RL-based multi-AP coordination method diminishes the collision probability among the APs; therefore, the method is a promising approach to improve time-resource efficiency. However, this method is vulnerable to frame transmissions of uncoordinated APs that are less aware of frame transmissions of other coordinated APs. To help the central agent experience even such unexpected frame transmissions, in addition to the central agent, the proposed method also competitively trains an adversarial AP that disturbs coordinated APs by causing frame collisions intensively. Besides, we propose to exploit a history of frame losses of a coordinated AP to promote reasonable competition between the central agent and adversarial AP. The simulation results indicate that the proposed method can avoid uncoordinated interference and thereby improve the minimum sum of the throughputs in the system compared to not considering the uncoordinated AP.

To achieve reliable small cell millimeter-wave wireless backhauls, this study installs small cell base stations (SBSs) on overhead messenger wires to gain flexibility in physical deployments of SBSs ensuring in the line-of-sight connections between SBSs and gateway BSs. These installations pose challenges in aligning directional beams, whereby complicated wind-forced dynamics in on-wire SBSs require frequent beam training, and consequently, a large signaling overhead. To address this, this study aims at demonstrating the feasibility of learning-based beam tracking where a beam tracking policy is learned a priori to fix beam misalignment caused by the wind-forced dynamics. Because wind-forced dynamics in SBSs can be three-dimensional (3D), the proposed beam tracking newly exploits the 3D position/velocity of the SBS as state information. As a solution to fix beam misalignment, the beam tracking policy is learned via deep reinforcement learning wherein the 3D information and beam direction are regarded as a state and an action, respectively, and the received signal power at a gateway BS is maximized. The simulation results depict the feasibility of learning an appropriate beam tracking policy to prevent beam misalignment induced by wind-forced 3D dynamics in on-wire SBSs.