Multi-agent reinforcement learning based joint uplink–downlink subcarrier assignment and power allocation for D2D underlay networks.

This paper investigates the joint uplink–downlink resource allocation in time-varying device-to-device (D2D) underlay wireless cellular networks. Specifically, we formulate the joint optimization problem of the joint uplink–downlink subcarrier assignment and power allocation (SAPA) of D2D pairs, with the purpose of maximizing the sum data rate (SDR) of all D2D pairs while ensuring the basic data rate requirements of both cellular users and D2D pairs. To accommodate the high dynamics of wireless networks, we develop an effective joint uplink-downlink SAPA scheme based on distributed deep reinforcement learning (DRL), wherein each D2D pair acts as an agent and adopts the model-free double-deep Q-network (DDQN) algorithm to solve the joint optimization problem. Moreover, in our proposed DDQN scheme, we assume that all agents maintain the same reward, thus collaborative behavior between agents is inspired to alleviate the mutual interference incurred by subcarrier reuses between the cellular users and D2D pairs. Numerical results show that our proposed DDQN method could quickly converge to the near-optimal performance, has low computational complexity and thus could be adopted in large-scale D2D underlay wireless cellular networks. [ABSTRACT FROM AUTHOR]