ASTPPO: A proximal policy optimization algorithm based on the attention mechanism and spatio–temporal correlation for routing optimization in software-defined networking.

Currently, existing research on deploying deep reinforcement learning on software-defined networks (SDN) to achieve route optimization does not consider the network's spatial–temporal correlation globally and has yet to reach the ultimate in performance. Given the above issues, this study proposes a Proximal Policy Optimization algorithm based on the Attention mechanism and Spatio–Temporal correlation (ASTPPO) to optimize the SDN routing issue. First, we extract temporal and spatial correlation features in state information using Gated Recurrent Units (GRU) and Graph Attention Networks (GAT), providing implicit information containing more environments for reinforcement learning decisions. Second, we use the skip-connect method to connect implicit and directly related information into a multi-layer perceptron, improving the model's learning efficiency and perceptual ability. Finally, we demonstrate the effectiveness of ASTPPO through static and dynamic traffic experiments. Benefitting from Spatio–Temporal correlation learning with a global view, ASTPPO performs better load balancing and congestion control under different traffic intensity requirements and network topologies than other reinforcement learning baseline algorithms. The simulation results show that the ASTPPO algorithm improved by 9.02% and 15.07%, respectively, compared with the second-best algorithm in static and dynamic traffic scenarios. [ABSTRACT FROM AUTHOR]