Graph cooperation deep reinforcement learning for ecological urban traffic signal control.

Cooperation between intersections in large-scale road networks is critical in traffic congestion. Currently, most traffic signals cooperate via pre-defined timing phases, which is extremely inefficient in real-time traffic scenarios. Most existing studies on multi-agent reinforcement learning (MARL) traffic signal control have focused on designing efficient communication methods, but have ignored the importance of how agents interact in cooperative communication. To achieve more efficient cooperation among traffic signals and alleviate urban traffic congestion, this study constructs a Graph Cooperation Q-learning Network Traffic Signal Control (GCQN-TSC) model, which is a graph cooperation network with an embedded self-attention mechanism that enables agents to adjust their attention in real time according to the dynamic traffic flow information, perceive the traffic environment quickly and effectively in a larger range, and help agents achieve more effective collaboration. Moreover, the Deep Graph Q-learning (DGQ) algorithm is proposed in this model to optimize the traffic signal control strategy according to the spatio-temporal characteristics of different traffic scenes and provide the optimal signal phase for each intersection. This study also integrates the ecological traffic concept into MARL traffic signal control, which aims to reduce traffic exhaust emissions. Finally, the proposed GCQN-TSC is experimentally validated both in a synthetic traffic grid and a real-world traffic network using the SUMO simulator. The experimental results show that GCQN-TSC outperforms other traffic signal control methods in almost all performance metrics, including average queue length and waiting time, as it can aggregate information acquired from collaborative agents and make network-level signal optimization decisions. [ABSTRACT FROM AUTHOR]