Information upwards, recommendation downwards: reinforcement learning with hierarchy for traffic signal control.

Traffic signal control (TSC) is a practical solution to the major problem of congestion in metropolitan areas. Reinforcement Learning (RL) techniques present powerful frameworks for optimizing traffic signal controllers that learn to respond to real-time traffic changes. Multiagent RL (MARL) techniques have been showing better results over centralized techniques (RL-based or not), where local intersection agents have partial observation of and control over the environment. Since in TSC the best decision does not depend only on local information, in the present paper we aim at increasing agents' views by using a hierarchical approach, where information is passed upwards, is then aggregated forming recommendations that are sent downwards. We divide the transportation network into regions, each controlled by a region agent; this is done at different hierarchical levels. The traffic signal controllers, located at the intersections, are the local agents at the hierarchy's bottom. Region agents can supervise intersection agents or other region agents. Evaluation of this approach in a synthetic traffic grid shows that the proposed hierarchical organization outperforms a fixed-time approach and an RL-based approach without hierarchy. [ABSTRACT FROM AUTHOR]