The crowd cooperation approach for formation maintenance and collision avoidance using multi-agent deep reinforcement learning.

Formation maintenance and collision avoidance allow multiple agents to form and maintain a specific formation in the environment and reach the final goal safely without any collisions, which are important component of the multi-agent/crowd cooperation field. We propose a two-level formation control method to simulate crowd formation. At the low level, a comprehensive group reward function is designed to train the agents to learn to collaborate to reach the goal and avoid collisions with other agents and obstacles. The Multi-Agent POsthumous Credit Assignment algorithm and curriculum learning are combined to solve the credit assignment problem in the shared group reward and the slow convergence problem of the policy in the reward-sparse environment. At the high level, we design a formation switch method to determine the compression or splitting of the formation for multiple agents passing through the narrow exit. Then the optimal reciprocal collision avoidance algorithm is adopted to compute a collision-free velocity for the team of multiple agents, on the basis of which the desired formation positions are calculated to avoid collisions between the team and the obstacles in the environment. Lastly, Hungarian algorithm is utilized to assign the formation positions to the agents as their respective target position to accelerate the process of forming. Experimental results demonstrate that our proposed approach enable multiple agents to constitute the formation shape from the starting positions quickly and maintain the shape well when moving toward the final goal. Furthermore, our model has a certain generalization ability to unseen formation shapes in the scenarios without and with obstacles. [ABSTRACT FROM AUTHOR]