Evolutionary End-to-End Autonomous Driving Model With Continuous-Time Neural Networks

The end-to-end paradigm has gained considerable attention in autonomous driving due to its anticipated performance. However, prevailing end-to-end paradigms predominantly employ one-shot training using imitation learning, resulting in models lacking evolutionary capabilities and struggling with long-tail scenarios. Furthermore, addressing these long-tail scenarios necessitates end-to-end models to simultaneously exhibit the generalizability of environmental representations and the robustness of control policies. Therefore, this paper proposes an end-to-end autonomous driving model called GPCT, using a Generative Perception network and a Continuous-Time brain neural network, with a Policy-Reward-Data-Aggregation (PRDA) mechanism. Specifically, the generative perception network extracts perceptual information from monocular camera inputs and undergoes distribution fitting and sampling to obtain environmental dynamics information. Subsequently, the sequential environmental dynamics information is fed into continuous-time brain neural networks to output the control information. The end-to-end model is then applied to on-policy scenarios using the PRDA mechanism to collect data for further training and evolution. Data is collected within the Carla simulator, followed by model training, and the utilization of a multi-round PRDA mechanism for data collection and training to facilitate model evolution. The algorithm's performance improves by 63.85% after five evolution experiments. In the transfer experiments, the proposed algorithm achieves a route completion rate close to 100% and maintains a driving score of around 60%, even surpassing the performance of systems equipped with multiple cameras and LiDAR. Furthermore, under heavy fog conditions, the route completion rate remains at 85%, showcasing generalizability and robustness.