Search

In Vehicular Ad Hoc Networks, reliable information transmission relies on an effective routing strategy. Most existing reinforcement learning-based routing methods are ineffective in dynamic network environments and cannot prevent inefficient network routing. Efficient network routing can be controlled by network traffic management, so this paper proposes an intelligent routing strategy based on Deep Reinforcement Learning to enhance routing performance. By integrating intersection forwarding and traffic awareness capabilities, this paper addresses the problem of local optimality and utilizes the Deep Q Network to make intersection forwarding decisions. The state space of this strategy consists of intersection nodes, road information between intersections, and forwarding packet information. When a vehicle node carrying a packet approaches an intersection based on the state space, the intersection node uses a neural network to select the optimal next-hop relay intersection from past learning experiences. It generates appropriate vehicle routing decisions based on information from the current and candidate relay intersections. Finally, we use the real taxi trajectory data of Beijing City to conduct extensive simulation experiments. Simulation results and analysis demonstrate that the proposed strategy outperforms related research regarding higher average packet delivery ratio, shorter average end-to-end delay, and lower average overhead ratio in dense and sparse traffic periods under real road environments. Consequently, this strategy provides efficient and reliable message transmission services for Vehicular Ad Hoc Networks. [ABSTRACT FROM AUTHOR]

With the development of artificial intelligence technology, deep reinforcement learning (DRL) has become a major approach to the design of intelligent vehicle-to-everything (V2X) routing protocols for vehicular ad hoc networks (VANETs). However, if the V2X routing protocol does not consider both real-time traffic conditions and historical vehicle trajectory information, the source vehicle may not transfer its packet to the correct relay vehicles and, finally, to the destination. Thus, this kind of routing protocol fails to guarantee successful packet delivery. Using the greater network flexibility and scalability of the software-defined network (SDN) architecture, this study designs a two-phase integrated DQN and RF Packet Routing Framework (IDRF) that combines the deep Q-learning network (DQN) and random forest (RF) approaches. First, the IDRF offline phase corrects the vehicle's historical trajectory information using the vehicle trajectory continuity algorithm and trains the DQN model. Then, the IDRF real-time phase judges whether vehicles can meet each other and makes a real-time routing decision to select the most appropriate relay vehicle after adding real-time vehicles to the VANET. In this way, the IDRF can obtain the packet transfer path with the shortest end-to-end delay. Compared to two DQN-based approaches, i.e., TDRL-RP and VRDRT, and traditional VANET routing algorithms, the IDRF exhibits significant performance improvements for both sparse and congested periods during intensive simulations of the historical GPS trajectories of 10,357 taxis within Beijing city. Performance improvements in the average packet delivery ratio, end-to-end delay, and overhead ratio of the IDRF over TDRL-RP and VRDRT under different numbers of pairs and transmission ranges are at least 3.56%, 12.73%, and 5.14% and 6.06%, 11.84%, and 7.08%, respectively. [ABSTRACT FROM AUTHOR]

Dealing with the packet-routing problem is challenging in the V2X (Vehicle-to-Everything) network environment, where it suffers from the high mobility of vehicles and varied vehicle density at different times. Many related studies have been proposed to apply artificial intelligence models, such as Q-learning, which is a well-known reinforcement learning model, to analyze the historical trajectory data of vehicles and to further design an efficient packet-routing algorithm for V2X. In order to reduce the number of Q-tables generated by Q-learning, grid-based routing algorithms such as the QGrid have been proposed accordingly to divide the entire network environment into equal grids. This paper focuses on improving the defects of these grid-based routing algorithms, which only consider the vehicle density of each grid in Q-learning. Hence, we propose a Software-Defined Directional QGrid (SD-QGrid) routing platform in this paper. By deploying an SDN Control Node (CN) to perform centralized control for V2X, the SD-QGrid considers the directionality from the source to the destination, real-time positions and historical trajectory records between the adjacent grids of all vehicles. The SD-QGrid further proposes the flows of the offline Q-learning training process and the online routing decision process. The two-hop trajectory-based routing (THTR) algorithm, which depends on the source–destination directionality and the movement direction of the vehicle for the next two grids, is proposed as a vehicle node to forward its packets to the best next-hop neighbor node in real time. Finally, we use the real vehicle trajectory data of Taipei City to conduct extensive simulation experiments with respect to four transmission parameters. The simulation results prove that the SD-QGrid achieved an over 10% improvement in the average packet delivery ratio and an over 25% reduction in the average end-to-end delay at the cost of less than 2% in average overhead, compared with two well-known Q-learning grid-based routing algorithms. [ABSTRACT FROM AUTHOR]