Back to Search Start Over

Multiple Unmanned Aerial Vehicle (multi-UAV) Reconnaissance and Search with Limited Communication Range Using Semantic Episodic Memory in Reinforcement Learning

Authors :
Boquan Zhang
Tao Wang
Mingxuan Li
Yanru Cui
Xiang Lin
Zhi Zhu
Source :
Drones, Vol 8, Iss 8, p 393 (2024)
Publication Year :
2024
Publisher :
MDPI AG, 2024.

Abstract

Unmanned Aerial Vehicles (UAVs) have garnered widespread attention in reconnaissance and search operations due to their low cost and high flexibility. However, when multiple UAVs (multi-UAV) collaborate on these tasks, a limited communication range can restrict their efficiency. This paper investigates the problem of multi-UAV collaborative reconnaissance and search for static targets with a limited communication range (MCRS-LCR). To address communication limitations, we designed a communication and information fusion model based on belief maps and modeled MCRS-LCR as a multi-objective optimization problem. We further reformulated this problem as a decentralized partially observable Markov decision process (Dec-POMDP). We introduced episodic memory into the reinforcement learning framework, proposing the CNN-Semantic Episodic Memory Utilization (CNN-SEMU) algorithm. Specifically, CNN-SEMU uses an encoder–decoder structure with a CNN to learn state embedding patterns influenced by the highest returns. It extracts semantic features from the high-dimensional map state space to construct a smoother memory embedding space, ultimately enhancing reinforcement learning performance by recalling the highest returns of historical states. Extensive simulation experiments demonstrate that in reconnaissance and search tasks of various scales, CNN-SEMU surpasses state-of-the-art multi-agent reinforcement learning methods in episodic rewards, search efficiency, and collision frequency.

Details

Language :
English
ISSN :
2504446X
Volume :
8
Issue :
8
Database :
Directory of Open Access Journals
Journal :
Drones
Publication Type :
Academic Journal
Accession number :
edsdoj.599f24f5d14e39a3454c2dc6477dfb
Document Type :
article
Full Text :
https://doi.org/10.3390/drones8080393