Search

In ocean observation missions, unmanned autonomous ocean observation platforms play a crucial role, with precise target detection technology serving as a key support for the autonomous operation of unmanned platforms. Among various underwater sensing devices, side-scan sonar (SSS) has become a primary tool for wide-area underwater detection due to its extensive detection range. However, current research on target detection with SSS primarily focuses on large targets such as sunken ships and aircraft, lacking investigations into small targets. In this study, we collected data on underwater small targets using an unmanned boat equipped with SSS and proposed an enhancement method based on the YOLOv7 model for detecting small targets in SSS images. First, to obtain more accurate initial anchor boxes, we replaced the original k-means algorithm with the k-means++ algorithm. Next, we replaced ordinary convolution blocks in the backbone network with Omni-dimensional Dynamic Convolution (ODConv) to enhance the feature extraction capability for small targets. Subsequently, we inserted a Global Attention Mechanism (GAM) into the neck network to focus on global information and extract target features, effectively addressing the issue of sparse target features in SSS images. Finally, we mitigated the harmful gradients produced by low-quality annotated data by adopting Wise-IoU (WIoU) to improve the detection accuracy of small targets in SSS images. Through validation on the test set, the proposed method showed a significant improvement compared to the original YOLOv7, with increases of 5.05% and 2.51% in mAP@0.5 and mAP@0.5: 0.95 indicators, respectively. The proposed method demonstrated excellent performance in detecting small targets in SSS images and can be applied to the detection of underwater mines and small equipment, providing effective support for underwater small target detection tasks.

In the field of underwater perception and detection, side-scan sonar (SSS) plays an indispensable role. However, the imaging mechanism of SSS results in slow information acquisition and high complexity, significantly hindering the advancement of downstream data-driven applications. To address this challenge, we designed an SSS image generator based on diffusion models. We developed a data collection system based on Autonomous Underwater Vehicles (AUVs) to achieve stable and rich data collection. For the process of converting acoustic signals into image signals, we established an image compensation method based on nonlinear gain enhancement to ensure the reliability of remote signals. On this basis, we developed the first controllable category SSS image generation algorithm, which can generate specified data for five categories, demonstrating outstanding performance in terms of the Fréchet Inception Distance (FID) and the Inception Score (IS). We further evaluated our image generator in the task of SSS object detection, and our cross-validation experiments showed that the generated images contributed to an average accuracy improvement of approximately 10% in object detection. The experimental results validate the effectiveness of the proposed SSS image generator in generating highly similar sonar images and enhancing detection accuracy, effectively addressing the issue of data scarcity.

Side-scan sonar plays a crucial role in underwater exploration, and the autonomous detection of side-scan sonar images is vital for detecting unknown underwater environments. However, due to the complexity of the underwater environment, the presence of a few highlighted areas on the targets, blurred feature details, and difficulty in collecting data from side-scan sonar, achieving high-precision autonomous target recognition in side-scan sonar images is challenging. This article addresses this problem by improving the You Only Look Once v7 (YOLOv7) model to achieve high-precision object detection in side-scan sonar images. Firstly, given that side-scan sonar images contain large areas of irrelevant information, this paper introduces the Swin-Transformer for dynamic attention and global modeling, which enhances the model’s focus on the target regions. Secondly, the Convolutional Block Attention Module (CBAM) is utilized to further improve feature representation and enhance the neural network model’s accuracy. Lastly, to address the uncertainty of geometric features in side-scan sonar target features, this paper innovatively incorporates a feature scaling factor into the YOLOv7 model. The experiment initially verified the necessity of attention mechanisms in the public dataset. Subsequently, experiments on our side-scan sonar (SSS) image dataset show that the improved YOLOv7 model has 87.9% and 49.23% in its average accuracy (mAP0.5) and (mAP0.5:0.95), respectively. These results are 9.28% and 8.41% higher than the YOLOv7 model. The improved YOLOv7 algorithm proposed in this paper has great potential for object detection and the recognition of side-scan sonar images.

Underwater vehicles heavily depend on the integration of inertial navigation with Doppler Velocity Log (DVL) for fusion-based localization. Given the constraints imposed by sensor costs, ensuring the optimization ability and robustness of fusion algorithms is of paramount importance. While filtering-based techniques such as Extended Kalman Filter (EKF) offer mature solutions to nonlinear problems, their reliance on linearization approximation may compromise final accuracy. Recently, Invariant EKF (IEKF) methods based on the concept of smooth manifolds have emerged to address this limitation. However, the optimization by matrix Lie groups must satisfy the “group affine” property to ensure state independence, which constrains the applicability of IEKF to high-precision positioning of underwater multi-sensor fusion. In this study, an alternative state-independent underwater fusion invariant filtering approach based on a two-frame group utilizing DVL, Inertial Measurement Unit (IMU), and Earth-Centered Earth-Fixed (ECEF) configuration is proposed. This methodology circumvents the necessity for group affine in the presence of biases. We account for inertial biases and DVL pole-arm effects, achieving convergence in an imperfect IEKF by either fixed observation or body observation information. Through simulations and real datasets that are time-synchronized, we demonstrate the effectiveness and robustness of the proposed algorithm.

Accurate underwater target detection is crucial for the operation of autonomous underwater vehicles (AUVs), enhancing their environmental awareness and target search and rescue capabilities. Current deep learning-based detection models are typically large, requiring substantial storage and computational resources. However, the limited space on AUVs poses significant challenges for deploying these models on the embedded processors. Therefore, research on model compression is of great practical importance, aiming to reduce model parameters and computational load without significantly sacrificing accuracy. To address the challenge of deploying large detection models, this paper introduces an automated pruning method based on dependency graphs and successfully implements efficient pruning on the YOLOv7 model. To mitigate the accuracy degradation caused by extensive pruning, we design a hybrid distillation method that combines output-based and feature-based distillation techniques, thereby improving the detection accuracy of the pruned model. Finally, we deploy the compressed model on an embedded processor within an AUV to evaluate its performance. Multiple experiments confirm the effectiveness of our proposed method in practical applications.

In the realm of underwater environment detection, achieving information matching stands as a pivotal step, forming an indispensable component for collaborative detection and research in areas such as distributed mapping. Nevertheless, the progress in studying the matching of underwater side-scan sonar images has been hindered by challenges including low image quality, intricate features, and susceptibility to distortion in commonly used side-scan sonar images. This article presents a comprehensive overview of the advancements in underwater sonar image processing. Building upon the novel SchemaNet image topological structure extraction model, we introduce a feature matching model grounded in side-scan sonar images. The proposed approach employs a semantic segmentation network as a teacher model to distill the DeiT model during training, extracting the attention matrix of intermediate layer outputs. This emulates SchemaNet’s transformation method, enabling the acquisition of high-dimensional topological structure features from the image. Subsequently, utilizing a real side-scan sonar dataset and augmenting data, we formulate a matching dataset and train the model using a graph neural network. The resulting model demonstrates effective performance in side-scan sonar image matching tasks. These research findings bear significance for underwater detection and target recognition and can offer valuable insights and references for image processing in diverse domains.

IntroductionAutonomous underwater vehicles (AUVs) are required to thoroughly scan designated areas during underwater missions. They typically follow a zig-zag trajectory to achieve full coverage. However, effective coverage can be challenging in complex environments due to the accumulation and drift of navigation errors. Possible solutions include surfacing for satellite positioning or underwater acoustic positioning using transponders on other vehicles. Nevertheless, surfacing or active acoustics can compromise stealth during reconnaissance missions in hostile areas by revealing the vehicle’s location.MethodsWe propose calibration and planning strategies based on error models and acoustic positioning to address this challenge. Acoustic markers are deployed via surface ships to minimize navigation errors while maintaining stealth. And a new path planning method using a traceless Kalman filter and acoustic localization is proposed to achieve full-area coverage of AUVs. By analyzing the statistics of accumulated sensor errors, we optimize the positions of acoustic markers to communicate with AUVs and achieve better coverage. AUV trajectory concealment is achieved during detection by randomizing the USV navigation trajectory and irregularizing the locations of acoustic marker.ResultsThe proposed method enables the cumulative determination of the absolute position of a target with low localization error in a side-scan sonar-based search task. Simulations based on large-scale maps demonstrate the effectiveness and robustness of the proposed algorithm.DiscussionSolving the problem of accumulating underwater localization errors based on inertial navigation by error modeling and acoustic calibration is a typical way. In this paper, we have implemented a method to solve the localization error in a search scenario where stealth is considered.

Deep learning-based object detection methods have demonstrated remarkable effectiveness across various domains. Recently, there has been growing interest in applying these techniques to underwater environments. Conventional optical imaging methods face severe limitations when operating in underwater conditions, restricting their ability to identify objects with good visibility and at close distances. Consequently, side-scan sonar (SSS) has emerged as a common equipment choice for underwater detection due to its compatibility with the characteristics of sound waves in water. This paper introduces a novel method, termed the Enhanced YOLOv7-Based Approach, for detecting small objects in SSS images. Building upon the widely-adopted YOLOv7 method, the proposed approach incorporates several enhancements aimed at improving detection accuracy. First, a dedicated detection layer tailored for small objects is added to the original network architecture. Additionally, two attention mechanisms are integrated within the backbone and neck components of the network, respectively, to strengthen the network’s focus on object features. Finally, the network features are recombined based on the BiFPN structure. Experimental results demonstrate that the proposed method outperforms mainstream object detection algorithms. In comparison to the original YOLOv7 network, it achieves a Precision of 95.5%, indicating a significant improvement of 4.8%. Moreover, its Recall reaches 87.0%, representing an enhancement of 5.1%, while the mean Average Precision (mAP) at an IoU threshold of 0.5 (mAP@.5) reaches 86.9%, reflecting a 6.7% improvement. Furthermore, the mAP@.5:.95 reaches 55.1%, a 4.8% enhancement. Therefore, the method presented in this paper enhances the performance of YOLOv7 for object detection in SSS images, providing a fresh perspective on small object detection based on SSS images and contributing to the advancement of underwater detection techniques.

Multi-vehicle collaborative mapping proves more efficient in constructing maps in unfamiliar underwater environments in comparison to single-vehicle methods. One of the pivotal hurdles of Simultaneous Localization and Mapping (SLAM) with multiple underwater vehicles is map registration. Due to the inadequate characteristics of the underwater grid maps, matching map features poses a challenge, and outliers between maps add to the complexity. We propose an algorithm to solve this problem. This approach employs the Gaussian Mixture Robust Branch and Bound (GMRBnB) algorithm with an interior point filtering technique. Feature point extraction, registration using the GMRBnB algorithm, inlier extraction based on density, and registration of the inlier are performed to obtain a more precise transformation matrix. The results of the simulation and experiments demonstrate that this technique heightens outlier tolerance and reinforces map registration accuracy. The proposed approach surpasses Iterative Closest Point (ICP) and Normal Distributions Transform (NDT) methods with respect to map registration quality.

In the realm of underwater exploration, particularly within the domain of autonomous detection, sidescan sonar stands as a pivotal sensor apparatus. Autonomous detection models necessitate a substantial volume of scanned sonar image data for optimal training, yet the challenges and costs associated with acquiring such data pose significant limitations on the deployment of autonomous detection models in underwater exploration scenarios. Consequently, there arises a demand for the development of cost-effective data augmentation techniques. In the present investigation, an initial collection of scanned sonar image data was conducted during lake trials, encompassing diverse environmental regions, including rocky terrain, shadowed areas, and aquatic bodies. Subsequently, a proprietary generative adversarial network (GAN) model was devised for the purpose of synthesizing scanned sonar data. The synthesized data underwent denoising and underwent post-processing via algorithmic methods. Subsequently, similarity metrics were computed to gauge the quality of the generated scanned sonar data. Furthermore, a semantic segmentation model was meticulously crafted and trained by employing authentic data. The generated data were subsequently introduced into this semantic segmentation model. The output outcomes demonstrated that the model exhibited preliminary labeling proficiency on the generated image data, requiring only minimal manual intervention to conform to the standards of a conventional dataset. Following the inclusion of the labeled data into the original dataset and the subsequent training of the network model utilizing the expanded dataset, there was an observed discernible enhancement in the segmentation performance of the model.

Underwater target detection technology plays a crucial role in the autonomous exploration of underwater vehicles. In recent years, significant progress has been made in the field of target detection through the application of artificial intelligence technology. Effectively applying AI techniques to underwater target detection is a highly promising area of research. However, the difficulty and high cost of underwater acoustic data collection have led to a severe lack of data, greatly restricting the development of deep-learning-based target detection methods. The present study is the first to utilize diffusion models for generating underwater acoustic data, thereby effectively addressing the issue of poor detection performance arising from the scarcity of underwater acoustic data. Firstly, we place iron cylinders and cones underwater (simulating small preset targets such as mines). Subsequently, we employ an autonomous underwater vehicle (AUV) equipped with side-scan sonar (SSS) to obtain underwater target data. The collected target data are augmented using the denoising diffusion probabilistic model (DDPM). Finally, the augmented data are used to train an improved YOLOv7 model, and its detection performance is evaluated on a test set. The results demonstrate the effectiveness of the proposed method in generating similar data and overcoming the challenge of limited training sample data. Compared to models trained solely on the original data, the model trained with augmented data shows a mean average precision (mAP) improvement of approximately 30% across various mainstream detection networks. Additionally, compared to the original model, the improved YOLOv7 model proposed in this study exhibits a 2% increase in mAP on the underwater dataset.

The AUV (Autonomous Underwater Vehicle) navigation process relies on the interaction of a variety of sensors. The side-scan sonar can collect underwater images and obtain semantic underwater environment information after processing, which will help improve the ability of AUV autonomous navigation. However, there is no practical method to utilize the semantic information of side scan sonar image. A new convolutional neural network model is proposed to solve this problem in this paper. The model is a standard codec structure, which extracts multi-channel features from the input image and then fuses them to reduce parameters and strengthen the weight of feature channels. Then, a larger convolution kernel is used to extract the features of large-scale sonar images more effectively. Finally, a parallel compensation link with a small-scale convolution kernel is added and spliced with features extracted from a large convolution kernel in the decoding part to obtain features of different scales. We use this model to conduct experiments on self-collected sonar data sets, which were uploaded on github. The experimental results show that ACC and MIoU reach 0.87 and 0.71, better than other classical small-order semantic segmentation networks. Furthermore, the 347.52 g FOLP and the number of parameters around 13 m also ensure the computing speed and portability of the network. The result can extract the semantic information of the side-scan sonar image and assist with AUV autonomous navigation and mapping.

Path planning obtains the trajectory from one point to another with the robot's kinematics model and environment understanding. However, as the localization uncertainty through the odometry sensors is inevitably affected, the position of the moving path will deviate further and further compared to the original path, which leads to path drift in GPS denied environments. This article proposes a novel path planning algorithm based on Dijkstra to address such issues. By combining statistical characteristics of localization error caused by dead-reckoning, the replanned path with minimum cumulative error is generated with uniforming distribution in the searching space. The simulation verifies the effectiveness of the proposed algorithm. In a real scenario with measurement noise, the results of the proposed algorithm effectively reduce cumulative error compared to the results of the conventional planning algorithm.

SLAM (Simultaneous Localization And Mapping) plays a vital role in navigation tasks of AUV (Autonomous Underwater Vehicle). However, due to a vast amount of image sonar data and some acoustic equipment's inherent high latency, it is a considerable challenge to implement real-time underwater SLAM on a small AUV. This paper presents a filter based methodology for SLAM algorithms in underwater environments. First, a multi-beam forward looking sonar (MFLS) is utilized to extract environmental features. The acquired sonar image is then converted to sparse point cloud format through threshold segmentation and distance-constrained filtering to solve the calculation explosion issue caused by a large amount of original data. Second, based on the proposed method, the DVL, IMU, and sonar data are fused, the Rao-Blackwellized particle filter (RBPF)-based SLAM method is used to estimate AUV pose and generate an occupancy grid map. To verify the proposed algorithm, the underwater vehicle is equipped as an experimental platform to conduct field tasks in both the experimental pool and wild lake, respectively. Experiments illustrate that the proposed approach achieves better performance in both state estimation and suppressing divergence.

Path planning is often considered as an important task in autonomous driving applications. Current planning method only concerns the knowledge of robot kinematics, however, in GPS denied environments, the robot odometry sensor often causes accumulated error. To address this problem, an improved path planning algorithm is proposed based on reinforcement learning method, which also calculates the characteristics of the cumulated error during the planning procedure. The cumulative error path is calculated by the map with convex target processing, while modifying the algorithm reward and punishment parameters based on the error estimation strategy. To verify the proposed approach, simulation experiments exhibited that the algorithm effectively avoid the error drift in path planning.