Search

Underwater computer vision plays a vital role in ocean research, enabling autonomous navigation, infrastructure inspections, and marine life monitoring. However, the underwater environment presents unique challenges, including color distortion, limited visibility, and dynamic light conditions, which hinder the performance of traditional image processing methods. Recent advancements in deep learning (DL) have demonstrated remarkable success in overcoming these challenges by enabling robust feature extraction, image enhancement, and object recognition. This review provides a comprehensive analysis of cutting-edge deep learning architectures designed for underwater object detection, segmentation, and tracking. State-of-the-art (SOTA) models, including AGW-YOLOv8, Feature-Adaptive FPN, and Dual-SAM, have shown substantial improvements in addressing occlusions, camouflaging, and small underwater object detection. For tracking tasks, transformer-based models like SiamFCA and FishTrack leverage hierarchical attention mechanisms and convolutional neural networks (CNNs) to achieve high accuracy and robustness in dynamic underwater environments. Beyond optical imaging, this review explores alternative modalities such as sonar, hyperspectral imaging, and event-based vision, which provide complementary data to enhance underwater vision systems. These approaches improve performance under challenging conditions, enabling richer and more informative scene interpretation. Promising future directions are also discussed, emphasizing the need for domain adaptation techniques to improve generalizability, lightweight architectures for real-time performance, and multi-modal data fusion to enhance interpretability and robustness. By critically evaluating current methodologies and highlighting gaps, this review provides insights for advancing underwater computer vision systems to support ocean exploration, ecological conservation, and disaster management.

SCoralNet (based on Faster R-CNN) is a new underwater coral detection framework that has been proposed to automatically localize and identify distinct coral species in images, allowing for fast and detailed annotation. Monitoring the coverage and abundance of underwater corals typically involves the annotation and processing of large amounts of underwater coral images. However, manually annotating a large number of images is time-consuming and labor-intensive, and CNN classifiers only provide simple classification annotations without capturing the images' finer details. SCoralNet's detection performance is improved by incorporating dilated convolutions into the backbone network. To successfully capture multi-scale and multi-level information from coral targets, a neck network called NASFPN is placed between the backbone and the detecting head. Seesaw Loss is used to reduce the impact of the dataset's long-tailed distribution on SCoralNet's classifier accuracy. CIoU loss is used to optimize the bounding box regression method. During inference, Soft-NMS is applied to suppress redundant coral detection boxes. To assess SCoralNet's effectiveness, a dataset called Coral-soft was developed using real-world photos of common soft coral species from the Sanya region of China. SCoralNet outperformed the original Faster R-CNN model on the Coral-soft dataset, with a 45.68% gain in mean average precision (mAP) and a 59.2% increase in mAP75. Furthermore, SCoralNet outperformed some advanced models in terms of overall performance. [ABSTRACT FROM AUTHOR]

Underwater computer vision has attracted increasing attention in the research community due to the recent advances in underwater platforms such as of rovers, gliders, autonomous underwater vehicles (AUVs), and the like, that now make possible the acquisition of vast amounts of imagery and video for applications such as biodiversity assessment, environmental monitoring, and search and rescue. Despite growing interest, underwater computer vision is still a relatively under-researched area, where the attention in the literature has been paid to the use of computer vision techniques for image restoration and reconstruction, where image formation models and image processing methods are used to recover colour corrected or enhanced images. This is due to the notion that these methods can be used to achieve photometric invariants to perform higher-level vision tasks such as shape recovery and recognition under the challenging and widely varying imaging conditions that apply to underwater scenes. In this paper, we review underwater computer vision techniques for image reconstruction, restoration, recognition, depth, and shape recovery. Further, we review current applications such as biodiversity assessment, management and protection, infrastructure inspection and AUVs navigation, amongst others. We also delve upon the current trends in the field and examine the challenges and opportunities in the area. [ABSTRACT FROM AUTHOR]

Fish assessment and monitoring are important for the development of a modern aquatic ecosystem. Fish are a vital part of the marine and freshwater environments. Morphological and computational details of fish, such as size, shape, and position, are important in fish observation and fisheries. Typically, manual, or low-efficient techniques are used to acquire fish details. However, existing typical methods are usually time-consuming, less accurate, and resource-intensive. Computer-aided methods are crucial for intelligent and automatic fish assessment. Two novel networks, namely parallel feature fusion-based segmentation network (PFFS-Net) and progressive information fusion-based segmentation network (PIFS-Net), were developed for pixel-wise fish segmentation. PFFS-Net is a base network that uses parallel feature fusion to achieve a better segmentation performance. PIFS-Net is the final model of this work and uses a progressive spatial feature fusion (SFF) mechanism to enhance segmentation accuracy. PIFS-Net also employs rapid feature reduction and pre-prediction low-level information fusion blocks to further boost performance.The proposed models were evaluated using the following three publicly available databases: semantic segmentation of underwater imagery (SUIM), DeepFish, and Large-scale fish. The proposed networks outperformed the state-of-the-art methods in challenging underwater conditions with superior computational efficiency. PIFS-Net needs only 2.02 million trainable parameters for its complete training. Automatic and accurate fish segmentation can be a major step towards an intelligent aquatic ecosystem. The codes of our algorithms and trained models are available on Github.

Fish assessment and monitoring are important for the development of a modern aquatic ecosystem. Fish are a vital part of the marine and freshwater environments. Morphological and computational details of fish, such as size, shape, and position, are important in fish observation and fisheries. Typically, manual, or low-efficient techniques are used to acquire fish details. However, existing typical methods are usually time-consuming, less accurate, and resource-intensive. Computer-aided methods are crucial for intelligent and automatic fish assessment. Two novel networks, namely parallel feature fusion-based segmentation network (PFFS-Net) and progressive information fusion-based segmentation network (PIFS-Net), were developed for pixel-wise fish segmentation. PFFS-Net is a base network that uses parallel feature fusion to achieve a better segmentation performance. PIFS-Net is the final model of this work and uses a progressive spatial feature fusion (SFF) mechanism to enhance segmentation accuracy. PIFS-Net also employs rapid feature reduction and pre-prediction low-level information fusion blocks to further boost performance. The proposed models were evaluated using the following three publicly available databases: semantic segmentation of underwater imagery (SUIM), DeepFish, and Large-scale fish. The proposed networks outperformed the state-of-the-art methods in challenging underwater conditions with superior computational efficiency. PIFS-Net needs only 2.02 million trainable parameters for its complete training. Automatic and accurate fish segmentation can be a major step towards an intelligent aquatic ecosystem. The codes of our algorithms and trained models are available on Github. [ABSTRACT FROM AUTHOR]

Because fish are vital to marine ecosystems, monitoring and accurate detection are crucial for assessing the potential for fisheries in these environments. Conventionally, fish-related assessment is conducted manually, which makes it labor-intensive and time-consuming. In addition, the assessments are challenging owing to underwater visibility limitations, which leads to poor detection accuracy. To overcome these problems, we propose two novel architectures for the automatic and high-performance segmentation of fish populations. In this study, the efficient fish segmentation network (EFS-Net) and multi-level feature accumulation-based segmentation network (MFAS-Net) are the base and final networks, respectively. In deep convolutional neural networks, the initial layers usually contain potential spatial information. Therefore, the EFS-Net employs a series of convolution layers in the early stage of the network for optimal feature extraction. To boost segmentation accuracy, the MFAS-Net uses an initial feature refinement and transfer block to refine potential low-level information and subsequently transfers it to the deep stages of the network. Moreover, the MFAS-Net employs multi-level feature accumulation that improves pixel-wise prediction for fish that are indistinct. The proposed networks are evaluated using two publicly available datasets, namely DeepFish and semantic segmentation of underwater imagery (SUIM), both of which contain challenging underwater fish segmentation images. The experimental results reveal that mean intersection-over-unions of 76.42% and 92.0% are attained by the proposed method for the DeepFish and SUIM datasets, respectively; these values are higher than those by the state-of-the-art methods such as A-LCFCN+PM and DPANet. In addition, high segmentation performance is achieved without compromising the computational efficiency of the networks. The MFAS-Net requires only 3.57 million trainable parameters to be fully trained. The proposed model and the complete code will be made available1.

Data augmentation is an established technique in computer vision to foster the generalization of training and to deal with low data volume. Most data augmentation and computer vision research are focused on everyday images such as traffic data. The application of computer vision techniques in domains like marine sciences has shown to be not that straightforward in the past due to special characteristics, such as very low data volume and class imbalance, because of costly manual annotation by human domain experts, and general low species abundances. However, the data volume acquired today with moving platforms to collect large image collections from remote marine habitats, like the deep benthos, for marine biodiversity assessment and monitoring makes the use of computer vision automatic detection and classification inevitable. In this work, we investigate the effect of data augmentation in the context of taxonomic classification in underwater, i.e., benthic images. First, we show that established data augmentation methods (i.e., geometric and photometric transformations) perform differently in marine image collections compared to established image collections like the Cityscapes dataset, showing everyday traffic images. Some of the methods even decrease the learning performance when applied to marine image collections. Second, we propose new data augmentation combination policies motivated by our observations and compare their effect to those proposed by the AutoAugment algorithm and can show that the proposed augmentation policy outperforms the AutoAugment results for marine image collections. We conclude that in the case of small marine image datasets, background knowledge, and heuristics should sometimes be applied to design an effective data augmentation method. [ABSTRACT FROM AUTHOR]

Multiple fish tracking in unconstrained marine videos is a highly challenging task. Trajectories of fishes convey critical information for the analysis of fish behavior. In this article, we have proposed deep fish tracking network (DFTNet) that incorporates Siamese network for encoding the appearance similarity and attention long short-term memory network to capture the motion similarity across subsequent frames. Finally, intersection-over-union matching score is computed to amalgamate spatial similarity cue in the final score. The proposed framework can provide joint optimization score to maintain the tracklet information encoding appearance, motion, and spatial similarity cues. We perform exhaustive experiments and compare the proposed approach with competing techniques over Fish4knowledge videos and achieve significant average reduction in identification (ID) switches by 60.9%. The source code is made publicly available at https://github.com/hemanth-s17/Deep-Fish-Tracker-Network. [ABSTRACT FROM AUTHOR]

Underwater survey and inspection are tasks of paramount relevance for a variety of applications. They are usually performed through the employment of optical and acoustic sensors installed aboard underwater vehicles, in order to capture details of the surrounding environment. The informative properties of the data are systematically affected by a number of disturbing factors, such as the signal energy absorbed by the propagation medium or diverse noise categories contaminating the resulting imagery. Restoring the signal properties in order to exploit the carried information is typically a tough challenge. Visual saliency refers to the computational modeling of the preliminary perceptual stages of human vision, where the presence of conspicuous targets within a surveyed scene activates neurons of the visual cortex, specifically sensitive to meaningful visual variations. In relatively recent years, visual saliency has been exploited in the field of automated underwater exploration. This work provides a comprehensive overview of the computational methods implemented and applied in underwater computer vision tasks, based on the extraction of visual saliency-related features.

A proposal is described for an underwater sensor combining an acoustic device with an optical one to automatically size juvenile bluefin tuna from a ventral perspective. Acoustic and optical information is acquired when the tuna are swimming freely and the fish cross our combined sensor’s field of view. Image processing techniques are used to identify and classify fish traces in acoustic data (echogram), while the video frames are processed by fitting a deformable model of the fishes’ ventral silhouette. Finally, the fish are sized combining the processed acoustic and optical data, once the correspondence between the two kinds of data is verified. The proposed system is able to automatically give accurate measurements of the tuna’s Snout-Fork Length (SFL) and width. In comparison with our previously validated automatic sizing procedure with stereoscopic vision, this proposal improves the samples per hour of computing time by 7.2 times in a tank with 77 juveniles of Atlantic bluefin tuna (Thunnus thynnus), without compromising the accuracy of the measurements. This work validates the procedure for combining acoustic and optical data for fish sizing and is the first step towards an embedded sensor, whose electronics and processing capabilities should be optimized to be autonomous in terms of the power supply and to enable real-time processing.

Underwater object detection and recognition using computer vision are challenging tasks due to the poor light condition of submerged environments. For intervention missions requiring grasping and manipulation of submerged objects, a vision system must provide an Autonomous Underwater Vehicles (AUV) with object detection, localization and tracking capabilities. In this paper, we describe the integration of a vision system in the MARIS intervention AUV and its configuration for detecting cylindrical pipes, a typical artifact of interest in underwater operations. Pipe edges are tracked using an alpha-beta filter to achieve robustness and return a reliable pose estimation even in case of partial pipe visibility. Experiments in an outdoor water pool in different light conditions show that the adopted algorithmic approach allows detection of target pipes and provides a sufficiently accurate estimation of their pose even when they become partially visible, thereby supporting the AUV in several successful pipe grasping operations. [ABSTRACT FROM AUTHOR]

A proposal is described for an underwater sensor combining an acoustic device with an optical one to automatically size juvenile bluefin tuna from a ventral perspective. Acoustic and optical information is acquired when the tuna are swimming freely and the fish cross our combined sensor's field of view. Image processing techniques are used to identify and classify fish traces in acoustic data (echogram), while the video frames are processed by fitting a deformable model of the fishes' ventral silhouette. Finally, the fish are sized combining the processed acoustic and optical data, once the correspondence between the two kinds of data is verified. The proposed system is able to automatically give accurate measurements of the tuna's Snout-Fork Length (SFL) and width. In comparison with our previously validated automatic sizing procedure with stereoscopic vision, this proposal improves the samples per hour of computing time by 7.2 times in a tank with 77 juveniles of Atlantic bluefin tuna (Thunnus thynnus), without compromising the accuracy of the measurements. This work validates the procedure for combining acoustic and optical data for fish sizing and is the first step towards an embedded sensor, whose electronics and processing capabilities should be optimized to be autonomous in terms of the power supply and to enable real-time processing. [ABSTRACT FROM AUTHOR]