Search

In underwater scenes, degraded underwater images caused by wavelength-dependent light absorption and scattering present huge challenges to vision tasks. Underwater image enhancement has attracted much attention due to the significance of vision-based applications in marine engineering and underwater robotics. Numerous underwater image enhancement algorithms have been proposed in the last few years. However, almost all existing approaches focus only on the enhancement of independent images. Considering that images photographed in the same underwater scene usually share similar degradation, related images can provide rich complementary information for each other’s enhancement. In this paper, we propose an Underwater Image Co-enhancement Network (UICoE-Net) based on an encoder-decoder Siamese architecture. For joint learning, we introduced correlation feature matching units into the multiple layers of our Siamese encoder-decoder structure in order to communicate the mutual correlation of the two branches. Extensive experiments using the Underwater Image Enhancement Benchmark (UIEB), Underwater Image Co-enhancement Dataset (UICoD) collected from an underwater video dataset with ground-truth reference and Stereo Quantitative Underwater Image Dataset (SQUID) dataset demonstrate the effectiveness of our method.

Due to the wavelength-dependent light attenuation, refraction and scattering, underwater images usually suffer from color distortion and blurred details. However, due to the limited number of paired underwater images with undistorted images as reference, training deep enhancement models for diverse degradation types is quite difficult. To boost the performance of data-driven approaches, it is essential to establish more effective learning mechanisms that mine richer supervised information from limited training sample resources. In this paper, we propose a novel underwater image enhancement network, called SGUIE-Net, in which we introduce semantic information as high-level guidance across different images that share common semantic regions. Accordingly, we propose semantic region-wise enhancement module to perceive the degradation of different semantic regions from multiple scales and feed it back to the global attention features extracted from its original scale. This strategy helps to achieve robust and visually pleasant enhancements to different semantic objects, which should thanks to the guidance of semantic information for differentiated enhancement. More importantly, for those degradation types that are not common in the training sample distribution, the guidance connects them with the already well-learned types according to their semantic relevance. Extensive experiments on the publicly available datasets and our proposed dataset demonstrated the impressive performance of SGUIE-Net. The code and proposed dataset are available at: https://trentqq.github.io/SGUIE-Net.html

Depth completion aims to recover pixelwise depth from incomplete and noisy depth measurements with or without the guidance of a reference RGB image. This task attracted considerable research interest due to its importance in various computer vision-based applications, such as scene understanding, autonomous driving, 3-D reconstruction, object detection, pose estimation, trajectory prediction, and so on. As the system input, an incomplete depth map is usually generated by projecting the 3-D points collected by ranging sensors, such as LiDAR in outdoor environments, or obtained directly from RGB-D cameras in indoor areas. However, even if a high-end LiDAR is employed, the obtained depth maps are still very sparse and noisy, especially in the regions near the object boundaries, which makes the depth completion task a challenging problem. To address this issue, a few years ago, conventional image processing-based techniques were employed to fill the holes and remove the noise from the relatively dense depth maps obtained by RGB-D cameras, while deep learning-based methods have recently become increasingly popular and inspiring results have been achieved, especially for the challenging situation of LiDAR-image-based depth completion. This article systematically reviews and summarizes the works related to the topic of depth completion in terms of input modalities, data fusion strategies, loss functions, and experimental settings, especially for the key techniques proposed in deep learning-based multiple input methods. On this basis, we conclude by presenting the current status of depth completion and discussing several prospects for its future research directions.

In recent years, saliency detection is playing vital role in many computer vision tasks. Due to the diversity of visual pattern in natural images, a specific saliency approach is hardly suitable for any type of images. The optimal selection of no-reference saliency map without the pixel-wise ground-truth mask as reference is an important problem, which directly influences the performance of the higher-level tasks. This is essentially a saliency quality assessment problem. Unlike other existing solutions, we propose CascNet for both highly efficient and accurate quality assessment. CascNet is consisted of Applicability Sorting Net (ASNet) and Comparing Net (CompNet), where ASNet serves as an efficient filter to only keep suitable saliency algorithms for the given image with no need of generating saliency maps beforehand, while CompNet is used for further fine assessment with a deep Siamese structure. Comprehensive experiments conducted on the commonly used salient object detection datasets THUS10000, DUT-OMRON and the currently most challenging dataset SOC demonstrate the good performance of the proposed method.

Achieving subjective and objective quality assessment of underwater images is of high significance in underwater visual perception and image/video processing. However, the development of underwater image quality assessment (UIQA) is limited for the lack of publicly available underwater image datasets with human subjective scores and reliable objective UIQA metrics. To address this issue, we establish a large-scale underwater image dataset, dubbed UID2021, for evaluating no-reference (NR) UIQA metrics. The constructed dataset contains 60 multiply degraded underwater images collected from various sources, covering six common underwater scenes (i.e., bluish scene, blue-green scene, greenish scene, hazy scene, low-light scene, and turbid scene), and their corresponding 900 quality improved versions are generated by employing 15 state-of-the-art underwater image enhancement and restoration algorithms. Mean opinion scores with 52 observers for each image of UID2021 are also obtained by using the pairwise comparison sorting method. Both in-air and underwater-specific NR IQA algorithms are tested on our constructed dataset to fairly compare their performance and analyze their strengths and weaknesses. Our proposed UID2021 dataset enables ones to evaluate NR UIQA algorithms comprehensively and paves the way for further research on UIQA. The dataset is available at https://github.com/Hou-Guojia/UID2021 .

In the co-segmentation problems of sparse related image group, how to mine correlations between common objects has been one of the most significant steps. In this paper, we first prioritize the processing efficiency and construct sparse cooperative graph for related images in feature space, instead of examining the full correlations of common foregrounds which lead to high computation cost and great risk of false connection. And more importantly, we design a flexible and efficient extending strategy to reuse the cooperative graph for incremental image data. Then, with the sparse correlations of the image group, we propose a unilateral proposal co-selection method for pair-wise region level co-segmentation. Finally, we have achieved the segmentation results with further pixel level refinement. Our experimental results on publicly available datasets show that, compared with the approaches using dense cooperative constraints, the proposed method can achieve more competitive results with extremely sparse correlative constraints, which shows its bright application prospects for sparse correlated image groups with incremental data due to its high efficiency and flexible extensibility.

Traffic sign detection is a very important part of intelligent assisted driving system. However, with the interference of various target sizes, geometric distortion, occlusion and motion blur, fast and accurate detection on large-size car camera image is extremely hard. To achieve both high efficient and accurate detection, we present a traffic sign detection method within a coarse-to-fine framework, which sequentially detects the targets in grid-level and image-level. We demonstrate that focusing first is a more effective detection strategy for small targets in wide detection space. We propose a target grid prediction network, which is a fully convolutional network for binary classification, to realize rapid coarse localization of the target and effectively guide the clipping and scaling of the target area. With the flexible potential target region extracting strategy, the detecting space can be significantly reduced. At the same time, the correctly extracted local areas for the targets can further facilitate the accurate detection of the subsequent traffic sign detector. In the experiments, our method achieves impressive performance in terms of both efficiency and accuracy. On the challenging Tsinghua-Tencent 100K (TT100K) dataset, our method achieves 20.9 FPS detection speed for 1600 × 1600 images and the F1-score of the proposed method for all-scale targets is 91.55.

In this letter, we present a novel rotation averaging pipeline, which is performed in a hierarchical manner. Unlike the traditional rotation averaging methods which focus on designing robust loss function to get rid of the impacts of the relative rotation outliers, here the outliers are detected and filtered by leveraging the well-known robust model estimation procedure, RANdom SAmple Consensus (RANSAC). During the RANSAC process, the minimal set is randomly sampled by random tree spanning on the Epipolar-geometry Graph (EG). As the RANSAC estimation result is sensitive to the size of minimal set, the EG is clustered into several sub-graphs, and the inner- and inter-cluster RANSAC-based rotation averaging are performed hierarchically. In addition, both random generation and optimal selection of the minimal set are performed in a weighted manner to make the rotation averaging pipeline more robust. Ablation studies and comparison experiments on the 1DSfM and San Francisco (SNF) datasets demonstrate the effectiveness of our proposed method.

Video segmentation is the foundation of many senior computer vision applications. For robotic vision applications, video object segmentation is facing more difficulties. Since the videos are usually collected from those complicated scenes and the camera perspective may be static for some time, object saliency and motion clues which are widely used in traditional video segmentation approaches could be unreliable. Besides, its online processing ability is of crucial importance. In this paper, we propose an online video object segmentation method, which is organized as a segmentation-by-detection framework. It first effectively segments out meaningful objects in short video frame batches with the help of object detectors, and then the segmentation of different batches can be associated with each other via bi-directional notebook based connecting strategy. Experiments conducted on public available datasets verify the above-mentioned property and show good performance of the proposed method.

Images captured underwater are often characterized by low contrast, color distortion, and noise. To address these visual degradations, we propose a novel scheme by constructing an adaptive color and contrast enhancement, and denoising (ACCE-D) framework for underwater image enhancement. In the proposed framework, Difference of Gaussian (DoG) filter and bilateral filter are respectively employed to decompose the high-frequency and low-frequency components. Benefited from this separation, we utilize soft-thresholding operation to suppress the noise in the high-frequency component. Specially, the low-frequency component is enhanced by using an adaptive color and contrast enhancement (ACCE) strategy. The proposed ACCE is an adaptive variational framework implemented in the HSI color space, which integrates data term and regularized term, as well as introduces Gaussian weight and Heaviside function to avoid over-enhancement and oversaturation. Moreover, we derive a numerical solution for ACCE, and adopt a pyramid-based strategy to accelerate the solving procedure. Experimental results demonstrate that our strategy is effective in color correction, visibility improvement, and detail revealing. Comparison with state-of-the-art techniques also validate the superiority of proposed method. Furthermore, we have verified the utility of our proposed ACCE-D for enhancing other types of degraded scenes, including foggy scene, sandstorm scene and low-light scene.

A heuristic four color labeling method is proposed to give robust initial foul-phase partition for Multiphase Multiple Piecewise Constant (MMPC) model.A regional adjacency cracking method is proposed to remove unnecessary adjacency constraints which impede the four color labeling.Compared with the random four color labeling, the color map of heuristic coloring shows better consistency for the homogenous regions.The heuristic four color labeling based approach reaches the good or even better segmentation with fewer iterations. Multilabel segmentation is an important research branch in image segmentation field. In our previous work, Multiphase Multiple Piecewise Constant and Geodesic Active Contour (MMPC-GAC) model was proposed, which can effectively describe multiple objects and background with intensity inhomogeneity. It can be approximately solved with Multiple Layer Graph (MLG) methods. To make the optimization more efficient and limit the approximate error, four-color labeling theorem was further introduced which can limit the MLG within three layers (representing four phases). However, the adopted random four-color labeling method usually provides chaotic color maps with obvious inhomogeneity for those semantic consistent regions. For this case, a new and alternative method named heuristic four-color labeling is proposed in this paper, which aims to generate more reasonable color maps with a global view of the whole image. And compared with the random four-color labeling strategy, the whole iterative algorithm based on our method usually produces better segmentations with faster convergence, particularly for images with clutters and complicated structures. This strategy is a good substitute for random coloring method when the latter produces unsatisfactory messy segmentation. Experiments conducted on public dataset demonstrate the effectiveness of the proposed method.

In this paper, a co-segmentation algorithm based on 3D heat diffusion named co-diffusion is proposed. The image set is considered as a metal cuboid, and the K heat sources with constant temperature, which maximize the sum of the temperature of the system under anisotropic heat diffusion, are found to cluster the image set. The co-diffusion co-segmentation is an intuitive extension of the diffusion segmentation in Kim et al. (Proceedings of ICCV, 2011) while the performance is greatly improved. Comparatively, the proposed algorithm advances in the following three aspects: (1) The proposed algorithm can obtain better optimization because the heat diffusion is directly solved in 3D image set space, while the algorithm in Kim et al. (2011) deals with many independent 2D heat diffusions and solves the optimization by approximate belief propagation. (2) The marginal gain of every candidate heat source is globally determined in the image set, which can effectively compensate the wrong segmentations caused by the locality of the 2D image diffusion (Kim et al. 2011). (3) The K heat sources are chosen in image set while the algorithm (Kim et al. 2011) appoints mandatory K heat sources to each image in set, which will inevitably cause wrong segmentations for some images. The superiority of the proposed co-diffusion segmentation method is examined and demonstrated through a large number of experiments by using some typical datasets.

As a mobile autonomous ocean observation platform, Autonomous Underwater Vehicle (AUV) has the advantage of long endurance and large range, being free from sea condition. The microstructure of velocity and temperature of horizontal and vertical profiles can be obtained with a selfdeveloped AUV, which carries two shear probes, and an FP07 thermistor mounted on a sting at the forward end of the vehicle. However, since the shear probes are essentially transverse force sensors, they are sensitive to vibration noise, and any vibration of the AUV will show up as a spurious signal in the measured shear signal, which will reduce the lower resolution limit of dissipation rate of turbulent kinetic energy. Focusing on the vibration noise of AUV turbulence measurement platform, the work has presented a flexible connection vibration reduction method based on extension spring, and the vibration test has been carried out. By comparing and analyzing the time-frequency domain characteristics of vibration acceleration before and after vibration damping, the results show that the RMS and energy of vibration acceleration are significantly reduced, especially vibration above 54 Hz, whereupon the average damping efficiency is more than 70%, which verifies the validity of this method.

This paper presents a novel approach about constraining trajectories of underwater gliders based on Deep Q Network(DQN) and the dynamic model of gliders. In terms of dynamics modeling, we choose the angle of attack and roll angle as the input of the glider' movement simulation, the profile motions in the vertical plane is manipulated by the angle of attack, and the gyratory motion is accomplished by controlling the roll angle. Since different roll angles allow the glider to turn with different radius, it can be used to counteract the influence of current on the motion state of the glider. And the determination of the proper roll angle to make the glider's trajectory consistent with the preset route is a sequential decision problem. We use the glider's motion space discretization and apply the deep reinforcement learning theory to acquire the optimal roll angle, thus finally achieve the goal that improves the accuracy of the glider's trajectory. The simulation results show that the proposed method has better performance and advantage to deploy compared with the conventional PID method.

Even though there have been a large amount of previous work on video segmentation techniques, it is still a challenging task to extract the video objects accurately without interactions, especially for those videos which contain irrelevant frames (frames containing no common targets). In this essay, a novel multivideo object cosegmentation method is raised to cosegment common or similar objects of relevant frames in different videos, which includes three steps: 1) object proposal generation and clustering within each video; 2) weighted graph construction and common objects selection; and 3) irrelevant frames detection and pixel-level segmentation refinement. We apply our method on challenging datasets and exhaustive comparison experiments demonstrate the effectiveness of the proposed method.

Object proposal quality assessment without ground truth as reference is a challenging task. Some existing methods measure the quality with hand-crafted metrics for subjective metrics, such as objectness and foreground confidence. Recently, deep learning is adopted for direct assessment for quantifiable metric, such as Intersection over Union (IoU). However, we find that IoU, the commonly used quality metric, is far from fully describing the quality of an object proposal. Proposals with the same IoU score may carry totally different amount of discriminative attribute. We introduce a new metric named Discriminative Information Richness (DIR) to characterize the discriminative degree of the given object proposal. DIR is derived from the response intensity of the projected deep feature maps, whose high correlation response indicates the discriminative regions. Besides, we design a convolutional neural network named DrlNet to simultaneously predict IoU scores and perceive the richness of the identification information. DrlNet is defined as a multi-metric joint deep regression network for both spatial covering prediction and discriminative information richness perception. Compared with the solely IoU based models, DrlNet can provide more comprehensive quality assessment. We perform comprehensive experiments on both PASCAL VOC dataset and COCO dataset. The experimental results show that our DrlNet performs well on both proposal selection and object detection tasks. Particularly, experimental results on COCO dataset demonstrate the good generalization ability of the proposed model.

Guo, T.; Song, D.; Li, K.; Li, C., and Yang, H., 2020. Pitch angle control with model compensation based on active disturbance rejection controller for underwater gliders. Journal of Coastal Research, 36(2), 424–433. Coconut Creek (Florida), ISSN 0749-0208.As the pitch angle control system of an underwater glider is nonlinear, multivariate, and has strong coupling, active disturbance rejection control (ADRC) was applied in the pitch angle control loop. To improve the control precision and responsiveness, a model compensation (MC) was presented to alleviate the estimation pressures for the extended state observer in the ADRC. The vast majority of parameters in the MC were estimated and identified so that the ADRC controller can work better. Simulation experiments illustrated that the MC based on ADRC was effective. For the pitch angle control, the overshoot was reduced by 4.5% and the settling time dropped to 90 seconds compared with the traditional ADRC. It demonstrated a more accurate motion trajectory control effect for an underwater glider. Besides, the MC-ADRC controller has better dynamic characteristics and antidisturbance ability, which ensure that the underwater glider is more anti-interference and flexible.

For interactive segmentation approaches, object segmentation in complicated background is cumbersome, and usually needs tedious interactions to refine the incomplete segmentations . In this paper, an adaptive optimal shape prior is proposed for easy interactive object segmentation. Different from the traditional shape priors which only provide loose constraint, our adaptive shape prior gives more accurate and individualized constraint by exploiting the shape information of incomplete segmentation. Moreover, by combining the non-rigid shape registration and a local shape consistency evaluation system presented in this paper, such adaptive optimal shape prior could be achieved automatically. Both of these contributions greatly lighten the burden on users and make interactive segmentation much easier. The comparison experiments on the newly-built TypShape dataset with the related algorithms have demonstrated good performance of the proposed algorithm.

In this paper, an object cosegmentation method based on shape conformability is proposed. Different from the previous object cosegmentation methods which are based on the region feature similarity of the common objects in image set, our proposed SaCoseg cosegmentation algorithm focuses on the shape consistency of the foreground objects in image set. In the proposed method, given an image set where the implied foreground objects may be varied in appearance but share similar shape structures, the implied common shape pattern in the image set can be automatically mined and regarded as the shape prior of those unsatisfactorily segmented images. The SaCoseg algorithm mainly consists of four steps: 1) the initial Grabcut segmentation; 2) the shape mapping by coherent point drift registration; 3) the common shape pattern discovery by affinity propagation clustering; and 4) the refinement by Grabcut with common shape constraint. To testify our proposed algorithm and establish a benchmark for future work, we built the CoShape data set to evaluate the shape-based cosegmentation. The experiments on CoShape data set and the comparison with some related cosegmentation algorithms demonstrate the good performance of the proposed SaCoseg algorithm.

Road traffic plays an important role in our modern life. Lane detection and tracking system is developed for improving active security in intelligent vehicle. We present an effective method to achieve lane detection based on Bayesian probability framework, which utilizes prior knowledge of lane to decrease error lane detections. Besides, propagated spatio-temporal constraints between frames are applied to a simple and robust tracking strategy. Our tracking strategy can deal with some challenging scenarios, such as worn lane markings and shadows of trees, and reduce the amount of calculation greatly. Experimental results show that the proposed algorithm is robust against noise, shadows and illumination variations in captured road image sequences.

In the paper we present a new Spatial Adjacent Bag of Features (SABOF) model, in which the spatial information is effectively integrated into the traditional BOF model to enhance the scene and object recognition performance. The SABOF model chooses the frequency of each keyword and the largest frequency of its neighboring pairs to construct the feature histogram. Using the feature histogram whose dimension is only twice larger than that of the original BOF model, the SABOF model drastically enhances the discrimination performance. Combining the Superpixel Adjacent Histogram (SAH) Fulkerson et al., 2009 [12] with multiple segmentations Pantofaru et al., 2008 [33] and Russell et al., 2006 [36], the SABOF method effectively deals with the segmentation and classification of objects with different sizes. Changing the segmentation scale parameter to obtain multiple superpixel segmentations and correspondingly adjusting the neighbor parameters of the SAH method multiple classifiers are trained so that, the SABOF method can fuse multiple results of these classifiers to obtain better classification performance than any single classifier. The superpixel-based conditional random field (CRF) is used to further improve the classification performance. The experimental results of scene classification and of object recognition and localization on classical data sets demonstrate the performance of the proposed model and algorithm.

A novel object specified segmentation approach based on the auto-adjusted shape prior by utilising the point set registration method is presented. Three steps are included in the proposed method: (i) initial segmentation, (ii) automatic shape prior adjustment by point set registration and (iii) object segmentation constrained by the adjusted shape prior. To repair the shape difference between the local targets and the given shape model, such as location, scale, rotation and local shape details, an excellent point set registration approach named coherent point drift (CPD) is adopted. The adjusted shape constraints under the coherent moving constraint implied in CPD give reliable boundary predictions. Experimental results on the ETHZ shape dataset have demonstrated the outstanding performance of the proposed method.

An image segmentation approach, which is based on heat diffusion and graph cuts optimisation, is proposed. The prior segmentation result is obtained by temperature maximisation on the heat diffusion system. In the random walk-based label-assigning process, due to lack of spatial dependencies of neighbouring pixels, the segmentation may deteriorate notably when pixels from disconnected regions of an image show similar features. To overcome this problem, a multilayer graph-based model is presented and image segmentation is considered as an energy minimisation problem. The parameters in the model are learned from the results of temperature maximisation on the heat diffusion system. It is shown that the presented variational model can be discretely optimised by the graph cuts method efficiently. Therefore, the spatial dependences of the neighbouring pixels can be integrated to obtain better segmentation results. A number of comparison experiments demonstrate the superiority of the proposed method.

In this paper, an object segmentation algorithm based on automatic shape constraint selection is proposed. Different from the traditional shape prior based object segmentation methods which only provide loose shape constraints, our proposed object segmentation gives more accurate shape constraint by selecting the most appropriate shape among the standard shape set. Furthermore, to overcome the inevitable differences between the true borders and the standard shapes, the Coherent Point Drift (CPD) is adopted to project the standard shapes to the local ones. A quantitative evaluating mechanism is introduced to pick out the most suitable shape prior. The proposed algorithm mainly consists of four steps: 1) the initial GrabCut segmentation; 2) standard shape projection by CPD registration; 3) rank the standard shapes according to the evaluation scores; 4) refine GrabCut segmentation with the chosen shape constraint. The comparison experiments with the related algorithms on Weizmann_horse dataset have demonstrated the good performance of the proposed algorithm.

Co-segmentation addresses the problem of simultaneously extracting the common targets appeared in multiple images. Multiple common targets involved object co-segmentation problem, which is very common in reality, has been a new research hotspot recently. In this paper, an unsupervised object co-segmentation method for indefinite number of common targets is proposed. This method overcomes the inherent limitation of traditional proposal selection-based methods for multiple common targets involved images while retaining their original advantages for objects extracting. For each image, the proposed multi-search strategy extracts each target individually and an adaptive decision criterion is raised to give each candidate a reliable judgment automatically, i.e., target or non-target. The comparison experiments conducted on public data sets iCoseg , MSRC , and a more challenging data set Coseg-INCT demonstrate the superior performance of the proposed method.