Search

In underwater scenes, degraded underwater images caused by wavelength-dependent light absorption and scattering present huge challenges to vision tasks. Underwater image enhancement has attracted much attention due to the significance of vision-based applications in marine engineering and underwater robotics. Numerous underwater image enhancement algorithms have been proposed in the last few years. However, almost all existing approaches focus only on the enhancement of independent images. Considering that images photographed in the same underwater scene usually share similar degradation, related images can provide rich complementary information for each other’s enhancement. In this paper, we propose an Underwater Image Co-enhancement Network (UICoE-Net) based on an encoder-decoder Siamese architecture. For joint learning, we introduced correlation feature matching units into the multiple layers of our Siamese encoder-decoder structure in order to communicate the mutual correlation of the two branches. Extensive experiments using the Underwater Image Enhancement Benchmark (UIEB), Underwater Image Co-enhancement Dataset (UICoD) collected from an underwater video dataset with ground-truth reference and Stereo Quantitative Underwater Image Dataset (SQUID) dataset demonstrate the effectiveness of our method.

In recent years, saliency detection is playing vital role in many computer vision tasks. Due to the diversity of visual pattern in natural images, a specific saliency approach is hardly suitable for any type of images. The optimal selection of no-reference saliency map without the pixel-wise ground-truth mask as reference is an important problem, which directly influences the performance of the higher-level tasks. This is essentially a saliency quality assessment problem. Unlike other existing solutions, we propose CascNet for both highly efficient and accurate quality assessment. CascNet is consisted of Applicability Sorting Net (ASNet) and Comparing Net (CompNet), where ASNet serves as an efficient filter to only keep suitable saliency algorithms for the given image with no need of generating saliency maps beforehand, while CompNet is used for further fine assessment with a deep Siamese structure. Comprehensive experiments conducted on the commonly used salient object detection datasets THUS10000, DUT-OMRON and the currently most challenging dataset SOC demonstrate the good performance of the proposed method.

Achieving subjective and objective quality assessment of underwater images is of high significance in underwater visual perception and image/video processing. However, the development of underwater image quality assessment (UIQA) is limited for the lack of publicly available underwater image datasets with human subjective scores and reliable objective UIQA metrics. To address this issue, we establish a large-scale underwater image dataset, dubbed UID2021, for evaluating no-reference (NR) UIQA metrics. The constructed dataset contains 60 multiply degraded underwater images collected from various sources, covering six common underwater scenes (i.e., bluish scene, blue-green scene, greenish scene, hazy scene, low-light scene, and turbid scene), and their corresponding 900 quality improved versions are generated by employing 15 state-of-the-art underwater image enhancement and restoration algorithms. Mean opinion scores with 52 observers for each image of UID2021 are also obtained by using the pairwise comparison sorting method. Both in-air and underwater-specific NR IQA algorithms are tested on our constructed dataset to fairly compare their performance and analyze their strengths and weaknesses. Our proposed UID2021 dataset enables ones to evaluate NR UIQA algorithms comprehensively and paves the way for further research on UIQA. The dataset is available at https://github.com/Hou-Guojia/UID2021 .

In the co-segmentation problems of sparse related image group, how to mine correlations between common objects has been one of the most significant steps. In this paper, we first prioritize the processing efficiency and construct sparse cooperative graph for related images in feature space, instead of examining the full correlations of common foregrounds which lead to high computation cost and great risk of false connection. And more importantly, we design a flexible and efficient extending strategy to reuse the cooperative graph for incremental image data. Then, with the sparse correlations of the image group, we propose a unilateral proposal co-selection method for pair-wise region level co-segmentation. Finally, we have achieved the segmentation results with further pixel level refinement. Our experimental results on publicly available datasets show that, compared with the approaches using dense cooperative constraints, the proposed method can achieve more competitive results with extremely sparse correlative constraints, which shows its bright application prospects for sparse correlated image groups with incremental data due to its high efficiency and flexible extensibility.

In this letter, we present a novel rotation averaging pipeline, which is performed in a hierarchical manner. Unlike the traditional rotation averaging methods which focus on designing robust loss function to get rid of the impacts of the relative rotation outliers, here the outliers are detected and filtered by leveraging the well-known robust model estimation procedure, RANdom SAmple Consensus (RANSAC). During the RANSAC process, the minimal set is randomly sampled by random tree spanning on the Epipolar-geometry Graph (EG). As the RANSAC estimation result is sensitive to the size of minimal set, the EG is clustered into several sub-graphs, and the inner- and inter-cluster RANSAC-based rotation averaging are performed hierarchically. In addition, both random generation and optimal selection of the minimal set are performed in a weighted manner to make the rotation averaging pipeline more robust. Ablation studies and comparison experiments on the 1DSfM and San Francisco (SNF) datasets demonstrate the effectiveness of our proposed method.

Video segmentation is the foundation of many senior computer vision applications. For robotic vision applications, video object segmentation is facing more difficulties. Since the videos are usually collected from those complicated scenes and the camera perspective may be static for some time, object saliency and motion clues which are widely used in traditional video segmentation approaches could be unreliable. Besides, its online processing ability is of crucial importance. In this paper, we propose an online video object segmentation method, which is organized as a segmentation-by-detection framework. It first effectively segments out meaningful objects in short video frame batches with the help of object detectors, and then the segmentation of different batches can be associated with each other via bi-directional notebook based connecting strategy. Experiments conducted on public available datasets verify the above-mentioned property and show good performance of the proposed method.

Images captured underwater are often characterized by low contrast, color distortion, and noise. To address these visual degradations, we propose a novel scheme by constructing an adaptive color and contrast enhancement, and denoising (ACCE-D) framework for underwater image enhancement. In the proposed framework, Difference of Gaussian (DoG) filter and bilateral filter are respectively employed to decompose the high-frequency and low-frequency components. Benefited from this separation, we utilize soft-thresholding operation to suppress the noise in the high-frequency component. Specially, the low-frequency component is enhanced by using an adaptive color and contrast enhancement (ACCE) strategy. The proposed ACCE is an adaptive variational framework implemented in the HSI color space, which integrates data term and regularized term, as well as introduces Gaussian weight and Heaviside function to avoid over-enhancement and oversaturation. Moreover, we derive a numerical solution for ACCE, and adopt a pyramid-based strategy to accelerate the solving procedure. Experimental results demonstrate that our strategy is effective in color correction, visibility improvement, and detail revealing. Comparison with state-of-the-art techniques also validate the superiority of proposed method. Furthermore, we have verified the utility of our proposed ACCE-D for enhancing other types of degraded scenes, including foggy scene, sandstorm scene and low-light scene.

A heuristic four color labeling method is proposed to give robust initial foul-phase partition for Multiphase Multiple Piecewise Constant (MMPC) model.A regional adjacency cracking method is proposed to remove unnecessary adjacency constraints which impede the four color labeling.Compared with the random four color labeling, the color map of heuristic coloring shows better consistency for the homogenous regions.The heuristic four color labeling based approach reaches the good or even better segmentation with fewer iterations. Multilabel segmentation is an important research branch in image segmentation field. In our previous work, Multiphase Multiple Piecewise Constant and Geodesic Active Contour (MMPC-GAC) model was proposed, which can effectively describe multiple objects and background with intensity inhomogeneity. It can be approximately solved with Multiple Layer Graph (MLG) methods. To make the optimization more efficient and limit the approximate error, four-color labeling theorem was further introduced which can limit the MLG within three layers (representing four phases). However, the adopted random four-color labeling method usually provides chaotic color maps with obvious inhomogeneity for those semantic consistent regions. For this case, a new and alternative method named heuristic four-color labeling is proposed in this paper, which aims to generate more reasonable color maps with a global view of the whole image. And compared with the random four-color labeling strategy, the whole iterative algorithm based on our method usually produces better segmentations with faster convergence, particularly for images with clutters and complicated structures. This strategy is a good substitute for random coloring method when the latter produces unsatisfactory messy segmentation. Experiments conducted on public dataset demonstrate the effectiveness of the proposed method.

Even though there have been a large amount of previous work on video segmentation techniques, it is still a challenging task to extract the video objects accurately without interactions, especially for those videos which contain irrelevant frames (frames containing no common targets). In this essay, a novel multivideo object cosegmentation method is raised to cosegment common or similar objects of relevant frames in different videos, which includes three steps: 1) object proposal generation and clustering within each video; 2) weighted graph construction and common objects selection; and 3) irrelevant frames detection and pixel-level segmentation refinement. We apply our method on challenging datasets and exhaustive comparison experiments demonstrate the effectiveness of the proposed method.

For interactive segmentation approaches, object segmentation in complicated background is cumbersome, and usually needs tedious interactions to refine the incomplete segmentations . In this paper, an adaptive optimal shape prior is proposed for easy interactive object segmentation. Different from the traditional shape priors which only provide loose constraint, our adaptive shape prior gives more accurate and individualized constraint by exploiting the shape information of incomplete segmentation. Moreover, by combining the non-rigid shape registration and a local shape consistency evaluation system presented in this paper, such adaptive optimal shape prior could be achieved automatically. Both of these contributions greatly lighten the burden on users and make interactive segmentation much easier. The comparison experiments on the newly-built TypShape dataset with the related algorithms have demonstrated good performance of the proposed algorithm.

In this paper, an object cosegmentation method based on shape conformability is proposed. Different from the previous object cosegmentation methods which are based on the region feature similarity of the common objects in image set, our proposed SaCoseg cosegmentation algorithm focuses on the shape consistency of the foreground objects in image set. In the proposed method, given an image set where the implied foreground objects may be varied in appearance but share similar shape structures, the implied common shape pattern in the image set can be automatically mined and regarded as the shape prior of those unsatisfactorily segmented images. The SaCoseg algorithm mainly consists of four steps: 1) the initial Grabcut segmentation; 2) the shape mapping by coherent point drift registration; 3) the common shape pattern discovery by affinity propagation clustering; and 4) the refinement by Grabcut with common shape constraint. To testify our proposed algorithm and establish a benchmark for future work, we built the CoShape data set to evaluate the shape-based cosegmentation. The experiments on CoShape data set and the comparison with some related cosegmentation algorithms demonstrate the good performance of the proposed SaCoseg algorithm.

In the paper we present a new Spatial Adjacent Bag of Features (SABOF) model, in which the spatial information is effectively integrated into the traditional BOF model to enhance the scene and object recognition performance. The SABOF model chooses the frequency of each keyword and the largest frequency of its neighboring pairs to construct the feature histogram. Using the feature histogram whose dimension is only twice larger than that of the original BOF model, the SABOF model drastically enhances the discrimination performance. Combining the Superpixel Adjacent Histogram (SAH) Fulkerson et al., 2009 [12] with multiple segmentations Pantofaru et al., 2008 [33] and Russell et al., 2006 [36], the SABOF method effectively deals with the segmentation and classification of objects with different sizes. Changing the segmentation scale parameter to obtain multiple superpixel segmentations and correspondingly adjusting the neighbor parameters of the SAH method multiple classifiers are trained so that, the SABOF method can fuse multiple results of these classifiers to obtain better classification performance than any single classifier. The superpixel-based conditional random field (CRF) is used to further improve the classification performance. The experimental results of scene classification and of object recognition and localization on classical data sets demonstrate the performance of the proposed model and algorithm.

A novel object specified segmentation approach based on the auto-adjusted shape prior by utilising the point set registration method is presented. Three steps are included in the proposed method: (i) initial segmentation, (ii) automatic shape prior adjustment by point set registration and (iii) object segmentation constrained by the adjusted shape prior. To repair the shape difference between the local targets and the given shape model, such as location, scale, rotation and local shape details, an excellent point set registration approach named coherent point drift (CPD) is adopted. The adjusted shape constraints under the coherent moving constraint implied in CPD give reliable boundary predictions. Experimental results on the ETHZ shape dataset have demonstrated the outstanding performance of the proposed method.

In this paper, an object segmentation algorithm based on automatic shape constraint selection is proposed. Different from the traditional shape prior based object segmentation methods which only provide loose shape constraints, our proposed object segmentation gives more accurate shape constraint by selecting the most appropriate shape among the standard shape set. Furthermore, to overcome the inevitable differences between the true borders and the standard shapes, the Coherent Point Drift (CPD) is adopted to project the standard shapes to the local ones. A quantitative evaluating mechanism is introduced to pick out the most suitable shape prior. The proposed algorithm mainly consists of four steps: 1) the initial GrabCut segmentation; 2) standard shape projection by CPD registration; 3) rank the standard shapes according to the evaluation scores; 4) refine GrabCut segmentation with the chosen shape constraint. The comparison experiments with the related algorithms on Weizmann_horse dataset have demonstrated the good performance of the proposed algorithm.