Search

Image segmentation is an important field of computer vision and has attracted significant research attention in the recent years. In this paper, we provide a survey of regional level set image segmentation models based on the energy functional similarity measure. Our survey begins with an introduction to region-based level set image segmentation and an overview of its general steps. Then the different segmentation models are summarized. We define and survey six categories of regional level set image segmentation models based on energy functional similarity measures. For every category, we present the mainstream approaches from the literature as examples. Experimental analyses are conducted to compare the segmentation performance of various methods, which allow us to draw meaningful conclusions about their mutual advantages and disadvantages. Finally, we conclude this survey by highlighting several promising directions which need to be further explored by the research community in the future.

Image retrieval is becoming more and more important due to the rapid increase of the number of images on the web. To improve the efficiency of computing the similarity of images, hashing has moved into the focus of research. This paper proposes a Deep Attention-based Hash (DAH) retrieval model, which combines an attention module and a convolutional neural network to obtain hash codes with strong representability. Our DAH has the following features: The Hamming distance between the hash codes generated by similar images is small and the Hamming distance of hash codes of dissimilar images has a larger constant value. The quantitative loss from Euclidean distance to Hamming distance is minimized. DAH has a high image retrieval precision: We thoroughly compare it with ten state-of-the-art approaches on the CIFAR-10 dataset. The results show that the Mean Average Precision (MAP) of DAH reaches more than 92% in terms of 12, 24, 36 and 48 bit hash codes on CIFAR-10, which is better than what the state-of- art methods used for comparison can deliver.

Household garbage images are highly diverse in color, texture and geometry, which poses significant challenges to garbage classification. Deep convolutional neural network (DCNN) have recently achieved remarkable progress due to their ability to learn high-level feature representations. It usually requires a large number of labelled image data for training a DCNN model. However, there are few public and mature data sets concerned on household garbage images. This severely limits the progress of research and the state of the art is not entirely clear. To address this problem, we introduce a new benchmark data set for household garbage image classification. This data set is called 30 Types of Household Garbage Images (HGI-30), which contains 6′000 images of 30 household garbage types, with complex backgrounds, different resolutions, and complicated variations in sample, pose, illumination and background. The publicly available HGI-30 data set allows researchers to develop more accurate and robust methods for both household garbage image processing and interpretation analysis of household garbage object. We further study the classification problem on this data set and propose a transfer learning based method, also provide a performance analysis, which serves as baseline result on this benchmark.

With the construction and development of civilized cities, image based garbage classification has gradually become an important concern in computer vision community. During the algorithms for image classification, the strong ability of Convolution Neural Networks (CNNs) in feature learning makes it the most successful approach at the moment. However, the parameters of CNNs model are very huge, and its training usually depends on a large amount of samples. In this article, we tackle the problem of lightweight neural network based garbage image classification, which aims to learn classifier with a small number of model parameters. Specifically, we utilize the MobileNetV2 for the backbone of feature extraction network and jointly train such two nets in a way of deep mutual learning. It realizes the information distillation between the teacher and the student. With this, we can significantly improve the learning ability of the MobileNetV2 based lightweight neural network. The experimental results on a self-assembled dataset show that our proposal effectively classifies the garbage and achieves a classification effect batter than the state of the arts in terms of testing accuracy, time and model size.

Object detection from satellite images is challenging and either computationally expensive or labor intense. Satellite images often cover large areas of more than 10 k m × 10 k m . They include objects of different scales, which makes it hard to detect all of them at the same image resolution. Considering that airplanes are usually located in airports, ships are often distributed in ports and sea areas, and that oil depots are typically found close to airports or ports, we propose a new hierarchical object detection framework for very high-resolution satellite images. Our framework prescribes two stages: (1) detecting airports and ports in down-sampled satellite images and (2) mapping the detected object back to the original high-resolution satellite images for detecting the smaller objects near them. In order to improve the efficiency of object detection, we further propose a contextual information based deep feature extraction approach for both of the hierarchical detection steps, as well as an inclined bounding box based arbitrarily-oriented object location mechanism suitable especially for the smaller objects. Comprehensive experiments on a public dataset and two self-assembled datasets (which we made publicly available) show the superior performance of our method compared to standalone state-of-the-art object detectors.

In this paper, a new hybrid diffusion-based level set method is proposed to efficiently address the complex image segmentation problem. Different from the traditional methods, the proposed method is performed on image diffusion space rather than intensity space. Firstly, the nonlinear diffusion based on total variation flow and additive operator splitting scheme is performed on the original intensity image to obtain the diffused image. Then, the local diffusion energy term is constructed by performing homomorphic unsharp masking operation on diffused image so as to implement a local piecewise constant search. To avoid trapping into local minimum produced by local energy, the global diffusion energy term is formed by approximating diffused image in a global piecewise constant way. Besides, the regularization energy term is included to have penalization effect on evolving contour length and maintenance of level set function being signed distance function. By minimizing the overall energy functional which is a linear combination of local energy, global energy and regularization energy, the evolving contour can be driven to approach the object boundary. The experiments on different characteristics of complex images have shown that the proposed method can achieve satisfying segmentation performance accompanied with some good properties, i.e. the robustness to initial parameter and contour setting, noise insensitivity, quick and stable convergence.

DNA-binding proteins (DBPs) play pivotal roles in many biological functions such as alternative splicing, RNA editing, and methylation. Many traditional machine learning (ML) methods and deep learning (DL) methods have been proposed to predict DBPs. However, these methods either rely on manual feature extraction or fail to capture long-term dependencies in the DNA sequence. In this paper, we propose a method, called PDBP-Fusion, to identify DBPs based on the fusion of local features and long-term dependencies only from primary sequences. We utilize convolutional neural network (CNN) to learn local features and use bi-directional long-short term memory network (Bi-LSTM) to capture critical long-term dependencies in context. Besides, we perform feature extraction, model training, and model prediction simultaneously. The PDBP-Fusion approach can predict DBPs with 86.45% sensitivity, 79.13% specificity, 82.81% accuracy, and 0.661 MCC on the PDB14189 benchmark dataset. The MCC of our proposed methods has been increased by at least 9.1% compared to other advanced prediction models. Moreover, the PDBP-Fusion also gets superior performance and model robustness on the PDB2272 independent dataset. It demonstrates that the PDBP-Fusion can be used to predict DBPs from sequences accurately and effectively; the online server is at http://119.45.144.26:8080/PDBP-Fusion/.

In this paper, an efficient and robust level set method is presented to segment the images with intensity inhomogeneity. The multi-scale segmentation idea is incorporated into energy functional construction and a new Hermite differential operator is designed to numerically solve the level set evolution equation. Firstly, the circular shape window is used to define local region so as to approximate the image as well as intensity inhomogeneity. Then, multi-scale statistical analysis is performed on intensities of local circular regions centered in each pixel. So, the multi-scale local energy term can be constructed by fitting multi-scale approximation of inhomogeneity-free image in a piecewise constant way. To avoid the time-consuming re-initialization procedure, a new double-well potential function is adopted to construct the penalty energy term. Finally, the multi-scale segmentation is performed by minimizing the total energy functional. Here, a new differential operator based on Hermite polynomial interpolation is proposed to solve the minimization. The experiments and comparisons with three popular local region-based methods on images with different levels of intensity inhomogeneity have demonstrated the efficiency and robustness of the proposed method.

Aircraft type recognition from remote sensing images has many civil and military applications. In images obtained with modern technologies such as high spatial resolution remote sensing, even details of aircraft can become visible. With this, the identification of aircraft types from remote sensing images becomes possible. However, the existing methods for this purpose have mostly been evaluated on different data sets and under different experimental settings. This makes it hard to compare their results and judge the progress in the field. Moreover, the data sets used are often not publicly available, which brings difficulties to reproduce the works for fair comparison. This severely limits the progress of research and the state of the art is not entirely clear. To address this problem, we introduce a new benchmark data set for aircraft type recognition from remote sensing images. This data set is called Multi-Type Aircraft Remote Sensing Images (MTARSI), which contains 9’385 images of 20 aircraft types, with complex backgrounds, different spatial resolutions, and complicated variations in pose, spatial location, illumination, and time period. The publicly available MTARSI data set allows researchers to develop more accurate and robust methods for both remote sensing image processing and interpretation analysis of remote sensing object. We also provide a performance analysis of state-of-the-art aircraft type recognition and deep learning approaches on MTARSI, which serves as baseline result on this benchmark.

The selection of motion feature directly affects the recognition effect of human action recognition method. Single feature is often affected by human appearance, environment, camera settings and other factors, and its recognition effect is limited. This paper propose a novel action recognition method by using selective ensemble learning, which is a special paradigm of ensemble learning. Moreover, this paper presents a fast and efficient action description feature and a novel recognition algorithm. Robust discriminant mixed features are learnt from behavioral video frames as behavioral descriptors, The recogniton algorithm using selective ensemble learning can achieve fast classification. Experimental results show that the proposed method achieves ideal recognition results on the self-built indoor behavior data set and public data set.

Image segmentation plays a critical role in computer vision and image processing. In this paper, we propose a new Local Chan–Vese (LCV) model by using the cosine function to express the data fitting term in traditional level set image segment models and present a new distance regularized based on a polynomial. We discuss two algorithms of the new model. The first algorithm is a traditional algorithm based on finite difference, which is slow. The second algorithm is a sweeping algorithm, which didn’t need to solve the Euler-Lagrange equation. The second algorithm only needs to calculate the energy change when a pixel was moved from the outside region to the inside region of evolving curves and vice versa. The second algorithm is high speed and can avoid solving the partial differential equation. There is no need for the reinitialization step, and stability conditions, and the distance regularization term. The experiments have shown the effectiveness of the two algorithms.

This paper presents a novel level set method for complex image segmentation, where the local statistical analysis and global similarity measurement are both incorporated into the construction of energy functional. The intensity statistical analysis is performed on local circular regions centered in each pixel so that the local energy term is constructed in a piecewise constant way. Meanwhile, the Bhattacharyya coefficient is utilized to measure the similarity between probability distribution functions for intensities inside and outside the evolving contour. The global energy term can be formulated by minimizing the Bhattacharyya coefficient. To avoid the time-consuming re-initialization step, the penalty energy term associated with a new double-well potential is constructed to maintain the signed distance property of level set function. The experiments and comparisons with four popular models on synthetic and real images have demonstrated that our method is efficient and robust for segmenting noisy images, images with intensity inhomogeneity, texture images and multiphase images. The intensity statistical analysis is performed on local circular regions.The global energy term is formulated by minimizing the Bhattacharyya coefficient.The penalty energy term associated with a new double-well potential is proposed.The proposed method is efficient and robust for segmenting complex images.

PM2.5 pollution is becoming more and more serious in China. A novel feature selection method based on multi-fractal dimension (MFD) and artificial bee colony algorithm (ABC) is proposed to improve the prediction accuracy of PM2.5. In this method, the MFD is used as the evaluation criterion of feature subsets, and an improved ABC is taken as the search strategy. In this paper, we first use the MFD+ABC method to select the optimal feature subset, then use SVR to predict the next day’s concentration of PM2.5 in Shanghai and Guangzhou. The experimental results show that the proposed method has better performance in both the number of selected features and the prediction accuracy.

In this paper, we describe a novel 3-D face recognition scheme for 3-D face recognition that can automatically identify faces from range images, and is insensitive to holes, facial expression, and hair. In our scheme, a number of carefully selected range images constitute a set of example faces, and another range image is chosen as a ldquogeneric face.rdquo The generic face is then warped to match each of the example faces in the least mean square sense. Each such warp is specified by a vector of displacement values. In feature extraction operation, when a target face image comes in, the generic face is warped to match it. The geometric transformation used in the warping is a linear combination of the example face warping vectors. The coefficients in the linear combination are adjusted to minimize the root mean square error. After the matching process is complete, the coefficients of the composite warp are used as features and passed to a Mahalanobis-distance-based classifier for face recognition. Our technique is tested on a data set containing more than 600 range images. Experimental results in the access-control scenario show the effectiveness of the extracted features.

This paper presents a novel level set method to segment medical image with intensity inhomogeneity (IIH). The multi-scale segmentation idea is incorporated and a new penalty energy term is proposed to eliminate the time-consuming re-initialization procedure. Firstly, the circular window is used to define the local region so as to approximate the image as well as IIH. Then, multi-scale statistical analysis is performed on intensities of local circular regions center in each pixel. The multi-scale energy term can be constructed by fitting multi-scale approximation of inhomogeneity-free image in a piecewise constant way. In addition, a new penalty energy term is constructed to enforce level set function to maintain a signed distance function near the zero level set. Finally, the multi-scale segmentation is performed by minimizing the total energy functional. The experiments on medical images with IIH have demonstrated the efficiency and robustness of the proposed method.

The popular Richardson-Lucy (RL) image deconvolution algorithm often produces undesirable ringing artifacts. In this paper, we propose a novel Dual Range Deringing (DRD) algorithm to address this problem. As a post-deconvolution scheme, the proposed approach follows RL deconvolution and removes ringing artifacts by utilizing information from both the input blurred image and the RL-deblurred image. DRD first marks smooth regions in the input blurred image that are likely to be subjected to ringing artifacts far away from any strong edge. It then identifies short-range ringing artifacts from the regions that surround strong edges in the RL-deblurred image. Once marked, both long- and short-range ringing artifacts are then suppressed by an edgepreserving deringing filter. We demonstrate the effectiveness of this procedure by performing experiments on a set of images blurred with various Point Spread Functions (PSFs). We compare DRD with state-of-the-art non-blind deconvolution algorithms and show that our results are virtually free of ringing artifacts with only minor detail losses. Moreover, DRD consists of computationally efficient local operations and is suitable for parallelization on modern GPUs.

Dermoscopy is a technique used to better visualize pigmented skin lesion and aid the clinician in determining if a lesion is benign or malignant. Automated segmentation of dermoscopy images is an important step for computer-aided diagnosis of melanoma. In this paper, we investigate how to use the spatial constraints present in pigmented lesions to improve the segmentation of dermoscopy images. We present an unsupervised segmentation algorithm that embeds these constraints into the feature space. The algorithm groups image pixels with homogeneous properties, and merges the pixel groups into a few super-regions. The optimal lesion- skin boundary is chosen from the set of all region boundaries, where the optimality is determined from the color and texture properties of the regions. We test our method on 67 dermoscopy images and compare the automatically generated segmentation with dermatologist-determined segmentation. The results demonstrate the advantage of incorporating domain-specific constraints into the segmentation process.

In this paper, a novel scheme is introduced for human facial feature extraction. Unlike previous methods that fit a 3D morphable model to 2D intensity images, our scheme utilizes 3D range images to extract features without requiring manually-defined initial landmark points. A linear transformation is used to achieve the mapping between the 3D model and a 3D range image, which makes the computation simple and fast. Moreover, our scheme is robust to the illumination and pose variations. In addition to features from range images, extra features can be obtained by examining optional 2D texture images. Using our scheme, we can also perform automatic eye/mouth corner localization. Experimental results show the high accuracy and robustness of our scheme.