Search

This paper presents an unsupervised and semi-automatic image segmentation approach where we formulate the segmentation as an inference problem based on unary and pairwise assignment probabilities computed using low-level image cues. The inference is solved via a probabilistic graph matching scheme, which allows rigorous incorporation of low-level image cues and automatic tuning of parameters. The proposed scheme is experimentally shown to compare favorably with contemporary semi-supervised and unsupervised image segmentation schemes, when applied to contemporary state-of-the-art image sets.

Using an efficient neural network for recognition and segmentation will definitely improve the performance and accuracy of the results; in addition to reduce the efforts and costs. This paper investigates and compares between results of four different artificial neural network models. The same algorithm has been applied for all with applying two major techniques, first, neural-segmentation technique, second, apply a new fusion equation. The neural techniques calculate the confidence values for each Prospective Segmentation Points (PSP) using the proposed classifiers in order to recognize the better model, this will enhance the overall recognition results of the handwritten scripts. The fusion equation evaluates each PSP by obtaining a fused value from three neural confidence values. CPU times and accuracies are also reported. Experiments that were performed of classifiers will be compared with each other and with the literature.

Previous approaches to texture analysis and segmentation use multichannel filtering by applying a set of filters in the frequency domain or a set of masks in the spatial domain. This paper presents two new texture segmentation algorithms based on multichannel filtering in conjunction with neural networks for feature extraction and segmentation. The features extracted by Gabor filters have been applied for image segmentation and analysis. Suitable choices of filter parameters and filter bank coverage in the frequency domain to optimize the filters are discussed. Here we introduce two methods to optimize Gabor filter bank. First, a Gabor filter bank with a flat response is implemented and the optimal feature dimension is extracted by competitive networks. Second, a subset of Gabor filter bank is selected to compose the best discriminative filters, so that each filter in this small set can discriminate a pair of textures in a given image. In both approaches, multilayer perceptrons are employed to segment the extracted features. The comparisons of segmentation results generated using the proposed methods and previous research using Gabor, discrete cosine transform (DCT), and Laws filters are presented. Finally, the segmentation results generated by applying the optimized filter banks to textured images are presented and discussed.

This paper aims to propose a new optimal hierarchical clustering approach to 3D mobile light detection and ranging (LiDAR) point clouds. The hierarchical clustering is performed on unorganized point clouds based on a proximity matrix that consists of a distance term and a direction term. In the dissimilarity calculation of two clusters, a pair of points from each of two clusters is selected, respectively, and Euclidean distances between the points are employed to define the distance term. The direction term is obtained by the differences of normal vectors at chosen points. The main contribution is that the cluster combination in the hierarchical clustering is optimized by a point-based graph model. The cluster combination is formulated as a problem of matching, optimized by finding the minimum-cost perfect matching in a bipartite graph. The results show that the proposed hierarchical clustering method succeeds in segmenting object from point clouds without any human–computer interaction and outperforms the state-of-the-art segmentation approaches in terms of completeness and correctness.

In this study the authors will look at the detection and segmentation of the iris and its influence on the overall performance of the iris-biometric tool chain. The authors will examine whether the segmentation accuracy, based on conformance with a ground truth, can serve as a predictor for the overall performance of the iris-biometric tool chain. That is: If the segmentation accuracy is improved will this always improve the overall performance? Furthermore, the authors will systematically evaluate the influence of segmentation parameters, pupillary and limbic boundary and normalisation centre (based on Daugman's rubbersheet model), on the rest of the iris-biometric tool chain. The authors will investigate if accurately finding these parameters is important and how consistency, that is, extracting the same exact region of the iris during segmenting, influences the overall performance., Comment: Published at IET Biometrics

In the previous chapter, you studied the theory behind image filters, along with the types and practical applications of filters in enhancing images.

In this work, a novel region switching based classification method is proposed for speech emotion classification using vowel-like regions (VLRs) and non-vowel-like regions (non-VLRs). In literature, normally the entire active speech region is processed for emotion classification. A few studies have been performed on segmented sound units, such as, syllables, phones, vowel, consonant and voiced, for speech emotion classification. This work presents a detailed analysis of emotion information contained independently in segmented VLRs and non-VLRs. The proposed region switching based method is implemented by choosing the features of either VLRs or non-VLRs for each emotion. The VLRs are detected by identifying hypothesized VLR onset and end points. Segmentation of non-VLRs is done by using the knowledge of VLRs and active speech regions. The performance is evaluated using EMODB, IEMOCAP and FAU AIBO databases. Experimental results show that both the VLRs and non-VLRs contain emotion-specific information. In terms of emotion classification rate, the proposed region switching based classification approach shows significant improvement in comparison to the classification approach by processing entire active speech region, and it outperforms other state-of-the-art approaches for all the three databases.

We propose an active learning approach to image segmentation that exploits geometric priors to speed up and streamline the annotation process. It can be applied for both background-foreground and multi-class segmentation tasks in 2D images and 3D image volumes. Our approach combines geometric smoothness priors in the image space with more traditional uncertainty measures to estimate which pixels or voxels are the most informative, and thus should to be annotated next. For multi-class settings, we additionally introduce two novel criteria for uncertainty. In the 3D case, we use the resulting uncertainty measure to select voxels lying on a planar patch, which makes batch annotation much more convenient for the end user compared to the setting where voxels are randomly distributed in a volume. The planar patch is found using a branch-and-bound algorithm that looks for a 2D patch in a 3D volume where the most informative instances are located. We evaluate our approach on Electron Microscopy and Magnetic Resonance image volumes, as well as on regular images of horses and faces. We demonstrate a substantial performance increase over other approaches thanks to the use of geometric priors., Comment: Extension of our previous paper arXiv:1508.04955

Manual detection and analysis of brain tumours is an exhaustive and time-consuming process. Further, it is subject to intra-observer and inter-observer variabilities. Automated brain tumour segmentation and analysis has thus gained much attention in recent years. However, the existing segmentation techniques do not meet the requirements of real-time use due to limitations posed by poor image quality and image complexity. This article proposes a hybrid approach for image segmentation by combining biorthogonal wavelet transform, skull stripping, fuzzy c-means threshold clustering, Canny edge detection, and morphological operations. Biorthogonal wavelet transform and skull stripping are essential pre-processing steps for analysis of brain images. Initially, biorthogonal wavelet transform is used to remove impulsive noise and skull stripping is employed to eliminate non-cerebral tissue regions from the acquired images, followed by segmentation using fuzzy c-means threshold clustering, Canny edge detection, and morphological processing. The performance of the proposed automated system is tested on standard datasets using performance measures such as Jaccard index, Dice similarity coefficient, execution time, and entropy. The proposed method achieves a Jaccard index and Dice similarity coefficient of 0.886 and 0.935, respectively, which indicate better overlap between the automated segmentation method and manual segmentation method performed by an expert radiologist. The average execution time and average entropy values obtained are 1.001 s and 0.202, respectively. The results obtained are discussed in view of some reported studies in terms of execution time and tumour area. Further work is needed to evaluate the proposed method in routine clinical practice and its effect on radiologists’ performances.

Objectness has recently become a standard step in many computer vision tasks. Among various techniques, those based on hierarchical image segmentation play a fundamental role for developments in new data modalities. In this paper, we address the problem of objectness in RGB-D images and propose a novel and effective approach, namely, hierarchical image segmentation ensemble (HISE). Different from existing image segmentation based methods that generate object segments or proposals largely by heuristics or empirical rules, HISE learns superpixel mergings with a hierarchical tree-structured ensemble, where individual merging models of the ensemble are formed by traversing different paths of the tree, and where both the merging accuracy and proposal diversity are emphasized. Furthermore, we use efficient feature measurements that support easy integration of additional clues. Extensive experiments conducted on the benchmark NYU-v2 RGB-D and SUN RGB-D data sets show the competency of our proposed method.

Segmentation is one of the important steps for image analysis. Multilevel thresholding image segmentation was more popular in image segmentation. Otsu and Kapur based methods are most popular for multilevel threshold image segmentation. Many authors implemented evolutionary algorithms for the optimal multilevel threshold selection based on the above methods. In this paper, an efficient approach for multilevel image segmentation has been proposed and implemented based on novel evolutionary algorithm Improved Differential Search (IDS). The feasibility of proposed approach/algorithm has been tested on standard gray scale images. To check the effectiveness of the proposed approach/algorithm, all experimental results are analyzed quantitatively and qualitatively. Keywords: Multilevel thresholding, Gray scale image segmentation, Improved differential search algorithm, Quantitative and qualitative analysis

In the context of digital image processing and analysis, the Binary Partition Tree (BPT) is a classical data-structure for the hierarchical modelling of images at different scales. BPTs belong both to the families of graph-based models and morphological hierarchies. They constitute an efficient way to define sets of nested partitions of image support, that further provide knowledge-guided reduced research spaces for optimization-based segmentation procedures. Basically, a BPT is built in a mono-feature way, i.e., for one given image, and one given metric, by merging pairs of connected image regions that are similar in the induced feature space. We propose in this work a generalization of the BPT construction framework, allowing to embed multiple features. The cornerstone of our approach relies on a collaborative strategy enabling to establish a consensus between different metrics, thus allowing to obtain a unified hierarchical segmentation space. In particular, this provides alternatives to the complex issue of arbitrary metric construction from several – possibly non-comparable – features. To reach that goal, we first revisit the BPT construction algorithm to describe it in a fully graph-based formalism. Then, we present the structural and algorithmic evolutions and impacts when embedding multiple features in BPT construction. We also discuss different ways to tackle the induced memory and time complexity issues raised by this generalized framework. Final experiments illustrate how this multi-feature framework can be used to build BPTs from multiple metrics computed through the (potentially multiple) image content(s), in particular in the context of remote sensing.

Computer vision is an area of the computer science which aims to make computers to have some functions owned by human vision system. During the research on computer vision, we often think about "how human see". Obviously, when a human observer views a scene, the observer sees not the whole complex scene, but rather a collection of objects. A common experience of segmentation is the way that an image can resolve itself into a figure, typically the significant, important object, and a ground, the background on which the figure lies. The capability of human beings to segment the complex scene into separated objects is so efficient that we regard it as a mystery. At meantime, it appeals the researchers of computer vision who want to let computer have the same capability. So the task, Image Segmentation, is presented. The chapter describes new achievements in the area of multimodal range and intensity image unsupervised segmentation. This chapter is organized as follows. Range sensors are described in section 2 followed by the current state of art survey in section 3. Sections 4 to 6 describe our fast range image segmentation method for scenes comprising general faced objects. This range segmentation method is based on a recursive adaptive probabilistic detection of step discontinuities (sections 4 and 5) which are present at object face borders in mutually registered range and intensity data. Detected face outlines guides the subsequent region growing step in section 6 where the neighbouring face curves are grouped together. Region growing based on curve segments instead of pixels like in the classical approaches considerably speed up the algorithm. The exploitation of multimodal data significantly improves the segmentation quality. The evaluation methodology a range segmentation benchmarks are described in section 7. Following sections show our experimental results of the proposed model (section 8), discuss its properties and conclude (section 9) the chapter.

Segmentation is the partitioning of an image into multiple regions (sets of pixels) according to a given criterion. The goal of segmentation is typically to locate objects of interest within the image. A wide variety of methods and algorithms are available to deal with the problem of the segmentation of images (Fu and Mui, 1981; Haralick and Shapiro, 1985; Pal and Pal, 1993). These methods can be broadly classified into four categories (Zhu and Yuille, 1996): • Edge-based techniques. • Region-based techniques. • Deformable models. • Global optimization approaches. The edge-based techniques are based on information about the boundaries of the image. Therefore, they try to locate the points in which abrupt changes occur in the levels of some property of the image, typically brightness (Canny, 1986; Rosenfeld and Kak, 1982). On the other hand, those methods that use spatial information of the image (e.g. color or texture) to produce the segmented image fit into the region-based techniques (Chen et al., 1992; Sahoo et al., 1988). These methods depend on the consistency of some relevant property in the different regions of the image. The deformable models are based on curves or surfaces defined within an image that moves due to the influence of certain forces. They can be classified into various groups, principally snakes, deformable templates and active contours (Blake and Isard, 1998; Kass et al., 1988). All of these techniques avoid the use of a global criterion when segmenting the image, which is contrary to the global optimization approaches (Geman and Geman, 1984; Kanungo et al., 1994). In this work a unified framework for image segmentation is proposed. The technique consists of two stages: a parallel seeded region growing algorithm (PSRG) and a region merging heuristic (RM). In Figure 1 the functional scheme of the proposed algorithm is shown. In the first step, different segmentations, performed in parallel, of the same input image are obtained. Each of these segmentations, which from now on will be called partial segmentations, are also generated in parallel using different number of processors. This way, the region growing algorithm uses a two level parallelism. Next, a region merging heuristic is applied to the oversegmented image created as result of combining the different initial

Rock particle image segmentation is typically the first and most difficult task [59]. All subsequent interpretation tasks, including particle size, shape and texture analysis, rely heavily on the quality of the segmentation results. Since rock particle images vary from one to another, it is difficult or impossible to design and develop one segmentation algorithm for all kinds of rock particle images. The presented segmentation algorithms were developed for just several types of rock particle images with a certain characteristics with respect to segmentation. In general, both two steps of rock particle image segmentation, i.e. segmentation based on gray level and segmentation based on shape and size of rock particle particles, are needed in most cases of rock particle application. The thresholding algorithms are for images where particles or particle clusters differ everywhere in intensity from the background. The thresholding algorithms are not sensitive to texture on particles, and have normally a low cost for processing. When particles are densely packed, and particles are surrounded by particles and some void spaces (background), the algorithm based on split-and-merge can be applied for the image. The algorithm based on split-and-merge has the advantage of producing higher level primitives, but the region so extracted may not correspond to actual particles, and the boundary of the extracted region is rough. The algorithm based on edge detection is suitable for the images without too much texture on the surface of particles. It has the disadvantage of producing low-level primitives (segments) even after considerable processing. As one example, the two algorithms are compared in Fig. 23. In Fig. 23(a), the original image is a

2D Image segmentation has been a main issue in image analysis since the very early years. Traditional literature usually classifies segmentation approaches as area-based or contourbased. In the second class, among dozens of different approaches, Active Contours have recently gained more and more interest. Active contours (also known as deformable models) are open or closed curves that can accurately fit to the contours of objects featuring almost any kind of shape. These models are called active because they automatically respond to specific characteristics of the points of the image, by changing their shape consequently. For example, an active contour can respond to the edgeness values of the image points. A particular type of active contour is the snake: it responds both to the characteristics of the points of the image (through the minimization of a quantity called external energy), and to specific internal laws ruling its shape and way of deformation, tending to minimize a quantity called internal energy (Kass et al., 1988; Lai & Chin, 1995). It usually consist of elastic curves that, located over an image, evolve from their initial shapes and positions in order to adapt themselves to the notable characteristics of the scene. This evolution comes as a result of the combined action of external and internal forces. The external forces lead the snakes towards features of the image, whereas internal forces model the elasticity of the curves. In a parametric representation, a snake appears as a curve u(s)=(x(s),y(s)), s [0,1], with u(0)=u(1). Its internal energy is often defined as

We study the problem of interactive segmentation and contour completion for multiple objects. The form of constraints our model incorporates are those coming from user scribbles (interior or exterior constraints) as well as information regarding the topology of the 2-D space after partitioning (number of closed contours desired). We discuss how concepts from discrete calculus and a simple identity using the Euler characteristic of a planar graph can be utilized to derive a practical algorithm for this problem. We also present specialized branch and bound methods for the case of single contour completion under such constraints. On an extensive dataset of ~ 1000 images, our experiments suggest that a small amount of side knowledge can give strong improvements over fully unsupervised contour completion methods. We show that by interpreting user indications topologically, user effort is substantially reduced.

In vivo quantitative assessment of structural and functional biomarkers is essential for understanding pathophysiology and identifying novel therapies for congenital heart disorders. Cardiac defect analysis through fixed tissue and histology has offered revolutionary insights into the tissue architecture, but section thickness limits the tissue penetration. This study demonstrated the potential of Light Sheet Fluorescence Microscopy (LSFM) for analyzing in vivo 4D (3d + time) cardiac contractility. Furthermore, we have described the utility of an improved feature detection framework for localizing cardiomyocyte nuclei in the zebrafish atrium and ventricle. Using the Hessian Difference of Gaussian (HDoG) scale space in conjunction with the watershed algorithm, we were able to quantify a statistically significant increase in cardiomyocyte nuclei count across different developmental stages.Furthermore, we assessed individual volumes and surface areas for the cardiomyocyte nuclei in the ventricle’s innermost and outermost curvature during cardiac systole and diastole. Using the segmented nuclei volumes from the feature detection, we successfully performed local area ratio analysis to quantify the degree of deformation suffered by the outermost ventricular region compared to the innermost ventricular region. This paper focuses on the merits of our segmentation and demonstrates its efficacy for cell counting and morphology analysis in the presence of anisotropic illumination across the field-of-view (FOV).

Prosodic features modelling pitch, energy, and duration play a major role in speech emotion recognition. Our word level features, especially duration and pitch features, rely on correct word segmentation and F0 extraction. For the FAU Aibo Emotion Corpus, the automatic segmentation of a forced alignment of the spoken word sequence and the automatically extracted F0 values have been manually corrected. Frequencies of different types of segmentation and F0errors are given and their influence on emotion recognition using different groups of prosodic features is evaluated. The classification results show that the impact of these errors on emotion recognition is small.

This paper presents a modified intuitionistic fuzzy clustering (IFCM) algorithm for medical image segmentation. IFCM is a variant of the conventional fuzzy C-means (FCM) based on intuitionistic fuzzy set (IFS) theory. Unlike FCM, IFCM considers both membership and nonmembership values. The existing IFCM method uses Sugeno’s and Yager’s IFS generators to compute nonmembership value. But for certain parameters, IFS constructed using above complement generators does not satisfy the elementary condition of intuitionism. To overcome this problem, this paper adopts a new IFS generator. Further, Hausdorff distance is used as distance metric to calculate the distance between cluster center and pixel. Extensive experimentations are carried out on standard datasets like brain, lungs, liver and breast images. This paper compares the proposed method with other IFS based methods. The proposed algorithm satisfies the elementary condition of intuitionism. Further, this algorithm outperforms other methods with the use of various cluster validity functions.

We present a hybrid approach that segments an object by using both color and depth information obtained from views captured from a low-cost RGBD camera and sparsely-located color cameras. Our system begins with generating dense depth information of each target image by using Structure from Motion and Joint Bilateral Upsampling. We formulate the multi-view object segmentation as the Markov Random Field energy optimization on the graph constructed from the superpixels. To ensure inter-view consistency of the segmentation results between color images that have too few color features, our local mapping method generates dense inter-view geometric correspondences by using the dense depth images. Finally, the pixel-based optimization step refines the boundaries of the results obtained from the superpixel-based binary segmentation. We evaluate the validity of our method under various capture conditions such as numbers of views, rotations, and distances between cameras. We compared our method with the state-of-the-art methods that use the standard multi-view datasets. The comparison verified that the proposed method works very efficiently especially in a sparse wide-baseline capture environment.

Many machine vision applications require to compute the size of the fuzzy object in the captured image sequences robustly. The size variation with the change of time is then utilized for the different purposes, e. g. data analysis, diagnosis and feedback control. To this end, robust image segmentation is required in the first place. Many state-of-the-art segmentation methods are based on iterative clustering, e.g. the expectation maximization (EM) method, the K-means method and the fuzzy C-means method. One drawback of the iterative learning based clustering methods is that they perform poorly when there are severe noise or outliers. Consequently, the hard segmentation results for the fuzzy images by these segmentation results are not robust enough and the computed sizes based on the hard segmentation results are not accurate either. In this paper, we propose a non-iterative clustering based approach to segment the fuzzy object from the fuzzy images. Instead of yielding a hard segmentation result, we utilize interval type-2 fuzzy logic to assign membership to the final segmentation result. Accordingly, we compute the size of the object based on the soft segmentation result. Experimental results show that the proposed non-iterative soft segmentation approach is more robust in computing the size of the fuzzy object than the hard approaches that yield a distinct segmentation result.

Pedestrian segmentation in infrared images is a challenging problem due to low SNR and inhomogeneous luminance distribution. In this paper, we first introduce background prior and object-center prior into infrared pedestrian segmentation, and propose a robust and efficient saliency-based scheme, which aims to obtain the accurate pedestrian object. First, background likelihood is developed to abstract the object representation based on the Gaussian mixture model soft decomposition. Second, by combining the shape information and the infrared character of the pedestrians, kernel density estimation-based foreground estimation is proposed to obtain the saliency iteratively with better adaptable for the fuzzy contour of the infrared object. Third, pedestrian boundary weight is employed to integrate the above two saliency maps for more intact and accurate results. Finally, pedestrians can be easily segmented from the infrared images, through any existing segmentation methods, as simple as the Otsu threshold method, on the obtained final saliency map. Extensive experiments on real infrared images captured by intelligent transportation systems demonstrate that our saliency algorithm consistently outperforms the state-of-the-art saliency detection methods, in terms of higher precision, F-measure, and lower mean absolute error. The effectiveness of our proposed segmentation algorithm is also evaluated by comparisons with the existing infrared segmentation methods and yields more precise and intact pedestrian regions.

Quantitative image analysis often depends on accurate classification of pixels through a segmentation process. However, imaging artifacts such as the partial volume effect and sensor noise complicate the classification process. These effects increase the pixel intensity variance of each constituent class, causing intensity values of one class to overlap with another. This increased variance makes threshold based segmentation methods insufficient due to ambiguous overlap regions in the pixel intensity distributions. The class ambiguity becomes even more complex for systems with more than two constituents, such as unsaturated moist granular media. In this paper, we propose an image processing workflow that improves segmentation accuracy for multiphase systems. First, the ambiguous transition regions between classes are identified and removed, which allows for global thresholding of single-class regions. Then the transition regions are classified using a distance function, and finally both segmentations are combined into one classified image. This workflow includes three methodologies for identifying transition pixels and we demonstrate on a variety of synthetic images that these approaches are able to accurately separate the ambiguous transition pixels from the single-class regions. One of these methods does not require any parameter tuning and is entirely unguided, whereas the other two require manual threshold specifications and are thus user-guided. For situations with typical amounts of image noise, misclassification errors and area differences calculated between each class of the synthetic images and the resultant segmented images range from 0.69 to 1.48% and 0.01 to 0.74%, respectively, showing the segmentation accuracy of this approach. We demonstrate that we are able to accurately segment x-ray microtomography images of moist granular media using these computationally efficient methodologies.

Image segmentation is a very important and pre-processing step in image analysis. The conventional multilevel thresholding methods are efficient for bi-level thresholding because of its simplicity, robustness, less convergence time and accuracy. However, a mass of computational cost is needed and efficiency is broken down as an exhaustive search is utilized for finding the optimal thresholds, which results in application of evolutionary algorithm and swarm intelligence to obtain the optimal thresholds. The main aim of image segmentation was to segregate the foreground from background. For the first time this paper established a naturally inspired firefly algorithm based multilevel image thresholding for image segmentation by maximizing Shannon entropy or Fuzzy entropy. The proposed algorithm is tested on standard set of images and results are compared with the Shannon entropy or Fuzzy entropy based methods that are optimized by Differential Evolution (DE), Particle Swarm Optimization (PSO) and bat algorithm (BA). It is demonstrated that the proposed method shows better performance in objective function, structural similarity index, peak signal to noise ratio, misclassification error and CPU time than state of art methods. Keywords: Image segmentation, Image thresholding, Fuzzy entropy, Shannon entropy, Particle Swarm Optimization, Firefly algorithm

Pedestrian detection in infrared images is always a challenging task. Segmentation is an important step of pedestrian detection. An accurate segmentation could provide more information for further analysis. In this paper, an improved Fuzzy C-Means clustering method, which incorporates geometric symmetry information, is proposed for infrared pedestrian segmentation. In the proposed method, symmetry information is introduced by Markov random field theory. Moreover, a new metric is utilized to handle the weak symmetry of pedestrian. In addition, a whole procedure is proposed to extract infrared pedestrians. The experimental results indicate that our method performs better for infrared pedestrian segmentation and obtains better segmentation results compared with other state-of-the-art methods.

Sub-cortical brain structure segmentation in Magnetic Resonance Images (MRI) has attracted the interest of the research community for a long time as morphological changes in these structures are related to different neurodegenerative disorders. However, manual segmentation of these structures can be tedious and prone to variability, highlighting the need for robust automated segmentation methods. In this paper, we present a novel convolutional neural network based approach for accurate segmentation of the sub-cortical brain structures that combines both convolutional and prior spatial features for improving the segmentation accuracy. In order to increase the accuracy of the automated segmentation, we propose to train the network using a restricted sample selection to force the network to learn the most difficult parts of the structures. We evaluate the accuracy of the proposed method on the public MICCAI 2012 challenge and IBSR 18 datasets, comparing it with different traditional and deep learning state-of-the-art methods. On the MICCAI 2012 dataset, our method shows an excellent performance comparable to the best participant strategy on the challenge, while performing significantly better than state-of-the-art techniques such as FreeSurfer and FIRST. On the IBSR 18 dataset, our method also exhibits a significant increase in the performance with respect to not only FreeSurfer and FIRST, but also comparable or better results than other recent deep learning approaches. Moreover, our experiments show that both the addition of the spatial priors and the restricted sampling strategy have a significant effect on the accuracy of the proposed method. In order to encourage the reproducibility and the use of the proposed method, a public version of our approach is available to download for the neuroimaging community Kaisar Kushibar and Jose Bernal hold FI-DGR2017 grant from the Catalan Government with reference numbers 2017FI_B00372 and 2017FI_B00476, respectively. This work has been partially supported by La Fundació la Marató de TV3, by Retos de Investigación TIN2014-55710-R, TIN2015-73563-JIN, and DPI2017-86696-R from the Ministerio de Ciencia y Tecnologia

Superpixel segmentation algorithms are to partition an image into perceptually coherence atomic regions by assigning every pixel a superpixel label. Those algorithms have been wildly used as a preprocessing step in computer vision works, as they can enormously reduce the number of entries of subsequent algorithms. In this work, we propose an alternative superpixel segmentation method based on Gaussian mixture model (GMM) by assuming that each superpixel corresponds to a Gaussian distribution, and assuming that each pixel is generated by first randomly choosing one distribution from several Gaussian distributions which are defined to be related to that pixel, and then the pixel is drawn from the selected distribution. Based on this assumption, each pixel is supposed to be drawn from a mixture of Gaussian distributions with unknown parameters (GMM). An algorithm based on expectation-maximization method is applied to estimate the unknown parameters. Once the unknown parameters are obtained, the superpixel label of a pixel is determined by a posterior probability. The success of applying GMM to superpixel segmentation depends on the two major differences between the traditional GMM-based clustering and the proposed one: data points in our model may be non-identically distributed, and we present an approach to control the shape of the estimated Gaussian functions by adjusting their covariance matrices. Our method is of linear complexity with respect to the number of pixels. The proposed algorithm is inherently parallel and can get faster speed by adding simple OpenMP directives to our implementation. According to our experiments, our algorithm outperforms the state-of-the-art superpixel algorithms in accuracy and presents a competitive performance in computational efficiency.

We present efficient algorithms for segmenting and classifying trajectories based on a movement model parameterised by a single parameter, like the Brownian bridge movement model. Segmentation is the problem of subdividing a trajectory into interior-disjoint parts such that each part is homogeneous in its movement characteristics. We formalise this using the likelihood of the model parameter, and propose a new algorithm for trajectory segmentation based on this. We consider the case where a discrete set of m parameter values is given and present an algorithm to compute an optimal segmentation with respect to an information criterion in O(nm) time for a trajectory with n sampling points. We also present an algorithm that efficiently computes the optimal segmentation if we allow the parameter values to be drawn from a continuous domain. Classification is the problem of assigning trajectories to classes of similar movement characteristics. The set of trajectories might for instance be the subtrajectories resulting from segmenting a trajectory, thus identifying movement phases. We give an algorithm to compute the optimal classification with respect to an information criterion in $$O(m^2 + km\log m)$$ time for m parameter values and k trajectories, assuming bitonic likelihood functions. We also show that classification is NP-hard if the parameter values are allowed to vary continuously and present an algorithm that solves the problem in polynomial time under mild assumptions on the input.

The graph-based subspace segmentation technique has garnered a lot of attention in the visual data representation problem. In general, data (e.g., tracks of moving objects) are drawn from multiple linear subspaces. Thus, how to build a block-diagonal affinity matrix is the critical problem. In this paper, we propose a novel graph-based method, Ensemble Subspace Segmentation under Blockwise constraints (ESSB), which unifies least squares regression and a locality preserving graph regularizer into an ensemble learning framework. Specifically, compact encoding using least squares regression coefficients helps achieve a block-diagonal representation matrix among all samples. Meanwhile, the locality preserving regularizer tends to capture the intrinsic local structure, which further enhances the block-diagonal property. Both the blockwise efforts, i.e., least squares regression and the sparse regularizer, work jointly and are formulated in the ensemble learning framework, making ESSB more robust and efficient, especially when handling high-dimensional data. Finally, an efficient optimization solution based on inexact augmented Lagrange multiplier is derived with theoretical time complexity analysis. To demonstrate the effectiveness of the proposed method, we consider three different applications: face clustering, object clustering, and motion segmentation. Extensive results of both accuracy and normalized mutual information on four benchmarks, i.e., YaleB, ORL, COIL and Hopkins155, are reported. Also, the evaluations of computational cost are provided, based on which the superiority of our proposed method in both accuracy and efficiency is demonstrated compared with 12 baseline algorithms.

Existing unsupervised video object segmentation generates object information from the whole video, which ignores analysis of the local clips. However, we observe that local clips and their relationships are also useful for the video object segmentation. For example, the simple background clips can be used to improve the segmentation of complex background clips. In this paper, we propose a novel unsupervised segmentation framework to segment the primary object based on two aspects, i.e., the complexity awareness of video clips and their segmentation propagation. The first one is used to select the simple clips with smooth backgrounds and the second one generates an object prior from the simple clips and propagates the object prior to help and improve the segmentation of the complex clips. A complexity awareness method using the static cues and the dynamic cues are proposed to evaluate the complexity of the video frames. A new object prior learning model based on the local part structure is designed and a local part-based prior propagation is proposed for the complex clip segmentation. To verify our method, we collect a new challenging video segmentation data set, in which each video contains diverse backgrounds. Experimental results demonstrate that our method outperforms several state-of-the-art methods both on a classical data set and our new data set.

In this paper, we propose a simple but effective method for fast image segmentation. We re-examine the locality-preserving character of spectral clustering by constructing a graph over image regions with both global and local connections. Our novel approach to build graph connections relies on two key observations: 1) local region pairs that co-occur frequently will have a high probability to reside on a common object; 2) spatially distant regions in a common object often exhibit similar visual saliency, which implies their neighborship in a manifold. We present a novel energy function to efficiently conduct graph partitioning. Based on multiple high quality partitions, we show that the generated eigenvector histogram based representation can automatically drive effective unary potentials for a hierarchical random field model to produce multi-class segmentation. Sufficient experiments, on the BSDS500 benchmark, large-scale PASCAL VOC and COCO datasets, demonstrate the competitive segmentation accuracy and significantly improved efficiency of our proposed method compared with other state of the arts.

In this paper, we address the challenging task of scene segmentation. In order to capture the rich contextual dependencies over image regions, we propose Directed Acyclic Graph-Recurrent Neural Networks (DAG-RNN) to perform context aggregation over locally connected feature maps. More specifically, DAG-RNN is placed on top of pre-trained CNN (feature extractor) to embed context into local features so that their representative capability can be enhanced. In comparison with plain CNN (as in Fully Convolutional Networks-FCN), DAG-RNN is empirically found to be significantly more effective at aggregating context. Therefore, DAG-RNN demonstrates noticeably performance superiority over FCNs on scene segmentation. Besides, DAG-RNN entails dramatically less parameters as well as demands fewer computation operations, which makes DAG-RNN more favorable to be potentially applied on resource-constrained embedded devices. Meanwhile, the class occurrence frequencies are extremely imbalanced in scene segmentation, so we propose a novel class-weighted loss to train the segmentation network. The loss distributes reasonably higher attention weights to infrequent classes during network training, which is essential to boost their parsing performance. We evaluate our segmentation network on three challenging public scene segmentation benchmarks: Sift Flow, Pascal Context and COCO Stuff. On top of them, we achieve very impressive segmentation performance.

Moving object segmentation and classification from compressed video plays an important role in intelligent video surveillance. Compared with H.264/AVC, High Efficiency Video Coding (HEVC) introduces a host of new coding features that can be further exploited for moving object segmentation and classification. In this paper, we present a real-time approach to segment and classify moving objects using unique features directly extracted from the HEVC compressed domain for video surveillance. In the proposed method, first, motion vector (MV) interpolation for intra-coded prediction unit (PU) and MV outlier removal are employed for preprocessing. Second, blocks with nonzero MVs are clustered into the connected foreground regions using the four-connectivity component labeling algorithm. Third, object region tracking based on temporal consistency is applied to the connected foreground regions to remove the noise regions. The boundary of moving object region is further refined by the coding unit size and PU size. Finally, a person–vehicle classification model using bag of spatial–temporal HEVC syntax words is trained to classify the moving objects, either persons or vehicles. The experimental results demonstrate that the proposed method provides solid performance and can classify moving persons and vehicles accurately.

These days, superpixel algorithms are widely used in computer vision and multimedia applications. However, existing algorithms are designed for planar images, which are less suited to deal with wide angle images. In this paper, we present a superpixel segmentation method for $\text{360}^\circ$ spherical images. Unlike previous methods, our approach explicitly considers the geometry for spherical images and makes clustering to spherical image pixels. It starts with the seeds defined by Hammersley points sampled on the sphere, then iterates between assignment step and update step, which are both based on the distance metric respecting spherical geometry. We evaluate our method on the transformed Berkeley segmentation dataset and panorama segmentation dataset collected by ourselves. Experimental results show that our method can gain better performance in terms of adherence to image boundaries and superpixel structural regularity. Furthermore, superpixels generated by our method can reserve the coherence across image boundaries and all have closed contours.

Semantic segmentation of traffic scenes has potential applications in intelligent transportation systems. Deep learning techniques can improve segmentation accuracy, especially when the information from depth maps is introduced. However, little research has been done on the application of depth maps to the segmentation of traffic scene. In this paper, we propose a method for semantic segmentation of traffic scenes based on RGB-D images and deep learning. The semi-global stereo matching algorithm and the fast global image smoothing method are employed to obtain a smooth disparity map. We present a new deep fully convolutional neural network architecture for semantic pixel-wise segmentation. We test the performance of the proposed network architecture using RGB-D images as input and compare the results with the method that only takes RGB images as input. The experimental results show that the introduction of the disparity map can help to improve the semantic segmentation accuracy and that our proposed network architecture achieves good real-time performance and competitive segmentation accuracy.

Human action segmentation is important for human action analysis, which is a highly active research area. Most segmentation methods are based on clustering or numerical descriptors, which are only related to data, and consider no relationship between the data and physical characteristics of human actions. Physical characteristics of human motions are those that can be directly perceived by human beings, such as speed, acceleration, continuity, and so on, which are quite helpful in detecting human motion segment points. We propose a new physical-based descriptor of human action by curvature sequence warp space alignment (CSWSA) approach for sequence segmentation in this paper. Furthermore, time series-warp metric curvature segmentation method is constructed by the proposed descriptor and CSWSA. In our segmentation method, descriptor can express the changes of human actions, and CSWSA is an auxiliary method to give suggestions for segmentation. The experimental results show that our segmentation method is effective in both CMU human motion and video-based data sets.

Unmanned aerial vehicles (UAV) are used to conduct a variety of recognition as well as specific missions such as target tracking and safe landing. The transmitted image sequences to be interpreted at ground station usually face limited requirements of the data transmission. In this paper, on the one hand, we handle a surveillance mission with segmenting an UAV video's content into semantic regions. A multi-class image segmentation approach is proposed considering UAV video-specific characteristics. After post-processing steps on the segmentation results, a support vector-machine classifier is used to recognize regions. A Markov model is introduced to combine the results from the previous frames in order to improve the accuracy. On the other hand, this study also assesses the influences of data reduction techniques on the proposed techniques. The comparisons between the untreated configuration and control conditions under manipulations of the frame rate, spatial resolution, and compression ratio, demonstrate how these data reduction techniques adversely influence the algorithm's performance. The experiments also point out the optimal configuration in order to obtain a trade-off between the target performance and requirements of the data transmission.

Full-reference image quality assessment is widely used in many applications, such as image compression, image transmission and image mosaic. The visual masking effect has a significant impact on the perception of the human visual system, which is ignored in previous image quality assessments. Combined with the visual masking effect, a full-reference image quality assessment method by edge-feature-based image segmentation (EFS) was proposed. First, the image is segmented into three parts: contour regions, edge-extension regions and slowly-varying regions. The pixels in different regions are then described by different low-level features in the light of the visual masking effect. Finally, the low-level features in each part are pooled by two complementary aspects: visual saliency and visual masking effect. Experimental results on four large-scale benchmark databases show that the proposed method has a better prediction accuracy in all distortion types than other state-of-the-art image quality assessment indices.

A novel hybrid fitting energy-based active contours model in the level set framework is proposed. The method fuses the local image fitting term and the global image fitting term to drive the contour evolution, and a special extra term that penalises the deviation of the level set function from a signed distance function is also included in the authors' method, so the complex and costly reinitialisation procedure is completely eliminated. Their model can efficiently segment the images with intensity inhomogeneity no matter where the initial curve is located in the image. In its numerical implementation, two efficient numerical schemes are used to ensure the sufficient efficiency of the evolution process, one is the algebraic multigrid, which is used for breaking the restrictions on time step; the other is the sparse field method, which is introduced for fast local computation, compared with the traditional schemes, these two strategies can further shorten the time consumption of the evolution process, this allows the level set to quickly reach the true target location. The extensive and promising experimental results on synthetic and real images demonstrate subjectively and objectively the performance of the proposed method.

Selective image segmentation is an important topic in medical imaging and real applications. In this paper, we propose a weighted variational selective image segmentation model which contains two steps. The first stage is to obtain a smooth approximation related to Mumford-Shah model to the target region in the input image. Using weighted function, the approximation provides a larger value for the target region and smaller values for other regions. In the second stage, we make use of this approximation and perform a thresholding procedure to obtain the object of interest. The approximation can be obtained by the alternating direction method of multipliers and the convergence analysis of the method can be established. Experimental results for medical image selective segmentation are given to demonstrate the usefulness of the proposed method. We also do some comparisons and show that the performance of the proposed method is more competitive than other testing methods.

This study deals with information fusion for image segmentation. The evidence theory (or the Dempster–Shafer theory) allows the modellisation of uncertainty and imprecision in the information as well as the combination of different sources. Here, this approach is used in an unsupervised framework to combine the stochastic watershed segmentation which provides several segmentation results, with a Hessian operator in order to obtain a unique and efficient segmentation. The method is tested on natural images from the Berkeley dataset and evaluated using several evaluation metrics. The fusion results surpass those obtained with stochastic watershed alone.

A heuristic four color labeling method is proposed to give robust initial foul-phase partition for Multiphase Multiple Piecewise Constant (MMPC) model.A regional adjacency cracking method is proposed to remove unnecessary adjacency constraints which impede the four color labeling.Compared with the random four color labeling, the color map of heuristic coloring shows better consistency for the homogenous regions.The heuristic four color labeling based approach reaches the good or even better segmentation with fewer iterations. Multilabel segmentation is an important research branch in image segmentation field. In our previous work, Multiphase Multiple Piecewise Constant and Geodesic Active Contour (MMPC-GAC) model was proposed, which can effectively describe multiple objects and background with intensity inhomogeneity. It can be approximately solved with Multiple Layer Graph (MLG) methods. To make the optimization more efficient and limit the approximate error, four-color labeling theorem was further introduced which can limit the MLG within three layers (representing four phases). However, the adopted random four-color labeling method usually provides chaotic color maps with obvious inhomogeneity for those semantic consistent regions. For this case, a new and alternative method named heuristic four-color labeling is proposed in this paper, which aims to generate more reasonable color maps with a global view of the whole image. And compared with the random four-color labeling strategy, the whole iterative algorithm based on our method usually produces better segmentations with faster convergence, particularly for images with clutters and complicated structures. This strategy is a good substitute for random coloring method when the latter produces unsatisfactory messy segmentation. Experiments conducted on public dataset demonstrate the effectiveness of the proposed method.

In this letter, permutation preference is used to represent the data points for the linkage clustering to segment the tracking points belonging to different motions. In order to exclude the impact of outliers, an alternate sampling and clustering strategy is performed, that iteratively alternates between sampling the hypotheses within the clusters and clustering the points with the permutation preference. As a result, points with similar permutation preferences are sampled as the hypotheses, and outliers are effectively excluded, thus, making the preferences more distinguishable and improving the clustering. The iterative interaction between sampling and clustering results in a good convergent result. The proposed method obtains robust segmentation results with the Hopkins 155 dataset, which are better than the results obtained by the state-of-the-art methods.

Most color images obtained from the real world usually contain complex areas, such as nature scene images, remote sensing images, and medical images. All these type of images are very difficult to be separated accurately and automatically for complex color and structures included. In this paper, we focus on detecting hybrid cues of color image to segment complex scene in a bottom-up framework. The main idea of the proposed segmentation method is based on a two-step procedure: 1) a reasonable superpixels computing method is conducted and 2) a Mumford–Shah (M–S) optimal merging model is proposed for presegment suerpixels. First, a set of seed pixels is positioned at the lowest texture energy map computed from structure tensor diffusion features. Next, we implement a growing procedure to extract superpixels from selected seed pixels with color and texture cues. After that, a color-texture histograms feature is defined to measure similarity between regions, and an M–S optimal merging process is executed by comparing the similarity of adjacent regions with standard deviation constraints to get final segmentation. Extensive experiments are conducted on the Berkeley segmentation database, some remote sensing images, and medical images. The results of experiments have verified that the segmentation effectiveness of the proposed method in segmenting complex scenes and indicated that it is more robust and accurate than conventional methods.

Semantic image segmentation is a challenging problem from image processing where deep convolutional neural networks (CNN) have been applied with great success in the recent years. It deals with pixel-wise classification of an input image, dividing it into regions of multiple object classes. However, CNNs are opaque models. Given a trained CNN, it is hard to tell which information encoded in the input image is important for the network to perform segmentation. Such information could be useful to judge whether a trained network learned to segment in a plausible way or how its performance can be improved. For a trained CNN, we formulate an optimization problem to extract relevant image fractions for semantic segmentation. We try to identify a subset of pixels that contain the relevant information for the segmentation of one selected object class. In experiments on the Cityscapes dataset, we show that this is an easy way to gain valuable insight into a CNN trained for semantic segmentation. Looking at the relevant image fractions, we can identify possible limits of a trained network and draw conclusions about possible improvements.

Coherent image regions can be used as good features for many computer vision tasks, such as object tracking, segmentation, and recognition. Most of previous region extraction methods, however, are not suitable for online applications because of their either heavy computations or unsatisfactory results. We propose a seed-based region growing and merging approach to generate simultaneously coherent and discriminative image regions. We present a quadtree-based seed initialization algorithm to adaptively place seeds into different image areas and then grow them into regions by a color- and edge-guided growing procedure. To merge these regions in different levels, we propose to use the generalized boundary strength to measure the quality of region merging result. In addition, we present a region merging algorithm of linear time complexity to perform efficient and effective region merging. Overall, our new approach simultaneously holds these advantages: 1) it is extremely fast with linear complexity in both time and space, which takes less than 50 ms to process an HVGA image; 2) it can give a direct control of the region number and well adapt to image regions with various sizes and shapes; and 3) it provides a tree-structured representation of the regions and thus can model the image from multiple scales. We evaluate the proposed approach on the standard benchmarks with extensive comparisons with the state-of-the-art methods. The experimental results demonstrate its good comprehensive performances. Example applications using the extracted regions as features for online object tracking and multiclass object segmentation also exhibit its potential for many computer vision tasks.

Given a sequence of snapshots of flu propagating over a population network, can we find a segmentation when the patterns of the disease spread change, possibly due to interventions? In this paper, we study the problem of segmenting graph sequences with labeled nodes. Memes on the Twitter network, diseases over a contact network, movie-cascades over a social network, etc. are all graph sequences with labeled nodes. Most related work on this subject is on plain graphs and hence ignores the label dynamics. Others require fix parameters or feature engineering. We propose SnapNETS , to automatically find segmentations of such graph sequences, with different characteristics of nodes of each label in adjacent segments. It satisfies all the desired properties (being parameter free, comprehensive and scalable) by leveraging a principled, multi-level, flexible framework which maps the problem to a path optimization problem over a weighted DAG. Also, we develop the parallel framework of SnapNETS which speeds up its running time. Finally, we propose an extension of SnapNETS to handle the dynamic graph structures and use it to detect anomalies (and events) in network sequences. Extensive experiments on several diverse real datasets show that it finds cut points matching ground-truth or meaningful external signals and detects anomalies outperforming non-trivial baselines. We also show that the segmentations are easily interpretable, and that SnapNETS scales near-linearly with the size of the input. Finally, we show how to use SnapNETS to detect anomaly in a sequence of dynamic networks.