Search

Novelty and outlier detection are both used for anomaly detection. This paper works through the method of One-Class support vector machine (SVM) which could estimate the contour of initial observations and can be applied in the problems of anomaly detection. In this paper, the experiments on both artificial and real-world data sets are performed to demonstrate the importance of kernel and kernel parameter choice and the corresponding sensitivity of the algorithm. Due to the desire for general regulations of kernel parameter selection, some general methods of selecting the kernel bandwidth parameter of RBF kernel are therefore investigated, including median heuristic method and Bayesian kernel learning method. Then, the experiments based on these methods are conducted to observe their effect. Consequently, some new discoveries together with current issues on these methods, as well as some applicable situations of these methods, are found in this paper. Furthermore, this paper also confirms the effectiveness of RBF kernel and that the initial value of its bandwidth parameter can be set by those two methods.

Unmanned aerial vehicles using light detection and ranging (UAV LiDAR) with high spatial resolution have shown great potential in forest applications because they can capture vertical structures of forests. Individual tree segmentation is the foundation of many forest research works and applications. The tradition fixed bandwidth mean shift has been applied to individual tree segmentation and proved to be robust in tree segmentation. However, the fixed bandwidth-based segmentation methods are not suitable for various crown sizes, resulting in omission or commission errors. Therefore, to increase tree-segmentation accuracy, we propose a self-adaptive bandwidth estimation method to estimate the optimal kernel bandwidth automatically without any prior knowledge of crown size. First, from the global maximum point, we divide the three-dimensional (3D) space into a set of angular sectors, for each of which a canopy surface is simulated and the potential tree crown boundaries are identified to estimate average crown width as the kernel bandwidth. Afterwards, we use a mean shift with the automatically estimated kernel bandwidth to extract individual tree points. The method is iteratively implemented within a given area until all trees are segmented. The proposed method was tested on the 7 plots acquired by a Velodyne 16E LiDAR system, including 3 simple plots and 4 complex plots, and 95% and 80% of trees were correctly segmented, respectively. Comparative experiments show that our method contributes to the improvement of both segmentation accuracy and computational efficiency.

In the present investigation two smart prediction tools, namely the general regression neural network (GRNN) and multiple regression analysis (MRA) models were developed to predict and compare some of the key machinability aspects like average kerf width, average surface roughness and material removal rate in the wire electrical discharge machining process of titanium grade 6. Pulse-on time, pulse-off time, wire feed and wire tension were considered as machining variables to develop the predictive model. In order to curtail cross-validation error in GRNN, optimized kernel bandwidth was utilized using the grid search method. The neural network and regression models were trained, validated and tested with measured data. A mathematical model was developed using multiple regression analysis. The ANOVA test was also conducted to determine the significant parameters affecting the responses. The results indicated that the predicted responses lie within ± 5% and ± 10% error for GRNN and MRA, respectively, which suggests that the GRNN model is more reliable and adequate than the regression model. A comparative study with previous research work was also done to confirm the novelty along with application potential of the proposed model.

We describe Kernel gradient subtractive clustering (KG-SC), a scheme for automatic selection of most representative exemplar points from large scale datasets. Exemplar selection is required for abstractions in machine learning and data science. Exemplars are more interesting and informative than just dividing objects into clusters since the exemplars are real data points that form an abstract view of the whole dataset and are used to store compressed information. Selecting exemplars and clustering data around them is also important for enhancing the performance in evolutionary computation, multi-optima evolutionary search and differential evolution. Currently, algorithms like Affinity Propagation and k-medoids can select representative exemplars from the data. k-medoids requires the number k of exemplars to be given in advance, as well as a dissimilarity matrix in memory. Affinity propagation automatically finds exemplars as well as their number but it requires a similarity matrix in main memory. The essence of the proposed solution relies on a Kernel averaged gradient descent subtractive clustering which automatically learns a suitable kernel bandwidth parameter from the data and then uses this bandwidth in the subtractive clustering step that automatically selects the exemplars without any prior knowledge of their number, and can also return their ranking. We compare the proposed KG-SC exemplar selection with affinity propagation exemplar selection and evaluate their solutions with widely used quality indexes. The experimental simulations on various benchmark datasets seem promising since both algorithms select well defined representative exemplars and can deliver high quality solutions.

Neuromodulations as observed in the extracellular electrical potential recordings obtained from Electroencephalograms (EEG) manifest as organized, transient patterns that differ statistically from their featureless noisy background. Leveraging on this statistical dissimilarity, we propose a noniterative robust classification algorithm to isolate, in time, these neuromodulations from the temporally disorganized but structured background activity while simultaneously incorporating temporal sparsity of the events. Specifically, we exploit the ability of correntropy to asses higher - order moments as well as imply the degree of similarity between two random variables in the joint space regulated by the kernel bandwidth. We test our algorithm on DREAMS Sleep Spindle Database and further elaborate on the hyperparameters introduced. Finally, we compare the performance of the algorithm with two algorithms designed on similar ideas; one of which is a quick, simple norm based technique while the other parallels the state-of-the-art Robust Principal Component Analysis (RPCA) to achieve classification. The algorithm is able to match the performance of the state-of-the-art techniques while saving tremendously on computation time and complexity., 4 pages, Engineering in Medicine and Biology

In this paper, we propose a gas classification technique based on extracting new features and support vector machines (SVM) in a chemical plant. First, various gases are collected using semiconductor gas seniors, and then we calculate the composition ratio of these gasses, which are defined as features. These extracted features are highly discriminative and quantify the presence of gas. Moreover, these features are used as the SVM input for classifying gas types. In addition, we apply a grid search technique in SVM for tuning hyper-parameters such as misclassification rate, C, and kernel bandwidth, σ, to improve the classification performance. To verify the proposed technique, we collect various gases composition using a cost-effective self-designed test rig. The experimental results indicate that the proposed method is highly capable of classifying various hazardous gases with good accuracy.

Graphical abstractDisplay Omitted HighlightsA PSO-KDE model is proposed to diagnosis of breast cancer.PSO is used to determine bandwidth and select feature subset in KDE based classifier.PSO-KDE solves the multiple criteria problem using two single objective functions.The aim is to minimize the number of features and the classification error. Machine learning techniques can be used in diagnosis of breast cancer to help pathologists and physicians for decision making process. Kernel density estimation is a popular non-parametric method which can be applied for the estimation of data in many diverse applications. Selection of bandwidth and feature subset in kernel density estimator significantly influences the classification performance. In this paper, a PSO-KDE model is proposed that hybridize the particle swarm optimization (PSO) and non-parametric kernel density estimation (KDE) based classifier to diagnosis of breast cancer. In the proposed model, particle swarm optimization is used to simultaneously determine the kernel bandwidth and select the feature subset in the kernel density estimation based classifier. Classification performance and the number of selected features are the criteria used to design the objective function of PSO-KDE. The performance of the PSO-KDE is examined on Wisconsin Breast Cancer Dataset (WBCD) and Wisconsin Diagnosis Breast Cancer Database (WDBC) using classification accuracy, sensitivity and specificity. Experimental results demonstrate that the proposed model has better average performance than GA-KDE model in diagnosis of breast cancer.

International audience; This paper investigates multivariate kernel density estimation for hyperspectral data. More specifically, it focuses on designing a data-driven, parameter-free and fast estimator that is able to deal with clusters of different size, shape and density. We first propose a new criterion to compare density estimates, based on how evenly they treat the different classes in the data. In addition, we propose a simple, parameter-free and non-iterative kernel bandwidth selection approach. We show that it is substantially more adaptive than existing estimators on six remote sensing hyperspectral datasets. © 2018 IEEE.

Most density-based clustering methods largely rely on how well the underlying density is estimated. However, like clustering, density estimation is also a challenging unsupervised learning problem, especially the determination of the kernel bandwidth. In this paper, we propose a density-based multilayer hierarchical clustering method, called the Deep Nearest Neighbor Descent (D-NND), which can largely alleviate the impact of the density estimation. Unlike previous density-based methods, D-NND learns the underlying density distribution layer by layer and at the same time makes the dataset sparsely and effectively organized into a directed Tree. The experiments on three real-world datasets and several challenging synthetic datasets demonstrate that the proposed method has strong ability to discover the underlying cluster structures and is not very sensitive to the density estimation method, the parameters and the clusters of multiple scales.

A wide variety of methods have been proposed in protein subnuclear localization to improve the prediction accuracy. However, one important trend of these means is to treat fusion representation by fusing multiple feature representations, of which, the fusion process takes a lot of time. In view of this, this paper novelly proposed a method by combining a new single feature representation and a new algorithm to obtain good recognition rate. Specifically, based on the position-specific scoring matrix (PSSM), we proposed a new expression, correlation position-specific scoring matrix (CoPSSM) as the protein feature representation. Based on the classic nonlinear dimension reduction algorithm, kernel linear discriminant analysis (KLDA), we added a new discriminant criterion and proposed a dichotomous greedy genetic algorithm (DGGA) to intelligently select its kernel bandwidth parameter. Two public datasets with Jackknife test and KNN classifier were used for the numerical experiments. The results showed that the overall success rate (OSR) with single representation CoPSSM is larger than that with many relevant representations. The OSR of the proposed method can reach as high as 87.444% and 90.3361% for these two datasets, respectively, outperforming many current methods. To show the generalization of the proposed algorithm, two extra standard datasets of protein subcellular were chosen to conduct the expending experiment, and the prediction accuracy by Jackknife test and Independent test is still considerable.

We apply a fast kernel method for mask-based single-channel speech enhancement. Specifically, our method solves a kernel regression problem associated to a non-smooth kernel function (exponential power kernel) with a highly efficient iterative method (EigenPro). Due to the simplicity of this method, its hyper-parameters such as kernel bandwidth can be automatically and efficiently selected using line search with subsamples of training data. We observe an empirical correlation between the regression loss (mean square error) and regular metrics for speech enhancement. This observation justifies our training target and motivates us to achieve lower regression loss by training separate kernel model per frequency subband. We compare our method with the state-of-the-art deep neural networks on mask-based HINT and TIMIT. Experimental results show that our kernel method consistently outperforms deep neural networks while requiring less training time.

The ability to discover projected clusters in high-dimensional data is essential for many machinelearning applications. Projective clustering of categorical data is currently a challenge due to the difficultiesin learning adaptive weights for categorical attributes coordinating with clusters optimization. In this paper,a probability-based learning framework is proposed, which allows both the attribute weights and the centerbasedclusters to be optimized by kernel density estimation on categorical attributes. A novel algorithm is thenderived for projective clustering on categorical data, based on the new learning approach for the kernel bandwidthselection problem. We show that the attribute weight substantially connects to the kernel bandwidth, whilethe optimized cluster center corresponds to the normalized frequency estimator of the categorical attributes.Experimental results on synthesis and real-world data show outstanding performance of the proposed method,which significantly outperforms state-of-the-art algorithms.

The self-organizing map (SOM) is one of the popular clustering and data visualization algorithms and has evolved as a useful tool in pattern recognition, data mining since it was first introduced by Kohonen. However, it is observed that the magnification factor for such mappings deviates from the information-theoretically optimal value of 1 (for the SOM it is 2/3). This can be attributed to the use of the mean square error to adapt the system, which distorts the mapping by oversampling the low probability regions. In this work, we first discuss the kernel SOM in terms of a similarity measure called correntropy induced metric (CIM) and empirically show that this can enhance the magnification of the mapping without much increase in the computational complexity of the algorithm. We also show that adapting the SOM in the CIM sense is equivalent to reducing the localized cross information potential, an information-theoretic function that quantifies the similarity between two probability distributions. Using this property we propose a kernel bandwidth adaptation algorithm for Gaussian kernels, with both homoscedastic and heteroscedastic components. We show that the proposed model can achieve a mapping with optimal magnification and can automatically adapt the parameters of the kernel function.

We introduce a differentiable, end-to-end trainable framework for solving pixel-level grouping problems such as instance segmentation consisting of two novel components. First, we regress pixels into a hyper-spherical embedding space so that pixels from the same group have high cosine similarity while those from different groups have similarity below a specified margin. We analyze the choice of embedding dimension and margin, relating them to theoretical results on the problem of distributing points uniformly on the sphere. Second, to group instances, we utilize a variant of mean-shift clustering, implemented as a recurrent neural network parameterized by kernel bandwidth. This recurrent grouping module is differentiable, enjoys convergent dynamics and probabilistic interpretability. Backpropagating the group-weighted loss through this module allows learning to focus on only correcting embedding errors that won't be resolved during subsequent clustering. Our framework, while conceptually simple and theoretically abundant, is also practically effective and computationally efficient. We demonstrate substantial improvements over state-of-the-art instance segmentation for object proposal generation, as well as demonstrating the benefits of grouping loss for classification tasks such as boundary detection and semantic segmentation.

Support vector data description (SVDD) is a popular technique for detecting anomalies. The SVDD classifier partitions the whole space into an inlier region, which consists of the region near the training data, and an outlier region, which consists of points away from the training data. The computation of the SVDD classifier requires a kernel function, and the Gaussian kernel is a common choice for the kernel function. The Gaussian kernel has a bandwidth parameter, whose value is important for good results. A small bandwidth leads to overfitting, and the resulting SVDD classifier overestimates the number of anomalies. A large bandwidth leads to underfitting, and the classifier fails to detect many anomalies. In this paper we present a new automatic, unsupervised method for selecting the Gaussian kernel bandwidth. The selected value can be computed quickly, and it is competitive with existing bandwidth selection methods.

This paper proposes a novel approach for multimedia signal reconstruction based on kernel density estimation (KDE). We make use of a vector formalism in which vectors consist of a first subvector containing a set of missing samples and a second one containing a set of available context samples. The missing subvector is reconstructed by a minimum mean square error estimator which employs a probability density function (pdf) obtained by KDE. As in any kernel-based method, the main issue to deal with is the estimation of an appropriate kernel bandwidth. We propose an adaptive procedure for bandwidth estimation (BE) especially conceived for signal reconstruction. Thus, unlike general KDE or kernel-based regression, which try to obtain a general fit, the focus of this BE procedure is on the specifi cm issing subvector. Also, in order to exploit local signal correlations, our BE proposal adopts a scaling approach in which the bandwidth is computed as the local covariance matrix scaled by two factors. These two scale factors are obtained by minimization of two dif- ferent approximations to the reconstruction error. The resulting reconstruction methodology is tested on a spatial error conceal- ment (EC) application in which intracoded images have been transmitted through an error prone channel. The experimental results show the superiority of the proposed approach over a wide range of existing EC techniques.

Aiming at the limitations of the traditional mean shift, such as invariable kernel bandwidth, an improved tracking algorithm with the following strategies is proposed. The target model and the candidate are described by the similarity between them is evaluated by Bhattacharyya coefficient. This algorithm firstly calculates the Bhattacharyya coefficient of the template target histogram and template background histogram and calculates the Bhattacharyya coefficient of the candidate target histogram of the current frame and template background histogram when tracking. Then judge the change tendency of the target by comparing the two coefficients and correct the tracking window width with plus or minus 10% increment for subsequent frames target tracking. The experiments prove that the present method has better stability and robustness than the traditional algorithm and the kernel bandwidth can adapt to the size change of the target.

Wind power is considered one of the most rapidly growing sources of electricity generation all over the world. This study proposes a new approach for wind power prediction. The proposed method is derived by integrating the kernel principal component analysis (KPCA) method with the evolutionary optimised general regression neural network (GRNN) and local prediction framework. Local prediction uses only a set of K nearest neighbours in the reconstructed embedded space with considering the more relevant historical instances. In the evolutionary optimised local GRNN (EOLGRNN), the kernel bandwidth (smooth parameter) that controls the smoothness of the approximation is coded in a chromosome and determined by the optimisation using evolutionary algorithm. In the proposed method, KPCA is used in the first stage to extract features and obtain kernel principal components which used to construct the phase space of the time series of input. Then, EOLGRNN is employed in the second stage to solve the wind power prediction problem. The proposed method is evaluated using real world dataset. The results show that the proposed method provides a much better prediction performance in comparison with other published methods employing the same data.

In order to solve the problem that traditional mean shift tracking algorithm based on color histogram is easily affected by similar color non-target object and the change of the size of target, a variable kernel bandwidth tracking algorithm is put forward based on contourlet histogram and information entropy. The target is modeled by contourlet histogram, as iterative parameters in mean shift algorithm, and according to the relationship between information entropy and the size of target, a scale-update item is added to mean shift algorithm to track the target that has a change in size. Experimental results show that the proposed tracking algorithm can not only effectively overcome the interference of similar color non-target object in the scene, but also adapt to the changes in target’s size stably, and this algorithm had a good robustness.

The mean shift MS algorithm is based on a statistical approach to the clustering problem. Specifically, the method is a variant of density estimation. We revisit in this article the MS paradigm and its use for clustering of remotely sensed images. Specifically, we investigate further the classification accuracy of remotely sensed images as a function of various MS parameters, such as the variant used, kernel type, dimensionality, kernel bandwidth, etc. We provide empirical assessment of the algorithm based on experiments with multi-temporal and multi-spectral remotely sensed data sets, representing agricultural and land-cover data. Although the classification accuracy and reliability seem comparable to those obtained by other unsupervised methods e.g. ISODATA, the MS algorithm provides several important operational advantages. The adaptation of the procedure to a parallel computational environment is also discussed and demonstrated.

This paper discusses the Correntropy Induced Metric (CIM) based Growing Neural Gas (GNG) architecture. CIM is a kernel method based similarity measurement from the information theoretic learning perspective, which quantifies the similarity between probability distributions of input and reference vectors. We apply CIM to find a maximum error region and node insert criterion, instead of euclidean distance based function in original GNG. Furthermore, we introduce the two types of Gaussian kernel bandwidth adaptation methods for CIM. The simulation experiments in terms of the affect of kernel bandwidth σ in CIM, the self-organizing ability, and the quantitative comparison show that proposed model has the superior abilities than original GNG.

Kernel Spectral Clustering (KSC) solves a weighted kernel principal component analysis problem in a primal-dual optimization framework. It builds an unsupervised model on a small subset of data using the dual solution of the optimization problem. This allows KSC to have a powerful out-of-sample extension property leading to good cluster generalization w.r.t. unseen data points. However, in the presence of noise that causes overlapping data, the technique often fails to provide good generalization capability. In this paper, we propose a two-step process for clustering noisy data. We first denoise the data using kernel principal component analysis (KPCA) with a recently proposed Model selection criterion based on point-wise Distance Distributions (MDD) to obtain the underlying information in the data. We then use the KSC technique on this denoised data to obtain good quality clusters. One advantage of model based techniques is that we can use the same training and validation set for denoising and for clustering. We discovered that using the same kernel bandwidth parameter obtained from MDD for KPCA works efficiently with KSC in combination with the optimal number of clusters k to produce good quality clusters. We compare the proposed approach with normal KSC and KSC with KPCA using a heuristic method based on reconstruction error for several synthetic and real-world datasets to showcase the effectiveness of the proposed approach.

In this work we present a framework for automatic acoustic detection of chainsaw sounds to detect illegal wood extraction in the Amazon Rainforest. Our approach was developed to be embedded into the sensor nodes of a Wireless Acoustic Sensor Network (WASN) to monitor the environment. First, we represent each sound by a set of Mel-Frequency Cepstral Coefficients (MFCCs) and, after that, we fit a Probability Density Function (PDF) with a kernel based on a multivariate Gaussian density estimation using only the target class. This One-Class classification method allows us to recognize only chainsaw sounds rejecting all the other possible environmental sounds, such as: animal's calls, weather noises or boat engines. In the experiments, we varied the number MFCCs coefficients and the Kernel bandwidth performing a leave-one-out cross validation to find the best combination. Finally, we found that the best parameter combination achieve 98% of accuracy showing a low FNR and a high TPR, fact that enhances the credibility of the system avoiding false alarms and making it an optimal choice for an WASN application.

We investigate the parameters that govern an unsupervised anomaly detection framework that uses nonlinear techniques to learn a better model of the non-anomalous data. A manifold or kernel-based model is learned from a small, uniformly sampled subset in order to reduce computational burden and under the assumption that anomalous data will have little effect on the learned model because their rarity reduces the likelihood of their inclusion in the subset. The remaining data are then projected into the learned space and their projection errors used as detection statistics. Here, kernel principal component analysis is considered for learning the background model. We consider spectral data from an 8-band multispectral sensor as well as panchromatic infrared images treated by building a data set composed of overlapping image patches. We consider detection performance as a function of patch neighborhood size as well as embedding parameters such as kernel bandwidth and dimension. ROC curves are generated over a range of parameters and compared to RX performance.

Domain adaptation learning (DAL) is a novel and effective technique to address pattern classification problems where the prior information for training is unavailable or insufficient. Its effectiveness depends on the discrepancy between the two distributions that respectively generate the training data for the source domain and the testing data for the target domain. However, DAL may not work so well when only the distribution mean discrepancy between source and target domains is considered and minimized. In this paper, we first construct a generalized projected maximum distribution discrepancy (GPMDD) metric for DAL on reproducing kernel Hilbert space (RKHS) based domain distributions by simultaneously considering both the projected maximum distribution mean and the projected maximum distribution scatter discrepancy between the source and the target domain. In the sequel, based on both the structure risk and the GPMDD minimization principle, we propose a novel domain adaptation kernelized support vector machine (DAKSVM) with respect to the classical SVM, and its two extensions called LS-DAKSVM and @m-DAKSVM with respect to the least-square SVM and the v-SVM, respectively. Moreover, our theoretical analysis justified that the proposed GPMDD metric could effectively measure the consistency not only between the RKHS embedding domain distributions but also between the scatter information of source and target domains. Hence, the proposed methods are distinctive in that the more consistency between the scatter information of source and target domains can be achieved by tuning the kernel bandwidth, the better the convergence of GPMDD metric minimization is and thus improving the scalability and generalization capability of the proposed methods for DAL. Experimental results on artificial and real-world problems indicate that the performance of the proposed methods is superior to or at least comparable with existing benchmarking methods.

The traditional mean shift based on kernel-histogram is of invariable kernel bandwidth and unable to represent accurately the color distribution of target and eliminate the accumulative tracking errors. To overcome these limitations, a tracking algorithm with the following improved strategies is proposed. Firstly, a modified second-order spatial histogram including spatial and color information is adopted to describe target model and the candidate, and Bhattacharyya distance is derived to evaluate the similarity between them. Secondly, target model is estimated repeatedly according to the target region parameter resulted from template drift correction which avoids the tracking window drifting problem. Then, an affine transform is established combining corner and edge detection to update the tracking window parameters. Finally, a Kalman filter or linear filter is selected to predict the target motion according to the Kalman residual error. The results of experiments show that the proposed algorithm works well and is robust against similarity distraction, scale and orientation variations and short-term occlusion.

In this letter, a method to optimally determine the kernel bandwidth of the Gaussian radial basis function (RBF) kernel for support vector (SV)-based hyperspectral anomaly detection is presented. In this method, the support of a local background distribution is first nonparametrically learned by a technique called SV data description (SVDD). The SVDD optimally models an enclosing hypersphere around the local background data in a high-dimensional feature space associated with the Gaussian RBF kernel. Any test pixel that lies outside this hypersphere surrounding the local background is considered an anomaly and, hence, a possible target pixel. Considerable improvement in detection performance due to kernel parameter optimization can be seen in the simulation results when the algorithm is applied to hyperspectral images.

A modified active subset selection method based on quadratic Renyi entropy and a fast cross-validation for fixed-size least squares support vector machines is proposed for classification and regression with optimized tuning process. The kernel bandwidth of the entropy based selection criterion is optimally determined according to the solve-the-equation plug-in method. Also a fast cross-validation method based on a simple updating scheme is developed. The combination of these two techniques is suitable for handling large scale data sets on standard personal computers. Finally, the performance on test data and computational time of this fixed-size method are compared to those for standard support vector machines and @n-support vector machines resulting in sparser models with lower computational cost and comparable accuracy.

The log-likelihood energy term in popular model-fitting segmentation methods, e.g. [39, 8, 28, 10], is presented as a generalized "probabilistic K-means" energy [16] for color space clustering. This interpretation reveals some limitations, e.g. over-fitting. We propose an alternative approach to color clustering using kernel K-means energy with well-known properties such as non-linear separation and scalability to higher-dimensional feature spaces. Our bound formulation for kernel K-means allows to combine general pair-wise feature clustering methods with image grid regularization using graph cuts, similarly to standard color model fitting techniques for segmentation. Unlike histogram or GMM fitting [39, 28], our approach is closely related to average association and normalized cut. But, in contrast to previous pairwise clustering algorithms, our approach can incorporate any standard geometric regularization in the image domain. We analyze extreme cases for kernel bandwidth (e.g. Gini bias) and demonstrate effectiveness of KNN-based adaptive bandwidth strategies. Our kernel K-means approach to segmentation benefits from higher-dimensional features where standard model fitting fails.

This paper presents an effective image structure classification method, which was recently proposed for selecting the key parameter of non-local kernel regression (NLKR) namely the kernel bandwidth. Meanwhile, to overcome the problem of intensive computation cost of the non-local patch searching in NLKR, a fast patch searching strategy is proposed according to the classified structure regions. The proposed structure-aware NLKR (SA-NLKR) is applied to the multiview image denoising problem and the effectiveness of the proposed algorithm is illustrated by experimental results.

A simple, efficient, and parameter-free approach is proposed for the problem of multiclass classification, and is especially useful when dealing with large-scale datasets in the presence of label noise. Grown out of one-class SVM, our approach enjoys several distinct features: First, its decision boundary is learned based on both positive and negative examples; Second, the internal parameters and especially the kernel bandwidth are self-tuned. Our approach is compared side-by-side with LIBSVM, arguably the most widely-used multiclass classification system, in a sequence of empirical evaluations, where our approach is shown to perform almost as well as their optimal parameter settings tuned for individual datasets, while consuming only a fraction of the processing time.

Kernel-based methods have become popular in machine learning; however, they are typically designed for numeric data. These methods are established in vector spaces, which are undefined for categorical data. In this paper, we propose a new kind of kernel trick, showing that mapping of categorical samples into kernel spaces can be alternatively described as assigning a kernel-based weight to each categorical attribute of the input space, so that common distance measures can be employed. A data-driven approach is then proposed to kernel bandwidth selection by optimizing feature weights. We also make use of the kernel-based distance measure to effectively extend nearest-neighbor classification to classify categorical data. Experimental results on real-world data sets show the outstanding performance of this approach compared to that obtained in the original input space.

Video surveillance systems are commonly designed based on background estimation and subtraction techniques to support further object detection and tracking procedures. In this paper, a new kernel-based adaptive learning approach, called Correntropy-based Adaptive Learning - CAL, is proposed to deal with stationary and non-stationary pixel dynamics to support video-based surveillance systems. To this end, an stochastic gradient based adaptive learning algorithm is enhanced using Correntropy cost function to learn, in a dynamic way, a background model that allows diminishing influence of intrinsic video artifacts. Moreover, automatic estimation of Correntropy kernel bandwidth value is carried out by means of shape variations of the CAL error distribution. Obtained results show that the CAL algorithm outperforms, in well-known real-world datasets, state of the art methodologies, suitably modeling the scene background with acceptable computational burden.

Discriminating Magnetic Resonance Images (MRI) allows supporting the analysis of physiological and pathological processes, however, finding MRI relationships posses a challenge when analyzing in voxel-based high-dimensional spaces. We introduce a kernel-based representation approach to support MRI discrimination. In this sense, inherent Inter-Slice Kernel relationship is employed to highlight brain structure distributions. Then, a generalized Euclidean metric is estimated by using a kernel-based centered alignment algorithm to code the correlation between MRI dependencies and prior demographic patient information. The proposed approach is tested on MRI data classification by considering patient gender and age categories. Attained results show that proposed methodology improves data interpretability and separability in comparison to state of the art algorithms based on MRI Voxel-wise features. Therefore, introduced kernel-based representation can be useful to support MRI clustering and similarity inference tasks required on template-based image segmentation and atlas construction.

We propose a kernel function estimation strategy to support machine learning tasks by analyzing the input samples using Renyi’s Information Metrics. Specifically, we aim to identify a Reproducing Kernel Hilbert Space spanning the most widely the information force among data points by the maximization of the information potential variability of Parzen-based pdf estimation. So, a Gaussian kernel bandwidth updating rule is obtained as a function of the forces induced by a given dataset. Our proposal is tested on synthetic and real-world datasets related to clustering and classification tasks. Obtained results show that presented approach allows to compute RKHS’s favoring data groups separability, attaining suitable learning performances in comparison with state of the art algorithms.

This paper proposes a new wavelet domain denoising algorithm. In the results of conventional wavelet domain denoising methods, ringing artifacts or wavelet-shaped noises are sometimes observed due to thresholding of small but important coefficients or due to generation of large coefficients in flat areas. In this paper, nonlocal means filtering is applied to each subband of wavelet decomposition, which can keep small coefficients and does not generate unwanted large coefficients. Since the performance of nonlocal means filtering depends on the appropriate kernel bandwidth, we also propose a method to find global and local kernel bandwidth for each subband. In comparison with conventional methods, the proposed method shows lower PSNR than BM3D when pseudo white Gaussian noise is added, but higher PSNR than the spatial nonlocal means filtering and wavelet thresholding methods. For the mixture noise or Poisson noise, which may better explain the real noise from camera sensors, the proposed method shows better or comparable results than the state-of-the-art methods. Also, it is believed that the proposed method shows better subjective quality for the noisy images captured in the low-illumination conditions.

This paper proposes a methodology for the application of multivariate kernel density estimation (KDE) to MMSE-based image/video error concealment (EC). We show that the estimation of the kernel bandwidth matrix for EC must follow a criterion different from that of typical KDE problems. In particular, we propose a bandwidth built as the product of a structure matrix and a scale factor obtained with a minimum square error criterion. We show that our proposal can achieve average PSNR improvements larger than 1 dB with respect to other state-of-the-art techniques.

In this paper, an optimal support vector based method to detect anomalies in hyperspectral images is presented. This method is based on a technique called Support Vector Data Description (SVDD) which learns the support of local background distribution by modeling an enclosing hypersphere around this data in a high dimensional feature space associated with Gaussian Radial Basis Function (RBF) kernel. Any test pixel that lies outside this hypersphere surrounding the local background is considered an anomaly and hence, a possible target pixel. For the Gaussian RBF kernel to perform well, the bandwidth parameter of the kernel function needs to be set optimally. The proposed algorithm is a two-step iterative method to optimize for this parameter: the volume of the enclosing hypersphere is minimized by optimizing the support vectors in one step and then subsequently further minimized with respect to the kernel bandwidth parameter in the next. Considerable increase in detection performance due to kernel parameter optimization can be seen in the simulation results when the algorithm is applied to real hyperspectral images.

We present a new image denoising method based on the nonlocal means filtering in the wavelet domain. A noisy image is first decomposed into subbands by wavelet transform and the nonlocal means filter is applied to each subband. It is also noted that the performance of the nonlocal means filter depends on the kernel bandwidth (size of the filter) and the image properties. Hence we propose a method to adjust the kernel bandwidth for each of the subband images, based on the estimation of noise statistics. This filtering method preserves the wavelet coefficients corresponding to the structures, while effectively suppressing noisy ones. Experimental results show that the proposed method provides comparable or sometimes higher peak signal-to-noise ratio (PSNR) than the state-of-the-art wavelet denoising methods and the spatial nonlocal means filter. Subjective comparison also shows that the proposed method provides better contrast than the spatial nonlocal means filter, and less ringing artifacts that commonly arise in the conventional wavelet denoising.

This paper addresses the problem of estimating multidimensional propagation time-delay parameters for multiple competitive sources. Complex-valued mixing parameters, measuring the high-order coherence of the acoustic waves recorded at microphone pairs, are estimated applying the Independent Component Analysis (ICA). A statistical framework is used to model the pdf of the whole underlying time-delay distribution and an approximated Gaussian kernel density estimator is derived. Especially for frequencies affected by high spatial aliasing (which occurs when using a large microphone spacing) an overestimation of the kernel bandwidth may reduce the quality of the estimated density or generate high likelihood regions in false locations. To reduce the interference problem among different sources and propagation dimensions we propose an enhancement of the original kernel through a twofold extension: 1) the introduction of non-Euclidean metrics in the multidimensional space 2) the adoption of self-clustering techniques to tackle the permutation problem from a data association point of view. The improved kernel explicitly models in one shot the high-order structure of the entire parameters estimated by ICA and offers a considerably better rejection of the interference across sources as well as across propagation dimensions. Extensive numerical simulations are reported to support the theoretical analysis.

In emission tomography (ET), fast developing Bayesian reconstruction methods can incorporate anatomical information derived from co-registered scanning modalities, such as magnetic resonance (MR) and computed tomography (CT). We propose a Bayesian image reconstruction method for single photon emission computed tomography (SPECT), using a joint entropy (JE) similarity measure to embed MR anatomical data. An optimized non-parametric Parzen window approach is used for fast and efficient estimation of the probability density function (PDF) of the JE metric. It is known that the quality of the Parzen estimates strongly depends on the kernel bandwidth of the smoothing function. When the density is over or under-smoothed, because of too large or small bandwidth value, this leads to an incorrect entropy estimate and, eventually, to a biased solution. To alleviate the problem of searching manually for the most suitable weight for the smoothing function and the number of bins for the histogram, we use an adaptive method to find these parameters automatically from the data on each iteration of the Bayesian algorithm. We assess the NRMSE-variance behaviour of the MAP-EM reconstruction method in relation to the quality of the PDF building. For the different bandwidth values of the Gaussian kernel for the density function, an emission image is reconstructed using MR data as a prior. Preliminary numerical experiments are performed using simulated co-registered 2D and 3D SPECT/MR data. Comparison of proposed technique with neighbourhood dependent anatomically-based prior is presented. Lesions are simulated to be apparent on the gray matter of the 3D SPECT data, but invisible on MRI. Preliminary results demonstrate that applying optimal density estimation for JE metric is feasible and more efficient compared to non-adaptive techniques

We propose a novel linear discriminant analysis method and demonstrate its superiority over existing linear methods. Based on information theory, we introduce a non-parametric estimate of mutual information with variable kernel bandwidth. Furthermore, we derive a gradient-based optimization algorithm for learning the optimal linear reduction vectors which maximizes the mutual information estimate. We evaluate the proposed method by running cross-validation on 2 data sets from the UCI repository, together with linear and nonlinear SVMs as classifiers. The result attests to the superority of the method over conventional LDA and its variant, aPAC.

A kernel-based anomaly detection technique called Kernel RX algorithm has been developed earlier by one of the authors, to be used as a prescreening tool that non-linearly detects anomalous pixel spectra in hyperspectral images. Targets of interest are then identified among the prescreened anomalous spectra based on reference spectral information using supervised classification/detection techniques. Kernel RX algorithm uses kernels like the Gaussian radial basis function (RBF) kernel to transform the given data into higher-dimensional (possibly infinite) feature space before detecting the anomalies. The efficiency of the algorithm depends on this transformation which in turn depends on the respective kernel parameters. The Gaussian RBF kernel has a parameter called bandwidth parameter. In this paper, a new method to determine the optimal full diagonal bandwidth parameters of the Gaussian RBF kernel is presented. First, cross-validation technique is used to estimate an optimal single bandwidth parameter. Then, the full diagonal parameters are estimated from this single parameter using the variances of the spectral bands of the hyperspectral image. It will be shown that the optimal full diagonal bandwidth parameters provide a better probability of detection at a given false alarm rate compared to the optimal single bandwidth parameter and other suboptimal bandwidth parameters when tested on hyperspectral imagery for military target detection.

Spectral Clustering (SC) is a kernel method to cluster data objects using eigenvectors derived from the data. One fundamental issue that cause poor cuts in SC is its sensitivity to outliers. Another fundamental problem is how to determine the kernel bandwidth from the data. In fact, these two problems are also closely related. One cannot be solved before solving the other. The answer lies in robust and nonparametric estimators of the data density. We propose Reverse Soft K-Nearest Neighbor Density Estimation (RSKNN) that determines the density around a data sample, thus this sample's potential (other used terms are weight or entropy), using all the other samples' nearest neighbors' scatter properties on the contrary to the common practice of using the nearest neighbors of a sample itself to determine its own density. The basic idea behind this can be summarized as “every sample have k neighbors that are the nearest but not every point can be in many points' k-nearest neighborhood”. To demonstrate the use of it, we apply it to SC. Our package to use in SC consists of using RSKNN for estimating the density and the samples with high potential to be cluster centers helps in: 1) spectral decomposition phase to improve the generalization of the spectral cut criterion, 2) robust calculation of covariance matrix to be used in distance calculations, and 3) automatically determining the kernel bandwidth.

Kernel density estimation is a nonparametric procedure for probability density modeling, which has found several applications in various fields. The smoothness and modeling ability of the functional approximation are controlled by the kernel bandwidth. In this paper, we describe a Bayesian estimation method for finding the bandwidth from a given data set. The proposed bandwidth estimation method is applied in three different computational-intelligence methods that rely on kernel density estimation: 1) scale space; 2) mean shift; and 3) quantum clustering. The third method is a novel approach that relies on the principles of quantum mechanics. This method is based on the analogy between data samples and quantum particles and uses the Schrodinger potential as a cost function. The proposed methodology is used for blind-source separation of modulated signals and for terrain segmentation based on topography information.

Within this paper we investigate the challenge to find an appropriate bandwidth in kernel density estimation. Kernel density estimation methods can be used in visualizing and analyzing spatial data, with the objective of understanding and potentially predicting event patterns. Our aim is to provide a computational tool to visually select the bandwidth parameter and to visually investigate the effect of different parameters within the kernel density calculations. With this tool the user is able to visually find the most appropriate bandwidth for the particular dataset and scale. The slider-tool includes a graphical user interface and can read point datasets. A specific number of kernel density maps are generated by using a range of bandwidths. In the graphical interface these kernel density maps can be drawn near real-time while the user changes the respective bandwidth using a slider tool. This helps to determine which bandwidth setting fits the needs and provide an appropriate visualization of a particular point dataset. Refereed/Peer-reviewed

An improved mean shift method for object tracking based on nonparametric clustering and adaptive bandwidth is presented in this paper. Based on partitioning the color space of a tracked object by using a modified nonparametric clustering, an appearance model of the tracked object is built. It captures both the color information and spatial layout of the tracked object. The similarity measure between the target model and the target candidate is derived from the Bhattacharyya coefficient. The kernel bandwidth parameters are automatically selected by maximizing the lower bound of a log-likelihood function, which is derived from a kernel density estimate using the bandwidth matrix and the modified weight function. The experimental results show that the method can converge in an average of 2.6 iterations per frame.

In this paper, two unsupervised methods for multi-band change detection are presented. Both methods model the multi-band difference image histogram in order to characterize different degrees of observed spectral changes. In the first approach, we extend the single-band change detection algorithm proposed by Prieto and Bruzzone in which a two-component mixture density is fit to the observed difference image histogram, where the components correspond to the changed and unchanged populations. The second approach employs the hierarchical modal associative clustering algorithm proposed by Li et al., in which a hierarchy of kernel densities at different bandwidths is employed to model the multi-band difference image histogram. The kernel density modes correspond to different scales of changes and are analyzed with respect to increasing kernel bandwidth so that changes occurring at different scales may be identified. Experiments, carried out on ASTER data; are conducted to display the changes captured by each method as well as to illustrate how the degrees of detected changes can be interpreted with respect to model complexity or scale.

Natural images consist of texture, structure and smooth regions and this makes the task of filtering challenging mainly when it aims at edge and texture preservation. In this paper, we present a novel adaptive filtering technique based on a partition of the image to "noisy smooth zones" and "texture or edge + noise" zones. To this end, an analysis of local features is used to recover a statistical model that associates to each pixel a probability measure corresponding to a membership degree for each class. This probability function is then encoded in a new denoising process based on a MPM (Marginal Posterior Mode) estimation technique. The posterior density is computed through a non parametric density estimation method with variable kernel bandwidth that aims to adapt the denoising process to image structure. In our algorithm the selection of the bandwidth relies on a non linear function of the membership probabilities. Encouraging, experimental results demonstrate the potential of our approach.