Search

The adaptive neurofuzzy inference system (ANFIS) is a structured multioutput learning machine that has been successfully adopted in learning problems without noise or outliers. However, it does not work well for learning problems with noise or outliers. High-accuracy real-time forecasting of traffic flow is extremely difficult due to the effect of noise or outliers from complex traffic conditions. In this study, a novel probabilistic learning system, probabilistic regularized extreme learning machine combined with ANFIS (probabilistic R-ELANFIS), is proposed to capture the correlations among traffic flow data and, thereby, improve the accuracy of traffic flow forecasting. The new learning system adopts a fantastic objective function that minimizes both the mean and the variance of the model bias. The results from an experiment based on real-world traffic flow data showed that, compared with some kernel-based approaches, neural network approaches, and conventional ANFIS learning systems, the proposed probabilistic R-ELANFIS achieves competitive performance in terms of forecasting ability and generalizability.

The Micro-expressions (MEs) are very important nonverbal communication clues. However, due to their local and short nature, spotting them is challenging. In this paper, we address this problem by using a dedicated local and temporal pattern (LTP) of facial movement. This pattern has a specific shape (S-pattern) when ME are displayed. Thus, by using a classical classification algorithm (SVM), MEs are distinguished from other facial movements. We also propose a global final fusion analysis on the whole face to improve the distinction between ME (local) and head (global) movements. However, the learning of S-patterns is limited by the small number of ME databases and the low volume of ME samples. Hammerstein models (HMs) are known to be a good approximation of muscle movements. By approximating each S-pattern with a HM, we can both filter outliers and generate new similar S-patterns. By this way, we perform a data augmentation for S-pattern training dataset and improve the ability to differentiate micro-expressions from other facial movements. The spotting results, performed on CASME and CASME, show that our proposed LTP outperforms the most popular spotting method in terms of F1-score. Adding the fusion process and data augmentation improve even more the spotting performance.

We consider calculation of the mode of compositional data, the data related to each other through a linear constraint. Compositional data arises in various extensions of the fuzzy sets theory (type-2 interval-valued, intuitionistic, hesitant fuzzy sets), biomedicine (relative abundance, genome sequencing, activity recognition), and data analytics (various wealth indices, interval-valued observations, traffic congestion, etc.). Mode is a pre-aggregation function in the case of single variable, used as a classical estimator robust to outliers, but its multivariate extensions face the challenges of high computational complexity and potential oversmoothing. In this work we present several novel techniques for mode estimation on the unit k-simplex representing compositional, interval-valued, and general vector-valued data. We highlight the re-weighted k-nearest neighbours algorithm based on the Choquet integral with respect to a 2-additive fuzzy measure, compare its performance against other approaches based on spatial partitioning, and illustrate its applications to aggregation of real-world interval-valued data sets.

Wireless Sensor Networks (WSNs) are developed significantly from the last decades and attracted the attention of scientific and industrial domains. In WSNs, sensor nodes distributed autonomously in harsh environments are easily vulnerable to faults and attacks that cause sensor readings unreliable and inaccurate. In this scenario, sensor readings that have differed considerably from healthy behaviors will be considered abnormal data or anomalies/outliers. The inclusion of such outliers in data analytics will inevitably affect the outcome of the decision-making process. Thus, detecting outliers in WSNs using data-driven approaches becomes a novel technique among the Machine Learning (ML) communities. Meanwhile, various research issues are there in measuring the performance of the deployed ML algorithms in detecting outliers in WSNs, which generally contains minimum resources in terms of computational capability and power sources to ensure data quality. Hence, this paper presents a comprehensive overview of the state-of-the-art Statistical and Artificial Intelligence (AI) based techniques used in WSNs to detect outliers in the view of architecture, type, degree, approach, and detection mode. Furthermore, each aforementioned outlier detection approach is presented with detailed discussions and future scope for developments.

This article explores the problem of semisupervised affinity matrix learning, that is, learning an affinity matrix of data samples under the supervision of a small number of pairwise constraints (PCs). By observing that both the matrix encoding PCs, called pairwise constraint matrix (PCM) and the empirically constructed affinity matrix (EAM), express the similarity between samples, we assume that both of them are generated from a latent affinity matrix (LAM) that can depict the ideal pairwise relation between samples. Specifically, the PCM can be thought of as a partial observation of the LAM, while the EAM is a fully observed one but corrupted with noise/outliers. To this end, we innovatively cast the semisupervised affinity matrix learning as the recovery of the LAM guided by the PCM and EAM, which is technically formulated as a convex optimization problem. We also provide an efficient algorithm for solving the resulting model numerically. Extensive experiments on benchmark datasets demonstrate the significant superiority of our method over state-of-the-art ones when used for constrained clustering and dimensionality reduction. The code is publicly available at https://github.com/jyh-learning/LAM.

The problem of representative selection amounts to sampling few informative exemplars from large datasets. Existing approaches to data selection often fall short of simultaneously handling non-linear data structures, sampling concise and non-redundant subsets, rejecting outliers, and yielding interpretable outcomes. This paper presents a novel representative selection approach, dubbed MOSAIC, for drawing descriptive sketches of arbitrary manifold structures. Resting upon a novel quadratic formulation, MOSAIC advances a multi-criteria selection approach that maximizes the global representation power of the sampled subset, ensures novelty of the samples by minimizing redundancy, and rejects disruptive information by effectively detecting outliers. Theoretical analyses shed light on geometrical characterization of the obtained sketch and reveal that the sampled representatives maximize a well-defined notion of data coverage in a transformed space. In addition, we present a highly scalable randomized implementation of the proposed algorithm shown to bring about substantial speedups. MOSAIC's superiority in achieving the desired characteristics of a representative subset all at once while exhibiting remarkable robustness to various outlier types is demonstrated via extensive experiments conducted on both real and synthetic data with comparisons to state-of-the-art algorithms.

Precise prediction of risk for dyslexia among children’s in earlier stages is a significant long-term aim in the field of cognitive computing. Producing such accurate results for detection of dyslexia from a dataset which consist of low-quality dataset and the presence of vague information is the toughest challenge among researchers. This paper aims at developing an evolving model to handle the impreciseness in the detection of dyslexia more intelligently. In this work, each instance is described in a neutrosophic domain by defining a membership degree of truthiness, indeterminacy, and falsity. These instances are neutrosophically clustered by applying Neutrosophic C-Means clustering (NCM) which forms four different clusters namely dyslexia, no dyslexia, control/revision and hyperactivity or other issues. The outlier and noise are the special categories of indeterminacy which often occurs in real datasets are promptly discovered and clustered. NCM is optimized by introducing Artificial Bacterial Foraging (ABF), especially when there is vagueness or imprecision in the selection of cluster centroids. With the merits of global searching, ABF selects more promising clustering during cluster re-computation. The interpreted results confirm that the role played by proposed ONCMC-ABF algorithm produces better results in the prediction of dyslexia with the low-quality dataset.

In this paper, we propose a novel probabilistic variant of iterative closest point (ICP) dubbed as CoBigICP. The method leverages both local geometrical information and global noise characteristics. Locally, the 3D structure of both target and source clouds are incorporated into the objective function through bidirectional correspondence. Globally, error metric of correntropy is introduced as noise model to resist outliers. Importantly, the close resemblance between normal-distributions transform (NDT) and correntropy is revealed. To ease the minimization step, an on-manifold parameterization of the special Euclidean group is proposed. Extensive experiments validate that CoBigICP outperforms several well-known and state-of-the-art methods., Comment: 6 pages, 4 figures. Accepted to IROS2020

In this paper, we have proposed a novel approach for detecting intrusive activities in databases by the use of clustering and information fusion through ensemble learning. We have applied OPTICS clustering on the transaction attributes for building user behavioral profiles. A transaction is initially passed through the clustering module for computing its cluster belongingness and an outlier factor that signifies its degree of outlierness. Depending on the outlier factor value, the transaction is classified as genuine or an outlier. Each outlier transaction is further analyzed by passing it onto an Ensemble Learner that applies three different aggregation methods, bagging, boosting and stacking. We have conducted experiments using stochastic models to demonstrate the effectiveness of the proposed system. The performance of the three different ensembles are evaluated and compared based on various metrics. Moreover, our system is found to exhibit better performance as compared to other approaches taken from the literature.

This article is concerned with a new outlier-resistant observer-based control problem for a class of networked systems (NSs) under the encoding–decoding communication mechanism (EDCM). In order to lighten the communication burden and enhance the data security, the EDCM is introduced in the observer-to-controller channel. In case of the measurement outliers, a specific saturation function is adopted in the observer structure to restrain the abnormal innovations so as to mitigate the side effects brought from the outliers. The aim of this article is to design an observer-based controller, such that in the presence of measurement outliers, the closed-loop NS achieves the input-to-state stability (ISS) under the EDCM. By means of the uniform quantization technique, a criterion is first established in terms of the sizes of the encoding alphabet and the encoding period so as to ensure the detectability of the NS, and the requirement is also given on the capacity of the communication channel at each time instant. Then, with the help of ISS theory, the desired controller is obtained with its gain matrix parameterized by the solution to a certain inequality that can be solved via standard software packages. Finally, the effectiveness of the derived theoretical results is verified through three numerical simulation examples.

Probabilistic linear discriminant analysis (PLDA) is a very effective feature extraction approach and has obtained extensive and successful applications in supervised learning tasks. It employs the squared L₂-norm to measure the model errors, which assumes a Gaussian noise distribution implicitly. However, the noise in real-life applications may not follow a Gaussian distribution. Particularly, the squared L₂-norm could extremely exaggerate data outliers. To address this issue, this article proposes a robust PLDA model under the assumption of a Laplacian noise distribution, called L1-PLDA. The learning process employs the approach by expressing the Laplacian density function as a superposition of an infinite number of Gaussian distributions via introducing a new latent variable and then adopts the variational expectation-maximization (EM) algorithm to learn parameters. The most significant advantage of the new model is that the introduced latent variable can be used to detect data outliers. The experiments on several public databases show the superiority of the proposed L1-PLDA model in terms of classification and outlier detection.

The linear regression model remains an important workhorse for data scientists. However, many data sets contain many more predictors than observations. Besides, outliers, or anomalies, frequently occur. This paper proposes an algorithm for regression analysis that addresses these features typical for big data sets, which we call “sparse shooting S”. The resulting regression coefficients are sparse, meaning that many of them are set to zero, hereby selecting the most relevant predictors. A distinct feature of the method is its robustness with respect to outliers in the cells of the data matrix. The excellent performance of this robust variable selection and prediction method is shown in a simulation study. A real data application on car fuel consumption demonstrates its usefulness.

Remarkable achievements by deep neural networks stand on the development of excellent stochastic gradient descent methods. Deep-learning-based machine learning algorithms, however, have to find patterns between observations and supervised signals, even though they may include some noise that hides the true relationship between them, more or less especially in the robotics domain. To perform well even with such noise, we expect them to be able to detect outliers and discard them when needed. We, therefore, propose a new stochastic gradient optimization method, whose robustness is directly built in the algorithm, using the robust student-t distribution as its core idea. We integrate our method to some of the latest stochastic gradient algorithms, and in particular, Adam, the popular optimizer, is modified through our method. The resultant algorithm, called t-Adam, along with the other stochastic gradient methods integrated with our core idea is shown to effectively outperform Adam and their original versions in terms of robustness against noise on diverse tasks, ranging from regression and classification to reinforcement learning problems.

Recently, several robust principle component analysis (RPCA) models have been proposed to improve the robustness of principle component analysis (PCA) model. However, an obvious problem that the improvement of robustness on outliers affects the discrimination of correct samples, has not been solved yet. In this paper, we aim to treat correct samples and outliers differently via proposing a truncated robust principle component analysis model (T-RPCA). The proposed T-RPCA model has high interpretation for the robustness of outliers and discrimination of correct samples. Moreover, we propose a general optimization framework named re-weighted (RW) framework to solve a general optimization problem and generalize two sub-frameworks upon it. 1) The first one orients a general truncation loss optimization problem which contains objective problem of T-RPCA model. 2) The second sub-framework focuses on a general singular-value based optimization problem which is useful in many problems. Besides, we provide rigorously theoretical guarantees for proposed model, optimization framework and sub-frameworks. Empirical studies demonstrate that the proposed T-PRCA outperform than previous RPCA methods for reconstruction and classification tasks.

Semisupervised learning (SSL) has been widely used in numerous practical applications where the labeled training examples are inadequate while the unlabeled examples are abundant. Due to the scarcity of labeled examples, the performances of the existing SSL methods are often affected by the outliers in the labeled data, leading to the imperfect trained classifier. To enhance the robustness of SSL methods to the outliers, this article proposes a novel SSL algorithm called Laplacian Welsch regularization (LapWR). Specifically, apart from the conventional Laplacian regularizer, we also introduce a bounded, smooth, and nonconvex Welsch loss which can suppress the adverse effect brought by the labeled outliers. To handle the model nonconvexity caused by the Welsch loss, an iterative half-quadratic (HQ) optimization algorithm is adopted in which each subproblem has an ideal closed-form solution. To handle the large datasets, we further propose an accelerated model by utilizing the Nyström method to reduce the computational complexity of LapWR. Theoretically, the generalization bound of LapWR is derived based on analyzing its Rademacher complexity, which suggests that our proposed algorithm is guaranteed to obtain satisfactory performance. By comparing LapWR with the existing representative SSL algorithms on various benchmark and real-world datasets, we experimentally found that LapWR performs robustly to outliers and is able to consistently achieve the top-level results.

Graph and subspace clustering methods have become the mainstream of multi-view clustering due to their promising performance. However, (1) since graph clustering methods learn graphs directly from the raw data, when the raw data is distorted by noise and outliers, their performance may seriously decrease; (2) subspace clustering methods use a “two-step” strategy to learn the representation and affinity matrix independently, and thus may fail to explore their high correlation. To address these issues, we propose a novel multi-view clustering method via learning a Low-Rank Tensor Graph (LRTG). Different from subspace clustering methods, LRTG simultaneously learns the representation and affinity matrix in a single step to preserve their correlation. We apply Tucker decomposition and l2;1-norm to the LRTG model to alleviate noise and outliers for learning a “clean” representation. LRTG then learns the affinity matrix from this “clean” representation. Additionally, an adaptive neighbor scheme is proposed to find the K largest entries of the affinity matrix to form a flexible graph for clustering. An effective optimization algorithm is designed to solve the LRTG model based on the alternating direction method of multipliers. Extensive experiments on different clustering tasks demonstrate the effectiveness and superiority of LRTG over seventeen state-of-the-art clustering methods.

Nonnegative matrix factorization (NMF) is a widely used hyperspectral unmixing model which decomposes a known hyperspectral data matrix into two unknown matrices, i.e., endmember matrix and abundance matrix. Due to the use of least-squares loss, the NMF model is usually sensitive to noise or outliers. To improve its robustness, we introduce a general robust loss function to replace the traditional least-squares loss and propose a general loss-based NMF (GLNMF) model for hyperspectral unmixing in this letter. The general loss function is a superset of many common robust loss functions and is suitable for handling different types of noise. Experimental results on simulated and real hyperspectral data sets demonstrate that our GLNMF model is more accurate and robust than existing NMF methods.

In data mining, outlier detection is applied in different domains. It has very large applications such as energy consumption analysis, forecasting hurricanes in meteorological data, fraud and intrusion detection, event detection and system monitoring in sensor networks, etc. Most of existing outlier detection techniques depend on the properties of a particular type of data and can not deal with a large volume of data well, which mean that there is a necessity for improved methodologies and techniques to be applied to a large amount of data with different types in other application areas. In this paper, a parallel outlier detection technique is developed to detect the outliers in the sequential data. Although there are many types of outliers, this paper concentrates on the contextual anomalies. The proposed technique uses a graph approach to detect the outliers. It is very flexible, fast, and no labeled data is needed comparing to many previous approaches. The experimental results show the detected contextual outliers in the sequential data, as well as the efficient scaling up to handle the massive data by increasing the number of processors. The results prove that the parallelism of the proposed technique is very valuable.

Anomaly detection suffers from unbalanced data since anomalies are quite rare. Synthetically generated anomalies are a solution to such ill or not fully defined data. However, synthesis requires an expressive representation to guarantee the quality of the generated data. In this paper, we propose a two-level hierarchical latent space representation that distills inliers' feature-descriptors (through autoencoders) into more robust representations based on a variational family of distributions (through a variational autoencoder) for zero-shot anomaly generation. From the learned latent distributions, we select those that lie on the outskirts of the training data as synthetic-outlier generators. And, we synthesize from them, i.e., generate negative samples without seen them before, to train binary classifiers. We found that the use of the proposed hierarchical structure for feature distillation and fusion creates robust and general representations that allow us to synthesize pseudo outlier samples. And in turn, train robust binary classifiers for true outlier detection (without the need for actual outliers during training). We demonstrate the performance of our proposal on several benchmarks for anomaly detection., To appear in IEEE Trans. on Neural Networks and Learning Systems

Detecting outliers in real-time is increasingly important for many real-world applications such as detecting abnormal heart activity, intrusions to systems, spams or abnormal credit card transactions. However, detecting outliers in data streams rises many challenges such as high-dimensionality, dynamic data distribution and unpredictable relationships. Our simulations demonstrate that some advanced solutions still show drawbacks. In this paper, first, we improve the capacity to detect outliers of both micro-clusters based algorithms (MCOD) and distance-based algorithms (Abstract-C and Exact-Storm) known for their performance. This is by adding a layer called LiCS that classifies online the K-nearest-neighbors (Knn) of each node based on their evolutionary status. This layer aggregates the results and uses a count threshold to better classify nodes. Experiments on SpamBase datasets confirmed that our technique enhances the accuracy and the precision of such algorithm and helps to reduce the unclassified nodes.Second, we propose a hybrid solution based on iterative majority voting and our LiCS. Experiments on real data proves that it outperforms discussed algorithms in terms of accuracy, precision and sensitivity in detecting outliers. It also minimizes the issue of unclassified instances and consolidate the different outputs of algorithms.

Multi-sensor data fusion has recently gained a wide attention within the Geomatics research community, as it helps overcome the limitations of a single sensor and enables a complete 3D model for the structure and a better object classification. This study develops a data fusion algorithm, which optimally combines sensor data from a terrestrial and an unmanned aerial system (UAS) to obtain an improved and a complete 3D mapping model of a structure. Terrestrial laser scanner (TLS) data are collected for the exterior of a building along with the DJI Phantom 4 Pro and terrestrial close-range Sony α7R camera images. A number of ground control points and targets are established throughout the scanned building for the photogrammetric process and scans registration. Different point cloud datasets are generated from the TLS, UAS and the terrestrial Sony camera images. The created point clouds from each individual sensor and the fused point clouds are used in different forms, namely the original, denoised and subsampled point clouds. The denoised point cloud dataset is generated through the application of the statistical outlier remover (SOR) filter on the original point clouds. The relative precision of the 3D models is investigated using the multiscale model-to-model cloud comparison (M3C2) method. The TLS-based 3D model is used as a reference. It is found that the precision of the Sony-based 3D model is higher than the other two models for the original and denoised datasets. The fused Sony/UAS-based model provides a complete 3D model with precision higher than the UAS-based model.

In the past years, the practical cross-domain machinery fault diagnosis problems have been attracting growing attention, where the training and testing data are collected from different operating conditions. The recent advances in closed-set domain adaptation have well addressed the basic problem where the fault mode sets are identical in the source and target domains. While some attempts have also been made on the partial and open-set domain adaptations, no prior information of the target-domain fault modes can be usually available in the real industries, that forms a challenging problem in transfer learning. This article proposes a universal domain adaptation method for fault diagnosis, where no explicit assumption is made on the target label set. A hybrid approach with source class-wise and target instance-wise weighting mechanism is proposed for selective adaptation. By using additional outlier identifier, the proposed method can automatically recognize the unknown fault modes while achieving class-level alignments for the shared health states, without knowing the target label set. Experiments on two rotating machine datasets validate the proposed method, which is promising for practical applications under strong data uncertainties.

The common spatial pattern (CSP) algorithm is a well-recognized spatial filtering method for feature extraction in motor imagery (MI)-based brain-computer interfaces (BCIs). However, due to the influence of nonstationary in electroencephalography (EEG) and inherent defects of the CSP objective function, the spatial filters, and their corresponding features are not necessarily optimal in the feature space used within CSP. In this work, we design a new feature selection method to address this issue by selecting features based on an improved objective function. Especially, improvements are made in suppressing outliers and discovering features with larger interclass distances. Moreover, a fusion algorithm based on the Dempster-Shafer theory is proposed, which takes into consideration the distribution of features. With two competition data sets, we first evaluate the performance of the improved objective functions in terms of classification accuracy, feature distribution, and embeddability. Then, a comparison with other feature selection methods is carried out in both accuracy and computational time. Experimental results show that the proposed methods consume less additional computational cost and result in a significant increase in the performance of MI-based BCI systems.

Most robust estimators require tuning the parameters of the algorithm for the particular application, a bottleneck for practical applications. The paper presents the multiple input structures with robust estimator (MISRE), where each structure, inlier or outlier, is processed independently. The same two constants are used to find the scale estimates over expansions for each structure. The inlier/outlier classification is straightforward since the data is processed and ordered with the relevant inlier structures listed first. If the inlier noises are similar, MISRE's performance is equivalent to RANSAC-type algorithms. MISRE still returns the correct inlier estimates when inlier noises are very different, while RANSAC-type algorithms do not perform as well. MISRE's failures are gradual when too many outliers are present, beginning with the least significant inlier structure. Examples from 2D images and 3D point clouds illustrate the estimation.

Recent advances in implementing machine learning (ML) methods in hydrology have given rise to a new, data-driven approach to hydrological modeling. Comparison of physically based and ML approaches has shown that ML methods can achieve a similar accuracy to the physically based ones and outperform them when describing nonlinear relationships. Global ML models have been already successfully applied for modeling hydrological phenomena such as discharge.However, a major problem related to large-scale water quality modeling has been the lack of available observation data with a good spatiotemporal coverage. This has affected the reproducibility of previous studies and the potential improvement of existing models. In addition to the observation data itself, insufficient or poor quality metadata has also discouraged researchers to integrate the already available datasets. Therefore, improving both, the availability, and quality of open water quality data would increase the potential to implement predictive modeling on a global scale.We aim to address the aforementioned issues by presenting the new Global River Water Quality Archive (GRQA) by integrating data from five existing global and regional sources:Canadian Environmental Sustainability Indicators program (CESI) Global Freshwater Quality Database (GEMStat) GLObal RIver Chemistry database (GLORICH) European Environment Agency (Waterbase) USGS Water Quality Portal (WQP) The resulting dataset contains a total of over 14 million observations for 41 different forms of some of the most important water quality parameters, focusing on nutrients, carbon, oxygen and sediments. Supplementary metadata and statistics are provided with the observation time series to improve the usability of the dataset. We report on developing a harmonized schema and reproducible workflow that can be adapted to integrate and harmonize further data sources. We conclude our study with a call for action to extend this dataset and hope that the provided reproducible method of data integration and metadata provenance shall lead as an example.

This paper proposes a novel method for offline outlier detection in nonlinear dynamical systems using an input–output dataset of a Topical Negative Pressure Wound Therapy Device, NPWT. The fundamental characteristics of an NPWT describe a chaotic system whose states vary over time and may result in unpredictable and possibly anomalous divergent behavior in the presence of perturbations and other unmodeled system dynamics, despite a quasi-stable controller. Bacterial Memetic Algorithm, BMA, is used to generate fuzzy rule-based models of the input–output dataset. The error definition in the fuzzy rule extraction features a novel application of the Canberra Distance. The optimal number of rules for identifying the outliers, validated against both artificial and real system datasets, is calculated from the sample of inferred fuzzy models. The optimal number of rules is two in both cases based on the maximum average-error-drop. Using three or more rules results in better error performance; however, the algorithm learns the nuances of the outlier patterns instead. Novel methods for creating the outlier list and determining the optimal number of rules for the outlier detection problem are proposed.

With the rapid development of Internet of Things (IoT) technologies, the detection and analysis of malware have become a matter of concern in the industrial application of Cyber-Physical System (CPS) that provides various services using the IoT paradigm. Currently, many advanced machine learning methods such as deep learning are popular in the research of malware detection and analysis, and some achievements have been made so far. However, there are also some problems. For example, considering the noise and outliers in the existing datasets of malware, some methods are not robust enough. Therefore, the accuracy of malware classification still needs to be improved. Aiming at this issue, we propose a novel method that combines the correntropy and the deep learning model. In our proposed method for malware detection and analysis, given the success of the mixture correntropy as an effective similarity measure in addressing complex datasets with noise, it is therefore incorporated into a popular deep learning model, i.e., Convolutional Neural Network (CNN), to reconstruct its loss function, with the purpose of further detecting the features of outliers. We present the detailed design process of our method. Furthermore, the proposed method is tested both on a real-world malware dataset and a popular benchmark dataset, to verify its learning performance.

This paper proposed a novel Robust Multi-View Continuous Subspace Clustering (RMVCSC) algorithm, which can untangle heavily mixed clusters by optimizing a single continuous objective. The proposed objective uses robust estimators to automatically clip specious inter-cluster connections while maintaining convincing intra-cluster correspondences in the common representation subspace learned from multiple views. The common representation subspace can reveal the underlying cluster structure in data. RMVCSC is optimized in an alternating minimization scheme, in which the clustering result and the common representation subspace are simultaneously optimized. Since different views can describe distinct perspectives of input data, the proposed algorithm has more accurate clustering performance than conventional algorithms by exploring information among multi-view data. In other words, the proposed algorithm optimizes a novel continuous objective in the simultaneously learned common representation subspace across multiple views. By using robust redescending estimators, the proposed algorithm is not prone to stick into bad local minima even with outliers in data. This kind of robust continuous clustering methods has never been used for multi-view clustering before. Moreover, the convergence of the proposed algorithm is theoretically proved, and the experimental results show that the proposed RMVCSC can outperform several very recent proposed algorithms in terms of clustering accuracy.

Multi-view clustering, which utilizes the ample information provided by multiple sources to obtain better performance, has attracted much attention. However, existing clustering algorithms either have no ability to offer confidence for each assignment or suffer from the disturbance of outliers. To address these problems, in this paper, we propose a novel multi-view fuzzy clustering method via transferring softmin to fuzzy models. To obtain fuzzy assignments, we utilize the softmin with temperature and further develop an efficient algorithm to solve the non-convex problem approximately. We also show another explanation for the algorithm from the aspect of the prior distribution of various views. Besides, we design a scalable robust loss function, which interpolates between l2-norm and the squared l2-norm, to enhance the robustness to outliers. Extensive experiments show the superiority of our model under different clustering metrics.

This article focuses on the challenges of hit finding and data reduction in serial crystallography (SX). An effective and reliable Bragg-peak-finding method, called robust peak finder (RPF), has been developed. RPF is based on the principle of robust statistics and can be used for SX data analysis., A peak-finding algorithm for serial crystallography (SX) data analysis based on the principle of ‘robust statistics’ has been developed. Methods which are statistically robust are generally more insensitive to any departures from model assumptions and are particularly effective when analysing mixtures of probability distributions. For example, these methods enable the discretization of data into a group comprising inliers (i.e. the background noise) and another group comprising outliers (i.e. Bragg peaks). Our robust statistics algorithm has two key advantages, which are demonstrated through testing using multiple SX data sets. First, it is relatively insensitive to the exact value of the input parameters and hence requires minimal optimization. This is critical for the algorithm to be able to run unsupervised, allowing for automated selection or ‘vetoing’ of SX diffraction data. Secondly, the processing of individual diffraction patterns can be easily parallelized. This means that it can analyse data from multiple detector modules simultaneously, making it ideally suited to real-time data processing. These characteristics mean that the robust peak finder (RPF) algorithm will be particularly beneficial for the new class of MHz X-ray free-electron laser sources, which generate large amounts of data in a short period of time.

BACKGROUND: Motor imagery electroencephalogram (MI-EEG) play an important role in the field of neurorehabilitation, and a fuzzy support vector machine (FSVM) is one of the most used classifiers. Specifically, a fuzzy c-means (FCM) algorithm was used to membership calculation to deal with the classification problems with outliers or noises. However, FCM is sensitive to its initial value and easily falls into local optima. OBJECTIVE: The joint optimization of genetic algorithm (GA) and FCM is proposed to enhance robustness of fuzzy memberships to initial cluster centers, yielding an improved FSVM (GF-FSVM). METHOD: The features of each channel of MI-EEG are extracted by the improved refined composite multivariate multiscale fuzzy entropy and fused to form a feature vector for a trial. Then, GA is employed to optimize the initial cluster center of FCM, and the fuzzy membership degrees are calculated through an iterative process and further applied to classify two-class MI-EEGs. RESULTS: Extensive experiments are conducted on two publicly available datasets, the average recognition accuracies achieve 99.89% and 98.81% and the corresponding kappa values are 0.9978 and 0.9762, respectively. CONCLUSION: The optimized cluster centers of FCM via GA are almost overlapping, showing great stability, and GF-FSVM obtains higher classification accuracies and higher consistency as well.

X-ray crystallography is the most popular approach for analyzing protein 3D structure. However, the success rate of protein crystallization is very low (2-10 percent). To reduce the cost of time and resources, lots of computation-based methods are developed to detect the protein crystallization. Improving the accuracy of predicting protein crystallization is very important for the determination of protein structure by X-ray crystallography. At present, many machine learning methods are used to predict protein crystallization. In this article, we propose a Fuzzy Support Vector Machine based on Linear Neighborhood Representation (FSVM-LNR) to predict the crystallization propensity of proteins. Proteins are represented by three types of features (PsePSSM, PSSM-DWT, MMI-PS), and these features are serially combined and fed into FSVM-LNR. FSVM-LNR can filter outliers by membership score, which is calculated via reconstruction residuals of $k$ k nearest samples. To evaluate the performance of our predictive model, we test FSVM-LNR on the datasets of TRAIN3587, TEST3585 and TEST500. Our method achieves better Mathew’s correlation coefficient (MCC) on TRAIN3587 (MCC: 0.56) and TEST3585 (MCC: 0.58). Although the performance of independent test is not the best on TEST500, FSVM-LNR also has a certain predictability (MCC: 0.70) in the identification of protein crystallization. The good performance on the datasets proves the effectiveness of our method and the better performance on large datasets further demonstrates the stability and superiority of our method.

Fuzzy information entropy based on fuzzy relation in fuzzy rough set theory is an important metric of uncertainty. However, the research of fuzzy information entropy for hybrid feature outlier detection has not been reported. On this basis, this paper constructs a hybrid feature outlier detection method based on fuzzy information entropy by using fuzzy approximate space with fuzzy similarity relation. Firstly, the adaptive fuzzy radius of standard deviation and hybrid fuzzy similarity are employed to construct the fuzzy approximate space, and the relative fuzzy entropy is defined based on the fuzzy information entropy. Then, two kinds of metrics are constructed to describe the outlier degree of object. Finally, the fuzzy entropy-based outlier factor is integrated to implement outlier detection, and the relevant fuzzy information entropy-based outlier detection algorithm (FIEOD) is designed. The FIEOD algorithm is compared with the main outlier detection algorithms on public data. The experimental results reveal that the proposed method has better effectiveness and adaptability.

The Gaussian process is a powerful statistical learning model and has been applied widely in nonlinear regression and classification. However, it fails to model multi-modal data from a non-stationary source since a prior Gaussian process is generally stationary. Based on the idea of the mixture of experts, the mixture of Gaussian processes was established to increase the model flexibility. On the other hand, the Gaussian process is also sensitive to outliers and thus robust Gaussian processes have been suggested to own the heavy-tailed property. In practical applications, the datasets may be multi-modal and contain outliers at the same time. In order to overcome these two difficulties together, we propose a mixture of robust Gaussian processes (MRGP) model and establish a precise hard-cut EM algorithm for learning its parameters. Since the exact solving process is intractable due to the fact that non-Gaussian probability density functions of the noises are adopted into the likelihood of the proposed model on the dataset, we employ a variational bounding method to approximate the marginal likelihood functions so that the hard-cut EM algorithm can be implemented effectively. Moreover, we conduct various experiments on both synthetic and real-world datasets to evaluate and compare our proposed MRGP method with several competitive nonlinear regression methods. The experimental results demonstrate that our MRGP model with the hard-cut EM algorithm is much more effective and robust than the competitive nonlinear regression models.

The stochastic multi-armed bandit problem is a standard model to solve the exploration–exploitation trade-off in sequential decision problems. In clinical trials, which are sensitive to outlier data, the goal is to learn a risk-averse policy to provide a trade-off between exploration, exploitation, and safety. In this paper, we present a risk-averse multi-armed bandit algorithm to solve a decision-making problem based on the social engagement behaviors of children with Autism Spectrum Disorder (ASD). The algorithm is carried out when children interact with a humanoid robot and imitate a sequence of the robot's movements. The proposed algorithm is based on the Best Empirical Sampled Average algorithm under Entropic Value-at-Risk as a risk measure to decide on the best sequence of movements that can improve the social engagement behaviors of the children with ASD while imitating the robot's movements. We provide a detailed experimental analysis to compare the performance of our proposed algorithm to some well-known risk-averse multi-armed bandit algorithms on some artificial scenarios and our real-world problem. The experimental results report that the proposed algorithm outperforms its competitors in terms of robustness, risk avoidance, and cumulative regret, promoting the social engagement behaviors of children with ASD when imitating a robot's movements.

Recently, robust-norm distance related principal component analysis (PCA) for feature extraction has been shown to be very effective for image analysis, which considers either minimization of reconstruction error or maximization of data variance in low-dimensional subspace . However, both of them are important for feature extraction. Furthermore, most of existing methods cannot obtain satisfactory results due to the utilization of inflexible robust norm for distance metric. To address these problems, this paper proposes a novel robust PCA formulation called Double L 2,p-norm based PCA (DLPCA) for feature extraction, in which the minimization of reconstruction error and the maximization of variance are simultaneously taken into account in a unified framework. In the reconstruction error function, we target to learn a latent subspace to bridge the relationship between the transformed features and the original features. To guarantee the objective to be insensitive to outliers, we take L2,p-norm as the distance metric for both reconstruction error and data variance. These characteristics make our method more applicable for feature extraction. We present an effective iterative algorithm to obtain the solution of this challenging work, and conduct theoretical analysis on the convergence of the algorithm. The experimental results on several databases show the effectiveness of our model.

Clustering is one of the most popular data mining methods for analyzing the time series, not only due to its exploratory power, but also because it is often a preprocessing step or subroutine for other techniques. In this paper, an incremental fuzzy clustering algorithm of time series (IFCTS) is proposed, in which the clustering is divided into two stages: offline and online clustering. During the offline clustering process, the fuzzy clustering validity evaluation index is introduced to FCM to automatically obtain the optimal number of initial clusters for off-line data. Then, in the online clustering process, IFCTS algorithm can dynamically update the existing clusters in the incremental data snapshot, distinguish outliers and control the creation of new clusters adaptively by combining with the previous clusters. The experimental results show that the proposed algorithm has good clustering accuracy and efficiency for both equal-length and unequal-length time series.

Recently a novel density-based clustering algorithm , namely, border-peeling (BP) clustering algorithm, is proposed to group data by iteratively identifying border points and peeling off them until separable areas of data remain. The BP clustering is able to correctly recognize the true structure of clusters and automatically detect the outliers on several test cases. However, there are some drawbacks in BP, and these may hinder its widespread application. The BP clustering might yield bad results on datasets with non-uniformly-distributed clusters. Especially, the BP clustering tends to over-partition the data with complex shape. To overcome these defects, a robust border-peeling clustering algorithm (named as ROBP) is proposed in this paper. Our method improves the BP clustering algorithm from two aspects: density influence (i.e. density estimation) and linkage criterion (i.e. association strategy). In density estimation, we use Cauchy kernel with longer tails instead of Gaussian kernel in the local scaling function, and further propose a kernel density estimator , i.e., the density estimator based on Cauchy kernel. It can calculate quickly and accurately the density influence value of each point. In association strategy, we design a linkage criterion based on the shared neighborhood information. The linkage criterion can create some links between peeled border points and their neighboring peeled border points, in order to avoid over-segmentation of the clusters. We integrate the proposed linkage criterion and the uni-directional association strategy, and further propose a bi-directional association strategy. In experiments, we compare ROBP with 7 representative density-based clustering (or hierarchical clustering) algorithms, including BP, DBSCAN, HDBSCAN, density peak (DP) clustering, DPC-KNN, DPC-DBFN and McDPC, on 8 synthetic datasets and 11 real-world datasets. Results show that the proposed algorithm outperforms 7 competitors in most cases. Moreover, we compare the robustness of ROBP and BP, and evaluate their running time. Experimental results indicate that ROBP is much more robust and reliable, as well as it is competitive to BP in computational efficiency.

DKIFCM (Density Based Kernelized Intuitionistic Fuzzy C Means) is the new proposed clustering algorithm that is based on outlier identification, kernel functions, and intuitionist fuzzy approach. DKIFCM is an inspiration from Kernelized Intuitionistic Fuzzy C Means (KIFCM) algorithm and it addresses the performance issue in the presence of outliers. It first identifies outliers based on density of data and then clusters are computed accurately by mapping the data to high dimensional feature space. Performance and effectiveness of various algorithms are evaluated on synthetic 2D data sets such as Diamond data set (D10, D12, and D15), and noisy Dunn data set as well as on high dimension real-world data set such as Fisher-Iris, Wine, and Wisconsin Breast Cancer Data-set. Results of DKIFCM are compared with results of other algorithms such as Fuzzy-C-Means (FCM), Intuitionistic FCM (IFCM), Kernel-Intuitionistic FCM (KIFCM), and density-oriented FCM (DOFCM), and the performance of proposed algorithm is found to be superior even in the presence of outliers and noise. Key advantages of DKIFCM are outlier identification, robustness to noise, and accurate centroid computation.

The rapid development of 5G communication technology will considerably help the expansion and gth Industrial Internet of things (IIoT). Indeed, both the data scale and dimension will significantly increase, leading to a challenging problem of the effective real-time stream clustering in the field of IIoT streaming mining. This paper proposes an efficient and novel real-time kernel function stream clustering method based on shared nearest-neighbor density for IIoT. In the proposed method, the projection technology is used to select the dimensions of high-dimensional data, while the Euler kernel function is used as the similarity measure. Furthermore, the micro-clusters are divided by the shared nearest-neighbor density, and the outliers are relearned. The main innovation lies in using the Euler kernel function to measure the similarity, reduce the sensitivity of outliers, and use the relearning strategy to improve the clustering quality of the data stream. The theoretical analysis and experimental comparisons on the simulated data sets show that the proposed method is very effective and represents a good solution for clustering real-time data streams of IIoT.

A robust multiple model strategy for linear parameter varying (LPV) systems identification relied on Student’s ${t}$ -distribution is studied in this article. Since most industrial processes operate over several working points to meet multiple production objectives, a single model is generally insufficient to describe the process behavior over the whole operating range. To cope with this issue, the multiple model strategy is employed and at the same time, the outliers and missing measurements problems are both considered in this article. The finite impulse response (FIR) model is chosen to describe the local behavior of the LPV system. The statistical scheme based on the ${t}$ -distribution is utilized to model the system noise so as to deal with the outliers. The problem of parameters estimation with incomplete outputs is addressed by using the expectation-maximization (EM) algorithm. The feasibility and effectiveness of the developed approach are proved via a mechanical unit.

Non-negative matrix factorization (NMF) has become one of the most powerful methods for clustering and feature selection. However, the performance of the traditional NMF method severely degrades when the data contain noises and outliers or the manifold structure of the data is not taken into account. In this article, a novel method called correntropy-based hypergraph regularized NMF (CHNMF) is proposed to solve the above problem. Specifically, we use the correntropy instead of the Euclidean norm in the loss term of CHNMF, which will improve the robustness of the algorithm. And the hypergraph regularization term is also applied to the objective function, which can explore the high-order geometric information in more sample points. Then, the half-quadratic (HQ) optimization technique is adopted to solve the complex optimization problem of CHNMF. Finally, extensive experimental results on multi-cancer integrated data indicate that the proposed CHNMF method is superior to other state-of-the-art methods for clustering and feature selection.

Data-driven inferential sensors have been increasingly applied to estimating important yet difficult-to-measure quality-relevant variables in industrial processes. However, complicated data characteristics (such as nonlinearities, non-Gaussianities, uncertainties, outlying data, etc.) existing in industrial datasets impose significant difficulties in developing inferential sensors with high accuracy. In particular, modeling process dynamics with sequential data is practically very important but quite challenging. In order to deal with such issues, this article proposes a dynamic variational Bayesian Student's t mixture regression (D-VBSMR) with hidden variables propagation. In D-VBSMR, the probabilistic mixture model structure can deal with nonlinearities, non-Gaussianities, and uncertainties; meanwhile, the Student's t-distribution with heavy tails enables D-VBSMR to be highly robust against outliers. Most importantly, the first-order Markov chain is used to connect hidden state variables, such that the process dynamic characteristic can be effectively modeled. In addition, parameter learning for D-VBSMR is also developed by adopting the variational inference framework and forward–backward algorithm. Experimental results on both a numerical example and a real industrial process validate the effectiveness and advantages of the proposed method.

Accurate and reliable wind direction prediction is important for improving wind power conversion efficiency and operation safety. In this paper, a two-phase deep learning model is proposed and constructed for high-performance short-term wind direction forecasting. In the first phase, a hybrid data processing strategy, including data reconstruction, outlier deletion, dimension reduction, and sequence decomposition, is proposed to extract the most meaningful information from practical data. Then, in the second phase, a robust echo state network is developed for wind direction forecasting. In addition, its hyper-parameters are optimized using an improved flower pollination algorithm (IFPA) to achieve high efficiency. Experiments conducted on data from real wind farms validate the proposed hybrid data processing method. Finally, comparisons with benchmark prediction models show that the proposed network achieves superior performance.

Due to the powerful capacity of exploring complementary and consistent information by generating a consensus affinity graph from multi-view data, multi-view graph clustering (MVGC) methods have attracted intensive attention. However, multi-view data is usually existed in high-dimensional space, where redundant and irrelevant features may result in the curse of dimensionality . Moreover, original data often mix with noise and outliers that will destroy the underlying clustering structure , such that unreliable and inaccurate affinity graphs will be generated. To alleviate the aforementioned problems, we propose a novel multi-view latent energy-preserving embedding (MLEE) method, which seamlessly integrates the clean embedding space learning and consensus affinity graph learning into a unified objective function. Concretely, for each view, we first learn the low-dimensional yet clean data by proposing a full-energy projection and recovering method. This can well reduce the redundancy and interference in the data. Furthermore, by leveraging adaptive neighbors graph learning (ANGL), the local manifold structure of the clean embedding data can be implicitly preserved. To integrate the complementary and consistent information of different views, an early-fusion scheme is proposed to directly yield a consensus graph for clustering purpose. Experiments on six benchmark datasets demonstrate that our method achieves state-of-the-art clustering performance.

The need to learn a good representation is a core problem central to AI. We present a self-supervised representation learning framework and demonstrate its use for few-shot classification and clustering. Our framework can be interpreted as repeatedly discovering new categories from learned embeddings and training a new embedding function with self-supervised signals to differentiate the discovered categories. In our framework, we first discover categories from unlabeled data. Next we post-process the previous partition results to remove outliers and derive prototypes of each category. We then construct few-shot learning tasks with previously selected data and augmented virtual data. Lastly, we iterative train the network through previous steps to learn the final representation. Our framework can considerably outperform previous baselines in unsupervised few-shot classification tasks on miniImageNet and Omniglot data sets. We also validate our learned representation on clustering tasks and demonstrate that our framework further improves upon the recent deep clustering methods.

International audience; Solar irradiance data from high-quality ground-based measurements are primordial for different solar energy applications. In order to achieve the required accuracy, quality control procedures are of great benefit. A variety of approaches have been proposed. In this sense, some approaches propose a visual representation of the routine, while others only provide a time series of binary flag values, and do not propose any specific visualization of the flagged data as opposed to non-flagged ones. In this regard, the present paper puts forward a complete routine including several quality control procedures for solar irradiance measurements by providing visual support for these different approaches. The visual tool in question was validated using five years research data with 10 minutes resolution of the global, diffuse and direct components of solar irradiation collected from three ground-based weather stations in Morocco. This visual tool puts forth a more precise idea of the measurement quality by detecting various errors, such as time shifts, outliers identification; either with one or two components, or consistency tests between the three components of solar radiation when available. The proposed tool can be regarded as a means of improving the detection rate of abnormal data as a first step in diagnosing the prominent causes of error.

The problems of the strip flatness defects are always severe in the tandem cold rolling process. It is of great significance to predict flatness for flatness control according to the process conditions of products. A prediction model based on convolutional neural network (CNN) was developed in this paper, and it can accurately predict strip flatness under various conditions. According to the distribution characteristics of industrial data collected in the rolling process, the isolated forest algorithm was used to eliminate outliers. Considering the special requirements of CNN on the dimension of input features, the data folding method was used to process the input features. Additionally, since strip flatness data is a vector rather than a scalar, and the length of this vector varies with strip width, which decreases the network's training accuracy. To deal with the problems, the loss function was modified. Taking the Inception module as the basic network structure and inspired by Wide & Deep learning, a strip flatness prediction model with high accuracy was developed. The optimal architecture and parameters of our network were determined through a lot of experimental explorations. The performances of BPNN (Back Propagation Neural Network), DNN (Deep Neural Network), and the proposed model were compared by mean square error (MSE) and coefficient of determination (R2). The result indicates that the proposed model has the highest prediction accuracy and better adaptability. It has the lowest MSE, 0.9891, and the highest R2, 0.9555. Finally, the fitting coefficients of Legendre polynomials were used to further prove the excellent prediction performance of the proposed model for strip flatness. Compared with other prediction models, it can obtain the lowest prediction error for the first quadratic and quartic components of strip flatness and it can be well-applied to tandem cold rolling production.

k-Means is a well-studied clustering problem that finds applications in many fields related to unsupervised learning. It is known that k-means clustering is highly sensitive to the isolated points (called outliers). Such outliers can significantly influence the final cluster configuration and should be removed to obtain quality solutions. In this paper, we study the k-means with outliers problem. Given a set of data points, the problem is to remove a set of outliers such that the k-means clustering cost of the remaining points is minimized. Designing efficient algorithms for this problem remains an active area of research due to its important role in dealing with noisy data. We consider a relaxed objective function and propose a local search algorithm for k-means with outliers. It is shown that the algorithm has better performance guarantees than previously implemented methods. In particular, it yields a constant-factor bi-criteria approximation solution to the problem. Moreover, we show experimentally that the algorithm performs much better than its provable guarantee and dominates other state-of-the-art methods for the problem.

Robust regression tools are commonly used to develop regression-type ratio estimators with traditional measures of location whenever data are contaminated with outliers. Recently, the researchers extended this idea and developed regression-type ratio estimators through robust minimum covariance determinant (MCD) estimation. In this study, the quantile regression with MCD-based measures of location is utilized and a class of quantile regression-type mean estimators is proposed. The mean squared errors (MSEs) of the proposed estimators are also obtained. The proposed estimators are compared with the reviewed class of estimators through a simulation study. We also incorporated two real-life applications. To assess the presence of outliers in these real-life applications, the Dixon chi-squared test is used. It is found that the quantile regression estimators are performing better as compared to some existing estimators.