Search

The adaptive neurofuzzy inference system (ANFIS) is a structured multioutput learning machine that has been successfully adopted in learning problems without noise or outliers. However, it does not work well for learning problems with noise or outliers. High-accuracy real-time forecasting of traffic flow is extremely difficult due to the effect of noise or outliers from complex traffic conditions. In this study, a novel probabilistic learning system, probabilistic regularized extreme learning machine combined with ANFIS (probabilistic R-ELANFIS), is proposed to capture the correlations among traffic flow data and, thereby, improve the accuracy of traffic flow forecasting. The new learning system adopts a fantastic objective function that minimizes both the mean and the variance of the model bias. The results from an experiment based on real-world traffic flow data showed that, compared with some kernel-based approaches, neural network approaches, and conventional ANFIS learning systems, the proposed probabilistic R-ELANFIS achieves competitive performance in terms of forecasting ability and generalizability.

Partial multilabel learning (PML) aims to learn from training data, where each instance is associated with a set of candidate labels, among which only a part is correct. The common strategy to deal with such a problem is disambiguation, that is, identifying the ground-truth labels from the given candidate labels. However, the existing PML approaches always focus on leveraging the instance relationship to disambiguate the given noisy label space, while the potentially useful information in label space is not effectively explored. Meanwhile, the existence of noise and outliers in training data also makes the disambiguation operation less reliable, which inevitably decreases the robustness of the learned model. In this article, we propose a prior label knowledge regularized self-representation PML approach, called PAKS, where the self-representation scheme and prior label knowledge are jointly incorporated into a unified framework. Specifically, we introduce a self-representation model with a low-rank constraint, which aims to learn the subspace representations of distinct instances and explore the high-order underlying correlation among different instances. Meanwhile, we incorporate prior label knowledge into the above self-representation model, where the prior label knowledge is regarded as the complement of features to obtain an accurate self-representation matrix. The core of PAKS is to take advantage of the data membership preference, which is derived from the prior label knowledge, to purify the discovered membership of the data and accordingly obtain more representative feature subspace for model induction. Enormous experiments on both synthetic and real-world datasets show that our proposed approach can achieve superior or comparable performance to state-of-the-art approaches.

Varied edge bundling methods have been used to reduce visual clutter in parallel coordinates plots (PCP). However, existing edge-bundled PCP do not scale well for visual analysis of multidimensional big data and often overplot the bundles in the area near the axes. In this study, we propose a scalable lightweight bundling method to support visual analysis of multidimensional big data in PCP. It helps the users discover trends and detect outliers in the data by bundling the edges between each two adjacent axes independently. We integrate human judgments into the two-dimensional data binning by novel interactions to accelerate the clustering process of the data. We use the frequency-based representation to render the clusters as histogram-like bundles to reveal the distribution of the data and eliminate the overplotting of the bundles. Based on our method, we build a lightweight web-based visual analytics system for exploring multidimensional big data in PCP. The scalability analysis of our method shows that its clustering time increases linearly with the size of the data. Its rendering time is independent of the size of the data. We conduct two case studies and a user study to compare our method with classic PCP and two state-of-the-art edge-bundled PCP.

Broad learning system (BLS) is a novel and efficient model, which facilitates representation learning and classification by concatenating feature nodes and enhancement nodes. In spite of the efficient properties, BLS is still suboptimal when facing with imbalance problem. Besides, outliers and noises in imbalanced data remain a challenge for BLS. To address the above issues, in this paper we firstly propose a weighted BLS, which assigns a weight to each training sample, and adopt a general weighting scheme, which augments the weight of samples from the minority class. To further explore the prior distribution of original data, we design a density based weight generation mechanism to guide the specific weight matrix generation and propose the adaptive weighted broad learning system (AWBLS). This mechanism considers the inter-class and intra-class distance simultaneously in the density calculation. Finally, we propose the incremental weighted ensemble broad learning system (IWEB) by utilizing a progressive mechanism to further improve the stability and robustness of AWBLS. Extensive comparative experiments on 30 real-world data sets verfy that IWEB outperforms most of the imbalance ensemble classification methods.

Multiview learning (MVL), which enhances the learners' performance by coordinating complementarity and consistency among different views, has attracted much attention. The multiview generalized eigenvalue proximal support vector machine (MvGSVM) is a recently proposed effective binary classification method, which introduces the concept of MVL into the classical generalized eigenvalue proximal support vector machine (GEPSVM). However, this approach cannot guarantee good classification performance and robustness yet. In this article, we develop multiview robust double-sided twin SVM (MvRDTSVM) with SVM-type problems, which introduces a set of double-sided constraints into the proposed model to promote classification performance. To improve the robustness of MvRDTSVM against outliers, we take L1-norm as the distance metric. Also, a fast version of MvRDTSVM (called MvFRDTSVM) is further presented. The reformulated problems are complex, and solving them are very challenging. As one of the main contributions of this article, we design two effective iterative algorithms to optimize the proposed nonconvex problems and then conduct theoretical analysis on the algorithms. The experimental results verify the effectiveness of our proposed methods.

Network data constantly evolves with new network applications and protocols. There is a need for robust techniques to detect anomalous behaviour. Offline models trained with static data lose validity when new variants of traffic emerge. They require retraining but the need for ground truth and lengthy training times make this task challenging. Meanwhile, online models which detect outliers in streaming data are susceptible to the curse of dimensionality and natural variability. Today's anomalies may be tomorrow's new traffic and existing methods do not provide a way to differentiate between them. We propose a framework that makes the most of both approaches: an offline deep learning model extracts features of normal traffic and provides a bias for an online outlier detection model to select data for training. The online model retains its previously learnt knowledge and retrains itself with new data. Online thresholds are updated in a drifting manner and the Mann-Whitney-U test is incorporated to prevent inaccurate updates. We perform analysis on the scores, develop heuristics to detect new traffic and evaluate using three deep learning models and four outlier detection methods on the UNSW-NB15 and CTU-13 datasets. The framework improves upon any individual offline or online models in isolation.

Virtualization brings the benefits of promoted utilization and scalability to the multi-tenant cloud platforms. However, the hypervisor, as one of the foundations in virtualization, is challenging to survive under various malicious attacks due to its large attack surface. This paper presents a novel group detection framework for the hypervisor code integrity, called Outlier. A large-scale cloud is split into individual Outlier detection groups. In an Outlier group, each host contains two parts. One is a distributed detection protocol, called Co-protocol, and the other is a detection interface, called Checker. The Co-protocol constructs trust for the integrity detection within an Outlier group. With the Co-protocol, each Checker conducts reliable integrity detection on the hypervisor code, and then the potential outlier host is perceived. We implement our Outlier prototype on the Xen hypervisor and evaluate its overhead. Experiments show that the introduction of the Outlier has few impacts on the performance of the virtualized systems.

A recommender system (RS) is highly efficient in filtering people's desired information from high-dimensional and sparse (HiDS) data. To date, a latent factor (LF)-based approach becomes highly popular when implementing a RS. However, current LF models mostly adopt single distance-oriented Loss like an L₂ norm-oriented one, which ignores target data's characteristics described by other metrics like an L₁ norm-oriented one. To investigate this issue, this article proposes an L₁-and-L₂-norm-oriented LF (L³F) model. It adopts twofold ideas: 1) aggregating L₁ norm's robustness and L₂ norm's stability to form its Loss and 2) adaptively adjusting weights of L₁ and L₂ norms in its Loss. By doing so, it achieves fine aggregation effects with L₁ norm-oriented Loss's robustness and L₂ norm-oriented Loss's stability to precisely describe HiDS data with outliers. Experimental results on nine HiDS datasets generated by real systems show that an L³F model significantly outperforms state-of-the-art models in prediction accuracy for missing data of an HiDS dataset. Its computational efficiency is also comparable with the most efficient LF models. Hence, it has good potential for addressing HiDS data from real applications.

Data-driven fault diagnosis primarily involves the identification of different fault locations and fault severities. Focusing on a chal-lenging task for which the target fault severities do not exist in the training samples, this study proposes a generalized cross-severity bearing fault diagnosis scheme based on a novel hierarchical cross-category inference framework. The proposed method uses an outlier detection scheme based on unsupervised feature mapping and local outlier probability calculation to identify the unseen samples. A neural network embedded with a tree-structured decision layer acts as a backbone to execute fault diagnosis at different hierarchies for different sample types, seen or unseen. Additionally, the metric learning method is used to support the approximate severity inference of the unseen samples after the fault locations are identified in the hierarchical model. Experiments performed on an aeronautical bearing test rig revealed that the proposed scheme is both feasible and superior to existing methods.

The measurement error evaluation of Power Metering Equipment (PME) is significant for the instrument design and accurate metering of electric energy, especially in extreme environmental stresses. However, actual measurement error prediction is often disturbed by environmental noise and insufficient feature information. To address this problem, an Improved Local Outlier Factor (ILOF) method is first proposed to detect potential outliers. And an optimized distance function and adaptive threshold constraint method based on boxplot are used to improve the outlier detection performance of ILOF. Next, an error prediction method, namely Kernel Support Vector Regression (KSVR), is presented to fuse measurement error and multiple extreme environmental stresses by using the proposed kernel approach. Integrating the ILOF and KSVR, examples from the extreme environmental region demonstrate that the proposed evaluation framework has a higher assessment performance. Compared with several state-of-art prediction methods, our framework has profound outlier identification and error prediction performance under small sample conditions.

Wireless Sensor Networks (WSNs) are developed significantly from the last decades and attracted the attention of scientific and industrial domains. In WSNs, sensor nodes distributed autonomously in harsh environments are easily vulnerable to faults and attacks that cause sensor readings unreliable and inaccurate. In this scenario, sensor readings that have differed considerably from healthy behaviors will be considered abnormal data or anomalies/outliers. The inclusion of such outliers in data analytics will inevitably affect the outcome of the decision-making process. Thus, detecting outliers in WSNs using data-driven approaches becomes a novel technique among the Machine Learning (ML) communities. Meanwhile, various research issues are there in measuring the performance of the deployed ML algorithms in detecting outliers in WSNs, which generally contains minimum resources in terms of computational capability and power sources to ensure data quality. Hence, this paper presents a comprehensive overview of the state-of-the-art Statistical and Artificial Intelligence (AI) based techniques used in WSNs to detect outliers in the view of architecture, type, degree, approach, and detection mode. Furthermore, each aforementioned outlier detection approach is presented with detailed discussions and future scope for developments.

Anomaly (outlier) detection is one of the most important problems of modern data analysis. The sources of anomalies are varying. They can be the results of database users' mistakes, operational errors or just missing values. The problem is very important because of the fast growth of large data sets. Therefore, in this study, we present detailed results of work on the concept of Granular Computing-based approach to anomaly detection, classification, and gradation. The aim of the study is to introduce an innovative solution that allows the use of information granules to identify and classify anomalies. The novelty of the proposed solution consists in the use of fuzzy semantics implied by the statistical properties of the data considered. Moreover, instead of the classic approach to detecting anomalies in the data, it is proposed to determine the degree of anomaly for the data transformed to the new resulting state space. Thanks to the use of an innovative approach using the universal descriptor space, it is possible to determine the degree of anomaly, and by using various aggregation methods one can also specify its type.

A milling cutter is one of the most important parts of machine tools. Its working status significantly influences the precision of workpieces. Due to the complex wear mechanism, the single sensor may be difficult to acquire the complete degradation information of milling cutters. Therefore, a feature learning based method is proposed to automatically extract features from multi-source data and predict the remaining useful life of cutting tools in real-time. Firstly, a statistic-based method is constructed to detect and delete the outliers hidden in the monitoring data. Secondly, the clean data is input into a multi-scale convolutional attention network (MSAN) to learn features and fuse multi-source data. At last, the fused data is used to predict the remaining useful life of cutting tools in a regression layer. Compared with traditional tool life prediction methods, the proposed method is able to fuse multi-source data through an attention feature learning model to conduct the life prediction of tools. Additionally, the data cleaning and model optimization methods are also proposed to promote engineering practicability. In order to validate the effectiveness of such method, the life testing experiments on milling cutters are conducted to obtain run-to-failure data. In those experiments, multi-sensor monitor data are acquired, which are used to conduct validation experiments testing the effectiveness of the proposed method. The results indicate the superiority of the proposed method in remaining useful life prediction milling cutters.

The development of industrial cyber-physical system (ICPS) provides a tight connection between the digital model and the industrial physical plant, which enables to use the data-driven methods to reflect the state of process running in the real world. However, due to the noisy and harsh industrial environment, the collected data are often corrupted to some extent. If the corrupted data are not detected in time, the data-driven model will inevitably degenerate and induce a poor process monitoring performance. In addition, the nonlinear characteristics between process variables due to the high complexities in physical plant bring challenges to the data-driven methods. In this article, a robust kernel dictionary learning method, which can overcome the negative influence of outliers and simultaneously extracts the nonlinear characteristics of industrial process, is proposed to address the above problems in ICPS. Our extensive experiments demonstrate that the proposed method has achieved significantly better and stable performance to deal with outlier detection and process monitoring in ICPS.

The problem of representative selection amounts to sampling few informative exemplars from large datasets. Existing approaches to data selection often fall short of simultaneously handling non-linear data structures, sampling concise and non-redundant subsets, rejecting outliers, and yielding interpretable outcomes. This paper presents a novel representative selection approach, dubbed MOSAIC, for drawing descriptive sketches of arbitrary manifold structures. Resting upon a novel quadratic formulation, MOSAIC advances a multi-criteria selection approach that maximizes the global representation power of the sampled subset, ensures novelty of the samples by minimizing redundancy, and rejects disruptive information by effectively detecting outliers. Theoretical analyses shed light on geometrical characterization of the obtained sketch and reveal that the sampled representatives maximize a well-defined notion of data coverage in a transformed space. In addition, we present a highly scalable randomized implementation of the proposed algorithm shown to bring about substantial speedups. MOSAIC's superiority in achieving the desired characteristics of a representative subset all at once while exhibiting remarkable robustness to various outlier types is demonstrated via extensive experiments conducted on both real and synthetic data with comparisons to state-of-the-art algorithms.

Cyberattacks in power systems that alter the input data of a load forecasting model have serious, potentially devastating consequences. Existing cyberattack-resilient work focuses mainly on enhancing attack detection. Although some outliers can be easily identified, more carefully designed attacks can escape detection and impact load forecasting. Here, a cyberattack-resilient load forecasting approach based on an adaptive robust regression method is proposed, where the observations are trimmed based on their residuals and the proportion of the trim is adaptively determined by an estimation of the contaminated data proportion. An extensive comparison study shows that the proposed method outperforms the standard robust regression in various settings.

Many research areas depend on group anomaly detection. The use of group anomaly detection can maintain and provide security and privacy to the data involved. This research attempts to solve the deficiency of the existing literature in outlier detection thus a novel hybrid framework to identify group anomaly detection from sequence data is proposed in this paper. It proposes two approaches for efficiently solving this problem: i) Hybrid Data Mining-based algorithm, consists of three main phases: first, the clustering algorithm is applied to derive the micro-clusters. Second, the kNN algorithm is applied to each micro-cluster to calculate the candidates of the group's outliers. Third, a pattern mining framework gets applied to the candidates of the group's outliers as a pruning strategy, to generate the groups of outliers, and ii) a GPU-based approach is presented, which benefits from the massively GPU computing to boost the runtime of the hybrid data mining-based algorithm. Extensive experiments were conducted to show the advantages of different sequence databases of our proposed model. Results clearly show the efficiency of a GPU direction when directly compared to a sequential approach by reaching a speedup of 451. In addition, both approaches outperform the baseline methods for group detection.

Modern metro systems in big cities have accumulated large amounts of passenger transit transaction data via the deployment of automatic fare collection (AFC) devices, which could facilitate a thorough analysis of passenger flow dynamics. To discover the underlying passenger flow patterns over all stations in an entire metro system, this paper proposes a multivariate functional principal component analysis (MFPCA) method. The functional integrity of a single daily passenger flow profile at each station is explicitly utilized, and the complex correlation among a multitude of daily passenger flow profiles from all stations is comprehensively modeled. Moreover, to simultaneously improve the interpretability of eigenvectors and mitigate the sensitivity to outliers, the MFPCA is formulated as a minimization problem with both a sparsity and a robustness penalty term. A computationally efficient algorithm is developed accordingly to obtain the eigenvectors. The superiority of our proposed sparse and robust MFPCA (SRMFPCA) is validated using a Hong Kong Mass Transit Railway (MTR) dataset. The derived sparse and smooth eigenvectors can be well interpreted as empirically meaningful passenger flow patterns. The results of our method can be further used as solid foundations for station clustering, correlation analysis and outlier identification.

The interactive exploration of large and evolving datasets is challenging as relationships between underlying variables may not be fully understood. There may be hidden trends and patterns in the data that are worthy of further exploration and analysis. We present a system that methodically explores multiple combinations of variables using a searchlight technique and identifies outliers. An iterative k-means clustering algorithm is applied to features derived through a split-apply-combine paradigm used in the database literature. Outliers are identified as singleton or small clusters. This algorithm is swept across the dataset in a searchlight manner. The dimensions that contain outliers are combined in pairs with other dimensions using a susbset scan technique to gain further insight into the outliers. We illustrate this system by anaylzing open health care data released by New York State. We apply our iterative k-means searchlight followed by subset scanning. Several anomalous trends in the data are identified, including cost overruns at specific hospitals, and increases in diagnoses such as suicides. These constitute novel findings in the literature, and are of potential use to regulatory agencies, policy makers and concerned citizens., 2018 International Joint Conference on Neural Networks (IJCNN)

With the increased proliferation of smart devices, the transit passengers of today expect a higher quality of service in the form of real-time traffic updates, accurate expected time-of-arrival (ETA) predictions. Providing these services requires public transit agencies and private transportation players to maintain full situational awareness of the city-wide traffic. However, most such agencies and companies are resource-constrained and do not have access to city-wide traffic data. The availability of sparsely sampled and outlier-corrupted traffic data renders the resulting traffic maps patchy and unreliable and necessitates the use of sophisticated real-time traffic interpolation and prediction algorithms. Moreover, since the traffic data is measured and collected in a sequential manner, the estimations must also be generated online. Thankfully, the traffic matrices are spatially and temporally structured, allowing the use of time-series and matrix/tensor completion algorithms. This work puts forth a generative model for the traffic density and subsequently uses a variational Bayesian formalism to learn the parameters of the model. Specifically, we consider low-rank traffic matrices whose subspace evolves according to a state-space model with possible sparse outliers. Unlike most matrix/tensor completion algorithms, the proposed model is equipped with automatic relevance determination priors that allow it to learn the parameters in an entirely data-driven manner. A forward-backward algorithm is proposed that enables the updates to be carried out at low-complexity. Simulations carried out on real traffic speed data demonstrate that the proposed algorithm better predicts the future traffic densities as compared to the state-of-the-art matrix/tensor completion algorithms.

Precise prediction of risk for dyslexia among children’s in earlier stages is a significant long-term aim in the field of cognitive computing. Producing such accurate results for detection of dyslexia from a dataset which consist of low-quality dataset and the presence of vague information is the toughest challenge among researchers. This paper aims at developing an evolving model to handle the impreciseness in the detection of dyslexia more intelligently. In this work, each instance is described in a neutrosophic domain by defining a membership degree of truthiness, indeterminacy, and falsity. These instances are neutrosophically clustered by applying Neutrosophic C-Means clustering (NCM) which forms four different clusters namely dyslexia, no dyslexia, control/revision and hyperactivity or other issues. The outlier and noise are the special categories of indeterminacy which often occurs in real datasets are promptly discovered and clustered. NCM is optimized by introducing Artificial Bacterial Foraging (ABF), especially when there is vagueness or imprecision in the selection of cluster centroids. With the merits of global searching, ABF selects more promising clustering during cluster re-computation. The interpreted results confirm that the role played by proposed ONCMC-ABF algorithm produces better results in the prediction of dyslexia with the low-quality dataset.

Since social media, virtual communities and networks rapidly grow, multi-view data become more popular. In general, multi-view data always contain different feature components in different views. Although these data are extracted in different ways (views) from diverse settings and domains, they are used to describe the same samples which make them highly related. Hence, applying (single-view) clustering methods for multi-view data poses difficulty in achieving desirable clustering results. Thus, multi-view clustering methods should be developed that will utilize available multi-view information. Most of multi-view clustering techniques currently use k-means due to its conceptual simplicity, and use fuzzy c-means (FCM) that the datapoints can belong to more than one cluster based on their membership degrees from 0 to 1. However, the use of k-means or FCM may degrade its performance due to the presence of noise and outliers, especially on large or high-dimensional datasets. The constraint imposed on the membership degrees of k-means and FCM tends to assign a corresponding high membership value to an outlier or a noisy data point. To address these drawbacks, possibilistic c-means (PCM) relaxes the membership constraint of k-means and FCM so that outliers and noisy datapoints can be properly identified. On the other hand, there are various extensions of k-means and FCM for multi-view data, but no extension of PCM for multi-view data was made in the literature. Thus, we use PCM in our proposed multi-view clustering model. In this paper, we propose novel weighted multi-view PCM algorithms designed for clustering multi-view data as well as view and feature weights on PCM approaches, called W-MV-PCM and W-MV-PCM with L2 regularization (W-MV-PCM-L2). In multi-view clustering, different views may vary with respect to its importance and each view may contain some irrelevant features. In the proposed algorithms, a learning schema is constructed to compute for the view weights, and feature weights within each view. This schema will be able to identify the importance of each view and, at the same time, it will also identify and select relevant features in each view. Comparisons of W-MV-PCM-L2 with existing multi-view clustering algorithms are made on both synthetic and real datasets. The experimental results are evaluated using accuracy rate (AR) and external validity indexes, such as Rand index (RI) and normalized mutual information (NMI). The proposed W-MV-PCM-L2 algorithm with comparisons of existing algorithms under criteria of AR, RI and NMI shows that it is a feasible and effective multi-view clustering algorithm.

To realize fault identification of unlabeled data and improve model generalization capability, domain adaptation has been increasingly applied to intelligent fault diagnosis of machinery. Nevertheless, traditional domain adaptation diagnosis models generally constrain different domains to have the same label space, which is not always hold in complex industrial scenarios. Consequently, a more practical scenario, i.e. partial-set transfer diagnosis (PSTD), is explored in this work, where the target label space is a subspace of source domain. A multi-weight domain adversarial network (MWDAN) is proposed to solve this issue, in which class-level and instance-level weighted mechanisms are jointly designed to quantify the transferability and importance of data example. Based on the proposed strategy, the positive transfer between shared classes is promoted while the negative effect caused by outlier classes is circumvented. As a result, MWDAN can learn discriminative representations for accurate fault diagnosis in target domain. Extensive experiments constructed on two mechanical systems demonstrate the outstanding performance of MWDAN.

The rapidly increasing incidence of Diabetes Mellitus (DM) has shown that DM is a serious disease that endangered human life in all parts of the world. The late stage of Type-II DM (T2DM) in particular is accompanied by complex complications. Healthcare systems with various data mining algorithms can help the endocrinologist to find whether patients have diabetes in the early detection of T2DM. In the present research, a novel and efficient binary logistic regression (BLR) is proposed founding on feature transformation of XGBoost (XGBoost-BLR) for accurately predicting the specific type of T2DM, and making the model adaptive to more than one dataset. In order to raise the identification ratio, the databases are executed by series of preprocessing procedures which include removing outliers, normalization, and missing value processing. We select features that have a more significant effect on the results by χ 2 test (CST). Then, the selected features are projected into high-dimensional feature space by XGBoost. Finally, the high-dimensional features generated can be modelled by the BLR application. The proposed XGBoost-BLR achieved a 94% and 98% identification rate for diabetes prediction in Pima Indians Diabetes Database (PIDD) and Early-Stage Diabetes Risk Prediction Database (ESDRPD).

The development of Wireless Sensor Networks (WSNs) has been attained in the past few years due to its important using in wide range of application. The readings of data derived from WSN nodes are not always accurate and may contain abnormal data. This paper proposed an anomaly detection and classification algorithm in WSNs. At first, an integration of Contourlet Transform (CT) algorithm and One Class Support Vector Machine (OCSVM) algorithm (CT-OCSVM) is utilized to detect outliers then Fuzzy Inference System (FIS) is used to identify the source of these outliers. The underlying aim of this paper focuses on treating the collected streams of data as raw datum of an image, which is then passed through some filters using CT to get compressed size of directional subbands coefficients. The coefficients of CT are examined by OCSVM algorithm to detect anomalies. Finally the source of anomalies is identified based on using FIS and by exploiting the spatial temporal correlation existing between the sensed data. The integrated algorithm is tested using different types of filters. Real datasets collected from a small WSN constructed in a local lab are used for testing the integrated algorithms. The simulation results have shown a high rate of accurate classification with high detection rate and low false alarm rate.

The linear regression model remains an important workhorse for data scientists. However, many data sets contain many more predictors than observations. Besides, outliers, or anomalies, frequently occur. This paper proposes an algorithm for regression analysis that addresses these features typical for big data sets, which we call “sparse shooting S”. The resulting regression coefficients are sparse, meaning that many of them are set to zero, hereby selecting the most relevant predictors. A distinct feature of the method is its robustness with respect to outliers in the cells of the data matrix. The excellent performance of this robust variable selection and prediction method is shown in a simulation study. A real data application on car fuel consumption demonstrates its usefulness.

This paper focuses on developing efficient and robust evaluation metrics for RANSAC hypotheses to achieve accurate 3D rigid registration. Estimating six-degree-of-freedom (6-DoF) pose from feature correspondences remains a popular approach to 3D rigid registration, where random sample consensus (RANSAC) is a well-known solution to this problem. However, existing metrics for RANSAC hypotheses are either time-consuming or sensitive to common nuisances, parameter variations, and different application scenarios, resulting in performance deterioration with respect to overall registration accuracy and speed. We alleviate this problem by first analyzing the contributions of inliers and outliers and then proposing several efficient and robust metrics with different designing motivations for RANSAC hypotheses. Comparative experiments on four standard datasets with different nuisances and application scenarios verify that our considered metrics can significantly improve the registration performance and are more robust than several state-of-the-art competitors, making them good gifts to practical applications. This work also draws an interesting conclusion, i.e., not all inliers are equal while all outliers should be equal, which may shed new light on this research problem.

One of the main challenges in autonomous robotic exploration and navigation in unknown and unstructured environments is determining where the robot can or cannot safely move. A significant source of difficulty in this determination arises from stochasticity and uncertainty, coming from localization error, sensor sparsity and noise, difficult-to-model robot-ground interactions, and disturbances to the motion of the vehicle. Classical approaches to this problem rely on geometric analysis of the surrounding terrain, which can be prone to modeling errors and can be computationally expensive. Moreover, modeling the distribution of uncertain traversability costs is a difficult task, compounded by the various error sources mentioned above. In this work, we take a principled learning approach to this problem. We introduce a neural network architecture for robustly learning the distribution of traversability costs. Because we are motivated by preserving the life of the robot, we tackle this learning problem from the perspective of learning tail-risks, i.e. the Conditional Value-at-Risk (CVaR). We show that this approach reliably learns the expected tail risk given a desired probability risk threshold between 0 and 1, producing a traversability costmap which is more robust to outliers, more accurately captures tail risks, and is more computationally efficient, when compared against baselines. We validate our method on data collected a legged robot navigating challenging, unstructured environments including an abandoned subway, limestone caves, and lava tube caves., Published in RA-L with ICRA presentation option (IEEE International Conference on Robotics and Automation, 2022)

Symptom identification and early detection are the first steps towards a health condition diagnosis. The COVID-19 virus causes pneumonia-like symptoms such as fever, cough, and shortness of breath. Many COVID-19 contraction tests necessitate extensive clinical protocols in medical settings. Clinical studies help with the accurate analysis of COVID-19, where the virus has already spread to the lungs in most patients. The majority of existing supervised machine learning-based disease detection techniques are based on clinical data like x-rays and computerized tomography. This is heavily reliant on a larger clinical study and does not emphasize early symptom detection. The aim of this study is to investigate anomalies in patient physiological data for early COVID-19 symptoms identification. In this context, two of the most prevalent symptoms, fever and cough, were examined in a two-fold manner utilizing an unsupervised machine learning model. To examine disease progression, physiological features from a chest-worn device were analyzed. First, a Single Vector Activity Index (SVAI) parameter is proposed to monitor the breathing and cough patterns. Second, the dataset's variance is examined using the DBSCAN method for clustering and outlier detection. Finally, the model accuracy is evaluated to identify outliers on real-time data based on feature dissimilarities, yielding an overall detection accuracy of 90.34%.

Despite the promising preliminary results, existing graph convolutional network (GCN) based multi-view learning methods directly use the graph structure as view descriptor, which may inhibit the ability of multi-view learning for multimedia data. The major reason is that, in real multimedia applications, the graph structure may contain outliers. Moreover, they fail to take advantage of the information embedded in the inaccurate clustering labels obtained from their proposed methods, resulting in inferior clustering results. These observations motivate us to study whether there is a better alternative GCN based framework for multi-view clustering. To this end, in this paper, we propose an end-to-end self-supervised graph convolutional network for multi-view clustering (SGCMC). Specifically, SGCMC constructs a new view descriptor for graph-structured data by mapping the raw node content into the complex space via Euler transformation, which not only suppresses outliers but also reveals non-linear patterns embedded in data. Meanwhile, the proposed SGCMC uses the clustering labels to guide the learning of the latent representation and coefficient matrix, and the latter in turn is used to conduct the subsequent node clustering. By this way, clustering and representation learning are seamlessly connected, with the aim to achieve better clustering results. Extensive experiments indicate that the proposed SGCMC outperforms the state-of-the-art methods.

In data mining, outlier detection is applied in different domains. It has very large applications such as energy consumption analysis, forecasting hurricanes in meteorological data, fraud and intrusion detection, event detection and system monitoring in sensor networks, etc. Most of existing outlier detection techniques depend on the properties of a particular type of data and can not deal with a large volume of data well, which mean that there is a necessity for improved methodologies and techniques to be applied to a large amount of data with different types in other application areas. In this paper, a parallel outlier detection technique is developed to detect the outliers in the sequential data. Although there are many types of outliers, this paper concentrates on the contextual anomalies. The proposed technique uses a graph approach to detect the outliers. It is very flexible, fast, and no labeled data is needed comparing to many previous approaches. The experimental results show the detected contextual outliers in the sequential data, as well as the efficient scaling up to handle the massive data by increasing the number of processors. The results prove that the parallelism of the proposed technique is very valuable.

As a flexible passive 3D sensing means, unsupervised learning of depth from monocular videos is becoming an important research topic. It utilizes the photometric errors between the target view and the synthesized views from its adjacent source views as the loss instead of the difference from the ground truth. Occlusion and scene dynamics in real-world scenes still adversely affect the learning, despite significant progress made recently. In this paper, we show that deliberately manipulating photometric errors can efficiently deal with these difficulties better. We first propose an outlier masking technique that considers the occluded or dynamic pixels as statistical outliers in the photometric error map. With the outlier masking, the network learns the depth of objects that move in the opposite direction to the camera more accurately. To the best of our knowledge, such cases have not been seriously considered in the previous works, even though they pose a high risk in applications like autonomous driving. We also propose an efficient weighted multi-scale scheme to reduce the artifacts in the predicted depth maps. Extensive experiments on the KITTI dataset and additional experiments on the Cityscapes dataset have verified the proposed approach's effectiveness on depth or ego-motion estimation. Furthermore, for the first time, we evaluate the predicted depth on the regions of dynamic objects and static background separately for both supervised and unsupervised methods. The evaluation further verifies the effectiveness of our proposed technical approach and provides some interesting observations that might inspire future research in this direction., Accepted by IEEE Sensors Journal. arXiv admin note: text overlap with arXiv:2003.01360

Detecting outliers in real-time is increasingly important for many real-world applications such as detecting abnormal heart activity, intrusions to systems, spams or abnormal credit card transactions. However, detecting outliers in data streams rises many challenges such as high-dimensionality, dynamic data distribution and unpredictable relationships. Our simulations demonstrate that some advanced solutions still show drawbacks. In this paper, first, we improve the capacity to detect outliers of both micro-clusters based algorithms (MCOD) and distance-based algorithms (Abstract-C and Exact-Storm) known for their performance. This is by adding a layer called LiCS that classifies online the K-nearest-neighbors (Knn) of each node based on their evolutionary status. This layer aggregates the results and uses a count threshold to better classify nodes. Experiments on SpamBase datasets confirmed that our technique enhances the accuracy and the precision of such algorithm and helps to reduce the unclassified nodes.Second, we propose a hybrid solution based on iterative majority voting and our LiCS. Experiments on real data proves that it outperforms discussed algorithms in terms of accuracy, precision and sensitivity in detecting outliers. It also minimizes the issue of unclassified instances and consolidate the different outputs of algorithms.

Environmental impact of pollutants can be analyzed effectively by acquiring fish behavioral signals in water with biological behavior sensors. However, a variety of factors, such as the complexity of biological organisms themselves, the device error and the environmental noise, may compromise the accuracy and timeliness of model predictions. The current methods lack prior knowledge about the fish behavioral signals corresponding to characteristic pollutants, and in the event of a pollutant invasion, the fish behavioral signals are poorly discriminated. Therefore, we propose a novel method based on Bayesian sequential, which utilizes multi-channel prior knowledge to calculate the outlier sequence based on wavelet feature followed by calculating the anomaly probability of observed values. Furthermore, the relationship between the anomaly probability and toxicity is analyzed in order to achieve forewarning effectively. At last, our algorithm for fish toxicity detection is verified by integrating the data on laboratory acceptance of characteristic pollutants. The results show that only one false positive occurred in the six experiments, the present algorithm is effective in suppressing false positives and negatives, which increases the reliability of toxicity detections, and thereby has certain applicability and universality in engineering applications.

As an effective tool to deal with uncertain decision-making, three-way decisions (TWD) have gained wide attention in many applications. Decision-theoretic rough sets (DTRSs) as a classic model of TWD contain two key elements, i.e., conditional probability and loss functions. In this paper, we study the determination of these two elements in depth via the information interaction and modification strategy, and further propose a novel model of TWD. First, fuzzy c-means (FCM) is used to cluster the condition attribute information and loss function information, respectively. Considering the interaction of two types of information, we design the corresponding fusion tactic of these clustering results to get equivalent classes and develop a new calculation method of conditional probability. Then, we use probabilistic hesitant fuzzy sets (P-HFSs) to aggregate the loss functions of different members of the same equivalence class and get the probabilistic hesitant fuzzy elements (P-HFEs) loss functions. In this case, P-HFSs not only reflect the hesitant situation of decision-makers, but also depict the proportion of different opinions. Regarding P-HFEs loss functions, we also investigate the modification methods of its outliers in detail. Moreover, based on the P-HFEs loss functions, we propose TWD with probabilistic hesitant fuzzy decision-theoretic rough sets (P-HFDTRSs). Finally, in order to verify the effectiveness of our proposed method, we develop a series of comparative experiments and discuss the decision results on six UCI datasets.

In the past years, the practical cross-domain machinery fault diagnosis problems have been attracting growing attention, where the training and testing data are collected from different operating conditions. The recent advances in closed-set domain adaptation have well addressed the basic problem where the fault mode sets are identical in the source and target domains. While some attempts have also been made on the partial and open-set domain adaptations, no prior information of the target-domain fault modes can be usually available in the real industries, that forms a challenging problem in transfer learning. This article proposes a universal domain adaptation method for fault diagnosis, where no explicit assumption is made on the target label set. A hybrid approach with source class-wise and target instance-wise weighting mechanism is proposed for selective adaptation. By using additional outlier identifier, the proposed method can automatically recognize the unknown fault modes while achieving class-level alignments for the shared health states, without knowing the target label set. Experiments on two rotating machine datasets validate the proposed method, which is promising for practical applications under strong data uncertainties.

Model-based market segmentation analyses often involve an ordinal dependent variable as ordinal responses are frequently collected in marketing research. In the Bayesian segmentation literature, there are models for an interval- or ratio-scaled dependent variable but there is not any general model for an ordinal dependent variable. In this manuscript, the authors propose a new Bayesian procedure to simultaneously perform segmentation and ordinal regression with variable selection within each derived segment. The procedure is robust to outliers and it also provides an option to include concomitant variables that allows the simultaneous profiling of the derived segments. The authors demonstrate that the practice of treating ordinal responses as interval- or ratio-scales to apply existing Bayesian segmentation procedures can lead to very misleading results and conclusions. Through simulation studies, the authors show that the proposed procedure outperforms several benchmark Bayesian segmentation models in parameter recovery, segment retention, and segment membership prediction for such data. Finally, they provide a commercial business customer satisfaction empirical application to illustrate the usefulness of the proposed model.

In recent times, the Internet of Things and big data analytics have become a hot research topic in mobile edge computing (MEC) desires wide-ranging research works for intelligent decision making. In this view, this paper designs an intelligent outlier detection with machine learning empowered big data analytics (IODML-BDA) for MEC. The proposed model involves an adaptive synthetic sampling-based outlier detection technique to eradicate the existence of outliers. Besides, the oppositional swallow swarm optimization (OSSO) based feature selection technique is used to choose an effective set of features. Finally, long short-term memory based classification model is employed to identify different class labels. The design of OSSO algorithm for feature selection with ADASYN technique for big data analytics show the novelty of the work. A comprehensive experimental analysis is carried out on GPS trajectories, movement prediction, water treatment plant, hepatitis, and Twitter datasets to confirm the experimental results. The experimentation outcomes pointed out that the proposed IODML-BDA model achieves the higher accuracy of 0.9735, 0.9816, 0.9798, 0.9896, and 0.9912 on the applied datasets.

Outlier detection is an important research area in the field of machine learning and data science. The presence of outliers in a dataset limits its true usefulness in a real-life scenario. Due to the varied challenges, researchers strive to find a general method to be useful for different datasets. In this paper, we have proposed an outlier detection technique based on unsupervised learning using an ensemble of three clustering algorithms, namely K-means, K-means++ and Fuzzy C-means. We have proposed a unique way to deal with clustered outliers. Outcomes of the three aforementioned clustering algorithms are combined intelligently to accumulate all the complementary information. To combine the decisions of the hard and soft clustering algorithms, we have proposed a novel probability-based technique, which assigns a membership value to each data point in the case of a hard clustering algorithm. Three cluster validity indices are used as our evaluation metrics, which measure the goodness of a cluster. Significant improvement of cluster validity indices is observed after removing the outliers, which ensures the removal of outliers has resulted in stringent clusters. The method is evaluated on eight datasets, among which, three datasets are comparatively large. Source code of this work is available at: https://github.com/biswarup9/Outlier-Detection-Using-an-Ensemble-of-Clustering-Algorithms- .

Machine learning (ML) has emerged as a powerful technique in the Earth system science, nevertheless, its potential to model complex atmospheric chemistry remains largely unexplored. Here, we applied ML to simulate the variability in urban ozone (O3) over Doon valley of the Himalaya. The ML model, trained with past variations in O3 and meteorological conditions, successfully reproduced the independent O3 data (r2 ~ 0.7). Model performance is found to be similar when the variation in major precursors (CO and NOx) were included in the model, instead of the meteorology. Further the inclusion of both precursors and meteorology improved the performance significantly (r2 = 0.86) and the model could also capture the outliers, which are crucial for air quality assessments. We suggest that in absence of high-resolution measurements, ML modeling has profound implications for unraveling the feedback between pollution and meteorology in the fragile Himalayan ecosystem.

This paper proposes a novel method for offline outlier detection in nonlinear dynamical systems using an input–output dataset of a Topical Negative Pressure Wound Therapy Device, NPWT. The fundamental characteristics of an NPWT describe a chaotic system whose states vary over time and may result in unpredictable and possibly anomalous divergent behavior in the presence of perturbations and other unmodeled system dynamics, despite a quasi-stable controller. Bacterial Memetic Algorithm, BMA, is used to generate fuzzy rule-based models of the input–output dataset. The error definition in the fuzzy rule extraction features a novel application of the Canberra Distance. The optimal number of rules for identifying the outliers, validated against both artificial and real system datasets, is calculated from the sample of inferred fuzzy models. The optimal number of rules is two in both cases based on the maximum average-error-drop. Using three or more rules results in better error performance; however, the algorithm learns the nuances of the outlier patterns instead. Novel methods for creating the outlier list and determining the optimal number of rules for the outlier detection problem are proposed.

With the rapid development of Internet of Things (IoT) technologies, the detection and analysis of malware have become a matter of concern in the industrial application of Cyber-Physical System (CPS) that provides various services using the IoT paradigm. Currently, many advanced machine learning methods such as deep learning are popular in the research of malware detection and analysis, and some achievements have been made so far. However, there are also some problems. For example, considering the noise and outliers in the existing datasets of malware, some methods are not robust enough. Therefore, the accuracy of malware classification still needs to be improved. Aiming at this issue, we propose a novel method that combines the correntropy and the deep learning model. In our proposed method for malware detection and analysis, given the success of the mixture correntropy as an effective similarity measure in addressing complex datasets with noise, it is therefore incorporated into a popular deep learning model, i.e., Convolutional Neural Network (CNN), to reconstruct its loss function, with the purpose of further detecting the features of outliers. We present the detailed design process of our method. Furthermore, the proposed method is tested both on a real-world malware dataset and a popular benchmark dataset, to verify its learning performance.

In recent years, the Internet of vehicles (IoV) technology becomes a research hotspot . However, it also becomes a hotbed for malicious attacks . In the IoV, frequent data transmission and complex connections among numerous different nodes increase the complexity and diversity of malicious attacks. In order to realize the accurate and rapid detection of malicious attacks in the IoV environment, in this paper, an intrusion detection method is proposed by combining oversampling, outlier detection and metric learning. The proposed approach improves intrusion detection effect in three main ways: 1) it oversamples the minority classes based on a novel strategy, 2) it introduces a new feature with basis of imbalance ratio, and 3) it reduces the outliers and rescales original samples actively to make the decision boundary clearer by combining outlier detection and distance metric learning. Furthermore, genetic algorithm is used to extract the optimal subset of features. The experimental results show that the proposed method can achieve 98.51% accuracy and maintain 0.82% false alarm rate on UNSW-NB15 dataset. Also, it outperforms the existing methods on ROAD, Car-Hacking and CAN-intrusion dataset for in-vehicle communications.

The classification in imbalanced datasets is one of the main problems for machine learning techniques. Support vector machine (SVM) is biased to the majority class samples, and the minority class samples may incorrectly be considered as noise. Therefore, SVM has poor predictive accuracy for imbalanced datasets and generates inaccurate classification models. Existing class imbalance learning (CIL) techniques can make SVM less sensitive to class imbalance, but these methods suffer from issues related to noise and outliers. Moreover, despite the solid theoretical basis and good classification performance, SVM is not appropriate for the classification of large-scale datasets because the training complexity of SVM is closely related to the dataset size. Class imbalance learning (CIL) using Fuzzy adaptive resonance theory (ART) and intuitionistic fuzzy twin SVM (CIL-FART-IFTSVM), which can be applied to address the class imbalance issue in the presence of noise and outliers and large scale datasets, is proposed to overcome these substantial difficulties. In this method, we modify the distribution of the datasets using fuzzy adaptive resonance theory (Fuzzy ART) as a clustering method to overcome the imbalance problem. Then, after data reduction, IFTSVM is utilized to find excellent non-parallel hyperplanes in the generated data points. Finally, a coordinate descent system with shrinking by an active set is applied to reduce the computational complexity. Forty-five imbalanced datasets are considered to validate the performance of the proposed CIL-FART-IFTSVM method. The Friedman test and the bootstrap technique with 95 % confidence intervals are applied to quantify the results statistically. The experimental results indicate that the method proposed in this paper has a better performance compared with other methods, and the training time is significantly better than that of other classifiers for large-scale datasets.

In the big data era, clustering is one of the most popular data mining method. Most clustering algorithms have complications like automatic cluster number determination, poor clustering precision, inconsistent clustering of various datasets and parameter-dependent, etc. A new fuzzy autonomous solution for clustering named Meskat-Mahmudul (MM) clustering algorithm was proposed to overcome the complexity of parameter-free automatic cluster number determination and clustering accuracy. The Meskat-Mahmudul clustering algorithm finds out the exact number of clusters based on the average silhouette method in multivariate mixed attribute dataset, including real-time gene expression dataset and missing values, noise, and outliers. Meskat-Mahmudul Extended K-Means (MMK) clustering algorithm enhances the K-Means algorithm, which serves the purpose of automatic cluster discovery and runtime cluster placement. Several validation methods are used to evaluate clusters and certify optimum cluster partitioning and perfection. Some datasets are used to assess the performance of the proposed algorithms to other algorithms in terms of time complexity and clustering efficiency. Finally, Meskat-Mahmudul clustering and Meskat-Mahmudul Extended K-Means clustering algorithms were found superior over conventional algorithms.

With the rapid development of maritime technologies, a huge amount of ocean data has been acquired through the state-of-the-art ocean equipment to get better understanding and development of ocean. The prediction and correction of oceanic observation data play a fundamental and important role in the oceanic relevant applications, including both civilian and military fields. On the basis of Argo data, aiming at predicting and correcting the oceanic observation data, we propose an ocean temperature and salinity prediction approach in this paper. In our approach, firstly, bounded nonlinear function is utilized for dataset quality control, which can effectively eliminate the influence of spikes or outliers in Argo data. Then, RBF neural network is used for high-resolution Argo dataset construction. Finally, a bidirectional LSTM framework is proposed to predict and analyze the ocean temperature and salinity on the basis of BOA Argo data. Experimental results demonstrate that the proposed bidirectional LSTM framework can accurately predict the ocean temperature and salinity and enable outstanding performance in oceanic observation data prediction and correction. The proposed approach is also important for the realization of Argo dataset automatic quality control.

Seeking reliable correspondences is a fundamental and significant work in computer vision. Recent work has demonstrated that the task can be effectively accomplished by utilizing a deep learning network based on multi-layer perceptrons, which uses the context normalization to deal with the input. However, the context normalization treats each correspondence equally, which will reduce the representation capability of potential inliers. To solve this problem, we propose a novel and effective Local-Global Self-Attention (LAGA) layer based on the self-attention mechanism, to capture contextual information of potential inliers from coarse to fine, and suppress outliers at the same time in processing the input. The global self-attention module is able to capture abundant global contextual information in the whole image, and the local self-attention module is used to obtain rich local contextual information in the local region. After that, to obtain richer contextual information and feature maps with stronger representative capacity, we combine global and local contextual information. The extensive experiments have shown that the networks with our proposed LAGA layer perform better than the original and other comparative networks in outdoor and indoor scenes for outlier removal and camera pose estimation tasks.

Since a large amount of data can be obtained in the oil production process nowadays and the operation environment is increasingly complicated, it is necessary to research unsupervised and robust fault detection methods for improving safety. In this paper, an online Bayesian-based technique with a novel decision fusion algorithm is proposed for unsupervised fault detection in the pumping unit. First, a new strategy to detect the working condition of the pumping unit by dynamometer card as well as five process measured variables is proposed. To deal with high-dimension data and outliers in dynamometer card, a robust Douglas-Peucker algorithm is developed for obtaining compressed data. A chord ratio index evaluating deviation degree of observations is defined, which can be used for removing outliers during approximation. Two norms are introduced for choosing the threshold in the proposed Douglas-Peucker algorithm. Moreover, a Bayesian-based online change point detection model is attempted for detecting univariate faults in the pumping unit. A decision fusion method derived from Bayesian probability formula is proposed for fusing univariate fault detection results. At last, the power of the proposed method is evaluated by numerical simulations and a real oil production process.