Search

Multivariate time series data (Mv-TSD) portray the evolving processes of the system(s) under examination in a multi-view manner. Factorization methods are salient for Mv-TSD analysis with the potentials of structural feature construction. However, research challenges remain in the derivation of factors due to scattered data distribution of Mv-TSD and intensive interferences/outliers embedded in the source data. The proposed Enhanced Bayesian Factorization approach (Enhanced-BF) addresses the challenges in three phases: (1) variant scale partitioning applies to Mv-TSD according to degree of amplitude and obtains the blocks of variant scales; (2) hierarchical Bayesian model for tensor factorization automatically derives the factors of each block with interferences suppressed; (3) Bayesian unification model merges those block factors to construct the final structural features. Enhanced-BF has been evaluated using a case study of brain data engineering with multivariate electroencephalogram (EEG). Experimental results indicate that the proposed method manifests robustness to the interferences and outperforms the counterparts in terms of operation efficiency and error when factorizing EEG tensor. Besides, Enhanced-BF excels in factorization-based analysis of ongoing autism spectrum disorder (ASD) EEG: 3 times speed-up in factorization and 87.35% accuracy in ASD discrimination. The latent factors can distinctly interpret the typical EEG characteristics of ASD subjects.

Semisupervised learning has been widely applied to deep generative model such as variational autoencoder. However, there are still limited work in noise-robust semisupervised deep generative model where the noise exists in both of the data and the labels simultaneously, which are referred to as outliers and noisy labels or compound noise. In this article, we propose a novel noise-robust semisupervised deep generative model by jointly tackling the noisy labels and outliers in a unified robust semisupervised variational autoencoder randomized generative adversarial network (URSVAE-GAN). Typically, we consider the uncertainty of the information of the input data in order to enhance the robustness of the variational encoder toward the noisy data in our unified robust semisupervised variational autoencoder (URSVAE). Subsequently, in order to alleviate the detrimental effects of noisy labels, a denoising layer is integrated naturally into the semisupervised variational autoencoder so that the variational inference is conditioned on the corrected labels. Moreover, to enhance the robustness of the variational inference in the presence of outliers, the robust β-divergence measure is employed to derive the novel variational lower bound, which already achieves competitive performance. This further motivates the development of URSVAE-GAN that collapses the decoder of URSVAE and the generator of a robust semisupervised generative adversarial network into one unit. By applying the end-to-end denoising scheme in the joint optimization, the experimental results demonstrate the superiority of the proposed framework by the evaluating on image classification and face recognition tasks and comparing with the state-of-the-art approaches.

The Micro-expressions (MEs) are very important nonverbal communication clues. However, due to their local and short nature, spotting them is challenging. In this paper, we address this problem by using a dedicated local and temporal pattern (LTP) of facial movement. This pattern has a specific shape (S-pattern) when ME are displayed. Thus, by using a classical classification algorithm (SVM), MEs are distinguished from other facial movements. We also propose a global final fusion analysis on the whole face to improve the distinction between ME (local) and head (global) movements. However, the learning of S-patterns is limited by the small number of ME databases and the low volume of ME samples. Hammerstein models (HMs) are known to be a good approximation of muscle movements. By approximating each S-pattern with a HM, we can both filter outliers and generate new similar S-patterns. By this way, we perform a data augmentation for S-pattern training dataset and improve the ability to differentiate micro-expressions from other facial movements. The spotting results, performed on CASME and CASME, show that our proposed LTP outperforms the most popular spotting method in terms of F1-score. Adding the fusion process and data augmentation improve even more the spotting performance.

As a novel similarity measure that is defined as the expectation of a kernel function between two random variables, correntropy has been successfully applied in robust machine learning and signal processing to combat large outliers. The kernel function in correntropy is usually a zero-mean Gaussian kernel. In a recent work, the concept of mixture correntropy (MC) was proposed to improve the learning performance, where the kernel function is a mixture Gaussian kernel, namely, a linear combination of several zero-mean Gaussian kernels with different widths. In both correntropy and MC, the center of the kernel function is, however, always located at zero. In the present work, to further improve the learning performance, we propose the concept of multikernel correntropy (MKC), in which each component of the mixture Gaussian kernel can be centered at a different location. The properties of the MKC are investigated and an efficient approach is proposed to determine the free parameters in MKC. Experimental results show that the learning algorithms under the maximum MKC criterion (MMKCC) can outperform those under the original maximum correntropy criterion (MCC) and the maximum MC criterion (MMCC).

Fault prognosis based on single model is generally inaccurate due to the varying working conditions. A multistage fault prognosis methodology combining stage identification with Bayesian networks (BNs) and time series approach with particular emphasis on the autoregressive moving average (ARMA) model is proposed to solve this problem. In the first stage, degradation data are identified, and outliers are marked by the Euclidean distance. Degenerate attributes of outliers are finely identified by BNs and matched to the corresponding model. In the second stage, the ARMA model is used for prognosis according to the results of the fine identification. Subsequently, the double-precision identification and ARMA submodel prognosis are carried out alternately throughout the prognosis process. Three degradation types of PMSM are simulated to verify the applicability of the method. Result shows that it can track the changes in the degradation in time and obtains better results.

Outlier detection in multivariate data is one of the critical challenges in preprocessing phase. Many outlier detection methods have been emerged for the past few years to perform outlier detection efficiently in multivariate datasets. The prediction accuracy cannot be improved without proper outlier analysis and the prediction model might not confirm the expected behavior. The generation of huge data in real time applications makes the outlier detection process more crucial and challenging. Most of the currently available detection methods are based on mean and covariance that are not suitable for handling large volume of datasets, they are suitable for handlind static data and simple data to detect outliers. They cannot cope up with large scale data. So, there is a need for an efficient outlier detection model to detect the outliers in multivariate datasets. The primary objective of this research work is to develop a robust model for outlier detection in multivariate data. To achieve this, the work proposed an enhanced Hidden Semi-Markov Model (HSMM) which allows arbitrary time distribution in its states to detect outliers. The proposed work utilized six benchmark datasets and the performance is compared with several outlier detection algorithms such as HMM, iForest, FastABOD, and Expose. The work achieves 98.2 % of accuracy which is significantly better for detecting outliers in multivariate dataset. The proposed work improvised the percentage of acheivements between 2% to 25% than the currently available models.. The experimental analysis shows that the proposed model performs well than the currently available models in terms of accuracy, and receiver operation curve (ROC).

K-means is one of ten popular clustering algorithms. However, k-means performs poorly due to the presence of outliers in real datasets. Besides, a different distance metric makes a variation in data clustering accuracy. Improve the clustering accuracy of k-means is still an active topic among researchers of the data clustering community from outliers removal and distance metrics perspectives. Herein, a novel modification of the k-means algorithm is proposed based on Tukey’s rule in conjunction with a new distance metric. The standard Tukey rule is modified to remove the outliers adaptively by considering whether the data is distributed to the left, right or even to the input data's mean value. The elimination of outliers is applied in the proposed modification of the k-means before calculating the centroids to minimize the outliers' influences. Meanwhile, a new distance metric is proposed to assign each data point to the nearest cluster. In this research, the modified k-means significantly improves the clustering accuracy and centroids convergence. Moreover, the proposed distance metric's overall performance outperforms most of the literature distance metrics. This manuscript's presented work demonstrates the significance of the proposed technique to improve the overall clustering accuracy up to 80.57% on nine standard multivariate datasets.

Outlier detection is widely used in many fields, such as intrusion detection, credit card fraud detection, medical diagnosis, and so on. Existing outlier detection algorithms are mostly designed for dealing with numeric or categorical attributes. However, data usually exist in the form of mixed attributes in real-world applications. In this article, we propose a novel mixed attribute outlier detection method based on multigranulation relative entropy by employing the neighborhood rough set. First, the neighborhood system is constructed by optimizing the mixed distance metric and the radius of the statistical value. Second, the neighborhood entropy is introduced as an uncertainty measure of data. Furthermore, the three kinds of multigranulation relative entropy-based matrices are defined by three kinds of attribute sequences, and the multigranulation relative entropy-based outlier factor is integrated to indicate the outlier degree of every object. Based on the proposed outlier detection model, the corresponding algorithm is designed. Finally, the proposed algorithm is compared with other nine algorithms through experiments on public data. The experimental results show that the proposed technique is adaptive and effective.

Extracting genes involved in cancer lesions from gene expression data is critical for cancer research and drug development. the method of feature selection has attracted much attention in the field of bioinformatics. Principal Component Analysis (PCA) is a widely used method for learning low-dimensional representation. Some variants of PCA have been proposed to improve the robustness and sparsity of the algorithm. However, the existing methods ignore the high-order relationships between data. In this paper, a new model named Robust Principal Component Analysis via Hypergraph Regularization (HRPCA) is proposed. In detail, HRPCA utilizes L2,1-norm to reduce the effect of outliers and make data sufficiently row-sparse. And the Hypergraph Regularization is introduced to consider the complex relationship between data. Important information hidden in the data are mined, and this method ensures the accuracy of the resulting data relationship information. Extensive experiments on multi-view biological data demonstrate that the feasible and effective of the proposed approach.

In one-class classification, the low variance directions in the training data carry crucial information to build a good model of the target class. Boundary-based methods like One-Class Support Vector Machine (OSVM) preferentially separates the data from outliers along the large variance directions. On the other hand, retaining only the low variance directions can result in sacrificing some initial properties of the original data and is not desirable, specially in case of limited training samples. This paper introduces a Covariance-guided One-Class Support Vector Machine (COSVM) classification method which emphasizes the low variance projectional directions of the training data without compromising any important characteristics. COSVM improves upon the OSVM method by controlling the direction of the separating hyperplane through incorporation of the estimated covariance matrix from the training data. Our proposed method is a convex optimization problem resulting in one global optimum solution which can be solved efficiently with the help of existing numerical methods. The method also keeps the principal structure of the OSVM method intact, and can be implemented easily with the existing OSVM libraries. Comparative experimental results with contemporary one-class classifiers on numerous artificial and benchmark datasets demonstrate that our method results in significantly better classification performance. HighlightsThe low-variance directions are crucial for one-class classification (OCC).A new method of OCC emphasizing the low-variance directions is proposed.The method incorporates covariance information into convex optimization problem.Can be implemented and solved efficiently with existing software.Comparative experiments with contemporary classifiers show positive results.

The evaluation of pilot brain activity is very important for flight safety. This study proposes a Hidden semi-Markov Model with Hierarchical prior to detect brain activity under different flight tasks. A dynamic student mixture model is proposed to detect the outlier of emission probability of HSMM. Instantaneous spectrum features are also extracted from EEG signals. Compared with other latent variable models, the proposed model shows excellent performance for the automatic inference of brain cognitive activity of pilots. The results indicate that the consideration of hierarchical model and the emission probability with t mixture model improves the recognition performance for Pilots' fatigue cognitive level.

The research of semantic segmentation based on unsupervised domain adaptation greatly alleviates the high-cost bottleneck of manual annotation in deep learning. Inevitably domain gap limits the ability of target domain to learn knowledge from source domain. Previous domain alignment strategies aim to explore maximum domain-invariant space to enlarge knowledge that target domain learns. They based on the assumption that images are symmetrical in categories covering all predefined categories. However, in practical training, there is class asymmetry in image samples inter domains. This will lead to outlier categories and negative transfer, because it lacks the guidance of the corresponding category of the source domain. To tackle this issue, we propose a partial domain adaptive method on semantic segmentation to guide target model to learn category-level knowledge selectively. PAM (Partial Adaptive Map) module is introduced to motivate the target model to obtain more knowledge from non-outliers and less knowledge from outliers to avoid negative transfer. We further analyze the effectiveness of this method from the perspective of JS divergence. Our method has achieved significant improvement without additional discriminators. Experiments on general datasets GTA5 and SYNTHIA can compare with SOTA methods.

Existing domain adaptation (DA) methods generally assume that different domains have identical label space, and the training data are only sampled from a single domain. This unrealistic assumption is quite restricted for real-world applications, since it neglects the more practical scenario, where the source domain can contain the categories that are not shared by the target domain, and the training data can be collected from multiple modalities. In this article, we address a more difficult but practical problem, which recognizes RGB images through training on RGB-D data under the label space inequality scenario. There are three challenges in this task: 1) source and target domains are affected by the domain mismatch issue, which results in that the trained models perform imperfectly on the test data; 2) depth images are absent in the target domain (e.g., target images are captured by smartphones), when the source domain contains both the RGB and depth data. It makes the ordinary visual recognition approaches hardly applied to this task; and 3) in the real world, the source and target domains always have different numbers of categories, which would result in a negative transfer bottleneck being more prominent. Toward tackling the above challenges, we formulate a deep model, called visual-depth matching network (VDMN), where two new modules and a matching component can be trained in an end-to-end fashion jointly to identify the common and outlier categories effectively. The significance of VDMN is that it can take advantage of depth information and handle the domain distribution mismatch under label inequality simultaneously. The experimental results reveal that VDMN exceeds the state-of-the-art performance on various DA datasets, especially under the label inequality scenario.

Robustness enhancement and feature selection are the two crucial issues to be resolved in robust palmprint recognition. However, existing regression-based methods are insufficient to handle outliers and select significant features. From a statistical viewpoint, we present a general framework to intrinsically resolve the two issues. By investigating the role of outliers in the formation of coding errors, we devise a double-Laplacian mixture-error model to faithfully fit the error distribution. Additionally, we design a supervised group-sparse regularizer to enforce the locality and group sparsity of the codes. Integrating the two parts into the framework produces a nonconvex constrained problem, for which we develop an iteratively reweighted l1-l1 minimization algorithm by combining the majorization-minimization strategy and the alternating direction method of multipliers. The weighted vector learned from the error model and the local group sparsity enforced by the regularizer enable our method to better handle outliers and select more significant features than the state-of-the-art methods. Extensive experimental results verify the flexibility and robustness of our method to various contaminations.

The exploration of single cell RNA-sequencing (scRNA-seq) technology generates a new perspective to analyze biological problems. One of the major applications of scRNA-seq data is to discover subtypes of cells by cell clustering. Nevertheless, it is challengeable for traditional methods to handle scRNA-seq data with high level of technical noise and notorious dropouts. To better analyze single cell data, a novel scRNA-seq data analysis model called Maximum correntropy criterion based Non-negative and Low Rank Representation (MccNLRR) is introduced. Specifically, the maximum correntropy criterion, as an effective loss function, is more robust to the high noise and large outliers existed in the data. Moreover, the low rank representation is proven to be a powerful tool for capturing the global and local structures of data. Therefore, some important information, such as the similarity of cells in the subspace, is also extracted by it. Then, an iterative algorithm on the basis of the half-quadratic optimization and alternating direction method is developed to settle the complex optimization problem. Before the experiment, we also analyze the convergence and robustness of MccNLRR. At last, the results of cell clustering, visualization analysis, and gene markers selection on scRNA-seq data reveal that MccNLRR method can distinguish cell subtypes accurately and robustly.

Today's large datacenters house a massive number of machines, each of which is being closely monitored with multivariate time series (e.g., CPU idle, memory utilization) to ensure service quality. Detecting outlier machine instances with multivariate time series is crucial for service management. However, it is a challenging task due to the multiple classes and various shapes, high dimensionality, and lack of labels of multivariate time series. In this paper, we propose DOMI, a novel unsupervised model that combines Gaussian mixture VAE with 1D-CNN, to detect outlier machine instances. Its core idea is to capture the normal patterns of machine instances by learning their latent representations that consider the shape characteristics, reconstruct input data by the learned representations, and apply reconstruction probabilities to determine outliers. Moreover, DOMI interprets the detected outlier instance based on the reconstruction probability changes of univariate time series. Extensive experiments have been conducted on the dataset collected from 1821 machines with a 1.5-month-period, which are deployed in ByteDance, a top global content service provider. DOMI achieves the best F1-Score of 0.94 and AUC score of 0.99, significantly outperforming the best performing baseline method by 0.08 and 0.03, respectively. Moreover, its interpretation accuracy is up to 0.93.

Locality-preserving projection (LPP) has been widely used in feature extraction. However, LPP does not use data category information and uses the 2 L -norm for distance measurement, which is highly sensitive to outliers. In this paper, we consider the LPP weight matrix from a supervised perspective and combine the low-rank regression method to propose a new model to discover and extract features. By using the L2,1-norm to constrain the loss function and the regression matrix, not only is the sensitivity to outliers reduced but the low-rank condition of the regression matrix is also restricted. Then, we propose a solution to the optimization problem. Finally, we apply the method to a series of face databases, handwriting digital datasets and palmprint datasets to test the performance, and the experimental results show that this method is effective compared with some existing methods.

In this paper, we proposed a more robust supervised hashing framework based on the Cauchy loss function and Supervised Discrete Hashing (SDH) called Robust Supervised Discrete Hashing (RSDH), which can learn a robust subspace consisted of binary codes. The Cauchy loss is used to measure the error between the label matrix and the product of the decomposed matrices. RSDH can not only reduce the outliers and noise of the hashing codes, but also achieve the more satisfactory retrieval effect. Image retrieval experiments demonstrate that RSDH performs better than the other hashing methods.

We perform an out-of-distribution analysis of ~12,000,000 semi-independent 128x128 pixel^2 SST regions, which we define as cutouts, from all nighttime granules in the MODIS R2019 Level-2 public dataset to discover the most complex or extreme phenomena at the ocean surface. Our algorithm (Ulmo) is a probabilistic autoencoder, which combines two deep learning modules: (1) an autoencoder, trained on ~150,000 random cutouts from 2010, to represent any input cutout with a 512-dimensional latent vector akin to a (non-linear) EOF analysis; and (2) a normalizing flow, which maps the autoencoder's latent space distribution onto an isotropic Gaussian manifold. From the latter, we calculate a log-likelihood value for each cutout and define outlier cutouts to be those in the lowest 0.1% of the distribution. These exhibit large gradients and patterns characteristic of a highly dynamic ocean surface, and many are located within larger complexes whose unique dynamics warrant future analysis. Without guidance, Ulmo consistently locates the outliers where the major western boundary currents separate from the continental margin. Buoyed by these results, we begin the process of exploring the fundamental patterns learned by Ulmo, identifying several compelling examples. Future work may find that algorithms like Ulmo hold significant potential/promise to learn and derive other, not-yet-identified behaviors in the ocean from the many archives of satellite-derived SST fields. As important, we see no impediment to applying them to other large, remote-sensing datasets for ocean science (e.g., sea surface height, ocean color)., Comment: 16 pages, 12 figures

Estimating endmembers and corresponding abundances from mixed pixels are essential steps for hyperspectral unmixing. In hyperspectral unmixing, obtaining accurate unmixing results is difficult since less prior knowledge is available. Besides, the unmixing results are influenced by noise and highly correlated endmembers, so that the obtained abundance maps exist small values which are not present in the image. In this paper, we separate each abundance map into a feature layer and a sparse layer to protect the obtained abundance maps from the above-mentioned factors. The feature layer represents the main information of the abundance map. And the sparse layer contains outliers dominated by the above-mentioned factors. In particular, we design a feature extraction regularization to describe the feature layer and use a weighted l 1 norm to describe the sparse layer. Then, under the framework of non-negative tensor factorization (NTF), we propose a novel unmixing algorithm named spatial feature extraction NTF (SFE-NTF) for hyperspectral unmixing. The proposed SFE-NTF is based on an augmented multiplicative algorithm. Experimental results on both synthetic and real hyperspectral data demonstrate that the proposed algorithm outperforms other state-of-the-art algorithms.

We present a novel model called One Class Minimum Spanning Tree (OCmst) for novelty detection problem that uses a Convolutional Neural Network (CNN) as deep feature extractor and graph-based model based on Minimum Spanning Tree (MST). In a novelty detection scenario, the training data is no polluted by outliers (abnormal class) and the goal is to recognize if a test instance belongs to the normal class or to the abnormal class. Our approach uses the deep features from CNN to feed a pair of MSTs built starting from each test instance. To cut down the computational time we use a parameter $\gamma$ to specify the size of the MST's starting to the neighbours from the test instance. To prove the effectiveness of the proposed approach we conducted experiments on two publicly available datasets, well-known in literature and we achieved the state-of-the-art results on CIFAR10 dataset., Comment: 16 pages

Probabilistic linear discriminant analysis (PLDA) is a very effective feature extraction approach and has obtained extensive and successful applications in supervised learning tasks. It employs the squared L₂-norm to measure the model errors, which assumes a Gaussian noise distribution implicitly. However, the noise in real-life applications may not follow a Gaussian distribution. Particularly, the squared L₂-norm could extremely exaggerate data outliers. To address this issue, this article proposes a robust PLDA model under the assumption of a Laplacian noise distribution, called L1-PLDA. The learning process employs the approach by expressing the Laplacian density function as a superposition of an infinite number of Gaussian distributions via introducing a new latent variable and then adopts the variational expectation-maximization (EM) algorithm to learn parameters. The most significant advantage of the new model is that the introduced latent variable can be used to detect data outliers. The experiments on several public databases show the superiority of the proposed L1-PLDA model in terms of classification and outlier detection.

Deep metric based triplet loss has been widely used to enhance inter-class separability and intra-class compactness of network features. However, the margin parameters in the triplet loss for current approaches are usually fixed and not adaptive to the variations among different expression pairs. Meanwhile, outlier samples like faces with confusing expressions, occlusion and large head poses may be introduced during the selection of the hard triplets, which may deteriorate the generalization performance of the learned features for normal testing samples. In this work, a new triplet loss based on class-pair margins and multistage outlier suppression is proposed for facial expression recognition (FER). In this approach, each expression pair is assigned with an order-insensitive or two order-aware adaptive margin parameters. While expression samples with large head poses or occlusion are firstly detected and excluded, abnormal hard triplets are discarded if their feature distances do not fit the model of normal feature distance distribution. Extensive experiments on seven public benchmark expression databases show that the network using the proposed loss achieves much better accuracy than that using the original triplet loss and the network without using the proposed strategies, and the most balanced performances among state-of-the-art algorithms in the literature.

Currently, most of the supervised change detection approaches require a training data set that contains samples from both the changed and the unchanged data. However, under certain condition, such as natural disaster and military attack, the changed data samples are very few or even not available but the unchanged data are abundant. In this letter, we develop a generative adversarial networks (GANs)-based one-class classification (OCC) technique for time series remote-sensing image change detection. The proposed method is only trained with the unchanged data instead of both the changed and unchanged data. To achieve this purpose, first, spatial-spectral features are extracted from the time series remote-sensing images. Second, a GAN model is trained to detect the changes only with the extracted features of unchanged data. Remarkably, to offset the outlier errors caused by the incomplete supervision information provided by unchanged data alone, changed data, instead of unchanged data, are generated to improve power of discriminator. Finally, testing data are classified by the trained discriminator of GAN to produce a binary change map. Experimental results obtained on two optical time series remote-sensing data sets confirmed the effectiveness of our proposed method.

In industrial applications of machinery fault diagnostics, transfer learning is often used to transfer the knowledge learned from the source domain including labeled data to the target domain containing unlabeled data. This method follows the assumption that the label space in the source and target domain is the same. However, the label space of the target domain may be unknown in reality. For ensuring the integrity of knowledge, labeled data with all common fault types are considered as the source domain. However, if the label space of the target domain is less than that of the source domain, the negative effect from outlier classes will influence the performance. For solving this problem, a reconstruction domain adaptation transfer network (RDATN) is proposed. RDATN mainly contains health condition recognition and domain adaptation. The former is constructed for health condition recognition, while another one contributes to extracting domain-invariant features. Additionally, for reducing negative effects from outlier classes, a class-level weight and a sample-level weight are applied in the loss function. To improve the performance, particle swarm optimization is applied to search for initial parameters for RDATN. The superiority of the proposed model is verified through three bearing datasets in two experiments.

Establishing reliable correspondences by a deep neural network is an important task in computer vision, and it generally requires permutation-equivariant architecture and rich contextual information. In this paper, we design a Permutation-Equivariant Split Attention Network (called PESA-Net), to gather rich contextual information for the feature matching task. Specifically, we propose a novel “Split–Squeeze–Excitation–Union” (SSEU) module. The SSEU module not only generates multiple paths to exploit the geometrical context of putative correspondences from different aspects, but also adaptively captures channel-wise global information by explicitly modeling the interdependencies between the channels of features. In addition, we further construct a block by fusing the SSEU module, Multi-Layer Perceptron and some normalizations. The proposed PESA-Net is able to effectively infer the probabilities of correspondences being inliers or outliers and simultaneously recover the relative pose by essential matrix. Experimental results demonstrate that the proposed PESA-Net relative surpasses state-of-the-art approaches for pose estimation and outlier rejection on both outdoor scenes and indoor scenes (i.e., YFCC100M and SUN3D). Source codes: https://github.com/x-gb/PESA-Net .

Point set registration is one of the challenging tasks in remote sensing image processing and analysis. Its critical step is to find the corresponding relationships between the fixed scene point set and the moving model point set that undergo different sorts of transformations. Existing algorithms primarily utilize different types of prior information to improve the registration performance, such as spatial consistency, local similarity, and uniform outliers. However, due to the lack of active evaluation on the prior and intermediate information during the registration process, these strategies are susceptible to large data transformations. In order to enhance the robustness and accuracy for point set registration, we propose in this article a novel framework, namely Registration-is-Evaluation (RisE). Based on a multigranular probability model, our method exploits and utilizes both prior and posterior information to dynamically evaluate the matching status. What is more, instead of adding an extra uniform prior, we unified the outliers, noise, missing points, and heavily warped points into our registration evaluation model and address them simultaneously. We also apply a novel point set descriptor, called local polar relative geometry (LPRG), to have a more robust local similarity measurement. It adopts the local polar coordinate to perform multiscale pooling and relative geometric computation. Based on our proposed method, the matching relationships and the spatial transformations can be actively evaluated to provide useful contextual guidance for the registration process. Experimental results on multiple data sets show that our algorithm outperforms the state-of-the-art methods, in terms of both accuracy and robustness under large data degradations.

Graph and subspace clustering methods have become the mainstream of multi-view clustering due to their promising performance. However, (1) since graph clustering methods learn graphs directly from the raw data, when the raw data is distorted by noise and outliers, their performance may seriously decrease; (2) subspace clustering methods use a “two-step” strategy to learn the representation and affinity matrix independently, and thus may fail to explore their high correlation. To address these issues, we propose a novel multi-view clustering method via learning a Low-Rank Tensor Graph (LRTG). Different from subspace clustering methods, LRTG simultaneously learns the representation and affinity matrix in a single step to preserve their correlation. We apply Tucker decomposition and l2;1-norm to the LRTG model to alleviate noise and outliers for learning a “clean” representation. LRTG then learns the affinity matrix from this “clean” representation. Additionally, an adaptive neighbor scheme is proposed to find the K largest entries of the affinity matrix to form a flexible graph for clustering. An effective optimization algorithm is designed to solve the LRTG model based on the alternating direction method of multipliers. Extensive experiments on different clustering tasks demonstrate the effectiveness and superiority of LRTG over seventeen state-of-the-art clustering methods.

Accurate point matching is widely used, and it is a critical and challenging process in feature-based image registration. To improve feature matching accuracy on putative matches with heavy outliers and similar local structures, an accurate and robust feature point matching algorithm based on minimum relative motion entropy (MRME) is proposed, in which the relative motion between the putative matches and their K-nearest neighbors is formulated. Based on the relative motion clustering result, the relative motion entropy is defined to find the coincident relative motions. According to relative motions with MRME, the outliers are removed in a two-stage feature match strategy. With quasi-linear time complexity, outliers with random or irregular relative motion are removed efficiently and accurately, while inliers with coincident relative motion are retained. Three data sets with repetitive patterns, viewpoint changes, low overlapping areas, and local deformations are used to demonstrate the performance of the proposed algorithm. MRME is shown to be more robust and accurate than ten state-of-the-art feature matching algorithms.

Although several algorithms have achieved automatic registration on optical and synthetic aperture radar (SAR) images, it is still a challenge to establish enough reliable correspondences between such images due to their different imaging mechanisms. For this purpose, we propose a robust point-feature-based registration method. Considering the inherent properties of optical image and SAR image, two different gradient operators are utilized to construct scale spaces and extract features. A new gradient operator is defined for SAR image, yielding a more consistent gradient with optical image gradient calculated by the multiscale Sobel operator. Then, a novel cascaded matching method called cascaded sample consensus (CSC) is put forward to increase the number of correct correspondences. In the first matching, a simple but effective scale constraint strategy is used to remove outliers for a robust initial transformation model. Considering the spatial location relationship in each correct matching pair, the second matching constructs precise search spaces of the best matching points for more correspondences. Experimental results finally verify the robustness and accuracy of the proposed algorithm.

Nonnegative matrix factorization (NMF) is a widely used hyperspectral unmixing model which decomposes a known hyperspectral data matrix into two unknown matrices, i.e., endmember matrix and abundance matrix. Due to the use of least-squares loss, the NMF model is usually sensitive to noise or outliers. To improve its robustness, we introduce a general robust loss function to replace the traditional least-squares loss and propose a general loss-based NMF (GLNMF) model for hyperspectral unmixing in this letter. The general loss function is a superset of many common robust loss functions and is suitable for handling different types of noise. Experimental results on simulated and real hyperspectral data sets demonstrate that our GLNMF model is more accurate and robust than existing NMF methods.

Existing data-driven fault diagnosis methods assume that the label sets of the training data and test data are consistent, which is usually not applicable for real applications since the fault modes that occur in the test phase are unpredictable. To address this problem, open set fault diagnosis (OSFD), where the test label set consists of a portion of the training label set and some unknown classes, is studied in this article. Considering the changeable operating conditions of machinery, OSFD tasks are further divided into shared-domain open set fault diagnosis (SOSFD) and cross-domain open set fault diagnosis (COSFD) in this article. For SOSFD, 1-D convolutional neural networks are trained for learning discriminative features and recognizing fault modes. For COSFD, due to the distribution discrepancy between the source and target domains, the deep model needs to learn domain-invariant features of shared classes and separate features of outlier classes. Thus, by utilizing the output of an additional domain classifier, a model named bilateral weighted adversarial networks is proposed to assign large weights to shared classes and small weights to outlier classes during the feature alignment. In the test phase, samples are classified according to the outputs of the deep model and unknown-class samples are rejected by the extreme value theory model. Experimental results on two bearing datasets demonstrate the effectiveness and superiority of the proposed method.

Anomaly detection suffers from unbalanced data since anomalies are quite rare. Synthetically generated anomalies are a solution to such ill or not fully defined data. However, synthesis requires an expressive representation to guarantee the quality of the generated data. In this paper, we propose a two-level hierarchical latent space representation that distills inliers' feature-descriptors (through autoencoders) into more robust representations based on a variational family of distributions (through a variational autoencoder) for zero-shot anomaly generation. From the learned latent distributions, we select those that lie on the outskirts of the training data as synthetic-outlier generators. And, we synthesize from them, i.e., generate negative samples without seen them before, to train binary classifiers. We found that the use of the proposed hierarchical structure for feature distillation and fusion creates robust and general representations that allow us to synthesize pseudo outlier samples. And in turn, train robust binary classifiers for true outlier detection (without the need for actual outliers during training). We demonstrate the performance of our proposal on several benchmarks for anomaly detection., To appear in IEEE Trans. on Neural Networks and Learning Systems

Feature selection plays an important role in many areas that are relevant to machine learning. Information theory is widely applied to feature selection methods because it can measure linear and nonlinear correlations among variables. However, previous information theory-based feature selection methods ignore two critical problems. First, features are collected or annotated manually, it is inevitable to make some incorrect features or outliers. Second, previous information theory-based methods pay much attention to the correlations among features while ignoring the correlations between samples. To address these problems, this paper proposes an information theory-based feature selection method: Preserving Similarity and Staring Decisis. The proposed method firstly employs the similarity between samples to obtain a new transformed data that avoids outliers or incorrect features in original data. Meanwhile, the new transformed data enhances the separability of features, which is beneficial to feature selection. Additionally, different from previous methods that consider all the already-selected features in the feature selection process, the proposed method stares the last already-selected feature only, which reduces the computational cost. Extensive experiments demonstrate that the proposed method outperforms nine state-of-the-art methods in terms of multiple criteria significantly.

With the fast growing of data-rich systems, dealing with complex decision problems with skewed input data sets and respective outliers is unavoidable. Generally, data skewness refers to a non-uniform distribution in a dataset, i.e. a dataset which contains asymmetries and/or outliers. Normalization is the first step of most multi-criteria decision making (MCDM) problems to obtain dimensionless data, from heterogeneous input data sets, that enable aggregation of criteria and thereby ranking of alternatives. Therefore, when in presence of outliers in criteria datasets, finding a suitable normalization technique is of utmost importance. As such, in this work, we compare seven normalization techniques (Max, Max-Min, Vector, Sum, Logarithmic, Target-based, and Fuzzification) on criteria datasets, which contain outliers to analyse their results for MCDM problems. A numerical example illustrates the behaviour of the chosen normalization techniques and an (ongoing) evaluation assessment framework is used to recommend the best normalization technique for this type of criteria.

We consider functional outlier detection from a geometric perspective, specifically: for functional data sets drawn from a functional manifold which is defined by the data's modes of variation in amplitude and phase. Based on this manifold, we develop a conceptualization of functional outlier detection that is more widely applicable and realistic than previously proposed. Our theoretical and experimental analyses demonstrate several important advantages of this perspective: It considerably improves theoretical understanding and allows to describe and analyse complex functional outlier scenarios consistently and in full generality, by differentiating between structurally anomalous outlier data that are off-manifold and distributionally outlying data that are on-manifold but at its margins. This improves practical feasibility of functional outlier detection: We show that simple manifold learning methods can be used to reliably infer and visualize the geometric structure of functional data sets. We also show that standard outlier detection methods requiring tabular data inputs can be applied to functional data very successfully by simply using their vector-valued representations learned from manifold learning methods as input features. Our experiments on synthetic and real data sets demonstrate that this approach leads to outlier detection performances at least on par with existing functional data-specific methods in a large variety of settings, without the highly specialized, complex methodology and narrow domain of application these methods often entail., Comment: 40 pages, 20 figures

Due to sensor drift, instrumental variation, or change of measurement object, the distributions of the datasets (domains) acquired by the sensors are often different. A domain adaptive framework called MCSP-SSS is proposed to reduce the distribution discrepancy across domains for high-accuracy sensor data classification. The framework consists of two key parts: Multi-Constraint Subspace Projection (MCSP) and Source Sample Selection (SSS). MCSP is a subspace-projection-based approach, which introduces four constraints to get an optimized projection matrix: Principal Component Analysis (PCA) is used to reduce the redundancy of features; Mean Distribution Discrepancy (MDD) is applied to minimize the difference between source and target data; Hilbert-Schmidt Independence Criterion (HSIC) is adopted to maximizing the dependence between features and labels; A so-called weighted-within-class scatter matrix is introduced to minimize the within-class variance to avoid samples with different labels to overlap in the subspace. SSS is designed to remove the outliers in projected source samples so as to reduce distribution discrepancy between the source and target samples projected by MCSP. Experimental results show that our framework can achieve the best accuracies in all classification tasks in comparison with other state-of-the-art approaches. The accuracy of MCSP-SSS is 3.57 percentage points (pps) higher on average than that of the second best approach for single-source domain adaptation, and 7.00 pps for multi-source domain adaptation. Source code is available at https://github.com/threedteam/tsc_subspace_projection .

Unsupervised subsequence outlier detection on multivariate time series (MTS) is a valuable problem in practice that can observably save the cost of labeling and provide interpretability in real applications. For the task, most of the classic methods are under two strong assumptions: (i) stationary MTS. it may have difficulty coping with the phenomenon of time drift. (ii) Attribute-level IIDness (independent and identically distributed), it may ignore the relationship between attribute when measuring the similarity between multivariate subsequences. The above assumptions limit the availability of existing methods in real scenarios. To address this issue, this paper introduces a novel coupled generalized local outlier detection model for MTS, which extends the traditional generalized local outlier detection model to cope with subsequence outlier detection tasks by incorporating a novel Non-IID similarity metric. Specifically, the proposed method mainly includes three aspects: (i) represents the MTS relationship in symbolic space which provides a lower complexity and satisfactory sensitivity. (ii) Proposes a Non-IID coupled similarity metric (TSDis) which considers the intrinsic intra-attribute and inter-attribute coupling between segments. (iii) Extends the traditional generalized local outlier detection model to handle subsequence outlier detection tasks by embedding Non-IID coupled similarity metric. Experimental results show the proposed method can utilize the potential characteristics of MTS effectively and stably. Meanwhile, it detects outliers more accurately than baseline approaches on 12 time-series datasets.

The common spatial pattern (CSP) algorithm is a well-recognized spatial filtering method for feature extraction in motor imagery (MI)-based brain-computer interfaces (BCIs). However, due to the influence of nonstationary in electroencephalography (EEG) and inherent defects of the CSP objective function, the spatial filters, and their corresponding features are not necessarily optimal in the feature space used within CSP. In this work, we design a new feature selection method to address this issue by selecting features based on an improved objective function. Especially, improvements are made in suppressing outliers and discovering features with larger interclass distances. Moreover, a fusion algorithm based on the Dempster-Shafer theory is proposed, which takes into consideration the distribution of features. With two competition data sets, we first evaluate the performance of the improved objective functions in terms of classification accuracy, feature distribution, and embeddability. Then, a comparison with other feature selection methods is carried out in both accuracy and computational time. Experimental results show that the proposed methods consume less additional computational cost and result in a significant increase in the performance of MI-based BCI systems.

The method presented herein performs the tasks of identification, modeling, and registration refinement of poles using Light Detection And Ranging (LiDAR) point clouds. This method starts with a preliminary segmentation followed by eliminating outliers by integrating Fourier series and robust estimators. Poles’ cross sectional shapes are then identified by a customized template matching. Afterwards, points belonging to poles are extracted and the as-built model and registration-refined version of poles are generated. The algorithm was tested on five datasets including point clouds of three industrial sites, an urban environment, and a pole-like monument with very different volumes, resolutions, and configurations. The developed method works robustly despite the challenges that are often present in LiDAR data comprising outliers, non-uniform point sampling, gaps, and registration error. The obtained results indicate that 116 out of 117 poles in all five datasets were identified with no false positives at the object level. At point level, it reached greater than 98% average precision and 98% average accuracy in all datasets. Poles were modeled with a sub-millimeter precision in all datasets except for the third dataset in which poles were modeled with millimeter precision.

Point clouds from scanners or stereo vision inevitably contain outliers which have negative effects on subsequent procedures. Previous works classify outliers according to their characteristics, which guide the design of targeted outlier removal methods. Thus, outlier classification is critical to the corresponding method and outlier removal effect. The proposed type-based outlier removal framework (TBORF) aims to classify outliers more elaborately by considering both the characteristics of the underlying point cloud and the outliers. Therefore, the designed outlier removal methods can be more targeted. To this end, the framework first quantifies the characteristics of the input point cloud using three proposed metrics. According to this quantitative result, the input point clouds are classified into four types. Meanwhile, three new single-criterion methods are proposed to improve the effect on specific types of outlier removal. Based on the point cloud classification and the proposed single-criterion methods, an appropriate combined method is carefully designed for each type of point cloud. Performance evaluations on both outdoor and indoor point cloud datasets demonstrate that TBORF can effectively remove various outliers, facilitating subsequent digital geometry processing operations.

The preparation of a dataset by merging multiple data sources using the data fusion method may lead to the loss of vital information from each multi-source dataset and a certain amount of correlative information among the multiple data sources. Based on the extensive analysis of “the unique characteristics” of multi-source outliers, we propose multi-source outlier detection techniques to reliably identify outliers in multiple datasets. Several real-world examples are considered to classify multi-source outliers into three types (Type I-III) depending on the correlation among datasets. We design a baseline algorithm, which is an intuitive solution, and an optimal algorithm known as multiple-data-sources oriented outlier detection (MOD) to obtain high-score outliers. In addition, we build the MOD+ method to speed up the outlier detection process. A new density metric combining kNN and RNN is introduced to evaluate the deviation degrees of multi-source outliers. The new outlier are applied to develop three outlier-join operators. MOD and MOD+ are adept at (1) mining outlier information from each one of multi-source datasets and (2) sensing correlative outlier information among these multiple datasets. We implement and evaluate the three outlier detection algorithms by using synthetic and real-world datasets. The experimental results demonstrate that the proposed methods are promising and practical in the context of detecting outliers from multi-source datasets.

Deblurring images with outliers has always been a significantly challenging problem. Previous methods mainly involved complex operations, such as outlier and light streak detection, or sophisticated image priors for blur-kernel estimation, which increased the difficulty of deblurring images. Therefore, we developed a simple, yet efficient, blind deblurring algorithm in this study for handling images with outliers. To eliminate the impact of outliers during the kernel estimation process, we employed a robust Welsch loss function to characterize the data-fidelity term of our model. We observed that this function could extract significant edges successfully. Therefore, the image regularization term was also described by the same function. Using this unified robust function to describe our model can considerably reduce the complexity of the algorithm. Moreover, we derived a flexible weight function from the Welsch function to further improve the efficiency of our algorithm. To finally obtain accurate latent images, we developed a robust non-blind deblurring approach based on this flexible weight function. The experimental results indicate that our approach outperforms state-of-the-art methods in deblurring images with or without outliers. Compared with the method specifically for outliers, the recovery performance of our method can be improved by 12.9% (considering a dataset with impulse noise), and the execution efficiency is about 1.5 times faster.

Outlier detection is a challenging task especially when outliers are defined by rare combinations of multiple variables. In this paper, we develop and evaluate a new method for the detection of outliers in multivariate data that relies on Principal Components Analysis (PCA) and three-sigma limits. The proposed approach employs PCA to effectively perform dimension reduction by regenerating variables, i.e., fitted points from the original observations. The observations lying outside the three-sigma limits are identified as the outliers. This proposed method has been successfully employed to two real life and several artificially generated datasets. The performance of the proposed method is compared with some of the existing methods using different performance evaluation criteria including the percentage of correct classification, precision, recall, and F-measure. The supremacy of the proposed method is confirmed by abovementioned criteria and datasets. The F-measure for the first real life dataset is the highest, i.e., 0.6667 for the proposed method and 0.3333 and 0.4000 for the two existing approaches. Similarly, for the second real dataset, this measure is 0.8000 for the proposed approach and 0.5263 and 0.6315 for the two existing approaches. It is also observed by the simulation experiments that the performance of the proposed approach got better with increasing sample size.

Detection of outliers or anomalies is one of the vital issues in pattern-driven data mining. Outlier detection detects the inconsistent behavior of individual objects. It is an important sector in the data mining field with several different applications such as detecting credit card fraud, hacking discovery and discovering criminal activities. It is necessary to develop tools used to uncover the critical information established in the extensive data. This paper investigated a novel method for detecting cluster outliers in a multidimensional dataset, capable of identifying the clusters and outliers for datasets containing noise. The proposed method can detect the groups and outliers left by the clustering process, like instant irregular sets of clusters (C) and outliers (O), to boost the results. The results obtained after applying the algorithm to the dataset improved in terms of several parameters. For the comparative analysis, the accurate average value and the recall value parameters are computed. The accurate average value is 74.05% of the existing COID algorithm, and our proposed algorithm has 77.21%. The average recall value is 81.19% and 89.51% of the existing and proposed algorithm, which shows that the proposed work efficiency is better than the existing COID algorithm.

Although facial beauty prediction (FBP) has achieved high accuracy on images captured in a constrained environment, it is still a challenging task on face images in-the-wild. Moreover, there is no FBP benchmark dataset that includes images with various ethnic, age, gender properties and unrestricted in terms of face expression and pose. In this work, the issue of FBP in real-life scenario is addressed and a multi-ethnic facial beauty dataset, namely MEBeauty, is introduced. All face images are captured in an unconstrained environment and rated by volunteers with various ethnicity, age and gender in order to avoid any cultural and social biases in beauty perception. Different well-known CNNs with layer-wise transfer learning are performed on the dataset. Moreover, the evaluation of knowledge learning from the face recognition task across FBP is conducted. The expected high number of aberrant and outlier faces is considered and the use of various robust loss functions in order to learn deep regression networks for facial beauty prediction is evaluated. Several FBP frameworks are performed on the proposed dataset and widely-used SCUT-FBP 5500 in order to compare their effectiveness on face images in constrained and unconstrained environments.

The most widely-used data for civilian Space Situational Awareness are the Two-Line Element sets (TLEs) provided by the USSTRATCOM. TLEs play critical roles in space monitoring and analysis activities, and are also used in many applications requiring position information of space objects. However, TLEs suffer from uneven data quality, and it is usually necessary to perform outlier or anomaly detections before utilizing these ephemerides. There are many such procedures available in the published literature, and thresholds are usually obtained through simulations or from experiences, which lacks theoretical soundness and thus is inflexible to use. A new filter based on the Expectation Maximization (EM) algorithm is proposed to ease some restrictions, in which outlier thresholds are determined based on the variances estimated in the polynomial regression and prediction, which makes the new filter theoretically sound and more flexible to use. The new filter can be applied to detect outliers in TLEs of objects in any orbital regions, under various space environments, and enduring different operations such as orbit maneuvers. Multiple experiments are presented to demonstrate the effectiveness of the proposed filter to detect outliers of different magnitudes in TLEs. Simulated outliers in the mean motion and eccentricity with the relative magnitude ≥ 5 × 10 - 3 are almost 100% detected, 74% – 98% of outliers in the inclination with the relative magnitude = 0.01 are detected, and 37% – 68% of those with the relative magnitude 5 × 10 - 3 detected. In addition, sequences can be more accurately identified, which help detect outliers in different sequences, making the proposed filter more applicable to complex situations.

This article presents a real-time outlier detection deep-learning (OD-DL)-based method using a hybridized artificial neural network (ANN) approach. We propose an unsupervised ANN scheme that runs in parallel, a denoising autoencoder (DAE) and a recurrent neural network (RNN). The DAE aims to reconstruct relevant/normal input data, whereas it seeks to ignore outliers; the RNN, with a recursive structure, is used to predict time-series data. As measurements arrive, two tasks are performed: 1) the outlier decision, which is based on a reconstruction error and an energy score criteria from the output difference between the DAE and the RNN; and 2) the training procedure for both DAE and RNN. The proposed OD-DL scheme is specifically targeted to address the outlier problem of the data generated by a Doppler velocity log (DVL) sensor installed on board of an autonomous underwater vehicle (AUV) to enhance the AUV navigation system performance. In particular, the DVL data enter into the OD-DL scheme whose output is fed into an AUV navigation system that runs an error-state Kalman filter that integrates the corrected DVL data with the measurements of an inertial measurement unit and a depth meter. The experimental results show that the AUV navigation system with the OD-DL method outperforms in terms of a more accurate estimated position when compared with the case that there is no outlier detection and with the case of a navigation system using a conventional outlier detection method, or other simpler deep-learning methods.