Search

Projection techniques are often used to visualize high-dimensional data, allowing users to better understand the overall structure of multi-dimensional spaces on a 2D screen. Although many such methods exist, comparably little work has been done on generalizable methods of inverse-projection -- the process of mapping the projected points, or more generally, the projection space back to the original high-dimensional space. In this paper we present NNInv, a deep learning technique with the ability to approximate the inverse of any projection or mapping. NNInv learns to reconstruct high-dimensional data from any arbitrary point on a 2D projection space, giving users the ability to interact with the learned high-dimensional representation in a visual analytics system. We provide an analysis of the parameter space of NNInv, and offer guidance in selecting these parameters. We extend validation of the effectiveness of NNInv through a series of quantitative and qualitative analyses. We then demonstrate the method's utility by applying it to three visualization tasks: interactive instance interpolation, classifier agreement, and gradient visualization.

In state-of-the-art, it has proven that multi-view ensemble learning performs better than classical machine learning algorithms, with the optimized setting of views (subsets of features) usually. Obtaining the appropriate number of views for a given dataset is a complex problem in multi-view ensemble learning (MEL). The finding of the total number of possible views is an NP-hard problem, i.e., equivalent to Bell number. Moreover, the complexity of multi-view learning increases over a higher number of views of the dataset. Therefore, it is highly required to consider a smaller number of views with higher accuracy for optimal performance of MEL. In this work, MEL using Multi-Objective Particle Swarm Optimization (MEL-MOPSO) method has been proposed. The two objectives (number of views of the data and classification accuracy of MEL) have considered where the trade-off between objectives has been performed while searching for an optimal solution using Particle Swarm Optimization (PSO) in the process of multiobjective optimization. The experiments have been done over sixteen high-dimensional datasets using four state-of-art view construction methods. The individual views of the dataset has been utilized to learn through a support vector machine algorithm. The quantitative and non-parametric statistical analyses show that the proposed method has performed effectively and efficiently.

We extend the theory on factor models by incorporating “local” factors into the model. Local factors affect only an unknown subset of the observed variables. This implies a continuum of eigenvalues of the covariance matrix, as is commonly observed in applications. We derive which factors are pervasive enough to be economically important and which factors are pervasive enough to be estimable using the common principal component estimator. We then introduce a new class of estimators to determine the number of those relevant factors. Unlike existing estimators, our estimators use not only the eigenvalues of the covariance matrix, but also its eigenvectors. We find that incorporating partial sums of the eigenvectors into our estimators leads to significant gains in performance in simulations.

This protocol aims to develop, validate, and deploy a prediction model using high dimensional data by both human and machine learning. The applicability is intended for clinical prediction in healthcare providers, including but not limited to those using medical histories from electronic health records. This protocol applies diverse approaches to improve both predictive performance and interpretability while maintaining the generalizability of model evaluation. However, some steps require expensive computational capacity; otherwise, these will take longer time. The key stages consist of designs of data collection and analysis, feature discovery and quality control, and model development, validation, and deployment.

Change point analysis aims to detect structural changes in a data sequence. It has always been an active research area since it was introduced in the 1950s. In modern statistical applications, however, high-throughput data with increasing dimensions are ubiquitous in fields ranging from economics, finance to genetics and engineering. For those problems, the earlier works are typically no longer applicable. As a result, the problem of testing a change point for high dimensional data sequences has been an important yet challenging task. In this paper, we first focus on models for at most one change point, and review recent state-of-art techniques for change point testing of high dimensional mean vectors and compare their theoretical properties. Based on that, we provide a survey of some extensions to general high dimensional parameters beyond mean vectors as well as strategies for testing multiple change points in high dimensions. Finally, we discuss some open problems for possible future research directions.

BACKGROUND: Breast cancer is a major disease causing panic among women worldwide. Since gene mutations are the root cause for cancer development, analyzing gene expressions can give more insights into various phenotype of cancer treatments. Breast Cancer subtype prediction from gene expression data can provide more information for cancer treatment decisions. OBJECTIVE: Gene expressions are complex for analysis due to its high dimensional nature. Machine learning algorithms such as k-Nearest Neighbors, Support Vector Machine (SVM) and Random Forest are used with selection of features for prediction of breast cancer subtypes. Prediction accuracy of the existing methods are affected due to high dimensional nature of gene expressions. The objective of the work is to propose an efficient algorithm for the prediction of breast cancer subtypes from gene expression. METHODS: For subtype prediction, a novel Hubness Weighted Support Vector machine algorithm (HWSVM) using bad hubness score as a weight measure to handle the outliers in the data has been proposed. Based on the various subtypes, features are projected into seven different feature sets and Ensemble based Hubness Aware Weighted Support Vector Machine (HWSVMEns) is implemented for breast cancer subtype prediction. RESULTS: The proposed algorithms have been compared with the classical SVM and other traditional algorithms such as Random Forest, k-Nearest Neighbor algorithms and also with various gene selection methods. CONCLUSIONS: Experimental results show that the proposed HWSVM outperforms other algorithms in terms of accuracy, precision, recall and F1 score due to the hubness weightage scheme and the ensemble approach. The experiments have shown an average accuracy of 92% across various gene expression datasets.

Feature selection technique is a knowledge discovery tool which provides an understanding of the problem through the analysis of the most relevant features. Feature selection aims at building better classifier by listing significant features which also helps in reducing computational overload. Due to existing high throughput technologies and their recent advancements are resulting in high dimensional data due to which feature selection is being treated as handy and mandatory in such datasets. This actually questions the interpretability and stability of traditional feature selection algorithms. The high correlation in features frequently produces multiple equally optimal signatures, which makes traditional feature selection method unstable and thus leading to instability which reduces the confidence of selected features. Stability is the robustness of the feature preferences it produces to perturbation of training samples. Stability indicates the reproducibility power of the feature selection method. High stability of the feature selection algorithm is equally important as the high classification accuracy when evaluating feature selection performance. In this paper, we provide an overview of feature selection techniques and instability of the feature selection algorithm. We also present some of the solutions which can handle the different source of instability.

The use of the Choquet integral in data fusion processes allows for the effective modelling of interactions and dependencies between data features or criteria. Its application requires identification of the defining capacity (also known as fuzzy measure) values. The main limiting factor is the complexity of the underlying parameter learning problem, which grows exponentially in the number of variables. However, in practice we may have expert knowledge regarding which of the subsets of criteria interact with each other, and which groups are independent. In this paper we study hierarchical aggregation processes, architecturally similar to feed-forward neural networks, but which allow for the simplification of the fitting problem both in terms of the number of variables and monotonicity constraints. We note that the Mobius representation lets us identify a number of relationships between the overall fuzzy measure and the data pipeline structure. Included in our findings are simplified fuzzy measures that generalise both k-intolerant and k-interactive capacities.

As one important technique of fuzzy clustering in data mining and pattern recognition, the possibilistic c-means algorithm (PCM) has been widely used in image analysis and knowledge discovery. However, it is difficult for PCM to produce a good result for clustering big data, especially for heterogenous data, since it is initially designed for only small structured dataset. To tackle this problem, the paper proposes a high-order PCM algorithm (HOPCM) for big data clustering by optimizing the objective function in the tensor space. Further, we design a distributed HOPCM method based on MapReduce for very large amounts of heterogeneous data. Finally, we devise a privacy-preserving HOPCM algorithm (PPHOPCM) to protect the private data on cloud by applying the BGV encryption scheme to HOPCM, In PPHOPCM, the functions for updating the membership matrix and clustering centers are approximated as polynomial functions to support the secure computing of the BGV scheme. Experimental results indicate that PPHOPCM can effectively cluster a large number of heterogeneous data using cloud computing without disclosure of private data.

In recent two years, covid-19 diseases is the most harmful diseases in entire world. This disease increase the high mortality rate in several developed countries. Earlier identification of covid -19 symptoms can avoid the over illness or death. However, there are several researchers are introduced different methodology to identification of diseases symptoms. But, identification and classification of covid-19 diseases is the difficult task for every researchers and doctors. In this modern world, machine learning techniques is useful for several medical applications. This study is more focused in applying machine learning classifier model as SVM for classification of diseases. By improve the classification accuracy of the classifier by using hyper parameter optimization technique as modified cuckoo search algorithm. High dimensional data have unrelated, misleading features, which maximize the search space size subsequent in struggle to process data further thus not contributing to the learning practise, So we used a hybrid feature selection technique as mRMR (Minimum Redundancy Maximum Relevance) algorithm. The experiment is conducted by using UCI machine learning repository dataset. The classifier is conducted to classify the two set of classes such as COVID-19, and normal cases. The proposed model performance is analysed by using different parametric metrics, which are explained in result section.

Graph-based methods have achieved impressive performance on semisupervised classification (SSC). Traditional graph-based methods have two main drawbacks. First, the graph is predefined before training a classifier, which does not leverage the interactions between the classifier training and similarity matrix learning. Second, when handling high-dimensional data with noisy or redundant features, the graph constructed in the original input space is actually unsuitable and may lead to poor performance. In this article, we propose an SSC method with novel graph construction (SSC-NGC), in which the similarity matrix is optimized in both label space and an additional subspace to get a better and more robust result than in original data space. Furthermore, to obtain a high-quality subspace, we learn the projection matrix of the additional subspace by preserving the local and global structure of the data. Finally, we intergrade the classifier training, the graph construction, and the subspace learning into a unified framework. With this framework, the classifier parameters, similarity matrix, and projection matrix of subspace are adaptively learned in an iterative scheme to obtain an optimal joint result. We conduct extensive comparative experiments against state-of-the-art methods over multiple real-world data sets. Experimental results demonstrate the superiority of the proposed method over other state-of-the-art algorithms.

Business intelligence and AI services often involve the collection of copious amounts of multidimensional personal data. Since these data usually contain sensitive information of individuals, the direct collection can lead to privacy violations. Local differential privacy (LDP) is currently considered a state-ofthe-art solution for privacy-preserving data collection. However, existing LDP algorithms are not applicable to high-dimensional data; not only because of the increase in computation and communication cost, but also poor data utility.In this paper, we aim at addressing thecurse-of-dimensionalityproblem in LDP-based high-dimensional data collection. Based on the idea of machine learning and data synthesis, we propose DP-Fed-Wae, an efficient privacy-preserving framework for collecting high-dimensional categorical data. With the combination of a generative autoencoder, federated learning, and differential privacy, our framework is capable of privately learning the statistical distributions of local data and generating high utility synthetic data on the server side without revealing users’ private information. We have evaluated the framework in terms of data utility and privacy protection on a number of real-world datasets containing 68–124 classification attributes. We show that our framework outperforms the LDP-based baseline algorithms in capturing joint distributions and correlations of attributes and generating high-utility synthetic data. With a local privacy guarantee ∈ = 8, the machine learning models trained with the synthetic data generated by the baseline algorithm cause an accuracy loss of 10% ~ 30%, whereas the accuracy loss is significantly reduced to less than 3% and at best even less than 1% with our framework. Extensive experimental results demonstrate the capability and efficiency of our framework in synthesizing high-dimensional data while striking a satisfactory utility-privacy balance.

Multinomial logistic regression (MLR) is a useful tool for solving multi-classification problems. The l q , 0 ( q ⩾ 1 ) norm is an ideal regularization term for characterizing group sparsity in multinomial logistic regression and selecting important features in the high dimensional data. However, l q , 0 regularized multinomial logistic regression ( l q , 0 -MLR) is nonconvex, discontinuous, and NP-hard. Thus, most prior studies adopted a continuous approximation of the l q , 0 norm. In this paper, we present a novel l q , 0 -proximal Newton algorithm ( l q , 0 - PNA ) to solve the l q , 0 -MLR. We first define a strong α -stationary point and prove that this point is a local minimizer of l q , 0 -MLR. We then convert such a point into a stationary equation and solve it by l q , 0 - PNA , which is a Newton-type method running on a group sparse subspace with a low computational cost. Furthermore, we establish a locally quadratic convergence of l q , 0 - PNA . Finally, numerical experiments on simulated and real data show the superiority of l q , 0 - PNA in terms of computational time and accuracy, when compared with six state-of-the-art solvers, especially for high dimensional data.

High dimensional data are frequently collected across research fields such as genomics, health sciences, economics, and social sciences. Recently, variable selection in the high dimensional setting has drawn great attention, with many effective methods developed to reduce the dimensionality of the data. However, most of these methods apply only to normally or near normally distributed outcomes in a linear regression model, while few studies focus on variable selection for skewed data. Simulation studies show that ignoring an appropriate transformation for the outcome can lead to biased inferences (e.g., missing important covariates). In this paper, we develop a variable selection procedure for the Box–Cox power transformation model by developing a penalized maximum likelihood estimate and deriving the consistency, oracle property, and asymptotic distribution of this estimate. Simulation studies demonstrate that the proposed method can yield higher sensitivity, while the naive method that without doing transformation can lead to lower sensitivity. We apply the proposed method to a gene expression study.

Most clustering algorithms rely in some fundamental way on a measure of either similarity or distance — either between objects themselves, or between objects and cluster centroids. When the dataset contains mixed attributes, defining a suitable measure can be problematic. This paper presents a general graph-based method for clustering mixed-attribute datasets that does not require any explicit measure of similarity or distance. Empirical results on a range of well-known datasets using a range of evaluation measures show that the method achieves performance competitive with traditional clustering algorithms that require explicit calculation of distance or similarity, as well as with more recently proposed clustering algorithms based on matrix factorization.

It is easy to lead to poor generalization in machine learning tasks using real-world data directly, since such data is usually high-dimensional dimensionality and limited. Through learning the low dimensional representations of high-dimensional data, feature selection can retain useful features for machine learning tasks. Using these useful features effectively trains machine learning models. Hence, it is a challenge for feature selection from high-dimensional data. To address this issue, in this paper, a hybrid approach consisted of an autoencoder and Bayesian methods is proposed for a novel feature selection. Firstly, Bayesian methods are embedded in the proposed autoencoder as a special hidden layer. This of doing is to increase the precision during selecting non-redundant features. Then, the other hidden layers of the autoencoder are used for non-redundant feature selection. Finally, compared with the mainstream approaches for feature selection, the proposed method outperforms them. We find that the way consisted of autoencoders and probabilistic correction methods is more meaningful than that of stacking architectures or adding constraints to autoencoders as regards feature selection. We also demonstrate that stacked autoencoders are more suitable for large-scale feature selection, however, sparse autoencoders are beneficial for a smaller number of feature selection. We indicate that the value of the proposed method provides a theoretical reference to analyze the optimality of feature selection.

Linear discriminant analysis(LDA) classification is widely used in the current field of machine learning and data mining. However, its computational complexity tends to cause high processing latency when implemented on Field Programmable Gate Array (FPGA), especially for high dimensional data. This letter proposes a method to reduce computational complexity, which uses principal component analysis (PCA) to obtain the most representative characteristics of the original data for projection. Instead of the original data, the dimension reduced data obtained by the projection will be used for classification. Moreover, when there is only one main characteristic value, not only the matrix inversion operation is avoided, but also a large number of matrix multiplication operations are simplified. The implementation of the classifier on FPGA is realized using High Level Synthesis(HLS), which can effectively save hardware development time. The results show that the method has an encouraging performance in reducing the execution time of the algorithm on FPGA, while the accuracy of the classifier can also be guaranteed.

Mining useful knowledge from high-dimensional data is a hot research topic. Efficient and effective sample classification and feature selection are challenging tasks due to high dimensionality and small sample size of microarray data. Feature selection is necessary in the process of constructing the model to reduce time and space consumption. Therefore, a feature selection model based on prior knowledge and rough set is proposed. Pathway knowledge is used to select feature subsets, and rough set based on intersection neighborhood is then used to select important feature in each subset, since it can select features without redundancy and deals with numerical features directly. In order to improve the diversity among base classifiers and the efficiency of classification, it is necessary to select part of base classifiers. Classifiers are grouped into several clusters by k-means clustering using the proposed combination distance of Kappa-based diversity and accuracy. The base classifier with the best classification performance in each cluster will be selected to generate the final ensemble model. Experimental results on three Arabidopsis thaliana stress response datasets showed that the proposed method achieved better classification performance than existing ensemble models.

O_LIVisualising data is a vital part of analysis, allowing researchers to find patterns, and assess and communicate the results of statistical modeling. In ecology, visualisation is often challenging when there are many variables (often for different species or other taxonomic groups) and they are not normally distributed (often counts or presence-absence data). Ordination is a common and powerful way to overcome this hurdle by reducing data from many response variables to just two or three, to be easily plotted. C_LIO_LIOrdination is traditionally done using dissimilarity-based methods, most commonly non-metric multidimensional scaling (nMDS). In the last decade however, model-based methods for unconstrained ordination have gained popularity. These are primarily based on latent variable models, with latent variables estimating the underlying, unobserved ecological gradients. C_LIO_LIDespite some major benefits, a major drawback of model-based ordination methods is their speed, as they typically taking much longer to return a result than dissimilarity-based methods, especially for large sample sizes. C_LIO_LIWe introduce copula ordination, a new, scalable model-based approach to unconstrained ordination. This method has all the desirable properties of model-based ordination methods, with the added advantage that it is computationally far more efficient. In particular, simulations show copula ordination is an order of magnitude faster than current model-based methods, and can even be faster than nMDS for large sample sizes, while being able to produce similar ordination plots and trends as these methods. C_LI

Access to high-dimensional data has made the use of machine learning in causal inference more common in recent years. The double/debiased machine learning (DML) estimator for the treatment effect i...

We outline an inherent weakness of tensor factorization models when latent factors are expressed as a function of side information and propose a novel method to mitigate this weakness. We coin our method \textit{Kernel Fried Tensor}(KFT) and present it as a large scale forecasting tool for high dimensional data. Our results show superior performance against \textit{LightGBM} and \textit{Field Aware Factorization Machines}(FFM), two algorithms with proven track records widely used in industrial forecasting. We also develop a variational inference framework for KFT and associate our forecasts with calibrated uncertainty estimates on three large scale datasets. Furthermore, KFT is empirically shown to be robust against uninformative side information in terms of constants and Gaussian noise.

The non-negative matrix factorization (NMF) method is applied in many fields, including pattern recognition, visual analysis, and biomedicine. However, existing NMF methods acquire low dimensional representation directly from high dimensional data, and the low-rank part of the data contains the most useful information. Furthermore, the NMF method is vulnerable to noise interference and is not adequately robust. Therefore, a robust semi non-negative low-rank graph embedding algorithm via the L21 norm (RSNLGEL21) is proposed to address the stated problems. The algorithm considers the effective low-rank structure and geometric information of the original data and constrains the coefficient matrix to be non-negative; however, it does not impose a non-negative constraint on the base matrix. In addition, the L21 norm is introduced into the graph embedding and data reconstruction functions slightly to improve its recognition ability and robustness. Moreover, this paper also provides an iterative formula to the RSNLGEL21 algorithm and the corresponding proof of convergence and to analyze its computational complexity. Experiments on the ORL, FEI, AR, and FERET image datasets shows that the RSNLGEL21 algorithm has certain advantages in clustering performance.

Quantile regression has become a popular alternative to least squares regression for providing a comprehensive description of the response distribution, and robustness against heavy-tailed error distributions. However, the nonsmooth quantile loss poses new challenges to distributed estimation in both computation and theoretical development. To address this challenge, we use a convolution-type smoothing approach and its Taylor expression to transform the nondifferentiable quantile loss function into a convex quadratic loss function, which admits a fast and scalable algorithm to perform optimization under massive and high-dimensional data. The proposed distributed estimators are both computationally and communication efficient. Moreover, only the gradient information is communicated at each iteration. Theoretically, we show that, after a certain number of iterations, the resulting estimator is statistically as efficient as the global estimator without any restriction on the number of machines. Both simulations and data analysis are conducted to illustrate the finite sample performance of the proposed methods.

Rapid advance of sensor technology is facilitating the collection of high-dimensional data streams (HDS). Apart from real-time detection of potential out-of-control (OC) patterns, post-signal fault...

We consider the General Linear Model (GLM) which includes multivariate analysis of variance (MANOVA) and multiple linear regression as special cases. In practice, there are several widely used criteria for GLM: Wilks’ lambda, Bartlett–Nanda–Pillai test, Lawley–Hotelling test, and Roy maximum root test. Limiting distributions for the first three mentioned tests are known under different asymptotic settings. In the present paper, we obtain computable error bounds for the normal approximation of the Lawley–Hotelling statistic when the dimension grows proportionally to the sample size. This result enables us to get more precise calculations of the p-values in applications of multivariate analysis. In practice, more and more often analysts encounter situations where the number of factors is large and comparable with the sample size. Examples include medicine, biology (i.e., DNA microarray studies), and finance.

Covariates in regressions may be linked to each other on a network. Knowledge of the network structure can be incorporated into regularized regression settings via a network penalty term. However, when it is unknown whether the connection signs in the network are positive (connected covariates reinforce each other) or negative (connected covariates repress each other), the connection signs have to be estimated jointly with the covariate coefficients. This can be done with an algorithm iterating a connection sign estimation step and a covariate coefficient estimation step. We develop such an algorithm, called 3CoSE, and show detailed simulation results and an application forecasting event times. The algorithm performs well in a variety of settings. We also briefly describe the publicly available R-package developed for this purpose.

In a variety of diverse applications, it is very desirable to perform a robust analysis of high-dimensional measurements without being harmed by the presence of a possibly larger percentage of outlying measurements. The minimum weighted covariance determinant (MWCD) estimator, based on implicit weights assigned to individual observations, represents a promising and flexible extension of the popular minimum covariance determinant (MCD) estimator of the expectation and scatter matrix of mlutivariate data. In this work, a regularized version of the MWCD denoted as the minimum regularized weighted covariance determinant (MRWCD) estimator is proposed. At the same time, it is accompanied by an outlier detection procedure. The novel MRWCD estimator is able to outperform other available robust estimators in several simulation scenarios, especially in estimating the scatter matrix of contaminated high-dimensional data.

The recent increase in applications of high-dimensional data poses a severe challenge to data analytics, such as supervised classification, particularly for online applications. To tackle this challenge, efficient and effective methods for feature extraction are critical to the performance of classification analysis. The objective of this work is to develop a new supervised feature extraction method for high-dimensional data. It is achieved by developing a clustered discriminant regression (CDR) to extract informative and discriminant features for high-dimensional data. In CDR, the variables are clustered into different groups or subspaces, within which feature extraction is performed separately. The CDR algorithm, which is a greedy approach, is implemented to obtain the solution toward optimal feature extraction. One numerical study is performed to demonstrate the performance of the proposed method for variable selection. Three case studies using healthcare and additive manufacturing data sets are accomplished to demonstrate the classification performance of the proposed methods for real-world applications. The results clearly show that the proposed method is superior over the existing method for high-dimensional data feature extraction. Note to Practitioners —This article forwards a new supervised feature extraction method termed clustered discriminant regression. This method is highly effective for classification analysis of high-dimensional data, such as images or videos, where the number of variables is much larger than the number of samples. In our case studies on healthcare and additive manufacturing, the performance of classification analysis based on our method is superior over the existing feature extraction methods, which is confirmed by using various popular classification algorithms. For image classification, our method with elaborately selected classification algorithms can outperform a convolutional neural network. In addition, the computation efficiency of the proposed method is also promising, which enables its online applications, such as advanced manufacturing process monitoring and control.

Most approaches to robot visual localization rely on local, global visual or semantic information as observation. In this paper, the combination of global visual and semantic information is used as landmark in the observation model of Bayesian filters. Introducing the improved Gaussian Process into observation models with visual information, The GP-Localize algorithm is extended to high dimensional data, which means that all the historical data with spatiotemporal correlation to achieve constant time and memory for persistent outdoor robot localization can be considered by the Bayesian filters. The other contribution of this paper is we combine the all above parts into a system for robot visual localization and apply it to two real-world outdoor datasets including unmanned ground vehicle (UGV) and unmanned aerial vehicle (UAV). According to the experimental results, it's no difficult to find there is a higher accuracy while using global vision and semantic results rather than just using single feature. By using the combined features and improved Gaussian process approximation method in Bayes filters, our system is more robust and practical than existing localization systems such as ORB-SLAM.

Hyperspectral images (HSI) classification has been a research hotspot in the remote sensing field. Deep learning methods such as Cube-CNN have been applied to address the HSI classification problem. However, mainstream frameworks exist a performance gap to train Cube-CNN, since they are not designed for processing high dimensional data like HSI. To close this gap, we propose a Multi-GPU Training Framework (MTFC) for Cube-CNN-based HSI classification. We first design a Parallel Neighbor Pixel Extraction (PNPE) algorithm for efficiently generating 3-dimensional cube samples from raw data. Then, to fully exploit massive GPU parallelism and realize unique characteristics of HSI and Cube-CNN, we employ optimizations in MTFC such as task division, fine-grained mapping between tasks and GPU thread blocks, shared memory usage reduction, etc. Finally, to further improve training speed, we take advantage of CUDA streams and multiple GPUs to train a mini-batches of data samples simultaneously. An extensive set of experiments highlights that MTFC constantly outperforms the two baselines Caffe and Theano for all measured metrics across all system configurations, while offering the same level of classification accuracy. The speedup is up to 3.6x when using a single GPU and MTFC can achieve a rough linear scaling on multiple GPUs.

In linear regression of Y on X(2 Rp) with parameters (2 Rp+1); statistical inference is unreliable when observations are obtained from gross-error model, F;G = (1??)F +G; instead of the assumed probability F;G is gross-error probability, 0 < < 1: When G is unit mass at (x; y); Residual's Inuence Index, RINFIN(x; y; ; ), measures the dierence in small x-perturbations of L2-residual, r(x; y); for model F and for F;G via r's x-partial derivatives. Asymptotic properties are presented for sample RINFIN that is successful in extracting indications for inuential and bad leverage cases in microarray data and simulated, high dimensional data. Its performance improves as p increases and can also be used in multiple response linear regression. RINFIN's advantage is that, whereas in inuence functions of L2-regression coecients each x-coordinate and r(x; y) appear in a sum as product with moderate size when (x; y) is bad leverage case and masking makes r(x; y) nearly vanish, RINFIN's x-partial derivatives convert the product in sum allowing for unmasking.

In disease screening, a biomarker combination developed by combining multiple markers tends to have a higher sensitivity than an individual marker. Parametric methods for marker combination rely on the inverse of covariance matrices, which is often a non-trivial problem for high-dimensional data generated by modern high-throughput technologies. Additionally, another common problem in disease diagnosis is the existence of limit of detection (LOD) for an instrument – that is, when a biomarker's value falls below the limit, it cannot be observed and is assigned an NA value. To handle these two challenges in combining high-dimensional biomarkers with the presence of LOD, we propose a resample-replace lasso procedure. We first impute the values below LOD and then use the graphical lasso method to estimate the means and precision matrices for the high-dimensional biomarkers. The simulation results show that our method outperforms alternative methods such as either substitute NA values with LOD values or remove observations that have NA values. A real case analysis on a protein profiling study of glioblastoma patients on their survival status indicates that the biomarker combination obtained through the proposed method is more accurate in distinguishing between two groups.

In high-dimensional data analysis, the curse of dimensionality reasons that points tend to be far away from the center of the distribution and on the edge of high-dimensional space. Contrary to thi...

High-dimensional data have commonly emerged in diverse fields, such as economics, finance, genetics, medicine, machine learning, and so on. In this paper, we consider the sparse quantile regression problem of high-dimensional data with heavy-tailed noise, especially when the number of regressors is much larger than the sample size. We bring the spirit of Lp-norm support vector regression into quantile regression and propose a robust Lp-norm support vector quantile regression for high-dimensional data with heavy-tailed noise. The proposed method achieves robustness against heavy-tailed noise due to its use of the pinball loss function. Furthermore, Lp-norm support vector quantile regression ensures that the most representative variables are selected automatically by using a sparse parameter. We use a simulation study to test the variable selection performance of Lp-norm support vector quantile regression, where the number of explanatory variables greatly exceeds the sample size. The simulation study confirms that Lp-norm support vector quantile regression is not only robust against heavy-tailed noise but also selects representative variables. We further apply the proposed method to solve the variable selection problem of index construction, which also confirms the robustness and sparseness of Lp-norm support vector quantile regression.

Composite quantile regression (CQR) is a robust and efficient estimation method. This paper studies CQR method for single-index models with ultra-high-dimensional data. We propose a penalized CQR estimator for single-index models and combine the debiasing technique with the CQR method to construct an estimator that is asymptotically normal, which enables the construction of valid confidence intervals and hypothesis testing. Both simulations and data analysis are conducted to illustrate the finite sample performance of the proposed methods.

For high-dimensional data with a large number of redundant features, existing feature selection algorithms still have the problem of “curse of dimensionality.” In view of this, the paper studies a new two-phase evolutionary feature selection algorithm, called clustering-guided integer brain storm optimization algorithm (IBSO-C). In the first phase, an importance-guided feature clustering method is proposed to group similar features, so that the search space in the second phase can be reduced obviously. The second phase applies oneself to finding optimal feature subset by using an improved integer brain storm optimization. Moreover, a new encoding strategy and a time-varying integer update method for individuals are proposed to improve the search performance of brain storm optimization in the second phase. Since the number of feature clusters is far smaller than the size of original features, IBSO-C can find an optimal feature subset fast. Compared with several existing algorithms on some real-world datasets, experimental results show that IBSO-C can find feature subset with high classification accuracy at less computation cost.

We develop an envelope model for joint mean and covariance regression in the large p, small n setting. In contrast to existing envelope methods, which improve mean estimates by incorporating estimates of the covariance structure, we focus on identifying covariance heterogeneity by incorporating information about mean-level differences. We use a Monte Carlo EM algorithm to identify a low-dimensional subspace that explains differences in both means and covariances as a function of covariates, and then use MCMC to estimate the posterior uncertainty conditional on the inferred low-dimensional subspace. We demonstrate the utility of our model on a motivating application on the metabolomics of aging. We also provide R code that can be used to develop and test other generalizations of the response envelope model.

This article is concerned with the development of fuzzy models realized with the aid of genetic programming (GP). The proposed architecture employs GP to form fuzzy logic expressions involving logic operators and information granules (fuzzy sets) located in the input space, used to predict information granules located in the output space. We propose an architecture realizing logic processing, with the structural optimization of the model accomplished by a multitree genetic programming and the parametric optimization completed by gradient-based learning. The granulation of information used in this architecture is developed using the Fuzzy C-means clustering algorithm. The novelty of this study is two-fold: 1) it comes with the flexibility of the logic-oriented structure of fuzzy models, and 2) our architecture is designed to handle high-dimensional data by alleviating the detrimental effect of distance concentration hampering the effectiveness of standard Takagi–Sugeno–Kang fuzzy rule-based models. The article is illustrated through some experiments that provide a detailed insight into the performance of the fuzzy models. A comprehensive comparative analysis is also covered.

As a principled dimension reduction technique, factor models have been widely adopted in applications. However, conducting a proper Bayesian factor analysis can be subtle in high-dimensional settings since it requires both a careful prescription of the prior distribution and a suitable computational strategy. We analyze issues of posterior inconsistency and sensitivity under different priors for high-dimensional sparse normal factor models, and show why adopting the n-orthonormal factor assumption can resolve these issues and lead to a more robust and efficient Bayesian analysis. We also provide an efficient Gibbs sampler to conduct the required computation, and show that it can be orders of magnitude more efficient than compared existing algorithms.

Many statistical methodologies for high-dimensional data assume the population is normal. Although a few multivariate normality tests have been proposed, to the best of our knowledge, none of them ...

Visualization of high‐dimensional data is counter‐intuitive using conventional graphs. Parallel coordinates are proposed as an alternative to explore multivariate data more effectively. Ho...