Search

Abstract The rapid advancement of spatial transcriptomics technologies has revolutionized our understanding of cell heterogeneity and intricate spatial structures within tissues and organs. However, the high dimensionality and noise in spatial transcriptomic data present significant challenges for downstream data analyses. Here, we develop GraphPCA, an interpretable and quasi-linear dimension reduction algorithm that leverages the strengths of graphical regularization and principal component analysis. Comprehensive evaluations on simulated and multi-resolution spatial transcriptomic datasets generated from various platforms demonstrate the capacity of GraphPCA to enhance downstream analysis tasks including spatial domain detection, denoising, and trajectory inference compared to other state-of-the-art methods.

Abstract The availability of many variables with predictive power makes their selection in a regression context difficult. This study considers robust and understandable low-dimensional estimators as building blocks to improve overall predictive power by optimally combining these building blocks. Our new algorithm is based on generalized cross-validation and builds a predictive model step-by-step from a simple mean to more complex predictive combinations. Empirical applications to annual financial returns and actuarial telematics data show its usefulness in the financial and insurance industries.

Abstract Assessment of similarity in species composition or abundance across sampled locations is a common goal in multi‐species monitoring programs. Existing ordination techniques provide a framework for clustering sample locations based on species composition by projecting high‐dimensional community data into a low‐dimensional, latent ecological gradient representing species composition. However, these techniques require specification of the number of distinct ecological communities present in the latent space, which can be difficult to determine in advance. We develop an ordination model capable of simultaneous clustering and ordination that allows for estimation of the number of clusters present in the latent ecological gradient. This model draws latent coordinates for each sample location from a Dirichlet process mixture model, affording researchers with probabilistic statements about the number of clusters present in the latent ecological gradient. The model is compared to existing methods for simultaneous clustering and ordination via simulation and applied to two empirical datasets; JAGS code to fit the proposed model is provided in an appendix. The first dataset concerns presence‐absence records of fish in the Doubs river in eastern France and the second dataset describes presence‐absence records of plant species in Craters of the Moon National Monument and Preserve (CRMO) in Idaho, USA. Results from both analyses align with existing ecological gradients at each location. Development of the Dirichlet process ordination model provides wildlife managers with data‐driven inferences about the number of distinct communities present across monitored locations, allowing for more cost‐effective monitoring and reliable decision‐making for conservation management.

This research is important to understand the behavior of parents of Cerebral Palsy (CP) children by reducing the complexity in understanding knowledge and economic dimensions, thus providing a structured approach to identifying factors that influence the psychology of children with CP. This study aimed to identify and reduce the dimensions that influence the behavior of parents of children with CFA.This study aimed to reduce the socio-economic dimensions and knowledge of the behavior of parents of Cerebral Palsy (CP) children using CFA. A quantitative approach with a cross-sectional design was used in this study to comprehensively examine the behavior of parents of children with CP. Purposive sampling was used to select 200 participants from various backgrounds. The instrument used was a 28-item questionnaire which was distributed online to collect data. The questionnaire has gone through rigorous testing, including CFA, determinant KMO test, Bartlett test PCA, and grouping of variables based on identified factors.Research methods , a quantitative approach with cross-sectional design, and purposive sampling were used to select 200 parents of children with CP from various backgrounds. The instrument in the form of a questionnaire with 28 statement items was distributed online. Data analysis was carried out using CFA, including the determinant test, Kaiser-Meyer-Olkin (KMO) test, Bartlett test, Principal Components Analysis (PCA), and grouping variables based on factors. Research in Lampung shows that the number of boys with CP is slightly higher, with the majority aged 1-5 years. Factor analysis identified three main dimensions: parental knowledge about CP, family economic situation, and parental behavior. Better knowledge and a stable economic situation are positively correlated with better parental behavior in caring for children with CP. Many parents experience high levels of stress due to the physical, emotional, and financial burden of caring for a child with CP. The research conclusion shows that the 28 statement items were successfully reduced to three main dimensions (knowledge, economics, and behavior) which have a significant relationship in the care of children with CP. These findings suggest the need for interventions to increase parental knowledge about CP and family economic stability to increase parental participation in child care.Findings the sex distribution of children with cerebral palsy in Lampung showed a slight excess in the number of boys, with the majority of children being in the age range of 1-5 years. Factor analysis identified three main dimensions: parental knowledge of CP, family economic situation, and parental behavior, where better knowledge and stable economic situation positively correlated with better parental behavior in caring for children with CP. Many parents experience high levels of stress in caring for a child with CP because of the physical, emotional, and financial burden that the condition brings. In conclusion, the 28 items of the statement are reduced to 3 dimensions or factors, including knowledge, economics, and behavior. Each dimension has a relationship with each other significantly to the behavior of parents of CP children.

Abstract Motivation Chemical space embedding methods are widely utilized in various research settings for dimensional reduction, clustering and effective visualization. The maps generated by the embedding process can provide valuable insight to medicinal chemists in terms of the relationships between structural, physicochemical and biological properties of compounds. However, these maps are known to be difficult to interpret, and the ‘‘landscape’’ on the map is prone to ‘‘rearrangement’’ when embedding different sets of compounds. Results In this study we present the Hilbert-Curve Assisted Space Embedding (HCASE) method which was designed to create maps by organizing structures according to a logic familiar to medicinal chemists. First, a chemical space is created with the help of a set of ‘‘reference scaffolds’’. These scaffolds are sorted according to the medicinal chemistry inspired Scaffold-Key algorithm found in prior art. Next, the ordered scaffolds are mapped to a line which is folded into a higher dimensional (here: 2D) space. The intricately folded line is referred to as a pseudo-Hilbert-Curve. The embedding of a compound happens by locating its most similar reference scaffold in the pseudo-Hilbert-Curve and assuming the respective position. Through a series of experiments, we demonstrate the properties of the maps generated by the HCASE method. Subjects of embeddings were compounds of the DrugBank and CANVASS libraries, and the chemical spaces were defined by scaffolds extracted from the ChEMBL database. Scientific contribution The novelty of HCASE method lies in generating robust and intuitive chemical space embeddings that are reflective of a medicinal chemist’s reasoning, and the precedential use of space filling (Hilbert) curve in the process. Availability https://github.com/ncats/hcase Graphical Abstract

This article explores the application of Variational AutoEncoders (VAEs) to insurance data. Previous research has demonstrated the successful implementation of generative models, especially VAEs, across various domains, such as image recognition, text classification, and recommender systems. However, their application to insurance data, particularly to heterogeneous insurance portfolios with mixed continuous and discrete attributes, remains unexplored. This study introduces novel insights into utilising VAEs for unsupervised learning tasks in actuarial science, including dimension reduction and synthetic data generation. We propose a VAE model with a quantile transformation for continuous (latent) variables, a reconstruction loss that combines categorical cross-entropy and mean squared error, and a KL divergence-based regularisation term. Our VAE model’s architecture circumvents the need to pre-train and fine-tune a neural network to encode categorical variables into n-dimensional representative vectors within a continuous vector space of dimension Rn. We assess our VAE’s ability to reconstruct complex insurance data and generate synthetic insurance policies using a motor portfolio. Our experimental results and analysis highlight the potential of VAEs in addressing challenges related to data availability in the insurance industry.

Sparse canonical correlation analysis (sCCA) has been a useful approach for integrating different high-dimensional datasets by finding a subset of correlated features that explain the most correlation in the data. In the context of microbiome studies, investigators are always interested in knowing how the microbiome interacts with the host at different molecular levels such as genome, methylol, transcriptome, metabolome and proteome. sCCA provides a simple approach for exploiting the correlation structure among multiple omics data and finding a set of correlated omics features, which could contribute to understanding the host-microbiome interaction. However, existing sCCA methods do not address compositionality, and its application to microbiome data is thus not optimal. This paper proposes a new sCCA framework for integrating microbiome data with other high-dimensional omics data, accounting for the compositional nature of microbiome sequencing data. It also allows integrating prior structure information such as the grouping structure among bacterial taxa by imposing a “soft” constraint on the coefficients through varying penalization strength. As a result, the method provides significant improvement when the structure is informative while maintaining robustness against a misspecified structure. Through extensive simulation studies and real data analysis, we demonstrate the superiority of the proposed framework over the state-of-the-art approaches.

A signal decomposition is presented that disentangles the deterministic and stochastic components of a multivariate time series. The dynamical component analysis (DyCA) algorithm is based on the assumption that an unknown set of ordinary differential equations (ODEs) describes the dynamics of the deterministic part of the signal. The algorithm is thoroughly derived and accompanied by a link to the GitHub repository containing the algorithm. The method was applied to both simulated and real-world data sets and compared to the results of principal component analysis (PCA), independent component analysis (ICA), and dynamic mode decomposition (DMD). The results demonstrate that DyCA is capable of separating the deterministic and stochastic components of the signal. Furthermore, the algorithm is able to estimate the number of linear and non-linear differential equations and to extract the corresponding amplitudes. The results demonstrate that DyCA is an effective tool for signal decomposition and dimension reduction of multivariate time series. In this regard, DyCA outperforms PCA and ICA and is on par or slightly superior to the DMD algorithm in terms of performance.

This paper delves into an in-depth exploration of speaker recognition methodologies, with a primary focus on three pivotal approaches: feature-level fusion, dimension reduction employing principal component analysis (PCA) and independent component analysis (ICA), and feature optimization through a genetic algorithm (GA) and the marine predator algorithm (MPA). This study conducts comprehensive experiments across diverse speech datasets characterized by varying noise levels and speaker counts. Impressively, the research yields exceptional results across different datasets and classifiers. For instance, on the TIMIT babble noise dataset (120 speakers), feature fusion achieves a remarkable speaker identification accuracy of 92.7%, while various feature optimization techniques combined with K nearest neighbor (KNN) and linear discriminant (LD) classifiers result in a speaker verification equal error rate (SV EER) of 0.7%. Notably, this study achieves a speaker identification accuracy of 93.5% and SV EER of 0.13% on the TIMIT babble noise dataset (630 speakers) using a KNN classifier with feature optimization. On the TIMIT white noise dataset (120 and 630 speakers), speaker identification accuracies of 93.3% and 83.5%, along with SV EER values of 0.58% and 0.13%, respectively, were attained utilizing PCA dimension reduction and feature optimization techniques (PCA-MPA) with KNN classifiers. Furthermore, on the voxceleb1 dataset, PCA-MPA feature optimization with KNN classifiers achieves a speaker identification accuracy of 95.2% and an SV EER of 1.8%. These findings underscore the significant enhancement in computational speed and speaker recognition performance facilitated by feature optimization strategies.

Abstract Background Joint analysis of multiple phenotypes in studies of biological systems such as Genome-Wide Association Studies is critical to revealing the functional interactions between various traits and genetic variants, but growth of data in dimensionality has become a very challenging problem in the widespread use of joint analysis. To handle the excessiveness of variables, we consider the sliced inverse regression (SIR) method. Specifically, we propose a novel SIR-based association test that is robust and powerful in testing the association between multiple predictors and multiple outcomes. Results We conduct simulation studies in both low- and high-dimensional settings with various numbers of Single-Nucleotide Polymorphisms and consider the correlation structure of traits. Simulation results show that the proposed method outperforms the existing methods. We also successfully apply our method to the genetic association study of ADNI dataset. Both the simulation studies and real data analysis show that the SIR-based association test is valid and achieves a higher efficiency compared with its competitors. Conclusion Several scenarios with low- and high-dimensional responses and genotypes are considered in this paper. Our SIR-based method controls the estimated type I error at the pre-specified level $$\alpha $$ α .

Financial markets are complex and dynamic, and accurately predicting market trends is crucial for traders and financial analysts. Ichimoku-based features have gained significant attention in financial market analysis due to their ability to capture essential market signals and patterns. This significant compression retains essential patterns related to trends, support/resistance levels, and trading signals. The reduced dimensionality improves computational efficiency and could allow for more accurate predictive modeling by traders. However, real-world testing is needed because compressing data risks losing useful nuances. In this study, we utilize an autoencoder for the dimensionality reduction of Ichimoku-based features in financial market analysis. The autoencoder, a neural network architecture, compresses high-dimensional data into a lower-dimensional representation by learning important features and patterns. The experiments conducted on a Euro/Dollar market dataset spanning 1990, comprising 16 columns with Ichimoku features, reveal the remarkable reduction of dataset size from 2,269,500 to 756,375, equivalent to a decrease of 66.67 %. These results highlight the efficiency of the proposed approach in reducing the dimensionality of financial market data, suggesting its potential as a valuable tool for traders and financial analysts to predict market trends and make informed decisions in the financial markets.

The progressive evolution of the spatial and temporal resolutions of Earth observation satellites has brought multiple benefits to scientific research. The increasing volume of data with higher frequencies and spatial resolutions offers precise and timely information, making it an invaluable tool for environmental analysis and enhanced decision-making. However, this presents a formidable challenge for large-scale environmental analyses and socioeconomic applications based on spatial time series, often compelling researchers to resort to lower-resolution imagery, which can introduce uncertainty and impact results. In response to this, our key contribution is a novel machine learning approach for dense geospatial time series rooted in superpixel segmentation, which serves as a preliminary step in mitigating the high dimensionality of data in large-scale applications. This approach, while effectively reducing dimensionality, preserves valuable information to the maximum extent, thereby substantially enhancing data accuracy and subsequent environmental analyses. This method was empirically applied within the context of a comprehensive case study encompassing the 2002–2022 period with 8-d-frequency-normalized difference vegetation index data at 250-m resolution in an area spanning 43,470 km2. The efficacy of this methodology was assessed through a comparative analysis, comparing our results with those derived from 1000-m-resolution satellite data and an existing superpixel algorithm for time series data. An evaluation of the time-series deviations revealed that using coarser-resolution pixels introduced an error that exceeded that of the proposed algorithm by 25 % and that the proposed methodology outperformed other algorithms by more than 9 %. Notably, this methodological innovation concurrently facilitates the aggregation of pixels sharing similar land-cover classifications, thus mitigating subpixel heterogeneity within the dataset. Further, the proposed methodology, which is used as a preprocessing step, improves the clustering of pixels according to their time series and can enhance large-scale environmental analyses across a wide range of applications.

Abstract Group decision-making and consensus modeling have always been important research topics. With the widespread use of the Internet, group decisions can be made online, in which a large number of decision-makers participate. Most of the existing studies on large-scale group decision-making consider 20–50 decision-makers. Therefore, there is a need for a framework that focuses on situations where thousands of decision-makers exist. As dimension reduction is one of the five primary challenges in large-scale group decision-making, in this study, after reviewing the existing approaches, a new model is presented using a statistical approach along with complex network analysis techniques. The opinions are generalized first, and then the network of opinions is built. This new method reduces the dimensions of the problem by considering a hierarchy of opinions. Different scenarios were designed for the evaluation. The results show that the effect of this generalization on dimension reduction depends on the parameters of the problem. We have shown that in a group decision scenario with 3000 decision-makers and 6 alternatives, 99% of the data was reduced. As dimension reduction is the main focus of the current research, the effect of consistency on the diversity of opinions has also been investigated, and the results show that opinion consistency affects opinion generalization, which in turn affects dimension reduction. In addition, in the performed simulations, three types of functions were used to calculate similarity. The aim was to determine the best similarity function for the decision problems whose purpose was to rank the available alternatives. The results show that Euclidean similarity is a strict criterion compared with Cosine similarity.

This study aims to evaluate the relationships between pre-treatment solution, variety, drying characteristics, and raisin quality in raisin production, using multivariate analysis methods. The study was conducted on raisins obtained by dipping Bineteti and Zeyti local seed grape varieties in 13 different pre-treatment solutions which were obtained by mixing potassium carbonate and sodium bicarbonate with olive oil, hazelnut oil, and sesame oil at different concentrations. The dipped grapes were dried in the sun on a concrete drying platform. In the study, data of 15 numerical variables related to drying characteristics and raisin quality were reduced to four principal components (PC1, PC2, PC3 and PC4) using the principal component analysis (PCA), and their score values were numerically obtained. Then, two grape varieties, 13 pre-treatment solutions, and the four principal components were analyzed by non-linear principal component analysis (NLPCA). In addition, a cluster analysis was performed to determine the prominent pre-treatment solutions in terms of drying characteristics and raisin quality. It was determined that the pre-treatment solutions were effective on L*, a*, b*, chroma (C*), hue (h°), a/b values, antioxidant activity, total phenolic content, and drying time constituting PC1. It was remarkable that the colour parameters in prominent clusters in the cluster analysis also form PC1 in PCA analysis. The best pre-treatment solutions were found to be the "5% K2CO3 + 1% olive oil" solution for the Bineteti variety and the "5% K2CO3 + 2% hazelnut oil" solution for the Zeyti variety. It was determined that the pre-treatment solutions recommended for the varieties increased raisin quality and shortened the drying time, and had positive effects on the total phenolic content and antioxidant activity.

Abstract Streaming adaptations of manifold learning based dimensionality reduction methods, such as Isomap, are based on the assumption that a small initial batch of observations is enough for exact learning of the manifold, while remaining streaming data instances can be cheaply mapped to this manifold. However, there are no theoretical results to show that this core assumption is valid. Moreover, such methods typically assume that the underlying data distribution is stationary and are not equipped to detect, or handle, sudden changes or gradual drifts in the distribution that may occur when the data is streaming. We present theoretical results to show that the quality of a manifold asymptotically converges as the size of data increases. We then show that a Gaussian Process Regression (GPR) model, that uses a manifold-specific kernel function and is trained on an initial batch of sufficient size, can closely approximate the state-of-art streaming Isomap algorithms, and the predictive variance obtained from the GPR prediction can be employed as an effective detector of changes in the underlying data distribution. Results on several synthetic and real data sets show that the resulting algorithm can effectively learn lower dimensional representation of high dimensional data in a streaming setting, while identifying shifts in the generative distribution. For instance, key findings on a Gas sensor array data set show that our method can detect changes in the underlying data stream, triggered due to real-world factors, such as introduction of a new gas in the system, while efficiently mapping data on a low-dimensional manifold.

Abstract EDXRF spectrometry is a well-established and often-used analytical technique in examining materials from which cultural heritage objects are made. The analytical results are traditionally subjected to additional multivariate analysis for archaeometry studies to reduce the initial data’s dimensionality based on informative features. Nowadays, artificial intelligence (AI) techniques are used more for this purpose. Different soft computing techniques are used to improve speed and accuracy. Choosing the most suitable AI method can increase the sustainability of the analytical process and postprocessing activities. An autoencoder neural network has been designed and used as a dimension reduction tool of initial $$40 \times 2048$$ 40 × 2048 data collected in the raw EDXRF spectra, containing information about the selected points’ elemental composition on the canvas paintings’ surface. The autoencoder network design enables the best possible reconstruction of the original EDXRF spectrum and the most informative feature extraction, which has been used for dimension reduction. Such configuration allows for efficient classification algorithms and their performances. The autoencoder neural network approach is more sustainable, especially in processing time consumption and experts’ manual work.

Abstract Background Unsupervised clustering and outlier detection are important in medical research to understand the distributional composition of a collective of patients. A number of clustering methods exist, also for high-dimensional data after dimension reduction. Clustering and outlier detection may, however, become less robust or contradictory if multiple high-dimensional data sets per patient exist. Such a scenario is given when the focus is on 3-D data of multiple organs per patient, and a high-dimensional feature matrix per organ is extracted. Methods We use principal component analysis (PCA), t-distributed stochastic neighbor embedding (t-SNE) and multiple co-inertia analysis (MCIA) combined with bagplots to study the distribution of multi-organ 3-D data taken by computed tomography scans. After point-set registration of multiple organs from two public data sets, multiple hundred shape features are extracted per organ. While PCA and t-SNE can only be applied to each organ individually, MCIA can project the data of all organs into the same low-dimensional space. Results MCIA is the only approach, here, with which data of all organs can be projected into the same low-dimensional space. We studied how frequently (i.e., by how many organs) a patient was classified to belong to the inner or outer 50% of the population, or as an outlier. Outliers could only be detected with MCIA and PCA. MCIA and t-SNE were more robust in judging the distributional location of a patient in contrast to PCA. Conclusions MCIA is more appropriate and robust in judging the distributional location of a patient in the case of multiple high-dimensional data sets per patient. It is still recommendable to apply PCA or t-SNE in parallel to MCIA to study the location of individual organs.

Nowcasts are closely related to big data. Currently, the most popular nowcast model-building approach is to use the factor bridge equation (BE) model or factor mixed data sampling (MIDAS) model, where the factors are extracted from large datasets using the principal component analysis (PCA) method. However, the PCA method only suits datasets approximating a hyperplane, while real-world datasets are sometimes unsuitable. This study proposes a kernel-based dimension reduction method. It is a natural extension of the PCA method and is called KTPCA. The KTPCA method can reduce the dimensionality of datasets without approximating a hyperplane. The dimension reduction performance of the iterative KTPCA method was superior to that of the PCA, Sparse PCA (SPCA), Randomized SPCA (RSPCA), and Robust SPCA (ROBSPCA) methods. In addition, the PCA and SPCA methods are competitive regarding dimension reduction performance. This study also proposes a nowcasting procedure using the iterative KTPCA method. This procedure can be applied to build nowcast models and update the forecasts of a target variable at a low frequency under real-time data flow of the original variables at a higher frequency.

Surface electromyographic signals (sEMG) usually have high-dimensional properties, and direct processing of these data consumes significant computational resources. Dimensionality reduction processing can reduce the dimension of the data and improve the real-time performance and response speed. This is especially important for application scenarios such as prosthetic control and rehabilitation training where rapid feedback is required. This paper proposes a feature fusion dimension reduction method for sEMG signals. This method is constructed based on the unique correlation between the features of sEMG. To test the performance of the new dimension reduction method, the sEMG signals from five leg movements were collected from eight subjects and the classification of the feature matrix before and after dimension reduction was tested by six classifiers. The results show that the feature matrix after fusion dimension reduction has excellent classification performance in the subsequent classification tasks. It produces up to 98.3% accuracy. And the highest comprehensive evaluation index can reach 0.9958. This paper also compares the new method with three commonly used dimensionality reduction methods. The results show that the performance of the new method is not only optimal but also extremely stable. Because its classification performance will not be lower than other dimensionality reduction methods due to the change of classifiers. This confirms that the new method has a higher utility value in sEMG signals processing compared to other dimension reduction methods.

Infantile spasms are a severe epileptic syndrome characterized by short muscular contractions lasting from 0.5 to 2 seconds. They are often misdiagnosed due to their atypical presentation, and treatment is frequently delayed, leading to stagnation or regression in psychomotor development and significant cognitive and motor sequelae. One promising approach to addressing this issue is the use of markerless computer vision techniques. In this paper, we introduce a novel approach for recognizing infantile spasms based exclusively on video data. We utilize an expanded 3D neural network pre-trained on an extensive human action recognition dataset called Kinetics. By employing this model, we extract features from short segments of varying sizes sampled from seizure videos, which allows us to effectively capture the spatio-temporal characteristics of infantile spasms. We then apply multiple classifiers to perform binary classification on these extracted features. The best system achieved an average area under the ROC curve of $0.813\pm 0.058$ for a 3-second window.

The Fiedler vector is the eigenvector associated with the algebraic connectivity of the graph Laplacian. It is central to graph analysis as it provides substantial information to learn the latent structure of a graph. In real-world applications, however, the data may be subject to heavy-tailed noise and outliers which deteriorate the structure of the Fiedler vector estimate and lead to a breakdown of popular methods. Thus, we propose a Robust Regularized Locality Preserving Indexing (RRLPI) Fiedler vector estimation method that approximates the nonlinear manifold structure of the Laplace Beltrami operator while minimizing the impact of outliers. To achieve this aim, an analysis of the effects of two fundamental outlier types on the eigen-decomposition of block affinity matrices is conducted. Then, an error model is formulated based on which the RRLPI method is developed. It includes an unsupervised regularization parameter selection algorithm that leverages the geometric structure of the projection space. The performance is benchmarked against existing methods in terms of detection probability, partitioning quality, image segmentation capability, robustness and computation time using a large variety of synthetic and real data experiments.

Federated Learning, a model-sharing method, and Data Collaboration, a non-model-sharing method, are recognized as data analysis methods for distributed data. In Federated Learning, clients send only the parameters of a machine learning model to the central server. In Data Collaboration, clients send data that has undergone irreversibly transformed through dimensionality reduction to the central server. Both methods are designed with privacy concerns, but privacy is not guaranteed. Differential Privacy, a theoretical and quantitative privacy criterion, has been applied to Federated Learning to achieve rigorous privacy preservation. In this paper, we introduce a novel method using PCA (Principal Component Analysis) that finds low-rank approximation of a matrix preserving the variance, aiming to apply Differential Privacy to Data Collaboration. Experimental evaluation using the proposed method show that differentially-private Data Collaboration achieves comparable performance to differentially-private Federated Learning.

Abstract The relationships between place (e.g., neighborhood) and HIV are commonly investigated. As measurements of place are multivariate, most studies apply some dimension reduction, resulting in one variable (or a small number of variables), which is then used to characterize place. Typical dimension reduction methods seek to capture the most variance of the raw items, resulting in a type of summary variable we call “disadvantage score”. We propose to add a different type of summary variable, the “vulnerability score,” to the toolbox of the researchers doing place and HIV research. The vulnerability score measures how place, as known through the raw measurements, is predictive of an outcome. It captures variation in place characteristics that matters most for the particular outcome. We demonstrate the estimation and utility of place-based vulnerability scores for HIV viral non-suppression, using data with complicated clustering from a cohort of people with histories of injecting drugs.

Abstract Background Enzymes play an irreplaceable and important role in maintaining the lives of living organisms. The Enzyme Commission (EC) number of an enzyme indicates its essential functions. Correct identification of the first digit (family class) of the EC number for a given enzyme is a hot topic in the past twenty years. Several previous methods adopted functional domain composition to represent enzymes. However, it would lead to dimension disaster, thereby reducing the efficiency of the methods. On the other hand, most previous methods can only deal with enzymes belonging to one family class. In fact, several enzymes belong to two or more family classes. Results In this study, a fast and efficient multi-label classifier, named PredictEFC, was designed. To construct this classifier, a novel feature extraction scheme was designed for processing functional domain information of enzymes, which counting the distribution of each functional domain entry across seven family classes in the training dataset. Based on this scheme, each training or test enzyme was encoded into a 7-dimenion vector by fusing its functional domain information and above statistical results. Random k-labelsets (RAKEL) was adopted to build the classifier, where random forest was selected as the base classification algorithm. The two tenfold cross-validation results on the training dataset shown that the accuracy of PredictEFC can reach 0.8493 and 0.8370. The independent test on two datasets indicated the accuracy values of 0.9118 and 0.8777. Conclusion The performance of PredictEFC was slightly lower than the classifier directly using functional domain composition. However, its efficiency was sharply improved. The running time was less than one-tenth of the time of the classifier directly using functional domain composition. In additional, the utility of PredictEFC was superior to the classifiers using traditional dimensionality reduction methods and some previous methods, and this classifier can be transplanted for predicting enzyme family classes of other species. Finally, a web-server available at http://124.221.158.221/ was set up for easy usage.

Pattern selection is crucial for optimizing the calibration process of optical proximity correction (OPC) models in computational lithography. However, it remains a challenge to achieve a balance between representative coverage and computational efficiency. This work presents a comprehensive evaluation of the feature vectors’ (FVs’) effectiveness in pattern selection for OPC model calibration, leveraging key performance indicators (KPIs) based on Kullback–Leibler divergence and distance ranking. Through the construction of autoencoder-based FVs and fast Fourier transform (FFT)-based FVs, we compare their efficacy in capturing critical pattern features. Validation experimental results indicate that autoencoder-based FVs, particularly augmented with the lithography domain knowledge, outperform FFT-based counterparts in identifying anomalies and enhancing lithography model performance. These results also underscore the importance of adaptive pattern representation methods in calibrating the OPC model with evolving complexities.

This study examines how telematics variables such as annual percentage driven, total miles driven, and driving patterns influence the distributional behaviour of conventional rating factors when incorporated into predictive models for capturing auto insurance risk in rate regulation. To effectively manage the complexity inherent in telematics data, we advocate for the adoption of non-negative sparse principal component analysis (NSPCA) as a structured approach for data dimensionality reduction. By emphasizing sparsity and non-negativity constraints, NSPCA enhances the interpretability and predictive power of models concerning both loss severity and claim counts. This methodological innovation aims to advance statistical analyses within insurance pricing frameworks, ensuring the robustness of predictive models and providing insights crucial for rate regulation strategies specific to the auto insurance sector. Results show that, to enhance auto insurance risk pricing models, it is essential to address data dimension reduction challenges when integrating telematics data variables. Our findings underscore that integrating telematics variables into predictive models maintains the integrity of risk relativity estimates associated with traditional policy variables.

We introduce a new method for releasing answers to statistical queries with differential privacy, based on the Johnson-Lindenstrauss lemma. The key idea is to randomly project the query answers to a lower dimensional space so that the distance between any two vectors of feasible query answers is preserved up to an additive error. Then we answer the projected queries using a simple noise-adding mechanism, and lift the answers up to the original dimension. Using this method, we give, for the first time, purely differentially private mechanisms with optimal worst case sample complexity under average error for answering a workload of $k$ queries over a universe of size $N$. As other applications, we give the first purely private efficient mechanisms with optimal sample complexity for computing the covariance of a bounded high-dimensional distribution, and for answering 2-way marginal queries. We also show that, up to the dependence on the error, a variant of our mechanism is nearly optimal for every given query workload.

Endpoint control stands as a pivotal determinant of steel quality. However, the data derived from the BOF steelmaking process are characterized by high dimension, with intricate nonlinear relationships between variables and diverse working conditions. Traditional dimension reduction does not fully use non-local structural information within manifold shapes. To address these challenges, the article introduces a novel approach termed supervised weighting-based local structure preserving projection. This method dynamically includes label information using sparse representation and constructs weighted submanifolds to mitigate the influence of irrelevant labels. Subsequently, trend match is employed to establish the same distribution datasets for the submanifold. The global and local initial neighborhood maps are then constructed, extracting non-local relations from the submanifold by analyzing manifold curvature. This process eliminates interference from non-nearest-neighbor points on the manifold while preserving the local geometric structure, facilitating adaptive neighborhood parameter change. The proposed method enhances the adaptability of the model to changing working conditions and improves overall performance. The carbon content prediction maintains a controlled error range of within ±0.02%, achieving an accuracy rate of 82.50%. The temperature prediction maintains a controlled error range of within ±10°C, achieving an accuracy rate of 79.00%.

Dimension reduction is an effective method for system’s resilience analysis. In this paper, we investigate the effect of network structure on the accuracy of resilience dimension reduction. First, we introduce the resilience dimension reduction method and define the evaluation indicator of the resilience dimension reduction method. Then, by adjusting node connections, preferential connection mechanisms, and connection probabilities, we generate artificial networks, small-world networks and social networks with tunable assortativity coefficients, average clustering coefficients, and modularities, respectively. Experimental results for the gene regulatory dynamics show that the network structures with positive assortativity, large clustering coefficient, and significant community can enhance the accuracy of resilience dimension reduction. The result of this paper indicates that optimizing network structure can enhance the accuracy of resilience dimension reduction, which is of great significance for system resilience analysis and provides a new perspective and theoretical basis for selecting dimension reduction methods in system resilience analysis.

Effective detection of stress situations plays an important role in combating it. This is the main source of motivation for research to identify and evaluate different psychological conditions. Different monitor signals are used to identify individuals' stress situations in daily life. Electroencephalogram (EEG) signals are the main component used to detect stress and depression. The long-term acquisition of this signals partially interrupts daily life and negatively affects it. Researchers are trying to develop wearable technologies that can eliminate this disadvantage. In this study, stress situations are detected utilizing different sensors without EEG signals. The achievements of three different classification methods for different dimensional feature spaces have been compared. The effects of the feature selection and dimension reduction methods on the system performance have been analyzed. During the dimension reduction process, Minimum Redundancy Maximum Relevance (MRMR), Anova, Chi-2, Relieff, Kruskal Wallis (KW) and Principal Component Analysis (PCA) methods are implemented. Support Vector Machines (SVM), Linear Discriminant Analysis (LDA) and k-Nearest Neighbor (k-NN) methods are used as classifiers. The best performance is achieved with 96.2 % accuracy in 15-dimensional by using LDA and PCA methods together.

Various data analysis research has recently become necessary in to find and select relevant features without class labels using Unsupervised Feature Selection (UFS) approaches. Despite the fact that several open-source toolboxes provide feature selection techniques to reduce redundant features, data dimensionality, and computation costs, these approaches require programming knowledge, which limits their popularity and has not adequately addressed unlabeled real-world data. Automatic UFS Toolbox (Auto-UFSTool) for MATLAB, proposed in this study, is a user-friendly and fully-automatic toolbox that utilizes several UFS approaches from the most recent research. It is a collection of 25 robust UFS approaches, most of which were developed within the last five years. Therefore, a clear and systematic comparison of competing methods is feasible without requiring a single line of code. Even users without any previous programming experience may utilize the actual implementation by the Graphical User Interface (GUI). It also provides the opportunity to evaluate the feature selection results and generate graphs that facilitate the comparison of subsets of varying sizes. It is freely accessible in the MATLAB File Exchange repository and includes scripts and source code for each technique. The link to this toolbox is freely available to the general public on: bit.ly/AutoUFSTool

Abstract Background Complex biological systems are described as a multitude of cell–cell interactions (CCIs). Recent single-cell RNA-sequencing studies focus on CCIs based on ligand–receptor (L–R) gene co-expression but the analytical methods are not appropriate to detect many-to-many CCIs. Results In this work, we propose scTensor, a novel method for extracting representative triadic relationships (or hypergraphs), which include ligand-expression, receptor-expression, and related L–R pairs. Conclusions Through extensive studies with simulated and empirical datasets, we have shown that scTensor can detect some hypergraphs that cannot be detected using conventional CCI detection methods, especially when they include many-to-many relationships. scTensor is implemented as a freely available R/Bioconductor package.

Abstract Background Dimension reduction, especially feature selection, is an important step in improving classification performance for high-dimensional data. Particularly in cancer research, when reducing the number of features, i.e., genes, it is important to select the most informative features/potential biomarkers that could affect the diagnostic accuracy. Therefore, researchers continuously try to explore more efficient ways to reduce the large number of features/genes to a small but informative subset before the classification task. Hybrid methods have been extensively investigated for this purpose, and research to find the optimal approach is ongoing. Social network analysis is used as a part of a hybrid method, although there are several issues that have arisen when using social network tools, such as using a single environment for computing, constructing an adjacency matrix or computing network measures. Therefore, in our study, we apply a hybrid feature selection method consisting of several machine learning algorithms in addition to social network analysis with our proposed network metric, called the corrected degree of domesticity, in a single environment, R, to improve the support vector machine classifier’s performance. In addition, we evaluate and compare the performances of several combinations used in the different steps of the method with a simulation experiment. Results The proposed method improves the classifier’s performance compared to using the whole feature set in all the cases we investigate. Additionally, in terms of the area under the receiver operating characteristic (ROC) curve, our approach improves classification performance compared to several approaches in the literature. Conclusion When using the corrected degree of domesticity as a network degree centrality measure, it is important to use our correction to compare nodes/features with no connection outside of their community since it provides a more accurate ranking among the features. Due to the nature of the hybrid method, which includes social network analysis, it is necessary to investigate possible combinations to provide an optimal solution for the microarray data used in the research.

Continuous semigroup theory is applied to proof the existence and uniqueness of a solution to a fluid-structure interaction (FSI) problem of non-stationary Stokes flow in two bulk domains, separated by a 2D elastic, permeable plate. The plate's curvature is proportional to the jump of fluid stresses across the plate and the flow resistance is modeled by Darcy's law. In the weak formulation of the considered physical problem, a linear operator in space is associated with a sum of two bilinear forms on the fluid and the interface domains, respectively. One attains a system of equations in operator form, corresponding to the weak problem formulation. Utilizing the sufficient conditions in the Lumer-Phillips theorem, we show that the linear operator is a generator of a contraction semigroup, and give the existence proof to the FSI problem.

We consider situations in which the clustering of some multivariate data is desired, which establishes an ordering of the clusters with respect to an underlying latent variable. As our motivating example for a situation where such a technique is desirable, we consider scatterplots of traffic flow and speed, where a pattern of consecutive clusters can be thought to be linked by a latent variable, which is interpretable as traffic density. We focus on latent structures of linear or quadratic shapes, and present an estimation methodology based on expectation–maximization, which estimates both the latent subspace and the clusters along it. The directed clustering approach is summarized in two algorithms and applied to the traffic example outlined. Connections to related methodology, including principal curves, are briefly drawn.

Hypertensive disorders in pregnancy, which include preeclampsia, eclampsia, and chronic hypertension, complicate approximately 10% of all pregnancies in the world, constituting one of the most serious causes of mortality and morbidity in gestation. To help predict the occurrence of hypertensive disorders, a study based on algorithms that help model this health problem using mathematical tools is proposed. This study proposes a fuzzy c-means (FCM) model based on the Takagi–Sugeno (T-S) type of fuzzy rule to predict hypertensive disorders in pregnancy. To test different modeling methodologies, cross-validation comparisons were made between random forest, decision tree, support vector machine, and T-S and FCM methods, which achieved 80.00%, 66.25%, 70.00%, and 90.00%, respectively. The evaluation consisted of calculating the true positive rate (TPR) over the true negative rate (TNR), with equal error rate (EER) curves achieving a percentage of 20%. The learning dataset consisted of a total of 371 pregnant women, of which 13.2% were diagnosed with a condition related to gestational hypertension. The dataset for this study was obtained from the Secretaría de Salud del Estado de Hidalgo (SSEH), México. A random sub-sampling technique was used to adjust the class distribution of the data set, and to eliminate the problem of unbalanced classes. The models were trained using a total of 98 samples. The modeling results indicate that the T-S and FCM method has a higher predictive ability than the other three models in this research.

Manifold learning-based approaches have emerged as prominent techniques for dimensionality reduction. Among these methods, t-Distributed Stochastic Neighbor Embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP) stand out as two of the most widely used and effective approaches. While both methods share similar underlying procedures, empirical observations indicate two distinctive properties: global data structure preservation and computational efficiency. However, the underlying mathematical principles behind these distinctions remain elusive. To address this gap, this study presents a comparative analysis of the subprocesses involved in these methods, aiming to elucidate the mathematical mechanisms underlying the observed distinctions. By meticulously examining the equation formulations, the mathematical mechanisms contributing to global data structure preservation and computational efficiency are elucidated. To validate the theoretical analysis, data are collected through a laboratory experiment, and an open-source dataset is utilized for validation across different datasets. The consistent alignment of results obtained from both balanced and unbalanced datasets robustly confirms the study’s findings. The insights gained from this study provide a deeper understanding of the mathematical underpinnings of t-SNE and UMAP, enabling more informed and effective use of these dimensionality reduction techniques in various applications, such as anomaly detection, natural language processing, and bioinformatics.

Dimension reduction is a technique used to transform data from a high-dimensional space into a lower-dimensional space, aiming to retain as much of the original information as possible. This approach is crucial in many disciplines like engineering, biology, astronomy, and economics. In this paper, we consider the following dimensionality reduction instance: Given an n-dimensional probability distribution p and an integer m

Brain functional connectivity is a useful biomarker for diagnosing brain disorders. Connectivity is measured using resting-state functional magnetic resonance imaging (rs-fMRI). Previous studies have used a sequential application of the graphical model for network estimation and machine learning to construct predictive formulas for determining outcomes (e.g., disease or health) from the estimated network. However, the resulting network had limited utility for diagnosis because it was estimated independent of the outcome. In this study, we proposed a regression method with scores from rs-fMRI based on supervised sparse hierarchical components analysis (SSHCA). SSHCA has a hierarchical structure that consists of a network model (block scores at the individual level) and a scoring model (super scores at the population level). A regression model, such as the multiple logistic regression model with super scores as the predictor, was used to estimate diagnostic probabilities. An advantage of the proposed method was that the outcome-related (supervised) network connections and multiple scores corresponding to the sub-network estimation were helpful for interpreting the results. Our results in the simulation study and application to real data show that it is possible to predict diseases with high accuracy using the constructed model.

Abstract The application of the security region methodology in a practical distribution system with large scale normally requires large computer memory and high computation time. To overcome this problem, this article proposes a decoupling and dimension reduction method, which can significantly accelerate the calculation of distribution system security region (DSSR) and is important for the application of the DSSR theory in large‐scale distribution systems. First, the definition of DSSR dimension reflecting the size of solution space and the time complexity is proposed. And the solution algorithm for DSSR dimension is also given. Second, a decoupling and dimension reduction method suitable for the analysis of DSSR is proposed. Following the method, an incidence matrix can be obtained from the DSSR expressions, which can be further divided into multiple block matrices. According to the feeder combinations of the block matrices, the distribution system can be decoupled into multiple sub‐networks for more efficient analysis. Finally, a 10kV distribution network is used in case study to validate the proposed method. The results for a time‐consuming calculation, that is, TSC curve calculation, show that the proposed method can reduce the computation time significantly, making the time‐consuming calculation suitable for the analysis of large‐scale cases.

This paper proposes the research on dimension reduction for visualization of simplified security region of integrated energy system considering renewable energy access. Reason: Integrated energy system is conducive to achieving low-carbon transition and efficient utilization of energy. As the penetration of renewable energy increases, the security of integrated energy system faces severe challenges. It holds significance for comprehensively studying N-1 security test method in the operation scenario of integrated energy system, the impact of renewable energy on the IES security region model, and the optimal selection for observation variables. Methods: Firstly, a model for interval security region of integrated energy system is constructed, according to the N-1 safety constraints of energy hub key equipment and pipeline. The imbalance between energy demand and supply caused by the renewable energy uncertainty is defined as a special N-1 fault. Then, an engineering simplification approach for interval security region model is proposed, in order to quantify the security boundary, security distance, and interval total supply capability. On the basis, a dimension reduction observation method based on observation variable optimization is proposed, the selection of observation variables help to expand the volume of area unaffected by renewable energy uncertainty. Results: To the existed problem in current research, an example is presented to verify the effectiveness of proposed method, which brings forward N-1 security test method in the operation scenario of integrated energy system, analyzes the impact of renewable energy on the security region model, and realizes the optimal selection for observation variables of simplified security region. Conclusions: Interval security region of integrated energy system can intuitively estimate the security status of operating points and observe the security boundaries. The simplified model can satisfy the needs of calculating precision. Optimal selection of observation variables is necessary, and the volume of area unaffected by renewable energy uncertainty can be effectively increased.

Abstract As an ecological strategy for species coexistence, some species adapt to a wide range of habitats, while others specialize in particular environments. Such ‘generalists’ and ‘specialists’ achieve normal ecological balance through a complex network of interactions between species. However, the role of these interactions in maintaining the coexistence of generalist and specialist species has not been elucidated within a general theoretical framework. Here, we analyze the ecological mechanism for the coexistence of specialist and generalist species in a class of mutualistic and competitive interaction ecosystems based on the network dimension reduction method. We find that ecological specialists and generalists can be identified based on the number of their respective interactions. We also find, using real‐world empirical network simulations, that the removal of ecological generalists can lead to the collapse of local ecosystems, which is rarely observed with the loss of ecological specialists.

The analysis of residential electricity consumption behavior (RECB) aims to deeply reveal the electricity consumption characteristics of customers that can be used to improve the electricity service performances from the large volume of load data. For analyzing the RECB, various clustering-based methods have been developed. However, with the dimension of power consumption data increasing, it is difficult for traditional methods to accurately identify users’ power consumption patterns. To this end, this paper proposes a novel analysis method for RECB with high-dimensional data based on uniform manifold approximation and projection-criteria importance through intercriteria correlation (UMAP-CRITIC) feature optimization and sparrow search algorithm (SSA)-assisted clustering. Specifically, the high-dimensional raw data combined with load characteristics indexes are first utilized to generate low-dimensional features by UMAP-based dimension reduction theory. Secondly, the contribution weight of each reduced feature to the intrinsic information of the original data is optimized based on the CRITIC weight method, and then the data feature set of RECB identification considering feature contributions is constructed. Finally, the SSA-based k-means is employed to obtain and analyze the various RECBs. The simulation results show that the proposed method can effectively identify high-dimensional electricity consumption data and precisely extract the electricity consumption information of residents.

Abstract Background Complex diseases such as neurodevelopmental disorders (NDDs) exhibit multiple etiologies. The multi-etiological nature of complex-diseases emerges from distinct but functionally similar group of genes. Different diseases sharing genes of such groups show related clinical outcomes that further restrict our understanding of disease mechanisms, thus, limiting the applications of personalized medicine approaches to complex genetic disorders. Results Here, we present an interactive and user-friendly application, called DGH-GO. DGH-GO allows biologists to dissect the genetic heterogeneity of complex diseases by stratifying the putative disease-causing genes into clusters that may contribute to distinct disease outcome development. It can also be used to study the shared etiology of complex-diseases. DGH-GO creates a semantic similarity matrix for the input genes by using Gene Ontology (GO). The resultant matrix can be visualized in 2D plots using different dimension reduction methods (T-SNE, Principal component analysis, umap and Principal coordinate analysis). In the next step, clusters of functionally similar genes are identified from genes functional similarities assessed through GO. This is achieved by employing four different clustering methods (K-means, Hierarchical, Fuzzy and PAM). The user may change the clustering parameters and explore their effect on stratification immediately. DGH-GO was applied to genes disrupted by rare genetic variants in Autism Spectrum Disorder (ASD) patients. The analysis confirmed the multi-etiological nature of ASD by identifying four clusters of genes that were enriched for distinct biological mechanisms and clinical outcome. In the second case study, the analysis of genes shared by different NDDs showed that genes causing multiple disorders tend to aggregate in similar clusters, indicating a possible shared etiology. Conclusion DGH-GO is a user-friendly application that allows biologists to study the multi-etiological nature of complex diseases by dissecting their genetic heterogeneity. In summary, functional similarities, dimension reduction and clustering methods, coupled with interactive visualization and control over analysis allows biologists to explore and analyze their datasets without requiring expert knowledge on these methods. The source code of proposed application is available at https://github.com/Muh-Asif/DGH-GO

Three-way clustering uses core region and fringe region to describe a cluster, which divide the dataset into three parts. The division helps identify the central core and outer sparse regions of a cluster. One of the main challenges in three-way clustering is the meaningful construction of the two sets. Aimed at handling high-dimensional data and improving the stability of clustering, this paper proposes a novel three-way clustering method. The proposed method uses dimensionality reduction techniques to reduce data dimensions and eliminate noise. Based on the reduced dataset, random sampling and feature extraction are performed multiple times to introduce randomness and diversity, enhancing the algorithm’s robustness. Ensemble strategies are applied on these subsets, and the k-means algorithm is utilized to obtain multiple clustering results. Based on these results, we obtain co-association frequency between different samples and fused clustering result using the single-linkage method of hierarchical clustering. In order to describe the core region and fringe region of each cluster, the similar class of each sample is defined by co-association frequency. The lower and upper approximations of each cluster are obtained based on similar class. The samples in the lower approximation of each cluster belong to the core region of the cluster. The differences between lower and upper approximations of each cluster are defined as fringe region. Therefore, a three-way explanation of each cluster is naturally formed. By employing various UC Irvine Machine Learning Repository (UCI) datasets and comparing different clustering metrics such as Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and Accuracy (ACC), the experimental results show that the proposed strategy is effective in improving the structure of clustering results.

The potential outcomes framework serves as a fundamental tool for quantifying causal effects. The average dose–response function μ(t) (also called the effect curve) is typically of interest when dealing with a continuous treatment variable (exposure). The focus of this work is to determine the impact of an extreme level of treatment, potentially beyond the range of observed values—that is, estimating μ(t) for very large t. Our approach is grounded in the field of statistics known as extreme value theory. We outline key assumptions for the identifiability of the extreme treatment effect. Additionally, we present a novel and consistent estimation procedure that can potentially reduce the dimension of the confounders to at most 3. This is a significant result since typically, the estimation of μ(t) is very challenging due to high-dimensional confounders. In practical applications, our framework proves valuable when assessing the effects of scenarios such as drug overdoses, extreme river discharges, or extremely high temperatures on a variable of interest.

According to the World Health Organization, lung cancer is a leading cause of death worldwide. This research aims to process the Computerized Tomography (CT) images of lung cancer patients for the early diagnosis of the disease. The images are processed using Convolutional Neural Network (CNN) in the first approach, where Artificial Neural Network (ANN) is employed to classify the images. In the second approach, the images are pre-processed and segmented before utilizing CNN and ANN. In the third method, all the pre-processed and segmented images are converted to numerical data via specific feature extraction algorithms in the last step. Besides, dimensional reduction and feature selection algorithms are employed to classify with three machine learning techniques, i.e., Gradient Boosting (GB), Random Forest (RF), and Support Vector Machine (SVM). An extensive comparative analysis is made to come up with the best technique. The comparisons are made by evaluating the methodologies on a set of lung CT scan images collected from a medical center. The results show that when either SVM or RF classification techniques are used, a 95% accuracy is obtained in diagnosing lung cancer.

Abstract Background Cluster analysis is utilized frequently in scientific theory and applications to separate data into groups. A key assumption in many clustering algorithms is that the data was generated from a population consisting of multiple distinct clusters. Clusterability testing allows users to question the inherent assumption of latent cluster structure, a theoretical requirement for meaningful results in cluster analysis. Results This paper proposes methods for clusterability testing designed for high-dimensional data by utilizing sparse principal component analysis. Type I error and power of the clusterability tests are evaluated using simulated data with different types of cluster structure in high dimensions. Empirical performance of the new methods is evaluated and compared with prior methods on gene expression, microarray, and shotgun proteomics data. Our methods had reasonably low Type I error and maintained power for many datasets with a variety of structures and dimensions. Cluster structure was not detectable in other datasets with spatially close clusters. Conclusion This is the first analysis of clusterability testing on both simulated and real-world high-dimensional data.

The key to vertex embedding is to learn low-dimensional representations of global graph information, and integrating information from multiple steps is an effective strategy. Existing research shows that the transition probability of each step can capture the relationship between different hops, and the graph’s global information can be obtained simultaneously. However, much of the current work simply concatenates representations of different hops into a global representation. In other words, they are unclear about the contribution of each $k$ -step ( $k\geq {1}$ ) structural information in the embedding. With this motivation, we propose a unified framework that focuses on considering the contributions of different steps in global graph representation. It reconsiders the contribution of different steps in the global representation from two perspectives: (i)We flexibly assign different weights to the different steps loss function, (ii) According to different $k$ , we design strategies that all $k$ -step representations are concatenated with different proportions to form a global representation. Based on this, we more effectively integrate global structural information into the learning process. In this paper, our proposed framework achieves competitive performance on vertex classification, link prediction, and visualization tasks on multiple datasets.

Temporal network data are increasingly available in various domains, and often represent highly complex systems with intricate structural and temporal evolutions. Due to the difficulty of processing such complex data, it may be useful to coarse grain temporal network data into a numeric trajectory embedded in a low-dimensional space. We refer to such a procedure as temporal network embedding, which is distinct from procedures that aim at embedding individual nodes. Temporal network embedding is a challenging task because we often have access only to discrete time-stamped events between node pairs, and, in general, the events occur with irregular intervals, making the construction of the network at a given time a nontrivial question already. We propose a method to generate trajectories of temporal networks embedded in a low-dimensional space given a sequence of time-stamped events as input. We realize this goal by combining the landmark multidimensional scaling (LMDS), which is an out-of-sample extension of the well-known multidimensional scaling method, and the framework of tie-decay temporal networks. This combination enables us to obtain a continuous-time trajectory describing the evolution of temporal networks. We then study mathematical properties of the proposed temporal network embedding framework. Finally, we showcase the method with empirical data of social contacts to find temporal organization of contact events and loss of them over a single day and across different days.