Search

Finite mixture modeling, expanded to matrix-valued data, faces several challenges. One of the major concerns is overparameterization resulting from the high number of parameters involved in a matrix mixture. In addition, an appropriate power transformation is very useful if the data are skewed. The R package MatTransMix is a new piece of software devoted to parsimonious models, based on spectral decomposition of covariance matrices, developed for fitting heterogeneous matrix-valued data providing model-based clustering results. The package implements a variety of parsimonious models obtained from various combinations of spectral decomposition and skewness parameters. The paper discusses some methodological foundations of the proposed models and elaborates the functions available in this package on carefully chosen examples.

A new approach to the analysis of heterogeneous categorical sequences is proposed. The first-order Markov model is employed in a finite mixture setting with initial state and transition probabilities being expressed as functions of time. The expectation–maximization algorithm approach to parameter estimation is implemented in the presence of positive equivalence constraints that determine which observations must be placed in the same class in the solution. The proposed model is applied to a dataset from the British Household Panel Survey to evaluate the association between the education background and life outcomes of study participants. The analysis of the survey data reveals many interesting relationships between the level of education and major life events.

For decades, Gaussian mixture models have been the most popular mixtures in literature. However, the adequacy of the fit provided by Gaussian components is often in question. Various distributions capable of modeling skewness or heavy tails have been considered in this context recently. In this paper, we propose a novel contaminated transformation mixture model that is constructed based on the idea of transformation to symmetry and can account for skewness, heavy tails, and automatically assign scatter to secondary components.

Finite mixtures present a powerful tool for modeling complex heterogeneous data. One of their most important applications is model-based clustering. It assumes that each data group can be reasonably described by one mixture model component. This establishes a one-to-one relationship between mixture components and clusters. In some cases, however, this relationship can be broken due to the presence of observations from the same class recorded in different ways. This effect can occur because of recording inconsistencies due to the use of different scales, operator errors, or simply various recording styles. The idea presented in this paper aims to alleviate this issue through modifications incorporated into mixture models. While the proposed methodology is applicable to a broad class of mixture models, in this paper it is illustrated on Gaussian mixtures. Several simulation studies and an application to a real-life data set are considered, yielding promising results.

In this paper, a new flexible approach to modeling data with multiple partial right-censoring points is proposed. This method is based on finite mixture models, flexible tool to model heterogeneity in data. A general framework to accommodate partial censoring is considered. In this setting, it is assumed that a certain portion of data points are censored and the rest are not. This situation occurs in many insurance loss data sets. A novel probability function is proposed to be used as a mixture component and the expectation-maximization algorithm is employed for estimating model parameters. The Bayesian information criterion is used for model selection. Additionally, an approach for the variability assessment of parameter estimates as well as the computation of quantiles commonly known as risk measures is considered. The proposed model is evaluated using a simulation study based on four common probability distribution functions used to model right skewed loss data and applied to a real data set with good results.

The paper discusses a new approach for incorporating hard constraints into the K-means algorithm for semi-supervised clustering. An analytic modification of the objective function of K-means is proposed that has not been previously considered in the literature.

The paper describes the medical data personalization problem by determining the individual characteristics needed to predict the number of days a patient spends in a hospital. The mathematical problem of patient information analysis is formalized, which will help identify critical personal characteristics based on conditioned space analysis. The condition space is given in cube form as a reflection of the functional relationship of the general parameters to the studied object. The dataset consists of 51 instances, and ten parameters are processed using different clustering and regression models. Days in hospital is the target variable. A condition space cube is formed based on clustering analysis and features selection. In this manner, a hierarchical predictor based on clustering and an ensemble of weak regressors is built. The quality of the developed hierarchical predictor for Root Mean Squared Error metric is 1.47 times better than the best weak predictor (perceptron with 12 units in a single hidden layer).

Existing clustering methods range from simple but very restrictive to complex but more flexible. The K-means algorithm is one of the most popular clustering procedures due to its computational speed and intuitive construction. Unfortunately, the application of K-means in its traditional form based on Euclidean distances is limited to cases with spherical clusters of approximately the same volume and spread of points. Recent developments in the area of merging mixture components for clustering show good promise. We propose a general framework for hierarchical merging based on pairwise overlap between components which can be readily applied in the context of the K-means algorithm to produce meaningful clusters. Such an approach preserves the main advantage of the K-means algorithm—its speed. The developed ideas are illustrated on examples, studied through simulations, and applied to the problem of digit recognition.

The article covers the formalization of time-dependent and time-independent data of various origins of the investigated object, which are necessary for personalization of patient’s data in the process of seeking an individual approach to choosing a treatment strategy. This allowed to simulate the warehouse of the state of the object being studied in time, as the Euclidean warehouse. The mathematical model of States` warehouse is constructed, which is a vector-matrix form for the recording of a system of differential equations of the first order. The analysis is made of the application of methods of cluster analysis, Bayesian approach and decision trees for finding personalized decisions about treatment and improvement of the general state of the patient.

The paper proposed the development of an approach to modeling the nature of the individual's illness based on the ensemble of machine learning algorithms. Some factors may adversely affect the conduct, interpretation and generalization of research results, and the understanding and interpretation of the phenomenon under study.

The study was conducted on the applying of machine learning methods to search for key clusters, which made it possible to investigate individual patient characteristics, and to determine distances between cluster instances. It was suggested to determine the average distance between instances in order to find optimal performance metrics. That will determine the optimal state of the instance of this cluster.

The study was conducted by applying machine learning and data mining methods to treatment personalization. This allows individual patient characteristics to be investigated. The personalization method was built on the clustering method and associative rules. It was suggested to determine the average distance between instances in order to find the optimal performance metrics. The formalization of the medical data preprocessing stage was proposed in order to find personalized solutions based on current standards and pharmaceutical protocols. The patient data model was built using time-dependent and time-independent parameters. Personalized treatment is usually based on the decision tree method. This approach requires significant computation time and cannot be parallelized. Therefore, it was proposed to group people by conditions and to determine deviations of parameters from the normative parameters of the group, as well as the average parameters. The novelty of the paper is the new clustering method, which was built from an ensemble of cluster algorithms, and the usage of the new distance measure with Hopkins metrics, which were 0.13 less than for the k-means method. The Dunn index was 0.03 higher than for the BIRCH (balanced iterative reducing and clustering using hierarchies) algorithm. The next stage was the mining of associative rules provided separately for each cluster. This allows a personalized approach to treatment to be created for each patient based on long-term monitoring. The correctness level of the proposed medical decisions is 86%, which was approved by experts.

The existing finite mixture modeling and model-based clustering literature focuses primarily on the analysis of multivariate data observed in the form of vectors, with each element representing a specific feature. In this setting, multivariate Gaussian mixture models have been the most commonly used. Due to severe modeling issues observed when normal components cannot provide adequate fit to groups, much attention has been paid to developing models capable of accounting for skewness in data. In our work, we target the problem of mixture modeling with components that can handle skewness in matrix-valued data. The proposed developments open a wide range of possible modeling capabilities, with numerous applications, as illustrated in this paper. A novel matrix mixture model is proposed. Its skewness parameters enjoy appealing interpretability. The corresponding estimation procedure and various ways of parameterization are discussed. Comprehensive simulation studies and applications to real-life datasets illustrate the efficiency of the proposed developments, supported by good results.

Grouping similar objects into common groups, also known as clustering, is an important problem of unsupervised machine learning. Various clustering algorithms have been proposed in literature. In recent years, the need to analyze large amounts of data has led to reconsidering some fundamental clustering procedures. One of them is the celebrated K-means algorithm popular among practitioners due to its speedy performance and appealingly intuitive construction. Unfortunately, the algorithm often shows poor performance unless data groups have spherical shapes and approximately same sizes. In many applications, this restriction is so severe that the use of the K-means algorithm becomes questionable, misleading, or simply incorrect. We propose an extension of K-means that preserves the speed and intuitive interpretation of the original algorithm while providing greater flexibility in modeling clusters. The idea of the proposed generalization relies on the exponential transformation of Manly originally designed to obtain near-normally distributed data. The suggested modification is derived and illustrated on several datasets with good results.

Studying crime trends and tendencies is an important problem that helps to identify socioeconomic patterns and relationships of crucial significance. Finite mixture models are famous for their flexibility in modeling heterogeneity in data. A novel approach designed for accounting for skewness in the distributions of matrix observations is proposed and applied to the United States crime data collected between 2000 and 2012 years. Then, the model is further extended by incorporating explanatory variables. A step-by-step model development demonstrates differences and improvements associated with every stage of the process. Results obtained by the final model are illustrated and thoroughly discussed. Multiple interesting conclusions have been drawn based on the developed model and obtained model-based clustering partition.

Finite mixture modeling is one of the most rapidly developing areas of statistics due to its modeling flexibility and appealing interpretability. Gaussian mixture models have been popular among researchers for decades proving their usefulness in various applications. However, when Gaussian mixture components do not provide an adequate fit for the data, more general models must be considered. Traditional remedies for deviation from normality include employing a more appropriate distribution as well as transforming data to near-normality. Merging both approaches by introducing a mixture model with components derived from the multivariate Manly transformation is proposed. Such mixture models show good performance in modeling skewness and have excellent interpretability. Forward and backward model selection algorithms are proposed to choose an appropriate multivariate transformation. At each step of these algorithms, a model with the specific combination of skewness parameters is estimated by means of the expectation–maximization algorithm. The developed technique is carefully illustrated on synthetic data and applied to several well-known datasets, with promising results.

A problem of finding variables responsible for classifying a particular observation as well as detecting those that interfered with the assignment made is considered. We address this problem by providing a formal argument supported by intuitive geometric interpretation.

A novel approach relying on the notion of mixture models is proposed for modeling and clustering directed weighted networks. The developed methodology can be used in a variety of settings including multilayer networks. Computational issues associated with the developed procedure are effectively addressed by the use of MCMC techniques. The utility of the methodology is illustrated on a set of experiments as well as applications to real-life data containing export trade amounts for European countries.

Finite mixture modeling is a popular statistical technique capable of accounting for various shapes in data. One popular application of mixture models is model-based clustering. This paper considers the problem of clustering regression autoregressive moving average time series. Two novel estimation procedures for the considered framework are developed. The first one yields the conditional maximum likelihood estimates which can be used in cases when the length of times series is substantial. Simple analytical expressions make fast parameter estimation possible. The second method incorporates the Kalman filter and yields the exact maximum likelihood estimates. The procedure for assessing variability in obtained estimates is discussed. We also show that the Bayesian information criterion can be successfully used to choose the optimal number of mixture components and correctly assess time series orders. The performance of the developed methodology is evaluated on simulation studies. An application to the analysis of tree ring data is thoroughly considered. The results are very promising as the proposed approach overcomes the limitations of other methods developed so far.

Finite mixture models represent one of the most popular tools for modeling heterogeneous data. The traditional approach for parameter estimation is based on maximizing the likelihood function. Direct optimization is often troublesome due to the complex likelihood structure. The expectation---maximization algorithm proves to be an effective remedy that alleviates this issue. The solution obtained by this procedure is entirely driven by the choice of starting parameter values. This highlights the importance of an effective initialization strategy. Despite efforts undertaken in this area, there is no uniform winner found and practitioners tend to ignore the issue, often finding misleading or erroneous results. In this paper, we propose a simple yet effective tool for initializing the expectation---maximization algorithm in the mixture modeling setting. The idea is based on model averaging and proves to be efficient in detecting correct solutions even in those cases when competitors perform poorly. The utility of the proposed methodology is shown through comprehensive simulation study and applied to a well-known classification dataset with good results.

Cluster analysis is a popular statistics and computer science technique commonly used in various areas of research. In this article, we investigate factors that can influence clustering performance in the model-based clustering framework. The four factors considered are the level of overlap, number of clusters, number of dimensions, and sample size. Through a comprehensive simulation study, we investigate model-based clustering in different settings. As a measure of clustering performance, we employ three popular classification indices capable of reflecting the degree of agreement in two partitioning vectors, thus making the comparison between the true and estimated classification vectors possible. In addition to studying clustering complexity, the performance of the three classification measures is evaluated.

Finite mixture models are well known for their flexibility in modeling heterogeneity in data. Model-based clustering is an important application of mixture models, which assumes that each mixture component distribution can adequately model a particular group of data. Unfortunately, when more than one component is needed for each group, the appealing one-to-one correspondence between mixture components and groups of data is ruined and model-based clustering loses its attractive interpretation. Several remedies have been considered in literature. We discuss the most promising recent results obtained in this area and propose a new algorithm that finds partitionings through merging mixture components relying on their pairwise overlap. The proposed technique is illustrated on a popular classification and several synthetic datasets, with excellent results.

Navigation patterns expressed by sequences of visited web-sites or categories can characterize the behavior and habits of users. Such web-page routes taken by individuals are commonly called clickstreams. Clustering clickstream sequences is a recent yet challenging problem with many applications. The main difficulty is related to the fact that one needs to group categorical data sequences rather than vectors and the majority of traditional clustering algorithms are not applicable in this setting. The time-related character of data suggests that dynamic models have a better promise than static ones. Model-based clustering relying on the mixture of first order Markov models will be considered. Since the number of distinct web-pages, and therefore the number of states in a Markov process, can be very high, such a mixture model involves a large number of parameters. Thus, grouping states by their similarity to reduce the number of parameters in the model is also proposed. Then, states are clustered along with users providing a biclustering framework. The developed methodology is illustrated on synthetic and real datasets with good results.

Finite mixture modeling is a popular technique for capturing heterogeneity in data. Although the vast majority of the theory developed in this area up to date deals with vector-valued data, some recent advancements have been made to expand the concept to matrix-valued data, for example, by means of matrix Gaussian mixture models. Unfortunately, matrix mixtures tend to suffer from the overparameterization issue due to a high number of parameters involved in the model. As a result, this may lead to problems such as overfitting and mixture order underestimation. One possible approach of addressing the overparameterization issue that has proven to be effective in the vector-valued framework is to consider various parsimonious models. One of the most popular classes of parsimonious models is based on the spectral decomposition of covariance matrices. An attempt to generalize this class and make it applicable in the matrix setting is made. Estimation procedures are thoroughly discussed for all models considered. The application of the proposed methodology is studied on synthetic and real-life data sets.

Finite mixtures provide a powerful tool for modeling heterogeneous data. Model-based clustering is a broadly used grouping technique that assumes the existence of the one-to-one correspondence between clusters and mixture model components. Although there are many directions of active research in the model-based clustering framework, very little attention has been paid to studying the specific nature of detected clustering solutions. In this paper, we develop an approach for assessing the variability in classifications carried out by the Bayes decision rule. The proposed technique allows assessing significance of each assignment made. We also apply the developed instrument for identifying influential observations that have impact on the structure of the detected partitioning. The proposed diagnostic methodology is studied and illustrated on synthetic data and applied to the analysis of three well-known classification datasets.

Cluster analysis is a popular technique in statistics and computer science with the objective of grouping similar observations in relatively distinct groups generally known as clusters. Semi-supervised clustering assumes that some additional information about group memberships is available. Under the most frequently considered scenario, labels are known for some portion of data and unavailable for the rest of observations. In this paper, we discuss a general type of semi-supervised clustering defined by so called positive and negative constraints. Under positive constraints, some data points are required to belong to the same cluster. On the contrary, negative constraints specify that particular points must represent different data groups. We outline a general framework for semi-supervised clustering with constraints naturally incorporating the additional information into the EM algorithm traditionally used in mixture modeling and model-based clustering. The developed methodology is illustrated on synthetic and classification datasets. A dendrochronology application is considered and thoroughly discussed.

The K -means algorithm is commonly used with the Euclidean metric. While the use of Mahalanobis distances seems to be a straightforward extension of the algorithm, the initial estimation of covariance matrices can be complicated. We propose a novel approach for initializing covariance matrices.

The paper discusses an approach based on the multivariate Delta method for approximating the distribution of posterior probabilities in finite mixture models. It can be used for developing distributions of many other characteristics involving posterior probabilities such as the entropy of fuzzy classification or expected cluster sizes. An application of the proposed methodology to clustering through merging mixture components is proposed and discussed. The methodology is studied and illustrated on simulated and well-known classification datasets with good results.

There is a vast variety of clustering methods available in the literature. The performance of many of them strongly depends on specific patterns in data. This paper introduces a clustering procedure based on the empirical likelihood method which inherits many advantages of the classical likelihood approach without imposing restrictive probability distribution constraints. The performance of the proposed procedure is illustrated on simulated and classification datasets with excellent results. The comparison of the algorithm with several well-known clustering methods is very encouraging. The procedure is more robust and has higher accuracy than the competitors.

Summary Secondary ion mass spectrometry is a popular physical technique that allows learning elemental and chemical compositions of various substances. In some cases, the mass spectrum of the sample studied is not observable without that of the liquid solution in which the investigated substance must be placed. To separate the informative part of the observed mixed spectrum from the irrelevant liquid solution part, we develop an approach that is based on finite mixtures. Finite mixture modelling is a statistical technique that is capable of accounting for various shapes and patterns in data. We discuss a mixture model constructed with bell-shaped discrete component distributions that can be employed in the mass spectrometry framework for extracting the informative part of the mixed spectrum of organic objects. The results for assessing variability in the mass spectrum extracted are derived. The procedure is thoroughly illustrated on simulated data sets and applied to several real life problems that are concerned with complex organic dyes placed in a glycerol liquid matrix. The results obtained have clear chemical interpretation and suggest that the approach proposed is very promising.

Model-based clustering is an increasingly popular area of cluster analysis that relies on probabilistic description of data by means of finite mixture models. Mixture distributions prove to be a powerful technique for modeling heterogeneity in data. In model-based clustering, each data group is seen as a sample from one or several mixture components. Despite attractive interpretation, model-based clustering poses many challenges. This paper discusses some of the most important problems a researcher might encounter while applying the model-based cluster analysis. WIREs Comput Stat 2013, 5:135–148. doi: 10.1002/wics.1248

An approach is proposed for initializing the expectation-maximization (EM) algorithm in multivariate Gaussian mixture models with an unknown number of components. As the EM algorithm is often sensitive to the choice of the initial parameter vector, efficient initialization is an important preliminary process for the future convergence of the algorithm to the best local maximum of the likelihood function. We propose a strategy initializing mean vectors by choosing points with higher concentrations of neighbors and using a truncated normal distribution for the preliminary estimation of dispersion matrices. The suggested approach is illustrated on examples and compared with several other initialization methods.

This paper discusses an alternative approach to the estimation procedure presented in a recently published paper. The authors developed a model-based clustering approach for regression time series and proposed the APECM procedure as an acceleration method for the expectation–maximization algorithm. The process of the estimation of model parameters was discussed in great detail. In this paper, we show how the proposed procedure can be modified to achieve substantial acceleration and better stability. In particular, numerical maximization suggested for the estimation of parameters can be replaced with analytical closed-form expressions, and inverting high dimensional matrices can be avoided entirely. A convenient approach for assessing variability in parameter estimates is also provided. The results of conducted experiments are very promising. © 2012 Wiley Periodicals, Inc. Statistical Analysis and Data Mining, 2012 © 2012 Wiley Periodicals, Inc.