Search

An unsupervised change point detection (CPD) framework assisted by a predictive machine learning model called "Predict and Compare" is introduced which is able to detect change points online under the presence of non-trivial trend patterns which must be prevented from triggering false positives. Different predictive models for the required time series forecasting (Predict) step together with different statistical tests for deciding about the proximity of predicted and actual data (Compare step) are allowed. Its performance is shown for the Predict step being carried out by either an LSTM recursive neural network or an ARIMA linear time series model together with the CUSUM rule as Compare step method. It shows to perform best in comparison to several other online CPD methods for detect times in the regime of low numbers of false positive detections. The method's good performance is based on its ability to detect structural changes in the presence complex underlying trend patterns. The use case concerns tribological wear for which change points separating the run-in, steady-state, and divergent wear phases are detected. [ABSTRACT FROM AUTHOR]

Microbiome data analysis often relies on the identification of a subset of potential biomarkers associated with a clinical outcome of interest. Robust ZeroSum regression, an elastic-net penalized compositional regression built on the least trimmed squares estimator, is a variable selection procedure capable to cope with the high dimensionality of these data, their compositional nature, and, at the same time, it guarantees robustness against the presence of outliers. The necessity of discovering "true" effects and to improve clinical research quality and reproducibility has motivated us to propose a two-step robust compositional knockoff filter procedure, which allows selecting the set of relevant biomarkers, among the many measured features having a nonzero effect on the response, controlling the expected fraction of false positives. We demonstrate the effectiveness of our proposal in an extensive simulation study, and illustrate its usefulness in an application to intestinal microbiome analysis. [ABSTRACT FROM AUTHOR]

A robust and sparse estimator for multinomial regression is proposed for high dimensional data. Robustness of the estimator is achieved by trimming the observations, and sparsity of the estimator is obtained by the elastic net penalty. In contrast to multi-group classifiers based on dimension reduction, this model is very appealing in terms of interpretation, since one obtains estimated coefficients individually for every group, and also the sparsity of the coefficients is group specific. Simulation studies are conducted to show the performance in comparison to the non-robust version of the multinomial regression estimator, and some real data examples underline the usefulness of this robust estimator particularly in terms of result interpretation and model diagnostics. [ABSTRACT FROM AUTHOR]

Compositional data are commonly known as multivariate observations carrying relative information. Even though the case of vector or even two-factorial compositional data (compositional tables) is already well described in the literature, there is still a need for a comprehensive approach to the analysis of multi-factorial relative-valued data. Therefore, this contribution builds around the current knowledge about compositional data a general theoretical framework for k-factorial compositional data. As a main finding it turns out that, similar to the case of compositional tables, also the multi-factorial structures can be orthogonally decomposed into an independent and several interactive parts and, moreover, a coordinate representation allowing for their separate analysis by standard analytical methods can be constructed. For the sake of simplicity, these features are explained in detail for the case of three-factorial compositions (compositional cubes), followed by an outline covering the general case. The three-dimensional structure is analyzed in depth in two practical examples, dealing with systems of spatial and time dependent compositional cubes. The methodology is implemented in the R package robCompositions. [ABSTRACT FROM AUTHOR]

It often occurs in practice that it is sensible to give different weights to the variables involved in a multivariate data analysis—and the same holds for compositional data as multivariate observations carrying relative information. It can be convenient to apply weights to better accommodate differences in the quality of the measurements, the occurrence of zeros and missing values, or generally to highlight some specific features of compositional parts. The characterisation of compositional data as elements of a Bayes space, which is as a natural generalisation of the ordinary Aitchison geometry, enables the definition of a formal framework to implement weighting schemes for the parts of a composition. This is formally achieved by considering a reference measure in the Bayes space alternative to the common uniform measure via the well-known chain rule. Unweighted centred logratio (clr) coefficients and isometric logratio (ilr) coordinates then allow us to express compositions in real space equipped with (unweighted) Euclidean geometry. The resulting elements of real space generated by the clr coefficients or ilr coordinates are invariant to the scale of the original compositions, but the actual scale of the weights matters. In this work, these formal developments are presented and used to introduce a general approach for weighting parts in compositional data analysis. The practical use is demonstrated on simulated and real-world data sets in the context of the earth sciences. [ABSTRACT FROM AUTHOR]

The important goals of mineral exploration geochemistry are detection and identification of underlying mineralization. This paper deals with element concentration data analyzed of surface geochemical samples acquired from soil horizons or plants. A new unsupervised procedure is proposed for this purpose when the samples have been taken on a regular or irregular grid in the area under investigation. The methodology is based on Generalized Additive Model fits on element concentration data. Then new data points are taken of the surface of the smooth fits across the entire sampling area as a regular grid. Pairwise log-ratios of elements are then calculated of these points, and curvature of the log-ratio pairs is computed. High curvature indicates abrupt spatial changes, which could point at locations of mineralized zones. A measure called c-value evaluates the overall curvature and thus serves as an importance measure of a log-ratio pair. The methodology is tested on two real surface geochemical data sets collected in areas with known underlying mineralization, and the results confirm existing knowledge of the underlying mineralization. [ABSTRACT FROM AUTHOR]

Even though the logratio methodology provides a range of both generic, mostly exploratory, and purpose-built coordinate representations of compositional data, simple pairwise logratios are preferred by many for multivariate analysis in the geochemical practice, principally because of their simpler interpretation. However, the logratio coordinate systems that incorporate them are predominantly oblique, resulting in both conceptual and practical problems. We propose a new approach, called backwards pivot coordinates, where each pairwise logratio is linked to one orthogonal coordinate system, and these systems are then used together to produce a concise output. In this work, principal component analysis and regression with compositional explanatory variables are used as primary methods to demonstrate the methodological and interpretative advantages of the proposal. In the applied part of this study, sediment compositions from the Jizera River, Czech Republic, were analysed using these techniques through backwards pivot coordinates. This allowed us to discuss grain size control of the element composition of sediments and clearly distinguish anthropogenically contaminated and uncontaminated strata in sediment depth profiles. [ABSTRACT FROM AUTHOR]

Detecting subcropping mineralizations but also deeply buried mineralizations is one important goal in geochemical exploration. The identification of useful indicators for mineralization is a difficult task, as mineralization might be influenced by many factors, including location, investigated media and depth. Here, a statistical method is proposed which indicates chemical elements related to mineralization along a transect. Moreover, the method determines the potential area of the deposit along a transect. The identification is based on general additive models (GAMs) for the element concentrations across the spatial coordinate(s). The log-ratios of the GAM fits are taken to compute the curvature, where high and narrow curvature is supposed to indicate the mineralization area. By defining a measure for the quantification of high curvature, the log-ratios can be ranked, and elements can be identified that are indicative of the anomaly patterns. [ABSTRACT FROM AUTHOR]

Benford's law became a prevalent concept for fraud and anomaly detection. It examines the frequencies of the leading digits of numbers in a collection of data and states that the leading digit is most often 1, with diminishing frequencies up to 9. In this paper we propose a multivariate approach to test whether the observed frequencies follow the theoretical Benford distribution. Our approach is based on the concept of compositional data, which examines the relative information between the frequencies of the leading digits. As a result, we introduce a multivariate test for Benford distribution. In simulation studies and examples we compare the multivariate test performance to the conventional chi-square and Kolmogorov-Smirnov test, where the multivariate test turns out to be more sensitive in many cases. A diagnostics plot based on relative information allows to reveal and interpret the possible deviations from the Benford distribution. [ABSTRACT FROM AUTHOR]

Many geological phenomena are regularly measured over time to follow developments and changes. For many of these phenomena, the absolute values are not of interest, but rather the relative information, which means that the data are compositional time series. Thus, the serial nature and the compositional geometry should be considered when analyzing the data. Multivariate time series are already challenging, especially if they are higher dimensional, and latent variable models are a popular way to deal with this kind of data. Blind source separation techniques are well-established latent factor models for time series, with many variants covering quite different time series models. Here, several such methods and their assumptions are reviewed, and it is shown how they can be applied to high-dimensional compositional time series. Also, a novel blind source separation method is suggested which is quite flexible regarding the assumptions of the latent time series. The methodology is illustrated using simulations and in an application to light absorbance data from water samples taken from a small stream in Lower Austria. [ABSTRACT FROM AUTHOR]

Negative correlations between elements, molecules, or minerals can indicate a variety of geochemical processes, such as ion exchange, incongruent mineral precipitation/dissolution, and redox reactions. However, compositional data (those composed of relative parts) can also exhibit negative correlations simply due to displacement of one part by another, such as the addition of sodium chloride lowering the concentration of all ions other than sodium and chloride in a solution. Apart from this practical problem, the question is more general: how to address the relationships between components in data carrying relative information. For this purpose, the symmetric pivot coordinates were developed which allow for the identification of both positive and negative correlations between two parts in compositional data in terms of their relative dominance to the other parts. Accordingly, the symmetric pivot coordinate approach aggregates all of the logratios with those two parts of interest. This may not be desirable if data quality problems occur, because such parts would contribute the same weight to the coordinate as parts with good data quality. As a way out, the new method of weighted symmetric coordinates focusing on pairwise associations is proposed. In this approach, variables with large logratio variances are down-weighted to suppress their effect on the remaining variables, which is also demonstrated in a small simulation study. Finally, the weighted symmetric pivot coordinates are applied to chemistry data from a series of waste water samples from oil and gas wells produced from the lower Eagle Ford Group in the U.S. Gulf Coast Basin. In particular, strong negative correlations between ions are examined using this method to reveal processes which occur as a function of depth, including clay diagenesis, de-dolomitization, kerogen maturation, and sulfate and carbonate mineral saturation. [ABSTRACT FROM AUTHOR]

Copyright of BHM Berg- und Hüttenmännische Monatshefte is the property of Springer Nature and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The receiver operating characteristic (ROC) curve is a graphical method commonly used to study the capacity of continuous variables (markers) to properly classify subjects into one of two groups. The decision made is ultimately endorsed by a classification subset on the space where the marker is defined. In this paper, we study graphical representations and propose visual forms to reflect those classification rules giving rise to the construction of the ROC curve. On the one hand, we use static pictures for displaying the classification regions for univariate markers, which are specially convenient when there is not a monotone relationship between the marker and the likelihood of belonging to one group. In those cases, there are two options to improve the classification accuracy: to allow for more flexibility in the classification rules (for example considering two cutoff points instead of one) or to transform the marker by using a function whose resulting ROC curve is optimal. On the other hand, we propose to build videos for visualizing the collection of subsets when several markers are considered simultaneously. A compilation of techniques for finding a rule that maximizes the area under the ROC curve is included, with a focus on linear combinations. We present a tool for the R software which generates those graphics, and we apply it to one real dataset. The R code is provided as Supplementary Material. [ABSTRACT FROM AUTHOR]

Outliers are encountered in all practical situations of data analysis, regardless of the discipline of application. However, the term outlier is not uniformly defined across all these fields since the differentiation between regular and irregular behaviour is naturally embedded in the subject area under consideration. Generalized approaches for outlier identification have to be modified to allow the diligent search for potential outliers. Therefore, an overview of different techniques for multivariate outlier detection is presented within the scope of selected kinds of data frequently found in the field of geosciences. In particular, three common types of data in geological studies are explored: spatial, compositional and flat data. All of these formats motivate new outlier concepts, such as local outlyingness, where the spatial information of the data is used to define a neighbourhood structure. Another type are compositional data, which nicely illustrate the fact that some kinds of data require not only adaptations to standard outlier approaches, but also transformations of the data itself before conducting the outlier search. Finally, the very recently developed concept of cellwise outlyingness, typically used for high-dimensional data, allows one to identify atypical cells in a data matrix. In practice, the different data formats can be mixed, and it is demonstrated in various examples how to proceed in such situations. [ABSTRACT FROM AUTHOR]

We propose a robust and sparse classification method based on the optimal scoring approach. It is also applicable if the number of variables exceeds the number of observations. The data are first projected into a low dimensional subspace according to an optimal scoring criterion. The projection only includes a subset of the original variables (sparse modeling) and is not distorted by outliers (robust modeling). In this low dimensional subspace classification is performed by minimizing a robust Mahalanobis distance to the group centers. The low dimensional representation of the data is also useful for visualization purposes. We discuss the algorithm for the proposed method in detail. A simulation study illustrates the properties of robust and sparse classification by optimal scoring compared to the non-robust and/or non-sparse alternative methods. Three real data applications are given. [ABSTRACT FROM AUTHOR]

Correlation coefficients are most popular in statistical practice for measuring pairwise variable associations. Compositional data, carrying only relative information, require a different treatment in correlation analysis. For identifying the association between two compositional parts in terms of their dominance with respect to the other parts in the composition, symmetric balances are constructed, which capture all relative information in the form of aggregated logratios of both compositional parts of interest. The resulting coordinates have the form of logratios of individual parts to a (weighted) 'average representative' of the other parts, and thus, they clearly indicate how the respective parts dominate in the composition on average. The balances form orthonormal coordinates, and thus, the standard correlation measures relying on the Euclidean geometry can be used to measure the association. Simulation studies provide deeper insight into the proposed approach, and allow for comparisons with alternative measures. An application from geochemistry (Kola moss) indicates that correlations based on symmetric balances serve as a sensitive tool to reveal underlying geochemical processes. [ABSTRACT FROM AUTHOR]

The log ratio methodology converts compositional data, such as concentrations of chemical elements in a rock, from their original Aitchison geometry to interpretable real orthonormal coordinates, thereby allowing meaningful statistical processing and visualization. However, it must be taken into account that the original concentrations can be flawed by detection limit or imprecision problems that can severely affect the resulting coordinates. This paper aims to construct such orthonormal log ratio coordinates, called weighted pivot coordinates, that capture the relevant relative information about an original component and treat the redundant information in a controlled manner. Theoretical developments are supported by a thorough simulation study. Weighted pivot coordinates are then applied to the geochemical mapping of catchment outlet sediments from the National Geochemical Survey of Australia illustrating their advantage over possible alternatives. [ABSTRACT FROM AUTHOR]

Long-term changes of plant phenological phases determined by complex interactions of environmental factors are in the focus of recent climate impact research. There is a lack of studies on the comparison of biogeographical regions in Europe in terms of plant responses to climate. We examined the flowering phenology of plant species to identify the spatio-temporal patterns in their responses to environmental variables over the period 1970-2010. Data were collected from 12 countries along a 3000-km-long, North-South transect from northern to eastern Central Europe. Biogeographical regions of Europe were covered from Finland to Macedonia. Robust statistical methods were used to determine the most influential factors driving the changes of the beginning of flowering dates. Significant species-specific advancements in plant flowering onsets within the Continental (3 to 8.3 days), Alpine (2 to 3.8 days) and by highest magnitude in the Boreal biogeographical regions (2.2 to 9.6 days per decades) were found, while less pronounced responses were detected in the Pannonian and Mediterranean regions. While most of the other studies only use mean temperature in the models, we show that also the distribution of minimum and maximum temperatures are reasonable to consider as explanatory variable. Not just local (e.g. temperature) but large scale (e.g. North Atlantic Oscillation) climate factors, as well as altitude and latitude play significant role in the timing of flowering across biogeographical regions of Europe. Our analysis gave evidences that species show a delay in the timing of flowering with an increase in latitude (between the geographical coordinates of 40.9 and 67.9), and an advance with changing climate. The woody species (black locust and small-leaved lime) showed stronger advancements in their timing of flowering than the herbaceous species (dandelion, lily of the valley). In later decades (1991-2010), more pronounced phenological change was detected than during the earlier years (1970-1990), which indicates the increased influence of human induced higher spring temperatures in the late twentieth century. [ABSTRACT FROM AUTHOR]

Compositional data, as they typically appear in geochemistry in terms of concentrations of chemical elements in soil samples, need to be expressed in log-ratio coordinates before applying the traditional statistical tools if the relative structure of the data is of primary interest. There are different possibilities for this purpose, like centered log-ratio coefficients, or isometric log-ratio coordinates. In both the approaches, geometric means of the compositional parts are involved, and it is unclear how measurement errors or detection limit problems affect their presentation in coordinates. This problem is investigated theoretically by making use of the theory of error propagation. Due to certain limitations of this approach, the effect of error propagation is also studied by means of simulations. This allows to provide recommendations for practitioners on the amount of error and on the expected distortion of the results, depending on the purpose of the analysis. [ABSTRACT FROM AUTHOR]

In regional geochemistry rock, sediment, soil, plant or water samples, collected in a certain region, are analyzed for concentrations of chemical elements. The observations are thus usually high dimensional, spatially dependent and of compositional nature. In this paper, a novel blind source separation approach for spatially dependent data is suggested. For the analysis, it is assumed that the multivariate observations are linear combinations or mixtures of latent components and that the spatial processes for these latent components are second order stationary and uncorrelated. In the present approach, the latent components are then recovered by simultaneously diagonalizing the covariance matrix and a local covariance (correlation) matrix. This method can be easily applied also in the context of compositional data after appropriate data transformations. The components obtained in this way are uncorrelated and easily interpretable, and can be used for dimension reduction and for visual presentation of different features of the data. To demonstrate the usefulness of the new method, the KOLA data are reanalyzed using the new procedure and the results are compared to the results coming from marginal principal component analysis and independent component analysis that ignore spatial dependence. [ABSTRACT FROM AUTHOR]

The Mahalanobis distance between pairs of multivariate observations is used as a measure of similarity between the observations. The theoretical distribution is derived, and the result is used for judging on the degree of isolation of an observation. In case of spatially dependent data where spatial coordinates are available, different exploratory tools are introduced for studying the degree of isolation of an observation from a fraction of its neighbors, and thus to identify local multivariate outliers. [ABSTRACT FROM AUTHOR]

The estimation of a mean of a proportion is a frequent task in statistical survey analysis, and often such ratios are estimated from compositions such as income components, wage components, tax components, etc. In practice, the weighted arithmetic mean is regularly used to estimate the center of the data. However, this estimator is not appropriate if the ratios are estimated from compositions, because the sample space of compositional data is the simplex and not the usual Euclidean space. We demonstrate that the weighted geometric mean is useful for this purpose. Even for different sampling designs, the weighted geometric mean shows excellent behavior. [ABSTRACT FROM AUTHOR]

Omitting variables in compositional data analysis may lead to a substantial change in results from that of multivariate statistical analysis. In particular, this is the case for principal component analysis and the compositional biplot, where both the interpretation of loadings and scores of the remaining subcomposition are affected. A stepwise procedure is introduced that allows for a reduction of the original composition to a subcomposition by avoiding a substantial change of the information, like those carried by the compositional biplot. The subcomposition is easier to handle and interpret. Numerical results give evidence of the usefulness of the procedure. [ABSTRACT FROM AUTHOR]

Compositional data, i.e. data including only relative information, need to be transformed prior to applying the standard discriminant analysis methods that are designed for the Euclidean space. Here it is investigated for linear, quadratic, and Fisher discriminant analysis, which of the transformations lead to invariance of the resulting discriminant rules. Moreover, it is shown that for robust parameter estimation not only an appropriate transformation, but also affine equivariant estimators of location and covariance are needed. An example and simulated data demonstrate the effects of working in an inappropriate space for discriminant analysis. [ABSTRACT FROM AUTHOR]

The L-median is a robust estimator of multivariate location with good statistical properties. Several algorithms for computing the L-median are available. Problem specific algorithms can be used, but also general optimization routines. The aim is to compare different algorithms with respect to their precision and runtime. This is possible because all considered algorithms have been implemented in a standardized manner in the open source environment R. In most situations, the algorithm based on the optimization routine NLM (non-linear minimization) clearly outperforms other approaches. Its low computation time makes applications for large and high-dimensional data feasible. [ABSTRACT FROM AUTHOR]

Object: The goal of this study was to develop a comprehensive magnetic resonance (MR) data analysis framework for handling very large datasets with user-friendly tools for parallelization and to provide an example implementation. Materials and methods: Commonly used software packages (AFNI, FSL, SPM) were connected via a framework based on the free software environment R, with the possibility of using Nvidia CUDA GPU processing integrated for high-speed linear algebra operations in R. Three hundred single-subject datasets from the 1,000 Functional Connectomes project were used to demonstrate the capabilities of the framework. Results: A framework for easy implementation of processing pipelines was developed and an R package for the example implementation of Fully Exploratory Network ICA was compiled. Test runs on data from 300 subjects demonstrated the computational advantages of a processing pipeline developed using the framework compared to non-parallelized processing, reducing computation time by a factor of 15. Conclusion: The feasibility of computationally intensive exploratory analyses allows broader access to the tools for discovery science. [ABSTRACT FROM AUTHOR]

The complexity of water bodies in the eu-potamal river corridor and the main delta channels of the Romanian Danube is exemplified by the macrophyte vegetation. Two hypotheses provided the background for our study: (a) is the macrophyte vegetation of large, permanently connected branches significantly separated from that of the main river channel; (b) is the macrophyte composition of the Danube main stem significantly altered when the river divides into the three large navigable Delta channels. Water bodies considered were two contiguous sections of the main river channel, two large branches remaining from the historical floodplain, and the three main Delta channels. We quantified macrophyte diversity and floristic variation. Our data set was prepared from the MIDCC-project data base, in which macrophyte occurrence, abundance and habitat parameters are stored for contiguous survey units of the whole Danube river corridor. Field survey method followed that of Kohler and the European Standard EN14184. Results confirmed our first hypothesis: permanently connected side branches still support significantly different macrophyte assemblages, making them important indicators of floodplain connectivity. The diversion of the Danube into its three large navigable delta channels significantly alters the macrophyte vegetation from the c. 300 km of main stem up-river, substantially supporting our second hypothesis. Our results largely enhance the knowledge on aquatic plant biodiversity in the eu-potamal Danube, forming a solid base for long-term studies. We also discuss the relevance of our results regarding the ecological, as well as the conservational, quality of rivers and their floodplains. [ABSTRACT FROM AUTHOR]