Search

Shortcomings of the correlation coefficient (Pearson's) as a measure for estimating and calculating the accuracy of predictive model properties are analysed. Here we discuss two such cases that can often occur in the application of the model in predicting properties of a new external set of compounds. The first problem in using the correlation coefficient is its insensitivity to the systemic error that must be expected in predicting properties of a novel external set of compounds, which is not a random sample selected from the training set. The second problem is that an external set can be arbitrarily large or small and have an arbitrary and uneven distribution of the measured value of the target variable, whose values are not known in advance. In these conditions, the correlation coefficient can be an overoptimistic measure of agreement of predicted values with the corresponding experimental values and can lead to a highly optimistic conclusion about the predictive ability of the model. Due to these shortcomings of the correlation coefficient, the use of standard error (root-mean-square-error) of prediction is suggested as a better quality measure of predictive capabilities of a model. In the case of classification models, the use of the difference between the real accuracy and the most probable random accuracy of the model shows very good characteristics in ranking different models according to predictive quality, having at the same time an obvious interpretation. [ABSTRACT FROM AUTHOR]

In the literature there can be found a wide collection of correlation and association coefficients used for different structures of data. Generally, some of the correlation coefficients are conventionally used for continuous data and others for categorical or ordinal observations. The aim of this paper is to verify the performance of various approaches to correlation coefficient estimation for several types of observations. Both simulated and real data were analysed. For continuous variables, Pearson's r2 and MIC were determined, whereas for categorized data three approaches were compared: Cramér's V, Joe's estimator, and the regression-based estimator. Two method of discretization for continuous data were used. The following conclusions were drawn: the regression-based approach yielded the best results for data with the highest assumed r2 coefficient, whereas Joe's estimator was the better approximation of true correlation when the assumed r2 was small; and the MIC estimator detected the maximal level of dependency for data having a quadratic relation. Moreover, the discretization method applied to data with a non-linear dependency can cause loss of dependency information. The calculations were supported by the R packages arules and minerva. [ABSTRACT FROM AUTHOR]

We study the problem of nonparametric dependence detection. Many existing methods may suffer severe power loss due to nonuniform consistency, which we illustrate with a paradox. To avoid such power loss, we approach the nonparametric test of independence through the new framework of binary expansion statistics (BEStat) and binary expansion testing (BET), which examine dependence through a novel binary expansion filtration approximation of the copula. Through a Hadamard transform, we find that the symmetry statistics in the filtration are complete sufficient statistics for dependence. These statistics are also uncorrelated under the null. By using symmetry statistics, the BET avoids the problem of nonuniform consistency and improves upon a wide class of commonly used methods (a) by achieving the minimax rate in sample size requirement for reliable power and (b) by providing clear interpretations of global relationships upon rejection of independence. The binary expansion approach also connects the symmetry statistics with the current computing system to facilitate efficient bitwise implementation. We illustrate the BET with a study of the distribution of stars in the night sky and with an exploratory data analysis of the TCGA breast cancer data. for this article are available online. [ABSTRACT FROM AUTHOR]

Medical studies often involve a comparison between two outcomes, each collected from a sample. The probability associated with, and confidence in the result of the study is of most importance, since one may argue that having been wrong with a percent could be what killed a patient. Sampling is usually done from a finite and discrete population and it follows a Bernoulli trial, leading to a contingency of two binomially distributed samples (better known as 2×2 contingency table). Current guidelines recommend reporting relative measures of association (such as the relative risk and odds ratio) in conjunction with absolute measures of association (which include risk difference or excess risk). Because the distribution is discrete, the evaluation of the exact confidence interval for either of those measures of association is a mathematical challenge. Some alternate scenarios were analyzed (continuous vs. discrete; hypergeometric vs. binomial), and in the main case—bivariate binomial experiment—a strategy for providing exact p-values and confidence intervals is proposed. Algorithms implementing the strategy are given.

For the analysis of square contingency tables with ordered categories, a measure was developed to represent the degree of departure from the conditional symmetry model in which there is an asymmetric structure of the cell probabilities with respect to the main diagonal of the table. The present paper proposes a novel measure for the departure from conditional symmetry based on the cumulative probabilities from the corners of the square table. In a given example, the proposed measure is applied to Japanese occupational status data, and the interpretation of the proposed measure is illustrated as the departure from a proportional structure of social mobility.

A log-linear analysis is a method providing a comprehensive scheme to describe the association for categorical variables in a contingency table. The log-linear model specifies how the expected counts depend on the levels of the categorical variables for these cells and provide detailed information on the associations. The aim of this paper is to present theoretical, as well as empirical, aspects of ordinal log-linear models used for contingency tables with ordinal variables. We introduce log-linear models for ordinal variables: linear-by-linear association, row effect model, column effect model and RC Goodman’s model. Algorithm, advantages and disadvantages will be discussed in the paper. An empirical analysis will be conducted with the use of R.

Numerical weather predictions (NWP) play a fundamental role in air quality management. The transport and deposition of all the pollutants (natural and/or anthropogenic) present in the atmosphere are strongly influenced by meteorological conditions such as, for example, precipitation and winds. Furthermore, the presence of particulate matter in the atmosphere favors the physical processes of nucleation of the hydrometeors, thus increasing the risk of even extreme weather events. In this framework of reference, the present work aimed to improve the quality of weather forecasts related to extreme events through the optimization of the weather research and forecasting (WRF) model. For this purpose, the simulation results obtained using the WRF model, where physical parametrizations of the cumulus scheme can be optimized, are reported. As a case study, we considered the extreme meteorological event recorded on 25 November 2016, which affected the whole territory of Sicily and, in particular, the area of Sciacca (Agrigento). In order, to evaluate the performance of the proposed approach, we compared the WRF model outputs with data obtained by a network of radar and weather stations. The comparison was performed through statistical methods on the basis of a “contingency table”, which allowed for ascertaining the best suited physical parametrizations able to reproduce this event.

In in most cases, the distribution of communications is unknown and one may summarize social network communications with categorical attributes in a contingency table. Due to the categorical nature of the data and a large number of features, there are many parameters to be considered and estimated in the model. Hence, the accuracy of estimators decreases. To overcome the problem of high dimensionality and unknown communications distribution, multiple correspondence analysis is used to reduce the number of parameters. Then the rescaled data are studied in a Dirichlet model in which the parameters should be estimated. Moreover, two control charts, Hotelling's T2 and multivariate exponentially weighted moving average (MEWMA), are developed to monitor the parameters of the Dirichlet distribution. The performance of the proposed method is evaluated through simulation studies in terms of average run length criterion. Finally, the proposed method is applied to a real case. [ABSTRACT FROM AUTHOR]

Processes with multiple correlated categorical quality characteristics are called multivariate categorical processes. These processes are usually shown by contingency tables and are characterized by log‐linear models. In this paper, two monitoring approaches including likelihood ratio test (LRT) and F test are developed to monitor multivariate categorical processes based on the contingency table in Phase I. In addition, a change point estimator for multivariate categorical processes is developed in Phase I. The performances of the two proposed approaches are evaluated in terms of probability of signal, while the performance of the proposed change point estimator is evaluated in terms of accuracy and precision criteria through simulation experiments. Meanwhile, we compare the performance of the two proposed control charts with an existing control chart called "−2LRT" control chart for multivariate categorical processes. In the end, a typical application of the proposed methods is illustrated in a real‐world health care system. [ABSTRACT FROM AUTHOR]

Agriculture is the dynamic axis of the productive matrix of the Ecuadorian economy. From studies on the transformation of the raw material in the different productive processes in the agricultural sector shortcomings have been detected, that lead to problems such as the search for ways to increase the added value in the production of tropical fruits to guarantee the productive processes by the producers of Canton Quevedo. Therefore, the aim of this research is to carry out a study to design a strategy for the generation of added value in the production of tropical fruits of Canton Quevedo. The study was carried out based on surveys applied to the producers and partners of APROBANEC, and the citizens of the canton. A representative random sample was selected to apply the survey. Other statistical methods were also used, such as the contingency table. The result of the study showed that the situation is negative, thus measures are proposed to reverse this situation. [ABSTRACT FROM AUTHOR]

Latent count models constitute an important modeling class in which a latent vector of counts, z, is summarized or corrupted for reporting, yielding observed data y=Tz where T is a known but non‐invertible matrix. The observed vector y generally follows an unknown multivariate distribution with a complicated dependence structure. Latent count models arise in diverse fields, such as estimation of population size from capture‐recapture studies; inference on multi‐way contingency tables summarized by marginal totals; or analysis of route flows in networks based on traffic counts at a subset of nodes. Currently, inference under these models relies primarily on stochastic algorithms for sampling the latent vector z, typically in a Bayesian data‐augmentation framework. These schemes involve long computation times and can be difficult to implement. Here, we present a novel maximum‐likelihood approach using likelihoods constructed by the saddlepoint approximation. We show how the saddlepoint likelihood may be maximized efficiently, yielding fast inference even for large problems. For the case where z has a multinomial distribution, we validate the approximation by applying it to a specific model for which an exact likelihood is available. We implement the method for several models of interest, and evaluate its performance empirically and by comparison with other estimation approaches. The saddlepoint method consistently gives fast and accurate inference, even when y is dominated by small counts. [ABSTRACT FROM AUTHOR]

The paper is a statistical analysis of some indicators that characterize the environment of jobseekers registered in the Employment Offces in Albania. The data being analysed relates to the period January 2017 - June 2019. The analysis is done on the data from the NES database (National Employment Service). A careful description is intended to reveal typical features and to recognize the situation and evolution over the years. Assessment combines a total analysis with a comparative analysis with indicators that affect the employability of the individuals. [ABSTRACT FROM AUTHOR]