Search

The aim of this study is to understand how prevalence—the ratio of instances of presence to total sample size—affects the estimation of three discrimination indexes commonly used in distribution modelling: the area under the receiver operating characteristic curve (AUC), the value of sensitivity at the threshold where sensitivity equals specificity (Se*), and the maximum value of the Youden index or true skill statistic (Y). For four sample size levels, samples of suitability scores for the instances of presence and absence with varying prevalences were simulated from known distributions so that the true values of the discrimination indexes were known, and the three indexes were empirically estimated (AUCest, Se*est, Yest). AUCest and Se*est are unbiased estimators, and the greatest precision is achieved with a balanced prevalence. As sample size increases, there is a larger prevalence interval around 0.5 in which precision is more or less stable. As a rule of thumb, in the case of n ≤ 100, at least ten observations of the rare state (either instances of presence or absence) should be considered, whereas the safety prevalence interval [0.01, 0.99] should be used for higher sample sizes. The lower the true discriminative power of the models, the higher the negative effect of prevalence. Yest is positively biased, and bias and precision become worse towards low and high prevalences. Highly unbalanced prevalences increase the imprecision in estimating the discrimination capacity of the models. Y is not recommended as a discrimination measure since it provides overoptimistic results. [ABSTRACT FROM AUTHOR]

Differences between climatic databases have been reported to alter the spatial predictions of species distribution models (SDM). In the present study, the global WorldClim v.2 database (WC) and the regional Iberian Climate Atlas (ICA) were compared in the geographical context of the Iberian Peninsula. Six climatic variables were considered: BIO1, BIO5 and BIO6 (temperature-related variables) and BIO12, BIO13 and BIO14 (precipitation-related variables). We performed regression analyses between values for each pair of homologous variables and generated quantile–quantile plots to compare the distribution of ranges within 10 × 10 grid cells. Pearson correlations were used to determine whether absolute differences between homologue variables were related to elevation. We modelled the occurrence of 48 woody plant species using either WC or ICA variables, and tested for differences in the estimated suitability values, discrimination power and importance of variables. Precipitation values varied considerably between databases, with WC variables reaching lower maximum and less variable values than ICA. Regarding temperature values, BIO1 had the highest correlation value between both datasets, whereas we observed substantial differences in the case of BIO5, which showed consistently lower values in WC than in ICA. Higher discrepancies between datasets, especially for temperature variables, were found in high elevation areas. As regards distribution models, the climate data source affected estimated suitability values, discrimination capacity and estimated variable importance. In addition, the rarer the species, the higher the uncertainty associated with the climate source. Climate data source is another uncertainty factor to add to all those that have already been highlighted in SDM. [ABSTRACT FROM AUTHOR]

Species distribution modelling has become a common approach in ecology in the last decades. As in any modelling exercise, evaluation of the predicted suitability surfaces is a key process, and the area under the receiver operating characteristic (ROC) curve (AUC) has become the most popular statistic for this purpose. A close covariation between the AUC and threshold-dependent discrimination measures (sensitivity Se and specificity Sp) raises into question the advantage of the threshold-independence of the AUC. In this study, the relationship between the AUC and several threshold-dependent discrimination measures is characterized in detail, and the sensitivity of the pattern to variations in the shape of the ROC curve is assessed. Hypothetical suitability values, coming from normal and skew-normal distributions, were simulated for both instances of presence and absence. The flexibility of the skew-normal distribution allowed for the simulation of a wide range of ROC curve configurations. The relationship between the AUC and threshold-dependent measures was graphically assessed; independently of the ROC curve shape, a nonlinear asymptotic relationship between the AUC and Se (and Sp) was obtained after applying the threshold that makes Se = Sp. A nonlinear asymptotic relationship between the AUC and the Youden index was also reported. These results imply that the AUC does not appropriately measure changes in the discrimination of models, and it is especially incapable of distinguishing between models with high discrimination capacity. Se or Sp derived from the application of the threshold that makes them equal is a preferred measure of discrimination power. Together with the rate of false positives and negatives, and with the prevalence of the species, these statistics provide more information about the discrimination capacity of the models than the AUC. [ABSTRACT FROM AUTHOR]