Örneklem Büyüklüğünün Tahmin-Doğrulama Metrikleri Üzerindeki Etkisinin İncelenmesi: Bir Simülasyon Çalışması.

Objective: The aim of this study is to introduce commonly used forecast-verification metrics and compare their performance for different sample sizes. Material and Methods: Forecast verification metrics are widely used as decision support tools in various scientific disciplines. If the prediction results are binary, metrics can be used to evaluate the discriminative power for prediction verification. These metrics are called thresholddependent metrics. In order to show the effect of different sample sizes on the performance of forecast-verification metrics, a simulation study was conducted considering nine different commonly used threshold-based metrics. Using the Python-random library, data were obtained for 35 different n values in the range of 10 ≤ n ≤ 1000. For the performance evaluation, the values and interpretation levels recommended in the literature for the Kappa coefficient were taken into account. Results: From the results of this study, where different sample sizes were considered, it was found that the effect of increasing or decreasing the sample size on forecast verification was almost constant. It was observed that for all sample sizes considered, around 50 percent of the estimates had "none or none to low" levels of interpretation for almost all metrics. When the metrics were ranked in terms of liberality by considering fair, moderate, substantial and perfect levels of verification together, the order was obtained as F, Odds Ratio Skill Score, Critical Success Index, Peirce Skill Score, Clayton Skill Score, Prediction Skill Index, Heidke Skill Score, Kappa and Gi lbert Skill Score. Conclusion: Forecast-verification metrics are more affected by the distribution of observations across cells in 2x2 crosstabs than by sample size. [ABSTRACT FROM AUTHOR]