Choosing the best metrics for quantifying the quality of the model in skewed binary classification problems.

Evaluating the performance of a predictive model trained on an imbalanced class distribution has encountered significant challenges as most of the widely used metrics are designed considering balanced class distribution datasets. This paper presents a study on four different biasing of target class variables in train set: (50% majority class, 50% minority class), (60% majority class, 40% minority class), (70% majority class, 30% minority class), (80% majority class, 20% minority class), and focuses on choosing the right metrics out of accuracy, precision, recall, f1 score and AUC score for a binary classification problem with a skewed class distribution. The algorithms considered are Logistic Regression, K Nearest Neighbors, Naïve Bayes. Analysis of all four biasing cases, five evaluation metrics for three algorithms have been presented in this paper. [ABSTRACT FROM AUTHOR]