Classification Methods, Reduced Datasets and Quality Analysis Applications

Modern industrial production lines are characterized by rapid dynamics, high noise levels, and low knowledge of the underlying physical phenomena. In these situations, inductive learning methods allow the system designer to infer a model of the relevant process phenomena by extracting information from experimental data. A wide range of inductive learning methods is available to the system designer, potentially ensuring different levels of accuracy on different problem domains. In this paper we consider the problem of designing an inductive classification system with optimal accuracy when domain knowledge is limited and the number of available experiments is small. By analyzing the formal properties of consistent learning methods and of accuracy estimators, we wish to convey to the reader the message that the common practice of aggressively pursuing error minimization with different training algorithms and classification families is unjustified. Our position is illustrated by analyzing a classification problem with industrial relevance.