Search

The most widely spread measure of performance, accuracy, suffers from a paradox: predictive models with a given level of accuracy may have greater predictive power than models with higher accuracy. Despite optimizing classification error rate, high accuracy models may fail to capture crucial information transfer in the classification task. We present evidence of this behavior by means of a combinatorial analysis where every possible contingency matrix of 2, 3 and 4 classes classifiers are depicted on the entropy triangle, a more reliable information-theoretic tool for classification assessment. Motivated by this, we develop from first principles a measure of classification performance that takes into consideration the information learned by classifiers. We are then able to obtain the entropy-modulated accuracy (EMA), a pessimistic estimate of the expected accuracy with the influence of the input distribution factored out, and the normalized information transfer factor (NIT), a measure of how efficient is the transmission of information from the input to the output set of classes. The EMA is a more natural measure of classification performance than accuracy when the heuristic to maximize is the transfer of information through the classifier instead of classification error count. The NIT factor measures the effectiveness of the learning process in classifiers and also makes it harder for them to "cheat" using techniques like specialization, while also promoting the interpretability of results. Their use is demonstrated in a mind reading task competition that aims at decoding the identity of a video stimulus based on magnetoencephalography recordings. We show how the EMA and the NIT factor reject rankings based in accuracy, choosing more meaningful and interpretable classifiers. Francisco José Valverde-Albacete has been partially supported by EU FP7 project LiMoSINe (contract 288024): www.limosine-project.eu Carmen Peláez Moreno has been partially supported by the Spanish Government-Comisión Interministerial de Ciencia y Tecnología project TEC2011–26807.

A crucial insight into handwriting dynamics is embodied in the idea that stable, robust handwriting movements correspond to attractors of an oscillatory dynamical system. We present a phase dynamic model of visuomotor performance involved in copying simple oriented lines. Our studies on human performance in copying oriented lines revealed a systematic error pattern in orientation of drawn lines, i.e., lines at certain orientation are drawn more accurately than at other values. Furthermore, human subjects exhibit "flips" in direction at certain characteristic orientations. It is argued that this flipping behavior has its roots in the fact that copying process is inherently ambiguous-a line of given orientation may be drawn in two different (mutually opposite) directions producing the same end result. The systematic error patterns seen in human copying performance is probably a result of the attempt of our visuomotor system to cope with this ambiguity and still be able to produce accurate copying movements. The proposed nonlinear phase-dynamic model explains the experimentally observed copying error pattern and also the flipping behavior with remarkable accuracy. � 2009 Springer-Verlag.

Using a single asymmetric relay feedback test, a method is proposed to identify four parameters of a transfer function model. The proposed method is used to identify all the parameters in a second order plus time delay model (SOPTD). The parameters estimated are of adequate accuracy for designing suitable controllers. The estimated SOPTD models have a step response behavior matching that of the actual process. The method can also be used for identifying open loop unstable transfer function models. For unstable systems, the closed loop step responses are compared. Simulation results are given for four case studies. ? 2003 Elsevier Ltd. All rights reserved.