Search

Due to importance of measuring the degree of resemblance, the similarity measure is widely adopted in various areas of the information systems (e.g., medical informatics and information retrieval) and in several applications like medical diagnostic, image processing, and pattern recognition. However, most of the existing similarity measures focus mainly on the degree of similarity without consulting expert(s) about the results. In this paper, an efficient tool for measuring similarity and agreement of objects that embeds experts’ opinions is proposed to assess similarity among features and agreement of opinions among experts. To obtain such robust measuring tool, three construction steps were followed. Firstly, adapting soft expert set as a general structure that consists of four components: objects, attributes, experts, and experts’ opinions. Secondly, representing the soft expert set, without losing stored information, in such a way as to fit the proposed similarity-agreement measure and make it simpler and more meaningful than the similar existing measures. Thirdly, axiomatizing the similarity-agreement measure for the case of two experts to simplify the model. Further, a diagnostic prediction application and its algorithm is discussed in this context, along with analysis of the experimental results. Analysis of performance of the proposed similarity-agreement measure revealed that it has high accuracy, sensitivity, and value of the F-measure and that it has better performance than existing state-of-the-art tools.

Agreement measures are useful metrics to both compare different evaluations of the same diagnostic outcomes and validate new rating systems or devices. While many of them have been proposed in the literature so far, Cohen’s n is still the de facto standard in gauging the agreement. Information Agreement (IA) is a novel two-observers information-theoretic-based metric introduced to overcome all the limitations and alleged pitfalls of Cohen’s n. It offers an operative meaning to the agreement since it measures – in both dichotomous and multi-value ordered-categorical cases-the information shared between two raters through the virtual diagnostic channel connecting them: the more information exchanged between the raters, the higher their agreement. Unfortunately, this measure is only able to deal with agreement matrices whose values are all strictly positive numbers. This work extends IA by admitting also 0 as a possible value for the entries of an agreement matrix. Moreover, a Python software library to compute the extended version of IA, together with some of the most used agreement measures, is presented and tested.