Search

Pedro Romero-Aroca,1 Marc Baget-Bernaldiz,1 Raul Navarro-Gil,1 Albert Feliu,2 Najla Maarof,3 Antonio Moreno,3 Julian Cristiano,3 Aida Valls3 1Ophthalmology Department, University Hospital Sant Joan, Institute of Health Research Pere Virgili (IISPV), Universitat Rovira & Virgili, Tarragona, Spain; 2Pediatric Department, University Hospital Sant Joan, Institute of Health Research Pere Virgili (IISPV), Universitat Rovira & Virgili, Tarragona, Spain; 3Department of Computer Engineering and Mathematics, Universitat Rovira & Virgili, ITAKA-Intelligent Technologies for Advanced Knowledge Acquisition, Tarragona, SpainCorrespondence: Pedro Romero-Aroca, Hospital Universitario Sant Joan de Reus, Avda. Dr. Josep Laporte 2, Reus, 43204, Spain, Email romeropere@gmail.comAim: The aim of the present study was to build a clinical decision support system (CDSS) that can predict the presence of diabetic retinopathy (DR) in type 1 diabetes (T1DM) patients.Material and Method: We built two versions of our CDSS to predict the presence of any-type DR and sight-threatening DR (STDR) in T1DM patients. The first version was trained using 324 T1DM and 826 T2DM patients. The second was trained with only the 324 T1DM patients.Results: The first version achieved an accuracy (ACC) = 0.795, specificity (SP) = 83%, and sensitivity (S) = 65.7% to predict the presence of any-DR, and an ACC = 0.918, SP = 87.1% and S = 87.8% for STDR. The second model achieved ACC = 0.799, SP = 87.5% and S = 86.3% when predicting any-DR and ACC = 0.937, SP = 90.9% and S = 83.0% for STDR.Conclusion: The two models better predict STDR than any-DR in T1DM patients. We will need a larger sample to strengthen our results.Keywords: diabetic retinopathy, type 1 diabetes mellitus, clinical decision support system

Diabetic retinopathy (DR) is one of the most common causes of vision loss or blindness globally. Early detection of retinal eye lesions like hard exudates, soft exudates, microaneurysms, and hemorrhages is crucial to detect DR in a human eye. Therefore, accurate segmentation of lesions from eye fundus images is essential to develop efficient automated DR detection systems. This paper presented a novel hard and soft exudates lesions segmentation method called Efficient Dual-Decoder Boosted Network (EDBNet). EDBNet is composed of the following main components: 1) pre-trained ImageNet ResNet50 encoder with Atrous Spatial Pyramid Pooling (ASPP), 2) UNet decoder block with Gated Skip Connections mechanism to enhance capture more details of fundus images, 3) dual-decoder boosted to improve the performance segmentation of retinal lesion in the eye fundus images, and fusion outputs of the dual-decoder boosted to generate enhanced exudates segmentation. The effectiveness of the proposed framework is assessed on the IDRiD publicly dataset in terms of accuracy, Area Under Precision-Recall (AUPR), IOU, and Dice metrics. EDBNet obtains 99.8, 74.4, 78.0, and 87.6% of soft exudates, respectively. For hard exudates, EDBNet achieves 99.5, 85.3, 80.3, and 89.1%, respectively. The experimental results also demonstrate that EDBNet outperforms many state-of-the-art methods.

Classification systems based Machine Learning hide the logic of their internal decision processes from the users. Hence, post-hoc explanations about their predictions are often required. This paper proposes Fuzzy-LORE, a method that generates local explanations for fuzzy-based Machine Learning systems. First, it learns a local fuzzy decision tree using a set of synthetic neighbours from the input instance. Then, it extracts from the logic of the fuzzy decision tree a meaningful explanation consisting of a set of decision rules (which explain the reasons behind the decision), a set of counterfactual rules (which inform of small changes in the instance’s features that would lead to a different outcome), and finally a set of specific counterfactual examples. Our experiments on a real-world medical dataset show that Fuzzy-LORE outperforms prior approaches and methods for generating local explanations.

Fuzzy Random Forests are well-known Machine Learning ensemble methods. They combine the outputs of multiple Fuzzy Decision Trees to improve the classification performance. Moreover, they can deal with data uncertainty and imprecision thanks to the use of fuzzy logic. Although many classification tasks are binary, in some situations we face the problem of classifying data into a set of ordered categories. This is a particular case of multi-class classification where the order between the classes is relevant, for example in medical diagnosis to detect the severity of a disease. In this paper, we explain how a binary Fuzzy Random Forest may be adapted to deal with ordinal classification. The work is focused on the prediction stage, not on the construction of the fuzzy trees. When a new instance arrives, the rules activation is done with the usual fuzzy operators, but the aggregation of the outputs given by the different rules and trees has been redefined. In particular, we present a procedure for managing the conflicting cases where different classes are predicted with similar support. The support of the classes is calculated using the OWA operator that permits to model the concept of majority agreement.

Background Taking care of chronic or long-term patients at home is an arduous task. Non-professional caregivers suffer the consequences of doing so, especially in terms of their mental health. Performing some simple activities through a mobile phone app may improve their mindset and consequently increase their positivity. However, each caregiver may need support in different aspects of positive mental health. In this paper, a method is defined to calculate the utility of a set of activities for a particular caregiver in order to personalize the intervention plan proposed in the app. Methods Based on the caregivers’ answers to a questionnaire, a modular averaging method is used to calculate the personal level of competence in each positive mental health factor. A reward-penalty scoring procedure then assigns an overall impact value to each activity. Finally, the app ranks the activities using this impact value. Results The results of this new personalization method are provided based on a pilot test conducted on 111 caregivers. The results indicate that a conjunctive average is appropriate at the first stage and that reward should be greater than penalty in the second stage. Conclusions The method presented is able to personalize the intervention plan by determining the best order of carrying out the activities for each caregiver, with the aim of avoiding a high level of deterioration in any factor.

Software projects are amongst the most common professional activities of computer engineers. Gaining competences on the design and development of software projects is, however, a complex issue that cannot be tackled within a single subject. In this article, we detail the design and implementation of a teaching methodology that aims at providing a comprehensive simulation of the whole life cycle of software projects throughout the coordination of the practical exercises of subjects of different courses. The proposal consists in developing the same software project in consecutive courses under the perspective of the three main roles involved in the project development: designer, developer and director. In the second course, the students exercise the developer role in teams of 4 people under the supervision of a student of the fourth course, who acts as the director . The director has previously proposed a formal design of the project in the third course ( designer role ). This methodology has been successfully deployed for 4 years in the Computer Engineering Degree at the Universitat Rovira i Virgili. The results show an improvement of the skills and competences related to the three involved subjects, which include better programming quality, better team coordination and fulfilment of deadlines, as well as a much practical view of the director’s role.

Deep neural network models have been proven to be very successful in image classification tasks, also for medical diagnosis, but their main concern is its lack of interpretability. They use to work as intuition machines with high statistical confidence but unable to give interpretable explanations about the reported results. The vast amount of parameters of these models make difficult to infer a rationale interpretation from them. In this paper we present a diabetic retinopathy interpretable classifier able to classify retine images into the different levels of disease severity and of explaining its results by assigning a score for every point in the hidden and input space, evaluating its contribution to the final classification in a linear way. The generated visual maps can be interpreted by an expert in order to compare its own knowledge with the interpretation given by the model., Submitted to Elsevier

Multi-class classification is a fundamental task in Machine Learning. However, complex models can be viewed as black boxes, making it difficult to gain insight into how the model makes its predictions and build trust in its decision-making process. This paper presents a novel method called Multi-Class Fuzzy-LORE (mcFuzzy-LORE) for explaining the decisions made by multi-class fuzzy-based classifiers such as Fuzzy Random Forests (FRF). mcFuzzy-LORE is an adaptation of the Fuzzy-LORE method that uses fuzzy decision trees as an alternative to classical decision trees, providing interpretable, human-readable rules that describe the reasoning behind the model’s decision for a specific input. The proposed method was evaluated on a private dataset that was used to train an FRF-based multi-class classifier that assesses the risk of developing diabetic retinopathy in diabetic patients. The results show that mcFuzzy-LORE outperforms prior classical LORE-based methods in the generation of counterfactual instances.

Diabetic retinopathy (DR) is a complication of diabetes mellitus that appears in the retina. Clinitians use retina images to detect DR pathological signs related to the occlusion of tiny blood vessels. Such occlusion brings a degenerative cycle between the breaking off and the new generation of thinner and weaker blood vessels. This research aims to develop a suitable retinal vasculature segmentation method for improving retinal screening procedures by means of computer-aided diagnosis systems. The blood vessel segmentation methodology relies on an effective feature selection based on Sequential Forward Selection, using the error rate of a decision tree classifier in the evaluation function. Subsequently, the classification process is performed by three alternative approaches: artificial neural networks, decision trees and support vector machines. The proposed methodology is validated on three publicly accessible datasets and a private one provided by Hospital Sant Joan of Reus. In all cases we obtain an average accuracy above 96% with a sensitivity of 72% in the blood vessel segmentation process. Compared with the state-of-the-art, our approach achieves the same performance as other methods that need more computational power. Our method significantly reduces the number of features used in the segmentation process from 20 to 5 dimensions. The implementation of the three classifiers confirmed that the five selected features have a good effectiveness, independently of the classification algorithm.

Despite the broad adoption of Machine Learning models in many domains, they remain mostly black boxes. There is a pressing need to ensure Machine Learning models that are interpretable, so that designers and users can understand the reasons behind their predictions. In this work, we propose a new method called C-LORE-F to explain the decisions of fuzzy-based black box models. This new method uses some contextual information about the attributes as well as the knowledge of the fuzzy sets associated to the linguistic labels of the fuzzy attributes to provide actionable explanations. The experimental results on three datasets reveal the effectiveness of C-LORE-F when compared with the most relevant related works.

The optic disc (OD) is the point where the retinal vessels begin. OD carries essential information linked to Diabetic Retinopathy and glaucoma that may cause vision loss. Therefore, accurate segmentation of the optic disc from eye fundus images is essential to develop efficient automated DR and glaucoma detection systems. This paper presents a deep learning-based system for OD segmentation based on an ensemble of efficient semantic segmentation models for medical image segmentation. The aggregation of the different DL models was performed with the ordered weighted averaging (OWA) operators. We proposed the use of a dynamically generated set of weights that can give a different contribution to the models according to their performance during the segmentation of OD in the eye fundus images. The effectiveness of the proposed system was assessed on a fundus image dataset collected from the Hospital Sant Joan de Reus. We obtained Jaccard, Dice, Precision, and Recall scores of 95.40, 95.10, 96.70, and 93.90%, respectively.

Random Forests are well-known Machine Learning classification mechanisms based on a collection of decision trees. In the last years, they have been applied to assess the risk of diabetic patients to develop Diabetic Retinopathy. The results have been good, despite the unbalance of data between classes and the inherent ambiguity of the problem (patients with similar data may belong to different classes). In this work we propose a new iterative method to update the set of trees in the Random Forest by considering trees generated from the data of the new patients that are visited in the medical centre. With this method, it has been possible to improve the results obtained with standard Random Forests.

A Fuzzy Decision Tree is a classification method consisting of a set of rules defined on fuzzy variables. The final class assignment is done according to the output of all the rules of the tree. Generally, the maximum operator is used to aggregate the results of the rules. However, some approaches based on more complex aggregation operators have appeared recently. In this work we propose to use Sugeno and Choquet integrals together with a Hierarchically ⊥-Decomposable Fuzzy Measure (HDFM) to aggregate the rules' values. The HDFM exploits the hierarchical structure of the fuzzy decision tree and takes into account the confidence value of the output together with the classification ambiguity of the rules. The HDFM is built using Sugeno-Weber t-conorms.We validate this approach on several classification problems and make a comparison of the performance with the state of art aggregation operators. Finally, a case study with a real dataset of diabetic patients is analyzed to predict the risk of suffering from diabetic retinopathy.

Purpose: To validate a clinical decision support system (CDSS) that estimates risk of diabetic retinopathy (DR) and to personalize screening protocols in type 2 diabetes mellitus (T2DM) patients. Methods: We utilized a CDSS based on a fuzzy random forest, integrated by fuzzy decision trees with the following variables: current age, sex, arterial hypertension, diabetes duration and treatment, HbA1c, glomerular filtration rate, microalbuminuria, and body mass index. Validation was made using the electronic health records of a sample of 101,802 T2DM patients. Diagnosis was made by retinal photographs, according to EURODIAB guidelines and the International Diabetic Retinopathy Classification. Results: The prevalence of DR was 19,759 patients (19.98%). Results yielded 16,593 (16.31%) true positives, 72,617 (71.33%) true negatives, 3165 (3.1%) false positives, and 9427 (9.26%) false negatives, with an accuracy of 0.876 (95% confidence interval [CI], 0.858–0.886), sensitivity of 84% (95% CI, 83.46–84.49), specificity of 88.5% (95% CI, 88.29–88.72), positive predictive value of 63.8% (95% CI, 63.18–64.35), negative predictive value of 95.8% (95% CI, 95.68–95.96), positive likelihood ratio of 7.30, and negative likelihood ratio of 0.18. The type 1 error was 0.115, and the type 2 error was 0.16. Conclusions: We confirmed a good prediction rate for DR from a representative sample of T2DM in our population. Furthermore, the CDSS was able to offer an individualized screening protocol for each patient according to the calculated risk confidence value. Translational Relevance: Results from this study will help to establish a novel strategy for personalizing screening for DR according to patient risk factors. The authors thank Phil Hoddy for his language assistance and for editing and correcting the English text. Supported by Instituto de Investigaciones Carlos III (research projects PI18/00169, PI12/01535, and PI15/01150) and by the European Regional Development Fund (FEDER). The authors thank Phil Hoddy for his language assistance and for editing and correcting the English text. Supported by Instituto de Investigaciones Carlos III (research projects PI18/00169, PI12/01535, and PI15/01150) and by the European Regional Development Fund (FEDER).

In the last years many Machine Learning-based systems have achieved an extraordinary performance in a wide range of fields, especially Natural Language Processing and Computer Vision. The construction of appropriate and meaningful explanations for the decisions of those systems, which are in may cases considered black boxes, is a hot research topic. Most of the state-of-the-art explanation systems in the field rely on generating synthetic instances to test them on the black box, analyse the results and build an interpretable model, which is then used to provide an understandable explanation. In this paper we focus on this generation process and propose Guided-LORE, a variant of the well-known LORE method. We conceptualise the problem of neighbourhood generation as a search and solve it using the Uniform Cost Search algorithm. The experimental results on four datasets reveal the effectiveness of our method when compared with the most relevant related works.

In content-based semantic recommender systems the items to be considered are defined in terms of a set of semantic attributes, which may take as values the concepts of a domain ontology. The aim of these systems is to suggest to the user the items that fit better with his/her preferences, stored in the user profile. When large ontologies are considered it is unrealistic to expect to have complete information about the user preference on each concept. In this work, we explain how the Weighted Ordered Weighted Averaging operator may be used to deduce the user preferences on all concepts, given the structure of the ontology and some partial preferential information. The parameters of the WOWA operator enable to establish the desired aggregation policy, which ranges from a full conjunction to a full disjunction. Different aggregation policies have been analyzed in a case study involving the recommendation of touristic activities in the city of Tarragona. Several profiles have been compared and the results indicate that different aggregation policies should be used depending on the type of user. The amount of information available in the ontology must be also taken into account in order to establish the parameters of the proposed algorithm.

Mining projects, as with many other extractive activities, result in divergent opinions during the planning process which may trigger conflict. Early understanding of stakeholders’ priorities can help with the design of better projects while reducing sources of potential conflict. The objective of this article is to present a rule-based method for decision support to be used in the first stages of the project to identify disagreements in the form of potentially conflictive criteria. The method proposed defines four attributes that should be evaluated for the different relevant criteria subject to potential conflict: the importance of the criterion, the perceived risk, the fairness, and the affected side’s willingness to make dialogue. These attributes are used to construct a rule-based system that can assess the degree of sensibility to conflict for each criterion. The application of this rule-based approach is explained using information from a real past conflict in Chile, where nine criteria were considered, four key decision makers were interviewed and 81 rules were created. The output given by the rule-based system was compared with the conflict degree given by the four decision makers. Results show that in 44.4% of the responses the rank level was the same, in 44% there was a slight subestimation and in 11% an overestimation of the criterion conflictive level. The method proposed incorporates new aspects into the analysis of sources of potential conflict and is simple enough for an anticipatory screening of potential disagreements around the criteria. In addition, this type of precautionary approach in the earlier stages of project appraisal can contribute to a better project design and a constructive industry-community dialogue.

Proposal of weighted kappa as loss function for ordinal regression in deep learning.Derivation of equations required for applying first order optimization algorithms.Solve three real-world ordinal classification problem.Performance comparison of the models trained by both log loss and kappa loss.Check stability of the kappa loss function using different type of data. Weighted Kappa is an index of reference used in many diagnosis systems to compare the agreement between different raters. This index can be also used to evaluate the performance of automatic classification methods against the gold standard given by an expert (or from a consensus of an expert group). On the other hand, in the last years, deep learning has achieved a great importance as a new machine learning method. The usual loss function used in deep learning for multi-class classification is the logarithmic loss. In this paper we explore the direct use of a weighted kappa loss function for multi-class classification of ordinal data, also known as ordinal regression. Three classification problems are solved in the paper using these two loss functions. Results confirm that better classification is made when the model is constructed with the optimization of kappa instead of logarithmic loss.

Background/aims To determine the incidence of any diabetic retinopathy (any-DR), sight-threatening diabetic retinopathy (STDR) and diabetic macular oedema (DMO) and their risk factors in type 1 diabetes mellitus (T1DM) over a screening programme. Methods Nine-year follow-up, prospective population-based study of 366 patients with T1DM and 15 030 with T2DM. Epidemiological risk factors were as follows: current age, age at DM diagnosis, sex, type of DM, duration of DM, arterial hypertension, levels of glycosylated haemoglobin (HbA1c), triglycerides, cholesterol fractions, serum creatinine, estimated glomerular filtration rate (eGFR) and urine albumin to creatinine ratio (UACR). Results Sum incidence of any-DR was 47.26% with annual incidence 15.16±2.19% in T1DM, and 26.49% with annual incidence 8.13% in T2DM. Sum incidence of STDR was 18.03% with annual incidence 5.77±1.21% in T1DM, and 7.59% with annual incidence 2.64±0.15% in T2DM. Sum incidence of DMO was 8.46% with annual incidence 2.68±038% in patients with T1DM and 6.36% with annual incidence 2.19±0.18% in T2DM. Cox's survival analysis showed that current age and age at diagnosis were risk factors at p

This study proposes a novel ranking method, called SentiRank, that integrates data coming from aspect-level sentiment analysis into a multi-criteria decision aiding procedure. The novelty of the work is the combination of the evaluation of the features of a set of products (based on user preferences) with the reviews in social networks about the same products. The contribution of this work is twofold: theoretical and practical. From the theoretical side, we propose an automatic method to extract the aspect categories and their evaluations from the online opinions of users. Next, we describe how to merge that information with the decision maker’s preferences about the features of a set of products. The ELECTRE method is used afterwards to rank the products with both types of input data. From the practical side, we have implemented a tool that can be used to rank a set ofs restaurants in Tarragona, using available data and reviews on the Web. The tests show that the ranking is effectively modified when online reviews are included in the analysis.

Background: To validate our deep learning algorithm (DLA) to read diabetic retinopathy (DR) retinographies. Introduction: Currently DR detection is made by retinography; due to its increasing diabe...

Retinal disorders are a severe health threat for older adults because they may lead to vision loss and blindness. Diabetic patients are particularly prone to suffer from Diabetic Retinopathy. Identifying relevant structural components in color fundus images like the optic disc (OD) is crucial to diagnose retinal diseases. Automatic OD detection is complex because of its location in an area where blood vessels converge, and color distribution is uneven. Several image processing techniques have been developed for OD detection so far, but vessel segmentation is sometimes required, increasing computational complexity and time. Moreover, precise OD segmentation methods utilize complex algorithms that need special hardware or extensive labeled datasets. We propose an OD detection approach based on the Modern Portfolio Theory of Markowitz to generate an innovative color fusion model. Specifically, the training phase calculates the optimal weights for each color channel. A fusion of weighted color channels is then applied in the testing phase. This approach acts as a powerful and real-time preprocessing stage. We use four heterogeneous datasets to validate the presented methodology. Three out of four datasets are publicly available (i.e., DRIVE, Messidor, and HRF), and the last corresponds to an in–house dataset acquired from Hospital Universitari Sant Joan de Reus (Spain). Two different segmentation methods are presented and compared with state-of-the-art computer vision techniques to analyze the model performance. An outstanding accuracy and overlap above 0.9 and 80%, respectively, and a minimal execution time of 0.05 s are reached. Therefore, our model could be integrated into daily clinical practice to accelerate the diagnosis of Diabetic Retinopathy due to its simplicity, performance, and speed.

Recommender systems aim to suggest lists of items that match accurately the user’s preferences. In the last years it has been argued that the diversity of the recommendations also plays an important role in the overall satisfaction of the user. Increasing the diversity of the suggestions may be beneficial both for the user and for retailers. This paper provides a brief review of the most popular diversification mechanisms and it introduces two new ones (Cluster Random and Cluster Quadratic) based on the semantic clustering of the domain objects. It also shows how the level of diversification may be dynamically adapted to the variety in the preferences of the user. A thorough evaluation of the diversification mechanisms on a Tourism recommender has been performed, reaching the conclusion that the new Cluster Quadratic diversification method achieves very competitive levels of precision and recall, while keeping an acceptable computational cost.

Diabetic retinopathy (DR) has become a major worldwide health problem due to the increase in blindness among diabetics at early ages. The detection of DR pathologies such as microaneurysms, hemorrhages and exudates through advanced computational techniques is of utmost importance in patient health care. New computer vision techniques are needed to improve upon traditional screening of color fundus images. The segmentation of the entire anatomical structure of the retina is a crucial phase in detecting these pathologies. This work proposes a novel framework for fast and fully automatic blood vessel segmentation and fovea detection. The preprocessing method involved both contrast limited adaptive histogram equalization and the brightness preserving dynamic fuzzy histogram equalization algorithms to enhance image contrast and eliminate noise artifacts. Afterwards, the color spaces and their intrinsic components were examined to identify the most suitable color model to reveal the foreground pixels against the entire background. Several samples were then collected and used by the renowned convexity shape prior segmentation algorithm. The proposed methodology achieved an average vasculature segmentation accuracy exceeding 96%, 95%, 98% and 94% for the DRIVE, STARE, HRF and Messidor publicly available datasets, respectively. An additional validation step reached an average accuracy of 94.30% using an in-house dataset provided by the Hospital Sant Joan of Reus (Spain). Moreover, an outstanding detection accuracy of over 98% was achieved for the foveal avascular zone. An extensive state-of-the-art comparison was also conducted. The proposed approach can thus be integrated into daily clinical practice to assist medical experts in the diagnosis of DR.

This work has been supported by REACTION (contract no. EVK2-CT2002-80025) and CASCADE (FP7, grant agreement no. 283068) projects, funded by the European Commission; DRYEX project, funded by the Spanish Ministry of Economy and Competiveness (CGL2014-59074-R); and ITAKA research group, under the Government of Catalonia grants 2009SGR-01523 and 2014SGR-652.

Background/aims To determine the changes in the incidence of diabetic retinopathy (DR), diabetic macular oedema (DMO) and their risk factors in a population-based study of patients with diabetes mellitus (DM) referred to our 16 Primary Health Care Areas (HCAs). Methods Prospective population-based study of a total of 15 396 Caucasian patients with DM, who represent 86.53% of the total patients with DM in our HCAs, were studied over an 8-year follow-up period. All patients were screened with a mean follow-up of 3.18±1.11 times for each patient over the 8 years. Results The yearly mean value of any DR was 8.37±2.19% (8.09%–8.99%); of advanced DR yearly mean value of 0.46±0.22% (0.03–0.78); and of DMO a yearly mean value of 2.19±0.18% (2%–2.49%). A clear increase was observed in the last 3 years, any DR increased from 8.09% in 2007 to 8.99% in 2014, and DMO from 2% in 2007 to 2.49% in 2014. These increases were more evident in some age groups. For patients with any DR aged 41–50 and 51–60 and for patients with advanced DR aged 41–50, 51–60 and 61–70, the increase was more marked, related to an increase in HbA1c values or to patients treated with insulin. Conclusions An increase in the incidence of DR and DMO was observed, especially in the younger patients aged between 31 and 70 years. This is linked to bad metabolic control of DM. Our results suggest a greater number of ocular complications in the near future, such as neovascular glaucoma, if these current findings are not addressed.

A new ELECTRE-III method is proposed handling hierarchical structure of criteria.ELECTRE-III-H generates a partial pre-order of alternatives in each node of the hierarchy.Concordance and discordance indices are adapted to aggregation of partial pre-orders.The pseudo-criteria concept is extended on intermediate criteria of the hierarchy.A real-case study about the evaluation of the quality of websites is presented. This paper proposes a method for ranking a set of alternatives evaluated using multiple and conflicting criteria that are organised in a hierarchical structure. The hierarchy permits the decision maker to identify different intermediate sub-problems of interest. In that way, the analysis of the criteria is done according to the subsets defined in the hierarchy, and following the precedence relations in a bottom-up approach. To deal with this type of hierarchical structures, an extension of the ELECTRE-III method, called ELECTRE-III-H, is presented. As all methods of ELECTRE family, this one also relies on building a binary outranking relation on the set of alternatives on the basis of concordance and discordance tests. The exploitation of this outranking relation generates a partial pre-order, establishing an indifference, preference or incomparability relation for each pair of alternatives. The idea of a bottom-up application of the classical ELECTRE-III method to sub-problems involving subsets of criteria at the intermediate levels of the hierarchy is infeasible because the evaluations of alternatives by criteria aggregating some sub-criteria have the form of partial pre-orders, and not complete pre-orders. Thus, we propose a new procedure for building outranking relations from a set of partial pre-orders, as well as a mechanism for propagating these pre-orders upwards in the hierarchy. With this method, the decision maker is able to analyse the problem in a decomposed way and gain information from the outputs obtained at intermediate levels. In addition, ELECTRE-III-H gives the decision maker the possibility to define a local preference model at each node of the hierarchy, according to his objectives and sub-problem characteristics. We show an application of this method to rank websites of tourist destination brands evaluated using a hierarchy with 4 levels.

Classification problems refer to the assignment of alternatives to predefined categories. In this work we focus on ordered classification, called sorting, in which the predefined categories indicate several degrees of interest or suitability of alternatives for a certain user. The assignment of alternatives is based on multiple conflicting criteria. This multi-criteria sorting approach is specially interesting for recommender systems aimed at finding the most suitable alternatives for each user. First, we study the ELECTRE-TRI-B sorting method, which follows the outranking approach based on comparing the evaluations of alternatives with the profile limits separating the categories. The complexity of some recommenders systems requires the extension of the classical ELECTRE-TRI-B method to manage a taxonomical organization of the set of criteria. In this paper we consider a set of criteria in the form of a hierarchy. The intermediate criteria in such a hierarchy correspond to different aspects of the recommendation procedure, such as content, context or cost. At each of these criteria, a sorting problem must be solved. Therefore, we propose extending ELECTRE-TRI-B to handle assignments of alternatives on several levels of the hierarchy. A hierarchical procedure for sorting is proposed, called ELECTRE-TRI-B-H. Secondly, the paper explains the integration of ELECTRE-TRI-B-H into a recommender system of touristic activities related to wine, called GoEno-Tur. This system is developed for the region of Tarragona, Catalonia (Spain), which is a well-recognized area of wine and cava production.