Search

Andreas Holzinger acknowledges funding support from the Austrian Science Fund (FWF), Project: P-32554 explainable Artificial Intelligence and from the European Union's Horizon 2020 research and innovation program under grant agreement 826078 (Feature Cloud). This publication reflects only the authors' view and the European Commission is not responsible for any use that may be made of the information it contains; Natalia Diaz-Rodriguez is supported by the Spanish Government Juan de la Cierva Incorporacion contract (IJC2019-039152-I); Isabelle Augenstein's research is partially funded by a DFF Sapere Aude research leader grant; Javier Del Ser acknowledges funding support from the Basque Government through the ELKARTEK program (3KIA project, KK-2020/00049) and the consolidated research group MATHMODE (ref. T1294-19); Wojciech Samek acknowledges funding support from the European Union's Horizon 2020 research and innovation program under grant agreement No. 965221 (iToBoS), and the German Federal Ministry of Education and Research (ref. 01IS18025 A, ref. 01IS18037I and ref. 0310L0207C); Igor Jurisica acknowledges funding support from Ontario Research Fund (RDI 34876), Natural Sciences Research Council (NSERC 203475), CIHR Research Grant (93579), Canada Foundation for Innovation (CFI 29272, 225404, 33536), IBM, Ian Lawson van Toch Fund, the Schroeder Arthritis Institute via the Toronto General and Western Hospital Foundation., Medical artificial intelligence (AI) systems have been remarkably successful, even outperforming human performance at certain tasks. There is no doubt that AI is important to improve human health in many ways and will disrupt various medical workflows in the future. Using AI to solve problems in medicine beyond the lab, in routine environments, we need to do more than to just improve the performance of existing AI methods. Robust AI solutions must be able to cope with imprecision, missing and incorrect information, and explain both the result and the process of how it was obtained to a medical expert. Using conceptual knowledge as a guiding model of reality can help to develop more robust, explainable, and less biased machine learning models that can ideally learn from less data. Achieving these goals will require an orchestrated effort that combines three complementary Frontier Research Areas: (1) Complex Networks and their Inference, (2) Graph causal models and counterfactuals, and (3) Verification and Explainability methods. The goal of this paper is to describe these three areas from a unified view and to motivate how information fusion in a comprehensive and integrative manner can not only help bring these three areas together, but also have a transformative role by bridging the gap between research and practical applications in the context of future trustworthy medical AI. This makes it imperative to include ethical and legal aspects as a cross-cutting discipline, because all future solutions must not only be ethically responsible, but also legally compliant., Austrian Science Fund (FWF) P-32554, European Union's Horizon 2020 research and innovation program 826078 965221, Spanish Government Juan de la Cierva Incorporacion IJC2019-039152-I, DFF Sapere Aude research leader grant, Basque Government KK-2020/00049, consolidated research group MATHMODE T1294-19, Federal Ministry of Education & Research (BMBF) 01IS18025 A 01IS18037I 0310L0207C, Ontario Research Fund RDI 34876, Natural Sciences Research Council NSERC 203475, Canadian Institutes of Health Research (CIHR) 93579, Canada Foundation for Innovation CGIAR CFI 29272 225404 33536, International Business Machines (IBM), Ian Lawson van Toch Fund, Schroeder Arthritis Institute via the Toronto General and Western Hospital Foundation

Although the European Commission proposed new legislation for the use of "high-risk artificial intelligence" earlier this year, the existing European fundamental rights framework already provides some clear guidance on the use of medical AI.

Nowadays, a vast amount of clinical data scattered across different sites on the Internet hinders users from finding helpful information for their well-being improvement. Besides, the overload of medical information (e.g., on drugs, medical tests, and treatment suggestions) have brought many difficulties to medical professionals in making patient-oriented decisions. These issues raise the need to apply recommender systems in the healthcare domain to help both, end-users and medical professionals, make more efficient and accurate health-related decisions. In this article, we provide a systematic overview of existing research on healthcare recommender systems. Different from existing related overview papers, our article provides insights into recommendation scenarios and recommendation approaches. Examples thereof are food recommendation, drug recommendation, health status prediction, healthcare service recommendation, and healthcare professional recommendation. Additionally, we develop working examples to give a deep understanding of recommendation algorithms. Finally, we discuss challenges concerning the development of healthcare recommender systems in the future.

International audience; Code quality is a requirement for successful and sustainable software development. The emergence of Artificial Intelligence and data driven Machine Learning in current applications makes customized solutions for both data as well as code quality a requirement. The diversity and the stochastic nature of Machine Learning algorithms require different test methods, each of which is suitable for a particular method. Conventional unit tests in test-automation environments provide the common, well-studied approach to tackle code quality issues, but Machine Learning applications pose new challenges and have different requirements, mostly as far the numerical computations are concerned. In this research work, a concrete use of property-based testing for quality assurance in the parameter learning algorithm of a probabilistic graphical model is described. The necessity and effectiveness of this method in comparison to unit tests is analyzed with concrete code examples for enhanced retraceability and interpretability, thus highly relevant for what is called explainable AI.

Background Increasing life expectancy results in more elderly people struggling with age related diseases and functional conditions. This poses huge challenges towards establishing new approaches for maintaining health at a higher age. An important aspect for age related deterioration of the general patient condition is frailty. The frailty syndrome is associated with a high risk for falls, hospitalization, disability, and finally increased mortality. Using predictive data mining enables the discovery of potential risk factors and can be used as clinical decision support system, which provides the medical doctor with information on the probable clinical patient outcome. This enables the professional to react promptly and to avert likely adverse events in advance. Methods Medical data of 474 study participants containing 284 health related parameters, including questionnaire answers, blood parameters and vital parameters from the Toledo Study for Healthy Aging (TSHA) was used. Binary classification models were built in order to distinguish between frail and non-frail study subjects. Results Using the available TSHA data and the discovered potential predictors, it was possible to design, develop and evaluate a variety of different predictive models for the frailty syndrome. The best performing model was the support vector machine (SVM, 78.31%). Moreover, a methodology was developed, making it possible to explore and to use incomplete medical data and further identify potential predictors and enable interpretability. Conclusions This work demonstrates that it is feasible to use incomplete, imbalanced medical data for the development of a predictive model for the frailty syndrome. Moreover, potential predictive factors have been discovered, which were clinically approved by the clinicians. Future work will improve prediction accuracy, especially with regard to separating the group of frail patients into frail and pre-frail ones and analyze the differences among them. Electronic supplementary material The online version of this article (10.1186/s12911-019-0747-6) contains supplementary material, which is available to authorized users.

Gleason grading of prostate cancer is an important prognostic factor, but suffers from poor reproducibility, particularly among non-subspecialist pathologists. Although artificial intelligence (A.I.) tools have demonstrated Gleason grading on-par with expert pathologists, it remains an open question whether and to what extent A.I. grading translates to better prognostication. In this study, we developed a system to predict prostate cancer-specific mortality via A.I.-based Gleason grading and subsequently evaluated its ability to risk-stratify patients on an independent retrospective cohort of 2807 prostatectomy cases from a single European center with 5–25 years of follow-up (median: 13, interquartile range 9–17). Here, we show that the A.I.’s risk scores produced a C-index of 0.84 (95% CI 0.80–0.87) for prostate cancer-specific mortality. Upon discretizing these risk scores into risk groups analogous to pathologist Grade Groups (GG), the A.I. has a C-index of 0.82 (95% CI 0.78–0.85). On the subset of cases with a GG provided in the original pathology report (n = 1517), the A.I.’s C-indices are 0.87 and 0.85 for continuous and discrete grading, respectively, compared to 0.79 (95% CI 0.71–0.86) for GG obtained from the reports. These represent improvements of 0.08 (95% CI 0.01–0.15) and 0.07 (95% CI 0.00–0.14), respectively. Our results suggest that A.I.-based Gleason grading can lead to effective risk stratification, and warrants further evaluation for improving disease management. Gleason grading is the process by which pathologists assess the morphology of prostate tumors. The assigned Grade Group tells us about the likely clinical course of people with prostate cancer and helps doctors to make decisions on treatment. The process is complex and subjective, with frequent disagreement amongst pathologists. In this study, we develop and evaluate an approach to Gleason grading based on artificial intelligence, rather than pathologists’ assessment, to predict risk of dying of prostate cancer. Looking back at tumors and data from 2,807 people diagnosed with prostate cancer, we find that our approach is better at predicting outcomes compared to grading by pathologists alone. These findings suggest that artificial intelligence might help doctors to accurately determine the probable clinical course of people with prostate cancer, which, in turn, will guide treatment. Wulczyn et al. utilise a deep learning-based Gleason grading model to predict prostate cancer-specific mortality in a retrospective cohort of radical prostatectomy patients. Their model enables improved risk stratification compared to pathologists’ grading and demonstrates the potential for computational pathology in the management of prostate cancer.

The most successful Machine Learning (ML) systems remain complex black boxes to end-users, and even experts are often unable to understand the rationale behind their decisions. The lack of transparency of such systems can have severe consequences or poor uses of limited valuable resources in medical diagnosis, financial decision-making, and in other high-stake domains. Therefore, the issue of ML explanation has experienced a surge in interest from the research community to application domains. While numerous explanation methods have been explored, there is a need for evaluations to quantify the quality of explanation methods to determine whether and to what extent the offered explainability achieves the defined objective, and compare available explanation methods and suggest the best explanation from the comparison for a specific task. This survey paper presents a comprehensive overview of methods proposed in the current literature for the evaluation of ML explanations. We identify properties of explainability from the review of definitions of explainability. The identified properties of explainability are used as objectives that evaluation metrics should achieve. The survey found that the quantitative metrics for both model-based and example-based explanations are primarily used to evaluate the parsimony/simplicity of interpretability, while the quantitative metrics for attribution-based explanations are primarily used to evaluate the soundness of fidelity of explainability. The survey also demonstrated that subjective measures, such as trust and confidence, have been embraced as the focal point for the human-centered evaluation of explainable systems. The paper concludes that the evaluation of ML explanations is a multidisciplinary research topic. It is also not possible to define an implementation of evaluation metrics, which can be applied to all explanation methods.

The main driver of the digital transformation currently underway is undoubtedly artificial intelligence (AI). The potential of AI to benefit humanity and its environment is undeniably enormous. AI can definitely help find new solutions to the most pressing challenges facing our human society in virtually all areas of life: from agriculture and forest ecosystems that affect our entire planet, to the health of every single human being. However, this article highlights a very different aspect. For all its benefits, the large-scale adoption of AI technologies also holds enormous and unimagined potential for new kinds of unforeseen threats. Therefore, all stakeholders, governments, policy makers, and industry, together with academia, must ensure that AI is developed with these potential threats in mind and that the safety, traceability, transparency, explainability, validity, and verifiability of AI applications in our everyday lives are ensured. It is the responsibility of all stakeholders to ensure the use of trustworthy and ethically reliable AI and to avoid the misuse of AI technologies. Achieving this will require a concerted effort to ensure that AI is always consistent with human values and includes a future that is safe in every way for all people on this planet. In this paper, we describe some of these threats and show that safety, security and explainability are indispensable cross-cutting issues and highlight this with two exemplary selected application areas: smart agriculture and smart health.

In medicine, particularly in radiology, there are great expectations in artificial intelligence (AI), which can “see” more than human radiologists in regard to, for example, tumor size, shape, morphology, texture and kinetics — thus enabling better care by earlier detection or more precise reports. Another point is that AI can handle large data sets in high-dimensional spaces. But it should not be forgotten that AI is only as good as the training samples available, which should ideally be numerous enough to cover all variants. On the other hand, the main feature of human intelligence is content knowledge and the ability to find near-optimal solutions. The purpose of this paper is to review the current complexity of radiology working places, to describe their advantages and shortcomings. Further, we give an AI overview of the different types and features as used so far. We also touch on the differences between AI and human intelligence in problem-solving. We present a new AI type, labeled “explainable AI,” which should enable a balance/cooperation between AI and human intelligence — thus bringing both worlds in compliance with legal requirements. For support of (pediatric) radiologists, we propose the creation of an AI assistant that augments radiologists and keeps their brain free for generic tasks.

The aim of the third United Nations (UN) Sustainable Development Goal, dedicated to “Good Health and Well-Being,” is that all people can access the health services they need without facing financial hardship. The goal has three targets: 1) 1 billion more people should benefit from universal health coverage, 2) 1 billion more people should be better protected from health emergencies, and 3) 1 billion more people should enjoy better health and well-being (World Health Organization, 2018).21 Artificial in-telligence (AI) is generally acknowl-edged as an important component in achieving these three targets.

The study of visual concept learning methodologies has been developed over the last years, becoming the state-of-the art research that challenges the reasoning capabilities of deep learning methods. In this paper we discuss the evolution of those methods, starting from the captioning approaches that prepared the transition to current cutting-edge visual question answering systems. The emergence of specially designed datasets, distilled from visual complexity, but with properties and divisions that challenge abstract reasoning and generalization capabilities, encourages the development of AI systems that will support them by design. Explainability of the decision making process of AI systems, either built-in or as a by-product of the acquired reasoning capabilities, underpins the understanding of those systems robustness, their underlying logic and their improvement potential.

Multimorbidity refers to the coexistence of two or more chronic diseases in one person. Therefore, patients with multimorbidity have multiple and special care needs. However, in practice it is difficult to meet these needs because the organizational processes of current healthcare systems tend to be tailored to a single disease. To improve clinical decision making and patient care in multimorbidity, a radical change in the problem-solving approach to medical research and treatment is needed. In addition to the traditional reductionist approach, we propose interactive research supported by artificial intelligence (AI) and advanced big data analytics. Such research approach, when applied to data routinely collected in healthcare settings, provides an integrated platform for research tasks related to multimorbidity. This may include, for example, prediction, correlation, and classification problems based on multiple interaction factors. However, to realize the idea of this paradigm shift in multimorbidity research, the optimization, standardization, and most importantly, the integration of electronic health data into a common national and international research infrastructure is needed. Ultimately, there is a need for the integration and implementation of efficient AI approaches, particularly deep learning, into clinical routine directly within the workflows of the medical professionals.

Extracting hierarchical properties from Whole Slide Images automatically and expanding the possibilities of visualising data from digital pathology will not only drastically improve speed and accuracy of the pathologists’ work but also simplify the necessary pre-processing steps for machine learning tool-chains. The introduced pipeline identifies and converts areas of interest into binary masks and finds groups of areas that share similar locations using k-Means. This grouping is evaluated by the Silhouette Score which serves as a measure of confidence for the separability of clusters. Found objects are compared using structural similarities and HU-Moments. These results are then stored as measures of similarities creating virtual groups of the most similar objects. Finally, the information on similarities combined with further structural parameters are visualised.

Training in the medical field depends on the teaching of theoretical and practical skills. The acquisition of theoretical basics can be usually achieved by studying various documents, while the training of practical skills requires more intensive discussion and a special setup, such as an appropriate laboratory environment or a mentor-mentee connection. Aside from the conditions for transferring and gaining practical skills, teaching methods mostly concentrate on the sharing of explicit knowledge that can be designated and transferred easily.Therefore, the presented work focuses on the tracking of how to approach the diagnosis in pathological examinations and the visualization of any interaction with the specimen by recording microscopical exploration of an experienced pathologist. Those interactions include the full navigation path through a specimen, the panning and magnification of areas of interest, the observation duration as well as the spoken comments. Each maneuver and annotation that is relevant for approaching the diagnosis is documented, processed and visually prepared for trainees/residents to be able to comprehend argumentation via the look through the eyes of an expert.The collected data serve as a base for various medical applications like learning management systems to support education in the medical field, as well as machine learning and artificial intelligence applications in the area of digital pathology.

Histopathology and cytopathology developed for the microscopic examination of tissue samples a specific terminology to describe type and shape of objects and patterns in the composition of nuclei, cells, tissue and anatomical elements. We map such a terminology to Bertin’s visual variables and propose three methods to describe “shape grammar” (1) With a formal, mathematical language, (2) with graphs and (3) by natural language descriptions. Finally, we propose practical applications of shape properties and shape grammar for explainability of AI algorithms in computational pathology.

International audience; Smart home environment plays a prominent role in improving the quality of life of the residents by enabling home automation, health care and safety through various Internet of Things (IoT) devices. However, a large amount of data generated by sensors in a smart home environment heighten security and privacy concerns among potential users. Some of the data can be sensitive as it contains information about users’ private activities, location, behavioural patterns and health status. Other concerns of the users are towards the distribution and sharing of data to third parties. In this chapter, we propose privacy-enabled smart home framework consisting of three major components: activity recognition and occupancy detection, privacy-preserving data management and voice assistant. The proposed platform includes unobtrusive sensors for multiple occupancy detection and activity recognition. The privacy-enabled voice assistant performs interaction with smart home. We also present a detailed description of system architecture with service middleware.

Conversational agents are gaining huge popularity in industrial applications such as digital assistants, chatbots, and particularly systems for natural language understanding (NLU). However, a major drawback is the unavailability of a common metric to evaluate the replies against human judgement for conversational agents. In this paper, we develop a benchmark dataset with human annotations and diverse replies that can be used to develop such metric for conversational agents. The paper introduces a high-quality human annotated movie dialogue dataset, HUMOD, that is developed from the Cornell movie dialogues dataset. This new dataset comprises 28,500 human responses from 9500 multi-turn dialogue history-reply pairs. Human responses include: (i) ratings of the dialogue reply in relevance to the dialogue history, and (ii) unique dialogue replies for each dialogue history from the users. Such unique dialogue replies enable researchers in evaluating their models against six unique human responses for each given history. Detailed analysis on how dialogues are structured and human perception on dialogue score in comparison with existing models are also presented.

The complexity of cancer diseases demands bioinformatic techniques and translational research based on big data and personalized medicine. Open data enables researchers to accelerate cancer studies, save resources and foster collaboration. Several tools and programming approaches are available for analyzing data, including annotation, clustering, comparison and extrapolation, merging, enrichment, functional association and statistics. We exploit openly available data via cancer gene expression analysis, we apply refinement as well as enrichment analysis via gene ontology and conclude with graph-based visualization of involved protein interaction networks as a basis for signaling. The different databases allowed for the construction of huge networks or specified ones consisting of high-confidence interactions only. Several genes associated to glioma were isolated via a network analysis from top hub nodes as well as from an outlier analysis. The latter approach highlights a mitogen-activated protein kinase next to a member of histondeacetylases and a protein phosphatase as genes uncommonly associated with glioma. Cluster analysis from top hub nodes lists several identified glioma-associated gene products to function within protein complexes, including epidermal growth factors as well as cell cycle proteins or RAS proto-oncogenes. By using selected exemplary tools and open-access resources for cancer research and differential network analysis, we highlight disturbed signaling components in brain cancer subtypes of glioma.

Within the last ten years, essential steps have been made to bring artificial intelligence (AI) successfully into the field of pathology. However, most pathologists are still far away from using AI in daily pathology practice. If one leaves the pathology annihilation model, this paper focuses on tasks, which could be solved, and which could be done better by AI, or image-based algorithms, compared to a human expert. In particular, this paper focuses on the needs and demands of surgical pathologists; examples include: Finding small tumour deposits within lymph nodes, detection and grading of cancer, quantification of positive tumour cells in immunohistochemistry, pre-check of Papanicolaou-stained gynaecological cytology in cervical cancer screening, text feature extraction, text interpretation for tumour-coding error prevention and AI in the next-generation virtual autopsy. However, in order to make substantial progress in both fields it is important to intensify the cooperation between medical AI experts and pathologists.

This paper reviews guidelines on how medical imaging analysis can be enhanced by Artificial Intelligence (AI) and Machine Learning (ML). In addition to outlining current and potential future developments, we also provide background information on chemical imaging and discuss the advantages of Explainable AI. We hypothesize that it is a matter of AI to find an invariably recurring parameter that has escaped human attention (e.g. due to noisy data). There is great potential in AI to illuminate the feature space of successful models.

An effective automatic diagnosis and grading of diabetic retinopathy would be very useful in the management of the diabetic retinopathy within the national health system. The detection of the presence of diabetic retinopathy features in the eyes is a challenging problem. Therefore, highly efficient and accurate image processing and machine learning techniques must be used in order to produce an effective automatic diagnosis of diabetic retinopathy. This chapter presents an up-to-date review on diabetic retinopathy detection systems that implement a variety of image processing techniques, including fuzzy image processing, along various machine learning techniques used for feature extraction and classification. Some background on diabetic retinopathy, with a focus on the diabetic retinopathy features and the diabetic retinopathy screening process, is included for better understanding. The chapter also highlights the available public databases, containing eye fundus images, which can be currently used in the diabetic retinopathy research. As the development of an automatic diabetic retinopathy screening system is a very challenging task, some of these challenges together with a discussion pertaining the automatic diabetic retinopathy screening are also presented in this chapter.

Diabetic retinopathy is an eye disease generated by long-standing diabetes, and it is one of the main causes of vision loss if not diagnosed and treated properly. Diabetic retinopathy consists of several types of lesions found in the retina of diabetic individuals. One of the important lesions of diabetic retinopathy is microaneurysms, which are small red dots that appear due to the local weakness of the capillary walls. This paper presents a novel automatic microaneurysms detection method, in retinal images by employing fuzzy image processing and deep learning. Firstly, the paper explores the existing systems of diabetic retinopathy screening, with a focus on the microaneurysms detection methods and deep learning classification. The proposed system consists of two parts, namely: image preprocessing with a combination of fuzzy image processing techniques, and also the microaneurysms classification using deep neural networks. This paper investigates the capability of a combination of different fuzzy image preprocessing techniques for the detection of microaneurysms in eye fundus images. In addition to the proposed microaneurysms detection system, the paper also highlights a novel dataset for the microaneurysms detection that includes the ground truth data. The purpose of the proposed automated microaneurysm detection with digital analysis of eye fundus images is to substitute current practice that is based on manual diagnosis and visual inspection, and eventually to contribute to producing a more reliable diabetic retinopathy screening system.

Recent advances in automatic machine learning (aML) allow solving problems without any human intervention. However, sometimes a human-in-the-loop can be beneficial in solving computationally hard problems. In this paper we provide new experimental insights on how we can improve computational intelligence by complementing it with human intelligence in an interactive machine learning approach (iML). For this purpose, we used the Ant Colony Optimization (ACO) framework, because this fosters multi-agent approaches with human agents in the loop. We propose unification between the human intelligence and interaction skills and the computational power of an artificial system. The ACO framework is used on a case study solving the Traveling Salesman Problem, because of its many practical implications, e.g. in the medical domain. We used ACO due to the fact that it is one of the best algorithms used in many applied intelligence problems. For the evaluation we used gamification, i.e. we implemented a snake-like game called Traveling Snakesman with the MAX–MIN Ant System (MMAS) in the background. We extended the MMAS–Algorithm in a way, that the human can directly interact and influence the ants. This is done by “traveling” with the snake across the graph. Each time the human travels over an ant, the current pheromone value of the edge is multiplied by 5. This manipulation has an impact on the ant’s behavior (the probability that this edge is taken by the ant increases). The results show that the humans performing one tour through the graphs have a significant impact on the shortest path found by the MMAS. Consequently, our experiment demonstrates that in our case human intelligence can positively influence machine intelligence. To the best of our knowledge this is the first study of this kind.

Data from National Statistical Institutes is generally considered an important source of credible evidence for a variety of users. Summarization and dissemination via traditional methods is a convenient approach for providing this evidence. However, this is usually comprehensible only for users with a considerable level of statistical literacy. A promising alternative lies in augmenting the summarization linguistically. Less statistically literate users (e.g., domain experts and the general public), as well as disabled people can benefit from such a summarization. This article studies the potential of summaries expressed in short quantified sentences. Summaries including, for example, “most visits from remote countries are of a short duration” can be immediately understood by diverse users. Linguistic summaries are not intended to replace existing dissemination approaches, but can augment them by providing alternatives for the benefit of diverse users of official statistics. Linguistic summarization can be achieved via mathematical formalization of linguistic terms and relative quantifiers by fuzzy sets. To avoid summaries based on outliers or data with low coverage, a quality criterion is applied. The concept based on linguistic summaries is demonstrated on test interfaces, interpreting summaries from real municipal statistical data. The article identifies a number of further research opportunities, and demonstrates ways to explore those.

Background There is potential for medical research on the basis of routine data used from general practice electronic health records (GP eHRs), even in areas where there is no common GP research platform. We present a case study on menopausal women with hypertension and metabolic syndrome (MS). The aims were to explore the appropriateness of the standard definition of MS to apply to this specific, narrowly defined population group and to improve recognition of women at high CV risk. Methods We investigated the possible uses offered by available data from GP eHRs, completed with patients interview, in goal of the study, using a combination of methods. For the sample of 202 hypertensive women, 47–59 years old, a data set was performed, consisted of a total number of 62 parameters, 50 parameters used from GP eHRs. It was analysed by using a mixture of methods: analysis of differences, cutoff values, graphical presentations, logistic regression and decision trees. Results The age range found to best match the emergency of MS was 51–55 years. Deviations from the definition of MS were identified: a larger cut-off value of the waist circumference measure (89 vs 80 cm) and parameters BMI and total serum cholesterol perform better as components of MS than the standard parameters waist circumference and HDL-cholesterol. The threshold value of BMI at which it is expected that most of hypertensive menopausal women have MS, was found to be 25.5. The other best means for recognision of women with MS include triglycerides above the threshold of 1.7 mmol/L and information on statins use. Prevention of CVD should focus on women with a new onset diabetes and comorbidities of a long-term hypertension with anxiety/depression. Conclusions The added value of this study goes beyond the current paradigm on MS. Results indicate characteristics of MS in a narrowly defined, specific population group. A comprehensive view has been enabled by using heterogenoeus data and a smart combination of various methods for data analysis. The paper shows the feasibility of this research approach in routine practice, to make use of data which would otherwise not be used for research.

Motivation Image augmentation is a frequently used technique in computer vision and has been seeing increased interest since the popularity of deep learning. Its usefulness is becoming more and more recognized due to deep neural networks requiring larger amounts of data to train, and because in certain fields, such as biomedical imaging, large amounts of labelled data are difficult to come by or expensive to produce. In biomedical imaging, features specific to this domain need to be addressed. Results Here we present the Augmentor software package for image augmentation. It provides a stochastic, pipeline-based approach to image augmentation with a number of features that are relevant to biomedical imaging, such as z-stack augmentation and randomized elastic distortions. The software has been designed to be highly extensible meaning an operation that might be specific to a highly specialized task can easily be added to the library, even at runtime. Although it has been designed as a general software library, it has features that are particularly relevant to biomedical imaging and the techniques required for this domain. Availability and implementation Augmentor is a Python package made available under the terms of the MIT licence. Source code can be found on GitHub under https://github.com/mdbloice/Augmentor and installation is via the pip package manager (A Julia version of the package, developed in parallel by Christof Stocker, is also available under https://github.com/Evizero/Augmentor.jl).

Kandinsky Figures and Kandinsky Patterns are mathematically describable, simple self-contained hence controllable test data sets for the development, validation and training of explainability in artificial intelligence. Whilst Kandinsky Patterns have these computationally manageable properties, they are at the same time easily distinguishable from human observers. Consequently, controlled patterns can be described by both humans and computers. We define a Kandinsky Pattern as a set of Kandinsky Figures, where for each figure an "infallible authority" defines that the figure belongs to the Kandinsky Pattern. With this simple principle we build training and validation data sets for automatic interpretability and context learning. In this paper we describe the basic idea and some underlying principles of Kandinsky Patterns and provide a Github repository to invite the international machine learning research community to a challenge to experiment with our Kandinsky Patterns to expand and thus make progress in the field of explainable AI and to contribute to the upcoming field of explainability and causability., 13 pages, 13 Figures

International audience; In this short editorial we present some thoughts on present and future trends in Artificial Intelligence (AI) generally, and Machine Learning (ML) specifically. Due to the huge ongoing success in machine learning, particularly in statistical learning from big data, there is rising interest of academia, industry and the public in this field. Industry is investing heavily in AI, and spin-offs and start-ups are emerging on an unprecedented rate. The European Union is allocating a lot of additional funding into AI research grants, and various institutions are calling for a joint European AI research institute. Even universities are taking AI/ML into their curricula and strategic plans. Finally, even the people on the street talk about it, and if grandma knows what her grandson is doing in his new start-up, then the time is ripe: We are reaching a new AI spring. However, as fantastic current approaches seem to be, there are still huge problems to be solved: the best performing models lack transparency, hence are considered to be black boxes. The general and worldwide trends in privacy, data protection, safety and security make such black box solutions difficult to use in practice. Specifically in Europe, where the new General Data Protection Regulation (GDPR) came into effect on May, 28, 2018 which affects everybody (right of explanation). Consequently, a previous niche field for many years, explainable AI, explodes in importance. For the future, we envision a fruitful marriage between classic logical approaches (ontologies) with statistical approaches which may lead to context-adaptive systems (stochastic ontologies) that might work similar as the human brain.

Data quality is of paramount importance for the smooth functioning of modern data-driven AI applications with machine learning as a core technology. This is also true for medical AI, where malfunctions due to "dirty data" can have particularly dramatic harmful implications. Consequently, data cleansing is an important part in improving the usability of (Big) Data for medical AI systems. However, it should not be overlooked that data cleansing can also have negative effects on data quality if not performed carefully. This paper takes an interdisciplinary look at some of the technical and legal challenges of data cleansing against the background of European medical device law, with the key message that technical and legal aspects must always be considered together in such a sensitive context.

Part 5: MAKE Explainable AI; International audience; Explainable AI is not a new field. Since at least the early exploitation of C.S. Pierce's abductive reasoning in expert systems of the 1980s, there were reasoning architectures to support an explanation function for complex AI systems, including applications in medical diagnosis , complex multi-component design, and reasoning about the real world. So explainability is at least as old as early AI, and a natural consequence of the design of AI systems. While early expert systems consisted of handcrafted knowledge bases that enabled reasoning over narrowly well-defined domains (e.g., INTERNIST, MYCIN), such systems had no learning capabilities and had only primitive uncertainty handling. But the evolution of formal reasoning architectures to incorporate principled probabilistic reasoning helped address the capture and use of uncertain knowledge. There has been recent and relatively rapid success of AI/machine learning solutions arises from neural network architectures. A new generation of neural methods now scale to exploit the practical applicability of statistical and algebraic learning approaches in arbitrarily high dimensional spaces. But despite their huge successes, largely in problems which can be cast as classification problems, their effectiveness is still limited by their un-debuggability, and their inability to "explain" their decisions in a human understandable and reconstructable way. So while AlphaGo or DeepStack can crush the best humans at Go or Poker, neither program has any internal model of its task; its representations defy interpretation by humans, there is no mechanism to explain their actions and behaviour, and furthermore, there is no obvious instructional value... the high performance systems can not help humans improve. Even when we understand the underlying mathematical scaffolding of current machine learning architectures, it is often impossible to get insight into the internal working of the models; we need explicit modeling and reasoning tools to explain how and why a result was achieved. We also know that a significant challenge for future AI is contextual adaptation, i.e., systems that incrementally help to construct explanatory models for solving real-world problems. Here it would be beneficial not to exclude human expertise, but to augment human intelligence with artificial intelligence.

The grand goal of Machine Learning is to develop software which can learn from previous experience—similar to how we humans do. Ultimately, to reach a level of usable intelligence, we need (1) to learn from prior data, (2) to extract knowledge, (3) to generalize—i.e., guessing where probability function mass/density concentrates, (4) to fight the curse of dimensionality, and (5) to disentangle underlying explanatory factors of the data—i.e., to make sense of the data in the context of an application domain. To address these challenges and to ensure successful machine learning applications in various domains an integrated machine learning approach is important. This requires a concerted international effort without boundaries, supporting collaborative, cross-domain, interdisciplinary and transdisciplinary work of experts from seven sections, ranging from data pre-processing to data visualization, i.e., to map results found in arbitrarily high dimensional spaces into the lower dimensions to make it accessible, usable and useful to the end user. An integrated machine learning approach needs also to consider issues of privacy, data protection, safety, security, user acceptance and social implications. This paper is the inaugural introduction to the new journal of MAchine Learning & Knowledge Extraction (MAKE). The goal is to provide an incomplete, personally biased, but consistent introduction into the concepts of MAKE and a brief overview of some selected topics to stimulate future research in the international research community.

Since pathology is supported by information technology new opportunities and questions have arisen. The digital age enables analyzing histopathological data with artificial intelligence methods to reveal further information and correlations. In this paper existing approaches to visualization of medical decision processes are presented as well as the relevance of explainability in decision making. The first step for implementing decision-paths in systems is to retrace an experienced pathologist’s diagnosis finding process. Recording a route through a landscape composed of human tissue in terms of a roadbook is one possible approach to collect information on how diagnoses are found. Choosing the roadbook metaphor provides a simple schema, that holds basic directions enriched with metadata regarding landmarks on a rally - in the context of pathology such landmarks provide information on the decision finding process.

The combination of ontologies with machine learning (ML) approaches is a hot topic and not yet extensively investigated but having great future potential. This is due to the general fact that both, ontologies and ML, constitute two indispensable technologies for domain-specific knowledge extraction, actively used in knowledge-based systems. Whilst the primary goal of both these approaches are the same, knowledge discovery, little is yet known about how the two sources of knowledge can be successfully integrated. The main data source in digital pathology are whole slide images. For the effective generation of sufficiently large and high-quality training data we need to extract in addition information from medical reports, containing non-standardized text. Since full annotation on pixel level would be impracticably expensive, a practical solution is in weakly-supervised ML. In the project described in this paper we used ontology-guided natural language processing (NLP) for term extraction and a decision tree built with an expert-curated classification system. This demonstrates the practical value of our solution to analyze and structure training data sets for ML and as a tool for the generation of biobank catalogues.

Artificial intelligence (AI) is currently regaining enormous interest due to the success of machine learning (ML), and in particular deep learning (DL). Image analysis, and thus radiomics, strongly benefits from this research. However, effectively and efficiently integrating diverse clinical, imaging, and molecular profile data is necessary to understand complex diseases, and to achieve accurate diagnosis in order to provide the best possible treatment. In addition to the need for sufficient computing resources, suitable algorithms, models, and data infrastructure, three important aspects are often neglected: (1) the need for multiple independent, sufficiently large and, above all, high-quality data sets; (2) the need for domain knowledge and ontologies; and (3) the requirement for multiple networks that provide relevant relationships among biological entities. While one will always get results out of high-dimensional data, all three aspects are essential to provide robust training and validation of ML models, to provide explainable hypotheses and results, and to achieve the necessary trust in AI and confidence for clinical applications.

We review some emerging trends in transduction, connectivity and data analytics for Point-of-Care Testing (POCT) of infectious and non-communicable diseases. The patient need for POCT is described along with developments in portable diagnostics, specifically in respect of Lab-on-chip and microfluidic systems. We describe some novel electrochemical and photonic systems and the use of mobile phones in terms of hardware components and device connectivity for POCT. Developments in data analytics that are applicable for POCT are described with an overview of data structures and recent AI/Machine learning trends. The most important methodologies of machine learning, including deep learning methods, are summarised. The potential value of trends within POCT systems for clinical diagnostics within Lower Middle Income Countries (LMICs) and the Least Developed Countries (LDCs) are highlighted.

Enhanced resolution of 7T magnetic resonance imaging (MRI) scanners has considerably advanced our knowledge of structure and function in human and animal brains. Post-industrialized countries are particularly prone to an ever-increasing number of ageing individuals and ageing-associated neurodegenerative diseases. Neurodegenerative diseases are associated with volume loss in the affected brain. MRI diagnoses and monitoring of subtle volume changes in the ageing/diseased brains have the potential to become standard diagnostic tools. Even with the superior resolution of 7T MRI scanners, the microstructural changes comprising cell types, cell numbers, and cellular processes, are still undetectable. Knowledge of origin, nature, and progression for microstructural changes are necessary to understand pathogenetic stages in the relentless neurodegenerative diseases, as well as to develop therapeutic tools that delay or stop neurodegenerative processes at their earliest stage. We illustrate the gap in resolution by comparing the identical regions of the post-mortem in situ 7 T MR images (virtual autopsy or virtopsy) with the histological observations in serial sections through the same brain. We also described the protocols and limitations associated with these comparisons, as well as the necessity of supercomputers and data management for "Big data". Analysis of neuron and/or glial number by using a body of mathematical tools and guidelines (stereology) is time-consuming, cumbersome, and still restricted to trained human investigators. Development of tools based on machine learning (ML) and artificial intelligence (AI) could considerably accelerate studies on localization, onset, and progression of neuron loss. Finally, these observations could disentangle the mechanisms of volume loss into stages of reversible atrophy and/or irreversible fatal cell death. This AI- and ML-based cooperation between virtopsy and histology could bridge the present gap between virtual reality and neuropathology. It could also culminate in the creation of an imaging-associated comprehensive database. This database would include genetic, clinical, epidemiological, and technical aspects that could help to alleviate or even stop the adverse effects of neurodegenerative diseases on affected individuals, their families, and society.

International audience; The most fundamental communication mechanism for interaction is dialogues involving speech, gesture, semantic and pragmatic knowledge. Various researches on dialogue management have been conducted focusing on standardized model for goal oriented applications using machine learning and deep learning models. The paper presents the overview on existing methods for dialogue manager training; their advantages and limitations. Furthermore, a new image-based method is used in Facebook bAbI Task 1 dataset in Out Of Vocabulary setting. The results show that using dialogue as an image performs well and helps dialogue manager in expanding out of vocabulary dialogue tasks in comparison to Memory Networks.

Serious games’ success depends on its capabilities to engage learners and to provide them with personalized gaming and learning experiences. Therefore, theoretically sound mechanisms for gaining a certain level of understanding of learning and gaming processes by the game is crucial. Consequently, AI and machine learning technologies increasingly enter the field. These technologies often fail, however, since serious games either pose highly complex problems (combining gaming and learning process) or do not provide the extensive data bases that would be required. One solution might be allowing human intelligence or intuition influence AI processes. In the present study, we investigated pathfinding algorithms with and without human interventions to the algorithms. As a testbed, we used a clone of the Travelling Salesman problem, the Travelling Snakesman game. We found some evidence that in this particular pathfinding problem human interventions result in superior results as the MAXMIN Ant System algorithm.

One-digit multiplication problems is one of the major fields in learning mathematics at the level of primary school that has been studied over and over. However, the majority of related work is focusing on descriptive statistics on data from multiple surveys. The goal of our research is to gain insights into multiplication misconceptions by applying machine learning techniques. To reach this goal, we trained a probabilistic graphical model of the students’ misconceptions from data of an application for learning multiplication. The use of this model facilitates the exploration of insights into human learning competence and the personalization of tutoring according to individual learner’s knowledge states. The detection of all relevant causal factors of the erroneous students answers as well as their corresponding relative weight is a valuable insight for teachers. Furthermore, the similarity between different multiplication problems - according to the students behavior - is quantified and used for their grouping into clusters. Overall, the proposed model facilitates real-time learning insights that lead to more informed decisions.

Many problems in clinical medicine are characterized by high complexity and non- linearity. Particularly, this is the case with aging diseases, chronic medical conditions that are known to tend to accumulate in the same person. This phenomenon is known as multimorbidity. In addition to the number of chronic diseases, the presence of integrated geriatric conditions and functional deficits, such as walking difficulties, of frailty (a general weakness associated with weight and muscle loss and low functioning) are important for the pre-diction of negative health outcomes of older people, such as hospitalization, dependency on others or pre- term mortality. In this work, we identified how frailty is associated with clinical phenotypes, which most reliably characterize the group of older patients from our local environment: the general practice attenders. We have performed cluster analysis, based on using a set of anthro-pometric and laboratory health indicators, routinely collected in electronic health records. Differences found among clusters in proportions of prefrail and frailversus non-frail patients have been explained with differences in the central values of the parameters used for clustering. Distribution patterns of chronic diseases and other geriatric conditions, found by the assessment of differences, were very useful in determining the clinical phenotypes derived by the clusters.Once more, this study demonstrates the most important aspect of any machine learning task: the quality of the data

Recent success in Artificial Intelligence (AI) and Machine Learning (ML) allow problem solving automatically without any human intervention. Autonomous approaches can be very convenient. However, in certain domains, e.g., in the medical domain, it is necessary to enable a domain expert to understand, why an algorithm came up with a certain result. Consequently, the field of Explainable AI (xAI) rapidly gained interest worldwide in various domains, particularly in medicine. Explainable AI studies transparency and traceability of opaque AI/ML and there are already a huge variety of methods. For example with layer-wise relevance propagation relevant parts of inputs to, and representations in, a neural network which caused a result, can be highlighted. This is a first important step to ensure that end users, e.g., medical professionals, assume responsibility for decision making with AI/ML and of interest to professionals and regulators. Interactive ML adds the component of human expertise to AI/ML processes by enabling them to re-enact and retrace AI/ML results, e.g. let them check it for plausibility. This requires new human-AI interfaces for explainable AI. In order to build effective and efficient interactive human-AI interfaces we have to deal with the question of how to evaluate the quality of explanations given by an explainable AI system. In this paper we introduce our System Causability Scale (SCS) to measure the quality of explanations. It is based on our notion of Causability (Holzinger et al., 2019) combined with concepts adapted from a widely accepted usability scale., Comment: 6 pages, 1 figure, 1 table, will appear in Springer/Nature KI - K\"unstliche Intelligenz (2020), Volume 34, Issue 2

Health systems worldwide are confronted with enormous amounts of heterogenous data from various clinical sources. Additionally, the trend toward precision medicine, in order to tailor decisions and therapies to the individual patient, generates enormous amounts of *omics data. Modern medicine is turning into a data science. The problem is that most of these data are in arbitrarily high dimensions, making manual analysis difficult, yet impossible for humans. This requires automated methods of data science, which is what machine learning provides. Automatic machine learning methods show impressive results in dealing with big data, because such algorithms can learn from training samples to gain knowledge from experience, thus helping to make decisions and predictions. The more data, the better these algorithms perform. Unfortunately, in medicine, we are often confronted with a small number of data sets, “little data,” rare events, and complex data, where automatic approaches suffer from insufficient training sets. This calls for interactive machine learning with a doctor in the loop, who can be beneficial in solving computationally hard problems, where human expertise can help to reduce an exponential search space through heuristic selection of data samples. Therefore, what would otherwise be an NP-hard problem reduces greatly in complexity through the input and the assistance of a medical professional involved in the machine learning pipeline.

AI follows the notion of human intelligence which is unfortunately not a clearly defined term. The most common definition given by cognitive science as mental capability, includes, among others, the ability to think abstract, to reason, and to solve problems from the real world. A hot topic in current AI/machine learning research is to find out whether and to what extent algorithms are able to learn abstract thinking and reasoning similarly as humans can do – or whether the learning outcome remains on purely statistical correlation. In this paper we provide some background on testing intelligence, report some preliminary results from 271 participants of our online study on explainability, and propose to use our Kandinsky Patterns as an IQ-Test for machines. Kandinsky Patterns are mathematically describable, simple, self-contained hence controllable test data sets for the development, validation and training of explainability in AI. Kandinsky Patterns are at the same time easily distinguishable from human observers. Consequently, controlled patterns can be described by both humans and computers. The results of our study show that the majority of human explanations was made based on the properties of individual elements in an image (i.e., shape, color, size) and the appearance of individual objects (number). Comparisons of elements (e.g., more, less, bigger, smaller, etc.) were significantly less likely and the location of objects, interestingly, played almost no role in the explanation of the images. The next step is to compare these explanations with machine explanations.

We propose a novel classification according to aggregation functions of mixed behaviour by variability in ordinal sums of conjunctive and disjunctive functions. Consequently, domain experts are empowered to assign only the most important observations regarding the considered attributes. This has the advantage that the variability of the functions provides opportunities for machine learning to learn the best possible option from the data. Moreover, such a solution is comprehensible, reproducible and explainable-per-design to domain experts. In this paper, we discuss the proposed approach with examples and outline the research steps in interactive machine learning with a human-in-the-loop over aggregation functions. Although human experts are not always able to explain anything either, they are sometimes able to bring in experience, contextual understanding and implicit knowledge, which is desirable in certain machine learning tasks and can contribute to the robustness of algorithms. The obtained theoretical results in ordinal sums are discussed and illustrated on examples.

Part 4: MAKE VIS; International audience; We describe a selection of challenges at the intersection of machine learning and data visualization and outline a subjective research agenda based on professional and personal experience. The unprecedented increase in the amount, variety and the value of data has been significantly transforming the way that scientific research is carried out and businesses operate. Within data science, which has emerged as a practice to enable this data-intensive innovation by gathering together and advancing the knowledge from fields such as statistics, machine learning, knowledge extraction, data management, and visualization, visualization plays a unique and maybe the ultimate role as an approach to facilitate the human and computer cooperation, and to particularly enable the analysis of diverse and heterogeneous data using complex computational methods where algorithmic results are challenging to interpret and operationalize. Whilst algorithm development is surely at the center of the whole pipeline in disciplines such as Machine Learning and Knowledge Discovery, it is visualization which ultimately makes the results accessible to the end user. Visualization thus can be seen as a mapping from arbitrarily high-dimensional abstract spaces to the lower dimensions and plays a central and critical role in interacting with machine learning algorithms, and particularly in interactive machine learning (iML) with including the human-in-the-loop. The central goal of the CD-MAKE VIS workshop is to spark discussions at this intersection of visualization, machine learning and knowledge discovery and bring together experts from these disciplines. This paper discusses a perspective on the challenges and opportunities in this integration of these discipline and presents a number of directions and strategies for further research.

Part 5: MAKE AAL; International audience; Human activity recognition using smart home sensors is one of the bases of ubiquitous computing in smart environments and a topic undergoing intense research in the field of ambient assisted living. The increasingly large amount of data sets calls for machine learning methods. In this paper, we introduce a deep learning model that learns to classify human activities without using any prior knowledge. For this purpose, a Long Short Term Memory (LSTM) Recurrent Neural Network was applied to three real world smart home datasets. The results of these experiments show that the proposed approach outperforms the existing ones in terms of accuracy and performance.

Part 6: MAKE Semantics; International audience; With Google’s Federated Learning & Facebook’s introduction of client-side NLP into their chat service, the era of client-side Machine Learning is upon us. While interesting ML approaches beyond the realm of toy examples were hitherto confined to large data-centers and powerful GPU’s, exponential trends in technology and the introduction of billions of smartphones enable sophisticated processing swarms of even hand-held devices. Such approaches hold several promises: 1. Without the need for powerful server infrastructures, even small companies could be scalable to millions of users easily and cost-efficiently; 2. Since data only used in the learning process never need to leave the client, personal information can be used free of privacy and data security concerns; 3. Since privacy is preserved automatically, the full range of personal information on the client device can be utilized for learning; and 4. without round-trips to the server, results like recommendations can be made available to users much faster, resulting in enhanced user experience. In this paper we propose an architecture for federated learning from personalized, graph based recommendations computed on client devices, collectively creating & enhancing a global knowledge graph. In this network, individual users will ‘train’ their local recommender engines, while a server-based voting mechanism aggregates the developing client-side models, preventing over-fitting on highly subjective data from tarnishing the global model.

The success of statistical machine learning (ML) methods made the field of Artificial Intelligence (AI) so popular again, after the last AI winter. Meanwhile deep learning approaches even exceed human performance in particular tasks. However, such approaches have some disadvantages besides of needing big quality data, much computational power and engineering effort; those approaches are becoming increasingly opaque, and even if we understand the underlying mathematical principles of such models they still lack explicit declarative knowledge. For example, words are mapped to high-dimensional vectors, making them unintelligible to humans. What we need in the future are context-adaptive procedures, i.e. systems that construct contextual explanatory models for classes of real-world phenomena. This is the goal of explainable AI, which is not a new field; rather, the problem of explainability is as old as AI itself. While rule-based approaches of early AI were comprehensible “glass-box” approaches at least in narrow domains, their weakness was in dealing with uncertainties of the real world. Maybe one step further is in linking probabilistic learning methods with large knowledge representations (ontologies) and logical approaches, thus making results re-traceable, explainable and comprehensible on demand.

In recent years user privacy has become an important aspect in the development of the Internet of Things (IoT) services due to their privacy invasive nature. However, there has been comparatively little research so far that aims to understanding users' notion of privacy in connection with IoT. In this work, we aim to understand how and if contextual factors affect users' privacy perceptions of IoT environments. To ascertain privacy perceptions, we deployed a public online survey (N=236) and contacted interviews (N=41) to explore factors that could have an influence. Although a lot of the participants identified privacy risks in IoT and rated the collected information items with high privacy ratings, we find that quite a large number of participants would still decide to have the offered IoT service if they find it useful and practical for their daily lives despite the infringement on their privacy. We conclude by highlighting and analyzing the qualitative comments of the participants and suggest possible solutions for the identified issues.