Search

Digital Supply Chain Surveillance (DSCS) is the proactive monitoring and analysis of digital data that allows firms to extract information related to a supply network, without the explicit consent of firms involved in the supply chain. AI has made DSCS to become easier and larger-scale, posing significant opportunities for automated detection of actors and dependencies involved in a supply chain, which in turn, can help firms to detect risky, unethical and environmentally unsustainable practices. Here, we define DSCS, review priority areas using a survey conducted in the UK. Visibility, sustainability, resilience are significant areas that DSCS can support, through a number of machine-learning approaches and predictive algorithms. Despite anecdotal narrative on the importance of explainability of algorithmic results, practitioners often prefer accuracy over explainability; however, there are significant differences between industrial sectors and application areas. Using a case study, we highlight a number of concerns on the unchecked use of AI in DSCS, such as bias or misinterpretation resulting in erroneous conclusions, which may lead to suboptimal decisions or relationship damage. Building on this, we develop and discuss a number of illustrative cases to highlight risks that practitioners should be aware of, proposing key areas of further research. [ABSTRACT FROM AUTHOR]

Artificial Intelligence (AI) has emerged as a complementary technology in supply chain research. However, the majority of AI approaches explored in this context afford little to no explainability, which is a significant barrier to a broader adoption of AI in supply chains. In recent years, the need for explainability has been a strong impetus for research in hybrid AI methodologies that combine neural architectures with logic-based reasoning, which are collectively referred to as Neurosymbolic AI. The aim of this paper is to provide a comprehensive overview of supply chain management literature that employs approaches within the neurosymbolic AI spectrum. To that end, a systematic review is conducted, followed by bibliometric, descriptive and thematic analyses on the identified studies. Our findings indicate that researchers have primarily focused on the limited subset of neurofuzzy approaches, while some supply chain applications, such as performance evaluation and sustainability, and sectors such as pharmaceutical and construction have received less attention. To help address these gaps, we propose five pillars of neurosymbolic AI research for supply chains and provide four use cases of applying unexplored neurosymbolic AI approaches to address typical problems in supply chain management, including a discussion of prerequisites for adopting such technologies. We envision that the findings and contributions of this survey will help encourage further research in neurosymbolic AI for supply chains and increase adoption of such technologies within supply chain practice. [ABSTRACT FROM AUTHOR]

The growing ubiquity of autonomous robots across different fields necessitates that agents adapt to diverse tasks and ensure transparency and intelligibility in their decision-making processes. This study presents a novel framework that combines a graph-structured world model with large language models (LLMs) to address these requirements. First, a latent space is created to capture the reachability between distinct states. Next, a graph is constructed within this latent space by clustering an offline dataset that effectively captures the complex dynamics of the environment. Subsequently, LLMs are employed to redefine the reward function and relabel the dataset, thereby establishing a well-defined Markov decision process based on the previously learned graph. This relabeling process ensures that the agent's decision space aligns with user intentions. Consequently, a predictive world model is obtained, offering insights into potential future states and facilitating graph-based planning. Moreover, by inputting the planned path of the agent in the graph-structured world model into LLMs, natural language explanations can be generated to provide transparency in the decision-making process. Experiments on the D4RL benchmark validated the effectiveness of our approach in long-horizon planning, its adaptability to different user tasks, and its inherent explainability. [ABSTRACT FROM AUTHOR]

AbstractSpatial interpolation is essential for handling sparsity and missing spatial data. Current machine learning-based spatial interpolation methods are subject to the statistical constraints of spatial stratified heterogeneity (SSH), normally involving separate modeling of each stratum and simple weighted averaging to integrate intra-stratum and inter-strata features. However, these models overlook the different contributions of inter-strata features to different locations within a stratum (heterogeneous inter-strata associations, HIA) and the explanation of spatial effects on the interpolation process, leading to suboptimal and unreliable interpolation outcomes. This article proposes a novel explainable spatial interpolation method considering SSH (X-SSHM). Spatial and environmental features are utilized to describe intra-stratum and inter-strata information, which are fed into random forest-based learners to achieve high-level semantic feature mapping. Geographically weighted regression is employed to integrate intra-stratum and inter-strata features to achieve a unified expression of SSH and HIA, obtaining the final interpolation result. Geographically weighted Shapley (GSHAP) is proposed to decompose the marginal contributions of intra-stratum and inter-strata features. Model performance is evaluated on simulated and soil organic matter datasets. X-SSHM outperformed five baselines regarding interpolation accuracy. Moreover, statistical methods validated X-SSHM’s ability to elucidate the mechanisms by which SSH, spatial autocorrelation and HIA affect the model interpolation process. [ABSTRACT FROM AUTHOR]

Determining the life stage of animals is crucial for assessing their overall health and making informed decisions for their care. However, traditional veterinary practices often rely on subjective methods, resulting in fuzziness or ambiguity. A quantitative metric via measurable vital signs has yet to be developed, which could be due to lack of a sophisticated methodology that can correlate the vital signs with age. To tackle this challenge, we present a method for equine life stage (young to old) classification from photoplethysmography (PPG) waveforms collected from 50 equine subjects with various ages, sex, and breeds. The data were collected using compact, wearable PPG sensors on their tails. The collected waveforms served as the input for the classification by an echo state network (ESN). It was found that the classification accuracy depended on the age split, suggesting a fussiness in classifying the youth and elder. Using the 17th year as the splitting age, the highest training and testing accuracies of 81.3% and 81.1% were achieved, respectively. In addition, input features from the reservoirs of the ESN were extracted and analyzed by a kernel principal component analysis, which afforded a 3D PCA map with clear clusters according to the age groups. This suggests that the ESN can learn hidden information from the PPG waves to classify the life stage. The proposed algorithm exhibits immense potential in veterinary medicine by offering a more objective and reliable approach to animal life stage assessment. [ABSTRACT FROM AUTHOR]

This paper presents a probabilistic model for various machine learning (ML) applications. While deep learning (DL) has produced state-of-the-art results in many domains, DL models are complex and over-parameterized, which leads to high uncertainty about what the model has learned, as well as its decision process. Further, DL models are not probabilistic, making reasoning about their output challenging. In contrast, the proposed model, referred to as Yet Another Discriminate Analysis(YADA), is less complex than other methods, is based on a mathematically rigorous foundation, and can be utilized for a wide variety of ML tasks including classification, explainability, and uncertainty quantification. YADA is thus competitive in most cases with many state-of-the-art DL models. Ideally, a probabilistic model would represent the full joint probability distribution of its features, but doing so is often computationally expensive and intractable. Hence, many probabilistic models assume that the features are either normally distributed, mutually independent, or both, which can severely limit their performance. YADA is an intermediate model that (1) captures the marginal distributions of each variable and the pairwise correlations between variables and (2) explicitly maps features to the space of multivariate Gaussian variables. Numerous mathematical properties of the YADA model can be derived, thereby improving the theoretic underpinnings of ML. Validation of the model can be statistically verified on new or held-out data using native properties of YADA. However, there are some engineering and practical challenges that we enumerate to make YADA more useful. [ABSTRACT FROM AUTHOR]

The automatic analysis of histology images is an open research field where machine learning techniques and neural networks, especially deep architectures, are considered successful tools due to their abilities in image classification. This paper proposes a granular computing methodology for histopathological image classification. It is based on embedding tiles of histopathology images using deep metric learning, where a self-organizing map is adopted to generate the granular structure in this learned embedding space. The SOM enables the implementation of an explainable mechanism by visualizing a knowledge space that the experts can use to analyze and classify the new images. Additionally, it provides confidence in the classification results while highlighting each important image fragment, with the benefit of reducing the number of false negatives. An exemplary case is when an image detail is indicated, with small confidence, as malignant in an image globally classified as benign. Another implemented feature is the proposal of additional labelled image tiles sharing the same characteristics to specify the context of the output decision. The proposed system was tested using three histopathology image datasets, obtaining the accuracy of the state-of-the-art black-box methods based on deep learning neural networks. Differently from the methodologies proposed so far for the same purpose, this paper introduces a novel explainable method for medical image analysis where the advantages of the deep learning neural networks used to build the embedding space for the image tiles are combined with the intrinsic explainability of the granular process obtained using the clustering property of a self-organizing map. [ABSTRACT FROM AUTHOR]

The remarkable success of Artificial Intelligence in advancing automated decision-making is evident both in academia and industry. Within the plethora of applications, ranking systems hold significant importance in various domains. This paper advocates for the application of a specific form of Explainable AI—namely, contrastive explanations—as particularly well-suited for addressing ranking problems. This approach is especially potent when combined with an Evaluative AI methodology, which conscientiously evaluates both positive and negative aspects influencing a potential ranking. Therefore, the present work introduces Evaluative Item-Contrastive Explanations tailored for ranking systems and illustrates its application and characteristics through an experiment conducted on publicly available data. [ABSTRACT FROM AUTHOR]

The geospatial domain increasingly relies on data-driven methodologies to extract actionable insights from the growing volume of available data. Despite the effectiveness of tree-based models in capturing complex relationships between features and targets, they fall short when it comes to considering spatial factors. This limitation arises from their reliance on univariate, axis-parallel splits that result in rectangular areas on a map. To address this issue and enhance both performance and interpretability, we propose a solution that introduces two novel bivariate splits: an oblique and Gaussian split designed specifically for geographic coordinates. Our innovation, called Geospatial Random Forest (geoRF), builds upon Geospatial Regression Trees (GeoTrees) to effectively incorporate geographic features and extract maximum spatial insights. Through an extensive benchmark, we show that our geoRF model outperforms traditional spatial statistical models, other spatial RF variations, machine learning and deep learning methods across a range of geospatial tasks. Furthermore, we contextualize our method's computational time complexity relative to baseline approaches. Our prediction maps illustrate that geoRF produces more robust and intuitive decision boundaries compared to conventional tree-based models. Utilizing impurity-based feature importance measures, we validate geoRF's effectiveness in highlighting the significance of geographic coordinates, especially in data sets exhibiting pronounced spatial patterns. [ABSTRACT FROM AUTHOR]

This paper presents the generation method for an explainable model based on the given information of a black box model using a concept of knowledge transfer to synthesize a dataset. The proposed method applies with GAN and Bee algorithm (BA) for data synthesis technique to synthesize a dataset by considering loss value in a knowledge transferring process to inherit the significance of features. The synthesized dataset is used to train for a proxy model as an explainable model. The result of the experiment indicates that knowledge transfer from Bee algo better than generative adversarial network (GAN) in terms of the coefficient of determination R2. In addition, explainable models from the synthesized data of the Bee-based method obtains F1 score superior to those from the GAN-based method in all datasets and settings. The dataset synthesized from the Bee-based method produces the explainable prediction model that has similar top-10 features according to similarity score of 0.6718 using shapley additive explanations (SHAP) feature importance which is higher than those from GAN-based method for 0.4218 in average. Additionally, experimental result to evaluate accuracy shows that F1 score from explainable models from the Bee-based method are closed to F1 score from a model generated from the original dataset. [ABSTRACT FROM AUTHOR]

Generative AI (GenAI) represents a shift from AI's ability to "recognize" to its ability to "generate" solutions for a wide range of tasks. As generated solutions and applications grow more complex and multi-faceted, new needs, objectives, and possibilities for explainability (XAI) have emerged. This work elaborates on why XAI has gained importance with the rise of GenAI and the challenges it poses for explainability research. We also highlight new and emerging criteria that explanations should meet, such as verifiability, interactivity, security, and cost considerations. To achieve this, we focus on surveying existing literature. Additionally, we provide a taxonomy of relevant dimensions to better characterize existing XAI mechanisms and methods for GenAI. We explore various approaches to ensure XAI, ranging from training data to prompting. Our paper provides a concise technical background of GenAI for non-technical readers, focusing on text and images to help them understand new or adapted XAI techniques for GenAI. However, due to the extensive body of work on GenAI, we chose not to delve into detailed aspects of XAI related to the evaluation and usage of explanations. Consequently, the manuscript appeals to both technical experts and professionals from other fields, such as social scientists and information systems researchers. Our research roadmap outlines over ten directions for future investigation. [ABSTRACT FROM AUTHOR]

Background: Amyotrophic lateral sclerosis (ALS) is a progressive neurodegenerative disease that results in death within a short time span (3-5 years). One of the major challenges in treating ALS is its highly heterogeneous disease progression and the lack of effective prognostic tools to forecast it. The main aim of this study was, then, to test the feasibility of predicting relevant clinical outcomes that characterize the progression of ALS with a two-year prediction horizon via artificial intelligence techniques using routine visits data. Methods: Three classification problems were considered: predicting death (binary problem), predicting death or percutaneous endoscopic gastrostomy (PEG) (multiclass problem), and predicting death or non-invasive ventilation (NIV) (multiclass problem). Two supervised learning models, a logistic regression (LR) and a deep learning multilayer perceptron (MLP), were trained ensuring technical robustness and reproducibility. Moreover, to provide insights into model explainability and result interpretability, model coefficients for LR and Shapley values for both LR and MLP were considered to characterize the relationship between each variable and the outcome. Results: On the one hand, predicting death was successful as both models yielded F1 scores and accuracy well above 0.7. The model explainability analysis performed for this outcome allowed for the understanding of how different methodological approaches consider the input variables when performing the prediction. On the other hand, predicting death alongside PEG or NIV proved to be much more challenging (F1 scores and accuracy in the 0.4-0.6 interval). Conclusions: In conclusion, predicting death due to ALS proved to be feasible. However, predicting PEG or NIV in a multiclass fashion proved to be unfeasible with these data, regardless of the complexity of the methodological approach. The observed results suggest a potential ceiling on the amount of information extractable from the database, e.g., due to the intrinsic difficulty of the prediction tasks at hand, or to the absence of crucial predictors that are, however, not currently collected during routine practice. [ABSTRACT FROM AUTHOR]

Background: Ageing is one of the most important challenges in our society. Evaluating how one is ageing is important in many aspects, from giving personalized recommendations to providing insight for long-term care eligibility. Machine learning can be utilized for that purpose, however, user reservations towards "black-box" predictions call for increased transparency and explainability of results. This study aimed to explore the potential of developing a machine learning-based healthy ageing scale that provides explainable results that could be trusted and understood by informal carers. Methods: In this study, we used data from 696 older adults collected via personal field interviews as part of independent research. Explanatory factor analysis was used to find candidate healthy ageing aspects. For visualization of key aspects, a web annotation application was developed. Key aspects were selected by gerontologists who later used web annotation applications to evaluate healthy ageing for each older adult on a Likert scale. Logistic Regression, Decision Tree Classifier, Random Forest, KNN, SVM and XGBoost were used for multi-classification machine learning. AUC OvO, AUC OvR, F1, Precision and Recall were used for evaluation. Finally, SHAP was applied to best model predictions to make them explainable. Results: The experimental results show that human annotations of healthy ageing could be modelled using machine learning where among several algorithms XGBoost showed superior performance. The use of XGBoost resulted in 0.92 macro-averaged AuC OvO and 0.76 macro-averaged F1. SHAP was applied to generate local explanations for predictions and shows how each feature is influencing the prediction. Conclusion: The resulting explainable predictions make a step toward practical scale implementation into decision support systems. The development of such a decision support system that would incorporate an explainable model could reduce user reluctance towards the utilization of AI in healthcare and provide explainable and trusted insights to informal carers or healthcare providers as a basis to shape tangible actions for improving ageing. Furthermore, the cooperation with gerontology specialists throughout the process also indicates expert knowledge as integrated into the model. [ABSTRACT FROM AUTHOR]

As they become more common, automated systems are also becoming increasingly opaque, challenging their users' abilities to explain and interpret their outputs. In this study, we test the predictions of fuzzy-trace theory—a leading theory of how people interpret quantitative information—on user decision making after interacting with an online decision aid. We recruited a sample of 205 online crowdworkers and asked them to use a system that was designed to detect URLs that were part of coordinated misinformation campaigns. We examined how user endorsements of system interpretability covaried with performance on this coordinated misinformation detection task and found that subjects who endorsed system interpretability displayed enhanced discernment. This interpretability was, in turn, associated with both objective mathematical ability and mathematical self-confidence. Beyond these individual differences, we evaluated the impact of a theoretically motivated intervention that was designed to promote sensemaking of system output. Participants provided with a "gist" version of system output, expressing the bottom-line meaning of that output, were better able to identify URLs that might have been part of a coordinated misinformation campaign, compared to users given the same information presented as verbatim quantitative metrics. This work highlights the importance of enabling users to grasp the essential, gist meaning of the information they receive from automated systems, which benefits users regardless of individual differences. [ABSTRACT FROM AUTHOR]

Battery lifetime prediction at early cycles is crucial for researchers and manufacturers to examine product quality and promote technology development. Machine learning has been widely utilized to construct data-driven solutions for high-accuracy predictions. However, the internal mechanisms of batteries are sensitive to many factors, such as charging/discharging protocols, manufacturing/storage conditions, and usage patterns. These factors will induce state transitions, thereby decreasing the prediction accuracy of data-driven approaches. Transfer learning is a promising technique that overcomes this difficulty and achieves accurate predictions by jointly utilizing information from various sources. Hence, we develop two transfer learning methods, Bayesian Model Fusion and Weighted Orthogonal Matching Pursuit, to strategically combine prior knowledge with limited information from the target dataset to achieve superior prediction performance. From our results, our transfer learning methods reduce root-mean-squared error by 41% through adapting to the target domain. Furthermore, the transfer learning strategies identify the variations of impactful features across different sets of batteries and therefore disentangle the battery degradation mechanisms and the root cause of state transitions from the perspective of data mining. These findings suggest that the transfer learning strategies proposed in our work are capable of acquiring knowledge across multiple data sources for solving specialized issues. [ABSTRACT FROM AUTHOR]

Algorithmic risk assessment tools, such as COMPAS, are increasingly used in criminal justice systems to predict the risk of defendants to reoffend in the future. This paper argues that these tools may not only predict recidivism, but may themselves causally induce recidivism through self-fulfilling predictions. We argue that such "performative" effects can yield severe harms both to individuals and to society at large, which raise epistemic-ethical responsibilities on the part of developers and users of risk assessment tools. To meet these responsibilities, we present a novel desideratum on algorithmic tools, called explainability-in-context, which requires clarifying how these tools causally interact with the social, technological, and institutional environments they are embedded in. Risk assessment practices are thus subject to high epistemic standards, which haven't been sufficiently appreciated to date. Explainability-in-context, we contend, is a crucial goal to pursue in addressing the ethical challenges surrounding risk assessment tools. [ABSTRACT FROM AUTHOR]

Complex Machine Learning (ML) models used to support decision-making in peer-to-peer (P2P) lending often lack clear, accurate, and interpretable explanations. While the game-theoretic concept of Shapley values and its computationally efficient variant Kernel SHAP may be employed for this aim, similarly to other existing methods, the latter makes the assumption that the features are independent. The assumption of uncorrelated features in credit risk management is fairly restrictive and, thus, prediction explanations coming from correlated features might result in highly misleading Shapley values, even when considering simple models. We therefore propose an evaluation of different dependent-feature estimation methods of Kernel SHAP for classification purposes in credit risk management. We show that dependent-feature estimation of Shapley values can improve the understanding of true prediction explanations, their robustness and is essential for better identifying the most relevant variables to default predictions coming from black-box ML models. RESEARCH HIGHLIGHTS: We propose estimation of feature-dependent Shapley values for P2P credit risk management We consider different linear and non-linear predictive models with varying degrees of dependence Dependent feature estimation of Shapley values can improve prediction explanations and their robustness Loan amount and interest rate are the most determinant features to loan default prediction explanations [ABSTRACT FROM AUTHOR]

Natural language inference (NLI) is a fundamental and quite challenging task in natural language processing, requiring efficient methods that are able to determine whether given hypotheses derive from given premises. In this paper, we apply explainability techniques to natural-language-inference methods as a means to illustrate the decision-making procedure of its methods. First, we investigate the performance and generalization capabilities of several transformer-based models, including BERT, ALBERT, RoBERTa, and DeBERTa, across widely used datasets like SNLI, GLUE Benchmark, and ANLI. Then, we employ stacking-ensemble techniques to leverage the strengths of multiple models and improve inference performance. Experimental results demonstrate significant improvements of the ensemble models in inference tasks, highlighting the effectiveness of stacking. Specifically, our best-performing ensemble models surpassed the best-performing individual transformer by 5.31% in accuracy on MNLI-m and MNLI-mm tasks. After that, we implement LIME and SHAP explainability techniques to shed light on the decision-making of the transformer models, indicating how specific words and contextual information are utilized in the transformer inferences procedures. The results indicate that the model properly leverages contextual information and individual words to make decisions but, in some cases, find difficulties in inference scenarios with metaphorical connections which require deeper inferential reasoning. [ABSTRACT FROM AUTHOR]

The proliferation of fake news threatens the integrity of information ecosystems, creating a pressing need for effective and interpretable detection mechanisms. Recent advances in machine learning, particularly with transformer-based models, offer promising solutions due to their superior ability to analyze complex language patterns. However, the practical implementation of these solutions often presents challenges due to their high computational costs and limited interpretability. In this work, we explore using content-based features to enhance the explainability and effectiveness of fake news detection. We propose a comprehensive feature framework encompassing characteristics related to linguistic, affective, cognitive, social, and contextual processes. This framework is evaluated across several public English datasets to identify key differences between fake and legitimate news. We assess the detection performance of these features using various traditional classifiers, including single and ensemble methods and analyze how feature reduction affects classifier performance. Our results show that, while traditional classifiers may not fully match transformer-based models, they achieve competitive results with significantly lower computational requirements. We also provide an interpretability analysis highlighting the most influential features in classification decisions. This study demonstrates the potential of interpretable features to build efficient, explainable, and accessible fake news detection systems. [ABSTRACT FROM AUTHOR]

The fundamental period is one of the most important parameters for the design of new structures as well as for estimating the capacity of existing ones. Thus, to estimate it, various design codes and researchers have adopted several approximate analytical equations based on a number of key structural parameters. To this end, the present study introduces a novel methodology for deriving the analytical equations for the fundamental period of reinforced concrete structures. The methodology is based on machine learning explainability techniques, specifically the so-called SHapley Additive exPlanations values. These values are commonly employed as an explainability tool. However, in the proposed novel approach they are employed as a basis to fit analytical curves, which allows the resulting equations to be constructed sequentially and in an informed manner while controlling the balance between accuracy and complexity. An extended dataset consisting of 4026 data points is employed, on which a Gradient Boosting Machine model is fitted. The model achieves excellent accuracy, with a coefficient of determination R 2 ≈ 0.99 , while the equations derived from the proposed formulation achieve an R 2 ≈ 0.95 and Mean Absolute Error ≈ 0.12 . This demonstrates the potential applicability of the proposed methodology in a wide array of similar engineering challenges. [ABSTRACT FROM AUTHOR]

A multitude of classifiers can be trained on the same data to achieve similar performances during test time while having learned significantly different classification patterns. When selecting a classifier, the machine learning practitioner has no understanding on the differences between models, their limits, where they agree and where they don't. But this choice will result in concrete consequences for instances to be classified in the discrepancy zone, since the final decision will be based on the selected classification pattern. Besides the arbitrary nature of the result, a bad choice could have further negative consequences such as loss of opportunity or lack of fairness. This paper proposes to address this question by analyzing the prediction discrepancies in a pool of best-performing models trained on the same data. A model-agnostic algorithm, DIG, is proposed to capture and explain discrepancies locally in tabular datasets, to enable the practitioner to make the best educated decision when selecting a model by anticipating its potential undesired consequences. [ABSTRACT FROM AUTHOR]

Background: Despite continuous performance improvements, especially in clinical contexts, a major challenge of Artificial Intelligence based Decision Support Systems (AI-DSS) remains their degree of epistemic opacity. The conditions of and the solutions for the justified use of the occasionally unexplainable technology in healthcare are an active field of research. In March 2024, the European Union agreed upon the Artificial Intelligence Act (AIA), requiring medical AI-DSS to be ad-hoc explainable or to use post-hoc explainability methods. The ethical debate does not seem to settle on this requirement yet. This systematic review aims to outline and categorize the positions and arguments in the ethical debate. Methods: We conducted a literature search on PubMed, BASE, and Scopus for English-speaking scientific peer-reviewed publications from 2016 to 2024. The inclusion criterion was to give explicit requirements of explainability for AI-DSS in healthcare and reason for it. Non-domain-specific documents, as well as surveys, reviews, and meta-analyses were excluded. The ethical requirements for explainability outlined in the documents were qualitatively analyzed with respect to arguments for the requirement of explainability and the required level of explainability. Results: The literature search resulted in 1662 documents; 44 documents were included in the review after eligibility screening of the remaining full texts. Our analysis showed that 17 records argue in favor of the requirement of explainable AI methods (xAI) or ad-hoc explainable models, providing 9 categories of arguments. The other 27 records argued against a general requirement, providing 11 categories of arguments. Also, we found that 14 works advocate the need for context-dependent levels of explainability, as opposed to 30 documents, arguing for context-independent, absolute standards. Conclusions: The systematic review of reasons shows no clear agreement on the requirement of post-hoc explainability methods or ad-hoc explainable models for AI-DSS in healthcare. The arguments found in the debate were referenced and responded to from different perspectives, demonstrating an interactive discourse. Policymakers and researchers should watch the development of the debate closely. Conversely, ethicists should be well informed by empirical and technical research, given the frequency of advancements in the field. [ABSTRACT FROM AUTHOR]

Background Purpose Methods Results Conclusions Deep learning‐based methods led to significant advancements in many areas of medical imaging, most of which are concerned with the reduction of artifacts caused by motion, scatter, or noise. However, with most neural networks being black boxes, they remain notoriously difficult to interpret, hindering their clinical implementation. In particular, it has been shown that networks exhibit invariances w.r.t. input features, that is, they learn to ignore certain information in the input data.To improve the interpretability of deep learning‐based low‐dose CT image denoising networks.We learn a complete data representation of low‐dose input images using a conditional variational autoencoder (cVAE). In this representation, invariances of any given denoising network are then disentangled from the information it is not invariant to using a conditional invertible neural network (cINN). At test time, image‐space invariances are generated by applying the inverse of the cINN and subsequent decoding using the cVAE. We propose two methods to analyze sampled invariances and to find those that correspond to alterations of anatomical structures.The proposed method is applied to four popular deep learning‐based low‐dose CT image denoising networks. We find that the networks are not only invariant to noise amplitude and realizations, but also to anatomical structures.The proposed method is capable of reconstructing and analyzing invariances of deep learning‐based low‐dose CT image denoising networks. This is an important step toward interpreting deep learning‐based methods for medical imaging, which is essential for their clinical implementation. [ABSTRACT FROM AUTHOR]

AbstractThe paper explores the synergy between wearable data and augmented Artificial Intelligence (AI) through findings from two interconnected studies. The first study (study 1) focuses on medical professionals’ perceptions of wearable data and AI, and the second study (Study 2) extends it focuses on how differences in the level of granularity in the data presented affect the professionals’ understanding, interpretation, and trust in AI recommendations. This system allows medical professionals to view AI-generated recommendations for sleep and activity improvement and explanations of the underlying rationale. While each study has distinct research questions, Study 2 builds upon Study 1's foundation. Both studies employed scenario-based interviews. Thematic analysis of Study 1 identified trust as a crucial factor in the acceptance of wearable data and AI, influencing Study 2's exploration of factors affecting trust, such as explainability, data granularity, representativeness, and user interaction. Study 2 highlighted varying perspectives on information sufficiency and data sharing from the AI system linked to professionals’ roles and tasks. The work offers insights into data granularity’s impact on engagement with AI recommendations. [ABSTRACT FROM AUTHOR]

The correlation between metasurface structures and their corresponding absorption spectra is inherently complex due to intricate physical interactions. Additionally, the reliance on Maxwell's equations for simulating these relationships leads to extensive computational demands, significantly hindering rapid development in this area. Numerous researchers have employed artificial intelligence (AI) models to predict absorption spectra. However, these models often act as black boxes. Despite training high-performance models, it remains challenging to verify if they are fitting to rational patterns or merely guessing outcomes. To address these challenges, we introduce the Explainable Encoder–Prediction–Reconstruction (EEPR) framework, which separates the prediction process into feature extraction and spectra generation, facilitating a deeper understanding of the physical relationships between metasurface structures and spectra and unveiling the model's operations at the feature level. Our model achieves a 66.23% reduction in average Mean Square Error (MSE), with an MSE of 2.843 ×   10 − 4 compared to the average MSE of 8.421 × 10 − 4 for mainstream networks. Additionally, our model operates approximately 500,000 times faster than traditional simulations based on Maxwell's equations, with a time of 3 × 10 − 3 seconds per sample, and demonstrates excellent generalization capabilities. By utilizing the EEPR framework, we achieve feature-level explainability and offer insights into the physical properties and their impact on metasurface structures, going beyond the pixel-level explanations provided by existing research. Additionally, we demonstrate the capability to adjust absorption by changing the metasurface at the feature level. These insights potentially empower designers to refine structures and enhance their trust in AI applications. [ABSTRACT FROM AUTHOR]

Road surface quality is essential for driver comfort and safety, making it crucial to monitor pavement conditions and detect defects in real time. However, the diversity of defects and the complexity of ambient conditions make it challenging to develop an effective and robust classification and detection algorithm. In this study, we adopted a semi-supervised learning approach to train ResNet-18 for image feature retrieval and then classification and detection of pavement defects. The resulting feature embedding vectors from image patches were retrieved, concatenated, and randomly sampled to model a multivariate normal distribution based on the only one-class training pavement image dataset. The calibration pavement image dataset was used to determine the defect score threshold based on the receiver operating characteristic curve, with the Mahalanobis distance employed as a metric to evaluate differences between normal and defect pavement images. Finally, a heatmap derived from the defect score map for the testing dataset was overlaid on the original pavement images to provide insight into the network's decisions and guide measures to improve its performance. The results demonstrate that the model's classification accuracy improved from 0.868 to 0.887 using the expanded and augmented pavement image data based on the analysis of heatmaps. [ABSTRACT FROM AUTHOR]

Featured Application: The information obtained from this study can be useful for coaches and performance analysts of teams. By taking into account the factors that contribute to securing one of the top positions in the league standings, coaches are encouraged to adopt ball possession strategies with patience and short passes to create gaps in the opponent's defense and, at the right moment, execute through balls aiming to enter the attacking third and create goal opportunities. On the other hand, teams are advised to avoid long passes, as these may lead to a loss of team cohesion and create large distances between the lines. Finally, this information is also useful for team scouts in identifying players who are suitable for implementing the aforementioned strategies. Introduction: Performance analysis is essential for coaches and a topic of extensive research. The advancement of technology and Artificial Intelligence (AI) techniques has revolutionized sports analytics. Aim: The primary aim of this article is to present a robust, explainable machine learning (ML) model that identifies the key factors that contribute to securing one of the top three positions in the standings of the French Ligue 1, ensuring participation in the UEFA Champions League for the following season. Materials and Methods: This retrospective observational study analyzed data from all 380 matches of the 2022–23 French Ligue 1 season. The data were obtained from the publicly-accessed website "whoscored" and included 34 performance indicators. This study employed Sequential Forward Feature Selection (SFFS) and various ML algorithms, including XGBoost, Support Vector Machine (SVM), and Logistic Regression (LR), to create a robust, explainable model. The SHAP (SHapley Additive Explanations) model was used to enhance model interpretability. Results: The K-means Cluster Analysis categorized teams into groups (TOP TEAMS, 3 teams/REST TEAMS, 17 teams), and the ML models provided significant insights into the factors influencing league standings. The LR classifier was the best-performing classifier, achieving an accuracy of 75.13%, a recall of 76.32%, an F1-score of 48.03%, and a precision of 35.17%. "SHORT PASSES" and "THROUGH BALLS" were features found to positively influence the model's predictions, while "TACKLES ATTEMPTED" and "LONG BALLS" had a negative impact. Conclusions: Our model provided satisfactory predictive accuracy and clear interpretability of results, which gave useful information to stakeholders. Specifically, our model suggests adopting a strategy during the ball possession phase that relies on short passes (avoiding long ones) and aiming to enter the attacking third and the opponent's penalty area with through balls. [ABSTRACT FROM AUTHOR]

Fruit quality assessment is vital for ensuring consumer satisfaction and marketability in agriculture. This study explores deep learning techniques for assessing fruit quality, focusing on practical deployment in resource-constrained environments. Two approaches were compared: training a convolutional neural network (CNN) from scratch and fine-tuning a pre-trained MobileNetV2 model through transfer learning. The performance of these models was evaluated using a subset of the Fruits-360 dataset chosen to simulate real-world conditions for small-scale producers. MobileNetV2 was selected for its compact size and efficiency, suitable for devices with limited computational resources. Both approaches achieved high accuracy, with the transfer learning model demonstrating faster convergence and slightly better performance. Feature map visualizations provided insight into the model's decision-making, highlighting damaged areas of fruits which enhances transparency and trust for end users. This study underscores the potential of deep learning models to modernize fruit quality assessment, offering practical, efficient, and interpretable tools for small-scale farmers. [ABSTRACT FROM AUTHOR]

It is expected that machine learning algorithms will enable better diagnosis, prognosis, and treatment in psychiatry. A central argument for deploying algorithmic methods in clinical decision-making in psychiatry is that they may enable not only faster and more accurate clinical judgments but also that they may provide a more objective foundation for clinical decisions. This article argues that the outputs of algorithms are never objective in the sense of being unaffected by human values and possibly biased choices. And it suggests that the best way to approach this is to ensure awareness of and transparency about the ethical trade-offs that must be made when developing an algorithm for mental health. [ABSTRACT FROM AUTHOR]

Prenatal alcohol exposure (PAE) refers to the exposure of the developing fetus due to alcohol consumption during pregnancy and can have life-long consequences for learning, behavior, and health. Understanding the impact of PAE on the developing brain manifests challenges due to its complex structural and functional attributes, which can be addressed by leveraging machine learning (ML) and deep learning (DL) approaches. While most ML and DL models have been tailored for adult-centric problems, this work focuses on applying DL to detect PAE in the pediatric population. This study integrates the pre-trained simple fully convolutional network (SFCN) as a transfer learning approach for extracting features and a newly trained classifier to distinguish between unexposed and PAE participants based on Tl-weighted structural brain magnetic resonance (MR) scans of individuals aged 2-8 years. Among several varying dataset sizes and augmentation strategy during training, the classifier secured the highest sensitivity of 88.47% with 85.04% average accuracy on testing data when considering a balanced dataset with augmentation for both classes. Moreover, we also preliminarily performed explainability analysis using the Grad-CAM method, highlighting various brain regions such as corpus callosum, cerebellum, pons, and white matter as the most important features in the model's decision-making process. Despite the challenges of constructing DL models for pediatric populations due to the brain's rapid development, motion artifacts, and insufficient data, this work highlights the potential of transfer learning in situations where data is limited. Furthermore, this study underscores the importance of preserving a balanced dataset for fair classification and clarifying the rationale behind the model's prediction using explainability analysis. [ABSTRACT FROM AUTHOR]

This paper presents a comparative analysis of deep learning techniques for anomaly detection and failure prediction. We explore various deep learning architectures on an IoT dataset, including recurrent neural networks (RNNs, LSTMs and GRUs), convolutional neural networks (CNNs) and transformers, to assess their effectiveness in anomaly detection and failure prediction. It was found that the hybrid transformer-GRU configuration delivers the highest accuracy, albeit at the cost of requiring the longest computational time for training. Furthermore, we employ explainability techniques to elucidate the decision-making processes of these black box models and evaluate their behaviour. By analysing the inner workings of the models, we aim at providing insights into the factors influencing failure predictions. Through comprehensive experimentation and analysis on sensor data collected from a water pump, this study contributes to the understanding of deep learning methodologies for anomaly detection and failure prediction and underscores the importance of model interpretability in critical applications such as prognostics and health management. Additionally, we specify the architecture for deploying these models in a real environment using the RAI4.0 metamodel, meant for designing, configuring and automatically deploying distributed stream-based industrial applications. Our findings will offer valuable guidance for practitioners seeking to deploy deep learning techniques effectively in predictive maintenance systems, facilitating informed decision-making and enhancing reliability and efficiency in industrial operations. [ABSTRACT FROM AUTHOR]

Land cover classification mapping is the process of assigning labels to different types of land surfaces based on overhead imagery. However, acquiring reference samples through fieldwork for ground truth can be costly and time‐intensive. Additionally, annotating high‐resolution satellite images poses challenges, as certain land cover types are difficult to discern solely from nadir images. To address these challenges, this study examined the feasibility of using street‐level imagery to support the collection of reference samples and identify land cover. We utilized 18,022 images captured in Japan, with 14 different land cover classes. Our approach involved using convolutional neural networks based on Inception‐v4 and DenseNet, as well as Transformer‐based Vision and Swin Transformers, both with and without pre‐trained weights and fine‐tuning techniques. Additionally, we explored explainability through Gradient‐Weighted Class Activation Mapping (Grad‐CAM). Our results indicate that using a Vision Transformer was the most effective method, achieving an overall accuracy of 86.12% and allowing for full explainability of land cover targets within an image. This paper proposes a promising solution for land cover classification from street‐level imagery, which can be used for semi‐automatic reference sample collection from geo‐tagged street‐level photos. [ABSTRACT FROM AUTHOR]

Controversy encompasses content that draws diverse perspectives, along with positive and negative feedback on a specific event, resulting in the formation of distinct user communities. we explore the explainability of controversy through the lens of SHAP (SHapley Additive exPlanations) method, aiming to provide a fair assessment of the individual contributions of different text features of tweets to controversy detection. We conduct an analysis of topic discussions on Twitter from a community perspective, investigating the role of text in accurately classifying tweets into their respective communities. To achieve this, we introduce a SHAP-based pipeline designed to quantify the influence of impactful text features on the predictions of three tweet classifiers. Text content alone offers interesting controversy detection accuracy. It can contain predictive features for controversy detection. For instance, negative connotations, pejorative tendencies and positive qualifying adjectives tend to impact the controversy model detection. [ABSTRACT FROM AUTHOR]

To avoid irreversible vision loss, early detection and diagnosis of Diabetic Retinopathy (DR) severity is critical. The percentage of people undertaking eye examinations has risen in recent years, increasing the burden on Ophthalmologists. Various Artificial Intelligence (AI) screening systems have recently been deployed to improve the accuracy of DR diagnosis. However, owing to their black-box nature, most successful AI screening systems are still held back in reality for medical decision aid. The need for an Explainable Artificial Intelligence (XAI) screening system to help Ophthalmologists in DR diagnosis is inevitable. The proposed work is divided into three phases: (i) pre-processing, (ii) optic disk localization, and (iii) DR severity classification. On top of the deep learning model, the proposed work implements a Local Interpretable Model-agnostic Explanations (LIME) explainer to describe what features of the retinal image took part in justifying the predictions. The proposed framework outputs a pixel-value tensor, explaining the possible pixel values contributing to the model's prediction. MESSIDOR data collection is used for experimental analysis. When compared with other deep learning models, the proposed framework achieved a better accuracy of 98.04%, sensitivity of 99.69%, specificity of 96.37%, f1-score of 96.99% and error rate of 3.60%. Incorporating explainable deep learning models for diabetic retinopathy severity grading improves diagnostic accuracy and provides clinicians with clear insights, enabling trust and informed decision-making in DR diagnosis. This proposed technique enormously advances more effective and responsible healthcare procedures. [ABSTRACT FROM AUTHOR]

Serendipity-oriented recommendation systems have been widely applied in major e-commerce and social platforms. Platform managers aim to enhance user satisfaction and increase platform sales by creating serendipitous encounters with information. Previous research has shown that the unexpectedness of encountering product information in serendipity-oriented recommendation systems can effectively stimulate positive emotions in customers, resulting in unplanned purchases, such as impulse buying. However, little research has focused on another critical aspect of encountering product information: perceived value. Our study suggests that encountering product information can positively affect the intention to purchase planned products (focal products) based on their perceived value. To explore this, we conducted three experiments and found that: (1) encountering product information positively influences planned product purchase intention (e.g., reduced decision-making time, improved focal product purchase intention), compared to the absence of encountering product information (precision-oriented recommendation systems); (2) this effect is mediated by customer inspiration; and (3) the characteristics of recommendation system strategies can moderate this effect. Specifically, when the strategy features exhibit a low level of explainability, the impact of encountering product information on customer inspiration and purchase intention is more significant than when a high level of explainability is presented. [ABSTRACT FROM AUTHOR]

Interpretable representations are the backbone of many explainers that target black-box predictive systems based on artificial intelligence and machine learning algorithms. They translate the low-level data representation necessary for good predictive performance into high-level human-intelligible concepts used to convey the explanatory insights. Notably, the explanation type and its cognitive complexity are directly controlled by the interpretable representation, tweaking which allows to target a particular audience and use case. However, many explainers built upon interpretable representations overlook their merit and fall back on default solutions that often carry implicit assumptions, thereby degrading the explanatory power and reliability of such techniques. To address this problem, we study properties of interpretable representations that encode presence and absence of human-comprehensible concepts. We demonstrate how they are operationalised for tabular, image and text data; discuss their assumptions, strengths and weaknesses; identify their core building blocks; and scrutinise their configuration and parameterisation. In particular, this in-depth analysis allows us to pinpoint their explanatory properties, desiderata and scope for (malicious) manipulation in the context of tabular data where a linear model is used to quantify the influence of interpretable concepts on a black-box prediction. Our findings lead to a range of recommendations for designing trustworthy interpretable representations; specifically, the benefits of class-aware (supervised) discretisation of tabular data, e.g., with decision trees, and sensitivity of image interpretable representations to segmentation granularity and occlusion colour. [ABSTRACT FROM AUTHOR]

Detecting and reacting to non-technical losses (NTL) is a fundamental activity that energy providers need to face in their daily routines. This is known to be challenging since the phenomenon of NTL is multi-factored, dynamic and extremely contextual, which makes artificial intelligence (AI) and, in particular, machine learning, natural areas to bring effective and tailored solutions. If the human factor is disregarded in the process of detecting NTL, there is a high risk of performance degradation since typical problems like dataset shift and biases cannot be easily identified by an algorithm. This paper presents a case study on incorporating explainable AI (XAI) in a mature NTL detection system that has been in production in the last years both in electricity and gas. The experience shows that incorporating this capability brings interesting improvements to the initial system and especially serves as a common ground where domain experts, data scientists, and business analysts can meet. [ABSTRACT FROM AUTHOR]

We examine whether data generated by explanation techniques, which promote a process of self-reflection, can improve classifier performance. Our work is based on the idea that humans have the ability to make quick, intuitive decisions as well as to reflect on their own thinking and learn from explanations. To the best of our knowledge, this is the first time that the potential of mimicking this process by using explanations generated by explainability methods has been explored. We found that combining explanations with traditional labeled data leads to significant improvements in classification accuracy and training efficiency across multiple image classification datasets and convolutional neural network architectures. It is worth noting that during training, we not only used explanations for the correct or predicted class, but also for other classes. This serves multiple purposes, including allowing for reflection on potential outcomes and enriching the data through augmentation. [ABSTRACT FROM AUTHOR]

The accuracy and flexibility of Deep Convolutional Neural Networks (DCNNs) have been highly validated over the past years. However, their intrinsic opaqueness is still affecting their reliability and limiting their application in critical production systems, where the black-box behavior is difficult to be accepted. This work proposes EBAnO, an innovative explanation framework able to analyze the decision-making process of DCNNs in image classification by providing prediction-local and class-based model-wise explanations through the unsupervised mining of knowledge contained in multiple convolutional layers. EBAnO provides detailed visual and numerical explanations thanks to two specific indexes that measure the features' influence and their influence precision in the decision-making process. The framework has been experimentally evaluated, both quantitatively and qualitatively, by (i) analyzing its explanations with four state-of-the-art DCNN architectures, (ii) comparing its results with three state-of-the-art explanation strategies and (iii) assessing its effectiveness and easiness of understanding through human judgment, by means of an online survey. EBAnO has been released as open-source code and it is freely available online. [ABSTRACT FROM AUTHOR]