Search

Time plays an integral role in video understanding, from disambiguating gestures like "swipe left" from "swipe right" to being the dimension over which activity unfolds. It is thus of great importance to leverage information across time in an effective manner to build successful systems for video understanding. By exploring time from three different perspectives, this thesis sheds light on which parts of video are informative, how label symmetries can be used to reduce the need for so much training data, and how activities can be explicitly modeled in the representation of video. Whilst symmetries in data have been well exploited for data augmentation purposes, the relationships among the labels used to supervise models have seen less attention. Both temporal and spatial relationships between fine-grained action labels are explored for the purposes of data augmentation and zero-shot learning. The dominant approach for applying deep learning methods to video for recognition has been to sample a sequence of frames at regular intervals as input. However, not all frames are made equal; some contain less information than others or are redundant to frames already sampled. This thesis presents a method used to quantify the importance of frames from the perspective of a trained model. A variety of models have been proposed for video recognition that model temporal relationships in different ways, but direct comparison has often been difficult due to differences in evaluation protocol. A benchmark study of these common models on the EPIC-KITCHENS dataset under a common evaluation protocol is presented to assess the relative merits of each one. One of the challenges faced in the video understanding community is the difficulty of scaling models up to longer video. A novel representation of video is proposed that explicitly models observed activities within a video up to a point in time to enable efficient processing of subsequent video. Additionally, a self-supervised pre-training procedure is introduced to bootstrap the model from long unlabelled videos.

Deep neural networks are becoming more and more popular due to their revolutionary success in diverse areas, such as computer vision, natural language processing, and speech recognition. However, the decision-making processes of these models are generally not interpretable to users. In various domains, such as healthcare, finance, or law, it is critical to know the reasons behind a decision made by an artificial intelligence system. Therefore, several directions for explaining neural models have recently been explored. In this thesis, I investigate two major directions for explaining deep neural networks. The first direction consists of feature-based post-hoc explanatory methods, that is, methods that aim to explain an already trained and fixed model (post-hoc), and that provide explanations in terms of input features, such as tokens for text and superpixels for images (feature-based). The second direction consists of self-explanatory neural models that generate natural language explanations, that is, models that have a built-in module that generates explanations for the predictions of the model. The contributions in these directions are as follows. First, I reveal certain difficulties of explaining even trivial models using only input features. I show that, despite the apparent implicit assumption that explanatory methods should look for one specific ground-truth feature-based explanation, there is often more than one such explanation for a prediction. I also show that two prevalent classes of explanatory methods target different types of ground-truth explanations without explicitly mentioning it. Moreover, I show that, sometimes, neither of these explanations is enough to provide a complete view of a decision-making process on an instance. Second, I introduce a framework for automatically verifying the faithfulness with which feature-based post-hoc explanatory methods describe the decision-making processes of the models that they aim to explain. This framework relies on the use of a particular type of model that is expected to provide insight into its decision-making process. I analyse potential limitations of this approach and introduce ways to alleviate them. The introduced verification framework is generic and can be instantiated on different tasks and domains to provide off-the-shelf sanity tests that can be used to test feature-based post-hoc explanatory methods. I instantiate this framework on a task of sentiment analysis and provide sanity tests on which I present the performances of three popular explanatory methods. Third, to explore the direction of self-explanatory neural models that generate natural language explanations for their predictions, I collected a large dataset of ~570K human-written natural language explanations on top of the influential Stanford Natural Language Inference (SNLI) dataset. I call this explanation-augmented dataset e-SNLI. I do a series of experiments that investigate both the capabilities of neural models to generate correct natural language explanations at test time, and the benefits of providing natural language explanations at training time. Fourth, I show that current self-explanatory models that generate natural language explanations for their own predictions may generate inconsistent explanations, such as ``There is a dog in the image.'' and ``There is no dog in the [same] image.''. Inconsistent explanations reveal either that the explanations are not faithfully describing the decision-making process of the model or that the model learned a flawed decision-making process. I introduce a simple yet effective adversarial framework for sanity checking models against the generation of inconsistent natural language explanations. Moreover, as part of the framework, I address the problem of adversarial attacks with exact target sequences, a scenario that was not previously addressed in sequence-to-sequence attacks, and which can be useful for other tasks in natural language processing. I apply the framework on a state of the art neural model on e-SNLI and show that this model can generate a significant number of inconsistencies. This work paves the way for obtaining more robust neural models accompanied by faithful explanations for their predictions.

The advent of deep neural networks (DNNs) have brought exciting new possibilities for the realization of automated driving functions. These data-driven methods have been widely applied to various driving tasks, including end-to-end urban driving. However, the use of these methods beyond simulated tests remains limited due to two significant shortcomings: (i) the lack of model transparency and (ii) the difficulty of generalizing beyond the training distribution. This dissertation aims to investigate methods for addressing the transparency and safety mitigation of learning-based controllers, specifically deep reinforcement learning (DRL) methods, to enable safe and predictable driving. To enhance interpretability, an interpretable and causal state representation, coined the driving forces, is proposed. This representation captures the causal relationship between the state and the produced control action by leveraging force features to encode the influence of internal and external factors on the ego vehicle. By training a DRL agent on this representation within a highway driving environment, the ability of the driving forces to encode and interpret the state-action causalities was demonstrated. Furthermore, an alternative paradigm for online adaptation by modifying the formulation of the driving forces is proposed to mitigate the behavior of the ego vehicle. The results showed that the ego vehicle was successful in mitigating its behavior and following desired new and unseen behaviors, without requiring modification to the underlying DNN. To address model transparency in black-box DNN-based driving policies, a knowledge distillation framework that combines interpretable decision trees with rule learning algorithms is proposed. This framework learns decision rule sets that represent the decision boundaries of the original driving policy. The driving forces are utilized to abstract the original state representation and ensure the interpretability of the learned explanations. This framework was tested on the highway driving policy, and the results demonstrated the ability to learn explanations of the decision boundaries with high precision. Moreover, the driving forces enabled visual representations of textual explanations, overcoming the loss of interpretability found in long rule lists. Two methods based on formal safety certification are proposed to provide stability and safety definitions on the learned control policies. A two-step framework (LSR-DDPG) is proposed that combines Lyapunov stability methods with continuous DRL control. The proposed framework learns a control Lyapunov function and its associated stable closed-loop dynamics, and then uses these models to regulate the objective of the DRL algorithm on learning stabilizing trajectories. The results demonstrated the ability to learn stable trajectories. To safely handle changing domain constraints, a minimally invasive control synthesis is proposed that combines control barrier functions with model predictive control. The proposed scheme was implemented on the DRL controller, and the ego vehicle was able to dynamically adjust its trajectory and behavior to changing domain constraints. These results provided necessary safety certifications that can be used to design safe DNN-based driving policies.

Deep neural networks (DNNs) can perform impressively in many natural language processing (NLP) tasks, but their black-box nature makes them inherently challenging to explain or interpret. Self-Explanatory models are a new approach to overcoming this challenge, generating explanations in human-readable languages besides task objectives like answering questions. The main focus of this thesis is the explainability of NLP tasks, as well as how attention methods can help enhance performance. Three different attention modules are proposed, SimpleAttention, CrossSelfAttention, and CrossModality. It also includes a new dataset transformation method called Two-Documents that converts every dataset into two separate documents required by the offered attention modules. The proposed ideas are incorporated in a faithful architecture in which a module produces an explanation and prepares the information vector for the subsequent layers. The experiments are run on the ERASER Benchmark’s CoS-E dataset, restricting them to the transformer used in the baseline and only training data from the dataset while it requires common sense knowledge to improve the accuracy. Based on the results, the proposed solution produced an explanation that outperformed Token F1 by about 4%, while being about 1% more accurate. Adviser: Stephen Scott

Abstract: The rise of Deep Learning (DL) and its assistance in learning complex feature representations significantly impacted Reinforcement Learning (RL). Deep Reinforcement Learning (DRL) made it possible to apply RL to complex real-world problems and even achieve human-level performance. One of these problems is related to robotics. Recently, DRL agents successfully learned optimal behavior in a range of robotic environments. The policy can provide much information from its learned representation. However, this policy is approximated using a neural network and, therefore, is a black box. Explainable Artificial Intelligence (XAI) is a new AI subfield focusing on interpreting Machine Learning models' behavior. A large part of XAI's literature has emerged on feature relevance techniques to explain a deep neural network (DNN) output processing on images. These techniques have been extended to explain Graph classification tasks using Graph Networks (GN). Nevertheless, these methods haven't been exploited to analyze the DRL agent's behavior learned to perform in a robotic environment. In this work, we proposed to analyze the representation learned by a DRL agent's policy in a robotic environment. We use graph structure to represent the robot's observation in an entity-relationship manner and graph neural networks as function approximators in DRL. For the interpretation phase, an explainability technique called Layer-wise Relevance Propagation (LRP), a feature relevance technique that had been successfully applied to explain image and graph classification tasks, is used to interpret the learned policy. We evaluate the information provided by the LRP on two simulated robotic environments on MuJoCo. The experiments and evaluation methods were delicately designed to effectively measure the value of knowledge gained by our approach to analyzing learned representations in the Deep Reinforcement Learning task.

The performance gains and expanded utilization of deep learning models in the fields of machine learning and natural language processing have been followed by a need for the internal mechanisms guiding them to be explainable and accompanied by methods allowing humans to diagnose and correct such models at inference time if needed. In contrast to post-hoc methods for explainability that train a secondary model to infer the decision reasoning of a primary model by using only its inputs and outputs, in-process methods offer faithful explanations of a model’s decisions by explicitly training the model to include such capabilities as an additional objective rather than trying to infer them in a post-hoc manner. Such methods should scale without sacrificing model performance and be sufficiently broad enough to incorporate diverse tasks and data types including sequential language, time series, and multi-modal data. Of particular interest is the analysis of such techniques for the learning of rich dense or sparse interpretable entity representations tied to knowledge bases. In this thesis we try to address these aims by developing efficient frameworks that handle different data types and provide diverse, in-process explainable techniques for transparent and trustworthy models. First, we show that it is feasible to learn dense entity representations from text via a dual encoder framework that encodes mentions and entities in the same dense vector space. Such representations can then be used for extremely fast entity linking where candidate entities are retrieved by approximate nearest neighbor search and generalize well to new datasets. During training the model leverages a novel negative mining algorithm which guides learning by iteratively constructing training batches to contain top candidates that were previously incorrectly ranked above the true entity. The technique dramatically improves model accuracy over iterations and the final batches can be viewed as samples most difficult for the model to learn. We then introduce a framework for learning in-process prototypes from an autoencoder that provides both instance-level and global explanations for time series classification. We explicitly optimize for increased prototype diversity which improves model accuracy and produces prototypes generated by learning regions of the latent space that highlight features the model uses for distinguishing amongst classes. We show that the prototypes are capable of learning real-world features - in our case-study ECG morphology related to bradycardia. Next we derive Biomedical Interpretable Entity Representations (BIER) in which dimensions correspond to fine-grained entity types, and values are predicted probabilities that a given entity is of the corresponding type. We propose a diagnostic method that exploits BIER’s final sparse and intermediate dense representations to facilitate model and entity type debugging and show BIERs achieve strong performance in biomedical tasks including named entity disambiguation and entity linking. We next propose a method for entity-based knowledge injection for the multimodal Knowledge-Based Visual Question Answering (KBVQA) task, which contains questions whose answers explicitly require external knowledge about named entities within an image, and study how it affects both task accuracy and an existing inprocess, bi-modal explainability technique. Our results show substantially improved performance on the KBVQA task without the need for additional costly pre-training, and we provide insights for when entity knowledge injection helps improve a model’s understanding. Finally, we introduce Intermediate enTity-based Sparse Interpretable Representation Learning (ItsIRL), an architecture that allows for fine-tuning of sparse, interpretable entity representations (IERs) on downstream tasks while preserving the semantics of the dimensions learned during pretraining. This approach surpasses prior IERs work and realizes competitive performance with dense models on biomedical tasks. We propose and study ‘counterfactual’ entity type manipulation techniques made possible by our architecture that allows fixing of ItsIRL errors that can surpass performance against dense non- interpretable models. Additionally, we propose a method to construct entity type based class prototypes for showing global semantic properties learned by our model, both for positive and negative instances.

The dependence on Natural Language Processing (NLP) systems has grown significantly in the last decade. Recent advances in deep learning have enabled language models to generate high-quality text at the same level as human-written text. If this growth continues, it can potentially lead to increased misinformation, which is a significant challenge. Although claim verification techniques exist, they lack proper explainability. Numerical scores such as Attention and Lime and visualization techniques such as saliency heat maps are insufficient because they require specialized knowledge. It is inaccessible and challenging for the nonexpert to understand black-box NLP systems. We propose a novel approach called, ExClaim for explainable claim verification using NLP rationalization. We demonstrate that our approach can predict a verdict for the claim but also justify and rationalize its output as a natural language explanation (NLE). We extensively evaluate the system using statistical and Explainable AI (XAI) metrics to ensure the outcomes are valid, verified, and trustworthy to help reinforce the human-AI trust. We propose a new subfield in XAI called Rational AI (RAI) to improve research progress on rationalization and NLE-based explainability techniques. Ensuring that claim verification systems are assured and explainable is a step towards trustworthy AI systems and ultimately helps mitigate misinformation.

As artificial intelligence (AI) systems increasingly impact the society, it is important to design and maintain models that are responsible and trustworthy. Models should not discriminate against certain individuals or a group of individuals (fairness), the decisions should be explainable, and they should be robust to adversarial attacks. Moreover, the trained models should be dynamically updated if the data changes over time, and methods to provide explanations for model decisions need to operate efficiently and in real-time. In this thesis, we address these challenges by developing frameworks that can account for more than one characteristic of responsible artificial intelligence. First, we evaluate existing black-box models using CERTIFAI: Counterfactual Explanations for Robustness, Transparency, Interpretability, and Fairness of AI models. CERTIFAI uses a custom genetic algorithm to produce counterfactual explanations, which are generated points close to the input point but belonging to a different class. These points can then be used to: provide explanations, measure feature importance, evaluate fairness based on an introduced notion called burden, and measure the robustness to adversarial attacks. We then introduce FASTER-CE: a novel set of algorithms to generate fast, sparse, and robust counterfactual explanations. The backbone of the proposed method is an autoencoder trained on the original dataset. Random samples from the latent space of the trained autoencoder are used to train linear models for each of the features in the dataset and for the black-box model predictions. Using these trained linear models and additional user-defined constraints, we easily compute the direction for counterfactual explanation search and generate multiple counterfactual explanations that are sparse, realistic, and robust to input manipulations. We show that FASTER-CE is much faster than other state of the art methods to generate counterfactual explanations in generating multiple explanations with several desirable, and often conflicting, properties. Additionally, we explore the trade-offs between the sparsity, proximity, validity, speed of generation, and the robustness of explanations. Next, we look into training a fairer model by creating a data augmentation based pre-processing bias mitigation technique that also lends itself towards bias disambiguation called FaiDA (fair data augmentation). We theoretically show that two different notions of fairness: statistical parity difference (independence) and average odds difference (separation) always change in the same direction using such an augmentation. We also show submodularity of the proposed fairness-aware augmentation approach that enables an efficient greedy algorithm. To make models fair and robust, we introduce an in-processing bias mitigation technique FaiR-N: Fair and Robust Neural Networks, that trains models with regularizers to improve on burden-based fairness and adversarial robustness. We show that models can be trained with these considerations without compromising significantly on accuracy, that improving on burden based fairness also improves other fairness measures, and also discuss trade-offs between fairness and adversarial robustness. We then focus on training models that are more fair and can also account for drift, where the drift could be with respect to accuracy and fairness. We propose FEAMOE, a mixture of experts framework aimed at learning fairer, more interpretable models that can also rapidly adjust to drifts in both the accuracy and fairness of a classifier. We illustrate our framework for three popular fairness measures and demonstrate how drift can be handled with respect to these fairness constraints, while also providing fast explanations. Our framework, as applied to a mixture of linear experts, is able to perform comparably to neural networks in terms of accuracy while producing fairer and more interpretable models that are dynamically updated to account for drift.

Machine Learning (ML) algorithms are widely used in our daily lives. The need to increase the accuracy of ML models has led to building increasingly powerful and complex algorithms known as black-box models which do not provide any explanations about the reasons behind their output. On the other hand, there are white-box ML models which are inherently interpretable while having lower accuracy compared to black-box models. To have a productive and practical algorithmic decision system, precise predictions may not be sufficient. The system may need to have transparency and be able to provide explanations, especially in applications with safety-critical contexts such as medicine, aerospace, robotics, and self-driving vehicles; or in socially-sensitive domains such as credit scoring and predictive policing. This is because having transparency can help explain why a certain decision was made and this, in turn, could be useful in discovering possible biases that lead to discrimination against any individual or group of people. Fairness and bias are other aspects that need to be considered in evaluating ML models. Therefore, depending on the application domain, accuracy, explainability, and fairness from bias may be necessary in building a practical and effective algorithmic decision system. However, in practice, it is challenging to have a model that optimizes all of these three aspects simultaneously. In this work, we study ML criteria that go beyond accuracy in two different problems: 1) in collaborative filtering recommendation, where we study explainability and bias in addition to accuracy; and 2) in robotic grasp failure prediction, where we study explainability in addition to prediction accuracy.

Recommender systems have become ubiquitous Artificial Intelligence (AI) tools that play an important role in filtering online information in our daily lives. Whether we are shopping, browsing movies, or listening to music online, AI recommender systems are working behind the scene to provide us with curated and personalized content, that has been predicted to be relevant to our interest. The increasing prevalence of recommender systems has challenged researchers to develop powerful algorithms that can deliver recommendations with increasing accuracy. In addition to the predictive accuracy of recommender systems, recent research has also started paying attention to their fairness, in particular with regard to the bias and transparency of their predictions. This dissertation contributes to advancing the state of the art in fairness in AI by proposing new Machine Learning models and algorithms that aim to improve the user's experience when receiving recommendations, with a focus that is positioned at the nexus of three objectives, namely accuracy, transparency, and unbiasedness of the predictions. In our research, we focus on state-of-the-art Collaborative Filtering (CF) recommendation approaches trained on implicit feedback data. More specifically, we address the limitations of two established deep learning approaches in two distinct recommendation settings, namely recommendation with user profiles and sequential recommendation. First, we focus on a state of the art pairwise ranking model, namely Bayesian Personalized Ranking (BPR), which has been found to outperform pointwise models in predictive accuracy in the recommendation with the user profiles setting. Specifically, we address two limitations of BPR: (1) BPR is a black box model that does not explain its outputs, thus limiting the user's trust in the recommendations, and the analyst's ability to scrutinize a model's outputs; and (2) BPR is vulnerable to exposure bias due to the data being Missing Not At Random (MNAR). This exposure bias usually translates into an unfairness against the least popular items because they risk being under-exposed by the recommender system. We propose a novel explainable loss function and a corresponding model called Explainable Bayesian Personalized Ranking (EBPR) that generates recommendations along with item-based explanations. Then, we theoretically quantify the additional exposure bias resulting from the explainability, and use it as a basis to propose an unbiased estimator for the ideal EBPR loss. This being done, we perform an empirical study on three real-world benchmarking datasets that demonstrate the advantages of our proposed models, compared to existing state of the art techniques. Next, we shift our attention to sequential recommendation systems and focus on modeling and mitigating exposure bias in BERT4Rec, which is a state-of-the-art recommendation approach based on bidirectional transformers. The bi-directional representation capacity in BERT4Rec is based on the Cloze task, a.k.a. Masked Language Model, which consists of predicting randomly masked items within the sequence, assuming that the true interacted item is the most relevant one. This results in an exposure bias, where non-interacted items with low exposure propensities are assumed to be irrelevant. Thus far, the most common approach to mitigating exposure bias in recommendation has been Inverse Propensity Scoring (IPS), which consists of down-weighting the interacted predictions in the loss function in proportion to their propensities of exposure, yielding a theoretically unbiased learning. We first argue and prove that IPS does not extend to sequential recommendation because it fails to account for the sequential nature of the problem. We then propose a novel propensity scoring mechanism, that we name Inverse Temporal Propensity Scoring (ITPS), which is used to theoretically debias the Cloze task in sequential recommendation. We also rely on the ITPS framework to propose a bidirectional transformer-based model called ITPS-BERT4Rec. Finally, we empirically demonstrate the debiasing capabilities of our proposed approach and its robustness to the severity of exposure bias. Our proposed explainable approach in recommendation with user profiles, EBPR, showed an increase in ranking accuracy of about 4% and an increase in explainability of about 7% over the baseline BPR model when performing experiments on real-world recommendation datasets. Moreover, experiments on a real-world unbiased dataset demonstrated the importance of coupling explainability and exposure debiasing in capturing the true preferences of the user with a significant improvement of 1% over the baseline unbiased model UBPR. Furthermore, coupling explainability with exposure debiasing was also empirically proven to mitigate popularity bias with an improvement in popularity debiasing metrics of over 10% on three real-world recommendation tasks over the unbiased UBPR model. These results demonstrate the viability of our proposed approaches in recommendation with user profiles and their capacity to improve the user's experience in recommendation by better capturing and modeling their true preferences, improving the explainability of the recommendations, and presenting them with more diverse recommendations that span a larger portion of the item catalog. On the other hand, our proposed approach in sequential recommendation ITPS-BERT4Rec has demonstrated a significant increase of 1% in terms of modeling the true preferences of the user in a semi-synthetic setting over the state-of-the-art sequential recommendation model BERT4Rec while also being unbiased in terms of exposure. Similarly, ITPS-BERT4Rec showed an average increase of 8.7% over BERT4Rec in three real-world recommendation settings. Moreover, empirical experiments demonstrated the robustness of our proposed ITPS-BERT4Rec model to increasing levels of exposure bias and its stability in terms of variance. Furthermore, experiments on popularity debiasing showed a significant advantage of our proposed ITPS-BERT4Rec model for both the short and long term sequences. Finally, ITPS-BERT4Rec showed respective improvements of around 60%, 470%, and 150% over vanilla BERT4Rec in capturing the temporal dependencies between the items within the sequences of interactions for three different evaluation metrics. These results demonstrate the potential of our proposed unbiased estimator to improve the user experience in the context of sequential recommendation by presenting them with more accurate and diverse recommendations that better match their true preferences and the sequential dependencies between the recommended items.

A number of competing concerns slow adoption of deep learning for computer vision on“edge” devices. Edge devices provide only limited resources for on-device algorithms to employ, constraining power, memory, and storage usage. Examples include mobile phones, autonomous vehicles, and virtual reality headsets, which demand both high accuracy and low latency, two objectives competing for resources.To tackle this sisyphean task, modern methods expend gargantuan amounts of computationto design solutions, exceeding thousands of GPU hours or years of GPU compute to design a single neural network. Not to mention, these works maximize just one performance metric – accuracy – under a single set of resource constraints. What if the set of resource constraints changes? If additional performance metrics rise to the forefront, such as explainability or generalization? Modern methods for designing efficient neural networks are handicapped by excessive computation requirements for goals too singularly and narrowly sighted.This thesis tackles the bottlenecks of modern methods directly, achieving state-of-the-artperformance by efficiently designing efficient deep neural networks. These improvements don’t only reduce computation or only improve accuracy; instead, our methods improve performance and reduce computational requirements, despite increasing search space size by orders of magnitude. We also demonstrate missed opportunities with performance metrics beyond accuracy, redesigning the task so that accuracy, explainability, and generalization improve jointly, an impossibility by conventional wisdom, which suggests explainability and accuracy participate in a zero-sum game.This thesis culminates in a set of models that set new flexibility and performance standards forproduction-ready models: those that are state-of-the-art accurate, explainable, generalizable, and configurable for any set of resource constraints in just CPU minutes.

The recent surge in highly successful, but opaque, machine-learning models has given rise to a dire need for interpretability. This work addresses the problem of interpretability with novel definitions, methodology, and scientific investigations, ensuring that interpretations are useful by grounding them in the context of real-world problems and audiences. We begin by defining what we mean by interpretability and some desiderata surrounding it, emphasizing the underappreciated role of context. We then dive into novel methods for interpreting/improving neural network models, focusing on how to best score, use, and distill interactions. Next, we turn from neural networks to relatively simple rule-based models, where we investigate how to improve predictive performance while maintaining an extremely concise model. Finally, we conclude with work on open-source software and data for facilitating interpretable data science. In each case, we dive into a specific context which motivates the proposed methodology, ranging from cosmology to cell biology to medicine. Code for everything is available at github.com/csinva.

In recent years, artificial intelligence has achieved remarkable progress by showing its applicable values in various industries. Convolutional neural networks (CNN) and their derivative approaches are well known for their robustness and accuracy in handling visual data. However, as a data-driven approach, CNN also has limitations. In exchange for good performance, CNN requires a large amount of training data and a heavy training process. The intricate neural network layer design also needs to be reconstructed and tuned by experienced researchers for different problems. Finally, the “curse of Blackbox” is a well-known disadvantage of the neural network, preventing CNN from providing a reasonable explanation for the prediction results. All the above limitations remind us that the most cutting-edge approach is still in the state of weak AI. This thesis proposes an approach called Abstractive Representation Model (ARM), which is different from the traditional data-driven approaches. This goal of experimenting with such a model is to address CNN’s weaknesses and possibly develop a new way of handling image data.

Artificial intelligence and machine learning (ML) solutions are being integrated into more software solutions than ever before. The inner working of ML models is difficult to understand making them black boxes. Explainable artificial intelligence (XAI) tools have been developed to make these ML models more understandable. In this work, we reviewed existing XAI solutions and their implications. We have grouped these tools based on how they operate and what kinds of information they can provide to the stakeholders. To illustrate this, we conducted a case study with the Metropolitan Sewer District of Greater Cincinnati to evaluate the use of XAI to make ML solutions more accountable and understandable. As part of our case study methodology, we investigated how these XAI tools can be integrated into the ML development pipeline. There is a clear need for explainability in ML based software and our case study and research investigated how these XAI tools can enhance ML testing and validation.

Recent years saw an explosive growth in the amount of digital information and the number of users who interact with this information through various platforms, ranging from web services to mobile applications and smart devices. This increase in information and users has naturally led to information overload which inherently limits the capacity of users to discover and find their needs among the staggering array of options available at any given time, the majority of which they may never become aware of. Online services have handled this information overload by using algorithmic filtering tools that can suggest relevant and personalized information to users. These filtering methods, known as Recommender Systems (RS), have become essential to recommend a range of relevant options in diverse domains ranging from friends, courses, music, and restaurants, to movies, books, and travel recommendations. Most research on recommender systems has focused on developing and evaluating models that can make predictions efficiently and accurately, without taking into account other desiderata such as fairness and transparency which are becoming increasingly important to establish trust with human users. For this reason, researchers have been recently pressed to develop recommendation systems that are endowed with the increased ability to explain why a recommendation is given, and hence help users make more informed decisions. Nowadays, state of the art Machine Learning (ML) techniques are being used to achieve unprecedented levels of accuracy in recommender systems. Unfortunately, most models are notorious for being black box models that cannot explain their output predictions. One such example is Matrix Factorization, a technique that is widely used in Collaborative Filtering algorithms. Unfortunately, like all black box machine learning models, MF is unable to explain its outputs. This dissertation proposes a new Cosine-based explainable Matrix Factorization model (CEMF) that incorporates a user-neighborhood explanation matrix (NSE) and incorporates a cosine based penalty in the objective function to encourage predictions that are explainable. Our evaluation experiments demonstrate that CEMF can recommend items that are more explainable and diverse compared to its competitive baselines, and that it further achieves this superior performance without sacrificing the accuracy of its predictions.

Embedding knowledge graphs is a common method used to encode information from the graph at hand projected in a low dimensional space. There are two shortcomings in the field of knowledge graph embeddings for link prediction. The first shortcoming is that, as far as we know, current software libraries to compute knowledge graph embeddings differ from the original papers proposing these embeddings. Certain implementations are faithful to the original papers, while others range from minute differences to significant variations. Due to these implementation variations, it is difficult to compare the same algorithm from multiple libraries and also affects our ability to reproduce results. In this report, we describe a new framework, AugmentedKGE (aKGE), to embed knowledge graphs. The library features multiple knowledge graph embedding algorithms, a rank-based evaluator, and is developed completely using Python and PyTorch. The second shortcoming is that, during the evaluation process of link prediction, the goal is to rank based on scores a positive triple over a (typically large) number of negative triples. Accuracy metrics used in the evaluation of link prediction are aggregations of the ranks of the positive triples under evaluation and do not typically provide enough details as to why a number of negative triples are ranked higher than their positive counterparts. Providing explanations to these triples aids in understanding the results of the link predictions based on knowledge graph embeddings. Current approaches mainly focus on explaining embeddings rather than predictions and single predictions rather than all the link predictions made by the embeddings of a certain knowledge graph. In this report, we present an approach to explain all these predictions by providing two metrics that serve to quantify and compare the explainability of different embeddings. From the results of evaluating aKGE, we observe that the accuracy metrics are better than the accuracy metrics obtained from the standard implementation of OpenKE. From the results of explainability, we observe that the horn rules obtained explain more than 50% of all the negative triples generated.

There have been many recent advancements in the field of reinforcement learning, starting from the Deep Q Network playing various Atari 2600 games all the way to Google Deempind's Alphastar playing competitively in the game StarCraft. However, as the field challenges more complex environments, the current methods of training models and understanding their decision making become less effective. Currently, the problem is partially dealt with by simply adding more resources, but the need for a better solution remains. This thesis proposes a reinforcement learning framework where a teacher or entity with domain knowledge of the task to complete can assist the agent in finding a policy, while also providing human observers a more understandable format for how the agent formulates its overall policy to complete a given task. This framework is evaluated on the Seaquest environment within the OpenAI Gym framework. Adviser: Stephen Scott

Black-box recommender system models are machine learning models that generate personalized recommendations without explaining how the recommendations were generated to the user or giving them a way to correct wrong assumptions made about them by the model. However, compared to white-box models, which are transparent and scrutable, black-box models are generally more accurate. Recent research has shown that accuracy alone is not sufficient for user satisfaction. One such black-box model is Matrix Factorization, a State of the Art recommendation technique that is widely used due to its ability to deal with sparse data sets and to produce accurate recommendations. Recent work has proposed new Matrix Factorization models that are explainable by incorporating explanations derived from semantic knowledge graphs, user neighborhood, or item neighborhood graphs into the model learning process. These Explainable Matrix Factorization (EMF) methods have the benefit of providing explanations without sacrificing accuracy. However, their explanations tend to be limited to only one explanation style. In this dissertation, we propose a framework comprising new machine learning methods to build explainable models that can make recommendations with multiple explanation-styles, by hybridizing multiple EMF models and by proposing new EMF models that explain recommendations using tags. The various pre-calculated explainability scores, leveraged in our proposed methods, have all been validated in prior work that conducted user studies to evaluate users’ satisfaction with each style individually. Unlike most existing work that generates explanations post-hoc, i.e., after the predictions have already been made, our framework is based on calculating explainability scores directly from available data, before the model is learned, and then using them as part of a regularization mechanism, to guide the model learning. Unlike post-hoc methods, our framework makes it possible to learn machine learning models that take into account the explanation scores, therefore ensuring higher transparency. Our evaluation experiments show that our proposed methods provide accurate recommendations while also providing users with multiple styles of explanations about how data was used to generate each recommendation. Each explanation style also provides additional decision-making information that empowers the user to either trust or scrutinize the recommendations. Although, rooted in the hybrid recommendation framework, our proposed methods make a significant step forward in explainable AI and beyond existing hybrid frameworks, because the proposed hybridization mechanisms make an intentional effort to take into account the individual models’ explanations and not only their output predicted ratings.

Large-scale distributed systems---such as supercomputers, cloud computing platforms, and distributed applications---routinely suffer from slowdowns and crashes due to software and hardware problems, resulting in reduced efficiency and wasted resources. These large-scale systems typically deploy monitoring or tracing systems that gather a variety of statistics about the state of the hardware and the software. State-of-the-art methods either analyze this data manually, or design unique automated methods for each specific problem. This thesis builds on the vision that generalized automated analytics methods on the data sets collected from these complex computing systems provide critical information about the causes of the problems, and this analysis can then enable proactive management to improve performance, resilience, efficiency, or security significantly beyond current limits. This thesis seeks to design scalable, automated analytics methods and frameworks for large-scale distributed systems that minimize dependency on expert knowledge, automate parts of the solution process, and help make systems more resilient. In addition to analyzing data that is already collected from systems, our frameworks also identify what to collect from where in the system, such that the collected data would be concise and useful for manual analytics. We focus on two data sources for conducting analytics: numeric telemetry data, which is typically collected from operating system or hardware counters, and end-to-end traces collected from distributed applications. This thesis makes the following contributions in large-scale distributed systems: (1) Designing a framework for accurately diagnosing previously encountered performance variations, (2) designing a technique for detecting (unwanted) applications running on the systems, (3) developing a suite for reproducing performance variations that can be used to systematically develop analytics methods, (4) designing a method to explain predictions of black-box machine learning frameworks, and (5) constructing an end-to-end tracing framework that can dynamically adjust instrumentation for effective diagnosis of performance problems.

Streaming applications are now the predominant tools for listening to music. What makes the success of such software is the availability of songs and especially their ability to provide users with relevant personalized recommendations. State of the art music recommender systems mainly rely on either Matrix factorization-based collaborative filtering approaches or deep learning architectures. Deep learning models usually use metadata for content-based filtering or predict the next user interaction (listening to a song) using a memory-based deep learning structure that learns from temporal sequences of user actions. Despite advances in deep learning models for song recommendation systems, none has taken advantage of the sequential nature of songs by learning sequence models that are based on content. Aside from the importance of prediction accuracy in recommendation systems, recent research has unveiled the importance of other significant aspects such as explainability and solving the cold start problem where a new user or item with no prior history of interactions joins an online platform. In this work, we propose a hybrid deep learning structure, called “SeER”, that uses collaborative filtering and deep sequence models on the MIDI content of songs for recommendation. Our approach aims to take advantage of the superior capabilities of re-current neural networks, the multidimensional time series aspect of songs, and the power of matrix factorization to: •provide more accurate personalized recommendations, •solve the item cold start problem which is in the case of where a new unrated song is added to the set of choices to recommend; and •generate a relevant explanation for a song recommendation using a novel explainability process we named “Segment Forward Propagation Explainability”. Our evaluation experiments show promising results compared to state of the art baseline and hybrid song recommender systems in terms of ranking evaluation. In addition, we demonstrate how our explanation mechanism can be used with generic sequential data beyond music, namely unstructured free text in two application domains: sentiment classification of online user reviews and delineating potential child abuse instances from medical examination reports.