Search

In the last decade, there has been a growing interest in applying AI technologies to implement complex data analytics over data streams. To this end, researchers in various fields have been organising a yearly event called the "Stream Reasoning Workshop" to share perspectives, challenges, and experiences around this topic. In this paper, the previous organisers of the workshops and other community members provide a summary of the main research results that have been discussed during the first six editions of the event. These results can be categorised into four main research areas: The first is concerned with the technological challenges related to handling large data streams. The second area aims at adapting and extending existing semantic technologies to data streams. The third and fourth areas focus on how to implement reasoning techniques, either considering deductive or inductive techniques, to extract new and valuable knowledge from the data in the stream. This summary is written not only to provide a crystallisation of the field, but also to point out distinctive traits of the stream reasoning community. Moreover, it also provides a foundation for future research by enumerating a list of use cases and open challenges, to stimulate others to join this exciting research area.

In the last decade, there has been a growing interest in applying AI technologies to implement complex data analytics over data streams. To this end, researchers in various fields have been organising a yearly event called the "Stream Reasoning Workshop" to share perspectives, challenges, and experiences around this topic. In this paper, the previous organisers of the workshops and other community members provide a summary of the main research results that have been discussed during the first six editions of the event. These results can be categorised into four main research areas: The first is concerned with the technological challenges related to handling large data streams. The second area aims at adapting and extending existing semantic technologies to data streams. The third and fourth areas focus on how to implement reasoning techniques, either considering deductive or inductive techniques, to extract new and valuable knowledge from the data in the stream. This summary is written not only to provide a crystallisation of the field, but also to point out distinctive traits of the stream reasoning community. Moreover, it also provides a foundation for future research by enumerating a list of use cases and open challenges, to stimulate others to join this exciting research area.

In-situ monitoring incorporating data from visual and other sensor technologies, allows the collection of extensive datasets during the Additive Manufacturing (AM) process. These datasets have potential for determining the quality of the manufactured output and the detection of defects through the use of Machine Learning during the manufacturing process. Open and annotated datasets derived from AM processes are necessary for the machine learning community to address this opportunity, which creates difficulties in the application of computer vision-related machine learning in AM. This systematic review investigates the availability of open image-based datasets originating from AM processes that align with a number of pre-defined selection criteria. The review identifies existing gaps among the current image-based datasets in the domain of AM, and points to the need for greater availability of open datasets in order to allow quality assessment and defect detection during additive manufacturing, to develop., Comment: 24 pages

Advanced image data augmentation techniques play a pivotal role in enhancing the training of models for diverse computer vision tasks. Notably, SalfMix and KeepAugment have emerged as popular strategies, showcasing their efficacy in boosting model performance. However, SalfMix reliance on duplicating salient features poses a risk of overfitting, potentially compromising the model's generalization capabilities. Conversely, KeepAugment, which selectively preserves salient regions and augments non-salient ones, introduces a domain shift that hinders the exchange of crucial contextual information, impeding overall model understanding. In response to these challenges, we introduce KeepOriginalAugment, a novel data augmentation approach. This method intelligently incorporates the most salient region within the non-salient area, allowing augmentation to be applied to either region. Striking a balance between data diversity and information preservation, KeepOriginalAugment enables models to leverage both diverse salient and non-salient regions, leading to enhanced performance. We explore three strategies for determining the placement of the salient region minimum, maximum, or random and investigate swapping perspective strategies to decide which part (salient or non-salient) undergoes augmentation. Our experimental evaluations, conducted on classification datasets such as CIFAR-10, CIFAR-100, and TinyImageNet, demonstrate the superior performance of KeepOriginalAugment compared to existing state-of-the-art techniques., Comment: This paper has been accepted at 20th International Conference on Artificial Intelligence Applications and Innovations 2024

Neurosymbolic background knowledge and the expressivity required of its logic can break Machine Learning assumptions about data Independence and Identical Distribution. In this position paper we propose to analyze IID relaxation in a hierarchy of logics that fit different use case requirements. We discuss the benefits of exploiting known data dependencies and distribution constraints for Neurosymbolic use cases and argue that the expressivity required for this knowledge has implications for the design of underlying ML routines. This opens a new research agenda with general questions about Neurosymbolic background knowledge and the expressivity required of its logic., Comment: 12 pages, 2 figures, submitted to NeSy 2024

The development of computer vision and in-situ monitoring using visual sensors allows the collection of large datasets from the additive manufacturing (AM) process. Such datasets could be used with machine learning techniques to improve the quality of AM. This paper examines two scenarios: first, using convolutional neural networks (CNNs) to accurately classify defects in an image dataset from AM and second, applying active learning techniques to the developed classification model. This allows the construction of a human-in-the-loop mechanism to reduce the size of the data required to train and generate training data.

The development of computer vision and in-situ monitoring using visual sensors allows the collection of large datasets from the additive manufacturing (AM) process. Such datasets could be used with machine learning techniques to improve the quality of AM. This paper examines two scenarios: first, using convolutional neural networks (CNNs) to accurately classify defects in an image dataset from AM and second, applying active learning techniques to the developed classification model. This allows the construction of a human-in-the-loop mechanism to reduce the size of the data required to train and generate training data.

The development of computer vision and in-situ monitoring using visual sensors allows the collection of large datasets from the additive manufacturing (AM) process. Such datasets could be used with machine learning techniques to improve the quality of AM. This paper examines two scenarios: first, using convolutional neural networks (CNNs) to accurately classify defects in an image dataset from AM and second, applying active learning techniques to the developed classification model. This allows the construction of a human-in-the-loop mechanism to reduce the size of the data required to train and generate training data.

The development of computer vision and in-situ monitoring using visual sensors allows the collection of large datasets from the additive manufacturing (AM) process. Such datasets could be used with machine learning techniques to improve the quality of AM. This paper examines two scenarios: first, using convolutional neural networks (CNNs) to accurately classify defects in an image dataset from AM and second, applying active learning techniques to the developed classification model. This allows the construction of a human-in-the-loop mechanism to reduce the size of the data required to train and generate training data.

Deep learning (DL) algorithms have shown significant performance in various computer vision tasks. However, having limited labelled data lead to a network overfitting problem, where network performance is bad on unseen data as compared to training data. Consequently, it limits performance improvement. To cope with this problem, various techniques have been proposed such as dropout, normalization and advanced data augmentation. Among these, data augmentation, which aims to enlarge the dataset size by including sample diversity, has been a hot topic in recent times. In this article, we focus on advanced data augmentation techniques. we provide a background of data augmentation, a novel and comprehensive taxonomy of reviewed data augmentation techniques, and the strengths and weaknesses (wherever possible) of each technique. We also provide comprehensive results of the data augmentation effect on three popular computer vision tasks, such as image classification, object detection and semantic segmentation. For results reproducibility, we compiled available codes of all data augmentation techniques. Finally, we discuss the challenges and difficulties, and possible future direction for the research community. We believe, this survey provides several benefits i) readers will understand the data augmentation working mechanism to fix overfitting problems ii) results will save the searching time of the researcher for comparison purposes. iii) Codes of the mentioned data augmentation techniques are available at https://github.com/kmr2017/Advanced-Data-augmentation-codes iv) Future work will spark interest in research community., Comment: We need to make a lot changes to make its quality better

Data augmentation has proven to be effective in training neural networks. Recently, a method called RandAug was proposed, randomly selecting data augmentation techniques from a predefined search space. RandAug has demonstrated significant performance improvements for image-related tasks while imposing minimal computational overhead. However, no prior research has explored the application of RandAug specifically for audio data augmentation, which converts audio into an image-like pattern. To address this gap, we introduce AudRandAug, an adaptation of RandAug for audio data. AudRandAug selects data augmentation policies from a dedicated audio search space. To evaluate the effectiveness of AudRandAug, we conducted experiments using various models and datasets. Our findings indicate that AudRandAug outperforms other existing data augmentation methods regarding accuracy performance., Comment: Paper has accepted at 25th Irish Machine Vision and Image Processing Conference

This volume contains the Technical Communications presented at the 39th International Conference on Logic Programming (ICLP 2023), held at Imperial College London, UK from July 9 to July 15, 2023. Technical Communications included here concern the Main Track, the Doctoral Consortium, the Application and Systems/Demo track, the Recently Published Research Track, the Birds-of-a-Feather track, the Thematic Tracks on Logic Programming and Machine Learning, and Logic Programming and Explainability, Ethics, and Trustworthiness.

The development of computer vision and in-situ monitoring using visual sensors allows the collection of large datasets from the additive manufacturing (AM) process. Such datasets could be used with machine learning techniques to improve the quality of AM. This paper examines two scenarios: first, using convolutional neural networks (CNNs) to accurately classify defects in an image dataset from AM and second, applying active learning techniques to the developed classification model. This allows the construction of a human-in-the-loop mechanism to reduce the size of the data required to train and generate training data., Comment: 4 pages, accepted at the Irish Machine Vision and Image Processing Conference (IMVIP), Galway, August 2023

Recent developments in in-situ monitoring and process control in Additive Manufacturing (AM), also known as 3D-printing, allows the collection of large amounts of emission data during the build process of the parts being manufactured. This data can be used as input into 3D and 2D representations of the 3D-printed parts. However the analysis and use, as well as the characterization of this data still remains a manual process. The aim of this paper is to propose an adaptive human-in-the-loop approach using Machine Learning techniques that automatically inspect and annotate the emissions data generated during the AM process. More specifically, this paper will look at two scenarios: firstly, using convolutional neural networks (CNNs) to automatically inspect and classify emission data collected by in-situ monitoring and secondly, applying Active Learning techniques to the developed classification model to construct a human-in-the-loop mechanism in order to accelerate the labeling process of the emission data. The CNN-based approach relies on transfer learning and fine-tuning, which makes the approach applicable to other industrial image patterns. The adaptive nature of the approach is enabled by uncertainty sampling strategy to automatic selection of samples to be presented to human experts for annotation.

Recent developments in in-situ monitoring and process control in Additive Manufacturing (AM), also known as 3D-printing, allows the collection of large amounts of emission data during the build process of the parts being manufactured. This data can be used as input into 3D and 2D representations of the 3D-printed parts. However the analysis and use, as well as the characterization of this data still remains a manual process. The aim of this paper is to propose an adaptive human-in-the-loop approach using Machine Learning techniques that automatically inspect and annotate the emissions data generated during the AM process. More specifically, this paper will look at two scenarios: firstly, using convolutional neural networks (CNNs) to automatically inspect and classify emission data collected by in-situ monitoring and secondly, applying Active Learning techniques to the developed classification model to construct a human-in-the-loop mechanism in order to accelerate the labeling process of the emission data. The CNN-based approach relies on transfer learning and fine-tuning, which makes the approach applicable to other industrial image patterns. The adaptive nature of the approach is enabled by uncertainty sampling strategy to automatic selection of samples to be presented to human experts for annotation.

Recent developments in in-situ monitoring and process control in Additive Manufacturing (AM), also known as 3D-printing, allows the collection of large amounts of emission data during the build process of the parts being manufactured. This data can be used as input into 3D and 2D representations of the 3D-printed parts. However the analysis and use, as well as the characterization of this data still remains a manual process. The aim of this paper is to propose an adaptive human-in-the-loop approach using Machine Learning techniques that automatically inspect and annotate the emissions data generated during the AM process. More specifically, this paper will look at two scenarios: firstly, using convolutional neural networks (CNNs) to automatically inspect and classify emission data collected by in-situ monitoring and secondly, applying Active Learning techniques to the developed classification model to construct a human-in-the-loop mechanism in order to accelerate the labeling process of the emission data. The CNN-based approach relies on transfer learning and fine-tuning, which makes the approach applicable to other industrial image patterns. The adaptive nature of the approach is enabled by uncertainty sampling strategy to automatic selection of samples to be presented to human experts for annotation.

Recent developments in in-situ monitoring and process control in Additive Manufacturing (AM), also known as 3D-printing, allows the collection of large amounts of emission data during the build process of the parts being manufactured. This data can be used as input into 3D and 2D representations of the 3D-printed parts. However the analysis and use, as well as the characterization of this data still remains a manual process. The aim of this paper is to propose an adaptive human-in-the-loop approach using Machine Learning techniques that automatically inspect and annotate the emissions data generated during the AM process. More specifically, this paper will look at two scenarios: firstly, using convolutional neural networks (CNNs) to automatically inspect and classify emission data collected by in-situ monitoring and secondly, applying Active Learning techniques to the developed classification model to construct a human-in-the-loop mechanism in order to accelerate the labeling process of the emission data. The CNN-based approach relies on transfer learning and fine-tuning, which makes the approach applicable to other industrial image patterns. The adaptive nature of the approach is enabled by uncertainty sampling strategy to automatic selection of samples to be presented to human experts for annotation., Comment: 7 pages, 9 figures, 1 table. Presented at The 6th IEEE Workshop on Human-in-the-Loop Methods and Future of Work in BigData (IEEE HMData 2022) December 2022

The popularity of deep learning has increased tremendously in recent years due to its ability to efficiently solve complex tasks in challenging areas such as computer vision and language processing. Despite this success, low-level neural activity reproduced by Deep Neural Networks (DNNs) generates extremely rich representations of the data. These representations are difficult to characterise and cannot be directly used to understand the decision process. In this paper we build upon our exploratory work where we introduced the concept of a co-activation graph and investigated the potential of graph analysis for explaining deep representations. The co-activation graph encodes statistical correlations between neurons’ activation values and therefore helps to characterise the relationship between pairs of neurons in the hidden layers and output classes. To confirm the validity of our findings, our experimental evaluation is extended to consider datasets and models with different levels of complexity. For each of the considered datasets we explore the co-activation graph and use graph analysis to detect similar classes, find central nodes and use graph visualisation to better interpret the outcomes of the analysis. Our results show that graph analysis can reveal important insights into how DNNs work and enable partial explainability of deep learning models.

Neural knowledge transfer methods aim to constrain the hidden representation of one neural network to be similar, or have similar properties, to another by applying specially designed loss functions between the two networks hidden layers. In this way the intangible knowledge encoded by the network's weights is transferred without having to replicate exact weight structures or alter the knowledge representation from its natural highly distributed form. Motivated by the need to enable greater transparency in evaluating such methods by bridging the gap between different experimental setups in the existing literature, the need to cast a wider net in comparing each method to a greater number of its peers and a desire to explore novel combinations of existing methods we conduct an experimental comparison of eight contemporary neural knowledge transfer algorithms and further explore the performance of some combinations. We conduct our experiments on an image classification task and measure relative performance gains over non-knowledge enhanced baseline neural networks in terms of classification accuracy. We observed (i) some interesting contradictions between our results and those reported in original papers, (ii) a general lack of correlation between any given methods standalone performance vs performance when used in combination with knowledge distillation, (iii) a general trend of older simpler methods outperforming newer ones and (iv) Contrastive Representation Distillation (CRD) achieving best performance.

Neural knowledge transfer methods aim to constrain the hidden representation of one neural network to be similar, or have similar properties, to another by applying specially designed loss functions between the two networks hidden layers. In this way the intangible knowledge encoded by the network's weights is transferred without having to replicate exact weight structures or alter the knowledge representation from its natural highly distributed form. Motivated by the need to enable greater transparency in evaluating such methods by bridging the gap between different experimental setups in the existing literature, the need to cast a wider net in comparing each method to a greater number of its peers and a desire to explore novel combinations of existing methods we conduct an experimental comparison of eight contemporary neural knowledge transfer algorithms and further explore the performance of some combinations. We conduct our experiments on an image classification task and measure relative performance gains over non-knowledge enhanced baseline neural networks in terms of classification accuracy. We observed (i) some interesting contradictions between our results and those reported in original papers, (ii) a general lack of correlation between any given methods standalone performance vs performance when used in combination with knowledge distillation, (iii) a general trend of older simpler methods outperforming newer ones and (iv) Contrastive Representation Distillation (CRD) achieving best performance.

This article explores the rapidly advancing innovation to endow robots with social intelligence capabilities in the form of multilingual and multimodal emotion recognition, and emotion-aware decision-making capabilities, for contextually appropriate robot behaviours and cooperative social human–robot interaction for the healthcare domain. The objective is to enable robots to become trustworthy and versatile social robots capable of having human-friendly and human assistive interactions, utilised to better assist human users’ needs by enabling the robot to sense, adapt, and respond appropriately to their requirements while taking into consideration their wider affective, motivational states, and behaviour. We propose an innovative approach to the difficult research challenge of endowing robots with social intelligence capabilities for human assistive interactions, going beyond the conventional robotic sense-think-act loop. We propose an architecture that addresses a wide range of social cooperation skills and features required for real human–robot social interaction, which includes language and vision analysis, dynamic emotional analysis (long-term affect and mood), semantic mapping to improve the robot’s knowledge of the local context, situational knowledge representation, and emotion-aware decision-making. Fundamental to this architecture is a normative ethical and social framework adapted to the specific challenges of robots engaging with caregivers and care-receivers.

This article explores the rapidly advancing innovation to endow robots with social intelligence capabilities in the form of multilingual and multimodal emotion recognition, and emotion-aware decision-making capabilities, for contextually appropriate robot behaviours and cooperative social human–robot interaction for the healthcare domain. The objective is to enable robots to become trustworthy and versatile social robots capable of having human-friendly and human assistive interactions, utilised to better assist human users’ needs by enabling the robot to sense, adapt, and respond appropriately to their requirements while taking into consideration their wider affective, motivational states, and behaviour. We propose an innovative approach to the difficult research challenge of endowing robots with social intelligence capabilities for human assistive interactions, going beyond the conventional robotic sense-think-act loop. We propose an architecture that addresses a wide range of social cooperation skills and features required for real human–robot social interaction, which includes language and vision analysis, dynamic emotional analysis (long-term affect and mood), semantic mapping to improve the robot’s knowledge of the local context, situational knowledge representation, and emotion-aware decision-making. Fundamental to this architecture is a normative ethical and social framework adapted to the specific challenges of robots engaging with caregivers and care-receivers.

Neural knowledge transfer methods aim to constrain the hidden representation of one neural network to be similar, or have similar properties, to another by applying specially designed loss functions between the two networks hidden layers. In this way the intangible knowledge encoded by the network's weights is transferred without having to replicate exact weight structures or alter the knowledge representation from its natural highly distributed form. Motivated by the need to enable greater transparency in evaluating such methods by bridging the gap between different experimental setups in the existing literature, the need to cast a wider net in comparing each method to a greater number of its peers and a desire to explore novel combinations of existing methods we conduct an experimental comparison of eight contemporary neural knowledge transfer algorithms and further explore the performance of some combinations. We conduct our experiments on an image classification task and measure relative performance gains over non-knowledge enhanced baseline neural networks in terms of classification accuracy. We observed (i) some interesting contradictions between our results and those reported in original papers, (ii) a general lack of correlation between any given methods standalone performance vs performance when used in combination with knowledge distillation, (iii) a general trend of older simpler methods outperforming newer ones and (iv) Contrastive Representation Distillation (CRD) achieving best performance.

Neural knowledge transfer methods aim to constrain the hidden representation of one neural network to be similar, or have similar properties, to another by applying specially designed loss functions between the two networks hidden layers. In this way the intangible knowledge encoded by the network's weights is transferred without having to replicate exact weight structures or alter the knowledge representation from its natural highly distributed form. Motivated by the need to enable greater transparency in evaluating such methods by bridging the gap between different experimental setups in the existing literature, the need to cast a wider net in comparing each method to a greater number of its peers and a desire to explore novel combinations of existing methods we conduct an experimental comparison of eight contemporary neural knowledge transfer algorithms and further explore the performance of some combinations. We conduct our experiments on an image classification task and measure relative performance gains over non-knowledge enhanced baseline neural networks in terms of classification accuracy. We observed (i) some interesting contradictions between our results and those reported in original papers, (ii) a general lack of correlation between any given methods standalone performance vs performance when used in combination with knowledge distillation, (iii) a general trend of older simpler methods outperforming newer ones and (iv) Contrastive Representation Distillation (CRD) achieving best performance.

In recent years we have seen significant advances in the technology used to both publish and consume Linked Data. However, in order to support the next generation of ebusiness applications on top of interlinked machine readable data suitable forms of access control need to be put in place. Although a number of access control models and frameworks have been put forward, very little research has been conducted into the security implications associated with granting access to partial data or the correctness of the proposed access control mechanisms. Therefore the contributions of this paper are two fold: we propose a query rewriting algorithm which can be used to partially restrict access to SPARQL 1.1 queries and updates; and we demonstrate how a set of criteria, which was originally used to verify that an access control policy holds over different database states, can be adapted to verify the correctness of access control via query rewriting.

Since the first conference held in Marseille in 1982, ICLP has been the premier international event for presenting research in logic programming. Contributions are solicited in all areas of logic programming and related areas, including but not restricted to: - Foundations: Semantics, Formalisms, Answer-Set Programming, Non-monotonic Reasoning, Knowledge Representation. - Declarative Programming: Inference engines, Analysis, Type and mode inference, Partial evaluation, Abstract interpretation, Transformation, Validation, Verification, Debugging, Profiling, Testing, Logic-based domain-specific languages, constraint handling rules. - Related Paradigms and Synergies: Inductive and Co-inductive Logic Programming, Constraint Logic Programming, Interaction with SAT, SMT and CSP solvers, Logic programming techniques for type inference and theorem proving, Argumentation, Probabilistic Logic Programming, Relations to object-oriented and Functional programming, Description logics, Neural-Symbolic Machine Learning, Hybrid Deep Learning and Symbolic Reasoning. - Implementation: Concurrency and distribution, Objects, Coordination, Mobility, Virtual machines, Compilation, Higher Order, Type systems, Modules, Constraint handling rules, Meta-programming, Foreign interfaces, User interfaces. - Applications: Databases, Big Data, Data Integration and Federation, Software Engineering, Natural Language Processing, Web and Semantic Web, Agents, Artificial Intelligence, Bioinformatics, Education, Computational life sciences, Education, Cybersecurity, and Robotics.

Due to its potential to solve complex tasks, deep learning is being used across many different areas. The complexity of neural networks however makes it difficult to explain the whole decision process used by the model, which makes understanding deep learning models an active research topic. In this work we address this issue by extracting the knowledge acquired by trained Deep Neural Networks (DNNs) and representing this knowledge in a graph. The proposed graph encodes statistical correlations between neurons’ activation values in order to expose the relationship between neurons in the hidden layers with both the input layer and output classes. Two initial experiments in image classification were conducted to evaluate whether the proposed graph can help understanding and explaining DNNs. We first show how it is possible to explore the proposed graph to find what neurons are the most important for predicting each class. Then, we use graph analysis to detect groups of classes that are more similar to each other and how these similarities affect the DNN. Finally, we use heatmaps to visualize what parts of the input layer are responsible for activating each neuron in hidden layers. The results show that by building and analysing the proposed graph it is possible to gain relevant insights of the DNN’s inner workings.

The ability to enhance deep representations with prior knowledge is receiving a lot of attention from the AI community as a key enabler to improve the way modern Artificial Neural Networks (ANN) learn. In this paper we introduce our approach to this task, which relies on a knowledge extraction algorithm, a knowledge injection algorithm and a common intermediate knowledge representation as an alternative to traditional neural transfer. As a result of this research, we envisage a knowledge-enhanced ANN, which will be able to learn, characterise and reuse knowledge extracted from the learning process, thus enabling more robust architecture-agnostic neural transfer, greater explainability and further integration of neural and symbolic approaches to learning.

Due to its potential to solve complex tasks, deep learning is being used across many different areas. The complexity of neural networks however makes it difficult to explain the whole decision process used by the model, which makes understanding deep learning models an active research topic. In this work we address this issue by extracting the knowledge acquired by trained Deep Neural Networks (DNNs) and representing this knowledge in a graph. The proposed graph encodes statistical correlations between neurons’ activation values in order to expose the relationship between neurons in the hidden layers with both the input layer and output classes. Two initial experiments in image classification were conducted to evaluate whether the proposed graph can help understanding and explaining DNNs. We first show how it is possible to explore the proposed graph to find what neurons are the most important for predicting each class. Then, we use graph analysis to detect groups of classes that are more similar to each other and how these similarities affect the DNN. Finally, we use heatmaps to visualize what parts of the input layer are responsible for activating each neuron in hidden layers. The results show that by building and analysing the proposed graph it is possible to gain relevant insights of the DNN’s inner workings.

The ability to enhance deep representations with prior knowledge is receiving a lot of attention from the AI community as a key enabler to improve the way modern Artificial Neural Networks (ANN) learn. In this paper we introduce our approach to this task, which relies on a knowledge extraction algorithm, a knowledge injection algorithm and a common intermediate knowledge representation as an alternative to traditional neural transfer. As a result of this research, we envisage a knowledge-enhanced ANN, which will be able to learn, characterise and reuse knowledge extracted from the learning process, thus enabling more robust architecture-agnostic neural transfer, greater explainability and further integration of neural and symbolic approaches to learning.

Due to its potential to solve complex tasks, deep learning is being used across many different areas. The complexity of neural networks however makes it difficult to explain the whole decision process used by the model, which makes understanding deep learning models an active research topic. In this work we address this issue by extracting the knowledge acquired by trained Deep Neural Networks (DNNs) and representing this knowledge in a graph. The proposed graph encodes statistical correlations between neurons’ activation values in order to expose the relationship between neurons in the hidden layers with both the input layer and output classes. Two initial experiments in image classification were conducted to evaluate whether the proposed graph can help understanding and explaining DNNs. We first show how it is possible to explore the proposed graph to find what neurons are the most important for predicting each class. Then, we use graph analysis to detect groups of classes that are more similar to each other and how these similarities affect the DNN. Finally, we use heatmaps to visualize what parts of the input layer are responsible for activating each neuron in hidden layers. The results show that by building and analysing the proposed graph it is possible to gain relevant insights of the DNN’s inner workings.

The ability to enhance deep representations with prior knowledge is receiving a lot of attention from the AI community as a key enabler to improve the way modern Artificial Neural Networks (ANN) learn. In this paper we introduce our approach to this task, which relies on a knowledge extraction algorithm, a knowledge injection algorithm and a common intermediate knowledge representation as an alternative to traditional neural transfer. As a result of this research, we envisage a knowledge-enhanced ANN, which will be able to learn, characterise and reuse knowledge extracted from the learning process, thus enabling more robust architecture-agnostic neural transfer, greater explainability and further integration of neural and symbolic approaches to learning.

Due to its potential to solve complex tasks, deep learning is being used across many different areas. The complexity of neural networks however makes it difficult to explain the whole decision process used by the model, which makes understanding deep learning models an active research topic. In this work we address this issue by extracting the knowledge acquired by trained Deep Neural Networks (DNNs) and representing this knowledge in a graph. The proposed graph encodes statistical correlations between neurons’ activation values in order to expose the relationship between neurons in the hidden layers with both the input layer and output classes. Two initial experiments in image classification were conducted to evaluate whether the proposed graph can help understanding and explaining DNNs. We first show how it is possible to explore the proposed graph to find what neurons are the most important for predicting each class. Then, we use graph analysis to detect groups of classes that are more similar to each other and how these similarities affect the DNN. Finally, we use heatmaps to visualize what parts of the input layer are responsible for activating each neuron in hidden layers. The results show that by building and analysing the proposed graph it is possible to gain relevant insights of the DNN’s inner workings.

The ability to enhance deep representations with prior knowledge is receiving a lot of attention from the AI community as a key enabler to improve the way modern Artificial Neural Networks (ANN) learn. In this paper we introduce our approach to this task, which relies on a knowledge extraction algorithm, a knowledge injection algorithm and a common intermediate knowledge representation as an alternative to traditional neural transfer. As a result of this research, we envisage a knowledge-enhanced ANN, which will be able to learn, characterise and reuse knowledge extracted from the learning process, thus enabling more robust architecture-agnostic neural transfer, greater explainability and further integration of neural and symbolic approaches to learning.

Much of the recent hype around artificial intelligence stems from recent advances in Neural Networks, currently the most widely used algorithm that succeeded where other approaches failed for decades. Neural Networks today can leverage large amounts of data to be trained to perform hard tasks such as recognising objects in an image or translating languages. The process they use to perform these tasks is equivalent to a complex pattern recognition procedure which uses some clever mathematics to expose the underlying structure in a body of data. Humans think in a more conceptual way. We build a mental model of our world. We have the ability to extract relationships such as causality between elements involved in learning to perform a task, and the ability to use background knowledge when learning. One of the key challenges in making more human-like artificial intelligence is incorporating these properties of natural learning into the neural network paradigm. Designing such a system which could utilise background knowledge in learning a new task would enable the networks to be trained on much less data, opening up a new world of opportunities for Neural Networks to be applied to tasks which were previously not feasible due to the scarce availability of data. In identifying these challenges, we have been inspired by recent seminal papers within the Deep Learning community, which call for new approaches to enhance deep representations with (common-sense) background knowledge. This is considered as a key enabler to significantly improve the ability of machines to learn new tasks faster and in a domain invariant way. The main practical challenges involved in this research are finding how best to extract and format relevant knowledge from a trained network, and finding how best to inject this knowledge into an untrained network.

Much of the recent hype around artificial intelligence stems from recent advances in Neural Networks, currently the most widely used algorithm that succeeded where other approaches failed for decades. Neural Networks today can leverage large amounts of data to be trained to perform hard tasks such as recognising objects in an image or translating languages. The process they use to perform these tasks is equivalent to a complex pattern recognition procedure which uses some clever mathematics to expose the underlying structure in a body of data. Humans think in a more conceptual way. We build a mental model of our world. We have the ability to extract relationships such as causality between elements involved in learning to perform a task, and the ability to use background knowledge when learning. One of the key challenges in making more human-like artificial intelligence is incorporating these properties of natural learning into the neural network paradigm. Designing such a system which could utilise background knowledge in learning a new task would enable the networks to be trained on much less data, opening up a new world of opportunities for Neural Networks to be applied to tasks which were previously not feasible due to the scarce availability of data. In identifying these challenges, we have been inspired by recent seminal papers within the Deep Learning community, which call for new approaches to enhance deep representations with (common-sense) background knowledge. This is considered as a key enabler to significantly improve the ability of machines to learn new tasks faster and in a domain invariant way. The main practical challenges involved in this research are finding how best to extract and format relevant knowledge from a trained network, and finding how best to inject this knowledge into an untrained network.

Much of the recent hype around artificial intelligence stems from recent advances in Neural Networks, currently the most widely used algorithm that succeeded where other approaches failed for decades. Neural Networks today can leverage large amounts of data to be trained to perform hard tasks such as recognising objects in an image or translating languages. The process they use to perform these tasks is equivalent to a complex pattern recognition procedure which uses some clever mathematics to expose the underlying structure in a body of data. Humans think in a more conceptual way. We build a mental model of our world. We have the ability to extract relationships such as causality between elements involved in learning to perform a task, and the ability to use background knowledge when learning. One of the key challenges in making more human-like artificial intelligence is incorporating these properties of natural learning into the neural network paradigm. Designing such a system which could utilise background knowledge in learning a new task would enable the networks to be trained on much less data, opening up a new world of opportunities for Neural Networks to be applied to tasks which were previously not feasible due to the scarce availability of data. In identifying these challenges, we have been inspired by recent seminal papers within the Deep Learning community, which call for new approaches to enhance deep representations with (common-sense) background knowledge. This is considered as a key enabler to significantly improve the ability of machines to learn new tasks faster and in a domain invariant way. The main practical challenges involved in this research are finding how best to extract and format relevant knowledge from a trained network, and finding how best to inject this knowledge into an untrained network.

Much of the recent hype around artificial intelligence stems from recent advances in Neural Networks, currently the most widely used algorithm that succeeded where other approaches failed for decades. Neural Networks today can leverage large amounts of data to be trained to perform hard tasks such as recognising objects in an image or translating languages. The process they use to perform these tasks is equivalent to a complex pattern recognition procedure which uses some clever mathematics to expose the underlying structure in a body of data. Humans think in a more conceptual way. We build a mental model of our world. We have the ability to extract relationships such as causality between elements involved in learning to perform a task, and the ability to use background knowledge when learning. One of the key challenges in making more human-like artificial intelligence is incorporating these properties of natural learning into the neural network paradigm. Designing such a system which could utilise background knowledge in learning a new task would enable the networks to be trained on much less data, opening up a new world of opportunities for Neural Networks to be applied to tasks which were previously not feasible due to the scarce availability of data. In identifying these challenges, we have been inspired by recent seminal papers within the Deep Learning community, which call for new approaches to enhance deep representations with (common-sense) background knowledge. This is considered as a key enabler to significantly improve the ability of machines to learn new tasks faster and in a domain invariant way. The main practical challenges involved in this research are finding how best to extract and format relevant knowledge from a trained network, and finding how best to inject this knowledge into an untrained network.

Smart city applications in the Big Data era require not only techniques dedicated to dynamicity handling, but also the ability to take into account contextual information, user preferences and requirements, and real-time events to provide optimal solutions and automatic configuration for the end user. In this paper, we present a specific functionality in the design and implementation of a declarative decision support component that exploits contextual information, user preferences and requirements to automatically provide optimal configurations of smart city applications. The key property of user-centricity of our approach is achieved by enabling users to declaratively specify constraints and preferences on the solutions provided by the smart city application through the Decision Support component, and automatically map these constraints and preferences to provide optimal responses targeting user needs. We showcase the effectiveness and flexibility of our solution in two real usecase scenarios: a multimodal travel planner and a mobile parking application. All the components and algorithms described in this paper have been defined and implemented as part of the Smart City Framework CityPulse (www.ict-citypulse.eu).

Stream reasoning is an emerging research area focused on providing continuous reasoning solutions for data streams. The high expressiveness of non-monotonic reasoning enables complex decision making by managing defaults, common-sense, preferences, recursion, and non-determinism, but it is computationally intensive. The exponential growth in the availability of streaming data on the Web has seriously hindered the applicability of state-of-the-art non-monotonic reasoners to be applied to streaming information in a scalable way. In this paper, we address the issue of scalability for non-monotonic stream reasoning based on Answer Set Programming (ASP), by analyzing input dependency. We introduce an input dependency graph to represent the relationships between input events based on the structure of a given logical rule set. The input dependency graph allows us to dynamically configure the streaming window size in order to maximise the scalability of the non-monotonic reasoner. We conduct an experimental evaluation to demonstrate the effectiveness and ability of our proposed approach in improving the scalability of disjunctive logic programming with ASP in dynamic environments.

Stream Reasoning refers to inference approaches and deduction mechanisms which are concerned with providing continuous inference capabilities over dynamic data. The paradigm shift from current batch-like approaches towards timely and scalable stream reasoning leverages the natural temporal order in data streams and applies windows-based processing to complex deduction tasks that go beyond continuous query processing such as those involving preferential reasoning, constraint ptimization, planning, uncertainty, non-monotonicity, non-determinism, and solution enumeration.

Smart city applications in the Big Data era require not only techniques dedicated to dynamicity handling, but also the ability to take into account contextual information, user preferences and requirements, and real-time events to provide optimal solutions and automatic configuration for the end user. In this paper, we present a specific functionality in the design and implementation of a declarative decision support component that exploits contextual information, user preferences and requirements to automatically provide optimal configurations of smart city applications. The key property of user-centricity of our approach is achieved by enabling users to declaratively specify constraints and preferences on the solutions provided by the smart city application through the Decision Support component, and automatically map these constraints and preferences to provide optimal responses targeting user needs. We showcase the effectiveness and flexibility of our solution in two real usecase scenarios: a multimodal travel planner and a mobile parking application. All the components and algorithms described in this paper have been defined and implemented as part of the Smart City Framework CityPulse (www.ict-citypulse.eu).

Stream reasoning is an emerging research area focused on providing continuous reasoning solutions for data streams. The high expressiveness of non-monotonic reasoning enables complex decision making by managing defaults, common-sense, preferences, recursion, and non-determinism, but it is computationally intensive. The exponential growth in the availability of streaming data on the Web has seriously hindered the applicability of state-of-the-art non-monotonic reasoners to be applied to streaming information in a scalable way. In this paper, we address the issue of scalability for non-monotonic stream reasoning based on Answer Set Programming (ASP), by analyzing input dependency. We introduce an input dependency graph to represent the relationships between input events based on the structure of a given logical rule set. The input dependency graph allows us to dynamically configure the streaming window size in order to maximise the scalability of the non-monotonic reasoner. We conduct an experimental evaluation to demonstrate the effectiveness and ability of our proposed approach in improving the scalability of disjunctive logic programming with ASP in dynamic environments.

Stream Reasoning refers to inference approaches and deduction mechanisms which are concerned with providing continuous inference capabilities over dynamic data. The paradigm shift from current batch-like approaches towards timely and scalable stream reasoning leverages the natural temporal order in data streams and applies windows-based processing to complex deduction tasks that go beyond continuous query processing such as those involving preferential reasoning, constraint ptimization, planning, uncertainty, non-monotonicity, non-determinism, and solution enumeration.

Smart city applications in the Big Data era require not only techniques dedicated to dynamicity handling, but also the ability to take into account contextual information, user preferences and requirements, and real-time events to provide optimal solutions and automatic configuration for the end user. In this paper, we present a specific functionality in the design and implementation of a declarative decision support component that exploits contextual information, user preferences and requirements to automatically provide optimal configurations of smart city applications. The key property of user-centricity of our approach is achieved by enabling users to declaratively specify constraints and preferences on the solutions provided by the smart city application through the Decision Support component, and automatically map these constraints and preferences to provide optimal responses targeting user needs. We showcase the effectiveness and flexibility of our solution in two real usecase scenarios: a multimodal travel planner and a mobile parking application. All the components and algorithms described in this paper have been defined and implemented as part of the Smart City Framework CityPulse (www.ict-citypulse.eu).

Stream Reasoning refers to inference approaches and deduction mechanisms which are concerned with providing continuous inference capabilities over dynamic data. The paradigm shift from current batch-like approaches towards timely and scalable stream reasoning leverages the natural temporal order in data streams and applies windows-based processing to complex deduction tasks that go beyond continuous query processing such as those involving preferential reasoning, constraint ptimization, planning, uncertainty, non-monotonicity, non-determinism, and solution enumeration.

Smart city applications in the Big Data era require not only techniques dedicated to dynamicity handling, but also the ability to take into account contextual information, user preferences and requirements, and real-time events to provide optimal solutions and automatic configuration for the end user. In this paper, we present a specific functionality in the design and implementation of a declarative decision support component that exploits contextual information, user preferences and requirements to automatically provide optimal configurations of smart city applications. The key property of user-centricity of our approach is achieved by enabling users to declaratively specify constraints and preferences on the solutions provided by the smart city application through the Decision Support component, and automatically map these constraints and preferences to provide optimal responses targeting user needs. We showcase the effectiveness and flexibility of our solution in two real usecase scenarios: a multimodal travel planner and a mobile parking application. All the components and algorithms described in this paper have been defined and implemented as part of the Smart City Framework CityPulse (www.ict-citypulse.eu).

Stream reasoning is an emerging research area focused on providing continuous reasoning solutions for data streams. The high expressiveness of non-monotonic reasoning enables complex decision making by managing defaults, common-sense, preferences, recursion, and non-determinism, but it is computationally intensive. The exponential growth in the availability of streaming data on the Web has seriously hindered the applicability of state-of-the-art non-monotonic reasoners to be applied to streaming information in a scalable way. In this paper, we address the issue of scalability for non-monotonic stream reasoning based on Answer Set Programming (ASP), by analyzing input dependency. We introduce an input dependency graph to represent the relationships between input events based on the structure of a given logical rule set. The input dependency graph allows us to dynamically configure the streaming window size in order to maximise the scalability of the non-monotonic reasoner. We conduct an experimental evaluation to demonstrate the effectiveness and ability of our proposed approach in improving the scalability of disjunctive logic programming with ASP in dynamic environments.

In Web stream processing, there are queries that integrate Web data of various velocity, categorized broadly as streaming (i.e., fast changing) and background (i.e., slow changing) data. The introduction of local views on the background data speeds up the query answering process, but requires maintenance processes to keep the replicated data up-to-date. In this work, we study the problem of maintaining local views in a Web setting, where background data are usually stored remotely, are exposed through services with constraints on the data access (e.g., invocation rate limits and data access patterns) and, contrary to the database setting, do not provide streams with changes over their content. Then, we propose an initial solution: WBM, a method to maintain the content of the view with regards to query and user-defined constraints on accuracy and responsiveness.