Search

Purpose: This paper aims to analyze the performance of artificial neural networks with filling methods in predicting the minimum fluidization velocity of different biomass types for bioenergy applications. Design/methodology/approach: An extensive literature review was performed to create an efficient database for training purposes. The database consisted of experimental values of the minimum fluidization velocity, physical properties of the biomass particles (density, size and sphericity) and characteristics of the fluidization (monocomponent experiments or binary mixture). The neural models developed were divided into eight different cases, in which the main difference between them was the filling method type (K-nearest neighbors [KNN] or linear interpolation) and the number of input neurons. The results of the neural models were compared to the classical correlations proposed by the literature and empirical equations derived from multiple regression analysis. Findings: The performance of a given filling method depended on the characteristics and size of the database. The KNN method was superior for lower available data for training and specific fluidization experiments, like monocomponent or binary mixture. The linear interpolation method was superior for a wider and larger database, including monocomponent and binary mixture. The performance of the neural model was comparable with the predictions of the most well-known correlations from the literature. Originality/value: Techniques of machine learning, such as filling methods, were used to improve the performance of the neural models. Besides the typical comparisons with conventional correlations, comparisons with three main equations derived from multiple regression analysis were reported and discussed. [ABSTRACT FROM AUTHOR]

In the contemporary data-driven era, the fields of machine learning, deep learning, big data, statistics, and data science are essential for forecasting outcomes and getting insights from data. This paper looks at how machine learning approaches can be used to anticipate solar power generation, assess heat exchanger heat transfer efficiency, and predict the thermo-physical properties of nanofluids. The review specifically focuses on the potential use of machine learning in solar thermal applications, perovskites, and photovoltaic power forecasting. Predictions of nanofluid characteristics and device performance may be more accurately made with the development of machine learning algorithms. The use of machine learning in the creation of new perovskites and the assessment of their effectiveness and stability is also included in the review. Additionally, the paper explores developments in artificial intelligence, particularly deep learning, in this area and offers insights into techniques for forecasting solar power, including PV production, cloud motion, and weather classification. [ABSTRACT FROM AUTHOR]

Condition-based maintenance (CBM) policy can avoid premature or late maintenance and reduce system failures and maintenance costs. Most existing CBM studies cannot solve the dimensional disaster problem in multi-component complex systems. Only some studies consider the constraint of maintenance resources when searching for the optimal maintenance policy, which is hard to apply to practical maintenance. This paper studies the joint optimization of the CBM policy and spare components inventory for the multi-component system in large state and action spaces. We use Markov Decision Process to model it and propose an improved deep reinforcement learning algorithm based on the stochastic policy and actor-critic framework. In this algorithm, factorization decomposes the system action into the linear combination of each component's action. The experimental results show that the algorithm proposed in this paper has better time performance and lower system cost compared with other benchmark algorithms. The training time of the former is only 28.5% and 9.12% of that of PPO and DQN algorithms, and the corresponding system cost is decreased by 17.39% and 27.95%, respectively. At the same time, our algorithm has good scalability and is suitable for solving Markov decision-making problems in large-scale state and action space. • Considering minor and major repair, we model the joint optimization of CBM and spare ordering for large multi-component systems based on MDP. • An improved DRL algorithm is presented to deal with the MDP model in large-scale discrete state and action space. • We validate our DRL algorithm has good time performance and optimal decision-making series solution via comparisons with DQN and PPO algorithm. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new deep learning algorithm based on the physics-informed neural network (PINN) for solving the Cahn–Hilliard (CH) equations. We adopt the discrete time model of the PINN and use Runge–Kutta methods to discretize the time variable. We introduce a novel loss function and update it at each time step so that the computational cost can be significantly reduced compared to some existing methods. Numerical results of several one-dimensional and two-dimensional CH equations are presented to validate the efficiency and accuracy of the proposed method. [ABSTRACT FROM AUTHOR]

Multimodal emotion recognition, that is, emotion recognition uses machine learning to generate multi-modal features on the basis of videos which has become a research hotspot in the field of artificial intelligence. Traditional multi-modal emotion recognition method only simply connects multiple modalities, and the interactive utilization rate of modal information is low, and it cannot reflect the real emotion under the conflict of modal features well. This article first proves that effective weighting can improve the discrimination between modalities. Therefore, this paper takes into account the importance differences between multiple modalities, and assigns weights to them through the importance attention network. At the same time, considering that there is a certain complementary relationship between the modalities, this paper constructs an attention network with complementary modalities. Finally, the reconstructed features are fused to obtain a multi-modal feature with good interaction. The method proposed in this paper is compared with traditional methods in public datasets. The test results show that our method is accurate in It performs well in both the rate and confusion matrix metrics. [ABSTRACT FROM AUTHOR]

Thin-walled structural problems have been a longstanding computational challenge. The research bottleneck in applying the standard numerical methods for such problems arises from their special geometrical configurations in which the thickness-to-length ratio of the thin-structures is usually up to the order of 10−6 or even smaller. In this paper, we present a method to solve such problems using physics-informed neural networks (PINNs) which are trained to satisfy the differential operator and the corresponding boundary/initial conditions. The PINNs-based method is meshless which is a key feature since mesh-based methods become infeasible for problems with ultra-thin shapes. Instead of using a mesh, the PINNs are trained on batches of randomly sampled collocation points. The algorithm is tested for a class of thin-walled structural problems, including elastic/piezoelectric thin-films as well as ultra-thin coating/substrate structures. We also present comparisons with numerical solutions obtained by using an advanced boundary element method (BEM). Accurate and reliable PINNs results can be achieved for a relative thickness-to-length ratio of the thin structures as small as 10−8, which is sufficient for modeling most of thin structures as used in smart materials. A self-contained MATLAB code and data-sets accompanying this paper are also provided. [ABSTRACT FROM AUTHOR]

The training effect of the neural network model is directly influenced by optimization algorithms. The Barzilai-Borwein(BB) method is used in the stochastic gradient descent (SGD) and other deep learning optimization algorithms because of its outstanding performance in terms of convergence speed. In order to improve the stability of BB step size and the output quality of network model in deep learning, this paper presents two improved optimization algorithms based on the BB method: BBbound and AdaBBbound. BBbound reduces the floating BB step size by generating the upper bound of the step in the current iteration. It avoids the occasional occurrence of long steps; AdaBBbound is an adaptive gradient method based on BB method with dynamic bound. The algorithm has a fast convergence effect in the early stage, then calculates learning rate and smoothly transitions to SGD by setting the better conditions of the BB method. We analyzed the performance of the two algorithms based on the initial conditions and the learning rate change curve. Meanwhile, we tested our algorithms on popular network models such as ResNet and DenseNet. The results showed that the new optimization algorithms have achieved high stability and significantly improved the performance of the network model. [ABSTRACT FROM AUTHOR]

To further enhance the reliability of Machine Learning (ML) systems, considerable efforts have been dedicated to developing privacy protection techniques. Recently, membership privacy has gained increasing attention, with a focus on determining whether a specific data point is present in the confidential training set of an ML model. However, most current attacks only prioritize attack accuracy and fail to extend their range to the evidence that contributes to the member/non-member classification. This limitation greatly reduces the practicality of Membership Inference Attacks (MIA), as real-world data typically includes multiple features, making it challenging to identify which features are involved in the sensitive training set. Therefore, this paper targets one of the fundamental challenges in membership inference attack: measuring the distance between an attack sample and a member sample. Specifically, we propose a novel threat model called Membership Reconstruction Attack (MRA), which aims to reconstruct the exact distribution of the target training set. MRA achieves this by marking each input dimension (e.g., pixels) according to its similarity to the target dataset in feature space. Our attack demonstrates its effectiveness across various settings, including different major datasets (MNIST, CIFAR-10, CIFAR-100) and different model architectures (AlexNet, ResNet, DenseNet, and generative models). Additionally, we evaluate MRA from the defenders' perspective and test several defense approaches against our attack. [ABSTRACT FROM AUTHOR]

Machine learning models, especially deep neural network (DNN) models, have widespread and valuable applications in business activities. Training a deep learning model for commercial use requires plenty of private data, expert knowledge, and computing resources. The huge commercial value of such trained models has attracted the attention of attackers. Attackers can construct a dataset by repeatedly querying the target model for the output of the requested samples and then train a substitute model on this dataset that functions similarly to the target model. In this paper, we propose a defense scheme based on physical unclonable function (PUF) against such black-box model extraction attacks. We deploy a PUF on the user side and the corresponding PUF model on the service provider side to ensure that only legitimate users can obtain the correct model predictions. Our experimental results show that by choosing a suitable fuzzy extractor threshold d , legitimate users can recover more than 99.5% of the prediction results with a little additional computational overhead to the service provider. We perform a model extraction attack in the most favorable case for the attacker, and the prediction accuracy of the obtained substitute model is only about 10%, which demonstrates the effectiveness of our proposed scheme. Compared to existing defenses, our scheme not only effectively prevents black-box model extraction attacks but also ensures that the accuracy of the prediction service for legitimate users is not affected. • Defense based on PUF obfuscation against black-box model extraction attacks. • A PUF and the corresponding PUF model ensure that legitimate users can obtain correct model predictions. • Legitimate users can correctly recover more than 99.5% of the predictions with a little additional computational overhead. • The prediction accuracy of the model extracted by the attacker is only about 10%. [ABSTRACT FROM AUTHOR]

• A new supervised fuzzy measure learning algorithm is proposed for combining classifiers. • Different from previous works, any cost function can be optimized, including those well-suited for classification problems. • Three new update policies are proposed for dealing with monotonicity constrains. • The benefit of using appropriate cost functions for classification is experimentally demonstrated. • The new proposal outperforms existing methods in both binary and multi-class problems. Fuzzy measure-based aggregations allow taking interactions among coalitions of the input sources into account. Their main drawback when applying them in real-world problems, such as combining classifier ensembles, is how to define the fuzzy measure that governs the aggregation and specifies the interactions. However, their usage for combining classifiers has shown its advantage. The learning of the fuzzy measure can be done either in a supervised or unsupervised manner. This paper focuses on supervised approaches. Existing supervised approaches are designed to minimize the mean squared error cost function, even for classification problems. We propose a new fuzzy measure learning algorithm for combining classifiers that can optimize any cost function. To do so, advancements from deep learning frameworks are considered such as automatic gradient computation. Therefore, a gradient-based method is presented together with three new update policies that are required to preserve the monotonicity constraints of the fuzzy measures. The usefulness of the proposal and the optimization of cross-entropy cost are shown in an extensive experimental study with 58 datasets corresponding to both binary and multi-class classification problems. In this framework, the proposed method is compared with other state-of-the-art methods for fuzzy measure learning. [ABSTRACT FROM AUTHOR]

• A new supervised fuzzy measure learning algorithm is proposed for combining classifiers. • Different from previous works, any cost function can be optimized, including those well-suited for classification problems. • Three new update policies are proposed for dealing with monotonicity constrains. • The benefit of using appropriate cost functions for classification is experimentally demonstrated. • The new proposal outperforms existing methods in both binary and multi-class problems. Fuzzy measure-based aggregations allow taking interactions among coalitions of the input sources into account. Their main drawback when applying them in real-world problems, such as combining classifier ensembles, is how to define the fuzzy measure that governs the aggregation and specifies the interactions. However, their usage for combining classifiers has shown its advantage. The learning of the fuzzy measure can be done either in a supervised or unsupervised manner. This paper focuses on supervised approaches. Existing supervised approaches are designed to minimize the mean squared error cost function, even for classification problems. We propose a new fuzzy measure learning algorithm for combining classifiers that can optimize any cost function. To do so, advancements from deep learning frameworks are considered such as automatic gradient computation. Therefore, a gradient-based method is presented together with three new update policies that are required to preserve the monotonicity constraints of the fuzzy measures. The usefulness of the proposal and the optimization of cross-entropy cost are shown in an extensive experimental study with 58 datasets corresponding to both binary and multi-class classification problems. In this framework, the proposed method is compared with other state-of-the-art methods for fuzzy measure learning. [ABSTRACT FROM AUTHOR]

Machine learning becomes popular and is used for a wide range of problems in various areas of applied sciences. In dynamical systems, machine learning methods are applied to solve differential equations. In this paper, we develop an artificial network to solve systems of ordinary differential equations. For the network, we use a multilayer perceptron networks, which is a fully connected feedforward network to predict the flow of a specific system. In order to improve the long time estimation of the method, we introduced an upper bound control strategy. To deal with stiff ODE systems(such as slow-fast coupled system), a new algorithm, Finite Neural Element method, is introduced. By numerical simulation, the novel algorithm is proved to have better efficiency and accuracy than direct machine learning method. • A deep neural network is developed to solve a bundle of solutions of the differential equations. • A control strategy for the neural network is introduced to increase the accuracy of the method. • The Finite Neural Element method is capable of solving systems with multiple time scales. [ABSTRACT FROM AUTHOR]

Federated learning is a privacy-preserving machine learning paradigm that can train a model with decentralized data. Classical federated learning systems are vulnerable to attacks from malicious clients. Although a number of efforts have been made to improve robustness of federated learning, existing solutions are either too costly or attack-specific. In this paper, we propose the Robust Ternary Gradients Aggregation (RTGA) algorithm, which can efficiently handle different attacks using two novel mechanisms. At the client-side, the ternary quantization mechanism compresses gradients using only two bits to store each coordinate of the gradient vector. We adopt the error-feedback mechanism to compensate for the difference between the compressed and actual gradients. At the server-side, the robust aggregation mechanism is designed to detect malicious clients according to the risk score of each client. The risk score is calculated by solving an optimization problem of the divergence among clients. Furthermore, we provide analysis of the compression and robustness properties of our proposed algorithm. Extensive experiments show that, compared with existing algorithms, RTGA can mitigate different types of attacks while costing much less communications. [ABSTRACT FROM AUTHOR]

Long-tailed distribution learning is one of the critical research fields of deep learning and has gradually become a research hotspot. Existing re-sampling methods for long-tailed data classification attempt to adjust the number of tail class samples to balance the overall feature space and achieve satisfactory results. However, the methods impair the representative ability of the learned features to a certain extent, which in turn affects the tail class feature space. In this paper, we propose a hybrid ResNet based on joint basic and attention modules to enhance the tail class feature space, which provides rich discriminative and representative features in the tail class feature space. Firstly, we use hybrid ResNet to extract features, where the basic module ResNet and the attention module ResNet extract head and tail class features, respectively. The enhancement of tail class features can reduce the dependence of the classifier on head class features. Secondly, we build a fusion loss function, which considers the tradeoff between head loss and tail loss for long-tailed distribution learning. Experimental results show that the proposed model outperforms several state-of-the-art models in the long-tailed classification. Our model was 2.67% better than the optimal method under the long-tailed Tiny-Imagenet-LT dataset with an imbalanced ratio of 100. [ABSTRACT FROM AUTHOR]

Federated learning (FL), as a distributed machine learning paradigm, essentially promises that multiple parties can jointly train the model collaboratively without sharing local data. Recent research demonstrates that the adversary can deduce the sensitive data through shared model updates. To protect the data privacy of the participants, differential privacy (DP) is deployed in various FL scenarios due to the lightweight computational overhead. However, the trade-off between the availability and privacy of local models is the fundamental problem that needs to be solved in DP applications. In this paper, we propose a fine-grained and privacy-aware FL framework (iDP-FL) to enable training data and model parameters to satisfy confidentiality while markedly improving the model's prediction accuracy. Specifically, we first design an individualized perturbation noise (IPN) algorithm that adds different artificial noises dependent on the importance of each participant's model weight. Then, we propose a perturbation mechanism on the aggregator side, a DP protection method under the premise of loss function convergence, which prevents the global model parameters from being stolen by malicious adversaries. Moreover, to achieve lightweight protection throughout the learning, we present an advanced bilateral perturbation (ABP) protocol to perform iterative training. Theoretical analysis indicates that iDP-FL provides the DP guarantee, which yields superior prediction accuracy and excellent privacy-preserving with the same privacy level. Finally, extensive experiments conducted on real-world datasets demonstrate that our approach shows significant advantages with limited privacy budgets, especially at small privacy losses. [ABSTRACT FROM AUTHOR]

For their outstanding ability of finding hidden patterns in data, deep learning models have been extensively applied in many different domains. However, recent works have shown that, if a set of requirements expressing inherent knowledge about the problem at hand is given, then neural networks often fail to comply with them. This represents a major drawback for deep learning models, as requirements compliance is normally considered a necessary condition for standard software deployment. In this paper, we propose a novel neuro-symbolic framework able to make any neural network compliant by design to a given set of requirements over the output space expressed in full propositional logic. This framework, called CCN+, integrates the requirements into the output layer of the neural network by applying multiple inference rules that ensure compliance with the requirements and adapts the standard binary cross-entropy loss function to the requirement output layer. As a result, not only the outputted predictions are guaranteed to be compliant with the requirements, but the neural network itself learns how to exploit the domain knowledge expressed by the requirements to get better performance. We conduct an extensive experimental evaluation of CCN+ on 19 real-world multi-label classification datasets with propositional logic requirements, including a challenging dataset for autonomous driving. Our experimental analysis confirms that CCN+ is able to outperform both its neural counterparts and the state-of-the-art models. [ABSTRACT FROM AUTHOR]

Few-shot learning, which aims to identify new classes with few samples, is an increasingly popular and crucial research topic in the machine learning. Recently, the development of deep learning has deepened the network structure of a few-shot model, thereby obtaining deeper features from the samples. This trend led to an increasing number of few-shot learning models pursuing more complex structures and deeper features. However, discarding shallow features and blindly pursuing the depth of sample feature levels is not reasonable. The features at different levels of the sample have different information and characteristics. In this paper, we propose a few-shot image classification model based on deep and shallow feature fusion and a coarse-grained relationship score network (HFFCR). First, we utilize networks with different depth structures as feature extractors and then fuse the two kinds of sample features. The fused sample features collect sample information at different levels. Second, we condense the fused features into a coarse-grained prototype point. Prototype points can better represent the information in this class and improve classification efficiency. Finally, we construct a relationship score network, concatenating the prototype points and query samples into a feature map and sending it into the network to calculate the relationship score. The classification criteria for learnable relationship scores reflect the information difference between the two samples. Experiments on three datasets show that HFFCR has advanced performance. [ABSTRACT FROM AUTHOR]

One-class classification is a machine learning problem, where training data has only one class. The objective is to determine if the input data is seen class or unseen class. Traditional deep learning algorithms are not suitable for this task since the algorithm can predict only class in training data. In this paper, the one-class classification algorithm using construction error of image transformation network (OCITN) is proposed. In particular, image transformation network (ITN) is introduced as a subtask, which transforms input image into one image, namely goal image. Moreover, the error of ITN, namely construction error (CE), is computed as a distance metric between the goal image and model output. ITN model is trained using only one-class images and is applied for testing images. Since the model is trained with only one-class images, the CE for one-class is small relative to other classes. Thus, one-class classification is made determining CE is large or small. The proposed method is experimented with using MNIST, Fashion MNIST, CIFAR10, CIFAR100, and Cat-vs-Dog datasets. OCITN shows good results where the goal image has high entropy. Additionally, the extension of OCITN, namely OCITNE, is implemented. This method shows the state of the art performance in MNIST (98.0), Fashion MNIST(95.6), and acceptable performance in CIFAR10(78.4). Furthermore, these methods provide high-speed processing, OCITN process 5291 images, and OCITNE 1261 images per second, 137 times and 33 times faster than state of the art. The source code used in this paper can be downloaded from: https://github.com/ToshiHayashi/OCITN. [ABSTRACT FROM AUTHOR]

Graph mining has been widely studied to analyze real-world graph properties and applied to various applications. In particular, graph subspace clustering performance, defined as partitioning high-dimensional graph data into several clusters by finding minimum weights for the edges, has been consistently improved by exploiting deep learning algorithms with Euclidean features extracted from Euclidean domains (image datasets). Most subspace clustering algorithms tend to extract features from the Euclidean domain to identify graph characteristics and structures, and hence are limited for real-world data applications in non-Euclidean domains. This paper proposes a self-supervised deep geometric subspace clustering algorithm optimized for non-Euclidean high-dimensional graph data by emphasizing spatial features and geometric structures while simultaneously reducing redundant nodes and edges. Quantitative and qualitative experimental results verified the proposed approach is effective for graph clustering compared with previous state-of-the-art algorithms on public datasets. [ABSTRACT FROM AUTHOR]

• A dual-attention network (DA-Net) is proposed to capture the local–global features for multivariate time series classification. • Squeeze-Excitation Window Attention (SEWA) layer is proposed to mine the local significant feature. • Sparse Self-Attention within Windows (SSAW) layer is proposed to handle the long-range dependencies. • DA-Net achieves superior performance on UEA datasets. Multivariate time series classification is one of the increasingly important issues in machine learning. Existing methods focus on establishing the global long-range dependencies or discovering the local critical sequence fragments. However, they often ignore the combined information from both global and local features. In this paper, we propose a novel network (called DA-Net) based on dual attention to mine the local–global features for multivariate time series classification. Specifically, DA-Net consists of two distinctive layers, i.e., the Squeeze-Excitation Window Attention (SEWA) layer and the Sparse Self-Attention within Windows (SSAW) layer. For the SEWA layer, we capture the local window-wise information by explicitly establishing window dependencies to prioritize critical windows. For the SSAW layer, we preserve rich activate scores with less computation to widen the window scope for capturing global long-range dependencies. Based on the two elaborated layers, DA-Net can mine critical local sequence fragments in the process of establishing global long-range dependencies. The experimental results show that DA-Net is able to achieve competing performance with state-of-the-art approaches on the multivariate time series classification. [ABSTRACT FROM AUTHOR]

Purpose: With the rise of social media platforms, an increasing number of cases of cyberbullying has reemerged. Every day, large number of people, especially teenagers, become the victim of cyber abuse. A cyberbullied person can have a long-lasting impact on his mind. Due to it, the victim may develop social anxiety, engage in self-harm, go into depression or in the extreme cases, it may lead to suicide. This paper aims to evaluate various techniques to automatically detect cyberbullying from tweets by using machine learning and deep learning approaches. Design/methodology/approach: The authors applied machine learning algorithms approach and after analyzing the experimental results, the authors postulated that deep learning algorithms perform better for the task. Word-embedding techniques were used for word representation for our model training. Pre-trained embedding GloVe was used to generate word embedding. Different versions of GloVe were used and their performance was compared. Bi-directional long short-term memory (BLSTM) was used for classification. Findings: The dataset contains 35,787 labeled tweets. The GloVe840 word embedding technique along with BLSTM provided the best results on the dataset with an accuracy, precision and F1 measure of 92.60%, 96.60% and 94.20%, respectively. Research limitations/implications: If a word is not present in pre-trained embedding (GloVe), it may be given a random vector representation that may not correspond to the actual meaning of the word. It means that if a word is out of vocabulary (OOV) then it may not be represented suitably which can affect the detection of cyberbullying tweets. The problem may be rectified through the use of character level embedding of words. Practical implications: The findings of the work may inspire entrepreneurs to leverage the proposed approach to build deployable systems to detect cyberbullying in different contexts such as workplace, school, etc and may also draw the attention of lawmakers and policymakers to create systemic tools to tackle the ills of cyberbullying. Social implications: Cyberbullying, if effectively detected may save the victims from various psychological problems which, in turn, may lead society to a healthier and more productive life. Originality/value: The proposed method produced results that outperform the state-of-the-art approaches in detecting cyberbullying from tweets. It uses a large dataset, created by intelligently merging two publicly available datasets. Further, a comprehensive evaluation of the proposed methodology has been presented. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new optimization method based on the Monte Carlo method. The proposed method is applied to several benchmark problems, and the result of applying it to the optimization of neural network is reported. Deep machine learning using neural networks is one of the important keywords to promote innovation in today's advanced information society. Therefore, research on large-scale, high-speed, and high-precision algorithms has been actively conducted. The author has developed an optimization method in which the search region for multivariate parameters constituting the objective function is divided into two regions for each parameter, the integral values of each divided region are numerically calculated by the Monte Carlo method, the magnitude of each integral value is compared, and the optimum point is judged to be in a small region. The proposed method was applied to 50 variable benchmark functions (Schwefel and Ackley Functions), and was compared with the results of genetic algorithm (GA) which was a representative of existing optimization methods. As a result, it was confirmed that the proposed method is faster and more accurate than GA. In addition, the proposed method is applied to machine learning by neural networks, specifically XOR gate circuits and IRS classification problems, and verified. The neural network optimized by MOST reproduced teacher data and test data faster and more accurately than conventional Adam and genetic algorithms. [ABSTRACT FROM AUTHOR]

• A dual-attention network (DA-Net) is proposed to capture the local–global features for multivariate time series classification. • Squeeze-Excitation Window Attention (SEWA) layer is proposed to mine the local significant feature. • Sparse Self-Attention within Windows (SSAW) layer is proposed to handle the long-range dependencies. • DA-Net achieves superior performance on UEA datasets. Multivariate time series classification is one of the increasingly important issues in machine learning. Existing methods focus on establishing the global long-range dependencies or discovering the local critical sequence fragments. However, they often ignore the combined information from both global and local features. In this paper, we propose a novel network (called DA-Net) based on dual attention to mine the local–global features for multivariate time series classification. Specifically, DA-Net consists of two distinctive layers, i.e., the Squeeze-Excitation Window Attention (SEWA) layer and the Sparse Self-Attention within Windows (SSAW) layer. For the SEWA layer, we capture the local window-wise information by explicitly establishing window dependencies to prioritize critical windows. For the SSAW layer, we preserve rich activate scores with less computation to widen the window scope for capturing global long-range dependencies. Based on the two elaborated layers, DA-Net can mine critical local sequence fragments in the process of establishing global long-range dependencies. The experimental results show that DA-Net is able to achieve competing performance with state-of-the-art approaches on the multivariate time series classification. [ABSTRACT FROM AUTHOR]

Graph mining has been widely studied to analyze real-world graph properties and applied to various applications. In particular, graph subspace clustering performance, defined as partitioning high-dimensional graph data into several clusters by finding minimum weights for the edges, has been consistently improved by exploiting deep learning algorithms with Euclidean features extracted from Euclidean domains (image datasets). Most subspace clustering algorithms tend to extract features from the Euclidean domain to identify graph characteristics and structures, and hence are limited for real-world data applications in non-Euclidean domains. This paper proposes a self-supervised deep geometric subspace clustering algorithm optimized for non-Euclidean high-dimensional graph data by emphasizing spatial features and geometric structures while simultaneously reducing redundant nodes and edges. Quantitative and qualitative experimental results verified the proposed approach is effective for graph clustering compared with previous state-of-the-art algorithms on public datasets. [ABSTRACT FROM AUTHOR]

• A deep metric learning method is presented for facial expression recognition. • Chi-squared distance is modified for deep metric learning in histogram space. • A new fully histogram based Convolutional Neural Network is proposed. • A triplet loss based on modified chi-squared distance is proposed to learn the CNN. Facial expression recognition is a challenging problem in machine learning. There is much research has been conducted in this field; however, the accuracy of facial expression recognition, especially in uncontrolled conditions, needs much improvement. In this paper, a deep histogram metric learning in a Convolutional Neural Network (CNN) is presented for facial expression recognition. The proposed CNN utilizes a histogram calculation layer to provide statistical description of feature maps at the output of the convolutional layers. To train the proposed CNN in histogram space, a learnable matrix (equivalent to the fully connected layer) is introduced in chi-squared distance equation. Then, the modified equation is used in the loss function. The recognition rates of the proposed CNN for seven-class facial expression recognition on four well-known databases including CK+, MMI, SFEW, and RAF-DB are 98.47%, 83.41%, 61.01%, and 89.28%, respectively. The results show superiority of the proposed CNN compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Despite that the emerging internet of things (IoT) increases the chance for users to interact with things, generating massive context data, the ground-truth user-thing interaction data is still quite sparse compared with the enormous numbers of users and things. Current existing things recommendation methods mostly adopt machine learning or deep learning (e.g., matrix factorization or neural network) as their basic models, which continuously results in that the data sparsity has a great impact on their performance. Although they integrate different contexts to alleviate the problem, the real effect is rather limited. To the end, in the paper, we study how to achieve better recommendation results in the case of the data sparsity. Along this line, the very critical challenge is how to learn user preferences without relying too much on big data. As a solution, we first build an interaction event experience graph (IEEG) from the user interaction data. Secondly, we develop a Non-Axiomatic Logic-based learning framework for novel things correlations, named NALC, running over IEEG, to mine the user-thing relationship. We measure the user-user relationship and the thing-thing relationship from real-time contexts. Furthermore, we fuse the user-user, the thing-thing and the user-thing relationships based on a product rule, to model the recommender. Finally, we evaluate the performance of our proposed model on the real-world dataset and demonstrate that the proposed model outperforms existing state-of-the-art models. [ABSTRACT FROM AUTHOR]

With the advance of machine learning technology and especially the explosive growth of big data, federated learning, which allows multiple participants to jointly train a high-quality global machine learning model, has gained extensive attention. However, in federated learning, it has been proved that inference attacks could reveal sensitive information from both local updates and global model parameters, which threatens user privacy greatly. Aiming at the challenge, in this paper, a privacy-preserving and lossless federated learning scheme, named CORK, is proposed for deep neural network. With CORK, multiple participants can train a global model securely and accurately with the assistance of an aggregation server. Specifically, we first design a drop-tolerant secure aggregation algorithm FTSA, which ensures the confidentiality of local updates. Then, a lossless model perturbation mechanism PTSP is proposed to protect sensitive data in global model parameters. Furthermore, the neuron pruning operation in PTSP can reduce the scale of models, which thus improves the computation and communication efficiency significantly. Detailed security analysis shows that CORK can resist inference attacks on both local updates and global model parameters. In addition, CORK is implemented with real MNIST and CIFAR-10 datasets, and the experimental results demonstrate that CORK is indeed effective and efficient. [ABSTRACT FROM AUTHOR]

• A new imputation approach based on convolutional neural networks is proposed. • A loss function for imputation is designed in the setting of end-to-end learning. • Fuzzy clustering driven data organization is proposed for creating spatial property. The presence of missing data is a challenging issue in processing real-world datasets. It is necessary to improve the data quality by imputing the missing values so that effective learning from data can be achieved. Recently, deep learning has become the most powerful type of machine learning techniques, which can be used for discovering the hidden knowledge that exists in a large dataset to make accurate predictions. In this paper, we propose an imputation method that involves using a convolutional neural network to impute the missing values. The missing value of each instance is imputed essentially by using a trained kernel. The weights of the kernel are determined by learning from the given data that are arranged spatially in the data matrix. The kernel carries out a weighted sum of neighboring elements in an array for imputing the missing values. In addition, in the absence of the true values with which the missing values are expected to be replaced, a loss function is designed without the need to know the true value. Our method is evaluated on UCI datasets in comparison with state-of-the-art methods. The experimental results show that the proposed approach performs closely to or better than other methods. [ABSTRACT FROM AUTHOR]

• A novel document classification measuring the relevance/importance of emotional categories is proposed. • The Term Frequency – Inverse Document Frequency (TF-IDF) measure is used to assess the emotional importance. • A fuzzy partition of the TF-IDF index is constructed to evaluate the relevance of emotional categories. • A document can be assigned to more than one emotional category, considering the most relevant ones. With the surge of the large volume of data availability, Machine Learning and mainly Deep Learning techniques are the leading solutions in classification and predictive tasks, targeted at data-efficient learning. These models learn by training on many diversified samples in a process that is computationally expensive or time-consuming. Moreover, in many real-world scenarios, the amount of available data for training is unsuitable, because it is unlabeled or covers only portions of the whole reference domain cases. This paper proposes an alternative approach for document classification that leverages the distribution of the data projected in the multi-dimensional feature space to assess the weight of features in the final classification. The approach does not rely on traditional iterative methods for classification but builds a relevance measure to assess the relevance/importance of the features describing the domain of interest. The idea is to harness this metric to select relevant features and then express the values calculated by these metrics in natural language by exploiting fuzzy variables and linguistic labels to make human comprehension more immediate. The approach has been employed for emotion extraction from social media messages. The novelty of this approach is twofold: first, the well-known TF-IDF measure was reinterpreted as a relevance measure of emotions discovered in text content. Then, the discovered emotion relevance was described by fuzzy linguistic labels, defined on an ad-hoc-designed fuzzy partition, to express the data classification in natural language, more suitable to human understanding. [ABSTRACT FROM AUTHOR]

• Introducing a method for arrhythmia classification of 12-lead electrocardiogram(ECG). • Classification is reduced to 1-lead ECG by teacher-student knowledge distillation(KD). • A teacher model is developed through 12-lead ECG signals. • A student model is developed through 1-lead ECG signals. • Teacher-student KD is based on the decompose feature maps within the middle layers. • The results showed competitive performance of the student as opposed to the teacher. Deep learning models developed through multi-lead electrocardiogram (ECG) signals are considered the leading methods for the automated detection of arrhythmia on computer systems. However, due to the amplitudes of input signals, these models generate too many parameters for practical use. Therefore, they are rarely used on devices with limited computational resources in the newly-emerged technology of the Internet of medical things (IoMT). Knowledge distillation was utilized in this paper to propose a method for bridging the gap between the arrhythmia classification model with multi-lead ECG signals and the arrhythmia classification model with single-lead ECG signals by minimizing the performance decline. The proposed method consists of a teacher model with advanced architecture and a student model with simple architecture. The teacher model was already developed through multi-lead ECG signals, whereas the student model was developed through single-lead signals under the supervision of the teacher. Despite its simplicity, the student model receives the dark knowledge of multi-lead ECG signals from the teacher by imitating the teacher's behavior in the development process. According to the results, the student model was nearly 262.18 times more compressed than its teacher. Moreover, the student experienced approximately 0.81% of accuracy decline in Chapman ECG with 10646 patients. [ABSTRACT FROM AUTHOR]

In multi-task learning, the performance is often sensitive to the relationships between tasks. Thus it is important to study how to exploit the complex relationships across different tasks. One line of research captures the complex task relationships, by increasing the model capacity and thus requiring a large training dataset. However in many real-world applications, the amount of labeled data is limited. In this paper, we propose a light weight and specially designed architecture, which aims to model task relationships for small or middle-sized datasets. The proposed framework learns a task-specific ensemble of sub-networks in different depths, and is able to adapt the model architecture for the given data. The task-specific ensemble parameters are learned simultaneously with the weights of the network by optimizing a single loss function defined with respect to the end task. The hierarchical model structure is able to share both general and specific distributed representations to capture the inherent relationships between tasks. We validate our approach on various types of tasks, including synthetic task, article recommendation task and vision task. The results demonstrate the advantages of our model over several competitive baselines especially when the tasks are less-related. [ABSTRACT FROM AUTHOR]

Accurate prediction of the stock indexes in the new energy market is of significant importance to both investors and policymakers. However, in response to the volatility and uncertainty characteristic of the new energy market, most scholars currently focus on training prediction methods using features from a single time scale, which cannot capture the fluctuations of new energy stock indexes under different time scales. Therefore, in this paper, a hybrid deep learning model Multi-kernel Parallel TemporalNet (MKP-TemporalNet) is proposed for predicting new energy stock indexes. This model initially incorporates an attention mechanism to calibrate the feature importance of multivariate time series dynamically, and then combines the characteristics of improved Temporal Convolutional Networks (iTCN) and Bidirectional Gated Recurrent Units (BiGRU) to enhance prediction accuracy effectively. In particular, the novelty of the iTCN lies in the development of a multi-kernel parallel convolution structure within a residual layout at the core of the temporal convolution module, to address the low efficiency of traditional TCN's single kernel convolution in extracting temporal features from input sequences at different time scales. Results from evaluating MKP-TemporalNet against several popular machine learning models on six new energy stock indexes confirm its predictive effectiveness in the new energy sector. [ABSTRACT FROM AUTHOR]

This paper addresses the biological plausibility of both backpropagation (BP) and contrastive Hebbian learning (CHL) used in the Boltzmann machines. The main claim of this paper is that CHL is a general learning algorithm that can be used to steer feedforward networks toward desirable outcomes, and steer them away from undesirable outcomes without any need for the specialized feedback circuit of BP or the symmetric connections used by the Boltzmann machines. After adding perturbations during the learning phase to all the neurons in the network, multiple feedforward outcomes are classified into Hebbian and anti-Hebbian sets based on the network predictions. The algorithm is applied to networks when optimizing a loss objective where BP excels and is also applied to networks with stochastic binary outputs where BP cannot be easily applied. The power of the proposed algorithm lies in its simplicity where both learning and gradient estimation through stochastic binary activations are combined into a single local Hebbian rule. We will also show that both Hebbian and anti-Hebbian correlations are evaluated from the readily available signals that are fundamentally different from CHL used in the Boltzmann machines. We will demonstrate that the new learning paradigm where Hebbian/anti-Hebbian correlations are based on correct/incorrect predictions is a powerful concept that separates this paper from other biologically inspired learning algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we consider the problem of approximating functions from a Korobov space on [ − 1 , 1 ] d by ReLU shallow neural networks and present a rate O (m − 2 5 (1 + 2 d) log ⁡ m) of uniform approximation by networks of m hidden neurons. This is achieved by combining a novel Fourier analysis approach and a probability argument. We apply our approximation theory to a learning algorithm for regression based on ReLU shallow neural networks and derive learning rates of order O (N − 4 (d + 2) 9 d + 8 log ⁡ N) for the excess generalization error with the sample size N when the regression function lies in the Korobov space. [ABSTRACT FROM AUTHOR]

The k Nearest Neighbors (k NN) search in high-dimensional space is a fundamental problem with various applications. In this paper, we try to solve this problem by using deep neural networks (DNNs). We apply DNNs to represent complex correlations between high-dimensional objects, enabling us to project similar objects into the same class, thus reducing the search cost. Based on DNNs, we propose two novel techniques to improve query efficiency while keeping accuracy. First, traditional DNNs typically demand extensive training data for achieving high accuracy. To decrease the training size, we design a multi-module DNN framework comprising several small modules. Each module learns to capture part of knowledge for the given query. The collective output of these sub-modules is then seamlessly integrated to form the final result. Second, with machine learning models, the size of candidates are unbounded. Thus, we design a linear-time data layout refinement algorithm, aiming to limit the number of candidates to a small constant. Empirically we find that our approach significantly outperforms the state-of-the-art methods in terms of both time efficiency and space efficiency while still attaining comparable or better accuracy. [ABSTRACT FROM AUTHOR]

Identifying and reconstructing hadronic τ decays (τ h) is an important task at current and future high-energy physics experiments, as τ h represent an important tool to analyze the production of Higgs and electroweak bosons as well as to search for physics beyond the Standard Model. The identification of τ h can be viewed as a generalization and extension of jet-flavour tagging, which has in the recent years undergone significant progress due to the use of deep learning. Based on a granular simulation with realistic detector effects and a particle flow-based event reconstruction, we show in this paper that deep learning-based jet-flavour-tagging algorithms are powerful τ h identifiers. Specifically, we show that jet-flavour-tagging algorithms such as LorentzNet and ParticleTransformer can be adapted in an end-to-end fashion for discriminating τ h from quark and gluon jets. We find that the end-to-end transformer-based approach significantly outperforms contemporary state-of-the-art τ h reconstruction and identification algorithms currently in use at the Large Hadron Collider. [ABSTRACT FROM AUTHOR]

The brain neurons' electrical activities represented by Electroencephalogram (EEG) signals are the most common data for diagnosing Epilepsy seizure, which is considered a chronic nervous disorder that cannot be controlled medically using surgical operation or medications with more than 40 % of Epilepsy seizure case. With the progress and development of artificial intelligence and deep learning techniques, it becomes possible to detect these seizures over the observation of the non-stationary-dynamic EEG signals, which contain important information about the mental state of patients. This paper provides a concerted deep machine learning model consisting of two simultaneous techniques detecting the activity of epileptic seizures using EEG signals. The time-frequency image of EEG waves and EEG raw waves are used as input components for the convolution neural network (CNN) and recurrent neural network (RNN) with long- and short-term memory (LSTM). Two processing signal methods have been used, Short-Time Fourier Transform (STFT) and Continuous Wavelet Transformation (CWT), have been used for generating spectrogram and scalogram images with sizes of 77 × 75 and 32 × 32, respectively. The experimental results showed a detection accuracy of 99.57 %, 99.57 % using CWT Scalograms, and 99.26 %, 97.12 % using STFT spectrograms as CNN input for the Bonn University dataset and the CHB-MIT dataset, respectively. Thus, the proposed models provide the ability to detect epileptic seizures with high success compared to previous studies. [ABSTRACT FROM AUTHOR]

• Biological age (BA) is a valuable index to identify an individual's health status. • We introduced a risk score (RS) to measure personal health risk quantitatively. • We proposed a RS-embedded deep representation learning algorithm for BA estimation. • We presented a new BA validation method to be applied to unlabeled and labeled data. • The proposed method outperformed existing methods in the RS and disease incidence. The health index measures a person's overall health status which provides useful information for people to manage their health, so developing a precise and relevant health index is urgent. Currently, many researchers have studied the biological age (BA) estimation, one of the beneficial health indices, by applying machine learning and deep learning techniques to health data. However, most of them have focused on the chronological age prediction or basic latent feature extraction methods. In this paper, we present a new algorithm to estimate BA, called Risk Score-Embedded Autoencoder-based BA (RSAE-BA). RSAE-BA can provide an accurate health index by using deep representation learning with an individual's health risk. We first proposed a notion of risk score (RS) calculation to monitor a person's health risk. Then we extracted latent features by using an autoencoder embedding the RS, and used them to generate BA. To evaluate RSAE-BA, we presented a new BA validation method using the RS, which is applicable to both unlabeled and labeled data. We compared the results of RSAE-BA with existing methods, and demonstrated the accuracy of RSAE-BA and its applicability to predict disease incidence. We believe that RSAE-BA will be a useful alternative method to measure a person's health. [ABSTRACT FROM AUTHOR]

In recent years, many applications are using various forms of deep learning models. Such methods are usually based on traditional learning paradigms requiring the consistency of properties among the feature spaces of the training and test data and also the availability of large amounts of training data, e.g., for performing supervised learning tasks. However, many real-world data do not adhere to such assumptions. In such situations transfer learning can provide feasible solutions, e.g., by simultaneously learning from data-rich source data and data-sparse target data to transfer information for learning a target task. In this paper, we survey deep transfer learning models with a focus on applications to text data. First, we review the terminology used in the literature and introduce a new nomenclature allowing the unequivocal description of a transfer learning model. Second, we introduce a visual taxonomy of deep learning approaches that provides a systematic structure to the many diverse models introduced until now. Furthermore, we provide comprehensive information about text data that have been used for studying such models because only by the application of methods to data, performance measures can be estimated and models assessed. [ABSTRACT FROM AUTHOR]

This paper studies the charging/discharging scheduling problem of plug-in electric vehicles (PEVs) in smart grid, considering the users' satisfaction with state of charge (SoC) and the degradation cost of batteries. The objective is to collectively determine the energy usage patterns of all participating PEVs so as to minimize the energy cost of all PEVs while ensuring the charging needs of PEV owners. The challenges herein are mainly in three folds: 1) the randomness of electricity price and PEVs' commuting behavior; 2) the unknown dynamics model of SoC; and 3) a large solution space, which make it challenging to directly develop a model-based optimization algorithm. To this end, we first reformulate the above energy cost minimization problem as a Markov game with unknown transition probabilities. Then a multi-agent deep reinforcement learning (DRL)-based data-driven approach is developed to solve the Markov game. Specifically, the proposed approach consists of two networks: an extreme learning machine (ELM)-based feedforward neural network (NN) for uncertainty prediction of electricity price and PEVs' commuting behavior and a Q network for optimal action-value function approximation. Finally, the comparison results with three benchmark solutions show that our proposed algorithm can not only adaptively decide the optimal charging/discharging policy by on-line learning process, but also yield a lower energy cost within an unknown market environment. [ABSTRACT FROM AUTHOR]

Predicting electricity consumption is not an easy task depending on many factors that affect energy consumption. Therefore, electricity utilities and governments are always searching for intelligent models to improve the accuracy of prediction and recently, deep learning becomes the most used field in prediction. In this paper, we introduce a deep learning model based on deep feedforward neural networks and Long Short-Term Memory. The deep feedforward neural networks architecture was inspired by the Inception Residual Network v2, which achieved the highest accuracy in image classification. We compared our proposed model to other recent deep learning models in two different datasets: dataset from the Distribution Network Station of Tetouan city in Morocco and dataset from the North American Utility. The proposed model achieved the smallest error of Root Mean Square Error comparing to its counterparts. [ABSTRACT FROM AUTHOR]

The integration of wind power with traditional electricity grids is challenging because of the volatile and intermittent nature of wind speed. Consequently, wind speed forecasting is an essential tool in the energy system because it supports economic, regulation, and operational efficiency issues decisions. The development of systems for accurate wind speed forecasting is a difficult task because it presents linear and nonlinear temporal patterns, and is differently influenced by climate and environmental variables. In this paper, we propose a method to search for the best combination of linear, nonlinear forecasts, and exogenous variables, aiming to increase the accuracy of wind speed forecasting. The proposed method performs: (i) the choice of the linear statistical model to forecast the time series; (ii) the error series modeling using a Long Short-Term Memory model; and (iii) the evolutionary search for the most suitable nonlinear combination of linear and nonlinear forecasts with exogenous variables. The proposed method is assessed using hourly and monthly time series of three stations located in northeast Brazil. The evaluation of the results using two traditional measures shows that the proposed method has a better performance than statistical techniques, Machine Learning models, and state-of-the-art hybrid systems in most series. [ABSTRACT FROM AUTHOR]

• Shockable and non-shockable ECG signals are analyzed. • A review study for shockable ECG signal recognition is discussed. • Unique bispectrum and recurrence plots are proposed for shockable and non-shockable ECG signals. • Performance of nonlinear features for differentiating shockable and non-shockable. • ECG signals is compared. Sudden cardiac death from lethal arrhythmia is a preventable cause of death. Ventricular fibrillation and tachycardia are shockable electrocardiographic (ECG)rhythms that can respond to emergency electrical shock therapy and revert to normal sinus rhythm if diagnosed early upon cardiac arrest with the restoration of adequate cardiac pump function. However, manual inspection of ECG signals is a difficult task in the acute setting. Thus, computer-aided arrhythmia classification (CAAC) systems have been developed to detect shockable ECG rhythm. Traditional machine learning and deep learning methods are now progressively employed to enhance the diagnostic accuracy of CAAC systems. This paper reviews the state-of-the-art machine and deep learning based CAAC expert systems for shockable ECG signal recognition, discussing their strengths, advantages, and drawbacks. Moreover, unique bispectrum and recurrence plots are proposed to represent shockable and non-shockable ECG signals. Deep learning methods are usually more robust and accurate than standard machine learning methods but require big data of good quality for training. We recommend collecting large accessible ECG datasets with a meaningful proportion of abnormal cases for research and development of superior CAAC systems. [ABSTRACT FROM AUTHOR]

Recently, deep learning has attracted increasing attention for soft sensor applications in industrial processes. Hierarchical features can be learned from massive process data by deep learning, which is the key step for quality variable prediction. However, few deep learning algorithms consider the neighborhood structure of data samples for feature extraction in industrial processes. In this paper, a novel stacked neighborhood preserving autoencoder (S-NPAE) is proposed to extract hierarchical neighborhood-preserving features. As for each NPAE, a novel loss function is proposed to reconstruct the input data and preserve the neighborhood structure of the input data simultaneously. By minimizing this loss function, NPAE can efficiently extract the neighborhood-preserved features from its input data. Then, the deep S-NPAE network is constructed by stacking multiple NPAEs in a hierarchical way. Finally, the extracted features can be used for accurate quality prediction in soft sensor modeling. The experimental results on an industrial hydrocracking process demonstrate the effectiveness of the proposed method when compared with other commonly used methods. [ABSTRACT FROM AUTHOR]

Technology has grown rapidly in recent years, and new solutions that rely on Machine Learning (ML) and Artificial Intelligence (AI) are introduced every day. With such fast-paced advancement, inspecting and fully comprehending how given models make decisions is becoming problematic. The complex decision-making process of these models has become a black box, making it challenging to unravel how they work; therefore, eXplainable Artificial Intelligence (XAI) methods are crucial for further development. This paper discusses how state-of-the-art techniques determine classifications and why they need to be revised to understand the prediction-generating process fully. It compares those existing solutions with the new method called Principal Image Sections Mapping - PRISM, which relies on Principal Component Analysis and allows visualising the most significant features recognised by a given Convolutional Neural Network. PRISM is implemented in a piece of software called TorchPRISM that can generate and present the clustering based on the method's output. The result can indicate ambiguous classes discrimination; thus, the possibility of automating the output analysis process is also discussed. The paper's main objective is to examine how PRISM enhances the current understanding of the decision-making process and introduce a tool that can facilitate analysing the output. PRISM implementation (TorchPRISM) can be found in the public GitHub repository: https://github.com/szandala/TorchPRISM • PRISM method can explain CNN's decisions in an understandable way for various users. • PRISM uses PCA to point features and highlights them using an RGB-coloured image mask. • PRISM's effectiveness is assessed based on qualitative and quantitative measures. • TorchPRISM software allows the identification of ambiguous classes in large datasets. • PRISM overcomes the deficiencies of existing eXplainable AI and saliency map methods. [ABSTRACT FROM AUTHOR]

This paper concerns the study of history dependent phenomena in heterogeneous materials in a two-scale setting where the material is specified at a fine microscopic scale of heterogeneities that is much smaller than the coarse macroscopic scale of application. We specifically study a polycrystalline medium where each grain is governed by crystal plasticity while the solid is subjected to macroscopic dynamic loads. The theory of homogenization allows us to solve the macroscale problem directly with a constitutive relation that is defined implicitly by the solution of the microscale problem. However, the homogenization leads to a highly complex history dependence at the macroscale, one that can be quite different from that at the microscale. In this paper, we examine the use of machine-learning, and especially deep neural networks, to harness data generated by repeatedly solving the finer scale model to: (i) gain insights into the history dependence and the macroscopic internal variables that govern the overall response; and (ii) to create a computationally efficient surrogate of its solution operator, that can directly be used at the coarser scale with no further modeling. We do so by introducing a recurrent neural operator (RNO), and show that: (i) the architecture and the learned internal variables can provide insight into the physics of the macroscopic problem; and (ii) that the RNO can provide multiscale, specifically FE 2 , accuracy at a cost comparable to a conventional empirical constitutive relation. [ABSTRACT FROM AUTHOR]

Scholarly venue recommendation is an emerging field due to a rapid surge in the number of scholarly venues concomitant with exponential growth in interdisciplinary research and cross collaboration among researchers. Finding appropriate publication venues is confronted as one of the most challenging aspects in paper publication as a larger proportion of manuscripts face rejection due to a disjunction between the scope of the venue and the field of research pursued by the research article. We present CLAVER--an integrated framework of Convolutional Layer, bi-directional LSTM with an Attention mechanism-based scholarly VEnue Recommender system. The system is the first of its kind to integrate multiple deep learning-based concepts, that requires only the abstract and the title of a manuscript to identify academic venues. An extensive and exhaustive set of experiments conducted on the DBLP dataset certify that the postulated model CLAVER performs better than most of the modern techniques as entrenched by standard metrics such as stability, accuracy, MRR, average venue quality, precision@k, nDCG@k and diversity. [ABSTRACT FROM AUTHOR]

Image caption is an important application field of artificial intelligence technique. When a machine can describe a picture reasonably like a human, it represents that the machine has higher intelligence to understand the picture. However, for complex machine learning tasks such as image caption, data annotation is time-consuming and laborious. Usually in a new application scenario, data annotation rarely results in poor model performance. A large number of easily available unlabeled image data make it possible to semi-supervised learning of image caption methods. Based on the existing end-to-end deep learning paradigm, a semi-supervised deep learning method is proposed in this paper, called N-gram + Pseudo Label NIC method. The method combines the current mainstream deep neural network method, e.g. the NIC (Neural Image Caption) model, and the semi-supervised deep learning idea of pseudo labels, and N-gram. This method generates pseudo labels by N-gram Search algorithm, and improves the effect of the model by utilizing the prior knowledge of the N-gram table and people's descriptive habits. This method has achieved better results than the original NIC model on different sub-data sets of Flickr 8K data set and MSCOCO data set of 0.5k, 1k, 2k and 3k under BLEU-1 evaluation criteria. [ABSTRACT FROM AUTHOR]

This paper introduces a variational analysis approach to the learning of a deep model formed via a nested composition of mappings. The fuzzy sets, being characterized by Gaussian type of membership functions, are used to represent unknown functions associated to the layers of the model. The learning of the deep model would require a quantification of the uncertainties on the signals across the layers of the deep model. We derive analytically the mathematical expressions for membership functions using variational optimization to quantify the uncertainties on variables. The most significant feature of the learning approach is that all of the unobserved variables and parameters, associated to the deep model, are averaged out where the averages are computed taking into account the uncertainties (on variables and parameters). The uncertainties are quantified by means of fuzzy sets with membership functions optimally learned from the observed data. A rigorous mathematical treatment of the learning problem results in the development of a competitive classification algorithm. The study is a theoretical contribution to the field of fuzzy machine learning, nevertheless, offering practical machine learning algorithms. [ABSTRACT FROM AUTHOR]

Security vulnerabilities are among the major concerns of modern software engineering. Successful results of machine learning techniques in various challenging applications have led to an emerging field of research to investigate the effectiveness of machine learning, and more recently, deep learning techniques, for the problem of software vulnerability analysis and discovery. In this paper, we explore the utilization of Graph Neural Networks as the latest trend and progress in the field of artificial neural networks. To this end, we propose an original neural vulnerability analysis approach, using customized intermediate graph representations of programs to train graph neural network models. Experimental results on a public suite of vulnerable programs show that the proposed approach is effective at the task of software vulnerability analysis. Additional empirical experiments answer complementary research questions about the proposed approach. In particular, we present experimental results for the challenging task of cross-project vulnerability analysis, with interesting insights on the capabilities of our novel approach. Furthermore, a software utility that was developed in the course of this study is also published as a further contribution to the research community. [ABSTRACT FROM AUTHOR]