Search

Label Distribution Learning (LDL) is a new learning paradigm to describe supervision as probability distribution and has been successfully applied in many real-world scenarios in recent years. In LDL applications, the availability of a large amount of labeled data guarantees the prediction performance. In this paper, we cogitate the active learning for LDL to reduce the annotation cost. The center element in practice any active learning strategy is building the criterion that measures the usefulness of the unlabeled data and decides the instances to be selected to label manually. We are probably the first to focus on active instance selecting for label distribution learning. We propose a strategy named Active Label Distribution Learning (ALDL) to select the most informative instances for LDL applications. The fundamental idea of the ALDL strategy is to quantify the degree of disagreement for each unlabeled instance by the committee consisted of selected LDL algorithms, and identify the instances to be labeled manually. ALDL maintains composing the committee with selected LDL algorithms and measure the value of unlabeled instances, and a weight vector is used both parts. Besides, we discuss the convergence and the parameter selecting of ALDL. Finally, compared with other active learning methods, the experimental results on the datasets show the effectiveness of our method.

Imbalanced learning in the multi-label scenario is a challenging issue, and it also exists in the training of deep neural network. Previous studies have demonstrated that the resampling methods are capable of reducing bias towards the majority group. Nonetheless, when being extended to neural network, these methods display some obvious drawbacks, such as the introduction of extra hyperparameters, the fixed training mode, etc. In order to eliminate the disadvantages, in this paper, an efficient training technique named Mini-Batch Gradient Descent with Stratified sampling (MBGD-Ss) is proposed to alliviate the issue with imbalanced data by dynamically sampling. In view of the specialty of multi-label domain, we put forward two specific strategies as Label Powerset based (SsLP) and Label-based (SsL), respectively. Particularly, SsLP takes the label combination (labelset) that appears in the dataset as a stratum, and SsL directly sets the label as a stratum. Extensive experiments validate the effectiveness of the proposed approach in decreasing the imbalance of sampled data. Moreover, the empirical analysis also shows that the proposed method can mitigate the classifier’s bias against labels, especially improve the prediction accuracy of minority labels.

This paper addresses the berthing control problem for automatic ships by using a virtual guide system based on heuristic dynamic programming (HDP) method. Firstly, by introducing an automatic virtual guide system, the berthing control problem can be transformed into a tracking control problem, and then can be further transformed into an optimal regulation problem. Secondly, the HDP method is used to solve the optimal regulation control problem of the marine surface ship with unknown ship model. Then, it is proven that the tracking error, the adaptation laws and the control inputs are uniformly bounded on the basis of the Lyapunov theory. Finally, simulations are carried out on an automatic model ship and the HDP method is compared with the backstepping control to verify the effectiveness of the developed control scheme.

Video action anticipation aims to predict future action categories from observed frames. Current state-of-the-art approaches mainly resort to recurrent neural networks to encode history information into hidden states, and predict future actions from the hidden representations. It is well known that the recurrent pipeline is inefficient in capturing long-term information which may limit its performance in predication task. To address this problem, this paper proposes a simple yet efficient Temporal Transformer with Progressive Prediction (TTPP) framework, which repurposes a Transformer-style architecture to aggregate observed features, and then leverages a light-weight network to progressively predict future features and actions. Specifically, predicted features along with predicted probabilities are accumulated into the inputs of subsequent prediction. We evaluate our approach on three action datasets, namely TVSeries, THUMOS-14, and TV-Human-Interaction. Additionally we also conduct a comprehensive study for several popular aggregation and prediction strategies. Extensive results show that TTPP not only outperforms the state-of-the-art methods but also more efficient.

Traditional classification algorithms suppose that the sample distribution among classes is balanced. Yet, such an assumption leads to biased performance over the majority class. This paper proposes a Relevant Information-based UnderSampling (RIUS) approach to select the most relevant examples from the majority class to improve the classification performance for imbalanced data scenarios. RIUS builds on the information-preservation principle that extracts the majority class’s underlying structure with fewer samples. Additionally, we couple our RIUS approach to the well-known Clustering-based Undersampling algorithm (CBUS) to enhance the data representation, and named this RIUS enhancement as CRIUS. Experimental results show that RIUS and CRIUS reveal the data’s relevant structure and reduce the loss of information by selecting the most informative instances.

Generative adversarial networks (GAN) has received great attention and made great progress since its emergence in 2014. In this paper, we focus on the theoretical achievements of GAN and discuss them in detail for readers who wish to know more about GAN. Based on the number of the implemented network architectures, we category the improved methods into two groups: GAN variants, which are composed of two networks, to improve the performance by adding some regularization to the loss function; hybrid GANs, which are usually combined with other generative models to improve the training stability. For GAN variants, we discuss the theoretical results of the distribution divergence, training dynamics and various improved methods. For hybrid GANs, we introduce the improved methods of combining encoder, autoencoder or VAE. We also cover some other important issues, such as the quantify metrics of generated samples and the basic construction structure. In addition, we discuss the advantages of the GAN over other deep generative models, the future directions worthy of study, as well as the open issues that the community should further address.

In this paper, we propose a novel Attentive Multi-View Deep Subspace Nets (AMVDSN), which deeply explores underlying consistent and view-specific information from multiple views and fuse them by considering each view’s dynamic contribution obtained by attention mechanism. Unlike most multi-view subspace learning methods that they directly reconstruct data points on raw data or only consider consistency or complementarity when learning representation in deep or shallow space, our proposed method seeks to find a joint latent representation that explicitly considers both consensus and view-specific information among multiple views, and then performs subspace clustering on learned joint latent representation. Besides, different views contribute differently to representation learning, we therefore introduce attention mechanism to derive dynamic weight for each view, which performs much better than previous fusion methods in the field of multi-view subspace clustering. The proposed algorithm is intuitive and can be easily optimized just by using Stochastic Gradient Descent (SGD) because of the neural network framework, which also provides strong non-linear characterization capability compared with traditional subspace clustering approaches. The experimental results on seven real-world data sets have demonstrated the effectiveness of our proposed algorithm against some state-of-the-art subspace learning approaches.

This work presents a study on fault diagnosis in vibration signal processing. Rather than building a fault model through frequently used approaches to handling the series data such as LSTM or hidden Markov field, this work processes the vibration signal by moving the time window to generate multiple samples and then transfers fault diagnosis into a traditional supervised learning problem. Stochastic configuration neural network (SCN) which gives a clear condition of guaranteeing high performance of randomly weighted neural networks is selected as the model for training and testing. Different classifiers are used to conduct a performance comparison, and their comparative advantages including why SCN particularly suitable for this type of learning and more discussions about the experimental results are shown. The paper provides a new scheme to processing vibration signal for fault diagnosis and some useful guidelines of building an appropriate model with high performance.

Knowledge distillation (KD) is one of the most popular ways for model compression. The key idea is to transfer the knowledge from a deep teacher model (T) to a shallower student (S). However, existing methods suffer from performance degradation due to the substantial gap between the learning capacities of S and T. To remedy this problem, this paper proposes Residual error based Knowledge Distillation (RKD), which further distills the knowledge by introducing an assistant model(A). Specifically, S is trained to mimic the feature maps of T, and A aids this process by learning the residual error between them. In this way, S and A complement with each other to get better knowledge from T. Furthermore, we devise an effective method to derive S and A from a given model without increasing the total computational cost. Extensive experiments show that our approach achieves appealing results on popular classification datasets, CIFAR-100 and ImageNet, surpassing state-of-the-art methods and keep strong robustness to adversarial samples.

Smoke detection based on video monitoring is of great importance for early fire warning. However, most of the smoke detection methods based on neural network only consider the normal weather. The harsh weather such as the fog environment is ignored. In this paper, we propose a smoke detection in normal and fog weather, which combines attention mechanism and feature-level and decision-level fusion module. First, a new fog smoke dataset with diverse positive and hard negative samples dataset is established through online collection and offline shooting. Then, an attention mechanism module combining spatial attention and channel attention is proposed to solve the problem of small smoke detection. Next, a lightweight feature-level and decision-level fusion module is proposed, which can not only improve the discrimination of smoke, fog and other similar objects, but also ensure the real-time performance of the model. Finally, a large number of comparative experiments on the existing dataset and our self-created dataset, show that our method can obtain higher detection accuracy rate, precision rate, recall rate, and F1 score from the perspective of overall, each category, small smoke and hard negative samples detection than the existing methods.

Deep convolutional networks (CNNs) are able to learn robust representations and empower many computer vision tasks such as object recognition. However, when applying CNNs to industrial visual systems, they usually suffer from domain shift that exists between the training data and testing data. Such shift can be caused by different environment, types of cameras and exteriors of objects, leading to degrading performance and hindering the practical applications of CNNs in real-world visual recognition. To tackle this problem, Adversarial domain adaptation (ADA) reduces such shift by min–max optimization. However, current CNNs with ADA are hard to train due to training instability of adversarial network. In this paper, we propose a unified and easy-to-train domain adaptation framework, namely Attention-based Domain-confused Adversarial Domain ADaptation (AD3). Our method leverages both adversarial and statistical domain alignment, allows flexibility for source and target feature extractors and simultaneously performs feature-level and attention-level alignment. The statistical domain alignment promotes and stabilizes the adversarial domain learning, which reduces the manual work of tuning the hyper-parameters. The experimental results validate that our method performs better adaptation and faster convergence for adversarial domain learning than existing state-of-the-art methods on DIGITS, Office-31 and VisDA domain adaptation benchmarks.

Judicial Decision Prediction (JDP) aims to determine judicial decisions per the fact description of a criminal case. It comprises multiple subtasks, i.e., law article prediction, charge prediction, and term of the penalty prediction. Besides, there exist three properties among the subtasks, i.e., Commonalities , Specificities , and Dependencies. Nonetheless, existing approaches are usually well-designed for only a specific subtask, or take one of the properties into consideration for multiple subtasks. In this paper, we propose a novel Commonalities-, Specificities- and Dependencies-Enhanced Multi-Task Learning Network, to unify multiple subtasks accompanied by the properties in a framework. Further, while handling the Dependencies, we elaborate a learning module to ensure each subtask to learn contributions from other subtasks to varying degrees, a denoising module to minimize noise interferences among subtasks, and a reinforcing module to guarantee further enhancement for each subtask. Experimental results on two widely used datasets demonstrate that our model significantly and consistently outperforms previous state-of-the-art methods on most evaluation metrics across all subtasks.

With the development of e-commerce platforms, user reviews have become a vital source of information to address the sparsity problems for enhancing the predictive performance of the recommendation systems (RSs). However, the traditional methods of the RSs used to model user/item latent features based on static vectors in an independent manner without considering the dynamic nature of the user-item interactions which potentially affect the accuracy of the recommendation process. Thus, this paper proposes a RS model that exploits neural attention techniques to learn user/item representations by jointly considering the fine-grained semantic information for the user-item pairs. The proposed model utilizes both review-based and interaction-specific features for the user/item reviews to learn heterogeneous user/item representations. First, a BiLSTM sequence encoder is used to learn the contextual information of words, and a Co-attention network is then designed to jointly capture the most relevant semantic information of reviews for the user-item pair. To better capture user/item latent factors comprehensively, interaction-specific features based on the rating scores are further integrated with the review-specific latent features via a shared hidden layer. Finally, an attentive factorization machine (FM) is then applied on the shared hidden layer of the integrated user/item features for the final prediction. We carry out a series of experiments using real-world datasets and the results demonstrate that our proposed method is better than the baseline approaches in terms of both rating prediction and ranking performance.

Three-dimensional object reconstruction technology has a wide range of applications such as augment reality, virtual reality, industrial manufacturing and intelligent robotics. Although deep learning-based 3D object reconstruction technology has developed rapidly in recent years, there remain important problems to be solved. One of them is that the resolution of reconstructed 3D models is hard to improve because of the limitation of memory and computational efficiency when deployed on resource-limited devices. In this paper, we propose 3D-RVP to reconstruct a complete and accurate 3D geometry from a single depth view, where R, V and P represent Reconstruction, Voxel and Point, respectively. It is a novel two-stage method that combines a 3D encoder-decoder network with a point prediction network. In the first stage, we propose a 3D encoder-decoder network with residual learning to output coarse prediction results. In the second stage, we propose an iterative subdivision algorithm to predict the labels of adaptively selected points. The proposed method can output high-resolution 3D models by increasing a small number of parameters. Experiments are conducted on widely used benchmarks of a ShapeNet dataset in which four categories of models are selected to test the performance of neural networks. Experimental results show that our proposed method outperforms the state-of-the-arts, and achieves about 2.7 % improvement in terms of the intersection-over-union metric.

Generalized zero-shot learning suffers from an extreme data imbalance problem, that is, the training data only come from seen classes while no unseen class data are available. Recently, a number of feature generation methods based on generative adversarial networks (GAN) have been proposed to address this problem. Existing feature generation methods, however, have never considered the under-constrained problem, and thus could generate an unrestricted visual feature corresponding to no meaningful object class. In this paper, we propose to equip the feature generation framework with a parallel inference network that projects visual feature to the semantic descriptor space, constraining to avoid the generation of unrestricted visual features. The two-parallel-stream framework (1) enables our method, termed inference guided feature generation (Inf-FG), to mitigate the under-constrained problem and (2) makes our Inf-FG applicable to transductive ZSL. Our Inf-FG learns the feature generator and the inference network simultaneously by aligning the joint distribution of visual features and semantic descriptors from the feature generator and the joint distribution from the inference network. We evaluate our approach on four benchmark ZSL datasets, including AWA, CUB, SUN, and FLO, on which our method improves our baselines on generalized zero-shot learning.

Cross-domain recommendation is becoming increasingly popular recently. Existing cross-domain recommendation often assumes that, a sufficient set of bridged users across domains is given in advance which disregards the scenario with insufficient bridged users. In this paper, we propose a novel Trust-aware Latent Space Mapping approach for Cross-domain Recommendation, called TLSM-CDR. This represents one of the first attempts to address the challenge of insufficient bridged users from the perspective of users’ trust relationships to facilitate user sharing cross-domain recommendation. First, our model employs the Probabilistic Matrix Factorization (PMF) to generate user and item matrices. Then, Deep Neural Network (DNN) and graph Laplacian are seamlessly incorporated into our trust-aware non-linear mapping function to capture the latent space relationships between both bridged and non-bridged users. Finally, we predict the optimized users’ rating matrix in the target domain. Extensive experiments conducted on two real-world datasets demonstrate that, our TLSM-CDR model significantly outperforms several state-of-the-art methods.

With the popularity of social networks, sentiment analysis has become one of the hottest topics in natural language processing (NLP). As the development of research on the fine-grained sentiment analysis, more and more researchers pay attention to aspect-level sentiment analysis. It aims to identify the same or different sentiment polarity in different aspects of the context. In this paper, a context and aspect memory network (CAMN) method is proposed to solve the problem of aspect level sentiment analysis. In this method, deep memory network, bi-directional long short-term memory network and multi-attention mechanism are introduced to better capture the sentiment features in short texts. It includes two strategies: one is to use the self-attention mechanism (i.e., CAMN-SA) to calculate the context relevance; the other is to use the encoder-decoder attention mechanism (i.e., CAMN-ED) to calculate the context and aspect relevance. In order to verify the function of each component in the proposed method, and to test the effect of different hops on the memory network, we conduct many experiments on three real-world datasets to compare the baseline models with our proposed method. Experimental results show that our proposed method can achieve better performance than the baseline models.

Generative Adversarial Network (GAN) is a thriving generative model and considerable efforts have been made to enhance the generation capabilities via designing a different adversarial framework of GAN (e.g., the discriminator and the generator) or redesigning the penalty function. Although existing models have been demonstrated to be very effective, their generation capabilities have limitations. Existing GAN variants either result in identical generated instances or generate simulation data with low quality when the training data are diverse and extremely limited (a dataset consists of a set of classes but each class holds several or even one single sample) or extremely imbalanced (a category holds a set of samples and other categories hold one single sample). In this paper, we present an innovative approach to tackle this issue, which jointly employs joint distribution and reparameterization method to reparameterize the randomized space as a mixture model and learn the parameters of this mixture model along with that of GAN. In this way, we term our approach Joint Distribution GAN (JDGAN). In our work, we show that the JDGAN can not only generate high quality simulation data with diversity, but also increase the overlapping area between the generating distribution and the raw data distribution. We proceed to conduct extensive experiments, utilizing MNIST, CIFAR10 and Mass Spectrometry datasets, all using extremely limited amounts of data, to demonstrate the significant performance of JDGAN in both achieving the smallest Frechet Inception Distance (FID) score and producing diverse generated data.

Scene understanding comes in many flavors, two of the most popular being object detection and semantic segmentation, which act as two important aspects for scene understanding, and are applied to many areas, such as autonomous driving and intelligent surveillance. Although much progress has already been made, the two tasks of object detection and semantic segmentation are often investigated independently. In practice, scene understanding is complicated, and comprises many sub-tasks, so that research of learning multiple tasks simultaneously with a single model is feasible. With the interrelated goals of these two tasks, there is a strong motivation to improve the object detection accuracy with the help of semantic segmentation, and vice versa. In this paper, we propose a loss-balanced multi-task model for simultaneous object detection and semantic segmentation. We explore multi-task learning with sharing parameters based on deep learning to realize improved object detection and segmentation, and propose a single-stage deep architecture based on multi-task learning, jointly performing object detection and semantic segmentation to boost each other. With no more computation load in the inference compared with the baselines of SSD and FCN, we show that these two tasks, object detection and semantic segmentation, benefit from each other. Experimental results on Pascal VOC and COCO show that our method improves much in object detection and semantic segmentation compared with the corresponding baselines of both tasks.

High voltage direct current (HVDC) transmission mode with modular multilevel converters (MMC) topology is the future direction of transmission engineering, and security is their fundamental issue. Submodule fault of MMC in HVDC is the most common problem, nevertheless, traditional time–frequency based diagnosis technology can’t achieve high accuracy. To solve this pain spot, a new diagnosis strategy based on the synchrosqueezing transform (SST) and genetic algorithm optimized deep convolution neural network (GA-DCNN) is proposed in this paper. Firstly, the time–frequency representations (TFRs) of the raw signals which is synthesized by ac current and inner circulating current of the MMC are calculated with SST. Then, DCNN is introduced to learn the underlying features from the TFRs, and its key hyperparameters are optimized with genetic algorithm. Meanwhile, batch normalization, dropout and data augment technologies are explored to prevent DCNN model from overfitting and improve model performance. Compared to traditional SVM and BP-based algorithms, SST-GA-DCNN achieve high diagnosis accuracy. The experimental results show the feasibility and applicability of the proposed fault diagnosis framework.

Accurate traffic prediction plays an important role in Intelligent Transportation System. This problem is very challenging due to the heterogeneity and dynamic spatio-temporal dependence of large-scale traffic data. Existing models often suffer two limitations: (1) They usually only consider one type of data in the input, or simply treat other collected time series data as features, ignoring the non-linear interactions among different series. In fact, heterogeneous data at a specific location has direct impacts on the predicted series. (2) The method based on graph convolutional network uses a fixed Laplacian matrix to model spatial correlation, without considering its dynamics. The aggregations also occur only in the neighborhood, making it difficult to capture long-range dependencies. In this paper, we propose a M ulti-Stage A ttention S patial- T emporal G raph N etworks (MASTGN). First, an internal attention mechanism is designed to capture the interactions among multiple time series collected by the same sensor. Second, to model the complex spatial correlations, we apply a dynamic neighborhood-based attention mechanism. Unlike the general attention-based methods that ignore the structure information of the road network, we use the adjacency relations as a prior to divide the nodes of a road network into different neighborhood sets. In this way, attention can capture spatial correlations both within the same order neighborhood, and among different neighborhoods dynamically. Furthermore, a temporal attention mechanism is used to extract the dynamic temporal dependencies. Experiments are conducted on two real traffic datasets, and the results verify the effectiveness of the proposed model.

Large-scale trajectory dataset is always required for self-driving and many other applications. In this paper, we focus on the trajectory generation problem, which aims to generate qualified trajectory dataset that is indistinguishable from real trajectories, for fulfilling the needs of large-scale trajectory data by self-driving simulation and traffic analysis tasks in data sparse cities or regions. We propose two advanced solutions, namely TrajGAN and TrajVAE, which utilize LSTM to model the characteristics of trajectories first, and then take advantage of Generative Adversarial Network (GAN) and Variational AutoEncoder (VAE) frameworks respectively to generate trajectories. In order of compare the similarity of existing trajectories in our dataset and the generated trajectories, we utilize multiple trajectory similarity metrics. Through several experiments, we demonstrate that our method is more accurate and stable than the baseline.

Metaphor identification helps improve the performance of various natural language understanding tasks such as word sense disambiguation and sentiment analysis. Though many efforts have been made to deal with metaphor identification, most existing studies largely overlook a fact of contextual inconsistency of the metaphors in natural language. We observe that the greater the semantic inconsistency between current word and contextual words is, the more likely the word belongs to the metaphorical category. In this paper, we formulate the metaphor identification as a sequential tagging problem, and then develop a novel contextual inconsistency based neural sequence labeling approach, which can leverage the semantic contextual inconsistency among words of a sentence to address the problem. We propose to rely on distance metric to measure the contextual inconsistency, and evaluate four widely used distance functions in experiments. Experimental results on publicly available datasets validate the benefit of the proposed model over state-of-the-art baselines for metaphor identification.

Bankruptcy prediction is a crucial application in financial fields to aid in accurate decision making for business enterprises. Many models may stagnate to low-accuracy results due to the uninformed choice of parameters. This paper presents a forward-thinking bankruptcy prediction model based on kernel extreme learning machine (KELM), which proposes a new efficient version of a fruit fly optimization (FOA) algorithm called LSEOFOA, to evolve and harmonize the penalty and the kernel parameter in KELM. The upgraded version of FOA is conceptualized based on three reorganizations. The first attempt is to include Levy's flight for improving exploration inclinations, and the second is based on slime mould algorithm (SMA) for avoiding premature convergence and enhancing the stability of the exploration and exploitation patterns. As the last modification, we utilized the elite opposition-based learning for accelerating the convergence. The algorithmic trends of this optimizer are verified, and then, it is verified on a bankruptcy prediction module. Therefore, to further demonstrate the superiority of the LSEOFOA method, comparison studies are performed using the conventional FOA and other variants of FOA and a set of advanced algorithms including EBOwithCMAR. Experimental results for every optimization task demonstrate that LSEOFOA can provide a high-performance and self-assured tradeoff between exploration and exploitation. Also, the developed KELM classifier is utilized for bankruptcy prediction, and its optimal parameters set are revealed by the proposed FOA. The effectiveness of the LSEOFOA-KELM model is rigorously evaluated using a financial dataset and comparison with KELM-based models with other competitive optimizers such as LSHADE-RSP. Overall research findings show that the proposed model is superior in terms of classification accuracy, Matthews correlation coefficient, sensitivity, and specificity. Towards more evolutionary and efficient prediction models, the proposed LSEOFOA-KELM prediction model can be regarded as a promising warning tool for financial decision making, with successful performance in bankruptcy prediction. Interested readers to the idea and related material of LSEOFOA-KELM can find the designed public web service at https://aliasgharheidari.com. Also, the info and source codes of the slime mould algorithm (SMA) in python, matlab and other languages are shared publicly at https://aliasgharheidari.com/SMA.html.

Point-of-interest (POI) recommendation is an important technique widely used in self-driving services. While POI recommendation aims to recommend unvisited POIs to self-driving users, users always expect their intended items can be suggested together with these POIs, e.g. what activities to perform at the recommended places. However, existing methods cannot well support such POI recommendation in a finer granularity. In this paper, we investigate this new problem and propose a novel POI-based item recommendation model via graph embedding. The model accurately captures the joint effect of geographical and temporal influences on both POI-level and item-level recommendation in a shared space, which can address data sparsity and cold start problems effectively. To optimize the model efficiently and accurately, a novel weighted negative sampling strategy is designed. Besides, we propose a novel fine-grained user dynamic preference modeling method, which can accurately capture dynamic user preferences in a finer granularity based on the embeddings of both POIs and items. Comprehensive experimental studies have been conducted on three datasets. Results show that our model achieves significant improvement over state-of-the-art baselines.

Multimodal sentiment analysis is one of the most attractive interdisciplinary research topics in artificial intelligence (AI). Different from other classification issues, multimodal sentiment analysis of human is a much finer classification problem. However, most current work accept all multimodalities as the input together and then output final results at one time after fusion and decision processes. Rare models try to divide their models into more than one fusion modules with different fusion strategies for better adaption of different tasks. Additionally, most recent multimodal sentiment analysis methods pay great focuses on binary classification, but the accuracy of multi-classification still remains difficult to improve. Inspired by the emotional processing procedure in cognitive science, both binary and multi-classification abilities are improved in our method by dividing the complicated problem into smaller issues which are easier to be handled. In this paper, we propose a Hierarchal Attention-BiLSTM (Bidirectional Long-Short Term Memory) model based on Cognitive Brain limbic system (HALCB). HALCB splits the multimodal sentiment analysis into two modules responsible for two tasks, the binary classification and the multi-classification. The former module divides the input items into two categories by recognizing their polarity and then sends them to the latter module separately. In this module, Hash algorithm is utilized to improve the retrieve accuracy and speed. Correspondingly, the latter module contains a positive sub-net dedicated for positive inputs and a negative sub-nets dedicated for negative inputs. Each of these binary module and two sub-nets in multi-classification module possesses different fusion strategy and decision layer for matching its respective function. We also add a random forest at the final link to collect outputs from all modules and fuse them at the decision-level at last. Experiments are conducted on three datasets and compare the results with baselines on both binary classification and multi-classification tasks. Our experimental results surpass the state-of-the-art multimodal sentiment analysis methods on both binary and multi-classification by a big margin.

Training machine learning models requires large amounts of data, which may contain personal sensitive information. Machine learning based on privacy protection has become a research hotspot. In this paper, differential privacy is applied to the ensemble learning, a branch of machine learning, to prevent privacy leakage in the classification process. We propose a differentially private ensemble learning algorithm for classification, which achieves privacy protection while ensures prediction accuracy. Firstly, we adopt the Bag of Little Bootstrap technique and the Jaccard similarity coefficient to generate a set of training data sets, and construct corresponding differentially private base classifiers by adding a carefully chosen amount of perturbation noise with a privacy budget allocation strategy. Furthermore, to reduce the impact of perturbation noise on the accuracy of prediction, an effective ensemble algorithm is proposed. Specifically, the base classifiers are selected based on some criterion functions, and the corresponding weights are assigned simultaneously. Then, the final result of the classification is obtained by a weighted voting scheme. Experiments are executed on 9 real data sets from the UCI Machine Learning Repository to demonstrate that our differentially private ensemble classification algorithm achieves a better trade-off in terms of privacy protection and prediction accuracy.

BayeSuites is the first web framework for learning, visualizing, and interpreting Bayesian networks (BNs) that can scale to tens of thousands of nodes while providing fast and friendly user experience. All the necessary features that enable this are reviewed in this paper; these features include scalability, extensibility, interoperability, ease of use, and interpretability. Scalability is the key factor in learning and processing massive networks within reasonable time; for a maintainable software open to new functionalities, extensibility and interoperability are necessary. Ease of use and interpretability are fundamental aspects of model interpretation, fairly similar to the case of the recent explainable artificial intelligence trend. We present the capabilities of our proposed framework by highlighting a real example of a BN learned from genomic data obtained from Allen Institute for Brain Science. The extensibility properties of the software are also demonstrated with the help of our BN-based probabilistic clustering implementation, together with another genomic-data example.

Dialogue-based multiple-choice machine reading comprehension (MRC) is one of most difficult and novel tasks because it requires more advanced reading comprehension skills, such as speaker’s intention analysis, non-extractive reasoning, commonsense knowledge. Previous models usually only compute attention scores from the fixed representation of entire dialogue, and also do not fully consider the contribution of dialogue, question, options, and their combinations respectively. In this paper, we introduce Utterance-focusing Multiway-matching Network (UMN), a simple but effective human mimicking model for dialogue-based multiple-choice MRC. First, two utterance-focusing mechanisms called ParaUF and AutoUF are proposed to extract the utterances that are most relevant to the question and option: ParaUF gets the bilinear weighted distance between each utterance of dialogue and question and option during training while AutoUF obtains the scores by the relevance, overlap and coverage (ROC) rules before training process. Second, we adopted the multiway-matching mechanism to capture the relationship among the question, option and selected utterances through calculating the attention weights between the quadruplet of four sequences: utterances, question, option and the concatenation of each two. We evaluate the proposed model on dialogue-based multiple-choice MRC tasks, DREAM, and outperformed recently published methods under the same pre-trained model. A series of detailed analysis is also conducted to interpret the differences of two utterance-focusing mechanisms and the effectiveness of the proposed multiway-matching mechanism.

Instance search has been conventionally addressed as an image retrieval issue. In the existing solutions, traditional hand-crafted features and global deep features have been widely adopted. Unfortunately, since the features are not directly derived from the exact area of an instance in an image, satisfactory performance from most of them is undesirable. In this paper, a compact instance level feature representation is proposed. The scheme basically consists of two convolutional neural network (CNN) pipelines. One is designed for localizing potential instances from an image, while another is trained to learn object-aware weights to produce distinctive features. The sensitivity to the unknown categories, the distinctiveness to different instances, and most importantly, the capability of localizing an instance in an image are all carefully considered in the feature design. Moreover, both pipelines only require image level annotations, which makes the framework feasible for large-scale image collections with variety of instances. To the best of our knowledge, this is the first piece of work that builds the instance level representation based on weakly supervised object detection.

Data driven fault diagnosis has attracted a lot of attention in recent years owing to its intelligent and accurate detection of fault categories. However, it is a challenge for its applications in real world. The abundant labeled data is extremely necessary for data driven fault diagnosis to train a favorable model. Even though enough labeled data is prepared for training a model, we still cannot ensure the data used for training and testing draw from identical distribution. In other words, the labeled source domain has different distribution compared with the unlabeled target domain. In this paper, we introduce the domain adaptation strategy into deep neural networks to propose a deep domain adaptation architecture, which realizes to learn knowledge from the labeled source domain to facilitate the target classification. In the proposed model, the conditional and marginal distribution are adapted together in multiple layers of neural network, which uses MMD to measure the distribution discrepancy. Besides, the relative importance between marginal and conditional distributions is explored, and an adaptively weighted strategy is further introduced to learn the relative importance of the two distributions. To evaluate the proposed method, we conduct the simulations on different workloads, sensor deployment locations, and even different platforms. The results show the superiority of the proposed model to other intelligent fault diagnosis methods, meanwhile verify the necessity of marginal and conditional distribution adaptation and adaptive weighted strategy.

For the operational status detection of a pipeline network, it is essential to obtain enough samples of each class during the network operation. However, the phenomenon of few actual leak samples give rise to the imbalanced dataset problem. To address this issue, this paper proposes a minor class-based status detection method using enhanced generative adversarial networks (enhanced-GANs). First, a generative model with a U-net structure is established to generate the required samples with the modified normal samples, and the L1 loss and L2 loss functions are utilized to update the network parameters. Then, output results and extracted features regarding different layers of discriminative network are added in the generative network loss function to improve the quality of the generated samples. Furthermore, based on the hidden features extracted by the trained discriminative network, an enhanced dual judgment scheme is proposed to improve the status detection performance. Finally, extensive experiments are carried out to evaluate the proposed method with the dataset collected from a practical pipeline network. The experiment results show that the proposed method can not only provide enough leak samples but also effectively improves the status detection accuracy for a pipeline network.

Personalized recommendation mostly employs users’ historical data to improve their user profiles, and these profiles are then used as the bases for recommendations. Because reviews can contain a large amount of information regarding user preferences and item features, they can be naturally into recommender systems (RSs) as contextual information, thus solving the problem of data sparsity and helping to provide personalized recommendations. The existing technology mainly extracts latent representations of users or items in an independent and static manner. We argue that static embedding cannot fully capture a user's preferences. Indeed, a user will have different preferences corresponding to different items. This type of review-based recommendation model cannot provide a personalized, and complete semantic explanation of a candidate recommendation item to a user. In this paper, we introduce an attention mechanism to explore the importance of specific sentences in reviews for different users and propose a novel attentive preference personalized recommendation with sentence-level explanations (APSE). The APSE employs the latent features of users and items and the latent factors of their pairwise interactions to obtain review representations. Then, the APSE uses probability matrix factorization to model additional high-level feature interactions based on these user-item pairs for rating prediction. We implement review feature learning in the APSE to exploit review data in which an attentive mechanism is used to highlight the influences of words and sentences to achieve focused paragraph embedding. Finally, the APSE also employs an explanation sentence judgment mechanism that implements the user-item pair interaction method to extract comments or statements that pertain to user preferences as recommendation interpretations. Experiments are performed on real-world datasets for validation. Additionally, we show the important words and sentences highlighted by the attentive mechanism. At the end of the experiment, a specific item explanation for a user is produced and compared with the user's existing comments. The results show that the performance of the APSE can exceed that of various recommended models when the available ratings are limited.

Text style transfer is an important task in the field of natural language generation. Because of the lack of parallel data, it is a challenge to address this problem in an unsupervised manner. Existing methods mainly focus on the two-style transfer task, i.e. from one source style to one target style. In this paper, we first propose the task of unsupervised text multi-style transfer to address the problem of efficient text transfer from a source style to multiple target styles. To tackle this new task, we present a novel model based on Non-Autoregressive Transformer (NAT). The model consists of two parts: a parameter-shared style-independent module and a style-dependent module. In practice, we only need to reinitialize the parameter of style-dependent modules and retrain the whole model which can converge fast. Experimental results show that our model not only performs well in two-style transfer task, but also promises good results in the multi-style scenario.

The excellent performance of deep neural networks mainly relies on the attributes of the dataset, such as the size, diversity, completeness. However, it is usually difficult to obtain a high-qualified training dataset for some scenarios. Inspired by the human visual cognition process with few sample learning, and strong robustness, we believe the experience knowledge is more powerful than large-scale data. To combine the power of knowledge and data, we propose a knowledge-guided semantic computing network (SCN) in this paper, which is constructed with a primary knowledge-guided semantic tree module and an auxiliary data-driven lightweight neural network module. The semantic tree module can calculate the classification results by a forward computing process rapidly. The lightweight neural network module can aid the semantic tree module for higher classification ability. We also propose a hinge cross-entropy loss function to train the SCN, which enables the SCN to focus on those misclassified training samples and further improve the classification accuracy. The experimental results on MNIST and GTSRB data sets prove that the SCN achieves excellent classification accuracy comparable to the state-of-the-art methods on the original training samples and higher classification accuracy than the state-of-the-art methods on few training samples. What is more, at BIM eps = 0.3 on MNIST and FGSM eps = 0.03 on GTSRB adversarial test samples, the proposed SCN(1/4) and SCN(1/8) obtain over 75% and 14% accuracy improvement than the original CapsNet.

Accurate and real-time traffic flow prediction plays an important role in improving the traffic planning capability of intelligent traffic systems. However, traffic flow prediction is a very challenging problem because the spatial–temporal correlation among roads is complex and changeable. Most of the existing methods do not reasonably analyze the dynamic spatial–temporal correlation caused by the changing relationship of traffic patterns among roads, thus cannot get satisfactory results in the medium and long-term traffic prediction. To address these issues, a novel M ultisensor D ata C orrelation G raph C onvolution N etwork model, named MDCGCN, is proposed in this paper. The MDCGCN model consists of three parts: recent, daily period and weekly period components, and each of which consists of two parts: 1) benchmark adaptive mechanism and 2) multisensor data correlation convolution block. The first part can eliminate the differences among the periodic data and effectively improve the quality of data input. The second part can effectively capture the dynamic temporal and spatial correlation caused by the changing relationship of traffic patterns among roads. Through substantial experiments conducted on two real data sets, results indicate that the proposed MDCGCN model can significantly improve the medium and long-term prediction accuracy for traffic networks of different sizes, and is superior to existing prediction methods.

Nowadays, there are many models/algorithms designed for redundant manipulator control. Nevertheless, two-manipulator system is usually more efficient than single manipulator. However, there are a few studies investigated for the two-manipulator system control. Specifically, the posture control of the two-manipulator system is investigated in this paper. By analyzing the kinematics and considering the control task, two posture control schemes are proposed for two manipulators, respectively. Then, a posture coordination control scheme is proposed in the form of standard quadratic programming (QP). The projection neural network (PNN) model is further developed to handle the posture coordination control scheme. In addition, a one-iteration discrete PNN (DPNN) model is proposed for convenient numerical algorithm development and digital hardware implementation. Finally, sufficient experiments based on the two-manipulator system verify the effectiveness and superiority of the posture coordination control scheme, PNN model and one-iteration DPNN model.

Causality extraction from natural language texts is a challenging open problem in artificial intelligence. Existing methods utilize patterns, constraints, and machine learning techniques to extract causality, heavily depending on domain knowledge and requiring considerable human effort and time for feature engineering. In this paper, we formulate causality extraction as a sequence labeling problem based on a novel causality tagging scheme. On this basis, we propose a neural causality extractor with the BiLSTM-CRF model as the backbone, named SCITE (Self-attentive BiLSTM-CRF wIth Transferred Embeddings), which can directly extract cause and effect without extracting candidate causal pairs and identifying their relations separately. To address the problem of data insufficiency, we transfer contextual string embeddings, also known as Flair embeddings, which are trained on a large corpus in our task. In addition, to improve the performance of causality extraction, we introduce a multihead self-attention mechanism into SCITE to learn the dependencies between causal words. We evaluate our method on a public dataset, and experimental results demonstrate that our method achieves significant and consistent improvement compared to baselines., Comment: 39 pages, 11 figures, 6 tables

Linguistic knowledge is useful for various NLP tasks, but the difficulty lies in the representation and application. We consider that linguistic knowledge is implied in a large-scale corpus, while classification knowledge, the knowledge related to the definitions of entity and relation types, is implied in the labeled training data. Therefore, a corpus subgraph is proposed to mine more linguistic knowledge from the easily accessible unlabeled data, and sentence subgraphs are used to acquire classification knowledge. They jointly constitute a relation knowledge graph (RKG) to extract relations from sentences in this paper. On RKG, entity recognition can be regarded as a property value filling problem and relation classification can be regarded as a link prediction problem. Thus, the multiple relation extraction can be treated as a reasoning process for knowledge completion. We combine statistical reasoning and neural network reasoning to segment sentences into entity chunks and non-entity chunks, then propose a novel Chunk Graph LSTM network to learn the representations of entity chunks and infer the relations among them. The experiments on two standard datasets demonstrate our model outperforms the previous models for multiple relation extraction.

Deep learning techniques have become significantly prevalent in many real-world problems including a variety of detection, recognition, and classification tasks. To obtain high-performance neural networks, an enormous amount of training datasets, memory, and time-consuming computations are required which has increased the demands for outsource training among users. As a result, the machine-learning-as-a-service(MLaaS) providers or a third party can gain an opportunity to put the model’s security at risk by training the model with malicious inputs. The malicious functionality inserted into the neural network by the adversary will be activated in the presence of specific inputs. These kinds of attacks to neural networks, called trojan or backdoor attacks, are very stealthy and hard to detect because they do not affect the network performance on clean datasets. In this paper, we refer to two important threat models and we focus on the detection and mitigation techniques against these types of attacks on neural networks which has been proposed recently. We summarize, discuss, and compare the defense methods and their corresponding results.

Weakly supervised semantic segmentation has been attracting increasing attention as it can alleviate the need for expensive pixel-level annotations through the use of image-level labels. Relevant methods mainly rely on the implicit object localization ability of convolutional neural networks (CNNs). However, generated object attention maps remain mostly small and incomplete. In this paper, we propose an Atrous Convolutional Feature Network (ACFN) to generate dense object attention maps. This is achieved by enhancing the context representation of image classification CNNs. More specifically, cascaded atrous convolutions are used in the middle layers to retain sufficient spatial details, and pyramidal atrous convolutions are used in the last convolutional layers to provide multi-scale context information for the extraction of object attention maps. Moreover, we propose an attentive fusion strategy to adaptively fuse the multi-scale features. Our method shows improvements over existing methods on both the PASCAL VOC 2012 and MS COCO datasets, achieving state-of-the-art performance.

Traffic accidents usually lead to severe human casualties and huge economic losses in real-world scenarios. Timely accurate prediction of traffic accidents has great potential to protect public safety and reduce economic losses. However, it is challenging to predict traffic accidents due to the complex causality of traffic accidents with multiple factors, including spatial correlations, temporal dynamic interactions and external influences in traffic-relevant heterogeneous data. To overcome the above issues, this paper proposes a novel Deep Spatio-Temporal Graph Convolutional Network, namely DSTGCN, to predict traffic accidents. The proposed model is composed of three components: the first component is the spatial learning layer which performs graph convolutional operations on spatial information to learn the correlations in space. The second component is the spatio-temporal learning layer which utilizes graph and standard convolutions to capture the dynamic variations in both spatial and temporal perspective. The third component is the embedding layer which aims to obtain meaningful and semantic representations of external information. To evaluate the proposed model, we collect large-scale real-world data, including accident records, citi-wide vehicle speeds, road networks, meteorological conditions, and Point-of-Interest distributions. Experimental results on real-world datasets demonstrate that DSTGCN outperforms both classical and state-of-the-art methods.

Objects at different spatial positions in an image exhibit different scales. Adaptive receptive fields are expected to capture suitable ranges of context for accurate pixel level semantic prediction. Recently, atrous convolution with different dilation rates has been used to generate features of multi-scales through several branches which are then fused for prediction. However, there is a lack of explicit interaction among the branches of different scales to adaptively make full use of the contexts. In this paper, we propose a Content-Adaptive Scale Interaction Network (CASINet) to exploit the multi-scale features for scene parsing. We build CASINet based on the classic Atrous Spatial Pyramid Pooling (ASPP) module, followed by a proposed contextual scale interaction (CSI) module, and a scale adaptation (SA) module. Specifically, in the CSI module, for each spatial position of some scale, instead of being limited by a fixed set of convolutional filters that are shared across different spatial positions for feature learning, we promote the adaptivity of the convolutional filters to spatial positions. We achieve this by the context interaction among the features of different scales. The SA module explicitly and softly selects the suitable scale for each spatial position and each channel. Ablation studies demonstrate the effectiveness of the proposed modules. We achieve state-of-the-art performance on three scene parsing benchmarks Cityscapes, ADE20K and LIP.

In recent decades, unmanned surface vehicles (USVs) are attracting increasing attention due to their underlying capability in autonomously undertaking complex maritime tasks in constrained environments. However, the autonomy level of USVs is still limited, especially when being deployed to conduct multiple tasks simultaneously. This paper, therefore, aims to improve USVs autonomy level by investigating and developing an effective and efficient task management algorithm for multi-USV systems. To better deal with challenging requirements such as allocating vast tasks in cluttered environments, the task management has been de-composed into two submissions, i.e., task allocation and task execution. More specifically, unsupervised learning strategies have been used with an improved K-means algorithm proposed to first assign different tasks for a multi-USV system then a self-organising map (SOM) been implemented to deal with the task execution problem based upon the assigned tasks for each USV. Differing to other work, the communication problem that is crucial for USVs in a constrained environment has been specifically resolved by designing a new competition strategy for K-means algorithm. Key factors that will influence the communication capability in practical applications have been taken into account. A holistic task management architecture has been designed by integrating both the task allocation and task execution algorithms, and a number of simulations in both simulated and practical maritime environments have been carried out to validate the effectiveness of the proposed algorithms.

The Internet of Vehicles (IoV) can obtain traffic information through a large number of data collected by sensors. However, the lack of data, abnormal data, and other low-quality problems have seriously restricted the development and application of the IoV. To solve the problem of missing data in a large-scale road network, the previous research achievements show that tensor decomposition method has the advantages in solving multi-dimensional data imputation problems, so we adopt this tensor mode to model traffic velocity data. A new method of data missing estimation with tensor heterogeneous ensemble learning based on FNN (Fuzzy Neural Network) named FNNTEL is proposed in this paper. The performance of this method is evaluated by our experiments and analysis. The proposed method is applied to be tested by the real data captured in Guangzhou and Tianjin of China respectively. A large number of experimental tests show that the performance of the new method is better than other commonly used technologies and different missing data generation models.

The identification of disease-associated miRNAs can help people better understand the pathogenesis of diseases from a genetic perspective. Therefore, the prediction of miRNA-disease associations has received increasing attention. In this paper, we propose a new computational method NCFM (Neural network-based Collaborative Filtering Method) to predict miRNA-disease associations based on deep neural network. Firstly, high-dimensional sparse vectors of diseases and miRNAs are mapped into low-dimensional dense vectors in implicit semantic space via embedding layer, which called disease embedding and miRNA embedding, respectively. Secondly, the neural collaborative filter layers model the latent feature interactions between miRNAs and diseases. Then, different from other methods using square error loss function, we propose a new pairwise loss function to optimizes our model from a ranking perspective. Finally, experiments show that our proposed method can effectively prioritize disease-related miRNAs with the highest AUC of 0.912 and 0.921 compared with other recent methods in 5-fold cross validation and LOOCV framework. In addition, we implement two types of case studies, including four diseases. For a disease, more than 90% of predicted miRNAs are validated by another official dataset, which further illustrates the effectiveness of NCFM.

With great value in real applications, sequential recommendation aims to recommend users the personalized sequential actions. To achieve better performance, it is essential to consider both long-term preferences and sequential patterns ( i . e ., short-term dynamics). Compared to widely used Recurrent Neural Network (RNN) and Convolutional Neural Network (CNN), Self-Attention Network (SAN) obtains a surge of interest due to fewer parameters, highly parallelizable computation, and flexibility in modeling dependencies. However, existing SAN-based models are inadequate in characterizing and distinguishing users’ long-term preferences and short-term demands since they do not emphasize the importance of the current interest and temporal order information of sequences. In this paper, we propose a novel multi-layer long- and short-term self-attention network (LSSA) for sequential recommendation. Specifically, we first split the entire sequence of a user into multiple sub-sequences according to the timespan. Then the first self-attention layer learns the user’s short-term dynamics based on the last sub-sequence, while the second one captures the user’s long-term preferences through the previous sub-sequences and the last one. Finally, we integrate the long- and short-term representations together to form the user’s final hybrid representation. We evaluate the proposed model on three real-world datasets, and our experimental results show that LSSA outperforms state-of-the-art methods with a wide margin.

Few-Shot Learning (FSL) aims at recognizing new categories from a few available samples. In this paper, we propose two strategies on the basis of Prototypical Networks [1] to improve the discriminativeness and representativeness of the visual prototypes for few-shot learning task. Firstly, we propose a reweighting mechanism to distribute different weights for accesses the instance representativeness to the class. Secondly, we propose an information-guidance mechanism to encode discriminative knowledge into the class prototypes to compensate for more information across classes. Extensive experimental results on two benchmark datasets empirically show that both the proposed strategies improve the Prototypical Networks and achieve the state-of-the-art performances. Besides, the information-guidance mechanism could be seamlessly combined into some existing approaches to substantially improve their performances on few-shot classification.

In this paper, we devise a distributed protocol to study the consensus of multi-agent systems (MASs) with input saturation over directed networks. Firstly, a low-gain feedback control method is applied to overcome the saturation constraints, and an observation system without saturation constraints is designed. The semiglobal consensus condition over directed strongly connected networks is developed by using Ricatti equation. Secondly, a novel of tree-type error scheme is proposed to solve the consensus of MASs with general directed spanning tree networks. Combination with inequality techniques and Lyapunov stability theory, some criteria for achieving semiglobal consensus are obtained. In addition, the semiglobal consensus of MASs with directed switching networks is further considered, in which the network topologies only need to contain a directed spanning tree in some time intervals. Some related conditions are derived to ensure the consensus. Finally, some numerical simulations are given to demonstrate the validity of the theoretical results.

Electroencephalogram (EEG) signal analysis, which is widely used for human-computer interaction and neurological disease diagnosis, requires a large amount of labeled data for training. However, the collection of substantial EEG data could be difficult owing to its randomness and non-stationary. Moreover, there is notable individual difference in EEG data, which affects the reusability and generalization of models. For mitigating the adverse effects from the above factors, transfer learning is applied in this field to transfer the knowledge learnt in one domain into a different but related domain. Transfer learning adjusts models with small-scale data of the task, and also maintains the learning ability with individual difference. This paper describes four main methods of transfer learning and explores their practical applications in EEG signal analysis in recent years. Finally, we discuss challenges and opportunities of transfer learning and suggest areas for further study.