Search

This paper focuses on the word disentanglement-based approaches for text style transfer. The related systems first remove the style attribute words in the given texts and then generate the target sentences using the remained neutral templates. Previous work retrieves the style words using intuitive heuristics, which lacks in-depth analysis and can hardly provide a precise word detection. The resulting error will further affect the generation phase and leads to the failure of the style transformation. In this paper, we formalize the style detection task as a dynamic decision process; each word in the given text is classified as a style or neutral content sequentially leveraging the word property, the context, and the decision history. As no background labels indicating the attribute words, reinforcement learning is deployed in our model. An evaluation (reward) function is designed to quantify the style shift and content loss simultaneously, providing an accurate estimation for the decision process. The efficiency of the proposal is verified on two types of style transfer tasks, i.e., sentiment transfer and formality transfer. The experimental results demonstrate the model's competitive performance compared to the state-of-the-art baselines once equipped with an insertion-based generation approach.

Intrusion Detection System (IDS) is an important tool to ensure network security, which can detect and prevent malicious behavior in time. However, the noise and redundancy of data often reduce the detection performance of classifiers. The traditional model of intrusion detection system cannot effectively solve this problem. Therefore, in this paper, autoencoders (AEs) are firstly used to reduce the dimension of the original data, and a hybrid model combining particle swarm optimization (PSO) and gray wolf optimization (GWO) is proposed to optimize the support vector machine (SVM) parameters. This method combines the two optimization algorithms and selects the optimal parameter values according to the locally enhanced particles to train the classifier. In this paper, the NSL-KDD benchmark dataset and UNSW-NB15 dataset are used to evaluate the proposed model, and the model is compared with other classification methods separately. The experimental results show that our hybrid optimization model has better performance in detection accuracy and provides good detection rate and false alarm rate.

In recent years, deep learning has developed rapidly and has shown great performance on many challenging machine learning tasks, such as image classification, natural language processing, and speech recognition. However, researchers have recently discovered that deep learning models have security risks and are easily affected by adversarial examples. The adversarial example is a sample formed by deliberately adding subtle perturbation that is invisible to the human in the dataset. It can make the classification classify incorrectly with a high degree of confidence, which poses more challenges for deep learning research. In this paper, we propose a defense model based on the supervision mechanism. The model adds supervision layers to the original convolutional neural network and improves the robustness and defense ability of the model by improving the loss function. The LeNet-5 and VGG networks are used as the original network models. The experimental results on MNIST and CIFAR-10 confirm that the method proposed in this paper will effectively increase the difficulty of the attackers.

Scientific creativity refers to the development of new and innovative ideas that foster technological growth and advancement. This paper attempts to identify creative individuals in scientific domain by evaluating their visuo-spatial reasoning ability using EEG analysis system. The main objective of the present work is achieved by first procuring EEG signals from the scalp of subjects involved in a mental paper folding task. The acquired signals are pre-processed and transformed into a topological map using correlation analysis to investigate the connectivity patterns amongst different brain lobes pertaining to different degrees of spatial abilities (high and low). The acquired brain networks depict strong interconnections amongst bilateral supramarginal gyrus, right anterior prefrontal and right superior temporal lobes for subject possessing high spatial skills. Since, the correlation based brain network analysis provides a qualitative illustration of the connectivity patterns, centrality induced quantitative analysis have been performed. Such an evaluation has been conducted using two categories of centrality measures viz. degree centrality and betweenness centrality. The results obtained from centrality analysis also infer the active involvement of the aforesaid brain regions during mental paper folding activity. Finally, the centrality based network features are transferred to a novel Capsule Graph Neural Network (CapsGNN) based classifier to categorize the desired class labels. Performance analysis undertaken confirms the superior functionality of the proposed network with respect to the conventional models. Moreover, statistical evaluation conducted also assures the supercilious performance of the proposed classifier. Thus, the present methodology can be utilized for recruiting creative individuals to different sectors of industries associated with innovation such as architecture, game designing and the like.

This paper considers the unsupervised domain adaptation problem, in which we want to find a good prediction function on the unlabeled target domain, by utilizing the information provided in the labeled source domain. A common approach to the domain adaptation problem is to learn a representation space where the distributional discrepancy of the source and target domains is small. Existing methods generally tend to match the marginal distributions of the two domains, while the label information in the source domain is not fully exploited. In this paper, we propose a representation learning approach for domain adaptation, which is addressed as JODAWAT. We aim to adapt the joint distributions of the feature-label pairs in the shared representation space for both domains. In particular, we minimize the Wasserstein distance between the source and target domains, while the prediction performance on the source domain is also guaranteed. The proposed approach results in a minimax adversarial training procedure that incorporates a novel split gradient penalty term. A generalization bound on the target domain is provided to reveal the efficacy of representation learning for joint distribution adaptation. We conduct extensive evaluations on JODAWAT, and test its classification accuracy on multiple synthetic and real datasets. The experimental results justify that our proposed method is able to achieve superior performance compared with various domain adaptation methods.

Aspect-based sentiment analysis analyses the fine-grained sentiment polarity of a particular attribute in a sentence. In addition to the fact that most papers have applied attention mechanisms and neural networks to fine-grained sentiment analysis, some studies have reported outstanding performance from graph-structured neural networks in aspect-based sentiment analysis. In this paper, we propose the use of a graph attention network (GAT) to embed external knowledge and grammatical relations (KD-GAT). First, we employ a GAT to extract the nodes and edges related to sentences in the knowledge graph. Second, we consider the influence of conjunctions and parts of speech based on the aspect-oriented syntactic dependency tree. Finally, we emphasize the effect of aspect position in the mechanism of multiple attention. Experiments on the SemEval 2014 and Twitter datasets show that our model can improve the ability to analyze sentiment and that the graph structure method can better integrate grammatical relations to understand a sentence.

Adversarial examples about deep learning models have been paid much attention in recent years, including single-label adversarial examples and multi-label adversarial examples. In this paper, for the first time, an empirical study of generating a multi-label adversarial example to hide all labels in a multi-label example is presented. The objective of hiding all labels in a multi-label example is to make deep learning models know nothing about the environments. That is very worthy of studying because deep learning models will say there is nothing, although the real input has more than one label. In the empirical study, we use five state-of-the-art multi-label attack algorithms, i.e., ML-CW, ML-DP, FGSM, MI-FGSM, MLA-LP, four popular datasets, i.e., VOC2007, VOC2012, NUS-WIDE and COCO, and two typical models ML-GCN and ASL for evaluation. We conduct extensive experiments and report the attack success rates, the amount of perturbations of the adversarial examples generated by state-of-the-art multi-label attack algorithms. We also report the attack performance when a typical defending algorithm based on JPEG compression is used. The work in this paper is beneficial to the future study of generating multi-label adversarial examples as well as defending them.

One of the most important processes in big data analysis is outlier detection, where anomalies of observed data are detected and eliminated to improve the system performance. Many approaches to outlier detection have been proposed so far for classification and prediction purposes. This paper focus on the outlier detection under a practical circumstance such that multiple organizations possess different data sets of a specific task, while they cannot directly share with each other from a privacy point of view; that is, each organization is not allowed to provide their sensitive data to other entities but they are keen to cooperate in data analysis for some reasons. To address this issue, we present a new outlier detection approach to the data analysis for multiple organizations using a decision tree ensemble based on the so-called federated learning scheme. In this paper, we extend an existing outlier detection mode called Isolation Forest to the federated learning concept so that data protection of each entity can be achieved by introducing an additive homomorphic encryption. The experimental results for several benchmark data sets demonstrate that the proposed privacy-preserving Isolation Forest (pp-iForest) achieves stable classification performance, which is almost the same as that under a single-organization setting, even when the number of organization is increased.

Multilingual chat systems are the need of the hour for organizations who render online conversational services to their customers. This can effectively be facilitated using the robust machine translation (MT) systems. Translating user-generated contents is regarded as one of the challenging tasks for MT. As for translating conversations, it is more challenging since the meaning of any particular utterance in a conversation usually depends on its context. Moreover, chats in conversational systems are usually informal, contain code-mixed (mix of more than one language), and other grammatical inconsistencies. In this paper, we use state-of-the-art Transformer models to build our MT systems and to translate conversations between the customer service agents and customers. We propose a novel method which effectively selects contextual information from the source text of conversation to be translated. We also employ a terminology-based pseudo in-domain corpus mining strategy for fine-tuning our translation model. We evaluate our methods on the German-English WMT20 Shared Task on Chat Translation dataset, and obtain 61.1 and 63.9 BLEU points on evaluation test sets for English-to-German and German-to-English, respectively, surpassing the present state-of-the-art MT systems by 0.7 BLEU and 1.5 BLEU points, respectively. In this paper, we use state-of-the-art Transformer models to build our MT systems and to translate conversations between the customer service agents and customers. We propose a novel method which effectively selects contextual information from the source text of conversation to be translated. We also employ a terminology-based pseudo in-domain corpus mining strategy for fine-tuning our translation model. We evaluate our methods on the German-English WMT20 Shared Task on Chat Translation dataset, and obtain 61.1 and 63.9 BLEU points on evaluation test sets for English-to-German and German-to-English, respectively, surpassing the present state-of-the-art MT systems by 0.7 BLEU and 1.5 BLEU points, respectively.

Transfer learning has been widely studied and gained increasing popularity to improve the accuracy of machine learning models by transferring some knowledge acquired in different training. However, no prior work has pointed out that transfer learning can strengthen privacy attacks on machine learning models. In this paper, we propose TransMIA (Transfer learning-based Membership Inference Attacks), which use transfer learning to perform membership inference attacks on the source model when the adversary is able to access the parameters of the transferred model. In particular, we propose a transfer shadow training technique, where an adversary employs the parameters of the transferred model to construct shadow models, to significantly improve the performance of membership inference when a limited amount of shadow training data is available to the adversary. We evaluate our attacks using two real datasets, and show that our attacks outperform the state-of-the-art that does not use our transfer shadow training technique. We also compare four combinations of the learning-based/entropy-based approach and the fine-tuning/freezing approach, all of which employ our transfer shadow training technique. Then we examine the performance of these four approaches based on the distributions of confidence values, and discuss possible countermeasures against our attacks., Comment: IJCNN 2021 conference paper

By using document-level contextual information, document-level neural machine translation can achieve better results than ordinary machine translation, but traditional document-level machine translation is difficult to focus on the contextual sentence articulation relations and deep positional relations within the discourse while utilizing document-level vocabulary, and the model can concentrate only on relatively shallow inter-sentential relations or positional information. In this paper, we consider that most adjacent sentences are connected in document translation, and such links help improve the quality of translation. We propose a document translation model that focuses more on inter-sentential relations based on the previous work, and propose two methods to strengthen the model's positional information input, and combine these two methods to enhance the traditional Transformer positional information input. This paper also proposes a method for inserting paragraph information to allow inter-sentential relations to be learned by the model, and uses the improved Transformer model for Chinese-Mongolian document translation. Experiments show that in the improved Transformer system, the BLEU scores are enhanced on the Chinese-Mongolian machine translation task after fusing positional information and inter-sentential relation information, and the translation achieves better performance.

In recent years, Convolutional Neural Networks (CNN s) have been widely applied in various applications due to its powerful learning capability. However, its lack of explainability hinders its further usage in tasks requiring high reliability. Therefore, interpretability technique is the key to the application and deployment of CNN models. As a typical interpretability technique for CNN, Class Activation Map (CAM) utilizing the gradient based weights and activation map is widely applied to traditional CNN models for offering visual interpretability. However, the activation map adopted by CAM cannot loyally quantify the relevance between input samples and activation values. Hence, in this paper, we propose a new interpretability approach called Salience-CAM employing salience scores to accurately measure the relevance between input samples and activation values. To evaluate the effectiveness of Salience-CAM, comprehensive experiments are conducted on 6 selected time series datasets. By leveraging an evaluation algorithm proposed in this paper, the experimental results show that our proposed Salience-CAM outperforms the baseline by discovering more discriminative features.

This paper concerns the cooperative spectrum sensing (CSS) for cognitive radio (CR) networks, where the secondary users (SUs) collaborate to detect the presence of the primary users (PUs). With CSS, the information from different SUs is first fused, then, the detection of the PU signal is implemented based on the fused information. Most of the previous works focus on the design of a mapping function that calculates the probability of the existence of the PU signal based on the fused information. In this paper, we study the fusion process which combines the information from all SUs based on the local property of each SU. A graph-based multi-view learning framework for CSS (GMCSS) is designed to better fuse the information from different SUs. In the proposed framework, the information from each SU is considered as a view of the state of the target wireless channel and is fused with the information from other SUs through a graph-based learning process. Simulation results demonstrate the effectiveness of our method.

Script event prediction is a significant task to extract event evolution rules from text. It requires to capture the intra-rationality and inter-relationship of events. The previous methods have made great progress, but they can not fully capture the inner features of events. This paper proposes an event representation layer with argument-level attention to capture the relationship between predicate and arguments, which benefit to assess the rationality of the event. Besides, we integrate the enhanced event representation with the multi-level event similarity scores to obtain more robust performance. The multi-level similarity consists of scene similarity and semantic consistency. We utilize the similarity between candidates and individual events to evaluate the scene similarity. Semantic consistency is evaluated by the hidden state of Gate Recurrent Unit (GRU). Unlike the previous work, we only use the last hidden state of GRU instead of all hidden states to reduce training time. In addition, we adopt adversarial training to improve the robustness and generalization of the model. The experimental results verify the effectiveness of our model compared to these baseline methods. This paper also demonstrates what the attention mechanism has learned to enhance the interpretability of our model.

Online anomalous taxi trajectory detection, which identifies anomalies from ongoing taxi trajectories, has become an important and fundamental concern in many real-world applications. Most of the existing studies define the anomalous trajectories as the ones deviating from the majority of routes or showing abnormal driving time and distance at the same time. However, due to the complexity of road conditions and the variety of passenger preferences, those methods have large false-positive rates, i.e., reporting many normal routes as anomalies. A high false-positive rate is harmful since false alarms will 1) bring unnecessary panic to passengers, 2) cause fiscally punishments to normal drivers, and 3) wastes human resource to deal with drivers' complaints. To this end, this paper proposes an online anomalous trajectory detection method, namely multidimensional criteria based anomalous trajectory (MCAT), to identify anomalous trajectories online. It judges anomalies by considering multidimensional criteria (similarity, time, distance) at the same time, reducing the false positives without sacrificing false negative rates. We evaluate the proposed method based on the real-world taxi data collected from Shanghai, China. The experimental results demonstrate that our method can outperform state-of-the-art methods in terms of accuracy, false-negative rate, and false-positive rate.

The task of reasoning over knowledge graph for factoid questions has received significant interest from the research community of natural language processing. Performing this task inevitably faces the issues of question complexity and reasoning efficiency. In this paper, we investigate modern reasoning approaches over knowledge graph to tackle complex factoid questions of diverse reasoning schemas with attractive speedup in computational efficiency. To this end, we propose two evidence retrieval strategies to generate concise and informative evidence graph of high semantic-relevance and factual coverage to the question. Then, we adopt DELFT, a graph neural networks based framework that takes the linguistic structure representation of a question and the evidence graph as input, to predict the answer by reasoning over the evidence graph. We evaluate the performance across several baselines in terms of effectiveness and efficiency on two real-world datasets, MOOCQA and MetaQA. The results show the superiority of message passing paradigm in delivering a robust reasoner with better answer quality and significantly improved computational efficiency.

Existing image captioning approaches fail to generate fine-grained captions due to the lack of rich encoding representation of an image. In this paper, we present an attentive contextual network (ACN) to learn the spatially transformed image features and dense multi-scale contextual information of an image to generate semantically meaningful captions. At first, we construct deformable network on intermediate layers of convolutional neural network (CNN) to cultivate spatial invariant features. And the multi-scale contextual features are produced by employing contextual network on top of last layers of CNN. Then, we exploit attention mechanism on contextual network to extract dense contextual features. Further, the extracted spatial and contextual features are combined to encode the holistic representation of an image. Finally, a multi-stage caption decoder with visual attention module is incorporated to generate fine-grained captions. The performance of the proposed approach is demonstrated on COCO dataset, the largest dataset for image captioning.

Spiking neural networks (SNNs) are promising to replicate the efficiency of the brain by utilizing a paradigm of spike-based computation. Training a deep SNN is of great importance for solving practical tasks as well as discovering the fascinating capability of spike-based computation. The biologically plausible scheme of local learning motivates many approaches that enable training deep networks in an efficient parallel way. However, most of the existing spike-based local learning approaches show relatively low performances on challenging tasks. In this paper, we propose a new spike-based temporal dependent local learning (TDLL) algorithm, where each hidden layer of a deep SNN is independently trained with an auxiliary trainable spiking projection layer, and temporal dependency is fully employed to construct local errors for adjusting parameters. We examine the performance of the proposed TDLL with various networks on the MNIST, Fashion-MNIST, SVHN and CIFAR-10 datasets. Experimental results highlight that our method can scale up to larger networks, and more importantly, achieves relatively high accuracies on all benchmarks, which are even competitive with the ones obtained by global backpropagation-based methods. This work therefore contributes to providing an effective and efficient local learning method for deep SNNs, which could greatly benefit the developments of distributed neuromorphic computing.

Adversarial attacks/perturbations are becoming important for NLP research, as it has been shown recently that text-attacking adversaries can degrade an NLP model's performance without the victim's knowledge. This has far-reaching implications, especially when an NLP system is deployed in critical applications such as health or finance. In fact, the robustness of state-of-the-art models such as NLP transformers have increasingly been scrutinised due to their vulnerability against adversarial perturbations such as TextFooler and BERT-Attack. These methods, however, focus on changing words, which at times, ruins the readability and semantics of the sample. This paper introduces Special Symbol Text Attacks ‘SSTA’, a technique to improve the performance of language adversarial perturbations using special symbols that have downstream task information associated with them even though that should not have been the case. Our tests show that introducing such symbols which are meaningless to a human within a sentence, can perturb the sample in a particular direction. When this technique is used with TextFooler, which is a recent benchmark for the creation of NLP adversaries, through the TextAttack framework, it can improve all main evaluation metrics on three sentiment classification tasks and fake news detection. A simple, novel and symbol-specific adversarial learning technique is then introduced to reduce the influence of such special symbols.

Generating text with high quality and sufficient diversity is a fundamental task in natural language generation. Although generative adversarial networks (GANs) achieve promising results in text generation, GAN-based language models suffer from mode collapse, i.e., the generator tends to sacrifice diversity and focus on limited text patterns with high quality. By contrast, maximum likelihood estimation (MLE) based language models could cover various text patterns and generate diversified samples with poor quality. This paper proposes a cooperative framework with GANs and entropic auto-encoders (EAEs), named GAN-EAE, to synthesize their advantages for text generation, where EAEs are powerful MLE-based generative models based on deterministic auto-encoders. By imitating the output distribution of EAEs, the generator shapes its output distribution closer to the real data distribution against mode collapse. Meanwhile, by learning the samples from the generator of GANs, EAEs subtly distribute probability mass on high quality patterns for improving generation quality. The similar samples obtained from the generator may raise mode collapse and should be downplayed during adversarial training. Thus, a sample re-weighting mechanism is adopted to improve diversity by measuring the inner distance of generated samples. Experimental results demonstrate that GAN-EAE could improve both GANs and EAEs to achieve state-of-the-art performance.