Search

This paper focuses on the word disentanglement-based approaches for text style transfer. The related systems first remove the style attribute words in the given texts and then generate the target sentences using the remained neutral templates. Previous work retrieves the style words using intuitive heuristics, which lacks in-depth analysis and can hardly provide a precise word detection. The resulting error will further affect the generation phase and leads to the failure of the style transformation. In this paper, we formalize the style detection task as a dynamic decision process; each word in the given text is classified as a style or neutral content sequentially leveraging the word property, the context, and the decision history. As no background labels indicating the attribute words, reinforcement learning is deployed in our model. An evaluation (reward) function is designed to quantify the style shift and content loss simultaneously, providing an accurate estimation for the decision process. The efficiency of the proposal is verified on two types of style transfer tasks, i.e., sentiment transfer and formality transfer. The experimental results demonstrate the model's competitive performance compared to the state-of-the-art baselines once equipped with an insertion-based generation approach.

In real world, facial expression images are easily disturbed by different external environments, such as face poses, illumination intensities, and occlusion, and features extracted from whole facial expression images often have a lot of noise, which leads to low recognition accuracy of the model. Inspired by the facts that local images are less disturbed by the environment and human beings usually recognize facial expressions by integrating some local information about human faces, this paper proposes a model for recognition of facial expressions, based on the fusion of local image features, that aims to recognize facial expressions by comprehensively considering local information about human faces. To particularly integrate the robust characteristics of local images, this paper constructs a visual self-attention network by transformation of the Transformer model in the field of natural language processing, which can take into account the interaction between local images and make the image feature more discriminative. To demonstrate the performance of the model, this paper conducts experiments on three constrained small datasets, CK+, Oulu-CASIA and MMI, and three unconstrained large datasets, FER2013, RAF-DB and Expw. The results show that the proposed model can attain the recognition accuracy of the state-of-the-art methods on all datasets, thereby verifying the effectiveness of the proposed method.

The common practice of quality monitoring in industry relies on manual inspection well-known to be slow, error-prone and operator-dependent. This issue raises strong demand for automated real-time quality monitoring developed from data-driven approaches thus alleviating from operator dependence and adapting to various process uncertainties. Nonetheless, current approaches do not take into account the streaming nature of sensory information while relying heavily on hand-crafted features making them application-specific. This paper proposes the online quality monitoring methodology developed from recently developed deep learning algorithms for data streams, Neural Networks with Dynamically Evolved Capacity (NADINE), namely NADINE++. It features the integration of 1-D and 2-D convolutional layers to extract natural features of time-series and visual data streams captured from sensors and cameras of the injection molding machines from our own project. Real-time experiments have been conducted where the online quality monitoring task is simulated on the fly under the prequential test-then-train fashion - the prominent data stream evaluation protocol. Comparison with the state-of-the-art techniques clearly exhibits the advantage of NADINE++ with 4.68\% improvement on average for the quality monitoring task in streaming environments. To support the reproducible research initiative, codes, results of NADINE++ along with supplementary materials and injection molding dataset are made available in \url{https://github.com/ContinualAL/NADINE-IJCNN2021}., This paper has been accepted for publication in IJCNN, 2021

The evolution of activation functions in deep neural networks is usually driven by fixed goals and incremental steps toward solving specific problems. We introduce differential equation activations (DiffAct), an umbrella of activation units to improve the process of learning activation functions in complex neural network structures on different data sets. Our stated objective is to enable a feed-forward neural network (FNN) to learn nonlinear activation functional forms from a family of solutions to an ordinary differential equation. In this paper, we also present new activation functions in parabolic forms, as offspring of the family of DiffActs. We demonstrate the effectiveness of the exercise of exploiting DiffAct based units in accomplishing comparable, sometimes superior, performance against single, fixed activation functions. The stated objective is to discover the internal dynamics of such activation functions via a common framework instead of producing State-of-the-Art (SOTA) results. The paper also hypothesizes on the ability to draw effective inferences on simple, 1-hidden layer architectures, wherever possible.

Access to large corpora with strongly labelled sound events is expensive and difficult in engineering applications. Many researches turn to address the problem of how to detect both the types and the timestamps of sound events with weak labels that only specify the types. This task can be treated as a multiple instance learning (MIL) problem, and a key to it in the sound event detection (SED) task is the design of a pooling function. The linear softmax pooling function achieves state-of-the-art performance since it can vary both the signs and the magnitudes of gradients. However, linear softmax pooling cannot flexibly deal with sound events of different time scales. In this paper, we propose a power pooling function which can automatically adapt to various sound events. By adding a trainable parameter to each event, power pooling can provide more accurate gradients for frames in a clip than other pooling functions. On both weakly supervised and semi-supervised SED datasets, the proposed power pooling function outperforms linear softmax pooling on both coarse-grained and fine-grained metrics. Specifically, it improves the event-based F1 score by 11.4% and 10.2% relatively on the two datasets. While this paper focuses on SED applications, the proposed method can be applied to MIL tasks in other domains.

Intrusion Detection System (IDS) is an important tool to ensure network security, which can detect and prevent malicious behavior in time. However, the noise and redundancy of data often reduce the detection performance of classifiers. The traditional model of intrusion detection system cannot effectively solve this problem. Therefore, in this paper, autoencoders (AEs) are firstly used to reduce the dimension of the original data, and a hybrid model combining particle swarm optimization (PSO) and gray wolf optimization (GWO) is proposed to optimize the support vector machine (SVM) parameters. This method combines the two optimization algorithms and selects the optimal parameter values according to the locally enhanced particles to train the classifier. In this paper, the NSL-KDD benchmark dataset and UNSW-NB15 dataset are used to evaluate the proposed model, and the model is compared with other classification methods separately. The experimental results show that our hybrid optimization model has better performance in detection accuracy and provides good detection rate and false alarm rate.

More and more online resources for computer science are introduced, used and released in scientific literature in recent years. Knowledge about the usage of these online resources can help researchers easily find the applicable resources for their works. However, most existing methods ignore the importance of the content of the online resource citations. To this end, we manually create SciR, a dataset that contains 3,012 annotation sentences for this task, and introduce a multi-task learning framework to automatically extract the entities and relations from the context of online resource citations in scientific papers. Furthermore, considering the words in a sentence usually play different roles under different relations. In this paper, we treat different relations as distinctive sub-spaces and model the correlations between words in sentence for each relation type by a supervised biaffine attention network. Based on this relation-aware attention network, our model can not only effectively obtain the word-level correlations under each relation, but also naturally avoid the problem of overlapping relations. To evaluate the effectiveness of our model, we conduct comprehensive experiments on three datasets and the experimental results demonstrate that our model outperforms other state-of-the-art methods on the two tasks of entity recognition and relation extraction.

In recent years, deep learning has developed rapidly and has shown great performance on many challenging machine learning tasks, such as image classification, natural language processing, and speech recognition. However, researchers have recently discovered that deep learning models have security risks and are easily affected by adversarial examples. The adversarial example is a sample formed by deliberately adding subtle perturbation that is invisible to the human in the dataset. It can make the classification classify incorrectly with a high degree of confidence, which poses more challenges for deep learning research. In this paper, we propose a defense model based on the supervision mechanism. The model adds supervision layers to the original convolutional neural network and improves the robustness and defense ability of the model by improving the loss function. The LeNet-5 and VGG networks are used as the original network models. The experimental results on MNIST and CIFAR-10 confirm that the method proposed in this paper will effectively increase the difficulty of the attackers.

This is a theory paper. It first raises a rarely reported but unethical practice in Artificial Intelligence (AI) called Post Selection Using Test Sets (PSUTS). Consequently, the popular error-backprop methodology in deep learning lacks an acceptable generalization power. All AI methods fall into two broad schools, connectionist and symbolic. PSUTS practices have two kinds, machine PSUTS and human PSUTS. The connectionist school received criticisms for its “scruffiness” due to a huge number of scruffy parameters and now the machine PSUTS; but the seemingly “clean” symbolic school seems more brittle than what is known because of using human PSUTS. This paper formally defines what PSUTS is, analyzes why error-backprop methods with random initial weights suffer from severe local minima, why PSUTS violates well-established research ethics, and how every paper that used PSUTS should have at least transparently reported PSUTS data. For improved transparency in future publications, this paper proposes a new standard for AI metrology, called developmental errors for all networks trained in a project that the selection of the luckiest network depends on, along with Three Conditions: (1) system restrictions, (2) training experience and (3) computational resources.

The current volume of data in the field of securities investment is increasing dramatically. Simultaneously, the linkage of data from multiple parties makes investment reasoning decisions more challenging than ever. In response to this problem, the financial field's knowledge graph can improve the efficiency, depth, and breadth of financial practitioners' information analysis. Some existing financial knowledge graphs analyze the shareholding relationship between companies. Still, because they are limited to observing data from the company's perspective, users without professional industry background cannot quickly find the industry factors of stock market changes. This paper proposes a financial knowledge graph from the industry chain's perspective. This paper builds upstream and downstream relationships between industries through Transformer-based bidirectional encoder to mine potential industry chain associations from text data and completes the long industry chain of the stock market. This paper also builds a visualization system to display and explore the connection between listed companies and industries. Users can inspect the industry chain's composition and each company's revenue status and stock market conditions in the industry chain. The experiment shows that when the market price fluctuation is detected, the stock price fluctuation can be traced back to its origin in the knowledge graph.

Scientific creativity refers to the development of new and innovative ideas that foster technological growth and advancement. This paper attempts to identify creative individuals in scientific domain by evaluating their visuo-spatial reasoning ability using EEG analysis system. The main objective of the present work is achieved by first procuring EEG signals from the scalp of subjects involved in a mental paper folding task. The acquired signals are pre-processed and transformed into a topological map using correlation analysis to investigate the connectivity patterns amongst different brain lobes pertaining to different degrees of spatial abilities (high and low). The acquired brain networks depict strong interconnections amongst bilateral supramarginal gyrus, right anterior prefrontal and right superior temporal lobes for subject possessing high spatial skills. Since, the correlation based brain network analysis provides a qualitative illustration of the connectivity patterns, centrality induced quantitative analysis have been performed. Such an evaluation has been conducted using two categories of centrality measures viz. degree centrality and betweenness centrality. The results obtained from centrality analysis also infer the active involvement of the aforesaid brain regions during mental paper folding activity. Finally, the centrality based network features are transferred to a novel Capsule Graph Neural Network (CapsGNN) based classifier to categorize the desired class labels. Performance analysis undertaken confirms the superior functionality of the proposed network with respect to the conventional models. Moreover, statistical evaluation conducted also assures the supercilious performance of the proposed classifier. Thus, the present methodology can be utilized for recruiting creative individuals to different sectors of industries associated with innovation such as architecture, game designing and the like.

Automatic modulation classification (AMC), which aims to identify the modulation types of unknown signals without any prior knowledge, plays a key role in intelligent wireless communication. In recent years, the outstanding achievements of deep learning in the fields of computer vision and nature language processing have promoted the continuous researches of deep learning in AMC. In view of the fact that the existing deep learning-based AMC ignores the frequency attribute of the modulated signals, this paper proposed an AMC approach based on deep learning which comprehensively considers the contribution of frequency and spatiotemporal characteristics of the modulated signals. Specifically, we utilize the sliding window to divide the raw signals into short-time signal slices. In order to obtain the representation of frequency domain of the signals, each short-time signal slice is decomposed into several modes by variational mode decomposition (VMD), which is an adaptive signal decomposition technique. Finally, all the modes are used as input of the convolutional neural network (CNN) as a tensor to learn high-level features from the frequency and spatiotemporal domain to identify the modulation type. In this paper, we verify the effectiveness of the proposed approach with 10 kinds of modulation types generated by SMW200A Vector Signal Generator. More importantly, the proposed approach achieves 99.2% of the overall classification accuracy at 8dB signal to noise ratio (SNR) and shows better robustness to noise than the existing AMC approaches.

In this paper we propose a novel data-level algorithm for handling data imbalance in the classification task, Synthetic Majority Undersampling Technique (SMUTE). SMUTE leverages the concept of interpolation of nearby instances, previously introduced in the oversampling setting in SMOTE. Furthermore, we combine both in the Combined Synthetic Oversampling and Undersampling Technique (CSMOUTE), which integrates SMOTE oversampling with SMUTE undersampling. The results of the conducted experimental study demonstrate the usefulness of both the SMUTE and the CSMOUTE algorithms, especially when combined with more complex classifiers, namely MLP and SVM, and when applied on datasets consisting of a large number of outliers. This leads us to a conclusion that the proposed approach shows promise for further extensions accommodating local data characteristics, a direction discussed in more detail in the paper.

This paper concerns interference estimation problem for device-to-device (D2D) communication networks. In the considered system, D2D users share common spectrum resources, such that the D2D links have interference with each other. To achieve effective interference management, it is necessary to have an accurate understanding of the interference relationship of the D2D devices, which is difficult since the locations and the number of mobile devices can vary over time. In this paper, we formulate an interference estimation problem for the D2D communication network where the D2D users change over time. Our objective is to get accurate estimations for the interference suffered by a generic D2D link and for the interference a D2D link introduces on other users. We propose a graph convolutional neural network (GCN) based estimation model which can estimate the interference of a D2D link based on the location information of the corresponding D2D pairs. Simulation results show the performance of our method.

This paper considers the unsupervised domain adaptation problem, in which we want to find a good prediction function on the unlabeled target domain, by utilizing the information provided in the labeled source domain. A common approach to the domain adaptation problem is to learn a representation space where the distributional discrepancy of the source and target domains is small. Existing methods generally tend to match the marginal distributions of the two domains, while the label information in the source domain is not fully exploited. In this paper, we propose a representation learning approach for domain adaptation, which is addressed as JODAWAT. We aim to adapt the joint distributions of the feature-label pairs in the shared representation space for both domains. In particular, we minimize the Wasserstein distance between the source and target domains, while the prediction performance on the source domain is also guaranteed. The proposed approach results in a minimax adversarial training procedure that incorporates a novel split gradient penalty term. A generalization bound on the target domain is provided to reveal the efficacy of representation learning for joint distribution adaptation. We conduct extensive evaluations on JODAWAT, and test its classification accuracy on multiple synthetic and real datasets. The experimental results justify that our proposed method is able to achieve superior performance compared with various domain adaptation methods.

Aspect-based sentiment analysis analyses the fine-grained sentiment polarity of a particular attribute in a sentence. In addition to the fact that most papers have applied attention mechanisms and neural networks to fine-grained sentiment analysis, some studies have reported outstanding performance from graph-structured neural networks in aspect-based sentiment analysis. In this paper, we propose the use of a graph attention network (GAT) to embed external knowledge and grammatical relations (KD-GAT). First, we employ a GAT to extract the nodes and edges related to sentences in the knowledge graph. Second, we consider the influence of conjunctions and parts of speech based on the aspect-oriented syntactic dependency tree. Finally, we emphasize the effect of aspect position in the mechanism of multiple attention. Experiments on the SemEval 2014 and Twitter datasets show that our model can improve the ability to analyze sentiment and that the graph structure method can better integrate grammatical relations to understand a sentence.

Adversarial examples about deep learning models have been paid much attention in recent years, including single-label adversarial examples and multi-label adversarial examples. In this paper, for the first time, an empirical study of generating a multi-label adversarial example to hide all labels in a multi-label example is presented. The objective of hiding all labels in a multi-label example is to make deep learning models know nothing about the environments. That is very worthy of studying because deep learning models will say there is nothing, although the real input has more than one label. In the empirical study, we use five state-of-the-art multi-label attack algorithms, i.e., ML-CW, ML-DP, FGSM, MI-FGSM, MLA-LP, four popular datasets, i.e., VOC2007, VOC2012, NUS-WIDE and COCO, and two typical models ML-GCN and ASL for evaluation. We conduct extensive experiments and report the attack success rates, the amount of perturbations of the adversarial examples generated by state-of-the-art multi-label attack algorithms. We also report the attack performance when a typical defending algorithm based on JPEG compression is used. The work in this paper is beneficial to the future study of generating multi-label adversarial examples as well as defending them.

One of the most important processes in big data analysis is outlier detection, where anomalies of observed data are detected and eliminated to improve the system performance. Many approaches to outlier detection have been proposed so far for classification and prediction purposes. This paper focus on the outlier detection under a practical circumstance such that multiple organizations possess different data sets of a specific task, while they cannot directly share with each other from a privacy point of view; that is, each organization is not allowed to provide their sensitive data to other entities but they are keen to cooperate in data analysis for some reasons. To address this issue, we present a new outlier detection approach to the data analysis for multiple organizations using a decision tree ensemble based on the so-called federated learning scheme. In this paper, we extend an existing outlier detection mode called Isolation Forest to the federated learning concept so that data protection of each entity can be achieved by introducing an additive homomorphic encryption. The experimental results for several benchmark data sets demonstrate that the proposed privacy-preserving Isolation Forest (pp-iForest) achieves stable classification performance, which is almost the same as that under a single-organization setting, even when the number of organization is increased.

Designing advanced neural architectures to tackle specific tasks involves weeks or even months of intensive investigation by experts with rich domain knowledge. In recent years, neural architecture search (NAS) has attracted the interest of many researchers due to its ability to automatically design efficient neural architectures. Among different search strategies, evolutionary algorithms have achieved significant successes as derivative-free optimization algorithms. However, the tremendous computational resource consumption of the evolutionary neural architecture search dramatically restricts its application. In this paper, we explore how fitness approximation-based evolutionary algorithms can be applied to neural architecture search and propose NAS-EA-FA to accelerate the search process. We further exploit data augmentation and diversity of neural architectures to enhance the algorithm, and present NAS-EA-FA V2. Experiments show that NAS-EA-FA V2 is at least five times faster than other state-of-the-art neural architecture search algorithms like regularized evolution and iterative neural predictor on NASBench-101, and it is also the most effective and stable algorithm on NASBench-201. All the code used in this paper is available at https://github.com/fzjcdt/NAS-EA-FA.

Multilingual chat systems are the need of the hour for organizations who render online conversational services to their customers. This can effectively be facilitated using the robust machine translation (MT) systems. Translating user-generated contents is regarded as one of the challenging tasks for MT. As for translating conversations, it is more challenging since the meaning of any particular utterance in a conversation usually depends on its context. Moreover, chats in conversational systems are usually informal, contain code-mixed (mix of more than one language), and other grammatical inconsistencies. In this paper, we use state-of-the-art Transformer models to build our MT systems and to translate conversations between the customer service agents and customers. We propose a novel method which effectively selects contextual information from the source text of conversation to be translated. We also employ a terminology-based pseudo in-domain corpus mining strategy for fine-tuning our translation model. We evaluate our methods on the German-English WMT20 Shared Task on Chat Translation dataset, and obtain 61.1 and 63.9 BLEU points on evaluation test sets for English-to-German and German-to-English, respectively, surpassing the present state-of-the-art MT systems by 0.7 BLEU and 1.5 BLEU points, respectively. In this paper, we use state-of-the-art Transformer models to build our MT systems and to translate conversations between the customer service agents and customers. We propose a novel method which effectively selects contextual information from the source text of conversation to be translated. We also employ a terminology-based pseudo in-domain corpus mining strategy for fine-tuning our translation model. We evaluate our methods on the German-English WMT20 Shared Task on Chat Translation dataset, and obtain 61.1 and 63.9 BLEU points on evaluation test sets for English-to-German and German-to-English, respectively, surpassing the present state-of-the-art MT systems by 0.7 BLEU and 1.5 BLEU points, respectively.

Vector of Locally Aggregated Descriptors (VLAD) method, which aggregates descriptors and produces a compact image representation, has achieved great success in the field of image classification and retrieval. However, the original VLAD method is a hard assignment strategy that only assigns each descriptor to the nearest neighbor visual word in dictionary, which leads to large quantization error. In this paper, improved VLAD based on adaptive bases and saliency weights is proposed to solve the above problem. The new method considers the local density distribution when assigning local descriptors, adaptively selects several nearest neighbor visual words, and takes the coding coefficients obtained by utilizing saliency as the weights of the selected visual words. Experimental results on Corel 10, 15 Scenes and UIUC Sports Event datasets show that the new coding method proposed in this paper achieves better classification performance compared with the existing five VLAD based methods and two commonly used representation methods.

Transfer learning has been widely studied and gained increasing popularity to improve the accuracy of machine learning models by transferring some knowledge acquired in different training. However, no prior work has pointed out that transfer learning can strengthen privacy attacks on machine learning models. In this paper, we propose TransMIA (Transfer learning-based Membership Inference Attacks), which use transfer learning to perform membership inference attacks on the source model when the adversary is able to access the parameters of the transferred model. In particular, we propose a transfer shadow training technique, where an adversary employs the parameters of the transferred model to construct shadow models, to significantly improve the performance of membership inference when a limited amount of shadow training data is available to the adversary. We evaluate our attacks using two real datasets, and show that our attacks outperform the state-of-the-art that does not use our transfer shadow training technique. We also compare four combinations of the learning-based/entropy-based approach and the fine-tuning/freezing approach, all of which employ our transfer shadow training technique. Then we examine the performance of these four approaches based on the distributions of confidence values, and discuss possible countermeasures against our attacks., Comment: IJCNN 2021 conference paper

By using document-level contextual information, document-level neural machine translation can achieve better results than ordinary machine translation, but traditional document-level machine translation is difficult to focus on the contextual sentence articulation relations and deep positional relations within the discourse while utilizing document-level vocabulary, and the model can concentrate only on relatively shallow inter-sentential relations or positional information. In this paper, we consider that most adjacent sentences are connected in document translation, and such links help improve the quality of translation. We propose a document translation model that focuses more on inter-sentential relations based on the previous work, and propose two methods to strengthen the model's positional information input, and combine these two methods to enhance the traditional Transformer positional information input. This paper also proposes a method for inserting paragraph information to allow inter-sentential relations to be learned by the model, and uses the improved Transformer model for Chinese-Mongolian document translation. Experiments show that in the improved Transformer system, the BLEU scores are enhanced on the Chinese-Mongolian machine translation task after fusing positional information and inter-sentential relation information, and the translation achieves better performance.

In recent years, Convolutional Neural Networks (CNN s) have been widely applied in various applications due to its powerful learning capability. However, its lack of explainability hinders its further usage in tasks requiring high reliability. Therefore, interpretability technique is the key to the application and deployment of CNN models. As a typical interpretability technique for CNN, Class Activation Map (CAM) utilizing the gradient based weights and activation map is widely applied to traditional CNN models for offering visual interpretability. However, the activation map adopted by CAM cannot loyally quantify the relevance between input samples and activation values. Hence, in this paper, we propose a new interpretability approach called Salience-CAM employing salience scores to accurately measure the relevance between input samples and activation values. To evaluate the effectiveness of Salience-CAM, comprehensive experiments are conducted on 6 selected time series datasets. By leveraging an evaluation algorithm proposed in this paper, the experimental results show that our proposed Salience-CAM outperforms the baseline by discovering more discriminative features.

This paper concerns the cooperative spectrum sensing (CSS) for cognitive radio (CR) networks, where the secondary users (SUs) collaborate to detect the presence of the primary users (PUs). With CSS, the information from different SUs is first fused, then, the detection of the PU signal is implemented based on the fused information. Most of the previous works focus on the design of a mapping function that calculates the probability of the existence of the PU signal based on the fused information. In this paper, we study the fusion process which combines the information from all SUs based on the local property of each SU. A graph-based multi-view learning framework for CSS (GMCSS) is designed to better fuse the information from different SUs. In the proposed framework, the information from each SU is considered as a view of the state of the target wireless channel and is fused with the information from other SUs through a graph-based learning process. Simulation results demonstrate the effectiveness of our method.

Script event prediction is a significant task to extract event evolution rules from text. It requires to capture the intra-rationality and inter-relationship of events. The previous methods have made great progress, but they can not fully capture the inner features of events. This paper proposes an event representation layer with argument-level attention to capture the relationship between predicate and arguments, which benefit to assess the rationality of the event. Besides, we integrate the enhanced event representation with the multi-level event similarity scores to obtain more robust performance. The multi-level similarity consists of scene similarity and semantic consistency. We utilize the similarity between candidates and individual events to evaluate the scene similarity. Semantic consistency is evaluated by the hidden state of Gate Recurrent Unit (GRU). Unlike the previous work, we only use the last hidden state of GRU instead of all hidden states to reduce training time. In addition, we adopt adversarial training to improve the robustness and generalization of the model. The experimental results verify the effectiveness of our model compared to these baseline methods. This paper also demonstrates what the attention mechanism has learned to enhance the interpretability of our model.

A new target detection network-Parameter Derivative Network is proposed for the first time in this paper, which is based on an anchor box-based detector. The parameter derivation module is the core of the network, and Its essence is to make use of image features to generate specific convolution kernels to extract the features of target objects. It solves the problem of reduced accuracy of small target objects in image detection. Compared with traditional convolution, this method can effectively obtain the context information of the target object and improve the detection accuracy. After the image features are extracted by the backbone network, they are sent to the parameter derivation module to perform a feature screening, which can effectively reduce the ratio between the foreground and the background in the feature map. It ensures the foreground context information and suppresses the background information at the same times. It is different from the traditional network feature extraction method. Firstly, the convolution kernel of the parameter derivation module is generated after the image is convoluted, and the convolution kernel is formed by processing the same channel, and then the feature map is convoluted. It can effectively suppress background information and reduce the imbalance of front background information. Secondly, in the parameter derivation module, the influence of average pooling and maximum pooling on image classification is verified in this paper. Compared with the maximum pooling, the average pooling has more local context information, which is more helpful for the detection and recognition accuracy of small target objects. Finally, the experimental results show that the parameter derivation module proposed in this paper is particularly effective in the detection and positioning of targets of different sizes without increasing any overhead, especially for small targets. At the same time, for targets of other sizes, this structure can also improve well and the maximum increase in detection performance is 0.36%.

Takagi-Sugeno-Kang (TSK) fuzzy system with Gaussian membership functions (MFs) is one of the most widely used fuzzy systems in machine learning. However, it usually has difficulty handling high-dimensional datasets. This paper explores why TSK fuzzy systems with Gaussian MFs may fail on high-dimensional inputs. After transforming defuzzification to an equivalent form of softmax function, we find that the poor performance is due to the saturation of softmax. We show that two defuzzification operations, LogTSK and HTSK, the latter of which is first proposed in this paper, can avoid the saturation. Experimental results on datasets with various dimensionalities validated our analysis and demonstrated the effectiveness of LogTSK and HTSK., Comment: Int'l Joint Conf. on Neural Networks (IJCNN), Shenzhen, China, July 2021

In this paper, we propose a load balancing algorithm that optimizes the sensor usability by using sensor computing capability and source needs for intrusion detection, which is able to track the attack on the network. This paper combines the entropy-based active learning model to identify intrusion patterns efficiently. From the results, we can see that the designed model can help and improve the decision boundary by increasing the training instance through pooling strategy and entropy uncertainty measure.

Online anomalous taxi trajectory detection, which identifies anomalies from ongoing taxi trajectories, has become an important and fundamental concern in many real-world applications. Most of the existing studies define the anomalous trajectories as the ones deviating from the majority of routes or showing abnormal driving time and distance at the same time. However, due to the complexity of road conditions and the variety of passenger preferences, those methods have large false-positive rates, i.e., reporting many normal routes as anomalies. A high false-positive rate is harmful since false alarms will 1) bring unnecessary panic to passengers, 2) cause fiscally punishments to normal drivers, and 3) wastes human resource to deal with drivers' complaints. To this end, this paper proposes an online anomalous trajectory detection method, namely multidimensional criteria based anomalous trajectory (MCAT), to identify anomalous trajectories online. It judges anomalies by considering multidimensional criteria (similarity, time, distance) at the same time, reducing the false positives without sacrificing false negative rates. We evaluate the proposed method based on the real-world taxi data collected from Shanghai, China. The experimental results demonstrate that our method can outperform state-of-the-art methods in terms of accuracy, false-negative rate, and false-positive rate.

The task of reasoning over knowledge graph for factoid questions has received significant interest from the research community of natural language processing. Performing this task inevitably faces the issues of question complexity and reasoning efficiency. In this paper, we investigate modern reasoning approaches over knowledge graph to tackle complex factoid questions of diverse reasoning schemas with attractive speedup in computational efficiency. To this end, we propose two evidence retrieval strategies to generate concise and informative evidence graph of high semantic-relevance and factual coverage to the question. Then, we adopt DELFT, a graph neural networks based framework that takes the linguistic structure representation of a question and the evidence graph as input, to predict the answer by reasoning over the evidence graph. We evaluate the performance across several baselines in terms of effectiveness and efficiency on two real-world datasets, MOOCQA and MetaQA. The results show the superiority of message passing paradigm in delivering a robust reasoner with better answer quality and significantly improved computational efficiency.

Existing image captioning approaches fail to generate fine-grained captions due to the lack of rich encoding representation of an image. In this paper, we present an attentive contextual network (ACN) to learn the spatially transformed image features and dense multi-scale contextual information of an image to generate semantically meaningful captions. At first, we construct deformable network on intermediate layers of convolutional neural network (CNN) to cultivate spatial invariant features. And the multi-scale contextual features are produced by employing contextual network on top of last layers of CNN. Then, we exploit attention mechanism on contextual network to extract dense contextual features. Further, the extracted spatial and contextual features are combined to encode the holistic representation of an image. Finally, a multi-stage caption decoder with visual attention module is incorporated to generate fine-grained captions. The performance of the proposed approach is demonstrated on COCO dataset, the largest dataset for image captioning.

This paper is in the scope of emotion recognition by employing a recurrent spiking neural network (BI-SNN) architecture for modelling, mapping, learning, classifying, visualising, and understanding of spatio-temporal Electroencephalogram (EEG) data related to different emotional states. It further explores, develops, and applies a methodology based on the NeuCube BI-SNN, that includes methods for EEG data encoding, data mapping into a 3-dimensional BI-SNN model, unsupervised learning using spike-timing dependent plasticity (STDP) rule, spike-driven supervised learning, output classification, network analysis, and model visualisation and interpretation. The research conducted to model different emotional subtypes through mapping both space (brain regions) and time (brain dynamics) components of EEG brain data into SNN architecture. Here, a benchmark EEG dataset was used to design an empirical study that consisted of different experiments for classification of emotions. The obtained accuracy of 94.83% for EEG classification of four types of emotions was superior when compared with traditional machine learning techniques. The BI-SNN models not only detected the brain activity patterns related to positive and negative emotions with a high accuracy, but also revealed new knowledge about the brain areas activated in relation to different emotions. The research confirmed that neural activation increased in the frontal sites of brain (F7, F3, AF4) associated with positive emotions, while in the case of the negative emotions, connectivity strength was concentrated in the frontal (F4, AF3, F7, F8) and parietal sites of the brain (P7, P8).

In recent studies, object detection of satellite remote sensing images by deep learning method has emerged as a major concern in the fields of environmental detection, military investigation and geography. The current object detection practice, such as Faster RCNN, SSD and YOLO (v1-v5), is using new CNN based deep learning method to replace conventional sliding window and handcraft detectors, and it has achieved impressive success for some specific fields in natural scene images. However, in the task of satellite remote sensing images, it's still a major challenge that the existing models cannot guarantee the multi-scale object detection and real-time monitoring with high accuracy. To address these challenges, in this paper, we proposed SeMo- YOLO, a novel one stage deep learning detector based on the combination of MobileNet, YOLOv3 and channel attention mechanism. Experiment result shows that SeMo-YOLO and it's tiny version can achieve significant improvement in the aspect of accuracy for multiscale remote sensing object detection. It has a mAP (@.5) of 92%, 94%, and 95% on RSOD, HRRSD2019 and SAR-SSDD datasets respectively, which has bring significant improvement in detection precision than the existing state-of-the-art Networks. In addition, the weight parameters of the proposed model are less than the existing YOLOv3 so as to it can achieve relatively higher detection speed near real time.

Spiking neural networks (SNNs) are promising to replicate the efficiency of the brain by utilizing a paradigm of spike-based computation. Training a deep SNN is of great importance for solving practical tasks as well as discovering the fascinating capability of spike-based computation. The biologically plausible scheme of local learning motivates many approaches that enable training deep networks in an efficient parallel way. However, most of the existing spike-based local learning approaches show relatively low performances on challenging tasks. In this paper, we propose a new spike-based temporal dependent local learning (TDLL) algorithm, where each hidden layer of a deep SNN is independently trained with an auxiliary trainable spiking projection layer, and temporal dependency is fully employed to construct local errors for adjusting parameters. We examine the performance of the proposed TDLL with various networks on the MNIST, Fashion-MNIST, SVHN and CIFAR-10 datasets. Experimental results highlight that our method can scale up to larger networks, and more importantly, achieves relatively high accuracies on all benchmarks, which are even competitive with the ones obtained by global backpropagation-based methods. This work therefore contributes to providing an effective and efficient local learning method for deep SNNs, which could greatly benefit the developments of distributed neuromorphic computing.

Radar signal recognition plays an significant role in civil applications. Corresponding to two types of intentional modulation signal and unintentional fingerprint signal, radar signal recognition has two kinds of tasks—automatic modulation classification and radar emitter identification. In this paper, we propose a Embedding Bottleneck Gated Recurrent Unit (EBGRU) network that can handle these two tasks separately. The EBGRU consists of three main processing steps. Firstly, the normalized signal pulses are trained in pulse embedding network containing several embedding methods: Pulse2Vec, GloVeP and EPMo, during which we regard the radar signal pulses as radar signal-linguistic sequences for the first time. Then, pulses embeddings are added to original pulses and are sampled to form latent representations of pulses through information bottleneck. Finally, the gated recurrent unit network is utilized to predict radar signal labels. Experiment results show that the proposed method has reached 95.33% on simulated modulation signals and 94.67% at real intercepted emitter signals with relatively less network parameters.

A text-to-image (T2I) generation method is mainly evaluated from two aspects, one is the quality and diversity of the generated images, and the other is the semantic consistency between the generated images and the input sentences. The feature extraction of the text is a very important part. In this paper, we propose a Transformer based Text-to-Image Network (TT2INet). we use the pre-trained Transformer model (ALBERT) to extract the sentence feature vectors and word feature vectors of the input sentences as the basis for the Generative Adversarial Networks (GANs) to generate images. In addition, we also added self-attention mechanism and spectral normalization method to the model. Adding a self-attention mechanism can make the model pay attention to more local features when generating images. Using the spectral normalization method can make the training of GANs more stable. The Inception Scores of our method on Oxford-102, CUB and COCO datasets are 3.90, 4.89 and 26.53, and R-precision scores are 92.55, 87.72 and 92.29, respectively.

Object proposals generated based on sparse points from the raw point cloud have been widely used in 3D object detection. However, following the above scheme, most existing proposal generators have two problems, one is that the features for proposal generation constrain the detection performance by containing insufficient information; the other is that the sparse points obtained from the raw point cloud are misaligned with their corresponding objects in location and feature aspects. In this paper, we propose a dense-to-sparse proposal generation approach for 3D object detection, which can deal with the two problems simultaneously. Our approach utilizes the 3D CNN backbone to output dense features as a supplement to the original sparse point features for proposal generation. Besides, an object-aware feature pooling module is designed to address the misalignment between sparse points and corresponding objects. Experiments on the KITTI dataset show that our method outperforms the existing sparse-style methods and other published state-of-the-art methods.

Adversarial attacks/perturbations are becoming important for NLP research, as it has been shown recently that text-attacking adversaries can degrade an NLP model's performance without the victim's knowledge. This has far-reaching implications, especially when an NLP system is deployed in critical applications such as health or finance. In fact, the robustness of state-of-the-art models such as NLP transformers have increasingly been scrutinised due to their vulnerability against adversarial perturbations such as TextFooler and BERT-Attack. These methods, however, focus on changing words, which at times, ruins the readability and semantics of the sample. This paper introduces Special Symbol Text Attacks ‘SSTA’, a technique to improve the performance of language adversarial perturbations using special symbols that have downstream task information associated with them even though that should not have been the case. Our tests show that introducing such symbols which are meaningless to a human within a sentence, can perturb the sample in a particular direction. When this technique is used with TextFooler, which is a recent benchmark for the creation of NLP adversaries, through the TextAttack framework, it can improve all main evaluation metrics on three sentiment classification tasks and fake news detection. A simple, novel and symbol-specific adversarial learning technique is then introduced to reduce the influence of such special symbols.

Generating text with high quality and sufficient diversity is a fundamental task in natural language generation. Although generative adversarial networks (GANs) achieve promising results in text generation, GAN-based language models suffer from mode collapse, i.e., the generator tends to sacrifice diversity and focus on limited text patterns with high quality. By contrast, maximum likelihood estimation (MLE) based language models could cover various text patterns and generate diversified samples with poor quality. This paper proposes a cooperative framework with GANs and entropic auto-encoders (EAEs), named GAN-EAE, to synthesize their advantages for text generation, where EAEs are powerful MLE-based generative models based on deterministic auto-encoders. By imitating the output distribution of EAEs, the generator shapes its output distribution closer to the real data distribution against mode collapse. Meanwhile, by learning the samples from the generator of GANs, EAEs subtly distribute probability mass on high quality patterns for improving generation quality. The similar samples obtained from the generator may raise mode collapse and should be downplayed during adversarial training. Thus, a sample re-weighting mechanism is adopted to improve diversity by measuring the inner distance of generated samples. Experimental results demonstrate that GAN-EAE could improve both GANs and EAEs to achieve state-of-the-art performance.

Prognostication of vehicle trajectories in unknown environments is intrinsically a challenging and difficult problem to solve. The behavior of such vehicles is highly influenced by surrounding traffic, road conditions, and rogue participants present in the environment. Moreover, the presence of pedestrians, traffic lights, stop signs, etc., makes it much harder to infer the behavior of various traffic agents. This paper attempts to solve the problem of spatio-temporal look-ahead trajectory prediction using a novel recurrent neural network called the Memory Neuron Network. The Memory Neuron Network (MNN) attempts to capture the input-output relationship between the past positions and the future positions of the traffic agents. The proposed prediction model is computationally less intensive and has a simple architecture as compared to other deep learning models that utilize LSTMs and GRUs. It is then evaluated on the publicly available NGSIM dataset and its performance is compared with several state-of-art algorithms. Additionally, the performance is also evaluated on a custom synthetic dataset generated from the CARLA simulator. It is seen that the proposed model outperforms the existing state-of-art algorithms. Finally, the model is integrated with the CARLA simulator to test its robustness in real-time traffic scenarios.

In this paper, we propose a deep weight global registration (DWG-Reg) algorithm for poor initialization and partially overlapping point clouds registration problem. Our DWG-Reg is based on three modules: a bidirectional nearest search strategy for correspondence, a convolutional network for correspondence confidence prediction which consists of Hybird Distance Generator, optimal annealing Parameter Prediction network and a robust kernel function, a weighted optimizer algorithm for closed-form pose estimation. Experimental results show that our DWG-Reg achieves state-of-the-art performance compared to existing non-deep learning and recent deep learning methods. Our source code will open at https://github.com/BiaoBiaoLi/DWG-Reg.

It is necessary for the development of self-driving to fulfill the requirements for safety, stability, and intelligence, especially in high-speed conditions. Therefore, the detection of pedestrians that may occur in highway scenarios during driving and understanding the meaning of their behaviors in advance are significantly important for the self-driving vehicle to make correct decisions. However, no existing datasets are available for behavior recognition in self-driving scenarios. In order to advance the task of interactive cognition between the vehicle and pedestrians, in this paper, we present a new dataset, called THU-IntrudBehavior, that collects lane intrusion behaviors of pedestrians that can be applied in real world highway scenarios. The dataset contains diverse behaviors of single or multiple pedestrians/cyclists that are simulated in different urban roads under various weather conditions. We describe annotations of each video and report several experimental results of baseline methods on our self-collected dataset. Our THU-IntrudBehavior dataset provides new support for behavior recognition in high-speed conditions for self-driving.

Spiking Neural Networks (SNNs) are a promising alternative to traditional deep learning methods since they perform event-driven information processing. However, a major drawback of SNNs is high inference latency. The efficiency of SNNs could be enhanced using compression methods such as pruning and quantization. Notably, SNNs, unlike their non-spiking counterparts, consist of a temporal dimension, the compression of which can lead to latency reduction. In this paper, we propose spatial and temporal pruning of SNNs. First, structured spatial pruning is performed by determining the layer-wise significant dimensions using principal component analysis of the average accumulated membrane potential of the neurons. This step leads to 10-14X model compression. Additionally, it enables inference with lower latency and decreases the spike count per inference. To further reduce latency, temporal pruning is performed by gradually reducing the timesteps while training. The networks are trained using surrogate gradient descent based backpropagation and we validate the results on CIFAR10 and CIFAR100, using VGG architectures. The spatiotemporally pruned SNNs achieve 89.04% and 66.4% accuracy on CIFAR10 and CIFAR100, respectively, while performing inference with 3-30X reduced latency compared to state-of-the-art SNNs. Moreover, they require 8-14X lesser compute energy compared to their unpruned standard deep learning counterparts. The energy numbers are obtained by multiplying the number of operations with energy per operation. These SNNs also provide 1-4% higher robustness against Gaussian noise corrupted inputs. Furthermore, we perform weight quantization and find that performance remains reasonably stable up to 5-bit quantization.

With the increase of data centers, the number of disks also grows rapidly. Therefore, the prediction of disk failures has become an important task for both academia and industry. Existing prediction schemes predict disk failure in the short prediction horizon or with a short time window. However, these schemes cannot achieve ideal performance for a long prediction horizon with a long time window. In this paper, we proposed a deep learning method that can effectively solve the above problems. We refine the Self-Monitoring, Analysis and Reporting Technology (SMART) attributes by using information entropy to select the most related attributes for prediction. Moreover, we proposed the Multiple Channel Convolutional Neural Network based LSTM (MCCNN-LSTM) model to predict whether disk failures will occur in a given disk in next few days. We further evaluate the MCCNN-LSTM model by comparing it with the state-of-the-art works. Extensive experiments show that our model can improve FDR (Fault Detection Rate) to 99.8% and reduce FAR (False Alarm Rate) to 0.2%.

Deep CNNs, though have achieved the state of the art performance in image classification tasks, remain a black-box to a human using them. There is a growing interest in explaining the working of these deep models to improve their trustworthiness. In this paper, we introduce a Posthoc Architecture-agnostic Concept Extractor (PACE) that automatically extracts smaller sub-regions of the image called concepts relevant to the black-box prediction. PACE tightly integrates the faithfulness of the explanatory framework to the black-box model. To the best of our knowledge, this is the first work that extracts class-specific discriminative concepts in a posthoc manner automatically. The PACE framework is used to generate explanations for two different CNN architectures trained for classifying the AWA2 and Imagenet-Birds datasets. Extensive human subject experiments are conducted to validate the human interpretability and consistency of the explanations extracted by PACE. The results from these experiments suggest that over 72% of the concepts extracted by PACE are human interpretable., Accepted at International Joint Conference on Neural Networks (IJCNN 2021)

Polandball comics are hand-drawn satirical content that portray personified countries in a unique style. Although certain parts of these comics, such as ball outlines, are easy to draw, some country flags are complex and require time, effort, and skill to depict correctly. Convolutional Neural Networks have shown success in image synthesis tasks but lack the ability to rescale and rotate images for texture mapping. The domain of Virtual Try-On Networks has made great progress in networks that can handle spatially invariant transforms. We show that similar methods can be used in another domain dependent on texture mapping, namely generating valid, rule-abiding Poland-ball characters given an outline and a country flag. To evaluate our method we make use of the Frechet Inception Distance where we achieved a score of 34.9. Multiple configurations of the model were evaluated to show that all modules used in the model contribute to the achieved performance. The main contributions in this paper are: a model that can be used by Polandball artists to aid in comic creation and a dataset with over 40,000 labeled Polandball characters for computer vision tasks.

Sequential recommendation that aims to predict user preference with historical user interactions becomes one of the most popular tasks in the recommendation area. The existing methods concentrated on user's sequential features among exposed items have achieved good performance. However, they only rely on single item prediction optimization to learn data representation, which ignores the association between context data and sequence data. In this paper, we propose a novel self-supervised learning based sequential recommendation network (SSLRN), which contrastively learns data correlation to promote data representation of users and items. We design two auxiliary contrastive learning tasks to regularize user and item representation based on mutual information maximization (MIM). In particular, the item contrastive learning captures sequential contrast feature with sequence-item MIM, and the user contrastive learning regularizes user latent representation with user-item MIM. We evaluate our model on five real-world datasets and the experimental results show that the proposed framework significantly and consistently outperforms state-of-the-art sequential recommendation techniques.

Electroencephalogram (EEG) signal has been widely applied in detecting human emotion. Individual differences limit the generalization in cross-subject classification since the release of emotion will be definitely different across persons even with the same emotion stimuli. Previous research utilizes domain adaptation to solve this problem in a leave-one-subject-out training that is learning common emotion-related features from many subjects and testing on a new subject, which requires abundant labeled data. This paper proposed a novel one-to-one domain adaptation method, the Task-specific Domain Adversarial Neural Network (T-DANN) which transfers knowledge from either one subject to predict on another subject or knowledge from one phase to predict on another phase within the same subject. Therefore, T-DANN is more flexible and requires much less data during training. T-DANN is an adversarial training method which adapts the conditional distribution between domains and adapts classification boundaries between classes simultaneously. Compared with data from different subjects, phase data from the same subject has much deeper correlation thus enhances the prediction of emotion in new phase. We evaluated our method on EEG emotion benchmark dataset SEED. The experiments showed that our proposed method outperformed other baseline methods in cross-subject adaptation. By cross-phase adaptation, our method achieved accuracy that approximately 3% lower than the state-of-the-art method but only used 1/14 labeled data, indicating the priority of our proposed method in real-time application.

Aspect-based sentiment analIsis (ABSA) aims to detect the sentiment polaritI of a specific aspect in an opinionated sentence. Current work focuses on exploiting the sIntactic tree to shorten the distance between the aspect term and context words. However, the “hard-pruning” strategI on the sIntactic tree maI lead to the reduction of importa nt sIntactic information. In this paper, we propose a novel sInt actical distance attention guided graph convolutional network (SDGCN) for ABSA. Our model is capable of fullI exploiting the sIntactic knowledge with a “soft pruning” strategI and learning crucial fine-grain sIntactic distance info rmation. AdditionallI, an effective denselI connected graph convolutional laIer is applied to avoid the over-sm oothing problem of standard GCN. Experiments conducted on three benchmark datasets show that our model achieves promising results comparing to the baseline models.