Search

Zero-shot learning (ZSL) refers to the design of predictive functions on new classes (unseen classes) of data that have never been seen during training. In a more practical scenario, generalized zero-shot learning (GZSL) requires predicting both seen and unseen classes accurately. In the absence of target samples, many GZSL models may overfit training data and are inclined to predict individuals as categories that have been seen in training. To alleviate this problem, we develop a parameter-wise adversarial training process that promotes robust recognition of seen classes while designing during the test a novel model perturbation mechanism to ensure sufficient sensitivity to unseen classes. Concretely, adversarial perturbation is conducted on the model to obtain instance-specific parameters so that predictions can be biased to unseen classes in the test. Meanwhile, the robust training encourages the model robustness, leading to nearly unaffected prediction for seen classes. Moreover, perturbations in the parameter space, computed from multiple individuals simultaneously, can be used to avoid the effect of perturbations that are too extreme and ruin the predictions. Comparison results on four benchmark ZSL data sets show the effective improvement that the proposed framework made on zero-shot methods with learned metrics. [ABSTRACT FROM AUTHOR]

The article introduces Split Variational Adversarial Training (SVAT) as a method to enhance risk-aware stock recommendation by leveraging adversarial perturbations. Topics include the development of SVAT to sensitize models to risky stock examples, generate risk indicators, and reduce investment risks by lowering model volatility.

Directly processing 3D point cloud data becomes dominant in classification and segmentation tasks. Present mainstream point based methods usually focus on learning in either geometric space (i. e. PointNet++) or semantic space (i. e. DGCNN). Owing to the irregular and unordered data property of point cloud, these methods still suffer from drawbacks of either ambiguous local features aggregation in geometric space or poor global features extraction in semantic space. While few prior works address these two defects simultaneously by fusing information from the dual spaces, we make a first attempt to develop a synergistic framework, called PointGS. Leveraging both the strength of geometric structure and semantic representation, PointGS establishes a mutual supervision mechanism that can bridge the two spaces and fuse complementary information for better analyzing 3D point cloud data. Compared with existing popular networks, our work attains obvious performance improvement on all three mainstream tasks without any sophisticated operations. • We make a first attempt to develop a fusion framework in point cloud analysis. • We build multi-scale geometric inputs and eschew pooling in semantic space. • We explore a new updating rule for simplifying semantic feature aggregation. [ABSTRACT FROM AUTHOR]

For image classification in computer vision, the performance of conventional deep neural networks (DNN) may usually drop when labeled training samples are limited. In this case, few-shot learning (FSL) or particularly zero-shot learning (ZSL), i.e. classification of target classes with few or zero labeled training samples, was proposed to imitate the strong learning ability of human. However, recent investigations show that most existing ZSL models may easily overfit and they tend to misclassify the target instance as one class seen in the training set. To alleviate this problem, we proposed an embedding based ZSL method with a self-focus mechanism, i.e. a focus-ratio that introduces the importance of each dimension, into the model optimization process. The objective function will be reconstructed according to these focus-ratios encouraging that the embedding model focus exclusively on important dimensions in the target space. As the self-focus module only takes part in the training process, the over-fitting knowledge is apportioned, and hence the rest embedding model can become more generalized for the new classes during test. Experimental results on four benchmarks, including AwA1, AwA2, aPY and CUB, show that our method outperforms the state-of-the-art methods on coarse-grained ZSL tasks while not affecting the performance of fine-grained ZSL. Additionally, several comparisons demonstrate the superiority of the proposed mechanism. [ABSTRACT FROM AUTHOR]

Generative Adversarial Networks (GANs) have drawn great attention recently since they are the powerful models to generate high-quality images. Although GANs have achieved great success, they usually suffer from unstable training and consequently may lead to the poor generations in some cases. Such drawback is argued mainly due to the difficulties in measuring the divergence between the highly complicated the real and fake data distributions, which are normally in the high-dimensional space. To tackle this problem, previous researchers attempt to search a proper divergence capable of measuring the departure of the complex distributions. In contrast, we attempt to alleviate this problem from a different perspective: while retaining the information as much as possible of the original high dimensional distributions, we learn and leverage an additional latent space where simple distributions are defined in a low-dimensional space; as a result, we can readily compute the distance between two simple distributions with an available divergence measurement. Concretely, to retain the data information, the mutual information is maximized between the variables for the high dimensional complex distributions and the low dimensional simple distributions. The departure of the resulting simple distributions are then measured in the original way of GANs. Additionally, for simplifying the optimization further, we optimize directly the lower bound for mutual information. Termed as SimpleGAN, we conduct the proposed approach over the several different baseline models, i.e., conventional GANs, DCGAN, WGAN-GP, WGAN-GP-res, and LSWGAN-GP on the benchmark CIFAR-10 and STL-10 datasets. SimpleGAN shows the obvious superiority on these baseline models. Furthermore, in comparison with the existing methods measuring directly the distribution departure in the high-dimensional space, our method clearly demonstrates its superiority. Finally, a series of experiments show the advantages of the proposed SimpleGAN. [ABSTRACT FROM AUTHOR]

We propose a new regularization method for deep learning based on the manifold adversarial training (MAT). Unlike previous regularization and adversarial training methods, MAT further considers the local manifold of latent representations. Specifically, MAT manages to build an adversarial framework based on how the worst perturbation could affect the statistical manifold in the latent space rather than the output space. Particularly, a latent feature space with the Gaussian Mixture Model (GMM) is first derived in a deep neural network. We then define the smoothness by the largest variation of Gaussian mixtures when a local perturbation is given around the input data point. On one hand, the perturbations are added in the way that would rough the statistical manifold of the latent space the worst. On the other hand, the model is trained to promote the manifold smoothness the most in the latent space. Importantly, since the latent space is more informative than the output space, the proposed MAT can learn a more robust and compact data representation, leading to further performance improvement. The proposed MAT is important in that it can be considered as a superset of one recently-proposed discriminative feature learning approach called center loss. We conduct a series of experiments in both supervised and semi-supervised learning on four benchmark data sets, showing that the proposed MAT can achieve remarkable performance, much better than those of the state-of-the-art approaches. In addition, we present a series of visualization which could generate further understanding or explanation on adversarial examples. [ABSTRACT FROM AUTHOR]

Field classification is a new extension of traditional classification frameworks that attempts to utilize consistent information from a group of samples (termed fields). By forgoing the independent identically distributed (i.i.d.) assumption, field classification can achieve remarkably improved accuracy compared to traditional classification methods. Most studies of field classification have been conducted on traditional machine learning methods. In this paper, we propose integration with a Bayesian framework, for the first time, in order to extend field classification to deep learning and propose two novel deep neural network architectures: the Field Deep Perceptron (FDP) and the Field Deep Convolutional Neural Network (FDCNN). Specifically, we exploit a deep perceptron structure, typically a 6-layer structure, where the first 3 layers remove (learn) a 'style' from a group of samples to map them into a more discriminative space and the last 3 layers are trained to perform classification. For the FDCNN, we modify the AlexNet framework by adding style transformation layers within the hidden layers. We derive a novel learning scheme from a Bayesian framework and design a novel and efficient learning algorithm with guaranteed convergence for training the deep networks. The whole framework is interpreted with visualization features showing that the field deep neural network can better learn the style of a group of samples. Our developed models are also able to achieve transfer learning and learn transformations for newly introduced fields. We conduct extensive comparative experiments on benchmark data (including face, speech, and handwriting data) to validate our learning approach. Experimental results demonstrate that our proposed deep frameworks achieve significant improvements over other state-of-the-art algorithms, attaining new benchmark performance. [ABSTRACT FROM AUTHOR]

Dimensionality Reduction (DR) is a fundamental topic of pattern classification and machine learning. For classification tasks, DR is typically employed as a pre-processing step, succeeded by an independent classifier training stage. However, such independent operation of the two stages often limits the final classification performance notably, as the generated subspace may not be maximally beneficial or appropriate to the learning task at hand. This problem is further accentuated for high-dimensional data classification in situations of the limited number of samples. To address this problem, we develop a novel joint learning model for classification, referred to as two-layer mixture of factor analyzers with joint factor loading (2L-MJFA). Specifically, the model adopts a special two-layer mixture or a mixture of mixtures structure, where each component represents each specific class as a mixture of factor analyzers (MFA). Importantly, all the involved factor analyzers are intentionally designed so that they share the same loading matrix. This, apart from operating as the DR matrix, largely reduces the parameters and makes the proposed algorithm very suitable to small dataset situations. Additionally, we propose a modified expectation maximization algorithm to train the proposed model. A series of simulation experiments demonstrate that what we propose significantly outperforms other state-of-the-art algorithms on various benchmark datasets. Finally, since factor analyzers are closely linked with Auto-encoder networks, the proposed idea could be of particular utility to the community of neural networks. [ABSTRACT FROM AUTHOR]

• We present a multi-stage residual feature pyramid gating strategy which designs soft gates to select and reinforce multi-scale semantic features. • We introduce an image pyramid gating strategy to preserve natural information to enhance network discrimination in an attention-based fashion. • Our method attains the best performance without pre-training and can be readily combined with general frameworks without structure modifications. Specifically, the prediction accuracy consistently achieves over 3% improvements than the baseline (without pre-training) under various model configurations. In this work, we propose a comprehensive aggregated residual gating structure, the Pyramid GAting Network (PGA-Net) for human pose estimation which can select, distill, and fuse semantic level and natural level information from multiple scales. In comparison, through utilizing multi-scale features, most existing state-of-the-art pose estimation methods are still limited in three aspects. First, multi-scale features contain massively redundant information, which is unfortunately not distilled by most existing approaches. Second, preferring deeper network structures to extract strong semantic features, the conventional methods often ignore original texture information fusion. Third, to attain a good parameter initialization, the current methods heavily rely on pre-training, which is very time-consuming or even unavailable. While better coping with the above problems, our proposed PGA-Net distills high-level semantic features and replenishes low-level original information to reinforce module representation capability. Meanwhile, PGA-Net demonstrates notable training stability and superior performance even without pre-training. Extensive experiments demonstrate that our method consistently outperforms previous approaches even without pre-training, enabling thus an end-to-end model training from scratch. In COCO benchmark, PGA-Net consistently achieves over 3% improvements than the baseline (without pre-training) under various model configurations. 1 1 The code is released at https://github.com/ssr0512/PGA-Net [ABSTRACT FROM AUTHOR]

Nonconvex optimization problems have always been one focus in deep learning, in which many fast adaptive algorithms based on momentum are applied. However, the full gradient computation of high-dimensional feature vector in the above tasks become prohibitive. To reduce the computation cost for optimizers on nonconvex optimization problems typically seen in deep learning, this work proposes a randomized block-coordinate adaptive optimization algorithm, named RAda, which randomly picks a block from the full coordinates of the parameter vector and then sparsely computes its gradient. We prove that RAda converges to a δ -accurate solution with the stochastic first-order complexity of O (1 / δ 2) , where δ is the upper bound of the gradient's square, under nonconvex cases. Experiments on public datasets including CIFAR-10, CIFAR-100, and Penn TreeBank, verify that RAda outperforms the other compared algorithms in terms of the computational cost. [ABSTRACT FROM AUTHOR]

We propose a fast algorithm for approximate matching of large graphs. Previous graph matching algorithms suffer from high computational complexity and therefore do not have good scalability. By using a new doubly stochastic projection, for matching two weighted graphs of n nodes, our algorithm has time complexity only O ( n 3 ) per iteration and space complexity O ( n 2 ) . We proved that our algorithm converges at a super-logarithmic rate. Experiments on large synthetic and real graphs (over 1000 nodes) were conducted to evaluate the performance of various algorithms. Results show that due to its fast convergence, our algorithm is more than an order of magnitude faster than the previous state-of-the-art algorithms, while maintaining comparable accuracy in large graph matching. [ABSTRACT FROM AUTHOR]

• We present a timely and comprehensive survey on robust adversarial training. • This survey offers the fundamentals of adversarial training, a unified theory that can be used to interpret various methods, and a comprehensive summarization of different methodologies. • This survey also addresses three important research focus in adversarial training: interpretability, robust generalization, and robustness evaluation, which can stimulate future inspirations as well as research outlook. Deep neural networks have achieved remarkable success in machine learning, computer vision, and pattern recognition in the last few decades. Recent studies, however, show that neural networks (both shallow and deep) may be easily fooled by certain imperceptibly perturbed input samples called adversarial examples. Such security vulnerability has resulted in a large body of research in recent years because real-world threats could be introduced due to the vast applications of neural networks. To address the robustness issue to adversarial examples particularly in pattern recognition, robust adversarial training has become one mainstream. Various ideas, methods, and applications have boomed in the field. Yet, a deep understanding of adversarial training including characteristics, interpretations, theories, and connections among different models has remained elusive. This paper presents a comprehensive survey trying to offer a systematic and structured investigation on robust adversarial training in pattern recognition. We start with fundamentals including definition, notations, and properties of adversarial examples. We then introduce a general theoretical framework with gradient regularization for defending against adversarial samples - robust adversarial training with visualizations and interpretations on why adversarial training can lead to model robustness. Connections will also be established between adversarial training and other traditional learning theories. After that, we summarize, review, and discuss various methodologies with defense/training algorithms in a structured way. Finally, we present analysis, outlook, and remarks on adversarial training. [ABSTRACT FROM AUTHOR]

Learning nonstationary data with concept drift has received much attention in machine learning and been an active topic in ensemble learning. Specifically, batch growing ensemble methods present one important direction for dealing with concept drift involved in nonstationary data. However, current batch growing ensemble methods combine all the available component classifiers only, each trained independently from a batch of non-stationary data. They simply discard interim ensembles and hence may lose useful information obtained from the fine-tuned interim ensembles. Distinctively, we introduce a comprehensive hierarchical approach called Dynamic Ensemble of Ensembles (DE 2 ). The novel method combines classifiers as an ensemble of all the interim ensembles dynamically from consecutive batches of nonstationary data. DE 2 includes two key stages: component classifiers and interim ensembles are dynamically trained; and the final ensemble is then learned by exponentially-weighted averaging with available experts, i.e., interim ensembles. Moreover. we engage Sparsity Learning to choose component classifiers selectively and intelligently. We also incorporate the techniques of Dynamic Weighted Majority, and Learn ++ .NSE for better integrating different classifiers dynamically. We perform experiments with two benchmark test sets in real nonstationary environments, and compare our DE 2 method to other conventional competitive ensemble methods. Experimental results confirm that our approach consistently leads to better performance and has promising generalization ability for learning in nonstationary environments. [ABSTRACT FROM AUTHOR]

Semi-supervised learning (SSL) is a typical learning paradigms training a model from both labeled and unlabeled data. The traditional SSL models usually assume unlabeled data are relevant to the labeled data, i.e., following the same distributions of the targeted labeled data. In this paper, we address a different, yet formidable scenario in semi-supervised classification, where the unlabeled data may contain irrelevant data to the labeled data. To tackle this problem, we develop a maximum margin model, named tri-class support vector machine (3C-SVM), to utilize the available training data, while seeking a hyperplane for separating the targeted data well. Our 3C-SVM exhibits several characteristics and advantages. First, it does not need any prior knowledge and explicit assumption on the data relatedness. On the contrary, it can relieve the effect of irrelevant unlabeled data based on the logistic principle and maximum entropy principle. That is, 3C-SVM approaches an ideal classifier. This classifier relies heavily on labeled data and is confident on the relevant data lying far away from the decision hyperplane, while maximally ignoring the irrelevant data, which are hardly distinguished. Second, theoretical analysis is provided to prove that in what condition, the irrelevant data can help to seek the hyperplane. Third, 3C-SVM is a generalized model that unifies several popular maximum margin models, including standard SVMs, Semi-supervised SVMs (S 3 VMs), and SVMs learned from the universum ( U -SVMs) as its special cases. More importantly, we deploy a concave–convex produce to solve the proposed 3C-SVM, transforming the original mixed integer programming, to a semi-definite programming relaxation, and finally to a sequence of quadratic programming subproblems, which yields the same worst case time complexity as that of S 3 VMs. Finally, we demonstrate the effectiveness and efficiency of our proposed 3C-SVM through systematical experimental comparisons. [ABSTRACT FROM AUTHOR]

Abstract: Classifier ensemble has been broadly studied in two prevalent directions, i.e., to diversely generate classifier components, and to sparsely combine multiple classifiers. While most current approaches are emphasized on either sparsity or diversity only, we investigate classifier ensemble focused on both in this paper. We formulate the classifier ensemble problem with the sparsity and diversity learning in a general mathematical framework, which proves beneficial for grouping classifiers. In particular, derived from the error-ambiguity decomposition, we design a convex ensemble diversity measure. Consequently, accuracy loss, sparseness regularization, and diversity measure can be balanced and combined in a convex quadratic programming problem. We prove that the final convex optimization leads to a closed-form solution, making it very appealing for real ensemble learning problems. We compare our proposed novel method with other conventional ensemble methods such as Bagging, least squares combination, sparsity learning, and AdaBoost, extensively on a variety of UCI benchmark data sets and the Pascal Large Scale Learning Challenge 2008 webspam data. Experimental results confirm that our approach has very promising performance. [Copyright &y& Elsevier]

Abstract: We consider the classifier ensemble problem in this paper. Due to its superior performance to individual classifiers, class ensemble has been intensively studied in the literature. Generally speaking, there are two prevalent research directions on this, i.e., to diversely generate classifier components, and to sparsely combine multiple classifiers. While most current approaches are emphasized on either sparsity or diversity only, we investigate the classifier ensemble by learning both sparsity and diversity simultaneously. We manage to formulate the classifier ensemble problem with the sparsity or/and diversity learning in a general framework. In particular, the classifier ensemble with sparsity and diversity can be represented as a mathematical optimization problem. We then propose a heuristic algorithm, capable of obtaining ensemble classifiers with consideration of both sparsity and diversity. We exploit the genetic algorithm, and optimize sparsity and diversity for classifier selection and combination heuristically and iteratively. As one major contribution, we introduce the concept of the diversity contribution ability so as to select proper classifier components and evolve classifier weights eventually. Finally, we compare our proposed novel method with other conventional classifier ensemble methods such as Bagging, least squares combination, sparsity learning, and AdaBoost, extensively on UCI benchmark data sets and the Pascal Large Scale Learning Challenge 2008 webspam data. The experimental results confirm that our approach leads to better performance in many aspects. [Copyright &y& Elsevier]

Abstract: Multivariate Gaussian distribution is a popular assumption in many pattern recognition tasks. The quadratic discriminant function (QDF) is an effective classification approach based on this assumption. An improved algorithm, called modified QDF (or MQDF in short) has achieved great success and is widely recognized as the state-of-the-art method in character recognition. However, because both of the two approaches estimate the mean and covariance by the maximum-likelihood estimation (MLE), they often lead to the loss of the classification accuracy when the number of the training samples is small. To attack this problem, in this paper, we engage the graphical lasso method to estimate the covariance and propose a new classification method called the graphical lasso quadratic discriminant function (GLQDF). By exploiting a coordinate descent procedure for the lasso, GLQDF can estimate the covariance matrix (and its inverse) more precisely. Experimental results demonstrate that the proposed method can perform better than the competitive methods on two artificial and nine real datasets (including both benchmark digit and Chinese character data). [Copyright &y& Elsevier]

Metric learning has been widely studied in machine learning due to its capability to improve the performance of various algorithms. Meanwhile, multi-task learning usually leads to better performance by exploiting the shared information across all tasks. In this paper, we propose a novel framework to make metric learning benefit from jointly training all tasks. Based on the assumption that discriminative information is retained in a common subspace for all tasks, our framework can be readily used to extend many current metric learning methods. In particular, we apply our framework on the widely used Large Margin Component Analysis (LMCA) and yield a new model called multi-task LMCA. It performs remarkably well compared to many competitive methods. Besides, this method is able to learn a low-rank metric directly, which effects as feature reduction and enables noise compression and low storage. A series of experiments demonstrate the superiority of our method against three other comparison algorithms on both synthetic and real data. [ABSTRACT FROM AUTHOR]

Abstract: Linear Discriminant Analysis (LDA) is an important dimensionality reduction algorithm, but its performance is usually limited on multi-class data. Such limitation is incurred by the fact that LDA actually maximizes the average divergence among classes, whereby similar classes with smaller divergence tend to be merged in the subspace. To address this problem, we propose a novel dimensionality reduction method called Maxi-Min Discriminant Analysis (MMDA). In contrast to the traditional LDA, MMDA attempts to find a low-dimensional subspace by maximizing the minimal (worst-case) divergence among classes. This “minimal” setting overcomes the problem of LDA that tends to merge similar classes with smaller divergence when used for multi-class data. We formulate MMDA as a convex problem and further as a large-margin learning problem. One key contribution is that we design an efficient online learning algorithm to solve the involved problem, making the proposed method applicable to large scale data. Experimental results on various datasets demonstrate the efficiency and the efficacy of our proposed method against five other competitive approaches, and the scalability to the data with thousands of classes. [Copyright &y& Elsevier]

The ECOC technique is a powerful tool to learn and combine multiple binary learners for multi-class classification. It generally involves three steps: coding, dichotomizers learning, and decoding. In previous ECOC methods, the coding step and the dichotomizers learning step are usually performed independently. This simplifies the learning problem but may lead to unsatisfactory decoding results. To solve this problem, we propose a novel model for learning the ECOC matrix and dichotomizers jointly from data. We formulate the model as a nonlinear programming problem and develop an efficient alternating minimization algorithm to solve it. Specifically, for fixed ECOC matrix, our model is decomposed into a group of mutually independent quadratic programming problems; while for fixed dichotomizers, it is a difference of convex functions problem and can be easily solved using the concave--convex procedure algorithm. Our experimental results on ten data sets from the UCI machine learning repository demonstrated the advantage of our model over state-of-the-art ECOC methods. [ABSTRACT FROM AUTHOR]

Abstract: This paper provides a sparse learning algorithm for Support Vector Classification (SVC), called Sparse Support Vector Classification (SSVC), which leads to sparse solutions by automatically setting the irrelevant parameters exactly to zero. SSVC adopts the L 0-norm regularization term and is trained by an iteratively reweighted learning algorithm. We show that the proposed novel approach contains a hierarchical-Bayes interpretation. Moreover, this model can build up close connections with some other sparse models. More specifically, one variation of the proposed method is equivalent to the zero-norm classifier proposed in (); it is also an extended and more flexible framework in parallel with the Sparse Probit Classifier proposed by . Theoretical justifications and experimental evaluations on two synthetic datasets and seven benchmark datasets show that SSVC offers competitive performance to SVC but needs significantly fewer Support Vectors. [ABSTRACT FROM AUTHOR]

Abstract: Kernel methods have been widely used in pattern recognition. Many kernel classifiers such as Support Vector Machines (SVM) assume that data can be separated by a hyperplane in the kernel-induced feature space. These methods do not consider the data distribution and are difficult to output the probabilities or confidences for classification. This paper proposes a novel Kernel-based Maximum A Posteriori (KMAP) classification method, which makes a Gaussian distribution assumption instead of a linear separable assumption in the feature space. Robust methods are further proposed to estimate the probability densities, and the kernel trick is utilized to calculate our model. The model is theoretically and empirically important in the sense that: (1) it presents a more generalized classification model than other kernel-based algorithms, e.g., Kernel Fisher Discriminant Analysis (KFDA); (2) it can output probability or confidence for classification, therefore providing potential for reasoning under uncertainty; and (3) multi-way classification is as straightforward as binary classification in this model, because only probability calculation is involved and no one-against-one or one-against-others voting is needed. Moreover, we conduct an extensive experimental comparison with state-of-the-art classification methods, such as SVM and KFDA, on both eight UCI benchmark data sets and three face data sets. The results demonstrate that KMAP achieves very promising performance against other models. [Copyright &y& Elsevier]

Abstract: Time series prediction, especially financial time series prediction, is a challenging task in machine learning. In this issue, the data are usually non-stationary and volatile in nature. Because of its good generalization power, the support vector regression (SVR) has been widely applied in this application. The standard SVR employs a fixed -tube to tolerate noise and adopts the -norm ( or 2) to model the functional complexity of the whole data set. One problem of the standard SVR is that it considers data in a global fashion only. Therefore it may lack the flexibility to capture the local trend of data; this is a critical aspect of volatile data, especially financial time series data. Aiming to attack this issue, we propose the localized support vector regression (LSVR) model. This novel model is demonstrated to provide a systematic and automatic scheme to adapt the margin locally and flexibly; while the margin in the standard SVR is fixed globally. Therefore, the LSVR can tolerate noise adaptively. The proposed LSVR is promising in the sense that it not only captures the local information in data, but more importantly, it establishes connection with several models. More specifically: (1) it can be regarded as the regression extension of a recently proposed promising classification model, the Maxi-Min Margin Machine; (2) it incorporates the standard SVR as a special case under certain mild assumptions. We provide both theoretical justifications and empirical evaluations for this novel model. The experimental results on synthetic data and real financial data demonstrate its advantages over the standard SVR. [Copyright &y& Elsevier]

Recent advances in both lightweight deep learning algorithms and edge computing increasingly enable multiple model inference tasks to be conducted concurrently on resource-constrained edge devices, allowing us to achieve one goal collaboratively rather than getting high quality in each standalone task. However, the high overall running latency for performing multi-model inferences always negatively affects the real-time applications. To combat latency, the algorithms should be optimized to minimize the latency for multi-model deployment without compromising the safety-critical situation. This work focuses on the real-time task scheduling strategy for multi-model deployment and investigating the model inference using an open neural network exchange (ONNX) runtime engine. Then, an application deployment strategy is proposed based on the container technology and inference tasks are scheduled to different containers based on the scheduling strategies. Experimental results show that the proposed solution is able to significantly reduce the overall running latency in real-time applications. [ABSTRACT FROM AUTHOR]

Whilst adversarial training has been shown as a promising wisdom to promote model robustness in computer vision and machine learning, adversarially trained models often suffer from poor robust generalization on unseen adversarial examples. Namely, there still remains a big gap between the performance on training and test adversarial examples. In this paper, we propose to tackle this issue from a new perspective of the inter-feature relationship. Specifically, we aim to generate adversarial examples which maximize the loss function while maintaining the inter-feature relationship of natural data as well as penalizing the correlation distance between natural features and adversarial counterparts. As a key contribution, we prove that training with such examples while penalizing the distance between correlations can help promote both the generalization on natural and adversarial examples theoretically. We empirically validate our method through extensive experiments over different vision datasets (CIFAR-10, CIFAR-100, and SVHN), against several competitive methods. Our method substantially outperforms the baseline adversarial training by a large margin, especially for PGD20 on CIFAR-10, CIFAR-100, and SVHN with around 20%, 15% and 29% improvements. [ABSTRACT FROM AUTHOR]

Whilst adversarial training has been proven to be one most effective defending method against adversarial attacks for deep neural networks, it suffers from over-fitting on training adversarial data and thus may not guarantee the robust generalization. This may result from the fact that the conventional adversarial training methods generate adversarial perturbations usually in a supervised way so that the resulting adversarial examples are highly biased towards the decision boundary, leading to an inhomogeneous data distribution. To mitigate this limitation, we propose to generate adversarial examples from a perturbation diversity perspective. Specifically, the generated perturbed samples are not only adversarial but also diverse so as to certify robust generalization and significant robustness improvement through a homogeneous data distribution. We provide theoretical and empirical analysis, establishing a foundation to support the proposed method. As a major contribution, we prove that promoting perturbations diversity can lead to a better robust generalization bound. To verify our methods' effectiveness, we conduct extensive experiments over different datasets (e.g., CIFAR-10, CIFAR-100, SVHN) with different adversarial attacks (e.g., PGD, CW). Experimental results show that our method outperforms other state-of-the-art (e.g., PGD and Feature Scattering) in robust generalization performance. • Improved adversarial robustness from a diversity perspective. • Showed that robust generalization can be upper bounded by a diversity term. • Achieved competitive performace against adversarial examples. [ABSTRACT FROM AUTHOR]

Object detection has wide applications in intelligent systems and sensor applications. Compared with two stage detectors, recent one stage counterparts are capable of running more efficiently with comparable accuracy, which satisfy the requirement of real-time processing. To further improve the accuracy of one stage single shot detector (SSD), we propose a novel Multi-Path fusion Single Shot Detector (MPSSD). Different from other feature fusion methods, we exploit the connection among different scale representations in a pyramid manner. We propose feature fusion module to generate new feature pyramids based on multiscale features in SSD, and these pyramids are sent to our pyramid aggregation module for generating final features. These enhanced features have both localization and semantics information, thus improving the detection performance with little computation cost. A series of experiments on three benchmark datasets PASCAL VOC2007, VOC2012, and MS COCO demonstrate that our approach outperforms many state-of-the-art detectors both qualitatively and quantitatively. In particular, for input images with size 512 × 512, our method attains mean Average Precision (mAP) of 81.8% on VOC2007 t e s t , 80.3% on VOC2012 t e s t , and 33.1% mAP on COCO t e s t - d e v 2015. [ABSTRACT FROM AUTHOR]

• cylinder primitive extraction is addressed as a clustering problem of possible cylinder candidates; • candidates are generated by cutting the scene through several random planes; • a novel cylinder similarity function based on dual quaternion algebra is defined; • the clustering process, based on Game Theory, inherently discards incompatible candidates; • the proposed method can deal with noisy, partially occluded and non-oriented point clouds. Generative Adversarial Networks (GAN) receive great attention recently due to its excellent performance in image generation, transformation, and super-resolution. However, less emphasis or study has been put on GAN for classification with super-resolution. Moreover, though GANs may fabricate images which perceptually looks realistic, they usually fabricate some fake details especially in character data; this would impose further difficulties when they are input for classification. In this paper, we propose a novel Generative Adversarial Classifier (GAC) for low-resolution handwriting character recognition. Specifically, we design an additional classifier component in GAC, leading to a novel three-player GAN model which is not only able to generate high-quality super-resolved images, but also favorable for classification. Experimental results show that our proposed method can obtain remarkable performance in handwriting characters with 8 × super-resolution, achieving new state-of-the-art on benchmark dataset CASIA-HWDB1.1, and MNIST. [ABSTRACT FROM AUTHOR]

Artificial Intelligence (AI) has been witnessed as one of the fastest growing enabling technologies to optimise the performance of various systems. With the rich data generated from these IoT systems, the world is entering into a data-driven services, management, and manufacturing era. With the wide use of IoT systems, mobile edge computing is the de facto tool to ensure real-time data processing and services provided by these IoT systems. [Extracted from the article]

Continual learning, a widespread ability in people and animals, aims to learn and acquire new knowledge and skills continuously. Catastrophic forgetting usually occurs in continual learning when an agent attempts to learn different tasks sequentially without storing or accessing previous task information. Unfortunately, current learning systems, e.g., neural networks, are prone to deviate the weights learned in previous tasks after training new tasks, leading to catastrophic forgetting, especially in a sequential multi-tasks scenario. To address this problem, in this paper, we propose to overcome catastrophic forgetting with the focus on learning a series of robotic tasks sequentially. Particularly, a novel hierarchical neural network's framework called Encoding Primitives Generation Policy Learning (E-PGPL) is developed to enable continual learning with two components. By employing a variational autoencoder to project the original state space into a meaningful low-dimensional feature space, representative state primitives could be sampled to help learn corresponding policies for different tasks. In learning a new task, the feature space is required to be close to the previous ones so that previously learned tasks can be protected. Extensive experiments on several simulated robotic tasks demonstrate our method's efficacy to learn control policies for handling sequentially arriving multi-tasks, delivering improvement substantially over some other continual learning methods, especially for the tasks with more diversity. • Continual learning is studied in robotic tasks to avoid catastrophic forgetting. • State primitives are restricted by previous ones to remember old tasks. • Different robotic tasks are learned with different policy networks. [ABSTRACT FROM AUTHOR]

Correlation filter has been proven to be an effective tool for a number of approaches in visual tracking, particularly for seeking a good balance between tracking accuracy and speed. However, correlation filter-based models are susceptible to wrong updates stemming from inaccurate tracking results. To date, very little effort has been devoted towards handling the correlation filter update problem. In this paper, we propose a novel approach to address the correlation filter update problem. In our approach, we update and maintain multiple correlation filter models in parallel, and we use deep reinforcement learning for the selection of an optimal correlation filter model among them. To facilitate the decision process in an efficient manner, we propose a decision-net to deal with target appearance modeling, which is trained through hundreds of challenging videos using proximal policy optimization and a lightweight learning network. An exhaustive evaluation of the proposed approach on the OTB100 and OTB2013 benchmarks shows that the approach is effective enough to achieve the average success rate of 62.3% and the average precision score of 81.2%, both exceeding the performance of traditional correlation filter-based trackers. [ABSTRACT FROM AUTHOR]

• To our knowledge, it is the first work to study intensively mixup methods within the GAN framework for adversarial data augmentation in the long-tailed oracle character recognition. • We propose two novel mixup-based methods, including Repatch to generalize samples in the mixup generator and the TailMix mechanism to replenish samples in the tail classes for the adversarial generator. • We extensively validate the proposed model on three benchmark oracle datasets (Oracle-AYNU, OBC306 and Oracle-20K). Experimental results show that our model can yield the state-of-the-art performance as opposed to other existing oracle recognition methods. Deciphering oracle bone script is of great significance to the study of ancient Chinese culture as well as archaeology. Although recent studies on oracle character recognition have made substantial progress, they still suffer from the long-tailed data situation that results in a noticeable performance drop on the tail classes. To mitigate this issue, we propose a generative adversarial framework to augment oracle characters in the problematic classes. In this framework, the generator produces synthetic data through convex combinations of all the available samples in the corresponding classes, and is further optimized through adversarial learning with the classifier and simultaneously the discriminator. Meanwhile, we introduce Repatch to generalize samples in the generator. Since tail classes do not have sufficient data for convex combinations, we propose the TailMix mechanism to generate suitable tail class samples from other classes. Experimental results show that our proposed algorithm obtains remarkable performance in oracle character recognition and achieves new state-of-the-art average (total) accuracy with 86.03% (89.46%), 86.54% (93.86%), 95.22% (96.17%) on the three datasets Oracle-AYNU, OBC306 and Oracle-20K, respectively. [ABSTRACT FROM AUTHOR]

Over the years, Machine Learning models have been successfully employed on neuroimaging data for accurately predicting brain age. Deviations from the healthy brain aging pattern are associated with the accelerated brain aging and brain abnormalities. Hence, efficient and accurate diagnosis techniques are required to elicit accurate brain age estimations. Several contributions have been reported in the past for this purpose, resorting to different data-driven modeling methods. Recently, deep neural networks (also referred to as deep learning) have become prevalent in manifold neuroimaging studies, including brain age estimation. In this review, we offer a comprehensive analysis of the literature related to the adoption of deep learning for brain age estimation with neuroimaging data. We detail and analyze different deep learning architectures used for this application, pausing at research works published to date quantitatively exploring their application. We also examine different brain age estimation frameworks, comparatively exposing their advantages and weaknesses. Finally, the review concludes with an outlook towards future directions that should be followed by prospective studies. The ultimate goal of this paper is to establish a common and informed reference for newcomers and experienced researchers willing to approach brain age estimation by using deep learning models. • We categorize brain age estimation models based on the deep learning architectures. • We classify deep learning architectures based on the slice/voxel-based input data. • We examine the modalities, size of data, performance and the datasets used. • We enumerate several challenges and trace directions that can be explored further. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel feature extraction technique, termed Three-Dimensional Local Energy-Based Shape Histogram (3D-LESH), and exploit it to detect breast cancer in volumetric medical images. The technique is incorporated as part of an intelligent expert system that can aid medical practitioners making diagnostic decisions. Analysis of volumetric images, slice by slice, is cumbersome and inefficient. Hence, 3D-LESH is designed to compute a histogram-based feature set from a local energy map, calculated using a phase congruency (PC) measure of volumetric Magnetic Resonance Imaging (MRI) scans in 3D space. 3D-LESH features are invariant to contrast intensity variations within different slices of the MRI scan and are thus suitable for medical image analysis. The contribution of this article is manifold. First, we formulate a novel 3D-LESH feature extraction technique for 3D medical images to analyse volumetric images. Further, the proposed 3D-LESH algorithmis, for the first time, applied to medical MRI images. The final contribution is the design of an intelligent clinical decision support system (CDSS) as a multi-stage approach, combining novel 3D-LESH feature extraction with machine learning classifiers, to detect cancer from breast MRI scans. The proposed system applies contrast-limited adaptive histogram equalisation (CLAHE) to the MRI images before extracting 3D-LESH features. Furthermore, a selected subset of these features is fed into a machine-learning classifier, namely, a support vector machine (SVM), an extreme learning machine (ELM) or an echo state network (ESN) classifier, to detect abnormalities and distinguish between different stages of abnormality. We demonstrate the performance of the proposed technique by its application to benchmark breast cancer MRI images. The results indicate high-performance accuracy of the proposed system (98%±0.0050, with an area under a receiver operating charactertistic curve value of 0.9900 ± 0.0050) with multiple classifiers. When compared with the state-of-the-art wavelet-based feature extraction technique, statistical analysis provides conclusive evidence of the significance of our proposed 3D-LESH algorithm. [ABSTRACT FROM AUTHOR]

Abstract Random forests algorithms have been widely used in many classification and regression applications. However, the theory of random forests lags far behind their applications. In this paper, we propose a novel random forests classification algorithm based on cooperative game theory. The Banzhaf power index is employed to evaluate the power of each feature by traversing possible feature coalitions. Hence, we call the proposed algorithm Banzhaf random forests (BRFs). Unlike the previously used information gain ratio, which only measures the power of each feature for classification and pays less attention to the intrinsic structure of the feature variables, the Banzhaf power index can measure the importance of each feature by computing the dependency among the group of features. More importantly, we have proved the consistency of BRFs, which narrows the gap between the theory and applications of random forests. Extensive experiments on several UCI benchmark data sets and three real world applications show that BRFs perform significantly better than existing consistent random forests on classification accuracy, and better than or at least comparable with Breiman’s random forests, support vector machines (SVMs) and k-nearest neighbors (KNNs) classifiers. Highlights • A novel random classification forests algorithm, called Banzhaf random forests (BRFs), is proposed. • The Banzhaf power index is employed to evaluate the power of each feature by traversing possible feature coalitions. • The consistency of BRFs is proved. [ABSTRACT FROM AUTHOR]

In this paper, we develop a novel transformer-based generative adversarial neural network called U-Transformer for generalized image outpainting problems. Different from most present image outpainting methods conducting horizontal extrapolation, our generalized image outpainting could extrapolate visual context all-side around a given image with plausible structure and details even for complicated scenery, building, and art images. Specifically, we design a generator as an encoder-to-decoder structure embedded with the popular Swin Transformer blocks. As such, our novel neural network can better cope with image long-range dependencies which are crucially important for generalized image outpainting. We propose additionally a U-shaped structure and multi-view Temporal Spatial Predictor (TSP) module to reinforce image self-reconstruction as well as unknown-part prediction smoothly and realistically. By adjusting the predicting step in the TSP module in the testing stage, we can generate arbitrary outpainting size given the input sub-image. We experimentally demonstrate that our proposed method could produce visually appealing results for generalized image outpainting against the state-of-the-art image outpainting approaches. • To our best knowledge, the proposed U-Transformer is the first transformer-based image outpainting framework able to obtain global features and keep high resolutions. • The TSP module leverages the multi-view LSTM network and self-attention blocks which can transfer incomplete latent features and support the generation of arbitrary output resolutions. • U-Transformer produces visually appealing results against the state-of-the-art image outpainting networks. It also outperforms the competitors in most terms of quantitative results. [ABSTRACT FROM AUTHOR]

Knowledge resources, e.g. knowledge graphs, which formally represent essential semantics and information for logic inference and reasoning, can compensate for the unawareness nature of many natural language processing techniques based on deep neural networks. This paper provides a focused review of the emerging but intriguing topic that fuses quality external knowledge resources in improving the performance of natural language processing tasks. Existing methods and techniques are summarised in three main categories: (1) static word embeddings, (2) sentence-level deep learning models, and (3) contextualised language representation models, depending on when, how and where external knowledge is fused into the underlying learning models. We focus on the solutions to mitigate two issues: knowledge inclusion and inconsistency between language and knowledge. Details on the design of each representative method, as well as their strength and limitation, are discussed. We also point out some potential future directions in view of the latest trends in natural language processing research. • First comprehensive review on knowledge fusion into deep learning for NLP. • A taxonomy for the related research and extensive overview of the state-of-the-art. • A focus on the theoretical formulation and optimisation methods. • Harmonise the mathematical notations across a plethora of publications. [ABSTRACT FROM AUTHOR]

3D point clouds are gradually becoming more widely used in the medical field, however, they are rarely used for 3D representation of intracranial vessels and aneurysms due to the time-consuming data reconstruction. In this paper, we simulate the incomplete intracranial vessels (including aneurysms) in the actual collection from different angles, then propose Multi-Scope Feature Extraction Network (MSENet) for Intracranial Aneurysm 3D Point Cloud Completion. MSENet adopts a multi-scope feature extraction encoder to extract the global features from the incomplete point cloud. This encoder utilizes different scopes to fuse the neighborhood information for each point fully. Then a folding-based decoder is applied to obtain the complete 3D shape. To enable the decoder to intuitively match the original geometric structure, we engage the original points coordinates input to perform residual linking. Finally, we merge and sample the complete but coarse point cloud from the decoder to obtain the final refined complete 3D point cloud shape. We conduct extensive experiments on both 3D intracranial aneurysm datasets and general 3D vision PCN datasets. The results demonstrate the effectiveness of the proposed method on three evaluation metrics compared to baseline: our model increases the F-score to 0.379 ( + 21.1 % )/ 0.320 ( + 7.7 % ), reduces Chamfer Distance score to 0.998 ( − 33.8 % )/ 0.974 ( − 6.4 % ), and reduces the Earth Mover's Distance to 2.750 ( 17.8 % )/ 2.858 ( − 0.8 % ). [ABSTRACT FROM AUTHOR]

Link prediction is a key problem in the field of undirected graph, and it can be used in a variety of contexts, including information retrieval and market analysis. By "undirected graphs", we mean undirected complex networks in this study. The ability to predict new links in complex networks has a significant impact on society. Many complex systems can be modelled using networks. For example, links represent relationships (such as friendships, etc.) in social networks, whereas nodes represent users. Embedding methods, which produce the feature vector of each node in a graph and identify unknown links, are one of the newest approaches to link prediction. The Deep Walk algorithm is a common graph embedding approach that uses pure random walking to capture network structure. In this paper, we propose an efficient model for link prediction based on a hill climbing algorithm. It is used as a cost function. The lower the cost is, the higher the accuracy for link prediction between the source and destination node will be. Unlike other algorithms that predict links based on a single feature, it takes advantage of multiple features. The proposed method has been tested over nine publicly available datasets, and its performance has been evaluated by comparing it to other frequently used indexes. Our model outperforms all of these measures, as indicated by its higher prediction accuracy. [ABSTRACT FROM AUTHOR]

Deep learning (DL)-based specific emitter identification (SEI) technique can automatically extract radio frequency (RF) fingerprint features in RF signals to distinguish between legal and illegal devices and enhance the security of wireless network. However, deep neural network (DNN) can easily be fooled by adversarial examples or perturbations of the input data. If a malicious device emits signals containing a specially designed adversarial samples, will the DL-based SEI still work stably to correctly identify the malicious device? To the best of our knowledge, this research is still blank, let alone the corresponding defense methods. Therefore, this paper designs two scenarios of attack and defense and proposes the corresponding implementation methods to specializes in the robustness of DL-based SEI under adversarial attacks. On this basis, detailed experiments are carried out based on the real-world data and simulation data. The attack scenario is that the malicious device adds an adversarial perturbation signal specially designed to the original signal, misleading the original system to make a misjudgment. Experiments based on three different attack generation methods show that DL-based SEI is very vulnerability. Even if the intensity is very low, without affecting the probability density distribution of the original signal, the performance can be reduced to about 50%, and at −22 dB it is completely invalid. In the defense scenario, the adversarial training (AT) of DL-based SEI is added, which can significantly improve the system's performance under adversarial attacks, with ≥60% improvement in the recognition rate compared to the network without AT. Further, AT has a more robust effect on white noise. This study fills the relevant gaps and provides guidance for future research. In the future research, the impact of adversarial attacks must be considered, and it is necessary to add adversarial training in the training process. [ABSTRACT FROM AUTHOR]

Alzheimer's disease (AD) is a prevalent neurodegenerative disease with no effective treatment. Efficient and rapid detection plays a crucial role in mitigating and managing AD progression. Deep learning-assisted smartphone-based microfluidic paper analysis devices (μPADs) offer the advantages of low cost, good sensitivity, and rapid detection, providing a strategic pathway to address large-scale disease screening in resource-limited areas. However, existing smartphone-based detection platforms usually rely on large devices or cloud servers for data transfer and processing. Additionally, the implementation of automated colorimetric enzyme-linked immunoassay (c-ELISA) on μPADs can further facilitate the realization of smartphone μPADs platforms for efficient disease detection. This paper introduces a new deep learning-assisted offline smartphone platform for early AD screening, offering rapid disease detection in low-resource areas. The proposed platform features a simple mechanical rotating structure controlled by a smartphone, enabling fully automated c-ELISA on μPADs. Our platform successfully applied sandwich c-ELISA for detecting the β-amyloid peptide 1–42 (Aβ 1–42, a crucial AD biomarker) and demonstrated its efficacy in 38 artificial plasma samples (healthy: 19, unhealthy: 19, N = 6). Moreover, we employed the YOLOv5 deep learning model and achieved an impressive 97 % accuracy on a dataset of 1824 images, which is 10.16 % higher than the traditional method of curve-fitting results. The trained YOLOv5 model was seamlessly integrated into the smartphone using the NCNN (Tencent's Neural Network Inference Framework), enabling deep learning-assisted offline detection. A user-friendly smartphone application was developed to control the entire process, realizing a streamlined "samples in, answers out" approach. This deep learning-assisted, low-cost, user-friendly, highly stable, and rapid-response automated offline smartphone-based detection platform represents a good advancement in point-of-care testing (POCT). Moreover, our platform provides a feasible approach for efficient AD detection by examining the level of Aβ 1–42, particularly in areas with low resources and limited communication infrastructure. The schematic diagram of the overview of an offline deep learning-assisted smartphone-based paper-based microfluidic platform for screening of Alzheimer's disease. [Display omitted] • Deep learning-assisted smartphone-based platform enables offline detection. • Fully automated microfluidic paper analysis devices (μPADs). • Smartphone-based platform for detection of biomarker of Alzheimer's disease. [ABSTRACT FROM AUTHOR]

Text detection in natural scene images is an important prerequisite for many content-based image analysis tasks. In this paper, we propose an accurate and robust method for detecting texts in natural scene images. A fast and effective pruning algorithm is designed to extract Maximally Stable Extremal Regions (MSERs) as character candidates using the strategy of minimizing regularized variations. Character candidates are grouped into text candidates by the single-link clustering algorithm, where distance weights and clustering threshold are learned automatically by a novel self-training distance metric learning algorithm. The posterior probabilities of text candidates corresponding to non-text are estimated with a character classifier; text candidates with high non-text probabilities are eliminated and texts are identified with a text classifier. The proposed system is evaluated on the ICDAR 2011 Robust Reading Competition database; the $f$ -measure is over 76%, much better than the state-of-the-art performance of 71%. Experiments on multilingual, street view, multi-orientation and even born-digital databases also demonstrate the effectiveness of the proposed method. Finally, an online demo of our proposed scene text detection system has been set up at http://prir.ustb.edu.cn/TexStar/scene-text-detection/. [ABSTRACT FROM PUBLISHER]

Anomaly detection under the open-set scenario is a challenging task that requires learning discriminative features to detect anomalies that were even unseen during training. As a cheap yet effective approach, data augmentation has been widely used to create pseudo anomalies for better training of such models. Recent wisdom of augmentation methods focuses on generating random pseudo instances that may lead to a mixture of augmented instances with seen anomalies, or out of the typical range of anomalies. To address this issue, we propose a novel saliency-guided data augmentation method, SaliencyCut, to produce pseudo but more common anomalies that tend to stay in the plausible range of anomalies. Furthermore, we deploy a two-head learning strategy consisting of normal and anomaly learning heads to learn the anomaly score of each sample. Theoretical analyses show that this mechanism offers a more tractable and tighter lower bound of the data log-likelihood. We then design a novel patch-wise residual module in the anomaly learning head to extract and assess anomaly features from each sample, facilitating the learning of discriminative representations of anomaly instances. Extensive experiments conducted on six real-world anomaly detection datasets demonstrate the superiority of our method to competing methods under various settings. Codes are available at: https://github.com/yjnanan/SaliencyCut. [Display omitted] • We propose a saliency-guided data augmentation for creating pseudo anomalies, namely SaliencyCut. • We deploy a two-head learning mechanism based on the theoretical lower bound. • We further design a novel patch-wise residual module to gather anomaly knowledge. • Comprehensive experiments on six real application datasets validate the effectiveness of our method. [ABSTRACT FROM AUTHOR]

• A multi-modal Generative Adversarial Network for traffic event detection. • Semi-supervised learning based on generative adversarial network. • Detecting traffic events with both sensor and social media data. • Evaluation based on a large, real-world multi-modal dataset. Advances in the Internet of Things have enabled the development of many smart city applications and expert systems that help citizens and authorities better understand the dynamics of the cities, and make better planning and utilisation of city resources. Smart cities are composed of complex systems that usually process and analyse big data from the Cyber, Physical, and Social worlds. Traffic event detection is an important and complex task in smart transportation modelling and management. We address this problem using semi-supervised deep learning with data of different modalities, e.g., physical sensor observations and social media data. Unlike most existing studies focusing on data of single modality, the proposed method makes use of data of multiple modalities that appear to complement and reinforce each other. Meanwhile, as the amount of labelled data in big data applications is usually extremely limited, we extend the multi-modal Generative Adversarial Network model to a semi-supervised architecture to characterise traffic events. We evaluate the model with a large, real-world dataset consisting of traffic sensor observations and social media data collected from the San Francisco Bay Area over a period of four months. The evaluation results clearly demonstrate the advantages of the proposed model in extracting and classifying traffic events. [ABSTRACT FROM AUTHOR]

Learning a cross-domain representation from labeled source domains to unlabeled target domains is an important research problem in representation learning. Despite the success of traditional adversarial methods, they proposed to align features from each domain only while neglecting the importance of labels, when fooling a special domain discriminator network. Thus, the discriminator of these approaches merely distinguishes whether the generated features are in-domain or not, which may lead to less class-discriminative features. In this paper, by considering the joint distributions of features and labels in both domains, we present Feature and Label Adversarial Networks (FLAN). As a result, FLAN can generate more discriminative features in both domains. Experimental results on standard unsupervised domain adaptation benchmarks have demonstrated that FLAN can outperform the state-of-art domain invariant representation learning methods. [ABSTRACT FROM AUTHOR]