Search

Reinforcement learning (RL) has demonstrated its ability to solve high dimensional tasks by leveraging non-linear function approximators. However, these successes are mostly achieved by ’black-box’ policies in simulated domains. When deploying RL to the real world, several concerns regarding the use of a ’black-box’ policy might be raised. In order to make the learned policies more transparent, we propose in this paper a policy iteration scheme that retains a complex function approximator for its internal value predictions but constrains the policy to have a concise, hierarchical, and human-readable structure, based on a mixture of interpretable experts. Each expert selects a primitive action according to a distance to a prototypical state. A key design decision to keep such experts interpretable is to select the prototypical states from trajectory data. The main technical contribution of the paper is to address the challenges introduced by this non-differentiable prototypical state selection procedure. Experimentally, we show that our proposed algorithm can learn compelling policies on continuous action deep RL benchmarks, matching the performance of neural network based policies, but returning policies that are more amenable to human inspection than neural network or linear-in-feature policies. [ABSTRACT FROM AUTHOR]

This paper serves as a survey of recent advances in large margin training and its theoretical foundations, mostly for (nonlinear) deep neural networks (DNNs) that are probably the most prominent machine learning models for large-scale data in the community over the past decade. We generalize the formulation of classification margins from classical research to latest DNNs, summarize theoretical connections between the margin, network generalization, and robustness, and introduce recent efforts in enlarging the margins for DNNs comprehensively. Since the viewpoint of different methods is discrepant, we categorize them into groups for ease of comparison and discussion in the paper. Hopefully, our discussions and overview inspire new research work in the community that aim to improve the performance of DNNs, and we also point to directions where the large margin principle can be verified to provide theoretical evidence why certain regularizations for DNNs function well in practice. We managed to shorten the paper such that the crucial spirit of large margin learning and related methods are better emphasized. [ABSTRACT FROM AUTHOR]

It is widely acknowledged that the success of deep learning is built upon large-scale training data and tremendous computing power. However, the data and computing power are not always available for many real-world applications. In this paper, we address the machine learning problem where it lacks training data and limits computing power. Specifically, we investigate domain adaptation which is able to transfer knowledge from one labeled source domain to an unlabeled target domain, so that we do not need much training data from the target domain. At the same time, we consider the situation that the running environment is confined, e.g., in edge computing the end device has very limited running resources. Technically, we present the Faster Domain Adaptation (FDA) protocol and further report two paradigms of FDA: early stopping and amid skipping. The former accelerates domain adaptation by multiple early exit points. The latter speeds up the adaptation by wisely skip several amid neural network blocks. Extensive experiments on standard benchmarks verify that our method is able to achieve the comparable and even better accuracy but employ much less computing resources. To the best of our knowledge, there are very few works which investigated accelerating knowledge adaptation in the community. This work is expected to inspire the topic for more discussion. [ABSTRACT FROM AUTHOR]

Conventional machine learning algorithms suffer the problem that the model trained on existing data fails to generalize well to the data sampled from other distributions. To tackle this issue, unsupervised domain adaptation (UDA) transfers the knowledge learned from a well-labeled source domain to a different but related target domain where labeled data is unavailable. The majority of existing UDA methods assume that data from the source domain and the target domain are available and complete during training. Thus, the divergence between the two domains can be formulated and minimized. In this paper, we consider a more practical yet challenging UDA setting where either the source domain data or the target domain data are unknown. Conventional UDA methods would fail this setting since the domain divergence is agnostic due to the absence of the source data or the target data. Technically, we investigate UDA from a novel view—adversarial attack—and tackle the divergence-agnostic adaptive learning problem in a unified framework. Specifically, we first report the motivation of our approach by investigating the inherent relationship between UDA and adversarial attacks. Then we elaborately design adversarial examples to attack the training model and harness these adversarial examples. We argue that the generalization ability of the model would be significantly improved if it can defend against our attack, so as to improve the performance on the target domain. Theoretically, we analyze the generalization bound for our method based on domain adaptation theories. Extensive experimental results on multiple UDA benchmarks under conventional, source-absent and target-absent UDA settings verify that our method is able to achieve a favorable performance compared with previous ones. Notably, this work extends the scope of both domain adaptation and adversarial attack, and expected to inspire more ideas in the community. [ABSTRACT FROM AUTHOR]

Classification of 3D objects – the selection of a category in which each object belongs – is of great interest in the field of machine learning. Numerous researchers use deep neural networks to address this problem, altering the network architecture and representation of the 3D shape used as an input. To investigate the effectiveness of their approaches, we conduct an extensive survey of existing methods and identify common ideas by which we categorize them into a taxonomy. Second, we evaluate 11 selected classification networks on two 3D object datasets, extending the evaluation to a larger dataset on which most of the selected approaches have not been tested yet. For this, we provide a framework for converting shapes from common 3D mesh formats into formats native to each network, and for training and evaluating different classification approaches on this data. Despite being partially unable to reach the accuracies reported in the original papers, we compare the relative performance of the approaches as well as their performance when changing datasets as the only variable to provide valuable insights into performance on different kinds of data. We make our code available to simplify running training experiments with multiple neural networks with different prerequisites. [ABSTRACT FROM AUTHOR]

The growing number of Internet of Thing (IoT) and Ultra-Reliable Low Latency Communications (URLCC) use cases in next generation communication networks calls for the development of efficient Forward Error Correction (FEC) mechanisms. These use cases usually imply using short to mid-sized information blocks and requires low-complexity and/or fast decoding procedures. This paper investigates the joint learning of short to mid block-length coding schemes and associated Belief-Propagation (BP) like decoders using Machine Learning (ML) techniques. An interpretable auto-encoder (AE) architecture is proposed, ensuring scalability to block sizes currently challenging for ML-based linear block code design approaches. By optimizing a coding scheme w.r.t. the targeted decoder, the proposed system offers a good complexity/performance trade-off compared to various codes from literature with length up to 128 bits. [ABSTRACT FROM AUTHOR]

Time series classification models have been garnering significant importance in the research community. However, not much research has been done on generating adversarial samples for these models. These adversarial samples can become a security concern. In this paper, we propose utilizing an adversarial transformation network (ATN) on a distilled model to attack various time series classification models. The proposed attack on the classification model utilizes a distilled model as a surrogate that mimics the behavior of the attacked classical time series classification models. Our proposed methodology is applied onto 1-nearest neighbor dynamic time warping (1-NN DTW) and a fully convolutional network (FCN), all of which are trained on 42 University of California Riverside (UCR) datasets. In this paper, we show both models were susceptible to attacks on all 42 datasets. When compared to Fast Gradient Sign Method, the proposed attack generates a larger faction of successful adversarial black-box attacks. A simple defense mechanism is successfully devised to reduce the fraction of successful adversarial samples. Finally, we recommend future researchers that develop time series classification models to incorporating adversarial data samples into their training data sets to improve resilience on adversarial samples. [ABSTRACT FROM AUTHOR]

Sparsity-constrained optimization problems are common in machine learning, such as sparse coding, low-rank minimization and compressive sensing. However, most of previous studies focused on constructing various hand-crafted sparse regularizers, while little work was devoted to learning adaptive sparse regularizers from given input data for specific tasks. In this paper, we propose a deep sparse regularizer learning model that learns data-driven sparse regularizers adaptively. Via the proximal gradient algorithm, we find that the sparse regularizer learning is equivalent to learning a parameterized activation function. This encourages us to learn sparse regularizers in the deep learning framework. Therefore, we build a neural network composed of multiple blocks, each being differentiable and reusable. All blocks contain learnable piecewise linear activation functions which correspond to the sparse regularizer to be learned. Furthermore, the proposed model is trained with back propagation, and all parameters in this model are learned end-to-end. We apply our framework to multi-view clustering and semi-supervised classification tasks to learn a latent compact representation. Experimental results demonstrate the superiority of the proposed framework over state-of-the-art multi-view learning models. [ABSTRACT FROM AUTHOR]

The efficiency of link adaptation in wireless communications relies greatly on the accuracy of channel knowledge and transmission mode selection. In this paper, a novel deep learning based link adaptation framework is proposed for the orthogonal frequency-division multiplexing (OFDM) systems with compressed-sensing-assisted index modulation, termed as OFDM-CSIM, communicating over millimeter-wave (mmWave) channels. To achieve link adaptation, a novel multi-layer sparse Bayesian learning (SBL) algorithm is proposed for accurately and instantaneously providing the required channel state information. Meanwhile, a deep neural networks (DNN)-assisted adaptive modulation algorithm is proposed to choose the best possible transmission mode to maximize the achievable throughput. Simulation results show that the proposed multi-layer SBL algorithm enables more accurate channel estimation than the conventional techniques. The DNN-based adaptive modulator is capable of achieving a higher throughput than the learning-assisted solution based on the $k$ nearest neighbor ($k$ -NN) algorithm, and also the classic average signal-to-noise ratio (SNR)-based solutions. Moreover, analysis shows that both the multi-layer SBL algorithm and the DNN-assisted adaptive modulator achieve better performance than their respective conventional counterparts while at a significantly lower computational complexity cost. [ABSTRACT FROM AUTHOR]

Image compression is one of the most fundamental techniques and commonly used applications in the image and video processing field. Earlier methods built a well-designed pipeline, and efforts were made to improve all modules of the pipeline by handcrafted tuning. Later, tremendous contributions were made, especially when data-driven methods revitalized the domain with their excellent modeling capacities and flexibility in incorporating newly designed modules and constraints. Despite great progress, a systematic benchmark and comprehensive analysis of end-to-end learned image compression methods are lacking. In this paper, we first conduct a comprehensive literature survey of learned image compression methods. The literature is organized based on several aspects to jointly optimize the rate-distortion performance with a neural network, i.e., network architecture, entropy model and rate control. We describe milestones in cutting-edge learned image-compression methods, review a broad range of existing works, and provide insights into their historical development routes. With this survey, the main challenges of image compression methods are revealed, along with opportunities to address the related issues with recent advanced learning methods. This analysis provides an opportunity to take a further step towards higher-efficiency image compression. By introducing a coarse-to-fine hyperprior model for entropy estimation and signal reconstruction, we achieve improved rate-distortion performance, especially on high-resolution images. Extensive benchmark experiments demonstrate the superiority of our model in rate-distortion performance and time complexity on multi-core CPUs and GPUs. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (DCNNs) are currently the method of choice both for generative, as well as for discriminative learning in computer vision and machine learning. The success of DCNNs can be attributed to the careful selection of their building blocks (e.g., residual blocks, rectifiers, sophisticated normalization schemes, to mention but a few). In this paper, we propose $\Pi$ Π -Nets, a new class of function approximators based on polynomial expansions. $\Pi$ Π -Nets are polynomial neural networks, i.e., the output is a high-order polynomial of the input. The unknown parameters, which are naturally represented by high-order tensors, are estimated through a collective tensor factorization with factors sharing. We introduce three tensor decompositions that significantly reduce the number of parameters and show how they can be efficiently implemented by hierarchical neural networks. We empirically demonstrate that $\Pi$ Π -Nets are very expressive and they even produce good results without the use of non-linear activation functions in a large battery of tasks and signals, i.e., images, graphs, and audio. When used in conjunction with activation functions, $\Pi$ Π -Nets produce state-of-the-art results in three challenging tasks, i.e., image generation, face verification and 3D mesh representation learning. The source code is available at https://github.com/grigorisg9gr/polynomial_nets. [ABSTRACT FROM AUTHOR]

With the popularity of location-aware devices (e.g., smart phones), large amounts of location-based social media data such as check-ins are generated. This stimulates plenty of studies for POI recommendations by applying machine learning techniques. However, most of the existing studies focus on POI recommendations in the same city or region, and fail to recommend POIs for users when they travel to a new city. In this paper, we propose a novel deep neural network, named as ST-TransRec, for crossing-city POI recommendations. It integrates the deep neural network, transfer learning technique, and density-based resampling method into a unified framework. In this model, the deep neural network is used to capture users’ preferences for POIs and learn the embeddings of POIs. Besides, the transfer learning technique is employed to bridge the gap between cities that results from the city-dependent features. As the distributions over POIs are imbalanced, we design a density-based spatial resampling model which enables POIs to be well matched across cities. We conduct extensive experiments on two real-world datasets. The experimental results show the advantages of ST-TransRec over the state-of-the-art methods for crossing-city POI recommendations. [ABSTRACT FROM AUTHOR]

This paper proposes a novel machine-learning approach for predicting AC-OPF solutions that features a fast and scalable training. It is motivated by the significant training time needed by existing machine-learning approaches for predicting AC-OPF. The proposed approach is a 2-stage methodology that exploits a spatial decomposition of the power network that is viewed as a set of regions. The first stage learns to predict the flows and voltages on the buses and lines coupling the regions, and the second stage trains, in parallel, the machine-learning models for each region. The predictions can then seed a power flow to eliminate the physical constraint violations, resulting in minor violations only for the operational bound constraints. Experimental results on the French transmission system (up to 6,700 buses) and large test cases from the pglib library (up to 9,000 buses) demonstrate the potential of the approach. Within a short training time, the approach predicts AC-OPF solutions with very high fidelity, producing significant improvements over the state-of-the-art. The proposed approach thus opens the possibility of training machine-learning models quickly to respond to changes in operating conditions. [ABSTRACT FROM AUTHOR]

A sum-product network (SPN) is a probabilistic model, based on a rooted acyclic directed graph, in which terminal nodes represent probability distributions and non-terminal nodes represent convex sums (weighted averages) and products of probability distributions. They are closely related to probabilistic graphical models, in particular to Bayesian networks with multiple context-specific independencies. Their main advantage is the possibility of building tractable models from data, i.e., models that can perform several inference tasks in time proportional to the number of edges in the graph. They are somewhat similar to neural networks and can address the same kinds of problems, such as image processing and natural language understanding. This paper offers a survey of SPNs, including their definition, the main algorithms for inference and learning from data, several applications, a brief review of software libraries, and a comparison with related models. [ABSTRACT FROM AUTHOR]

In this paper, a new method is proposed for creating an optimized ensemble classifier. The proposed method mitigates the issue of class imbalances by partitioning the input data into its various data classes. The partitions are then clustered incrementally to generate a pool of class pure data clusters. The generated data clusters are then balanced by adding samples from all classes which are closest to the cluster centroid. In this manner all generated data clusters are balanced and classifiers trained on such a data cluster are unbiased as well. This creates a diverse input space for training of base classifiers. The pool of clusters is then utilized to train a set of diverse base classifiers to generate the base classifier pool. The pool of classifiers is then treated as a combinatorial problem of optimization and an evolutionary algorithm is incorporated. The proposed approach generates an optimized ensemble classifier that can not only achieve the highest classification accuracy but also has a lower component size as well. The proposed approach is tested on 31 benchmark datasets from UCI machine learning repository and results are compared with existing state-of-the-art ensemble classifiers as well. [ABSTRACT FROM AUTHOR]

Millimeter wave (mmWave) and massive MIMO systems are intrinsic components of 5G and beyond. These systems rely on using beamforming codebooks for both initial access and data transmission. Current beam codebooks, however, generally consist of a large number of narrow beams that scan all possible directions, leading to large training overhead. Further, these codebooks do not normally account for hardware impairments or possible non-uniform array geometries, and their calibration process is expensive. To overcome these limitations, this paper develops an efficient online machine learning framework that learns how to adapt the codebook beam patterns to the specific deployment, surrounding environment, user distribution, and hardware characteristics. This is done by designing a novel complex-valued neural network architecture in which the neuron weights directly model the beamforming weights of the analog phase shifters, accounting for the key hardware constraints. This model learns the codebook beams through online and self-supervised training avoiding the need for explicit channel state information. This respects the practical situations where the channel is imperfect or hard to obtain. Simulation results highlight the capability of the proposed solution in learning environment and hardware aware beam codebooks, which reduce the training overhead and improve the robustness against possible hardware impairments. [ABSTRACT FROM AUTHOR]

Representation learning has been proven to play an important role in the unprecedented success of machine learning models in numerous tasks, such as machine translation, face recognition and recommendation. The majority of existing representation learning approaches often require a large number of consistent and noise-free labels. However, due to various reasons such as budget constraints and privacy concerns, labels are very limited in many real-world scenarios. Directly applying standard representation learning approaches on small labeled data sets will easily run into over-fitting problems and lead to sub-optimal solutions. Even worse, in some domains such as education, the limited labels are usually annotated by multiple workers with diverse expertise, which yields noises and inconsistency in such crowdsourcing settings. In this paper, we propose a novel framework which aims to learn effective representations from limited data with crowdsourced labels. Specifically, we design a grouping based deep neural network to learn embeddings from a limited number of training samples and present a Bayesian confidence estimator to capture the inconsistency among crowdsourced labels. Furthermore, to expedite the training process, we develop a hard example selection procedure to adaptively pick up training examples that are misclassified by the model. Extensive experiments conducted on three real-world data sets demonstrate the superiority of our framework on learning representations from limited data with crowdsourced labels, comparing with various state-of-the-art baselines. In addition, we provide a comprehensive analysis on each of the main components of our proposed framework and also introduce the promising results it achieved in our real production to fully understand the proposed framework. To encourage reproducible results, we make our code available online at https://github.com/tal-ai/RECLE. [ABSTRACT FROM AUTHOR]

Many signal processing and machine learning problems can be formulated as consensus optimization problems which can be solved efficiently via a cooperative multi-agent system. However, the agents in the system can be unreliable due to a variety of reasons: noise, faults and attacks. Providing erroneous updates leads the optimization process in a wrong direction, and degrades the performance of distributed machine learning algorithms. This paper considers the problem of decentralized learning using ADMM in the presence of unreliable agents. First, we rigorously analyze the effect of erroneous updates (in ADMM learning iterations) on the convergence behavior of the multi-agent system. We show that the algorithm linearly converges to a neighborhood of the optimal solution under certain conditions and characterize the neighborhood size analytically. Next, we provide guidelines for network design to achieve a faster convergence to the neighborhood. We also provide conditions on the erroneous updates for exact convergence to the optimal solution. Finally, to mitigate the influence of unreliable agents, we propose ROAD, a robust variant of ADMM, and show its resilience to unreliable agents with an exact convergence to the optimum. [ABSTRACT FROM AUTHOR]

We give an overview of recent developments in the modeling of radiowave propagation, based on machine learning (ML) algorithms. We identify the input and output specification and the architecture of the model as the main challenges associated with ML-driven propagation models. Relevant papers are discussed and categorized based on their approach to each of these challenges. Emphasis is given on presenting the prospects and open problems in this promising and rapidly evolving area. [ABSTRACT FROM AUTHOR]

The proliferation of machine learning services in the last few years has raised data privacy concerns. Homomorphic encryption (HE) enables inference using encrypted data but it incurs 100x–10,000x memory and runtime overheads. Secure deep neural network (DNN) inference using HE is currently limited by computing and memory resources, with frameworks requiring hundreds of gigabytes of DRAM to evaluate small models. To overcome these limitations, in this paper we explore the feasibility of leveraging hybrid memory systems comprised of DRAM and persistent memory. In particular, we explore the recently-released Intel® Optane™ PMem technology and the Intel® HE-Transformer nGraph® to run large neural networks such as MobileNetV2 (in its largest variant) and ResNet-50 for the first time in the literature. We present an in-depth analysis of the efficiency of the executions with different hardware and software configurations. Our results conclude that DNN inference using HE incurs on friendly access patterns for this memory configuration, yielding efficient executions. [ABSTRACT FROM AUTHOR]

Adversarial attacks on deep neural networks (DNNs) have been found for several years. However, the existing adversarial attacks have high success rates only when the information of the victim DNN is well-known or could be estimated by the structure similarity or massive queries. In this paper, we propose to Attack on Attention (AoA), a semantic property commonly shared by DNNs. AoA enjoys a significant increase in transferability when the traditional cross entropy loss is replaced with the attention loss. Since AoA alters the loss function only, it could be easily combined with other transferability-enhancement techniques and then achieve SOTA performance. We apply AoA to generate 50000 adversarial samples from ImageNet validation set to defeat many neural networks, and thus name the dataset as DAmageNet. 13 well-trained DNNs are tested on DAmageNet, and all of them have an error rate over 85 percent. Even with defenses or adversarial training, most models still maintain an error rate over 70 percent on DAmageNet. DAmageNet is the first universal adversarial dataset. It could be downloaded freely and serve as a benchmark for robustness testing and adversarial training. [ABSTRACT FROM AUTHOR]

Many learning algorithms such as kernel machines, nearest neighbors, clustering, or anomaly detection, are based on distances or similarities. Before similarities are used for training an actual machine learning model, we would like to verify that they are bound to meaningful patterns in the data. In this paper, we propose to make similarities interpretable by augmenting them with an explanation. We develop BiLRP, a scalable and theoretically founded method to systematically decompose the output of an already trained deep similarity model on pairs of input features. Our method can be expressed as a composition of LRP explanations, which were shown in previous works to scale to highly nonlinear models. Through an extensive set of experiments, we demonstrate that BiLRP robustly explains complex similarity models, e.g., built on VGG-16 deep neural network features. Additionally, we apply our method to an open problem in digital humanities: detailed assessment of similarity between historical documents, such as astronomical tables. Here again, BiLRP provides insight and brings verifiability into a highly engineered and problem-specific similarity model. [ABSTRACT FROM AUTHOR]

Artificial Neural networks (ANNs) are widely used to solve classification problems for many machine learning applications. When errors occur in the computational units of an ANN implementation due to for example radiation effects, the result of an arithmetic operation can be changed, and therefore, the predicted classification class may be erroneously affected. This is not acceptable when ANNs are used in many safety-critical applications, because the incorrect classification may result in a system failure. Existing error-tolerant techniques usually rely on physically replicating parts of the ANN implementation or incurring in a significant computation overhead. Therefore, efficient protection schemes are needed for ANNs that are run on a processor and used in resource-limited platforms. A technique referred to as Selective Neuron Re-Computation (SNRC), is proposed in this paper. As per the ANN structure and algorithmic properties, SNRC can identify the cases in which the errors have no impact on the outcome; therefore, errors only need to be handled by re-computation when the classification result is detected as unreliable. Compared with existing temporal redundancy-based protection schemes, SNRC saves more than 60 percent of the re-computation (more than 90 percent in many cases) overhead to achieve complete error protection as assessed over a wide range of datasets. Different activation functions are also evaluated. [ABSTRACT FROM AUTHOR]

Many real–world domains involve information naturally represented by graphs, where nodes denote basic patterns while edges stand for relationships among them. The graph neural network (GNN) is a machine learning model capable of directly managing graph–structured data. In the original framework, GNNs are inductively trained, adapting their parameters based on a supervised learning environment. However, GNNs can also take advantage of transductive learning, thanks to the natural way they make information flow and spread across the graph, using relationships among patterns. In this paper, we propose a mixed inductive–transductive GNN model, study its properties and introduce an experimental strategy that allows us to understand and distinguish the role of inductive and transductive learning. The preliminary experimental results show interesting properties for the mixed model, highlighting how the peculiarities of the problems and the data can impact on the two learning strategies. [ABSTRACT FROM AUTHOR]

Network embedding algorithms learn a mapping from the discrete representation of nodes to continuous vector spaces that preserve the proximities of nodes. The techniques have a wide range of applications in various downstream tasks such as node classification, link prediction, and network alignment. Despite recent efforts to the design of novel models, little attention has been paid to understanding the instability of network embedding. In this paper, we fill this gap by investigating several state-of-the-art network embedding methods. Node embeddings form a geometric shape in the latent space. Characterizing the geometry is critical to figure out the variance of network embedding. Hence, we define two metrics to characterize the geometric properties and find that node embeddings tremble in different instantiations of an embedding space. Then, we formally define the stability of node embeddings as the invariance of the nearest neighbors of nodes. Experimental results show that existing embedding approaches have significant amounts of instability. We explore the influence factors that affect the stability of different methods and find that both network structure and algorithm models affect the stability of node embeddings significantly. Finally, we examine the implications of embedding instability for downstream tasks and find remarkable impacts on the performance. [ABSTRACT FROM AUTHOR]

Seizure prediction for drug-refractory epilepsy patients can improve their quality of life, reduce their anxiety, and help them take the necessary precautions. Nowadays, numerous deep learning algorithms have been proposed to predict seizure onset and obtain better performance than traditional machine learning methods. However, these methods require a large set of parameters and large hardware resources; they also have high energy consumption. Therefore, these methods cannot be implemented on compact, low-power wearable, or implantable medical devices. The devices should operate on a real-time basis to continually inform the epileptic patients. In this paper, we describe energy-efficient and hardware-friendly methods to predict the epileptic seizures. A model of only 45 kB was obtained by the neural architecture search and was evaluated across three datasets. The overall sensitivity, false prediction rate, and area under receiver operating characteristic curve were 99.81%, 0.005/h, 1 and 93.48%, 0.063/h, 0.977 and 85.19%, 0.116/h, 0.933, respectively, for three public datasets. This model was further reduced with several methods for model compression. The performances for the model sizes less than 50 kB for scalp EEG data and less than 10 kB for intracranial EEG data outperformed all the other models of similar model sizes. In particular, the energy consumption estimation was less than 10 mJ per inference for scalp EEG signal and less than 0.5 mJ per inference for intracranial EEG signal, which meet the requirements for low-power wearable and implantable devices, respectively. [ABSTRACT FROM AUTHOR]

This paper analyzes regularization terms proposed recently for improving the adversarial robustness of deep neural networks (DNNs), from a theoretical point of view. Specifically, we study possible connections between several effective methods, including input-gradient regularization, Jacobian regularization, curvature regularization, and a cross-Lipschitz functional. We investigate them on DNNs with general rectified linear activations, which constitute one of the most prevalent families of models for image classification and a host of other machine learning applications. We shed light on essential ingredients of these regularizations and re-interpret their functionality. Through the lens of our study, more principled and efficient regularizations can possibly be invented in the near future. [ABSTRACT FROM AUTHOR]

Multi-view representation learning is a promising and challenging research topic, which aims to integrate multiple data information from different views to improve the learning performance. The recent deep Gaussian processes (DGPs) have the advantages of good uncertainty estimates, powerful non-linear mapping ability and great generalization capability, which can be used as an excellent data representation learning method. However, DGPs only focus on single view data and are rarely applied to the multi-view scenario. In this paper, we propose a multi-view representation learning algorithm with deep Gaussian processes (named MvDGPs), which inherits the advantages of deep Gaussian processes and multi-view representation learning, and can learn more effective representation of multi-view data. The MvDGPs consist of two stages. The first stage is multi-view data representation learning, which is mainly used to learn more comprehensive representations of multi-view data. The second stage is classifier design, which aims to select an appropriate classifier to better employ the representations obtained in the first stage. In contrast with DGPs, MvDGPs support asymmetrical modeling depths for different views of data, resulting in better characterizations of the discrepancies among different views. Experimental results on real-world multi-view data sets verify the effectiveness of the proposed algorithm, which indicates that MvDGPs can integrate the complementary information in multiple views to discover a good representation of the data. [ABSTRACT FROM AUTHOR]

Few-shot learning has recently generated increasing popularity in machine learning, which addresses the fundamental yet challenging problem of learning to adapt to new tasks with the limited data. In this paper, we propose a new probabilistic framework that learns to fast adapt with external memory. We model the classifier parameters as distributions that are inferred from the support set and directly applied to the query set for prediction. The model is optimized by formulating as a variational inference problem. The probabilistic modeling enables better handling prediction uncertainty due to the limited data. We impose a discriminative constraint on the feature representations by exploring the class structure, which can improve the classification performance. We further introduce a memory unit to store task-specific information extracted from the support set and used for the query set to achieve explicit adaption to individual tasks. By episodic training, the model learns to acquire the capability of adapting to specific tasks, which guarantees its performance on new related tasks. We conduct extensive experiments on widely-used benchmarks for few-shot recognition. Our method achieves new state-of-the-art performance and largely surpassing previous methods by large margins. The ablation study further demonstrates the effectiveness of the proposed discriminative learning and memory unit. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the detection problem for a massive machine-type communication (mMTC) system that has correlated user activities. Two deep learning assisted algorithms are proposed to exploit the user activity correlation to facilitate channel estimation and user identification. Due to the dependency among user activities, conventional element-wise minimum mean square error (MMSE) denoiser used in the orthogonal approximate message passing (OAMP) algorithm cannot achieve satisfying performance during the two-step iterative process. Therefore, we propose a deep learning modified OAMP (DL-mOAMP) algorithm, which iteratively modifies the user activity ratio via exploiting the user activity correlation in the MMSE denoiser based on the estimated sequence during each OAMP iteration. Moreover, given a specific false alarm probability, a constant threshold employed in the conventional user identification is not optimal in the presence of user activity correlation. Thus, we propose a neural network framework that is dedicated to the user identification (DL-mOAMP-UI algorithm), which minimizes the missed detection probability under a pre-determined false alarm probability. Numerical results show that the proposed DL-mOAMP algorithm provides a substantial mean squared error performance gain compared to the conventional OAMP algorithm and the DL-mOAMP-UI algorithm can further improve the user identification accuracy of an mMTC system. [ABSTRACT FROM AUTHOR]

Arrhythmia detection and classification is a crucial step for diagnosing cardiovascular diseases. However, deep learning models that are commonly used and trained in end-to-end fashion are not able to provide good interpretability. In this paper, we address this deficiency by proposing the first novel interpretable arrhythmia classification approach based on a human-machine collaborative knowledge representation. Our approach first employs an AutoEncoder to encode electrocardiogram signals into two parts: hand-encoding knowledge and machine-encoding knowledge. A classifier then takes as input the encoded knowledge to classify arrhythmia heartbeats with or without human in the loop (HIL). Experiments and evaluation on the MIT-BIH Arrhythmia Database demonstrate that our new approach not only can effectively classify arrhythmia while offering interpretability, but also can improve the classification accuracy by adjusting the hand-encoding knowledge with our HIL mechanism. [ABSTRACT FROM AUTHOR]

The papers in this special section explore research on the latest topics related to the hardware acceleration of machine learning. [ABSTRACT FROM AUTHOR]