Search

Enabling a neural network to sequentially learn multiple tasks is of great significance for expanding the applicability of neural networks in real-world applications. However, artificial neural networks face the well-known problem of catastrophic forgetting. What is worse, the degradation of previously learned skills becomes more severe as the task sequence increases, known as the long-term catastrophic forgetting. It is due to two facts: first, as the model learns more tasks, the intersection of the low-error parameter subspace satisfying for these tasks becomes smaller or even does not exist; second, when the model learns a new task, the cumulative error keeps increasing as the model tries to protect the parameter configuration of previous tasks from interference. Inspired by the memory consolidation mechanism in mammalian brains with synaptic plasticity, we propose a confrontation mechanism in which Adversarial Neural Pruning and synaptic Consolidation (ANPyC) is used to overcome the long-term catastrophic forgetting issue. The neural pruning acts as long-term depression to prune task-irrelevant parameters, while the novel synaptic consolidation acts as long-term potentiation to strengthen task-relevant parameters. During the training, this confrontation achieves a balance in that only crucial parameters remain, and non-significant parameters are freed to learn subsequent tasks. ANPyC avoids forgetting important information and makes the model efficient to learn a large number of tasks. Specifically, the neural pruning iteratively relaxes the current task’s parameter conditions to expand the common parameter subspace of the task; the synaptic consolidation strategy, which consists of a structure-aware parameter-importance measurement and an element-wise parameter updating strategy, decreases the cumulative error when learning new tasks. Our approach encourages the synapse to be sparse and polarized, which enables long-term learning and memory. ANPyC exhibits effectiveness and generalization on both image classification and generation tasks with multiple layer perceptron, convolutional neural networks, and generative adversarial networks, and variational autoencoder. The full source code is available at https://github.com/GeoX-Lab/ANPyC. [ABSTRACT FROM AUTHOR]

Recent decades have witnessed rapid development of deep neural networks (DNN). As DNN learning is becoming more and more important to numerous intelligent system, ranging from self driving car to video surveillance system, significant research efforts have been devoted to explore how to improve DNN model's robustness and reliability against adversarial example attacks. Distinguish from previous study, we address the problem of adversarial training with ensemble based approach and propose a novel boosting based black-box attack scheme call BBAS to facilitate high diverse adversarial example generation. BBAS not only separates example generation from the settings of the trained model but also enhance the diversity of perturbation over class distribution through seamless integration of stratified sampling and ensemble adversarial training. This leads to reliable and effective training example selection. To validate and evaluate the scheme from different perspectives, a set of comprehensive tests have been carried out based on two large open data sets. Experimental results demonstrate the superiority of our method in terms of effectiveness. [ABSTRACT FROM AUTHOR]

We propose an online algorithm for sequential learning in the contextual multiarmed bandit setting. Our approach is to partition the context space and, then, optimally combine all of the possible mappings between the partition regions and the set of bandit arms in a data-driven manner. We show that in our approach, the best mapping is able to approximate the best arm selection policy to any desired degree under mild Lipschitz conditions. Therefore, we design our algorithm based on the optimal adaptive combination and asymptotically achieve the performance of the best mapping as well as the best arm selection policy. This optimality is also guaranteed to hold even in adversarial environments since we do not rely on any statistical assumptions regarding the contexts or the loss of the bandit arms. Moreover, we design an efficient implementation for our algorithm using various hierarchical partitioning structures, such as lexicographical or arbitrary position splitting and binary trees (BTs) (and several other partitioning examples). For instance, in the case of BT partitioning, the computational complexity is only log-linear in the number of regions in the finest partition. In conclusion, we provide significant performance improvements by introducing upper bounds (with respect to the best arm selection policy) that are mathematically proven to vanish in the average loss per round sense at a faster rate compared to the state of the art. Our experimental work extensively covers various scenarios ranging from bandit settings to multiclass classification with real and synthetic data. In these experiments, we show that our algorithm is highly superior to the state-of-the-art techniques while maintaining the introduced mathematical guarantees and a computationally decent scalability. [ABSTRACT FROM AUTHOR]

We report a method to generate de novo protein designs through a generative adversarial neural network, MolShapeGAN, that can rapidly produce a large variety of nanoarchitected material designs inspired by proteins. The proteomic molecular designs generated by MolShapeGAN model are examined using LAMMPS coarse-grained simulations by applying tensile deformation to the longest axis of each structure, to assess mechanical properties. In order to facilitate nanomechanical optimization, we develop a transformer neural network, denoted as VoxelPerceiver, that predicts mechanical properties directly from the molecular architecture in an end-to-end fashion. The assessment of key nanomechanical properties, such as maximum tensile stress, Von Mises stress mean, and Von Mises standard deviation, offer a materiomic design paradigm by which tailored nanomechanical properties can be achieved, and by which important insights can be gained about the particularities of nanomechanical responses of molecular structures. Optimization to achieve desired mechanical properties is performed both using a brute-force grid search and Bayesian optimization. We also report manufactured samples of scaled-up architected models of the protein designs using 3D printing. [ABSTRACT FROM AUTHOR]

The origins of the payment procedure by summons are not clearly known. In this paper an unknown precedent is localized in an early phase of post-classical roman procedure. The influence of some other juridical doctrines of diverse origins, could probably lead to the creation of a special procedure, that initially was nothing more than a technique, or a procedural phase. [ABSTRACT FROM AUTHOR]

The image factorization problem is the key challenge in Temporal Psycho-Visual Modulation (TPVM). In this paper, we present an end-to-end learned model for image-based non-negative matrix factorization. We decompose a set of images into a small number of image bases which can be used to reconstruct all the images by linearly combining the bases. During the process, the image bases, as well as their weights for the linear combination, are unknown. The method is based on conditional GAN and a variational sample disturber. Traditional NMF methods suffer from slow computational speed and poor generalization ability. A deep neural network shows potential in these two aspects. We conduct adversarial training in our model to generate better bases for image restoration. Our method outperforms other CNN based methods on several public datasets. Compared to traditional NMF algorithms, our model generates image bases that preserve details. Our model has advantages in speed as well as generalization ability. [ABSTRACT FROM AUTHOR]