Search

Convolutional neural networks (CNN) have been proved to be an effective method in the field of artificial intelligence (AI), and large-scale deploying CNN to embedded devices, no doubt, will greatly promote the development and application of AI into the practical industry. However, mainly due to the space-time complexity of CNN, computing power, memory bandwidth and flexibility are performance bottlenecks. In this paper, a framework containing model compression and hardware acceleration is proposed to solve the above problems. This framework consists of a mixed pruning method, data storage optimization for efficient memory utilization and an accelerator for mapping CNN on field programmable gate array (FPGA). The mixed pruning method is used to compress the model, and data bit-width is reduced to 8-bit by data quantization. Accelerator based on FPGA makes it flexible, configurable and efficient for CNN implementation. The model compression is evaluated on NVIDIA RTX2080Ti, and the results illustrate that the VGG16 is compressed by $30\times $ and the fully convolutional network (FCN) is compressed by $11\times $ within 1% accuracy loss. The compressed model is deployed and accelerated on ZCU102, which is up to $1.7\times $ and $24.5\times $ better in energy efficiency compared with RTX2080Ti and Intel i7 7700. [ABSTRACT FROM AUTHOR]

Deep neural network (DNN) has recently attracted tremendous attention in various fields. But the computing operation requirement and the memory bottleneck limit the energy efficiency of hardware implementations. Binary quantization is proposed to relieve the pressure of hardware design. And the Computing-In-Memory (CIM) is regarded as a promising method to resolve the memory wall challenge. However, the binary computing paradigm is mismatched with the CIM scheme, which incurs complex circuits and peripheral to realize binary operation in previous works. To overcome previous issues, this work presents Binary Representation Computation-In-Memory (BR-CIM) with several key features. (1) A lightweight computation unit is realized within the 6T SRAM array to accelerate binary computing and enlarge signal margin; (2) The reconfigurable computing scheme and mapping method support extendable bit precision to satisfy the accuracy requirement of various applications; (3) Simultaneous computing and weight loading is supported by column circuitry, which shortens the data loading latency; Several experiments are conducted to estimate algorithm accuracy, the computing latency, and power consumption. The energy efficiency reaches up to 1280 TOPs/W for binary representation. And the algorithm accuracy achieves 97.82%/76.4% on MNIST/CIFAR-100 dataset. [ABSTRACT FROM AUTHOR]

In this paper, a novel paradigm for blind source separation in the presence of nonlinear mixtures is presented. In particular, the paper addresses the problem of post-nonlinear mixing followed by another instantaneous mixing system. This model is called here the post-nonlinear-linear model. The method is based on the use of the recently introduced flexible activation function whose control points are adaptively changed: a neural model based on adaptive B-spline functions is employed. The signal separation is achieved through an information maximization criterion. Experimental results and comparison with existing solutions confirm the effectiveness of the proposed architecture. [ABSTRACT FROM AUTHOR]

This paper focuses on studying the H∞ and generalized H2 filtering problems for a class of delayed neural networks. The time-varying delay is only required to be continuous and bounded. Delay-dependent criteria are proposed such that the resulting filtering error system is globally exponentially stable with a guaranteed H∞ or generalized H2 performance. It is also shown that the designs of the desired filters are achieved by solving a set of linear matrix inequalities, which can be facilitated efficiently by resorting to standard numerical algorithms. It should be noted that, based on a novel bounding technique, several slack variables are introduced to reduce the conservatism of the derived conditions. Three examples with simulation results are provided to illustrate the effectiveness and performance of the developed approaches. [ABSTRACT FROM AUTHOR]

In this paper, the problem of stability analysis for a class of neural networks with distributed delays is investigated. Applying the M-matrix theory and new analysis technique, novel sufficient conditions for the existence, uniqueness, and global asymptotic stability of the equilibrium point of neural networks with distributed delays are derived. The new stability criteria can be applied to the case when the nondelayed terms cannot dominate the delayed terms, which have great significance in the design and application of neural networks with distributed delays. Three illustrative examples are presented which demonstrate the usefulness of the proposed results. [ABSTRACT FROM AUTHOR]

A neural network is said to be convergent (or completely stable) when each trajectory tends to an equilibrium point (a stationary state). A stronger property is that of absolute stability, which means that convergence holds for any choice of the neural network parameters, and any choice of the nonlinear functions, within specified and well characterized sets. In particular, the property of absolute stability requires that the neural network be convergent also when, for some parameter values, it possesses nonisolated equilibrium points (e.g., a manifold of equilibria). Such a property, which is really well suited for solving several classes of signal processing tasks in real time, cannot be in general established via the classical LaSalle approach, due to its inherent limitations to study convergence in situations where the neural network has nonisolated equilibrium points. In this paper, a new method to address absolute stability is developed, based on proving that the total length of the neural network trajectories is finite. A fundamental result on absolute stability is given, under the hypothesis that the neural network possesses a Lyapunov function, and the nonlinearities involved (neuron activations, inhibitions, etc.) are modeled by analytic functions. At the core of the proof of finiteness of trajectory length is the use of some basic inequalities for analytic functions due to Łojasiewicz. The result is applicable to a large class of neural networks, which includes the networks pro- posed by Vidyasagar, the Hopfield neural networks, and the standard cellular neural networks introduced by Chua and Yang. [ABSTRACT FROM AUTHOR]

In this paper, we propose a neural network for ranking a given set of N numbers In 0(1/N) time. The ordering of a set of numbers based on their relative magnitudes, which is analogous to sorting,. is a fundamental operation in many algorithms. In comparison with other sorting networks reported in the literature, the,: proposed network requires fewer neurons, and fewer interconnections between neurons. The interconnections use nonlinear synapses which are composed of comparators, and do not require any weighted interconnections between neurons, as used in conventional neural networks. The proposed network has many applications, including as a component of self-organizing feature maps aid other systems where sorting is a frequent operation. [ABSTRACT FROM AUTHOR]

The design of systems implementing low precision neural networks with emerging memories such as resistive random access memory (RRAM) is a significant lead for reducing the energy consumption of artificial intelligence. To achieve maximum energy efficiency in such systems, logic and memory should be integrated as tightly as possible. In this work, we focus on the case of ternary neural networks, where synaptic weights assume ternary values. We propose a two-transistor/two-resistor memory architecture employing a precharge sense amplifier, where the weight value can be extracted in a single sense operation. Based on experimental measurements on a hybrid 130 nm CMOS/RRAM chip featuring this sense amplifier, we show that this technique is particularly appropriate at low supply voltage, and that it is resilient to process, voltage, and temperature variations. We characterize the bit error rate in our scheme. We show based on neural network simulation on the CIFAR-10 image recognition task that the use of ternary neural networks significantly increases neural network performance, with regards to binary ones, which are often preferred for inference hardware. We finally evidence that the neural network is immune to the type of bit errors observed in our scheme, which can therefore be used without error correction. [ABSTRACT FROM AUTHOR]

Convolutional neural network (CNN) offers significant accuracy in image detection. To implement image detection using CNN in the Internet of Things (IoT) devices, a streaming hardware accelerator is proposed. The proposed accelerator optimizes the energy efficiency by avoiding unnecessary data movement. With unique filter decomposition technique, the accelerator can support arbitrary convolution window size. In addition, max-pooling function can be computed in parallel with convolution by using separate pooling unit, thus achieving throughput improvement. A prototype accelerator was implemented in TSMC 65-nm technology with a core size of 5 mm2. The accelerator can support major CNNs and achieve 152GOPS peak throughput and 434GOPS/W energy efficiency at 350 mW, making it a promising hardware accelerator for intelligent IoT devices. [ABSTRACT FROM PUBLISHER]

Neuromorphic computing is an attractive avenue of research for processing and learning complex real-world data. With technology migration into nano and molecular scales several area and power efficient approaches to the design and implementation of artificial neural networks have been proposed. The discovery of the memristor has further enabled the realization of denser nanoscale logic and memory systems by facilitating the implementation of multilevel logic. Specifically, the innate reconfigurability of memristors can be exploited to realize synapses in artificial neural networks. This work focuses on the development of a variation-tolerant training methodology to efficiently reconfigure memristive synapses in a Trainable Threshold Gate Array (TTGA) system. The training process is inspired from the gradient descent machine learning algorithm commonly used to train artificial threshold neural networks, perceptrons. The design and CMOS/Nano implementation of the TTGA system from trainable perceptron based threshold gates is detailed and results are provided to showcase the training process and performance characteristics of the proposed system. Also shown are the results for training a 1T1M (1 Transistor and 1 Memristor) multilevel memristive memory and its performance characteristics. [ABSTRACT FROM PUBLISHER]

The growing interest in pulse-mode processing by neural networks is encouraging the development of hardware implementations of massively parallel networks of integrate-and-fire neurons distributed over multiple chips. Address-event representation (AER) has long been considered a convenient transmission protocol for spike based neuromorphic devices. One missing, long-needed feature of AER-based systems is the ability to acquire data from complex neuromorphic systems and to stimulate them using suitable data. We have implemented a general-purpose solution in the form of a peripheral component interconnect (PCI) board (the PCI-AER board) supported by software. We describe the main characteristics of the PCI-AER board, and of the related supporting software. To show the functionality of the PCI-AER infrastructure we demonstrate a reconfigurable multichip neuromorphic system for feature selectivity which models orientation tuning properties of cortical neurons. [ABSTRACT FROM AUTHOR]