Search

Neuromorphic processors have gained momentum recently due to their high energy efficiency in artificial intelligence applications compared to DNN accelerators. Most neuromorphic processors are executing SNNs (Spiking Neural Networks). Liquid State Machine (LSM), as the spiking version of reservoir computing, shows advantages and great potential in image classification, speech recognition, language translation, etc.. Comparing with other SNN models, LSM has the characteristics of easy to train and low resource utilization, which is suitable for low-power and resource-constrained edge computing scenarios. In this paper, we propose a novel design of a neuromorphic processor, LSMCore, aiming at LSM acceleration. LSMCore supports both training and inference of LSM. It consists of 256 input neurons, 1024 liquid neurons, and 1.31M synapses. Besides, multiple optimization techniques, including weight quantization for reducing storage, zero-skipping for decreasing dynamic sparsity, and mini-batch training are adopted in this processor. The experimental results show that the frequency of LSMCore achieves 400 MHz, the power is 4.9W and the area is 18.49 mm2 with a 40nm library. Comparing with the baseline, LSMCore achieves up to $80.7\times $ ($49.6\times $), $91.3\times $ ($56.3\times $), and $83.1\times $ ($56.8\times $) speedup on MNIST, N-MNIST, and Free Spoken Digital Dataset (FSDD) respectively for training (inference), while the accuracy of LSMCore on these three datasets are 96.8%, 97.6%, and 90% respectively. [ABSTRACT FROM AUTHOR]

In-memory computing (IMC) is an effective solution for energy-efficient artificial intelligence applications. Analog IMC amortizes the power consumption of multiple sensing amplifiers with an analog-to-digital converter (ADC) and simultaneously completes the calculation of multi-line data with a high parallelism degree. Based on a universal one-transistor one-magnetic tunnel junction (MTJ) spin transfer torque magnetic RAM (STT-MRAM) cell, this paper demonstrates a novel tunneling magnetoresistance (TMR) ratio magnifying method to realize analog IMC. Previous concerns including low TMR ratio and analog calculation nonlinearity are addressed using device-circuit interaction. The TMR is magnified $7500\times $ using a latch structure in combination with the device. Peripheral circuits are minimally modified to enable in-memory matrix-vector multiplication. A current mirror with a feedback structure is implemented to enhance analog computing linearity and calculation accuracy. The proposed design maximumly supports 1024 2-bit input and 1-bit weight multiply-and-accumulate (MAC) computations simultaneously. The proposal is simulated using the 28-nm CMOS process and MTJ compact model. The integral nonlinearity is reduced by 57.6% compared with the conventional structure. 9.47-25.4 TOPS/W is realized with 2-bit input, 1-bit weight, and 4-bit output convolution neural network (CNN). [ABSTRACT FROM AUTHOR]

When applied to artificial intelligence edge devices, the conventionally von Neumann computing architecture imposes numerous challenges (e.g., improving the energy efficiency), due to the memory-wall bottleneck involving the frequent movement of data between the memory and the processing elements (PE). Computing-in-memory (CIM) is a promising candidate approach to breaking through this so-called memory wall bottleneck. SRAM cells provide unlimited endurance and compatibility with state-of-the-art logic processes. This paper outlines the background, trends, and challenges involved in the further development of SRAM-CIM macros. This paper also reviews recent silicon-verified SRAM-CIM macros designed for logic and multiplication-accumulation (MAC) operations. [ABSTRACT FROM AUTHOR]

This paper provides a review of design approaches towards artificial intelligence (AI) System-on-Chip. AI algorithms have progressed over the past decades from perceptron-based neural network (NN) and neuro-fuzzy (NF) system to today’s deep neural network (DNN) and neuromorphic computing. Recent DNN hardware accelerators focus on energy-efficient integration of digital circuits to realize real-time DNN operation while neuromorphic processors deploy new memory technologies with analog computation for low power consumption. However, different design approaches can be applied to such processor implementation with their pros and cons. This paper reviews from the early processor designs for NN and NF in both mixed-mode and digital implementations to the recent DNN SoC designs that we have proposed for a decade. The former content deals with NN and NF processors used as a functional building block of a machine vision SoC, while the latter concentrates on integration of the whole DNN function. We also provide a discussion on the approaches, and provide perspective on future research directions. [ABSTRACT FROM AUTHOR]

The Internet of Things (IoT) is an interface with the physical world that usually operates in random-sparse-event (RSE) scenarios. This article discusses main challenges of IoT chips: power consumption, power supply, artificial intelligence (AI), small-signal acquisition, and evaluation criteria. To overcome these challenges, many works recently aimed at IoT system design have emerged. This work reviews the architecture and circuit innovations that have contributed to IoT developments. This paper does not cover security of IoT. Event-driven architectures and nonuniform sampling ADCs significantly reduce the long-term average power. Besides, embedding AI engines in IoT nodes (AIoT) is one critical trend. The computing-in-memory technique improves the energy efficiency of the AI engine. Asynchronous spike neural networks (ASNNs) AI engines show low power potential. In addition to data processing, small-signal acquisition is also critical. The charge-domain analog-front-end (AFE) techniques such as floating inverter-based amplifiers improve energy efficiency. In addition to the above low power and high energy efficiency technologies, energy harvesting can also enhance the lifetime of AIoT devices. This article discusses recent ambient RF and natural energy harvesting approaches and high-efficiency DC-DC with a wide load range. Finally, novel evaluation criteria are introduced to establish benchmark standards for AIoT chips. [ABSTRACT FROM AUTHOR]

Recently, there have been many studies attempting to take advantage of advancements in Artificial Intelligence (AI) in Analog and Mixed-Signal (AMS) circuit design. Automated circuit sizing optimization and improving the accuracy of performance models are the two predominant uses of AI in AMS circuit design. This paper first introduces and explains the basic concepts in AI especially the ones that are more suitable to this application. Next, it surveys some recent studies of various AI techniques for AMS circuit design. Then, it discusses the main approaches as well as the pros and cons of each method. Finally, it gives meaningful insights about the current challenges and open issues, as well as recommends approaches for specific applications. [ABSTRACT FROM AUTHOR]

This paper presents an improved baseline correction filter to be used in the readout front-end ASICs of gaseous detector systems in high energy physics experiments. The digital filter is based on an FIR section and used in the data pipeline running in real-time thought the input signals, allowing adequate data compression by the subsequent blocks in the processing chain. The proposed design was optimized to cope with the power and area trade-offs while hardening the block using redundancy and a novel automated design system based on artificial intelligence. This optimizer is also discussed including its logic and improvements using genetic algorithms with dynamic mutation and cross-over probabilities connected to run-time EDA tools. The designed filter was fabricated in 130 nm process, packaged and tested in the Peletron particle accelerator at the University of Sao Paulo. The device was irradiated and received a total dose of 70 krad during about one week of exposition to alpha particles without any measured failure, validating the design and solving the vulnerability with a small increase in the channel area of just 0.07%. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNN) have been proved to be an effective method in the field of artificial intelligence (AI), and large-scale deploying CNN to embedded devices, no doubt, will greatly promote the development and application of AI into the practical industry. However, mainly due to the space-time complexity of CNN, computing power, memory bandwidth and flexibility are performance bottlenecks. In this paper, a framework containing model compression and hardware acceleration is proposed to solve the above problems. This framework consists of a mixed pruning method, data storage optimization for efficient memory utilization and an accelerator for mapping CNN on field programmable gate array (FPGA). The mixed pruning method is used to compress the model, and data bit-width is reduced to 8-bit by data quantization. Accelerator based on FPGA makes it flexible, configurable and efficient for CNN implementation. The model compression is evaluated on NVIDIA RTX2080Ti, and the results illustrate that the VGG16 is compressed by $30\times $ and the fully convolutional network (FCN) is compressed by $11\times $ within 1% accuracy loss. The compressed model is deployed and accelerated on ZCU102, which is up to $1.7\times $ and $24.5\times $ better in energy efficiency compared with RTX2080Ti and Intel i7 7700. [ABSTRACT FROM AUTHOR]

Deep neural network (DNN) has been widely studied due to its high performance and usability for various applications such as image classification, detection, segmentation, translation, and action recognition. Thanks to the universal applications and high performance of DNN algorithm, DNN is adopted for various AI platforms, including edge/mobile devices as well as cloud servers. However, high-performance DNN requires a large amount of computation and memory access, making it challenging to implement DNN operation on edge/mobile. There have been several ways to solve these problems, including algorithms as well as hardware for DNN. Algorithms that help accelerate DNN in hardware enable much more efficient operation of high-performance AI. This article aims to provide an overview of the recent hardware and algorithm co-design schemes enabling efficient processing of DNNs. Specifically, it will provide algorithm optimization methods for DNN structure, neurons, synapses, and data types. This paper also introduces optimization methods for hardware architectures, PE array, data-path control, and microarchitecture of PE. And we will also show examples of DNN algorithm and hardware co-designed ASICs. [ABSTRACT FROM AUTHOR]

This paper evaluates the performance of the discrete-time linear extended state observer (ESO) for a disturbed nonlinear discrete-time system with uncertainties. Since the performance of the ESO is dominated by the choice of the gain parameter, the goal of this paper is to analyze the behavior of the observation error when the gain parameter varies from zero to infinity. We show that for the ESO considered, there exists a certain threshold such that if the gain parameter is smaller than this threshold, the observation error at steady state will become unbounded. An upper bound on the asymptotic observation error is also proposed, which is proved to be bounded and convergent utilizing set-theoretic techniques. For the case where the closed-loop matrix of the ESO is nonnegative, a tighter upper bound on the asymptotic observation error is also proposed. The proposed theoretic results are illustrated by the simulation results and the experimental verifications on a direct current motor loading platform. [ABSTRACT FROM AUTHOR]

Computing-in-memory (CIM) is a promising approach to reduce the latency and improve the energy efficiency of deep neural network (DNN) artificial intelligence (AI) edge processors. However, SRAM-based CIM (SRAM-CIM) faces practical challenges in terms of area overhead, performance, energy efficiency, and yield against variations in data patterns and transistor performance. This paper employed a circuit-system co-design methodology to develop a SRAM-CIM unit-macro for a binary-based fully connected neural network (FCNN) layer of the DNN AI edge processors. The proposed SRAM-CIM unit-macro supports two binarized neural network models: an XNOR neural network (XNORNN) and a modified binary neural network (MBNN). To achieve compact area, fast access time, robust operations, and high energy-efficiency, our proposed SRAM-CIM uses a split-wordline compact-rule 6T SRAM and circuit techniques, including a dynamic input-aware reference generation (DIARG) scheme, an algorithm-dependent asymmetric control (ADAC) scheme, a write disturb-free (WDF) scheme, and a common-mode-insensitive small offset voltage-mode sensing amplifier (CMI-VSA). A fabricated 65-nm 4-Kb SRAM-CIM unit-macro achieved 2.4- and 2.3-ns product-sum access times for a FCNN layer using XNORNN and MBNN, respectively. The measured maximum energy efficiency reached 30.49 TOPS/W for XNORNN and 55.8 TOPS/W for the MBNN modes. [ABSTRACT FROM AUTHOR]

Silicon-based static random access memories (SRAM) and digital Boolean logic have been the workhorse of the state-of-the-art computing platforms. Despite tremendous strides in scaling the ubiquitous metal-oxide-semiconductor transistor, the underlying von-Neumann computing architecture has remained unchanged. The limited throughput and energy-efficiency of the state-of-the-art computing systems, to a large extent, result from the well-known von-Neumann bottleneck. The energy and throughput inefficiency of the von-Neumann machines have been accentuated in recent times due to the present emphasis on data-intensive applications such as artificial intelligence, machine learning, and cryptography. A possible approach towards mitigating the overhead associated with the von-Neumann bottleneck is to enable in-memory Boolean computations. In this paper, we present an augmented version of the conventional SRAM bit-cells, called the X-SRAM, with the ability to perform in-memory, vector Boolean computations, in addition to the usual memory storage operations. We propose at least six different schemes for enabling in-memory vector computations, including NAND, NOR, IMP (implication), XOR logic gates, with respect to different bit-cell topologies − the 8T cell and the 8+T Differential cell. In addition, we also present a novel ‘read-compute-store’ scheme, wherein the computed Boolean function can be directly stored in the memory without the need of latching the data and carrying out a subsequent write operation. The feasibility of the proposed schemes has been verified using the predictive transistor models and detailed Monte-Carlo variation analysis. As an illustration, we also present the efficacy of the proposed in-memory computations by implementing advanced encryption standard algorithm on a non-standard von-Neumann machine wherein the conventional SRAM is replaced by X-SRAM. Our simulations indicated that up to 75% of memory accesses can be saved using the proposed techniques. [ABSTRACT FROM AUTHOR]

Autonomous mobile robots are receiving a lot of attention for many applications, such as package delivery and smart surveillance, however, the battery capacity is limited to implement intelligent decision making in robots because of the heavy computational costs. In this paper, an ultra-low-power artificial intelligence processor (AIP) is proposed for real-time decision making of autonomous mobile robots. To achieve low power consumption while maintaining high performance, it adopts four key features: 1) an 8-thread tree search processor for real-time path planning; 2) a reinforcement learning accelerator for the avoidance of unexpected obstacles; 3) a 3-level transposition table cache for the reduction of duplicated computations; and 4) a PVT compensation circuit for the stable operation at near-threshold voltage. The proposed 16 mm $^{{{2}}}$ AIP is fabricated using 65-nm triple-well CMOS technology. It consumes only 1.1 mW at 0.55 V supply voltage and 7 MHz operating frequency, and 151 mW at 1.2 V supply voltage and 245 MHz operating frequency. The AIP achieves fast search speed (470 000 search/s) and low energy consumption (79 nJ/search), and it is successfully applied to a battery-powered robot system for autonomous navigation without any collision in dynamic indoor environments. [ABSTRACT FROM PUBLISHER]

Deep neural network (DNN) has recently attracted tremendous attention in various fields. But the computing operation requirement and the memory bottleneck limit the energy efficiency of hardware implementations. Binary quantization is proposed to relieve the pressure of hardware design. And the Computing-In-Memory (CIM) is regarded as a promising method to resolve the memory wall challenge. However, the binary computing paradigm is mismatched with the CIM scheme, which incurs complex circuits and peripheral to realize binary operation in previous works. To overcome previous issues, this work presents Binary Representation Computation-In-Memory (BR-CIM) with several key features. (1) A lightweight computation unit is realized within the 6T SRAM array to accelerate binary computing and enlarge signal margin; (2) The reconfigurable computing scheme and mapping method support extendable bit precision to satisfy the accuracy requirement of various applications; (3) Simultaneous computing and weight loading is supported by column circuitry, which shortens the data loading latency; Several experiments are conducted to estimate algorithm accuracy, the computing latency, and power consumption. The energy efficiency reaches up to 1280 TOPs/W for binary representation. And the algorithm accuracy achieves 97.82%/76.4% on MNIST/CIFAR-100 dataset. [ABSTRACT FROM AUTHOR]

This work presents a fully-digital 64 Kb non-volatile ReRAM based compute-in-memory (CIM) macro for the modern artificial intelligence (AI) edge devices, using 65 nm technology. This digital CIM architecture effectively removes the analog-design issues, related to process variations, noise susceptibility, and data-conversion overhead. Hence, it offers no accuracy loss and high energy-efficiency for the computation. To incorporate the digital computation, a novel NAND logic based 3.25T1R bitcell is proposed. The digital behaviour of this cell makes it superior to the conventional 1T1R based analog bitcell. Also, with the inherent non-volatility of ReRAM, the proposed cell can be a good substitute for SRAM-based CIM architectures with $4.62\times $ , $1.96\times $ , $3.96\times $ , and $5.12\times $ lower area than the XNOR-based 12T, Twin-8T, 8T, and 6T SRAM cell respectively. Moreover, the proposed CIM architecture allows full reconfigurabiliy from 1 to 16b precision for both input and weight. It also allows activating any number of parallel inputs, ranging from 1 to 128. According to simulation results, the proposed macro successfully operates up to 166.6 MHz for 1/8/15b input/weight/output precision and achieves 27.28 TOPS/W without any accuracy loss. Removing sense amplifiers for the ReRAM mode of the proposed work claims additional area and power savings. [ABSTRACT FROM AUTHOR]

Emerging advanced System-on-Chip (SoC) designs contain more and more complicated functions to be accelerated. This presents a challenge to conventional design approaches which use different hardware architectures or separate hardware accelerators to implement the various functions. To tackle this challenge, for the first time, we propose a generalized hardware accelerator called “Huicore” to speed up diverse functions on the same substrate. Through the analysis and transformation of mathematical characteristics, we reveal the commonality of many complicated functions using the CORDIC algorithm. Then we explore a reconfigurable architecture to implement them. The proposed reconfigurable accelerator can not only accelerate the implementation of many complicated functions, but also has small area, low power consumption and high precision. It is very suitable for integration in a SoC system to accelerate the implementation of various applications. [ABSTRACT FROM AUTHOR]

This paper considers a class of nonsymmetric cooperative neural networks (NNs) where the neurons are fully interconnected and the neuron activations are modeled by piecewise linear (PL) functions. The solution semiflow generated by cooperative PLNNs is monotone but, due to the horizontal segments in the neuron activations, is not eventually strongly monotone (ESM). The main result in this paper is that it is possible to prove a peculiar form of the Limit Set Dichotomy for this class of cooperative PLNNs. Such a form is slightly weaker than the standard form valid for ESM semiflows, but this notwithstanding it permits to establish a result on convergence analogous to that valid for ESM semiflows. Namely, for almost every choice of the initial conditions, each solution of a fully interconnected cooperative PLNN converges toward an equilibrium point, depending on the initial conditions, as t \to +\infty. From a methodological viewpoint, this paper extends some basic techniques and tools valid for ESM semiflows, in order that they can be applied to the monotone semiflows generated by the considered class of cooperative PLNNs. [ABSTRACT FROM PUBLISHER]

This paper develops a neural network for solving the general nonsmooth convex optimization problems. The proposed neural network is modeled by a differential inclusion. Compared with the existing neural networks for solving nonsmooth convex optimization problems, this neural network has a wider domain for implementation. Under a suitable assumption on the constraint set, it is proved that for a given nonsmooth convex optimization problem and sufficiently large penalty parameters, any trajectory of the neural network can reach the feasible region in finite time and stays there thereafter. Moreover, we can prove that the trajectory of the neural network constructed by a differential inclusion and with arbitrarily given initial value, converges to the set consisting of the equilibrium points of the neural network, whose elements are all the optimal solutions of the primal constrained optimization problem. In particular, we give the condition that the equilibrium point set of the neural network coincides with the optimal solution set of the primal constrained optimization problem and the condition ensuring convergence to the optimal solution set in finite time. Furthermore, illustrative examples show the correctness of the results in this paper, and the good performance of the proposed neural network. [ABSTRACT FROM AUTHOR]

The paper addresses analytical and practical aspects of integration of equation-based (classical) and signal-derived [artificial neural network (ANN)] dynamic models for transient analysis of large-scale dynamical systems. Our signal-based part is based on two ANNs, and is derived from measurements at boundary points. In this paper, we describe this hybrid modeling technique, and focus on: 1) a least square-based mechanism for on-line correction of dynamic variable predictions that is based on actual operating conditions; 2) the resilience of the algorithm to missing measurements due to failed communication links; and 3) a complete two-way interaction between the differential-algebraic equation based subsystem and the ANN-based subsystem. The paper demonstrates the feasibility of implementing our approach in standard power system software by integrating the ANN-based model with the transient analysis toolbox from Matlab. We illustrate capabilities of the proposed approach for transient analysis on a benchmark multi-machine example derived from the New England/New-York interconnected power system. [ABSTRACT FROM AUTHOR]

Recurrent neural networks of the Lotka-Volterra model have been proven to possess characteristics which are desirable in some neural computations. A clear understanding of the dynamical properties of a recurrent neural network is necessary for efficient applications of the network. This paper studies the global convergence of general Lotka-Volterra recurrent neural networks with variable delays. The contributions of this paper are: 1) sufficient conditions are established for lower positive boundedness of the networks; 2) global exponential stability conditions are obtained for the networks. These conditions are totally independent of the variable delays which are therefore allowed to be uncertain; 3) novel Lyapunov functionals are constructed to establish delays dependent conditions for global asymptotic stability, and 4) simulation results and examples are provided to supplement and illustrate the theoretical contributions presented. [ABSTRACT FROM AUTHOR]

Multistable networks have attracted much interest in recent years, since multistability is of primary importance for some applications of recurrent neural networks where monostability exhibits some restrictions. This paper focuses on the analysis of dynamical property for a class of additive recurrent neural networks with nonsaturating linear threshold transfer functions. A milder condition is derived to guarantee the boundedness and global attractivity of the networks as compared to that presented in [6]. Dynamical properties of the equilibria of two-dimensional networks are analyzed theoretically, and the relationships between the equilibria features and network parameters (synaptic weights and external inputs) are revealed. In addition, the sufficient and necessary conditions for coexistence of multiple equilibria are obtained, which confirmed the observations in [14] with a cortex-inspired silicon circuit. The results obtained in this paper are applicable to both symmetric and nonsymmetric networks. Simulation examples are used to illustrate the theory developed in this paper. [ABSTRACT FROM AUTHOR]

This paper deals with the global convergence and stability of the Hopfield-type neural networks under the critical condition that M1(F) = L-1DF - (FW+WTF)/(2) is nonnegative for any diagonal matrix F, where W is the weight matrix of the network, L = diag(L1, L2,.. . , LN) with 4 being the Lipschitz constant of gi and G(u) = (g1(u1),g2(u2),. . ,gN(uN))T is the activation mapping of the network. Many stability results have been obtained for the Hopfield-type neural networks in the noncritical case that M1 (F) is positive definite for some positive definite diagonal matrix F. However, very few results are available on the global convergence and stability of the networks in the critical case. In this paper, by exploring two intrinsic features of the activation mapping, two generic global convergence results are established in the critical case for the Hopfield-type neural networks, which extend most of the previously known globally asymptotic stability criteria to the critical case. The results obtained discriminate the critical dynamics of the networks, and can be applied directly to a group of Hopfield-type neural network models. An example has also been presented to demonstrate both theoretical importance and practical significance of the critical results obtained. [ABSTRACT FROM AUTHOR]

In this paper, novel call admission control (CAC) algorithms are developed based on cellular neural networks. These algorithms can achieve high network utilization by performing CAC in real-time, which is imperative in supporting quality of service (QoS) communication over packet-switched networks. The proposed solutions are of basic significance in access technology where a subscriber population (connected to the Internet via an access module) needs to receive services. In this case, QoS can only be preserved by admitting those user configurations which will not overload the access module. The paper treats CAC as a set separation problem where the separation surface is approximated based on a training set. This casts CAC as an image processing task in which a complex admission pattern is to be recognized from a couple of initial points belonging to the training set. Since CNNs can implement any propagation models to explore complex patterns, CAC can then be carried out by a CNN. The major challenge is to find the proper template matrix which yields high network utilization. On the other hand, the proposed method is also capable of handling three-dimensional separation surfaces, as in a typical access scenario there are three traffic classes (e.g., two type of Internet access and one voice over asymmetric digital subscriber line. [ABSTRACT FROM AUTHOR]

In this paper, a novel paradigm for blind source separation in the presence of nonlinear mixtures is presented. In particular, the paper addresses the problem of post-nonlinear mixing followed by another instantaneous mixing system. This model is called here the post-nonlinear-linear model. The method is based on the use of the recently introduced flexible activation function whose control points are adaptively changed: a neural model based on adaptive B-spline functions is employed. The signal separation is achieved through an information maximization criterion. Experimental results and comparison with existing solutions confirm the effectiveness of the proposed architecture. [ABSTRACT FROM AUTHOR]

The increasing demand for data-intensive computing applications, such as artificial intelligence (AI) and more specifically machine learning (ML), raises the need for novel computing hardware architectures capable of massive parallelism in performing core algebraic operations. Among the new paradigms, in-memory computing (IMC) with analogue devices is attracting significant interest for its large-scale integration potential, together with unrivaled speed and energy performance. Here, we present a fully-analogue, universal primitive capable of executing linear algebra operations such as regression, generalized least-square minimization and linear system solution with and without preconditioning. We study the impact of the main circuit parameters on accuracy and bandwidth with analytical closed-form expressions and SPICE simulations. Scaling challenges due to parasitic resistance/capacitance and their impact on key parameters such as bandwidth and accuracy are discussed. Finally, a comparison with existing solvers belonging to the same IMC framework is made to assess advantages and disadvantages of the proposed circuit. [ABSTRACT FROM AUTHOR]

This paper proposes a novel algebraic integer (AI) based multi-encoding of Daubechies-4 and -6 2-D wavelet filters having error-free integer-based computation. Digital VLSI architectures employing parallel channels are proposed, physically realized and tested. The multi-encoded AI framework allows a multiplication-free and computationally accurate architecture. It also guarantees a noise-free computation throughput the multi-level multi-rate 2-D filtering operation. A single final reconstruction step (FRS) furnishes filtered and down-sampled image outputs in fixed-point, resulting in low levels of quantization noise. Comparisons are provided between Daubechies-4 and -6 designs in terms of SNR, PSNR, hardware structure, and power consumptions, for different word lengths. SNR and PSNR improvements of approximately 30% were observed in favour of AI-based systems, when compared to 8-bit fixed-point schemes (six fractional bits). Further, FRS designs based on canonical signed digit representation and on expansion factors are proposed. The Daubechies-4 and -6 4-level VLSI architectures are prototyped on a Xilinx Virtex-6 vcx240t-1ff1156 FPGA device at 282 MHz and 146 MHz, respectively, with dynamic power consumption of 164 mW and 339 mW, respectively, and verified on FPGA chip using an ML605 platform. [ABSTRACT FROM PUBLISHER]

This paper proposes a novel constructive training algorithm for cascade neural networks. By reformulating the cascade neural network as a linear-in-the-parameters model, we use the orthogonal least squares (OLS) method to derive a novel objective function for training new hidden units. With this objective function, the sum of squared errors (SSE) of the network can be maximally reduced after each new hidden unit is added, thus leading to a network with less hidden units and better generalization performance. Furthermore, the proposed algorithm considers both the input weights training and output weights training in an integrated framework, which greatly simplifies the training of output weights. The effectiveness of the proposed algorithm is demonstrated by simulation results. [ABSTRACT FROM PUBLISHER]

In this paper, the existence of oscillations for a type of recurrent neural network with time delays between neural interconnections is investigated. A new set of criteria is proposed to guarantee the existence of oscillations for a network model. The results are shown to be the generalization of some published works dealing with no delay systems. [ABSTRACT FROM PUBLISHER]

Different from gradient-based neural networks, a special kind of recurrent neural network (RNN) has recently been proposed by Zhang et al. for online matrix inversion. Such an RNN is designed based on a matrix-valued error function instead of a scalar-valued error function. In addition, it was depicted in an implicit dynamics instead of an explicit dynamics. In this paper, we develop and investigate a discrete-time model of Zhang neural network (termed as such and abbreviated as ZNN for presentation convenience), which is depicted by a system of difference equations. Comparing with Newton iteration for matrix inversion, we find that the discrete-time ZNN model incorporates Newton iteration as its special case. Noticing this relation, we perform numerical comparisons on different situations of using ZNN and Newton iteration for matrix inversion. Different kinds of activation functions and different step-size values are examined for superior convergence and better stability of ZNN. Numerical examples demonstrate the efficacy of both ZNN and Newton iteration for online matrix inversion. [ABSTRACT FROM AUTHOR]

This paper focuses on studying the H∞ and generalized H2 filtering problems for a class of delayed neural networks. The time-varying delay is only required to be continuous and bounded. Delay-dependent criteria are proposed such that the resulting filtering error system is globally exponentially stable with a guaranteed H∞ or generalized H2 performance. It is also shown that the designs of the desired filters are achieved by solving a set of linear matrix inequalities, which can be facilitated efficiently by resorting to standard numerical algorithms. It should be noted that, based on a novel bounding technique, several slack variables are introduced to reduce the conservatism of the derived conditions. Three examples with simulation results are provided to illustrate the effectiveness and performance of the developed approaches. [ABSTRACT FROM AUTHOR]

In this paper, a biologically inspired, CNN-based, multi-channel, texture boundary detection technique is presented. The proposed approach is similar to human vision system. The algorithm is simple and straightforward such that it can be implemented on the cellular neural networks (CNNs). CNN contains several important advantages, such as efficient real-time processing capability and feasible very large-scale integration (VLSI) implementation. The proposed algorithm also had been widely tested on synthetic texture images. Those texture images are randomly selected from the Brodatz textures database [1]. According to our simulation results, the boundaries of uniform textures can be detected quite successfully. For the nonuniform or nonregular textures, the results also indicate meaningful properties, and the properties also are consistent to the human visual sensation. The proposed algorithm also has been implemented on the CNN Universal Machine (CNN-UM), and yields similar results as the simulation on the PC. Based on the efficient performance of CNN-UM, the algorithm becomes very fast. [ABSTRACT FROM AUTHOR]

In this paper, discrete-time blind-source separation (BSS) of instantaneous mixtures is studied. Decorrelation-based sufficient criteria for BSS of stationary and nonstionary sources are derived based on nonstationarity and nonwhiteness. A gradient algorithm is proposed based on these criteria. A batch-data algorithm and an on-line algorithm are developed based on the corollaries of the BSS criteria. These algorithms are especially useful for the separation of nonstationary sources. They are robust to additive white noises if the time-delayed decorrelation and the nonstationarity of the sources are considered simultaneously in the algorithms. Experiment results show the effectiveness and performance of the proposed algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we continue to explore application of nonsmooth analysis to the study of global asymptotic robust stability (GARS) of delayed neural networks. In combination with Lyapunov theory, our approach gives several new types of sufficient conditions ensuring GARS. A significant common aspect of our results is their low computational complexity. It is demonstrated that the reported results can be verified either by conducting spectral decompositions of symmetric matrices associated with the uncertainty sets of network parameters, or by solving a semidefinite programming problem. Nontrivial examples are constructed to compare with some closely related existing results. [ABSTRACT FROM AUTHOR]

In this paper, a new Lyapunov functional-based method is proposed for the stability analysis of delayed cellular neural networks (DCNN). Global exponential stability conditions are obtained for the general DCNN, the Hopfield neural networks (]INNs), and delayed HNNs with monotonic nondecreasing and nonconstant activation functions. [ABSTRACT FROM AUTHOR]

In this paper, the complete stability of cellular neural networks with time-varying delays is analyzed using the induction method and the contraction mapping principle. Delay-dependent and delay-independent conditions are obtained for locally stable equilibrium points to be located anywhere, which differ from the existing results on complete stability where the existence of equilibrium points in the saturation region is necessary for complete stability and locally stable equilibrium points can be in the saturation region only. In addition, some existing stability results in the literature are special cases of a new result herein. Simulation results are also discussed by use of two illustrative examples. [ABSTRACT FROM AUTHOR]

This paper investigates synchronization dynamics of a general model of complex delayed networks as well as the effects of time delays. Some simple yet generic criteria ensuring delay-independent and delay-dependent synchronization are derived, which are less conservative than those reported so far in the literature. Moreover, a scale ‘v’ denoted by a function of the smallest and the second largest eigenvalues of coupling matrix is presented to analyze the effects of time delays on synchronization of the networks. Furthermore, various kinds of coupling schemes, including small-world networks and scale-free networks, are studied. It is shown that, if the coupling delays are less than a positive threshold, then the network will be synchronized. On the other hand, with the increase of the coupling delays, the synchronizability of the network will be restrained and even eventually desynchronized. The results are illustrated by a prototype composed of the chaotic Duffing oscillators. Numerical simulations are also given to verify theoretical results. [ABSTRACT FROM AUTHOR]

Blind source separation (BSS) aims at recovering statistically independent source signals from their linear mixtures without knowing the mixing coefficients. Besides independent component analysis, nonlinear principal component analysis (NPCA) is shown to be another useful tool for solving this problem, but it requires that the measured data be prewhitened. By taking into account the autocorrelation matrix of the measured data, we present in this paper a modified NPCA criterion, and develop a least-mean-square (LMS) algorithm and a recursive least-squares algorithm. They can perform the online BSS using directly the unwhitened observations. Since a natural gradient learning is applied and the prewhitening process is removed, the proposed algorithms work more efficiently than the existing NPCA algorithms, as verified by computer simulations on man-made sources as well as practical speech signals. [ABSTRACT FROM AUTHOR]

Overfitting has long been recognized as a problem endemic to models with a large number of parameters. The usual method of avoiding this problem in neural networks is to avoid fitting the data too precisely, and this technique cannot determine the exact model size directly. In this paper, we describe an alternative, information theoretic criterion to determine the number of neurons in the optimal model. When applied to the time series prediction problem we find that models which minimize the description length (DL) of the data, both generalize well and accurately capture the underlying dynamics. We illustrate our method with several computational and experimental examples. [ABSTRACT FROM AUTHOR]

In this paper, we develop an advanced architecture and come up with a comprehensive design methodology of genetically optimized hybrid fuzzy neural networks (gHFNNs). The construction of gHFNN exploits fundamental technologies of computational intelligence (CI), namely fuzzy sets, neural networks, and genetic algorithms (GAs). The architecture of the gHFNN results from a highly synergistic usage of the genetic optimization-driven hybrid system being generated by combining fuzzy neural networks (FNNs) with polynomial neural networks (PNN). FNN contributes to the formation of the premise part of the overall network structure of the gHFNN. The consequence part of the gHFNN is designed using PNNs. The optimization of the FNN is realized with the aid of a standard backpropagation learning algorithm and genetic optimization. As the consequence part of the gHFNN, the development of the PNN dwells on two general optimization mechanisms: the structural optimization is realized via GAs whereas in the case of the parametric optimization we proceed with a standard least square method-based learning (optimization). Through the consecutive process of such structural and parametric optimization, an optimized PNN becomes generated in a dynamic fashion. To evaluate the performance of the gHFNNs, we experimented with a number of representative numerical examples. A comparative analysis demonstrates that the proposed gHFNNs are neurofuzzy systems with higher accuracy as well as more superb predictive capability than other models available in the literature. [ABSTRACT FROM AUTHOR]

In this paper, the problem of stability analysis for a class of neural networks with distributed delays is investigated. Applying the M-matrix theory and new analysis technique, novel sufficient conditions for the existence, uniqueness, and global asymptotic stability of the equilibrium point of neural networks with distributed delays are derived. The new stability criteria can be applied to the case when the nondelayed terms cannot dominate the delayed terms, which have great significance in the design and application of neural networks with distributed delays. Three illustrative examples are presented which demonstrate the usefulness of the proposed results. [ABSTRACT FROM AUTHOR]

This paper is concerned with global robust stability of a general class of discrete-time interval neural networks which contain time-invariant uncertain parameters with their values being unknown but bounded in given compact sets. We first introduce the concept of diagonally constrained interval neural networks and present a necessary and sufficient condition for global robust stability of the interval networks regardless of the bounds of nondiagonal uncertain parameters of state feedback and connection weight matrices. Then we extend the result to general interval neural networks. Finally, simulation results illustrate the characteristics of the main results. [ABSTRACT FROM AUTHOR]

In this paper, using a method based on nonsmooth analysis and the Lyapunov method, several new sufficient conditions are derived to ensure existence and global asymptotic stability of the equilibrium point for delayed Cohen-Grossberg neural networks. The obtained criteria can be checked easily in practice and have a distinguished feature from previous studies, and our results do not need the smoothness of the behaved function, boundedness of the activation function and the symmetry of the connection matrices. Moreover, two examples are exploited to illustrate the effectiveness of the proposed criteria in comparison with some existing results. [ABSTRACT FROM AUTHOR]

In this paper, the global exponential stability and periodicity of a class of recurrent neural networks with time delays are addressed by using Lyapunov functional method and inequality techniques. The delayed neural network includes the well-known Hopfield neural networks, cellular neural networks, and bidirectional associative memory networks as its special cases. New criteria are found to ascertain the global exponential stability and periodicity of the recurrent neural networks with time delays, and are also shown to be different from and improve upon existing ones. [ABSTRACT FROM AUTHOR]

This paper proposes a new fuzzy neural network (FNN) capable of parameter self-adapting and structure self-cone structing to acquire a small number of fuzzy rules for interpreting the embedded knowledge of a system from the given training data set. The proposed FNN is inherently a modified Takagi-Sugeno-Kang (TSK)-type fuzzy-rule-based model with neural network's learning ability. There are no rules initiated at the beginning and they are created and adapted through an on-line learning processing that performs simultaneous structure. and parameter identification. In the structure identification of the precondition part, the input space is partitioned in a flexible way according to the newly proposed on-line independent component analysis (ICA) mixture model. The input space is thus represented by linear combinations of independent, non-Gaussian densities. The first input training pattern is assigned to the first rule initially by the on-line ICA mixture model. Afterwards, some additional significant terms (input variables) selected by the on-line ICA mixture model will be added to the consequent part (forming a liner equation of input variables) incrementally or create a new rule in the learning processing. The combined precondition and consequent structure identification scheme can make the network grow dynamically and efficiently. In the parameter identification, the consequent parameters are tuned by the back propagation rule and the precondition parameters are turned by the on-line ICA mixture model. Both the structure and parameter identifications are done simultaneously to form a fast learning scheme. The derived on-line ICA mixture model also provide a natural linear transformation for each input variable to enhance the knowledge representation ability of the proposed FNN and reduce the required rules and achieve higher accuracy efficiently. In order to demonstrate the performance of the proposed FNN, several experiments covering the areas of system identification, classification, and image segmentation are carried out. Our experiments show that the proposed FNN can achieve significant improvements in the convergence speed and prediction accuracy with simpler networks structure. [ABSTRACT FROM AUTHOR]

In this paper, we investigate synchronization of an array of linearly coupled identical connected neural networks with delays; Variational method is used to investigate local synchronization. Global exponential stability is studied, too. We do not assume that the coupling matrix A is symmetric or irreducible. The linear matrix inequality approach is used to judge synchronization with global convergence property. [ABSTRACT FROM AUTHOR]

A neural network is said to be convergent (or completely stable) when each trajectory tends to an equilibrium point (a stationary state). A stronger property is that of absolute stability, which means that convergence holds for any choice of the neural network parameters, and any choice of the nonlinear functions, within specified and well characterized sets. In particular, the property of absolute stability requires that the neural network be convergent also when, for some parameter values, it possesses nonisolated equilibrium points (e.g., a manifold of equilibria). Such a property, which is really well suited for solving several classes of signal processing tasks in real time, cannot be in general established via the classical LaSalle approach, due to its inherent limitations to study convergence in situations where the neural network has nonisolated equilibrium points. In this paper, a new method to address absolute stability is developed, based on proving that the total length of the neural network trajectories is finite. A fundamental result on absolute stability is given, under the hypothesis that the neural network possesses a Lyapunov function, and the nonlinearities involved (neuron activations, inhibitions, etc.) are modeled by analytic functions. At the core of the proof of finiteness of trajectory length is the use of some basic inequalities for analytic functions due to Łojasiewicz. The result is applicable to a large class of neural networks, which includes the networks pro- posed by Vidyasagar, the Hopfield neural networks, and the standard cellular neural networks introduced by Chua and Yang. [ABSTRACT FROM AUTHOR]

In this paper, we show that any N × M-dimensional delayed cellular neural network described using cloning templates can have no more than 3N×M isolated equilibrium points and 2 N×M of these equilibrium points located in saturation regions are locally exponentially stable. In addition, we give the conditions for the equilibrium points to be locally exponentially stable when the equilibrium points locate the designated saturation region. These conditions improve and extend the existing stability results in the literature. The conditions are also very easy to be verified and can be checked by direct examination of the templates, regardless of the number of cells. Finally, the validity and performance of the results are illustrated by use of two numerical examples. [ABSTRACT FROM AUTHOR]

This paper proposes a new image-descreening technique based on texture classification using a cellular neural network (CNN) with template trained by genetic algorithm (GA), called GA-CNN. Instead of using the fixed filters for image descreening, we are equipped with a more pliable mechanism for classifications in screening patterns. Using CNN makes it possible to get an accurate texture classification result in a faster speed by its superiority of implementable hardware and the flexible choices of templates. The use of the GA here helps us to look for the most appropriate template for CNNs more adaptively and methodically. The evolved parameters in the template for CNNs can not only provide a quicker classification mechanism but also help us with a better texture classification for screening patterns. After the class of screening patterns in the querying images is determined by the trained GA-CNN-based texture classification system, the recommendatory filters are induced to solve the screening problems. The induction of the classification in screening patterns has simplified the choke of filters and made it valueless to determine a new structured filter. Eventually, our comprehensive methodology is going to be topped off with more desirable results and the indication for the decrease in time complexity. [ABSTRACT FROM AUTHOR]

Lyapunov functionals and Lyapunov functions coupled with the Razumildiin technique are still the most popular tools in studying the stability of large-scale retarded nonlinear systems. However, it is generally difficult to construct Lyapunov functionals or functions that satisfy the strong conditions required in the classical stability theory. In this paper, we show that for some delay differential systems such as additive neural networks with delays, we can weaken the condition that the Lyapunov functional or function is positive definite, by using the equivalence between the state stability and the output stability. We apply our general theory to obtain some new stability conditions for cellular neural network models. It is represented that it is easy to construct Lyapunov functionals or functions satisfied conditions of our theorems. [ABSTRACT FROM AUTHOR]

In this paper, we propose a neural network for ranking a given set of N numbers In 0(1/N) time. The ordering of a set of numbers based on their relative magnitudes, which is analogous to sorting,. is a fundamental operation in many algorithms. In comparison with other sorting networks reported in the literature, the,: proposed network requires fewer neurons, and fewer interconnections between neurons. The interconnections use nonlinear synapses which are composed of comparators, and do not require any weighted interconnections between neurons, as used in conventional neural networks. The proposed network has many applications, including as a component of self-organizing feature maps aid other systems where sorting is a frequent operation. [ABSTRACT FROM AUTHOR]