Search

This paper presents a novel recurrent neural network for solving nonlinear convex programming problems subject to nonlinear inequality constraints. Under the condition that the objective function is convex and all constraint functions are strictly convex or that the objective function is strictly convex and the constraint function is convex, the proposed neural network is proved to be stable in the sense of Lyapunov and globally convergent to an exact optimal solution. Compared with the existing neural networks for solving such nonlinear optimization problems, the proposed neural network has two major advantages. One is that it can solve convex programming problems with general convex inequality constraints. Another is that it does not require a Lipschitz condition on the objective function and constraint function. Simulation results are given to illustrate further the global convergence and performance of the proposed neural network for constrained nonlinear optimization. [ABSTRACT FROM AUTHOR]

An efficient adaptive algorithm in real-time applications should make optimal use of the available computing power for reaching some specific design goals. Relying on appropriate strategies, the spatial resolution/temporal rate can be traded against computational complexity; and sensitivity traded against robustness, in an adaptive process. In this paper, we present an algorithmic framework where a spatial multigrid computing is placed within a temporal multirate structure, and at each spatial grid point, the computation is based on an adaptive multiscale approach. The algorithms utilize an analogic (analog and logic) architecture consisting of a high-resolution optical sensor, a low-resolution cellular sensor-processor and a digital signal processor. The proposed framework makes the acquisition of a spatio-temporally consistent image flow possible even in case of extreme variations (relative motion) in the environment. It ideally supports the handling of various difficult problems on a moving platform including terrain identification, navigation parameter estimation, and multitarget tracking. The proposed spatio-temporal adaptation relies on a feature-based optical-flow estimation that can be efficiently calculated on available cellular nonlinear network (CNN) chips. The quality of the adaptation is evaluated compared to nonadaptive spatio-temporal behavior where the input flow is oversampled, thus resulting in redundant data processing with an unnecessary waste of computing power. We also use a visual navigation example recovering the yaw-pitch-roll parameters from motion-field estimates in order to analyze the adaptive hierarchical algorithmic framework proposed and highlight the application potentials in the area of unmanned air vehicles. [ABSTRACT FROM AUTHOR]

In this paper, an on-chip implementation of the active contour technique called pixel-level snakes is proposed. This is based on an optimized cellular neural network (CNN) algorithm with capabilities to support changes in the contour topology. The entire algorithm has been implemented on a 64 × 64 CNN universal machine chip-set architecture for which the results of the time performance measurements are given. To illustrate the validity and capabilities of the proposed implementation some on-chip experiments are also included. [ABSTRACT FROM AUTHOR]

It is widely accepted that using a set of cellular neural networks (CNNs) in parallel can achieve higher level information processing and reasoning functions either from application or biologics points of views. Such an integrated CNN system can solve more complex intelligent problems. In this paper, we propose a novel framework for automatically constructing a multiple-CNN integrated neural system in the form of a recurrent fuzzy neural network. This system, called recurrent fuzzy CNN (RFCNN), can automatically learn its proper network structure and parameters simultaneously. The structure learning includes the fuzzy division of the problem domain and the creation of fuzzy rules and CNNs. The parameter learning includes the tuning of fuzzy membership functions and CNN templates. In the RFCNN, each learned fuzzy rule corresponds to a CNN. Hence, each CNN takes care of a fuzzily separated problem region, and the functions of all CNNs are integrated through the fuzzy inference mechanism. A new online adaptive independent component analysis mixture-model technique is proposed for the structure learning of RFCNN, and the ordered-derivative calculus is applied to derive the recurrent learning rules of CNN templates in the parameter-learning phase. The proposed RFCNN provides a solution to the current dilemma on the decision of templates and/or fuzzy rules in the existing integrated (fuzzy) CNN systems. The capability of the proposed RFCNN is demonstrated on the real-world defect inspection problems. Experimental results show that the proposed scheme is effective and promising. [ABSTRACT FROM AUTHOR]

Reaction-diffusion dynamics have been proposed to explain pattern-formation behavior: in a variety of systems, e.g., chemical and biological. This paper describes a cellular neural network chip that exhibits spatially organized patterns of activity, which are formed by reaction diffusion. The chip contains four 32 × 32 cell arrays of transistors which are locally coupled and operate in weak inversion. Typical patterns consist of alternating regions of high- and low-drain currents with a preferred width, but no preferred orientation. Experimental, theoretical, and simulation results for this chip are in complete concordance, demonstrating that conventional very large-scale integration technology can be an ideal substrate for studying the spatio-temporal dynamics and applications of reaction-diffusion. The chip, which was fabricated in a 0.5-pm process, settles to steady-state patterns within several hundred microseconds and dissipates 10.55 mW. [ABSTRACT FROM AUTHOR]

The promising potential of cellular neural networks (CNN) has resulted in the development of several template design methods. The CNN universal machine (CNN-UM), a programmable CNN, has made it possible to create image-processing algorithms that run on this platform. However, very large-scale integration implementations of CNN-UMs presented parameter deviations that do not occur on ideal CNN structures. Consequently, new design methods were developed aiming at more robust templates. Although these new templates were indeed more robust, erroneous behavior can still be observed. An alternative for chip-independent robustness is chip-specific optimization, where the template is targeted to an individual chip. This paper describes a solution proposal in this sense to automatically tune templates in order to make the chip react as an ideal CNN structure. The approach uses measurements of actual CNN-UM chips as part of the cost function for a global optimization method to find an optimal template given an initial approximation. Further improvements are achieved by generating chip-specific robust templates by doing a search for the best template among the optimal ones. The tuned templates are therefore customized versions that are expected to be much less sensitive to imperfections on the operation of CNN-UM chips. Results are presented for the binary and grayscale cases, including the case of grayscale output. It is expected that as this technique matures, it will give CNN-UM chips enough reliability to compete with digital systems in terms of robustness in addition to advantages of speed. [ABSTRACT FROM AUTHOR]

This paper describes the design of an adaptive control system for recording neural signals from tripolar cuff electrodes. The control system is based on an adaptive version of the true-tripole amplifier configuration and was developed to compensate for possible errors in the cuff electrode balance by continuously adjusting the gains of the two differential amplifiers. Thus, in the presence of cuff imbalance, the output signal-to-interference ratio is expected to be significantly increased, in turn reducing the requirement for post-filtering to reasonable levels and resulting in a system which is fully implantable. A realization in 0.8-μm CMOS technology is described and simulated and preliminary measured results are presented. Gain control is achieved by means of current-mode feedback and many of the system blocks operate in the current-mode domain. The chip has a core area of 0.4 mm² and dissipates 3 mW from ± 2.5 V power supplies. Measurements indicate that the adaptive control system is expected to be capable of compensating for up to ±5% errors in the tripolar cuff electrode balance. [ABSTRACT FROM AUTHOR]

This paper describes the electronic implementation of a four-layer cellular neural network architecture implementing two components of a functional model of neurons in the visual cortex: linear orientation selective filtering and half wave rectification. Separate ON and OFF layers represent the positive and negative outputs of two-phase quadrature Gabor-type filters, whose orientation and spatial-frequency tunings are electronically adjustable. To enable the construction of a multichip network to extract different orientations in parallel, the chip includes an address event representation (AER) transceiver that accepts and produces two-dimensional images that are rate encoded as spike trains. It also includes routing circuitry that facilitates point-to-point signal fan in and fan out. We present measured results from a 32 × 64 pixel prototype, which was fabricated in the TSMC0.25-μm process on a 3.84 by 2.54 mm die. Quiescent power dissipation is 3 mW and is determined primarily by the spike activity on the AER bus. Settling times are on the order of a few milliseconds. In comparison with a two-layer network implementing the same filters, this network results in a more symmetric circuit design with lower quiescent power dissipation, albeit at the expense of twice as many transistors. [ABSTRACT FROM AUTHOR]

In this paper, two new architectures for high-speed CMOS wave-pipelined current-mode A/D converters (WP-LADCs) are proposed and analyzed. In the new WP-IADC architectures, the wave-pipelined theory is applied to both pipeline structures, called full WP-IADC (FWP-IADC) and Indirect transfer WP-IADC (ITW-IADC). In the FWP-IADC, each stage uses the full current-mode wave-pipelined structure without Switched-current cell circuits. In the ITWP-LADC, the switched-current cells are incorporated into the wave-pipelined stages which are divided into several sections with controlled clocks. Therefore, the proposed ITWP-IADC performs optimally in terms of speed and accuracy in the WP-IADCs. Generally, the proposed WP-LADCs have the advantages of high speed, high input frequency, high efficiency of timing usage, high clock-period flexibility in switched-current cells for precision enhancement, and reduced number of switched-current cells in the overall data path for linearity improvement. According to the theoretical analysis on the proposed WP-LADC structures, the minimum sampling clock period is proportional to the intrinsic delay of the current mirror and the increased rise/fall time in each wave-pipelined stage. The HSPICE simulation results reveal that, under Nyquist rate sampling in 8-b resolution, a sampling rate of 20 and 54 MHz can he achieved for FWP-IADC and two-section ITWP-IADC, respectively, if four wave-pipelined sections are used, the ITWP-LADC can be operated at 166 MHz at En input frequency of 8 MHz. To experimentally verify the correct function of the proposed WP-IADC structures, the proposed new architecture of the FWP-LADC Is implements by using 0.35-μm CMOS technology. The measurement results successfully demonstrate the feasibility of wave-pipelined IADC architectures in applications of high-speed ADCs. [ABSTRACT FROM AUTHOR]

Efficient deployment of deep neural networks (DNNs) emerges with the exploding demand for artificial intelligence on edge devices. Mixed-precision inference with both compressed model and reduced computation cost enlightens a way for accurate and efficient DNN deployments. Despite obtaining mixed-precision DNN models at the algorithmic level, there still lacks sufficient hardware support. In this work, we propose BitSystolic, a neural processing unit based on a systolic array structure. In BitSystolic, the numerical precision of both weights and activations can be configured in the range of 2 - 8 bits, fulfilling different requirements across mixed-precision models and tasks. Moreover, BitSystolic can support various data flows presented in different types of neural layers (e.g., convolution, fully-connected, and recurrent neural layers) and adaptive optimization of data reuse by switching between the matrix-matrix mode and vector-matrix mode. We designed and fabricated the proposed BitSystolic composed of a $16\times 16$ systolic array. Our measurement results show that BitSystolic features the unified power efficiency of up to 26.7 TOPS/W with 17.8 mW peak power consumption across various layer types. [ABSTRACT FROM AUTHOR]

The design of systems implementing low precision neural networks with emerging memories such as resistive random access memory (RRAM) is a significant lead for reducing the energy consumption of artificial intelligence. To achieve maximum energy efficiency in such systems, logic and memory should be integrated as tightly as possible. In this work, we focus on the case of ternary neural networks, where synaptic weights assume ternary values. We propose a two-transistor/two-resistor memory architecture employing a precharge sense amplifier, where the weight value can be extracted in a single sense operation. Based on experimental measurements on a hybrid 130 nm CMOS/RRAM chip featuring this sense amplifier, we show that this technique is particularly appropriate at low supply voltage, and that it is resilient to process, voltage, and temperature variations. We characterize the bit error rate in our scheme. We show based on neural network simulation on the CIFAR-10 image recognition task that the use of ternary neural networks significantly increases neural network performance, with regards to binary ones, which are often preferred for inference hardware. We finally evidence that the neural network is immune to the type of bit errors observed in our scheme, which can therefore be used without error correction. [ABSTRACT FROM AUTHOR]

Recent years have seen an increasing interest in the development of artificial intelligence circuits and systems for edge computing applications. In-memory computing mixed-signal neuromorphic architectures provide promising ultra-low-power solutions for edge-computing sensory-processing applications, thanks to their ability to emulate spiking neural networks in real-time. The fine-grain parallelism offered by this approach allows such neural circuits to process the sensory data efficiently by adapting their dynamics to the ones of the sensed signals, without having to resort to the time-multiplexed computing paradigm of von Neumann architectures. To reduce power consumption even further, we present a set of mixed-signal analog/digital circuits that exploit the features of advanced Fully-Depleted Silicon on Insulator (FDSOI) integration processes. Specifically, we explore the options of advanced FDSOI technologies to address analog design issues and optimize the design of the synapse integrator and of the adaptive neuron circuits accordingly. We present circuit post-layout simulation results and demonstrate the circuit’s ability to produce biologically plausible neural dynamics with compact designs, optimized for the realization of large-scale spiking neural networks in neuromorphic processors. [ABSTRACT FROM AUTHOR]

Convolutional neural network (CNN) offers significant accuracy in image detection. To implement image detection using CNN in the Internet of Things (IoT) devices, a streaming hardware accelerator is proposed. The proposed accelerator optimizes the energy efficiency by avoiding unnecessary data movement. With unique filter decomposition technique, the accelerator can support arbitrary convolution window size. In addition, max-pooling function can be computed in parallel with convolution by using separate pooling unit, thus achieving throughput improvement. A prototype accelerator was implemented in TSMC 65-nm technology with a core size of 5 mm2. The accelerator can support major CNNs and achieve 152GOPS peak throughput and 434GOPS/W energy efficiency at 350 mW, making it a promising hardware accelerator for intelligent IoT devices. [ABSTRACT FROM PUBLISHER]

A multiplier-less architecture based on algebraic integer representation for computing the Daubechies 6-tap wavelet transform for 1-D/2-D signal processing is proposed. This architecture improves on previous designs in a sense that it minimizes the number of parallel 2-input adder circuits. The algorithm was achieved using brute-force numerical optimization of the algebraic integer representation. The proposed architecture furnishes exact computation up to the final reconstruction step, which is the operation that maps the exactly computed filtered results from algebraic integer representation to fixed-point. Compared to our recent work, this architecture shows a reduction of 27·n-16 adder circuits, where n is the number of wavelet decomposition levels. The design is physically implemented for a 4-level 1-D/2-D decomposition using a Xilinx Virtex-6 vcx240t-1ff1156 field programmable gate array (FPGA) device operating at up to a maximum clock frequency of 344/ 168 MHz. The FPGA implementation of 1-D/2-D are tested using hardware co-simulation using an ML605 board with clock of 100 MHz. A 45 nm CMOS synthesis of 2-D designs show improved clock frequency of better than 306 MHz for a supply voltage of 1.1 V. [ABSTRACT FROM AUTHOR]

A 4th order RF LC-based \Sigma\Delta ADC clocked at 3.6 GHz and centered at 900 MHz is presented. A simple design methodology is used to derive a robust architecture with a minimum number of feedback coefficients. The simplicity of the ADC architecture results in a significant performance enhancement and power consumption reduction. An efficient algorithm for the tuning and calibration of the \Sigma\Delta LC-based loop filter is also presented. The ADC, suitable for cognitive Software Defined Radio applications, is implemented in a standard 130 nm CMOS technology. It achieves a 52 dB SFDR and a 50 dB SNR in a 28 MHz BW and consumes only 15 mW from a 1.2 V supply. The Figure of Merit of the ADC is 1.0 pJ/bit, which is to date the best reported FoM for an RF ADC. The effect of the clock jitter on the ADC performance is also measured and presented. [ABSTRACT FROM PUBLISHER]

A fast simulation method for time-derivative cellular neural networks (TDCNN) is proposed. Using forward Euler approximation (FEA) for the derivative of the cell state and the backward Euler approximation (BEA) for the derivatives of the neighboring cell states enables the recursive computation of the cell state and provides a speed advantage of orders of magnitude. The state equations are then packed into a vector-matrix form which enables the previously empirically given time constraint to be expressed as a matrix condition. It is shown that using both FEA and BEA leads to a second-order difference equation whose corresponding second-order differential equation is derived and shown to yield the same simulation results. [ABSTRACT FROM PUBLISHER]

Neuromorphic computing is an attractive avenue of research for processing and learning complex real-world data. With technology migration into nano and molecular scales several area and power efficient approaches to the design and implementation of artificial neural networks have been proposed. The discovery of the memristor has further enabled the realization of denser nanoscale logic and memory systems by facilitating the implementation of multilevel logic. Specifically, the innate reconfigurability of memristors can be exploited to realize synapses in artificial neural networks. This work focuses on the development of a variation-tolerant training methodology to efficiently reconfigure memristive synapses in a Trainable Threshold Gate Array (TTGA) system. The training process is inspired from the gradient descent machine learning algorithm commonly used to train artificial threshold neural networks, perceptrons. The design and CMOS/Nano implementation of the TTGA system from trainable perceptron based threshold gates is detailed and results are provided to showcase the training process and performance characteristics of the proposed system. Also shown are the results for training a 1T1M (1 Transistor and 1 Memristor) multilevel memristive memory and its performance characteristics. [ABSTRACT FROM PUBLISHER]

The growing interest in pulse-mode processing by neural networks is encouraging the development of hardware implementations of massively parallel networks of integrate-and-fire neurons distributed over multiple chips. Address-event representation (AER) has long been considered a convenient transmission protocol for spike based neuromorphic devices. One missing, long-needed feature of AER-based systems is the ability to acquire data from complex neuromorphic systems and to stimulate them using suitable data. We have implemented a general-purpose solution in the form of a peripheral component interconnect (PCI) board (the PCI-AER board) supported by software. We describe the main characteristics of the PCI-AER board, and of the related supporting software. To show the functionality of the PCI-AER infrastructure we demonstrate a reconfigurable multichip neuromorphic system for feature selectivity which models orientation tuning properties of cortical neurons. [ABSTRACT FROM AUTHOR]

Global robust convergence properties of continuous-time neural networks with discrete delays are studied. By employing suitable Lyapunov functionals, we derive a set of delay-independent sufficient conditions for the existence, uniqueness, and global robust asymptotic stability of the equilibrium point. The conditions can be easily verified as they can be expressed in terms of the network parameters only. Some numerical examples are given to compare our results with previous robust stability results derived in the literature. One of our main results is shown to improve and generalize a previously published result. Other results proved to establish a new set of robust stability criteria for delayed neural networks. [ABSTRACT FROM AUTHOR]

A spectral technique is proposed for investigating periodic oscillations and bifurcations in cellular nonlinear networks. The method consists of the following three fundamental steps: 1) an accurate estimation of the whole set of stable and unstable limit cycles is provided through the application of the describing function technique; 2) a detailed characterization of each limit cycle is obtained via the harmonic balance (HB) technique, by exploiting as input parameter the single harmonic approximation, provided by the describing function technique; 3) limit cycle stability and bifurcations are studied by computing the Floquet's multipliers and exploiting both a time-domain and an HR-based technique. [ABSTRACT FROM AUTHOR]

Cellular neural networks (CNNs) are dynamical systems, described by a large set of coupled nonlinear differential equations. The equilibrium-point analysis is an important step for understanding the global dynamics and for providing design rules. We yield a set of sufficient conditions (and a simple algorithm for checking them) ensuring the existence of at least one stable equilibrium point. Such conditions give rise to simple constraints, that extend the class of CNNs, for which the existence of a stable equilibrium point is rigorously proved. In addition, they are suitable for design and easy to check, because they are directly expressed in term of the template elements. [ABSTRACT FROM AUTHOR]

A recently reported result concerning the global robust stability of the equilibrium point of Hopfield-type delayed neural networks is revisited. It is shown that the result is not always right. [ABSTRACT FROM AUTHOR]

This May 2004 issue of the journal IEEE Transactions on Circuits and System has been organized into sections that reflect some of the major themes that have arisen over recent years. Each section contains articles detailing advances in theory, application, design, and implementation consistent with the broad multidisciplinary scope of cellular neural/nonlinear network (CNN). The first section highlights the current state of the art in single-layer CNNs, where the state of each cell is a scalar. The second section highlights recent developments in multilayer CNNs, which can exhibit a richer range of dynamical behaviors than the single-layer CNN. The final section contains articles that pave the way for the future development of CNN technology and active wave computing.