Search

Artificial intelligence is used for various applications and is promising as an indispensable infrastructure in future societies. Neural networks are representative technologies that imitate human brains and exhibit various advantages. However, the size is bulky, the power is huge, and some advantages are not demonstrated because they are executed on Neumann-type computers. Neuromorphic systems are biomimetic systems from the hardware level to implement neuron and synapse elements, and the size is compact, the power is low, and the operation is robust. However, because the conventional ones are not composed of fully optimized hardware, the power is not yet minimal, and extra control circuits must be used. In this article, we developed a neuromorphic system using memcapacitors and autonomous local learning. By using memcapacitors, the power can be minimal, and by using autonomous local learning, the control circuits to handle the synapse elements can be deleted. First, the memcapacitors are completed in a cross-bar array, where the ferroelectric layers are sandwiched between the horizontal and perpendicular electrodes. The polarization and capacitance exhibit hysteresis due to the dielectric polarization. Next, autonomous local learning is introduced as follows. During the training phase, associative patterns to be memorized are directly sent, relatively high voltages are applied, and dielectric polarizations are induced. During the operation phase, relatively low voltages are applied, and input signals are weighted with the capacitances of the memcapacitors, summed, and transferred as the output signals. Finally, the experimental system is set up, and the experimental results are acquired. The memorized patterns during the training phase, distorted patterns as the input signals during the operation phase, and retrieved patterns as the output signals in the operation phase are shown. Researchers found that the retrieved patterns are completely the same as the memorized patterns. This means that the neuromorphic system works as an associative memory.

Dynamic memristor (DM)-cellular neural networks (CNNs), which replace a linear resistor with flux-controlled memristor in the architecture of each cell of traditional CNNs, have attracted researchers' attention. Compared with common neural networks, the DM-CNNs have an outstanding merit: when a steady state is reached, all voltages, currents, and power consumption of DM-CNNs disappeared, in the meantime, the memristor can store the computation results by serving as nonvolatile memories. The previous study on stability of DM-CNNs rarely considered time delay, while delay is quite common and highly impacts the stability of the system. Thus, taking the time delay effect into consideration, we extend the original system to DM-D(delay)CNNs model. By using the Lyapunov method and the matrix theory, some new sufficient conditions for the global asymptotic stability and global exponential stability with a known convergence rate of DM-DCNNs are obtained. These criteria generalized some known conclusions and are easily verified. Moreover, we find DM-DCNNs have 3ⁿ equilibrium points (EPs) and 2ⁿ of them are locally asymptotically stable. These results are obtained via a given constitutive relation of memristor and the appropriate division of state space. Combine with these theoretical results, the applications of DM-DCNNs can be extended to other fields, such as associative memory, and its advantage can be used in a better way. Finally, numerical simulations are offered to illustrate the effectiveness of our theoretical results.

In this work, we are interested in a class of Bi-directional associative memory neural networks (BAMNNs) with hybrid time-varying delays and D operator. We establish some sufficient conditions to guarantee the existence and global exponential stability of measure-pseudo almost periodic solutions of the considered BAMNNs, using contraction mapping principle, exponential dichotomy theory, differential inequality techniques, and the properties of the measure-pseudo almost periodic functions. A numerical example along with a graphical illustration are presented to support the effectiveness and fasibility of the obtained results. The results in this paper are original and complement the previous outcomes.

In this paper, a physical Sr 0.97 Ba 0.03 TiO 3 - x (SBT) memristor with the voltage threshold characteristic was prepared. For characterizing the electrical characteristics of SBT memristor accurately, an adaptive voltage threshold memristor model was proposed. The synaptic plasticity of SBT memristor with different pulse stimulation was analyzed. Moreover, a Pavlov associative memory circuit was designed in the LTSPICE environment, which is composed of neuron circuit and memristive synaptic circuit. The neuron circuit can generate spike signals when the input signals exceed the threshold, and the synaptic weight can be tunable continuously by spike signals. Different from other memristive synaptic circuits, the proposed memristive synaptic circuit is based on physical SBT memristor, and the change of synaptic weight can be truly reflected in the circuit. In Pavlov’s dog experiment, when the presynaptic neuron receives the conditioned stimulus (CS) before the unconditioned stimulus (US), the synaptic weight of SBT memristor is increasing and the associative memory is building. When the presynaptic neuron receives the CS alone, or presynaptic neuron receives the CS after the US, the synaptic weight of SBT memristor is decreasing and the associative memory is losing. This phenomenon is highly similar to the building and losing processes of biological associative memory. These experimental results verify the feasibility and applicability of SBT memristor as electronic synapse in neuromorphic applications, and pave the way towards further development of artificial neural networks.

Artificial Neural Network (ANN) is a form of computation inspired by the arrangement and purpose of the brain. Pattern Recognition is one of the demanding areas under Artificial Neural Network and pattern association is one of the major tasks through which pattern recognition is realized. Nowadays researchers are taking interest in the combination of evolutionary algorithms and ANN. In the present work, an attempt is being made to analyze the performance of Hopfield Associative Memory with Monte Carlo adaptation rule and evolutionary searching (Genetic Algorithm) for pattern storage and recall. The training pattern set has been taken a set of five Greek alphabet letters as different objects. The aim is to store Patterns with MC adaptation rule and get the best weight matrices for efficient recalling of any given set patterns. The neural networks architectures have been taken up for storing the given set of inputs, for which conventional Hebbian and Monte Carlo adaptation rule is considered. During the mutation process, various mutation probabilities are supposed to be evaluated. The detailed simulated results thus obtained will be presented with the help of tables and graph.

First, the cerebellar model articulation controller (CMAC), invented in the early 1970s by J S Albus, and the associative memory system (AMS), developed for learning control systems by H Tolle, E Ersu and J Militzer in the early 1980s, are briefly described. The underlying mathematics of the AMS learning or training algorithm is then given with a geometrical interpretation from which its convergence properties may be deduced. These are illustrated for some simple cases. The original algorithm devised by Albus is very simple to compute but is slow to converge, and the second part of the paper investigates various methods of speeding up the algorithm. From an application of these new algorithms to test cases one is strongly recommended for further evaluation. The results reported here are of relevance also to the topical and rapidly growing field of neural computing.

A new concept of a multi-valued associative memory is introduced, generalizing a similar one in fuzzy neural networks. We expand the results on fuzzy associative memory with thresholds, to the case of a multi-valued one: we introduce the novel concept of such a network without numbers, investigate its properties, and give a learning algorithm in the multi-valued case. We discovered conditions under which it is possible to store given pairs of network variable patterns in such a multi-valued associative memory. In the multi-valued neural network, all variables are not numbers, but elements or subsets of a lattice, i.e., they are all only partially-ordered. Lattice operations are used to build the network output by inputs. In this paper, the lattice is assumed to be Brouwer and determines the implication used, together with other lattice operations, to determine the neural network output. We gave the example of the network use to classify aircraft/spacecraft trajectories., Comment: This is a version with correct Theorem 3 (Theorem 2 in published variant)

Learning from a few examples (one/few-shot learning) on the fly is a key challenge for on-device machine intelligence. We present the first chip-level demonstration of one-shot learning with Stanford Associative memory for Programmable, Integrated Edge iNtelligence via life-long learning and Search (SAPIENS), a resistive random access memory (RRAM)-based non-volatile associative memory (AM) chip that serves as the backend for memory-augmented neural networks (MANNs). The 64-kb fully integrated RRAM-CMOS AM chip performs long-term feature embedding and retrieval, demonstrated on a 32-way one-shot learning task on the Omniglot dataset. Using only one example per class for 32 unseen classes during on-chip learning, SAPIENS achieves 79% measured inference accuracy on Omniglot, comparable to edge software model accuracy using five-level quantization (82%). It achieves an energy efficiency of 118 GOPS/W at 200 MHz for in-memory L1 distance computation and prediction. Multi-bank measurements on the same chip show that increasing the capacity from three banks (24 kb) to eight banks (64 kb) improves the chip accuracy from 73.5% to 79%, while minimizing the accuracy excursion due to bank-to-bank variability.

In this paper, an ideal current source forgetting memristor model is proposed. Based on the model, three kinds of synapses with long- and short-term memory are designed: the series forgetting memristor synapse, the forgetting memristor bridge synapse written independently and the forgetting memristor bridge synapse written in batches. Combined with the forgetting property, the long- and short-term weight of the forgetting synapse can be controlled by the long- and short-term resistance of memristors. Compared the three forgetting synapses, the series forgetting memristor synapse has the lowest requirement for memristors, the forgetting memristor bridge synapse written independently is the most flexible, and the forgetting memristor bridge synapse written in batches is the most convenient. Compared with traditional synapses, forgetting synapses with long- and short-term memory have multi-weight storage. When forgetting synapses are applied to associative memory, it can be found more patterns are stored in the neural network and different patterns are recalled at different time due to the forgetting effect.

An associative memory is an artificial neural network model designed to store and recall input-output patterns pairs by association. A new method for obtaining two associative memories +M and -M is presented in this article, which uses a framework focused on two new binary operations boxplus and boxminus, and a unary operation called projection. The conditions to obtaining perfect recovery and the noise boundaries that the memories can tolerate are studied. Memories +M and -M are robust to additive and subtractive noise, respectively, both types converge in one-step and operate in heteroassociative and autoassociative modes. The performance of the proposed memories is tested against other memory models under identical conditions using the gamma binary distance for measuring similarity between binary patterns. The computer simulation results based on the average of 500 trials with the binary set of 26 lowercase letters of 7x7 pixel size, showed that the proposed memories in autoassociative mode exhibited a gamma binary distance of above .8, .75 and .7 by distorting up to 10 pixels by additive, subtractive, and mixed noise, respectively, which implied that the recovered images had a high similarity. In absence of noise the performance was excellent, i.e., the 100% of the recovered images were identical.

Alzheimer’s disease (AD), characterized by cognitive impairment, mainly affects middle-aged and elderly people. As the aging process of the world continues to intensify, AD harms people’s life, economy and society more and more seriously. Therefore, it has become an urgent problem to study the pathogenesis of AD and seek treatment on this basis. Hybrid synapse, autapse and spatial correlated noise in diverse neural activities have been investigated separately, however, theoretically understanding combination of them still has not been fully studied. Here in this paper, a neural network with multiple associative memory abilities is established from the perspective of the degeneration of associative memory ability in AD patients under the conditions of hybrid synapse, autapse and spatial correlated noise. In order to explore the pathogenesis, a synaptic loss and synaptic compensation model are established to analyze the associative memory ability of AD in different degrees of disease. The simulation results demonstrate the effectiveness of the proposed models and pave a way in the study of dynamic mechanism with higher bio-interpretability in neural networks.

The interplay between structure and function affects the emerging properties of many natural systems. Here we use an adaptive neural network model that couples activity and topological dynamics and reproduces the experimental temporal profiles of synaptic density observed in the brain. We prove that the existence of a transient period of relatively high synaptic connectivity is critical for the development of the system under noise circumstances, such that the resulting network can recover stored memories. Moreover, we show that intermediate synaptic densities provide optimal developmental paths with minimum energy consumption, and that ultimately it is the transient heterogeneity in the network that determines its evolution. These results could explain why the pruning curves observed in actual brain areas present their characteristic temporal profiles and they also suggest new design strategies to build biologically inspired neural networks with particular information processing capabilities., The authors acknowledge financial support from the Spanish Ministry of Science and Technology, and the Agencia Espanola de Investigacion (AEI), Spain under grant FIS2017-84256-P (FEDER funds) and from the Consejeria de Conocimiento, Investigacion Universidad, Junta de Andalucia and European Regional Development Funds, Spain, Refs. SOMM17/6105/UGR and A-FQM-175UGR18. APM also acknowledges support from ``Obra Social La Caixa, Spain'' (ID 100010434 with code LCF/BQ/ES15/10360004) and from ZonMw, Netherlands and the Dutch Epilepsy Foundation, Netherlands, project number 95105006. SJ acknowledges support from the Alan Turing Institute under EPSRC, United Kingdom Grant No. EP/N510129/1.

Associative memory is a widespread self-learning method in biological livings, which enables the nervous system to remember the relationship between two concurrent events. The significance of rebuilding associative memory at a behavior level is not only to reveal a way of designing a brain-like self-learning neuromorphic system but also to explore a method of comprehending the learning mechanism of a nervous system. In this paper, an associative memory learning at a behavior level is realized that successfully associates concurrent visual and auditory information together (pronunciation and image of digits). The task is achieved by associating the large-scale artificial neural networks (ANNs) together instead of relating multiple analog signals. In this way, the information carried and preprocessed by these ANNs can be associated. A neuron has been designed, named signal intensity encoding neurons (SIENs), to encode the output data of the ANNs into the magnitude and frequency of the analog spiking signals. Then, the spiking signals are correlated together with an associative neural network, implemented with a three-dimensional (3-D) memristor array. Furthermore, the selector devices in the traditional memristor cells limiting the design area have been avoided by our novel memristor weight updating scheme. With the novel SIENs, the 3-D memristive synapse, and the proposed memristor weight updating scheme, the simulation results demonstrate that our proposed associative memory learning method and the corresponding circuit implementations successfully associate the pronunciation and image of digits together, which mimics a human-like associative memory learning behavior.

In traditional neural network integration, people adopt Boosting, Bagging and other methods to integrate traditional neural networks. The integration is complex, time-consuming and laborious, difficult to popularize and apply. This paper is not a continuation of this method, but another integration which is called by us morphological neural network integration (MNNI) or morphological associative memory integration (MAMI). These networks used in MAMI are a network family, with 10 family members, unified in the morphological associative memory framework. Various morphological associative memory networks can be directly used as individual networks to learn and work separately, and then synthesize to draw conclusions. The results of some experiments show that this method is not only feasible in theory, but also effective in practice. It can avoid the complexity of traditional integration method, make the integration structure simple and clear, easy to operate, save time, and therefore is a method of neural network integration with research and application value. The contribution of this paper lies in that: (1) it proposed the concept and method of MNNI and, (2) verified the effectiveness of MNNI through experiments and, (3) it has the characteristics of simplicity, saving time and labor and cost, with a good application prospect and, (4) thus promoting the development of morphological neural networks in theory and practice.

A complex-valued Hopfield neural network (CHNN) has been applied as a multistate neural associative memory. Although synchronous mode accelerates the recall process, a CHNN with a projection rule may be trapped at a cycle of length two. A hyperbolic-valued Hopfield neural network (HHNN) with a conventional projection rule converges to a fixed point in synchronous mode. A noise robust projection rule improves the noise tolerance of HHNN, though it is not able to be applied to an HHNN in synchronous mode. In this work, we proposed hybrid mode, that is, asynchronous mode after synchronous mode. The conventional projection rule is employed in synchronous mode, and the noise robust projection rule is employed in asynchronous mode. Thus, the HHNN in hybrid mode converges in both modes. The HHNN in hybrid mode is expected to provide better noise tolerance than an HHNN in synchronous mode and faster recall than an HHNN in asynchronous mode. Computer simulations imply that our expectations are achieved.

This paper addresses the robust stability of recurrent neural networks (RNNs) with time-varying delays and input perturbation, where the time-varying delays include discrete and distributed delays. By employing the new $\psi $ -type integral inequality, several sufficient conditions are derived for the robust stability of RNNs with discrete and distributed delays. Meanwhile, the robust boundedness of neural networks is explored by the bounded input perturbation and $\mathcal {L}^{1}$ -norm constraint. Moreover, RNNs have a strong anti-jamming ability to input perturbation, and the robustness of RNNs is suitable for associative memory. Specifically, when input perturbation belongs to the specified and well-characterized space, the results cover both monostability and multistability as special cases. It is revealed that there is a relationship between the stability of neural networks and input perturbation. Compared with the existing results, these conditions proposed in this paper improve and extend the existing stability in some literature. Finally, the numerical examples are given to substantiate the effectiveness of the theoretical results.

A complex-valued multistate Hopfield neural network (CMHNN) is a multistate model of a Hopfield neural network and has been applied as a multistate neural associative memory. CMHNNs require many resources for weight parameters. To reduce the number of weight parameters, twin-multistate activation functions were introduced. A quaternion-valued twin-multistate Hopfield neural network (QTMHNN) is the first model to employ a twin-multistate activation function. A bicomplex-valued twin-multistate Hopfield neural network (BTMHNN) was also introduced. Weak noise tolerance is a disadvantage of QTMHNNs and BTMHNNs. To improve the noise tolerance, a BTMHNN can be modified to a bicomplex-valued twin–hyperbolic Hopfield neural network (BTHHNN). A BTMHNN is defined by the decomposition of a bicomplex number to a pair of complex numbers, whereas a BTHHNN is defined by decomposition of a bicomplex number to a pair of hyperbolic numbers. Computer simulations have improved the noise tolerance of BTHHNNs.

This paper is concerned with multidirectional associative memory neural network with distributed delays on almost-periodic time scales. Some sufficient conditions on the existence, uniqueness and the global exponential stability of almost-periodic solutions are established. An example is presented to illustrate the feasibility and effectiveness of the obtained results.

Associative memory and filling-in are two essential functions of the human brain. To implement these two brain-inspired functions in hardware, we proposed a memristor-based bidirectional associative memory (BAM) circuit in this paper. This circuit combines an online algorithm with a memristor array adjustment process, thus makes the circuit more universal for various tasks. The proposed circuit is constructed out of memristive synaptic circuits, IN modules and ACT modules. The memristive synaptic circuits utilize memristor arrays to represent weight matrix and operate corresponding operations hence make computing-in-memory and process information in parallel, which simplifies the complexity of circuit and improves the processing speed. The IN modules employ transistors as switches to choose the input layer hence can get initial information flow bidirectionally. The ACT modules perform activation function and can output continuous arbitrary real numbers. Thereby, both binary and gray-scale images can be tested in the proposed circuit. In addition to the hetero-association and filling-in results given in detail, the retrieval rates of the proposed circuit with the impact of different degrees of noise and the number of stored patterns are also evaluated and compared with software-based BAM. The simulation has experimented via MATLAB and PSpice, and the corresponding results show a remarkable performance of the proposed circuit. The influence of memristor’s stuck-at-fault is also considered. In comparison with software-based BAM and similar memristor-based neural network circuit, the proposed circuit performs better in processing speed.

This paper considers a class of delayed Cohen-Grossberg-type bi-directonal associative memory neural networks with impulses. By using Mawhin continuation theorem and constructing a new Lyapunov function, some sufficient conditions are presented to guarantee the existence and stability of periodic solutions for the impulsive neural network systems. A simulation example is carried out to illustrate the efficiency of the theoretical results.

Most conventional memristor-based Pavlov associative memory neural network circuits have been working on realizing the learning and forgetting functions of simultaneous conditioning. However, the time interval between unconditional stimulus and conditional stimulus is a critical variable in classical conditioning. Different unconditional and conditional stimulus intervals evoke associative memory of brains with different rates. For example, learning in simultaneous conditioning is less effective than delay conditioning. Therefore, a memristor-based neural network circuit of delay and simultaneous conditioning is designed. The proposed circuit consists of learning states detection module, voltage control module, and synapse module. Many functions, such as short-delay conditioning learning, long-delay conditioning learning, simultaneous conditioning learning, experience learning, and two types of forgetting are implemented by the circuit. In particular, the so-called experience learning is that learning the forgotten knowledge will be faster than before and learning again will forget more slowly. This function is more bionic. The correctness of the design is demonstrated through simulations using PSPICE.

Massive green tides caused by Ulva prolifera have annually occurred in the Yellow Sea since 2007, which has attracted much attention from the government and society. There are associations between the green tides in the Yellow Sea and social response in the social media (i.e., Sina Weibo), which are bidirectional and could be captured by the bidirectional neural network. For instance, how to detect daily U. prolifera green tides by fusing remote sensing data with social media data, and how to use the observed U. prolifera green tides to infer the social response are two challenges of State Oceanic Administration, China. This article first illustrated that there are bidirectional associations between green tides and Sina Weibo data. Then, this article introduced a bidirectional spatio-temporal associative memory neural network (BSAMNN) model for modeling this bidirectional association from the spatio-temporal perspective. BSAMNN first extracted six characteristics from green tides and nine characteristics from the social responses in 2016–2019. Second, these characteristics were split by year, and the characteristics in 2016–2018 were, respectively, put into the bidirectional associative memory neural network (BAM), which is a two-layer artificial neural network. Based on the BAM results and the observed data, the residual network was constructed. Third, BSAMNN extracted the spatio-temporal rules from the characteristics in 2016–2018 as the constraints through the mining rule algorithm and modify the results via sea surface wind and ocean surface current. Last, BSAMNN put the characteristics in 2019 into BAM and used the residual network to modify the results, which was constrained by the spatio-temporal rules. The feasibility and reliability of our approach were demonstrated by using the U. prolifera green tides in 2019. The average accuracy, false alarm rate, and missing alarm rate of BSAMNN results were 0.69, 0.25, and 0.31, respectively, which was 0.11 higher, 0.10 lower, and 0.11 lower than that of the traditional BAM. The results indicated that our method is an effective alternative of linking the U. prolifera green tides and its public sentiments on social media.

Many data-driven applications, including computer vision, machine learning, speech recognition, and medical diagnostics show tolerance to computation error. These applications are often accelerated on GPUs, but the performance improvements require high energy usage. In this article, we present DRAAW, an approximate computing technique capable of accelerating GPGPU applications at a warp level. In GPUs, warps are groups of threads which issued together across multiple cores. The slowest thread dictates the pace of the warp, so DRAAW identifies these bottlenecks and avoids them during approximation. We alleviate computation costs by using an approximate lookup table which tracks recent operations and reuses them to exploit temporal locality within applications. To improve neural network performance, we propose neuron aware approximation, a technique which profiles operations within network layers and automatically configures DRAAW to ensure computations with more impact on the output accuracy are subject to less approximation. We evaluate our design by placing DRAAW within each core of an Nvidia Kepler Architecture Titan. DRAAW improves throughput by up to $2.8\times $ and improves energy-delay product (EDP) by $5.6\times $ for six GPGPU applications while maintaining less than 5% output error. We show neuron aware approximation accelerates the inference of six neutral networks by $2.9\times $ and improves EDP by $6.2\times $ with less than 1% impact on prediction accuracy.

This paper is concerned with the synchronization of bi-directional associative memory (BAM) neural networks with diffusion and time-varying delays, in which both synchronizing and desynchronizing impulses are considered. Due to the existence of the diffusion effect in artificial neural networks, synchronization control involves the dynamical behaviors of master–slave systems both in space and time. An impulsive pinning controller is designed for realizing the spatio-temporal synchronization of neural networks. By the Lyapunov functional theory and the comparison principle, exponential synchronization criteria are established, which consist of algebraic inequalities and are easy to implement. Two numerical examples show that controlling only a portion of neurons in the slave neural network can achieve the synchronization of master–slave systems.

A complex-valued Hopfield neural network (CHNN) with a multistate activation function is a multistate model of neural associative memory. The weight parameters need a lot of memory resources. Twin-multistate activation functions were introduced to quaternion- and bicomplex-valued Hopfield neural networks. Since their architectures are much more complicated than that of CHNN, the architecture should be simplified. In this work, the number of weight parameters is reduced by bicomplex projection rule for CHNNs, which is given by the decomposition of bicomplex-valued Hopfield neural networks. Computer simulations support that the noise tolerance of CHNN with a bicomplex projection rule is equal to or even better than that of quaternion- and bicomplex-valued Hopfield neural networks. By computer simulations, we find that the projection rule for hyperbolic-valued Hopfield neural networks in synchronous mode maintains a high noise tolerance.

Implementing associative memory experiments with nanoscale memristors is an interesting subject, which can allow robots to mimic human thinking. This paper is concerned with a new emotion model of memristor-based neural network and its circuit implementation. The model has three inputs and two outputs, which can feel happy when it receives good news or feel sad when it receives bad news. Unknown news can be recognized through associative memory, which simulates human emotions. Associative learning and three kinds of forgetting process together make up the full-function emotion model. In addition, the Ag/AgInSbTe/Ta-based model closely related to the actual physical properties of memristors is used to design synaptic structures. The circuits of memristor-based neural networks are also simplified. Furthermore, the new emotion model is able to adjust the changing rate of emotion. The process reflects the fact that humans learn the same thing faster at the second time, compared with the first time. Finally, PSPICE is used to simulate all the circuits of the emotion model. The presented emotion model in this paper offers more possibilities for designing intelligent machines.

Significance Development of computational models of memory is a subject of long-standing interest at the intersection of machine learning and neuroscience. Our main finding is that overparameterized neural networks trained using standard optimization methods provide a simple mechanism for implementing associative memory. Remarkably, this mechanism allows for the storage and retrieval of sequences of examples. This finding also sheds light on inductive biases in overparameterized networks: while there are many functions that can achieve zero training loss in the overparameterized regime, our result shows that increasing depth and width in neural networks leads to maps that are more contractive around training examples, thereby allowing for storage and retrieval of more training examples., Identifying computational mechanisms for memorization and retrieval of data is a long-standing problem at the intersection of machine learning and neuroscience. Our main finding is that standard overparameterized deep neural networks trained using standard optimization methods implement such a mechanism for real-valued data. We provide empirical evidence that 1) overparameterized autoencoders store training samples as attractors and thus iterating the learned map leads to sample recovery, and that 2) the same mechanism allows for encoding sequences of examples and serves as an even more efficient mechanism for memory than autoencoding. Theoretically, we prove that when trained on a single example, autoencoders store the example as an attractor. Lastly, by treating a sequence encoder as a composition of maps, we prove that sequence encoding provides a more efficient mechanism for memory than autoencoding.

Memristive neuromorphic systems are emerging potential hardware platforms to implement artificial neural networks. Combining features of memristive neuromorphic systems with associative memory, this article proposes an associative-memory-based reconfigurable memristive neuromorphic system. In the proposed system, there are two neural networks: 1) the neural network for computing acceleration and 2) the neural network mimicking associative memory. Then, a case study of the system is presented, which includes an associative memory network to realize apple recognition and a computing acceleration network for iris classification. The associative memory network depends on associative learning to achieve the recognition function. In addition, during the corresponding forgetting process, the connections of the related synaptic circuits are cut off and sent to a synaptic circuit block, realizing variable circuit topology. Further, the synaptic circuits in the block are applied to construct the iris classification network, accomplishing the reconfiguration of the proposed system. The circuit structure of this classification network matches backpropagation (BP) algorithm well. Meanwhile, the network reaches a relatively high classification accuracy after training. In an iteration of the training, all the synaptic circuits that need to change can adjust weights synchronously, which improves training speed.

An implementation of associative memory based on a Hopfield network is described. In the proposed approach, memory addresses are regarded as training vectors of the artificial neural network. The efficiency of memory search is directly associated with solving the overfitting problem. A method for dividing the training and input network vectors into parts, the processing of which requires a smaller number of neurons, is proposed. Results of a series of experiments conducted on Hopfield network models with different numbers of neurons trained with different numbers of vectors and operated under different noise conditions are presented.

In this paper, we first established the fractional order memristive multidirectional associative memory neural networks (FMMAMNNs) model, and then considered its fixed-time synchronization control problem. On the basis of sliding model control and Lyapunov stability theorem, a fractional order sliding mode controller is constructed. By adding this controller to the response system, the error of the driver-response systems gradually converges to 0 in a fixed time. Compared with the previous researches, this paper considers a more complex model, and the proposed control theories can ensure that the setting time is only related to the model and controller, but not to the initial states of the system. Besides, the control theories are also applicable to the integer order models. Finally, two numerical simulations are given, the results show the validity of the theories.

An associative memory is a special type of artificial neural network that has the purpose of store input patterns with their corresponding output patterns and efficiently recall a pattern from a noise-distorted version. Presented in this article is a new framework for constructing a binary associative memory model based on two new autoinverse operations called extended XOR/XNOR; these new operations are generated from the XOR/XNOR operations, respectively. Two types of associative memory are generated with this model: the max type (XOR-AM max), which is constructed with the maximum of the extended XOR operation, and the min type (XOR-AM min), which is constructed with the minimum of the extended XNOR operation. The XOR-AM max exhibits tolerance against the presence of patterns distorted by dilative noise, whereas the XOR-AM min exhibits tolerance against the presence of patterns distorted by erosive noise; both types of memory converge in a single step, use the same extended XOR/XNOR operator for learning and recalling phases, operate in heteroassociative and autoassociative modes, and show infinite storage capacity for the autoassociative mode. Finally, computer simulation results are presented for the new memories based on the extended XOR/XNOR (XOR-AM), which have better or equal performance compared to other associative memories. For the experiments with mixed noise, the conditions established by the kernel method proposed by Ritter for Morphological Associative Memories were conserved, and the solution algorithm proposed by Hattori for the construction of the kernel patterns of these memories was modified.

Due to the fully-connected complex structure of Artificial Neural Networks (ANNs), systems based on ANN may consume much computational time, energy and space. Therefore, intense research has been recently centered on changing the topology and design of ANNs to obtain high performance. To explore the influence of network structure on ANNs complex systems topologies have been applied in these networks to have more efficient and less complex structures while they are more similar to biological systems at the same time. In this paper, the methodology and results of some recent papers are summarized and discussed in which the authors investigated the efficacy of random complex networks on the performance of Hopfield associative memory and multi-layer ANNs compared with ANNs with small-world, scale-free and regular structures.

Olfactory memory is one of the most ancient cognitive systems in the mammal brain and has its own neural structures and mechanisms. Most behavioral models of olfactory memory in laboratory animals use several training sessions, which significantly hinders the detection and study of the neural networks of the brain involved in the acquisition of this type of memory and which undergo derangement in pathology. We have developed a mouse model of the acquisition of conditioned reflex freezing in response to the odor of amyl acetate in which learning time was 40–80 sec and the associative memory, was long-term and persisted for at least two weeks. Our model allows for precise identification of the moment of formation of olfactory memory to provide for subsequent studies of the neural networks involved in this process in health and disease.

In this paper, a full-regulated neural network (NN) with a double hidden layer recurrent neural network (DHLRNN) structure is designed, and an adaptive global sliding-mode controller based on the DHLRNN is proposed for a class of dynamic systems. Theoretical guidance and adaptive adjustment mechanism are established to set up the base width and central vector of the Gaussian function in the DHLRNN structure, where six sets of parameters can be adaptively stabilized to their best values according to different inputs. The new DHLRNN can improve the accuracy and generalization ability of the network, reduce the number of network weights, and accelerate the network training speed due to the strong fitting and presentation ability of two-layer activation functions compared with a general NN with a single hidden layer. Since the neurons of input layer can receive signals which come back from the neurons of output layer in the output feedback neural structure, it can possess associative memory and rapid system convergence, achieving better approximation and superior dynamic capability. Simulation and experiment on an active power filter are carried out to indicate the excellent static and dynamic performances of the proposed DHLRNN-based adaptive global sliding-mode controller, verifying its best approximation performance and the most stable internal state compared with other schemes.

In this paper, a self-regulated double hidden layer output feedback neural network (DHLFNN) is presented to control an active power filter (APF) system as a current controller, which is conducive to the improvement of the response characteristic and power quality. First, a global sliding mode controller is introduced because it is effective in achieving overall robustness during the system response. A new output feedback neural structure that has two hidden layers is proposed to make the parameters adaptively adjust themselves and stabilize to their best values. A higher accuracy and stronger generalization ability can be also obtained by reducing the number of network weights and accelerating the network training speed owing to the strong fitting and presentation ability of two-layer activation functions. Furthermore, the designed feedback loops of the neural network play a significant role in possessing associative memory and rapid system convergence. This proposed double hidden layer output feedback neural based global sliding mode controller is simulated on the model of APF and the results show the excellent static and dynamic properties. Experimental results under three cases and comparisons are provided using a fully digital control system to validate the superior performance of the proposed DHLFNN controller.

Chaotic neural networks is widely used in associative memory because of its abundant chaotic behavior. The bridge synaptic circuit of the memristor has been mostly used in artificial neural networks, because of its synapse-like and non-volatile properties, but the weight addition circuit has a complicated structure, the high power consumption and the high complexity of the network, so the associative memory neural network circuit is still less implemented. In this paper, the memory characteristics of the threshold memristor is used to build the synaptic circuit, on the one hand, when the continuous voltage is applied to the memristor to alter its memristance, it can realize continuous synaptic weights from - 1 to 1. Synaptic weight circuit has simple structure and low energy consumption, due to the configurability of the threshold memristor, and different weights can be obtained in the same circuits to achieve the function of associative memory. On the other hand, we can realize self-associative memory, hetero-associative memory, the separation of superimposed patterns, many-to-many associative memory and application in the three-view drawing, through simulation experiments. Because of the nanoscale characteristics of memristor, the hardware implementation of large-scale chaotic neural network will has simplified structure and be integrated easily.

In this paper, an image encryption algorithm is proposed for the first time, which uses coding deoxyribonucleic acid (DNA) sequences to transcript its complementary messenger ribonucleic acid (mRNA) sequence. A binary associative memory neural network (BAM NN) is trained to produce the required transfer ribonucleic acid (tRNA) sequences to bond with the mRNA sequences to assign the correct amino acids. DNA and mRNA sequences, together with tRNA sequences, provide the permutation and diffusion operations for the pixels in colour digital imaging and communications in medicine images. Then, keys, which are determined by the generated amino acids, are used to encrypt the diffused cipher pixels further. Also key sharing has been carried out using Universal Software Radio Peripheral (USRP), cloud infrastructure and One Time Password (OTP) generation mechanisms.The different sequence formation can also identify any alterations made by the intruder in the amino acid, which results in the wrong set of key generation. Various decryption quality analyses, statistical and differential attack analyses (namely mean square error, unified average changing intensity, peak signal-to-noise ratio, number of pixel changing rate, entropy, mean absolute error, key sensitivity, cropping attacks, and chosen-plaintext attacks), and correlation tests were carried out to confirm the robustness of the proposed algorithm.

The passivity, low power consumption, memory characteristics and nanometer size of memristors make them the best choice to simulate synapses in artificial neural networks. In this paper, based on the proposed associative memory rules, we design a memristor neural network with plasticity synapses, which can perform analog operations similar to its biological behavior. For the memristor neural network circuit, we also construct a relatively simple Pavlov’s dog experiment simulation circuit, which can effectively reduce the complexity and power consumption of the network. Some advanced neural activities including learning, associative memory and three kinds of forgetting are realized based on the spiking-rate-dependent plasticity rule. Finally, the Simulation program with integrated circuit emphasis is used to simulate the circuit. The simulation results not only prove the correctness of the design, but also help to realize more efficient, simpler and more complex analog circuit of memristor neural network and then help to realize more intelligent, smaller and low-power brain chips.

Denial of Service (DoS) attack is a serious threat to Software Defined Network (SDN). Although many research efforts have been devoted to identify new features for DoS attack detection, the existing approaches are not able to detect various types of DoS attacks. In SDN, DoS attacks against data plane are mainly organized in two ways: 1) DoS attack with multiple flow entries (M-DoS) to exhaust the Ternary Content-Addressable Memory (TCAM) resource of the switch. 2) DoS attack with a single well-designed entry (S-DoS) to overwhelm the target link and further impact the controller. To detect these two attacks, we propose a new approach by extracting six features of flow table, and using the back propagation (BP) neural network to construct the classifier. Test results of test-bed experiments indicate that the accurate detection probability of proposed approach is 98.9%, which can effectively distinguish M-DoS flows and S-DoS flows without being affected by Flash crowd scene.

The development of hardware neural networks, including neuromorphic hardware, has been accelerated over the past few years. However, it is challenging to operate very large-scale neural networks with low-power hardware devices, partly due to signal transmissions through a massive number of interconnections. Our aim is to deal with the issue of communication cost from an algorithmic viewpoint and study learning algorithms for energy-efficient information processing. Here, we consider two approaches to finding spatially arranged sparse recurrent neural networks with the high cost-performance ratio for associative memory. In the first approach following classical methods, we focus on sparse modular network structures inspired by biological brain networks and examine their storage capacity under an iterative learning rule. We show that incorporating long-range intermodule connections into purely modular networks can enhance the cost-performance ratio. In the second approach, we formulate for the first time an optimization problem where the network sparsity is maximized under the constraints imposed by a pattern embedding condition. We show that there is a tradeoff between the interconnection cost and the computational performance in the optimized networks. We demonstrate that the optimized networks can achieve a better cost-performance ratio compared with those considered in the first approach. We show the effectiveness of the optimization approach mainly using binary patterns and apply it also to gray-scale image restoration. Our results suggest that the presented approaches are useful in seeking more sparse and less costly connectivity of neural networks for the enhancement of energy efficiency in hardware neural networks.

A hybrid analog/digital very large-scale integration (VLSI) implementation of a spiking neural network with programmable synaptic weights was designed. The synaptic weight values are stored in an asynchronous module, which is interfaced to a fast current-mode event-driven DAC for producing synaptic currents with the appropriate amplitude values. It acts as a transceiver, receiving asynchronous events for input, performing neural computations with hybrid analog/digital circuits on the input spikes, and eventually producing digital asynchronous events in output. Input, output, and synaptic weight values are transmitted to/from the chip using a common communication protocol based on the address event representation (AER). Using this representation, it is possible to interface the device to a workstation or a microcontroller and explore the effect of different types of spike-timing dependent plasticity (STDP) learning algorithms for updating the synaptic weights values in the CAM module.

With the rapid development of social economy, urban rail transit has become the mainstream of people's travel mode. For the reasonable dispatch of vehicles, rail transit operation departments have an increasing demand for short-term passenger flow forecasting. Through the analysis of the passenger flow data of Chongqing Metro from October 1, 2018, to October 15, 2018, the GRU neural network is used to analyze the Chongqing Metro stations from October 16, 2018, to October 30, 2018, to predict the arrival passenger flow data, and use the logit model-based transfer method selection combination model to obtain the transfer sharing rate, and finally obtain the predicted passenger flow of each transfer method. The comparison with other models and optimization algorithms shows that this model has better prediction accuracy, stability, and robustness. GRU neural network has associative memory function, simple structure, small amount of calculation, and anti-noise ability, so it is worth popularizing in subway passenger flow prediction applications.

Should the internal structure of a system matter when it comes to autonomy? While there is still no consensus on a rigorous, quantifiable definition of autonomy, multiple candidate measures and related quantities have been proposed across various disciplines, including graph-theory, information-theory, and complex system science. Here, I review and compare a range of measures related to autonomy and intelligent behavior. To that end, I analyzed the structural, information-theoretical, causal, and dynamical properties of simple artificial agents evolved to solve a spatial navigation task, with or without a need for associative memory. By contrast to standard artificial neural networks with fixed architectures and node functions, here, independent evolution simulations produced successful agents with diverse neural architectures and functions. This makes it possible to distinguish quantities that characterize task demands and input-output behavior, from those that capture intrinsic differences between substrates, which may help to determine more stringent requisites for autonomous behavior and the means to measure it.

International audience; Distributed smart sensors are more and more used in applications such as biomedical or domestic monitoring. However, each sensor broadcasts data wirelessly to the others or to an aggregator, which leads to energy-hungry sensor nodes and an increased latency at the network level. To tackle both challenges, this work proposes to distribute part of the processing elements in each sensor node and presents a 28nm FDSOI ASIC implementation of an associative memory using clique-based neural networks (CBNNs) coupled with an integrated SRAM memory. It consumes 43pJ for a single inference, which is 6.5 times better than state-of-the-art associative memories implementations, for the same memory size.

Synchronization of dual servo motor control has become an important issue due to many applications in engineering fields, such as electric vehicle, robotics, electronics production machines, and others. This paper studies cerebellar model articulation controller (CMAC) neural network (NN) controller to synchronize two servo motors with dynamic LuGre friction model. CMAC is kind of NN method represented by associative memory with more powerful properties. Cross-coupling control structure is employed to synchronize two servo motors in this study. To investigates the performance, MATLAB Simulink is applied to simulate the control design of dual servo motor. The simulation results exhibit that CMAC controller has better output trajectory and works well for two servo motors with different parameters of LuGre friction model.

The concept of spherical seperabilty was innovated by authors. In this research paper, based on the concept of spherical seperability, novel associative memories are proposed. The dynamics of the associative memories is shown to lead to a stable state or a cycle of length atmost 2, starting in an initial condition.

Owing to the quick spread of minute health-related experiences, the distillation of knowledge from such unstructured narratives is an extremely challenging task. In spite of the success of neural networks methods in improving learning structural reliability, they result in inadequate accuracy of the bio-medical sentiment classification when employing less useful features sets. Therefore, they lack discriminatory potential. In this study, we propose to add a deep associative memory into neural networks for an effective sentiment decomposition, which emphasizes correctly on bio-medical entities related to the extraction of different data-object properties, and contextual-semantics dependencies for a given aspect. The underlying trust of these measures is behind the ability to compute the completion of unseen medical patterns, where comprehensive bio-medical distributed representations are used for representing the formal medical connections from PubMed databases. Experiments on a biomedical sentiment analysis task show that the model provides comprehensive embeddings with meaningful medical patterns. It achieved an average performance of 87% on varied large online datasets. It also outperforms baselines in discovering and identifying medical natural concepts. We provide meaningful support to bio-medical sentiment analysis applications in social networks. Indeed, the facets of this study might be used in many health concerns such as analyzing change in health status or unexpected situations.

Memristor is a computational and area efficient substitute for resistive synapse in neural networks as it provides tunable and non-volatile storage of synaptic weights. Full CMOS circuit realisation of a 6 × 6 Bidirectional Associative Memory (BAM) neural network with CMOS memristor synapse is implemented in this paper. To show the BAM neural network’s ability to recall its pattern, we have taken the training set of three Tetris pattern pairs, and corresponding synapse weights are calculated using MATLAB. We have integrated the proposed tunable CMOS memristor emulator in the crossbar for storing the synaptic weights. The CMOS circuit implementation of the BAM neural network is validated with the simulation results in 0.18µm CMOS technology.

A perturbation fuzzy neural network (PFNN) fractional-order sliding mode control (FSMC)strategy is developed for a micro gyroscope system. The PFNN is used to identify the unknown system dynamics. A sine-cosine perturbation type membership function is employed to decrease the neural rule's uncertainty. The proposed controller includes a PFNN estimator and a FSMC, where the PFNN has two feedback loop, transmitting more complex information with the function of associative memory. Simulation results verified the validity of the proposed control method.