Search

We discuss possible interpretations of “(in)efficient” and “(un)predictable” systems when those concepts are applied to discrete networks and their dynamics. We are interested in computational notions of the network dynamics’ predictability and efficiency. Deterministic discrete network’s dynamics is efficient if it settles fairly quickly into an appropriate stationary pattern; that is, efficiency is essentially synonymous with a short transient chain until a stationary system state is reached. An inefficient dynamical system, then, is one that takes a long time to converge to a stationary behavior. The issue of (in)efficiency is related to, but different from, that of the network dynamics’ (un)predictability. We call dynamics predictable, if the “ultimate destiny” (i.e., convergence to a stationary behavior) can be efficiently determined or predicted, without performing a step-by-step simulation of the network's dynamics—unless such simulation itself can be done efficiently. For discrete systems with finite configuration spaces, all properties of their dynamics are decidable; however, for non-trivial network sizes, various properties may or may not be possible to predict within reasonable computational resources. In that context, we discuss how computational complexity of reachability problems and enumeration problems about several classes of cellular and network automata relate to whether those networks’ dynamics are predictable. We then discuss some examples of cellular automata and other Boolean networks whose dynamics are (i) both predictable and efficient, (ii) efficient but unpredictable, (iii) predictable yet inefficient, and finally (iv) both unpredictable and inefficient. In particular, we briefly overview several classes of Boolean networks that fall into the categories (i), (ii) and (iv).

The machine learning model influences the pattern recognition based on extraction of relative patterns, but not enough capable of efficiently processing the data set that needs layered interaction. So, deep learning model takes the advantage of artificial neural network for processing the data in layered abstraction by exploring massive parallelism, although the classical implementations of such model may not be competent due to the processing and storage of large neural networks. Current research explores quantum computation potentials and utilizes its significance for supporting inherent parallelism, as the machine perform exponential operations in single step of execution. The possible classical designs of pattern recognition using deep learning are Hopfield network and Boltzmann machine and their equivalent quantum models can remain effective and overcome the processing limitations of classical model by incorporating Grover’s search method and quantum Hebbian learning. The aim of writing this article is to introduce the necessity of deep learning model, an emergence of quantum computations, discussion over requisites of data preprocessing techniques, then to propose the classical equivalent quantum deep learning model along with the algorithms, complexity comparison and speedup analysis followed by conclusive aspects that proves effectiveness of quantum deep learning model and future works.

An approach for automatic formation of the structure of the Hopfield neural network model and its training (parameter settings) for solving systems of linear algebraic equations (SLAE) of arbitrary order is proposed. Adapted for solving of SLAE model of network is configured automatically in the environment for modeling Simulink, interacting with the Matlab computing system, which allows the user to vary the input data (order, constants vector and coefficients matrix system of linear algebraic equations). The results of research on the quality of the finding for solutions to SLAEs are presented.

This chapter focuses on the mutually connected neural networks. Section 4.1 describes the Hopfield network, and Sect. 4.2 the Boltzmann machine.

The world is in a frustrating situation, which is exacerbating due to the time-consuming process of the COVID-19 vaccine design and production. This chapter provides a comprehensive investigation of fundamentals, state-of-the-art and some perspectives to speed up the process of the design, optimization and production of the medicine for COVID-19 based on Deep Learning (DL) methods. The proposed platforms are able to be used as predictors to forecast antigens during the infection disregarding their abundance and immunogenicity with no requirement of growing the pathogen in vitro. First, we briefly survey the latest achievements and fundamentals of some DL methodologies, including Deep Boltzmann Machines (DBM), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Hopfield network and Long Short-Term Memory(LSTM). These techniques help us to reach an integrated approach for drug development by non-conventional antigens. We then propose several DL-based platforms to utilize for future applications regarding the latest publications and medical reports. Considering the evolving date on COVID-19 and its ever-changing nature, we believe this survey can give readers some useful ideas and directions to understand the application of Artificial Intelligence (AI) to accelerate the vaccine design not only for COVID-19 but also for many different diseases or viruses.

In this paper, we examine Hopfield network composed of multi-state neurons for storing sequence data as limit cycles of the network. Earlier, we had presented uni-modal data - particularly text, speech and audio data storage and retrieval in bipolar Hopfield based associative memory architecture. We extended this to multi-modal data and we demonstrated that Hopfield can indeed work as content addressable memory for multi-modal data. This paper is a step towards realising a more wider definition of multi-modality. We present a M-ary Hopfield associative memory model for storing limit cycle data. The proposed system uses a dual weight learning mechanism to exhibit limit cycle behavior in which sequence data can be stored and retrieved. We particularly deal with a) sequence of images and b) movie clip data as instances of limit cycle data. We also propose and use a two stage firing mechanism to retrieve the stored sequence data from the limit cycles. We present a trade-off behavior between the number of cycles and length of cycles the network can store and we demonstrate that the network capacity is still of the order of network size i.e., O(N) for limit cycle data. This represents a first of its kind attempt for sequence storage and retrieval in Hopfield network as limit-cycles, particularly with image-sequence and movie-content data of real-world scales.

NP Complete problems belong to a computational class of problems that have no known polynomial time solutions. Many popular and practically useful problems in the field of optimization and graph theory which have real life application are known to be NP Complete and solving them exactly is intractable. The existing problem with these is that there is no known efficient way to locate a solution in the first place, the most notable characteristic of NP-complete problems is that no fast solution to them is known. However approximate solutions can be obtained in polynomial time. Hopfield networks are one of the ways to obtain approximate solution to the problems in polynomial time. Exploiting the reducibility property and the capability of Hopfield Networks to provide approximate solutions in polynomial time we propose a Hopfield Network based approximation engine to solve these NP complete problems.

Personalized medicine aims at identifying specific targets for treatment considering the gene expression profile of each patient individually. We discuss the challenges for personalized oncology to take off and present an approach based on hub inhibition that we are developing. That is, the subtraction of RNA-seq data of tumoral and non-tumoral surrounding tissues in biopsies allows the identification of up-regulated genes in tumors of patients. Targeting connection hubs in the subnetworks formed by the interactions between the proteins of up-regulated genes is a suitable strategy for the inhibition of tumor growth and metastasis in vitro. The most relevant protein targets may be further analyzed for drug repurposing by computational biology. The subnetworks formed by the interactions between the proteins of up-regulated genes allow the inference by Shannon entropy of the number of targets to be inhibited according to the tumor aggressiveness. There are common targets between tumoral tissues but many others are personalized at a molecular level. We also consider additional measures and more sophisticated modeling. This approach is necessary to improve the rational choice of therapeutic targets and the description of network dynamics. The modeling of attractors through Hopfield Network and ordinary differential equations are given here as examples.

We consider the problem of finding representative prototypes within a set of data and solve it using Hopfield networks. Our key idea is to minimize the mean discrepancy between kernel density estimates of the distributions of data points and prototypes. We show that this objective can be cast as a quadratic unconstrained binary optimization problem which is equivalent to a Hopfield energy minimization problem. This result is of current interest as it suggests that vector quantization can be accomplished via adiabatic quantum computing.

The paper describes an application of interval methods to train recurrent neural networks and investigate their behavior. The HIBA_USNE multithreaded interval solver for nonlinear systems and algorithmic differentiation using ADHC are used. Using interval methods, we can not only train the network, but precisely localize all stationary points of the network. Preliminary numerical results for continuous Hopfield-like networks are presented.

In this chapter, four different types of neural networks are described: Radial Basis Functions-RBF, Self-Organizing Maps-SOM, the Hopfield, and the deep neural networks. RBF uses a different approach in the design of a neural network based on the hidden layer (unique in the network) composed of neurons in which radial basis functions are defined, hence the name of Radial Basis Functions, and which performs a nonlinear transformation of the input data supplied to the network. The SOM network, on the other hand, has an unsupervised learning model and has the originality of autonomously grouping input data on the basis of their similarity without evaluating the convergence error with external information on the data, but evaluating the quantization error on map network. With the Hopfield network, the learning model is supervised and with the ability to store information and retrieve it through even partial content of the original information. The network is associated with an energy function to be minimized during its evolution with a succession of states until reaching a final state corresponding to the minimum of the energy function. This feature allows it to be used to solve and set up an optimization problem in terms of the objective function to be associated with an energy function. The chapter concludes with the description of the foundation of the newly fashionable methods based on convolutional neural networks (CNN), by now the most widespread since 2012, based on the deep learning architecture (deep learning).

Multitask algorithms typically use task similarity information as a bias to speed up and improve the performance of learning processes. Tasks are learned jointly, sharing information across them, in order to construct models more accurate than those learned separately over single tasks. In this contribution, we present the first multitask model, to our knowledge, based on Hopfield Networks (HNs), named HoMTask. We show that by appropriately building a unique HN embedding all tasks, a more robust and effective classification model can be learned. HoMTask is a transductive semi-supervised parametric HN, that minimizes an energy function extended to all nodes and to all tasks under study. We provide theoretical evidence that the optimal parameters automatically estimated by HoMTask make coherent the model itself with the prior knowledge (connection weights and node labels). The convergence properties of HNs are preserved, and the fixed point reached by the network dynamics gives rise to the prediction of unlabeled nodes. The proposed model improves the classification abilities of singletask HNs on a preliminary benchmark comparison, and achieves competitive performance with state-of-the-art semi-supervised graph-based algorithms.

This chapter first introduces the continuous-time Hopfield neural networks. The existence and uniqueness of equilibrium, as well as its stability and instability, of continuous-time Hopfield networks are analyzed, and less conservative yet more general results are established. Then, in light of the continuous-time architecture of Hopfield networks, the impulsive Hopfield neural networks with transmission delays are formulated and explained. Many evolutionary processes, particularly biological systems, that exhibit impulsive dynamical behaviors, can be described by the impulsive Hopfield neural networks. Fundamental issues such as the global exponential stability, the existence and uniqueness of the equilibrium of such impulsive Hopfield networks are established. A numerical example is given for illustration and interpretation of the theoretical results.

We consider the problem of writing on a Hopfield network. We cast the problem as a supervised learning problem by observing a simple link between the update equations of Hopfield network and recurrent neural networks. We compare the new writing protocol to existing ones and experimentally verify its effectiveness. Our method not only has a better ability of noise recovery, but also has a bigger capacity compared to the other existing writing protocols.

It is commonly known that Hopfield Networks suffer from spurious states and from low storage capacity. To eliminate the spurious states Bistable Gradient Networks (BGN) introduce neurons with bistable behavior. The weights in BGN are calculated in analogy to those of Hopfield Networks, associated with Hebbian learning. Unfortunately, those networks still suffer from small storage capacity, resulting in high reconstruction errors when used to reconstruct noisy patterns. This paper proposes a new type of neural network consisting of neurons with a sigmoid hyperbolic tangent transfer function and a direct feedback. The feedback renders the neuron bistable. Furthermore, instead of using Hebbian learning which has some drawbacks when applied to overlapped patterns, we use the first order Contrastive Divergence (CD1) learning rule. We call these Networks Bistable Sigmoid Networks (BSN). When recalling patterns from the MNIST database the reconstruction error is zero even for high load providing no noise is applied. For an increasing noise level or an increasing amount of patterns the error rises only moderate.

This work presents an Ensemble of Attractor Neural Networks (EANN) model for gesture retrieval. 2D single-stroke gestures were captured and tested offline by the ensemble. The ensemble was compared to a single attractor with the same complexity, i.e. with equal connectivity. We show that the ensemble of neural networks improves the gesture retrieval in terms of capacity and quality of the gestures retrieval, regarding the single network. The ensemble was able to improve the retrieval of correlated patterns with a random assignment of pattern subsets to the ensemble modules. Thus, optimizing the ensemble input is a possibility for maximizing the patterns retrieval. The proposed EANN proved to be robust for gesture recognition with large initial noise promising to be robust for gesture invariants.

Recently we presented text storage and retrieval in an auto-associative memory framework using the Hopfield neural-network. This realized the ideal functionality of Hopfield network as a content-addressable information retrieval system. In this paper, we extend this result to multi-modal patterns, namely, images with text captions and show that the Hopfield network indeed can store and retrieve such multi-modal patterns even in an auto-associative setting. Within this framework, we examine two central issues such as (i) performance characterization to show that the O(N) capacity of the Hopfield network for a network of size N neurons under the Pseudo-inverse learning rule is still retained in the multi-modal case, and (ii) the retrieval dynamics of the multi-modal pattern (i.e., image and caption together) under various types of queries such as image\(+\)caption, image only and caption only, in line with a typical multi-modal retrieval system where the entire multi-modal pattern is expected to be retrieved even with a partial query pattern from any of the modalities. We present results related to these two issues on a large database of 7000\(+\) captioned-images and establish the practical scalability of both the storage capacity and the retrieval robustness of the Hopfield network for content-addressable retrieval of multi-modal patterns. We point to the potential of this work to extend to a more wider definition of multi-modality as in multi-media content, with various modalities such as video (image sequence) synchronized with sub-title text, speech, music and non-speech.

We tested the performance of the Ensemble of Attractor Neural Networks (EANN) model for fingerprint learning and retrieval. The EANN model has proved to increase the random patterns storage capacity, when compared to a single attractor of equal connectivity. In this work, we tested the EANN with real patterns, i.e. fingerprints dataset. The EANN improved the retrieval performance for real patterns more than tripling the capacity of the single attractor with the same number of connections. The EANN modules can also be specialized for different patterns sets according to their characteristics, i.e. pattern/network sparseness (activity). Three EANN modules were assigned with skeletonized fingerprints (low activity), binarized (original) fingerprints (medium activity), and dilated/thickened fingerprint (high activity), and their retrieval was checked. The more sparse the code the larger the storage capacity of the module. The EANN demonstrated to improve the retrieval capacity of the single network, and it can be very helpful for module specialization for different types of real patterns.

In this paper, we study the d-dimensional knapsack problem (d-KP). The problem d-KP is a generalized version of the well-known knapsack problem (1-KP) which is known to be an NP-complete problem. It is also known that there is no fully polynomial-time approximation scheme for d-KP for \(d >1\) unless \(P=NP\). In this work, we design an approximation algorithm for d-KP based on the Hopfield networks. Experimental results show that our proposed algorithm outperforms a well-known greedy algorithm in many cases.

The use of binary and multilevel memristors in the hardware neural networks implementation necessitates their weight coefficients quantization. In this paper we investigate the Hopfield network weights quantization influence on its information capacity and resistance to input data distortions. It is shown that, for a weight level number of the order of tens, the quantized weights Hopfield-Hebb network capacitance approximates its continuous weights version capacity. For a Hopfield projection network, similar result can be achieved only for a weight levels number of the order of hundreds. Experiments have shown that: (1) binary memristors should be used in Hopfield-Hebb networks, reduced by zeroing all weights in a given row which moduli are strictly less than the maximum weight in the row; (2) in the Hopfield projection networks with quantized weights, multilevel memristors with a weight levels number significantly more than two should be used, with a specific levels number depending on the stored reference vectors dimension, their particular set and the permissible input data noise level.

Energy-based models are a specific class of neural networks. The simplest energy model is the Hopfield Network dating back from the 1980s (Hopfield Proc Nat Acad Sci USA 79(8):2554–2558, 1982, [1]). Hopfield networks are often thought to be very simple, but they are quite different from what we have seen before.

In recent years, the continuous Hopfield network has become the most required tool to solve quadratic problems (QP). But, it suffers from some drawbacks, such as, the initial states. This later affect the convergence to the optimal solution and if a bad starting point is arbitrarily specified, the infeasible solution is generated. In this paper, we examine this issue and try to provide a new technique to choose a good starting point in order to give a good optimal solution for any quadratic problems (QP). Numerical simulations are provided to demonstrate the performance of this new technique applied to task assignment problems.

The concept of modeling a machine which can adapt to the dynamic changes in environment has fascinated the field of Artificial Intelligence. Machine Learning has made inroads in every possible domain. New techniques are developed which can mimic human like responses and thoughts. Cognitive computing has developed renewed interest in the community with advent of Artificial Neural Nets (ANN). In this paper, we present a biological inspired approach to building a augmented memory recall model which can learn usage access patterns and reconstruct from them when presented with noisy or broken concepts. We use Hopfield Networks in a distributed parallel architecture like Hadoop. We also present a mechanism for augmenting the memory capacity of Hopfield Nets. Our model is tested on a real world dataset by parallelizing the learning process thereby increasing the computing power to recognize patterns.

We describe neural network model of unconscious processing as it defined by the symmetrical logic of Matte Blanco. The model system consists of hierarchy of ensembles of Hopfield network representing definite classes of objects. Patterns in each of a network are considered as in some sense identical representatives of given class. These networks generate their self-reproducible descendants which can exchange patterns with each other and generate self-reproducible networks of a higher level representing wider classes of objects. We also give some examples of applications of this model.

A new approach to the construction of multilayer neural network approximate solutions for evolutionary partial differential equations is considered. The approach is based on the application of the recurrence relations of the Euler, Runge-Kutta, etc. methods to variable length intervals. The resulting neural-like structure can be considered as a generalization of a feedforward multilayer network or a recurrent Hopfield network. This analogy makes it possible to apply known methods to the refinement of the obtained solution, for example, the backpropagation algorithm. Earlier, a similar approach has been successfully used by the authors in the case of ordinary differential equations. Computational experiments are performed on one test problem for the one-dimensional (in terms of spatial variables) heat equation. Explicit formulas are derived for the dependence of the resulting neural network output on the number of layers. It was found that the error tends to zero with an increasing number of layers, even without the use of the network learning.

The paper describes the architecture of the information expert system designed and dedicated to stock exchange data processing, analyzing and presentation. The system uses Artificial Neural Networks (ANN) combined with technical analysis and fractal analysis to predict future price of stock exchange assets. It also enables selection of companies, which assets will be increased. The selection of companies is based on comparing algorithms implementing Hopfield network. The system displays a list of the companies with the highest expected profit sorted in descending order.