Search

The three-state Ising neural network with synchronous updating and variable dilution is discussed starting from the appropriate Hamiltonians. The thermodynamic and retrieval properties are examined using replica mean-field theory. Capacity-temperature phase diagrams are derived for several values of the pattern activity and different gradations of dilution, and the information content is calculated. The results are compared with those for sequential updating. The effect of self-coupling is established. Also the dynamics is studied using the generating function technique for both synchronous and sequential updating. Typical flow diagrams for the overlap order parameter are presented. The differences with the signal-to-noise approach are outlined., Comment: 21 pages Latex, 12 eps figures and 1 ps figure

The time evolution of an exactly solvable layered feedforward neural network with three-state neurons and optimizing the mutual information is studied for arbitrary synaptic noise (temperature). Detailed stationary temperature-capacity and capacity-activity phase diagrams are obtained. The model exhibits pattern retrieval, pattern-fluctuation retrieval and spin-glass phases. It is found that there is an improved performance in the form of both a larger critical capacity and information content compared with three-state Ising-type layered network models. Flow diagrams reveal that saddle-point solutions associated with fluctuation overlaps slow down considerably the flow of the network states towards the stable fixed-points., Comment: 17 pages Latex including 6 eps-figures

We introduce a spherical Hopfield-type neural network involving neurons and patterns that are continuous variables. We study both the thermodynamics and dynamics of this model. In order to have a retrieval phase a quartic term is added to the Hamiltonian. The thermodynamics of the model is exactly solvable and the results are replica symmetric. A Langevin dynamics leads to a closed set of equations for the order parameters and effective correlation and response function typical for neural networks. The stationary limit corresponds to the thermodynamic results. Numerical calculations illustrate our findings., 9 pages Latex including 3 eps figures, Addition of an author in the HTML-abstract unintentionally forgotten, no changes to the manuscript

We study supervised learning and generalisation in coupled perceptrons trained on-line using two learning scenarios. In the first scenario the teacher and the student are independent networks and both are represented by an Ashkin-Teller perceptron. In the second scenario the student and the teacher are simple perceptrons but are coupled by an Ashkin-Teller type four-neuron interaction term. Expressions for the generalisation error and the learning curves are derived for various learning algorithms. The analytic results find excellent confirmation in numerical simulations., Latex, 21 pages, 9 figures, iop style files included

A complete self-control mechanism is proposed in the dynamics of neural networks through the introduction of a time-dependent threshold, determined in function of both the noise and the pattern activity in the network. Especially for sparsely coded models this mechanism is shown to considerably improve the storage capacity, the basins of attraction and the mutual information content of the network., Comment: 4 pages, 6 Postscript figures

Using a signal-to-noise analysis, the effects of nonlinear modulation of the Hebbian learning rule in the multi-class proximity problem are investigated. Both random classification and classification provided by a Gaussian and a binary teacher are treated. Analytic expressions are derived for the learning and generalization rates around an old and a new prototype. For the proximity problem with binary inputs but Q′-state outputs, it is shown that the optimal modulation is a combination of a hyperbolic tangent and a linear function. As an illustration, numerical results are presented for the two-class and the Q′=3 multi-class problem.

We derive exact equations that determine the spectra of undirected and directed sparsely connected regular graphs containing loops of arbitrary length. The implications of our results to the structural and dynamical properties of networks are discussed by showing how loops influence the size of the spectral gap and the propensity for synchronization. Analytical formulas for the spectrum are obtained for specific length of the loops., Comment: 4 pages, 4 figures

Optimal capacities of perceptrons with graded input-output relations are computed within the Gardner approach (1988). The influence of desired output precision, stability with respect to input errors, and output-pattern statistics are analysed and discussed.

We study the L\'evy spin-glass model with the replica and the cavity method. In this model each spin interacts through a finite number of strong bonds and an infinite number of weak bonds. This hybrid behaviour of L\'evy spin glasses becomes transparent in our solution: the local field contains a part propagating along a backbone of strong bonds and a Gaussian noise term due to weak bonds. Our method allows to determine the complete replica symmetric phase diagram, the replica symmetry breaking line and the entropy. The results are compared with simulations and previous calculations using a Gaussian ansatz for the distribution of fields., Comment: 20 pages, 7 figures

We use the cavity method to study parallel dynamics of disordered Ising models on a graph. In particular, we derive a set of recursive equations in single site probabilities of paths propagating along the edges of the graph. These equations are analogous to the cavity equations for equilibrium models and are exact on a tree. On graphs with exclusively directed edges we find an exact expression for the stationary distribution of the spins. We present the phase diagrams for an Ising model on an asymmetric Bethe lattice and for a neural network with Hebbian interactions on an asymmetric scale-free graph. For graphs with a nonzero fraction of symmetric edges the equations can be solved for a finite number of time steps. Theoretical predictions are confirmed by simulation results. Using a heuristic method, the cavity equations are extended to a set of equations that determine the marginals of the stationary distribution of Ising models on graphs with a nonzero fraction of symmetric edges. The results of this method are discussed and compared with simulations.

We derive critical noise levels for Gallager codes on asymmetric channels as a function of the input bias and the temperature. Using a statistical mechanics approach we study the space of codewords and the entropy in the various decoding regimes. We further discuss the relation of the convergence of the message passing algorithm with the endogeny property and complexity, characterizing solutions of recursive equations of distributions for cavity fields.

We require total invariance of a lagrangian under supersymmetry transformations and we observe that special variables are singled out. They are identical to those entering the Nicolai mapping. We show that a similarity transformation is connected with the introduction of these new variables. We give a stochastic formulation of this transformation using the Cameron-Martin formula.

We study the thermodynamic properties of spin systems with bond-disorder on small-world hypergraphs, obtained by superimposing a one-dimensional Ising chain onto a random Bethe graph with p-spin interactions. Using transfer-matrix techniques, we derive fixed-point equations describing the relevant order parameters and the free energy, both in the replica symmetric and one step replica symmetry breaking approximation. We determine the static and dynamic ferromagnetic transition and the spinglass transition within replica symmetry for all temperatures, and demonstrate corrections to these results when one step replica symmetry breaking is taken into account. The results obtained are in agreement with Monte-Carlo simulations., 9 pages, 4 figures

The inclusion of a macroscopic adaptive threshold is studied for the retrieval dynamics of both layered feedforward and fully connected neural network models with synaptic noise. These two types of architectures require a different method to be solved numerically. In both cases it is shown that, if the threshold is chosen appropriately as a function of the cross-talk noise and of the activity of the stored patterns, adapting itself automatically in the course of the recall process, an autonomous functioning of the network is guaranteed. This self-control mechanism considerably improves the quality of retrieval, in particular the storage capacity, the basins of attraction and the mutual information content., Comment: 12 pages, 10 figures

We study the thermodynamic properties of spin systems on small-world hypergraphs, obtained by superimposing sparse Poisson random graphs with p-spin interactions onto a one-dimensional Ising chain with nearest-neighbor interactions. We use replica-symmetric transfer-matrix techniques to derive a set of fixed-point equations describing the relevant order parameters and free energy, and solve them employing population dynamics. In the special case where the number of connections per site is of the order of the system size we are able to solve the model analytically. In the more general case where the number of connections is finite we determine the static and dynamic ferromagnetic-paramagnetic transitions using population dynamics. The results are tested against Monte-Carlo simulations., 14 pages, 7 figures; Added 2 figures. Extended results

The inclusion of a macroscopic adaptive threshold is studied for the retrieval dynamics of layered feedforward neural network models with synaptic noise. It is shown that if the threshold is chosen appropriately as a function of the cross-talk noise and of the activity of the stored patterns, adapting itself automatically in the course of the recall process, an autonomous functioning of the network is guaranteed. This self-control mechanism considerably improves the quality of retrieval, in particular the storage capacity, the basins of attraction and the mutual information content.

The optimal capacity of a diluted Blume-Emery-Griffiths neural network is studied as a function of the pattern activity and the embedding stability using the Gardner entropy approach. Annealed dilution is considered, cutting some of the couplings referring to the ternary patterns themselves and some of the couplings related to the active patterns, both simultaneously (synchronous dilution) or independently (asynchronous dilution). Through the de Almeida-Thouless criterion it is found that the replica-symmetric solution is locally unstable as soon as there is dilution. The distribution of the couplings shows the typical gap with a width depending on the amount of dilution, but this gap persists even in cases where a particular type of coupling plays no role in the learning process., 9 pages Latex, 2 eps figures

For the retrieval dynamics of sparsely coded attractor associative memory models with synaptic noise the inclusion of a macroscopic time-dependent threshold is studied. It is shown that if the threshold is chosen appropriately as a function of the cross-talk noise and of the activity of the memorized patterns, adapting itself automatically in the course of the time evolution, an autonomous functioning of the model is guaranteed. This self-control mechanism considerably improves the quality of the fixed-point retrieval dynamics, in particular the storage capacity, the basins of attraction and the mutual information content., Comment: 5 pages Latex, 1 ps and 4 eps figures, to appear in the proceedings of the 2004 International Joint Conference on Neural Networks, Budapest (IEEE)

The effects of a variable amount of random dilution of the synaptic couplings in Q-Ising multi-state neural networks with Hebbian learning are examined. A fraction of the couplings is explicitly allowed to be anti-Hebbian. Random dilution represents the dying or pruning of synapses and, hence, a static disruption of the learning process which can be considered as a form of multiplicative noise in the learning rule. Both parallel and sequential updating of the neurons can be treated. Symmetric dilution in the statics of the network is studied using the mean-field theory approach of statistical mechanics. General dilution, including asymmetric pruning of the couplings, is examined using the generating functional (path integral) approach of disordered systems. It is shown that random dilution acts as additive gaussian noise in the Hebbian learning rule with a mean zero and a variance depending on the connectivity of the network and on the symmetry. Furthermore, a scaling factor appears that essentially measures the average amount of anti-Hebbian couplings., Comment: 15 pages, 5 figures, to appear in the proceedings of the Conference on Noise in Complex Systems and Stochastic Dynamics II (SPIE International)

Using the generating functional analysis an exact recursion relation is derived for the time evolution of the effective local field of the fully connected Little-Hopfield model. It is shown that, by leaving out the feedback correlations arising from earlier times in this effective dynamics, one precisely finds the recursion relations usually employed in the signal-to-noise approach. The consequences of this approximation as well as the physics behind it are discussed. In particular, it is pointed out why it is hard to notice the effects, especially for model parameters corresponding to retrieval. Numerical simulations confirm these findings. The signal-to-noise analysis is then extended to include all correlations, making it a full theory for dynamics at the level of the generating functional analysis. The results are applied to the frequently employed extremely diluted (a)symmetric architectures and to sequence processing networks., 26 pages, 3 figures

The parallel dynamics of the fully connected Blume-Emery-Griffiths neural network model is studied for arbitrary temperature. By employing a probabilistic signal-to-noise approach, a recursive scheme is found determining the time evolution of the distribution of the local fields and, hence, the evolution of the order parameters. A comparison of this approach is made with the generating functional method, allowing to calculate any physical relevant quantity as a function of time. Explicit analytic formula are given in both methods for the first few time steps of the dynamics. Up to the third time step the results are identical. Some arguments are presented why beyond the third time step the results differ for certain values of the model parameters. Furthermore, fixed-point equations are derived in the stationary limit. Numerical simulations confirm our theoretical findings., Comment: 26 pages in Latex, 8 eps figures

The thermodynamic and retrieval properties of fully connected Blume-Emery-Griffiths networks, storing ternary patterns, are studied using replica mean-field theory. Capacity-temperature phase diagrams are derived for several values of the pattern activity. It is found that the retrieval phase is the largest in comparison with other three-state neuron models. Furthermore, the meaning and stability of the so-called quadrupolar phase is discussed as a function of both the temperature and the pattern activity. Where appropriate, the results are compared with the diluted version of the model., Comment: 13 pages, 6 figures, iopart style

A Blume-Emery-Griffiths perceptron model is introduced and its optimal capacity is calculated within the replica-symmetric Gardner approach, as a function of the pattern activity and the imbedding stability parameter. The stability of the replica-symmetric approximation is studied via the analogue of the Almeida-Thouless line. A comparison is made with other three-state perceptrons., Comment: 10 pages, 8 figures

We investigate an XY spin-glass model in which both spins and interactions (or couplings) evolve in time, but with widely separated time-scales. For large times this model can be solved using replica theory, requiring two levels of replicas, one level for the spins and one for the couplings. We define the relevant order parameters, and derive a phase diagram in the replica-symmetric approximation, which exhibits two distinct spin-glass phases. The first phase is characterized by freezing of the spins only, whereas in the second phase both spins and couplings are frozen. A detailed stability analysis leads also to two distinct corresponding de Almeida-Thouless lines, each marking continuous replica-symmetry breaking. Numerical simulations support our theoretical study., Comment: 30 pages, 15 figures, LaTeX (J. Phys. A style)

Ashkin-Teller type perceptron models are introduced. Their maximal capacity per number of couplings is calculated within a first-step replica-symmetry-breaking Gardner approach. The results are compared with extensive numerical simulations using several algorithms., Comment: 8 pages in Latex with 2 eps figures, RSB1 calculations has been added

A self-control mechanism for the dynamics of a three-state fully-connected neural network is studied through the introduction of a time-dependent threshold. The self-adapting threshold is a function of both the neural and the pattern activity in the network. The time evolution of the order parameters is obtained on the basis of a recently developed dynamical recursive scheme. In the limit of low activity the mutual information is shown to be the relevant parameter in order to determine the retrieval quality. Due to self-control an improvement of this mutual information content as well as an increase of the storage capacity and an enlargement of the basins of attraction are found. These results are compared with numerical simulations., 8 pages, 8 ps.figures

The optimal capacity of graded-response perceptrons storing biased and spatially correlated patterns with non-monotonic input-output relations is studied. It is shown that only the structure of the output patterns is important for the overall performance of the perceptrons., 4 pages, 4 figures

The thermodynamic and retrieval properties of the Ashkin-Teller neural network model storing an infinite number of patterns are examined in the replica-symmetric mean-field approximation. In particular, for linked patterns temperature-capacity phase diagrams are derived for different values of the two-neuron and four-neuron coupling strengths. This model can be considered as a particular non-trivial generalisation of the Hopfield model and exhibits a number of interesting new features. Some aspects of replica-symmetry breaking are discussed., Latex, 12 pages, 5 ps figures, minor changes: some comments added

The parallel dynamics of the asymmetric extremely diluted Ashkin-Teller neural network is studied using signal-to-noise analysis techniques. Evolution equations for the order parameters are derived, both at zero and finite temperature. The retrieval properties of the network are discussed in terms of the four-spin coupling strength and the temperature. It is shown that the presence of a four-spin coupling enhances the retrieval quality., Comment: 6 pages, 2 eps figures

The inclusion of a threshold in the dynamics of layered neural networks with variable activity is studied at arbitrary temperature. In particular, the effects on the retrieval quality of a self-controlled threshold obtained by forcing the neural activity to stay equal to the activity of the stored paterns during the whole retrieval process, are compared with those of a threshold chosen externally for every loading and every temperature through optimisation of the mutual information content of the network. Numerical results, mostly concerning low activity networks are discussed., Comment: 15 pages, Latex2e, 6 eps figures

The replica method is applied to a neural network model with state-dependent synapses built from those patterns having a correlation with the state of the system greater than a certain threshold. Replica-symmetric and first-step replica-symmetry-breaking results are presented for the storage capacity at zero temperature as a function of this threshold value. A comparison is made with existing results based upon mean-field equations obtained by using a statistical method., 9 pages, including 2 figures

We investigate an XY spin-glass model in which both spins and couplings evolve in time: the spins change rapidly according to Glauber-type rules, whereas the couplings evolve slowly with a dynamics involving spin correlations and Gaussian disorder. For large times the model can be solved using replica theory. In contrast to the XY-model with static disordered couplings, solving the present model requires two levels of replicas, one for the spins and one for the couplings. Relevant order parameters are defined and a phase diagram is obtained upon making the replica-symmetric Ansatz. The system exhibits two different spin-glass phases, with distinct de Almeida-Thouless lines, marking continuous replica-symmetry breaking: one describing freezing of the spins only, and one describing freezing of both spins and couplings., Comment: 7 pages, Latex, 3 eps figures

Perceptrons with graded input-output relations and a limited output precision are studied within the Gardner-Derrida canonical ensemble approach. Soft non- negative error measures are introduced allowing for extended retrieval properties. In particular, the performance of these systems for a linear and quadratic error measure, corresponding to the perceptron respectively the adaline learning algorithm, is compared with the performance for a rigid error measure, simply counting the number of errors. Replica-symmetry-breaking effects are evaluated., Comment: 26 pages, 10 ps figures

An Ashkin-Teller neural network, allowing for two types of neurons is considered in the case of low loading as a function of the strength of the respective couplings between these neurons. The storage and retrieval of embedded patterns built from the two types of neurons, with different degrees of (in)dependence is studied. In particular, thermodynamic properties including the existence and stability of Mattis states are discussed. Furthermore, the dynamic behaviour is examined by deriving flow equations for the macroscopic overlap. It is found that for linked patterns the model shows better retrieval properties than a corresponding Hopfield model., Comment: 20 pages, 6 figures, Latex with postscript figures in one tar.gz file

This paper studies two concepts of time delay in a few-particle scattering. The first is a global time delay that refers to the total advancement or retardation of the entire wave-packet motion owing to the presence of interactions not contained in the asymptotic Hamiltonian. The second type, the angular time delay, is the early or late arrival of a particle in a counter subtending an angle theta with respect to the incident beam direction. In the two-body problem the magnitude of this time delay is known to be (d/dE) f (E,theta), where f (E,theta) is the scattering amplitude at energy E. The definition of these two kinds of time delay in the three-body problem is discussed. A generalization of the relation between angular time delay and the scattering amplitude is provided that is valid for elastic, rearrangement, and breakup scattering. The interdependence of these two kinds of multichannel time delay is established. Possible physical applications of the resulting theory are discussed. (AIP)

Two stochastic representations of two-dimensional non-abelian supersymmetric magnetic field systems are studied. In the case of SU(2) symmetry, the first is written in terms of a Markov process with continuous component coupled to a discrete component, provided a positivity condition is satisfied. The second is based upon an embedded U(1) model and leads to a representation of the euclidean wave function as a stochastic average with respect to gaussian white noise. The relation with the Nicolai map for the abelian model is discussed.

Using Faddeev’s form of time‐dependent scattering theory, we give an abstract definition of time delay valid for multichannel scattering. For the three‐body scattering problem we find an explicit relation, that is valid on the energy shell, between the time‐delay operator and the S operators and their energy derivatives.

Stochastic identities for supersymmetric models on a manifold are derived. For reasons connected with ordering ambiguities, only quantum mechanical models are considered. In particular, the N = 2 σ-model, the N = 2 kahlerian Wess-Zumino model and the N = 1 spinning particle coupled with a vector potential are treated. Common features of these identities are discussed.

Low-energy scattering for Schrodinger operators of the type H= −Δ + V in L2(R) with ∫Rdx V(x) = 0 is considered. The possibility of zero-energy eigenstates of H is taken into account explicitly. In particular, a Laurent expansion for the transition operator and recursion relations for its coefficients are provided and the leading behaviour of the scattering operator is given in all possible cases.

The single scattering optical potential was in the multiple scattering approaches of Watson and KMT. Since the kinematics of single scattering is three-body in nature, one builds a three-body model of this term. This approach can include the proper kinematics for the struck nucleon, the identity of the target nucleons, and the binding interaction of the struck nucleon. Integral equations of the Faddeev type are derived for both the Watson and KMT single scattering optical potentials. Unitarity relations are investigated and one observes that these relations can be expanded in order to identify the intermediate states responsible for the absorptive parts. The transition amplitudes to the inelastic states implicit in the model are extracted and evaluated. This permits one to understand the physical meaning of the imaginary part in precise terms. The same procedure is applied to the closure and impulse approximations for the single scattering term and their implicit inelastic states and reaction amplitudes are identified. These approximations are evaluated by analyzing the inelastic data. It is concluded that the impulse approximation to the Watson single scattering term should provide the best two-body approximation to a single-scattering optical potential.

A study of the time evolution and a stability analysis of the phases in the extremely diluted Blume-Emery- Griffiths neural network model are shown to yield new phase diagrams in which fluctuation retrieval may drive pattern retrieval. It is shown that saddle-point solutions associated with fluctuation overlaps slow down the flow of the network states towards the retrieval fixed points. A comparison of the performance with other three-state networks is also presented., One of the authors (D.B.) wants to thank T. Verbeiren and J. Busquets Blanco for critical discussions and the Fund for Scientific Research–Flanders, Belgium for financial support. D.R.C.D. acknowledges a Ramón y Cajal grant from the Spanish Ministry of Science (MCyT), and thanks the K.U. of Leuven, Belgium, for a visiting grant. E.K. warmly thanks for hospitality the Abdus Salam International Center for Theoretica Physics, Trieste, and is financially supported by Grant No. DGI (MCyT) BFM 2001-291-C02-01. The work of W.K.T. was partially supported by the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq), Brazil, and the same author acknowledges the Fundacão de Amparo à Pesquisa do Estado de Rio Grande do Sul (FAPERGS), Brazil for a visiting scientist grant to the IFUFRGS.

The categorization properties of an attractor network of three-state neurons which infers three-state concepts from examples are studied. The evolution equations governing the parallel dynamics at zero temperature for the overlap between the state of the network and the examples, the state of the network and the concepts as well as the neuron activity are discussed in the limit of extreme dilution. A transition from a retrieval region to a categorization region is found when the number of examples or their correlations are increased. If the pattern activity is small enough, the examples (concepts) are very well retrieved (categorized) for an appropriate choice of the zero-activity threshold of the neurons., 17 pages incl. figures, LaTeX, uses epsfig.sty