Search

Attractor models are simplified models used to describe the dynamics of firing rate profiles of a pool of neurons. The firing rate profile, or the neuronal activity, is thought to carry information. Continuous attractor neural networks (CANNs) describe the neural processing of continuous information such as object position, object orientation, and direction of object motion. Recently it was found that in one-dimensional CANNs, short-term synaptic depression can destabilize bump-shaped neuronal attractor activity profiles. In this article, we study two-dimensional CANNs with short-term synaptic depression and spike frequency adaptation. We found that the dynamics of CANNs with short-term synaptic depression and CANNs with spike frequency adaptation are qualitatively similar. We also found that in both kinds of CANNs, the perturbative approach can be used to predict phase diagrams, dynamical variables, and speed of spontaneous motion.

The present study investigates the influence of attention modulation on neural tuning functions under a Gaussian correlation structure. Recent experiments have shown that attention modulates the tuning curve via its height and base rate. Inspired by this experimental finding, we previously showed the effective size of attention modulation (i.e. the critical length) on the neural population that enhances encoding accuracy. The previous result, however, was obtained under the assumption of uncorrelated firing, i.e. stimulus-conditional independence of neural responses. A question still remains whether the above findings can be applied to correlated firing. It is important to investigate this issue partly because neural firings are usually correlated but even more so because common attentional inputs may cause correlated firings. The present study first provides the general framework of attention modulation in relation to an attended stimulus and an actual stimulus and then shows the existence of a critical length under a Gaussian correlation structure. In order to improve encoding accuracy, measured by the Fisher information, the height and the base rate should be increased when the attended stimulus is in the critical length from the peak of the tuning curve and decreased otherwise. Furthermore, we confirm that a similar nature of the critical length also holds even when the neural decoder uses an uncorrelated unfaithful model. Thus, the existence of the critical length seems to be a ubiquitous phenomenon in attention modulation, and so its implications are discussed.

In this paper, we discuss dynamical behaviors of recurrently asymmetrically connected neural networks. We propose a new approach to study global convergence of the networks. Better test conditions for global convergence are given.

In this paper, we discuss dynamical behaviors of recurrently asymmetrically connected neural networks in detail. We propose an effective approach to study global and local stability of the networks. Many of well known existing results are unified in our framework, which gives much better test conditions for global and local stability. Sufficient conditions for the uniqueness of the equilibrium point and its stability conditions are given, too.

The natural gradient method has an ideal dynamic behavior which resolves the slow learning speed of the standard gradient descent method caused by plateaus. However, it is required to calculate the Fisher information matrix and its inverse, which makes the implementation of the natural gradient almost impossible. To solve this problem, a preliminary study has been proposed concerning an adaptive method of calculating an estimate of the inverse of the Fisher information matrix, which is called the adaptive natural gradient learning method. In this paper, we show that the adaptive natural gradient method can be extended to be applicable to a wide class of stochastic models: regression with an arbitrary noise model and classification with an arbitrary number of classes. We give explicit forms of the adaptive natural gradient for these models. We confirm the practical advantage of the proposed algorithms through computational experiments on benchmark problems.

Dynamical properties of a neural auto-associative memory with two-stage neurons are investigated theoretically. The two-stage neuron is a model whose output is determined by a two-stage nonlinear function of the internal field of the neuron (internal field is a weighted sum of outputs of the other neurons). The model is general, including nonmonotonic neurons as well as monotonic ones. Recent studies on associative memory revealed superiority of nonmonotonic neurons to monotonic ones. The present paper supplies theoretical verification on the high performance of nonmonotonic neurons and proves that the capacity of the auto-associative memory with two-stage neurons is O(n/ radicallog n), in contrast to O(n/log n) of simple threshold neurons. There is also a discussion of recall processes, where the radius of basin of attraction of memorized patterns is clarified. An intuitive explanation on why the performance is improved by nonmonotonic neurons is also provided by showing the correspondence of the recall processes of the two-stage-neuron net and orthogonal learning.