Search

The development of an 'artificial brain' is one of the greatest challenges in artificial intelligence, and its success will have innumerable benefits in many and diverse fields, including the creation of autonomous robotic agents. Most research effort is spent on modelling vertebrate brains. Yet smaller brains can display comparable cognitive sophistication, while being more experimentally accessible and amenable to modelling.

This paper presents a new brain-inspired, switching control approach for a car-like autonomous vehicle using a basal ganglia (BG) model as an action selection mechanism. The problem domain has challenging nonholonomic and state constraints which imply no single stabilizing controller solution is possible by time-invariant smooth state feedback. To allow the BG make the correct controller selection from a family of candidate motion controllers, a fuzzy logic-based salience model using reference and tracking error only is developed, and applied in a soft switching control mechanism. To demonstrate the effectiveness of our approach for motion tracking control, we show effective control for a circular trajectory tracking application. The performance and advantages of the proposed fuzzy salience model and the BG-based soft switching control scheme against a traditional single control method are compared.

We present a novel behavioural task for the investigation of how actions are added to an agent’s repertoire. In the task, free exploration of the range of possible movements with a manipulandum, such as a joystick, is recorded. A subset of these movements trigger a reinforcing signal. Our interest is in how those elements of total behaviour which cause an unexpected outcome are identified and stored. This process is necessarily prior to the attachment of value to different actions [Redgrave, P., Gurney, K.: The short-latency dopamine signal: A role in discovering novel actions? Nat. Rev. Neurosci. 7(12), 967–975 (2006)]. The task allows for critical tests of reinforcement prediction error theories [e.g. Schultz, W., Dayan, P., Montague, P.: A neural substrate of prediction and reward. Science 275, 1593–1599 (1997)], as well as providing a window on a number of other issues in action learning. The task provides a paradigm where the exploratory motive drives learning, and as such we view it as in the tradition of Thorndike [Animal intelligence (1911)]. Our task is easily scalable in difficulty, is adaptable across species and provides rich set of behavioural measures throughout the action-learning process. Targets can be defined in spatial, temporal or kinematic/gestural terms, and the task also allows the concatenation of actions to be investigated. Action learning requires integration across spatial, kinematic and temporal dimensions. The task affords insight into these (and into the process of integration).

We introduce a biologically motivated, formal framework or “ontology” for dealing with many aspects of action discovery which we argue is an example of intrinsically motivated behaviour (as such, this chapter is a companion to that by Redgrave et al. in this volume). We argue that action discovery requires an interplay between separate internal forward models of prediction and inverse models mapping outcomes to actions. The process of learning actions is driven by transient changes in the animal’s policy (repetition bias) which is, in turn, a result of unpredicted, phasic sensory information (“surprise”). The notion of salience as value is introduced and broken down into contributions from novelty (or surprise), immediate reward acquisition, or general task/goal attainment. Many other aspects of biological action discovery emerge naturally in our framework which aims to guide future modelling efforts in this domain.

Our interest is in the neural circuitry which supports the discovery and encoding of novel actions. We discuss the significant existing literature which identifies the basal ganglia, a complex of subcortical nuclei, as important in both the selection of actions and in reinforcement learning. We discuss the complementarity of these problems of action selection and action learning. Two basic mechanisms of biasing action selection are identified: (a) adjusting the relative strengths of competing inputs and (b) adjusting the relative sensitivity of the receiver of reinforced inputs. We discuss the particular importance of the phasic dopamine signal in the basal ganglia and its proposed role in conveying a reward prediction error. Temporal constraints of this signal limit the information it can convey to immediately surprising sensory events, thus—we argue—making it inappropriate to convey information regarding the economic value of actions (as proposed by the reward prediction error hypothesis). Rather, we suggest this signal is ideal to support the identification of novel actions and their encoding via the biasing of future action selection.

A novel soft switching control approach is presented in this paper for autonomous vehicles by using a new functional model for Basal Ganglia (BG). In the proposed approach, a family of fundamental controllers is treated as each of a set of basic controllers are thought of as an ‘action’ which may be selected by the BG in a soft switching regime for real-time control of autonomous vehicle systems. Three controllers, i.e., conventional Proportional-Integral-Derivative (PID) controller, a PID structure-based pole-zero placement controller, and a pole only placement controller are used in this paper to support the proposed soft switching control strategy. To demonstrate the effectiveness of the proposed soft switching approach for nonlinear autonomous vehicle control (AVC), the throttle, brake and steering subsystems are focused on in this paper because they are three key subsystems in the whole AVC system. Simulation results are provided to illustrate the performance and effectiveness of the proposed soft switching control approach by applying it to the abovementioned subsystems.

Huntington's is a genetic, progressive neuro-degenerative disease, causing massive network degradation effecting the medium spiny neurons of the striatum in the basal ganglia (a set of sub-cortical nuclei, believed to be critical for action selection). Despite substantial striatal cell atrophy, some cognitive functions have been shown to improve in manifest Huntington's disease patients over healthy and pre-symptomatic Huntington's disease patients. Using a detailed model of the striatal microcircuit, we show that combining current ideas about the underlying causes of the disease could lead to the counter-intuitive result of improved competitive network dynamics for signal selection.

A general problem in robotics is how to best utilize sensors to classify the robot's environment. The BIOTACT project (BIOmimetic Technology for vibrissal Active Touch) is a collaboration between biologists and engineers that has led to many distinctive robots with artificial whisker sensing capabilities. One problem is to construct classifiers that can recognize a wide range of whisker sensations rather than constructing different classifiers for specific features. In this article, we demonstrate that a stationary naive Bayes classifier can perform such a general classification by applying it to various robot experiments. This classifier could be a key component of a robot able to learn autonomously about novel environments, where classifier properties are not known in advance.

Navigating the visual world is a challenging problem for autonomous agents, which must be flexible, robust, and preferably easily extensible in order to meet changing task demands. Here, we outline the rationale for an approach to constructing such agents biomimetically, even if they appear ‘over engineered’ at first glance, using the problem of gaze redirection in an attentional task.

We present a biologically inspired neural network model of visual orienting (using saccadic eye movements) in which targets are preferentially selected according to their reward value. Internal representations of visual features that guide saccades are developed in a self-organised map whose plasticity is modulated under reward. In this way, only those features relevant for acquiring rewarding targets are generated. As well as guiding the formation of feature representations, rewarding stimuli are stored in a working memory and bias future saccade generation. In addition, a reward prediction error is used to initiate retraining of the self-organised map to generate more efficient representations of the features when necessary.

This paper describes emergent neurobiological characteristics of an intelligent multiple-controller that has been developed for controlling the throttle, brake and steering subsystems of a validated vehicle model. Simulation results demonstrate the effectiveness of the proposed approach. Importantly, the controller exhibits discrete behaviours, governed by its two component controllers. These controllers are selected according to task demands by a fuzzy-logic based supervisor. The system therefore displays `action selection' under central switched control, as has been proposed to take place in the vertebrate brain. In addition, the supervisor and modular controllers have analogues with the higher and lower levels of functionality associated with the strata in layered brain architectures. Several further similarities are identified between the biology and the vehicle controller. We conclude that advances in neuroscience and control theory have reached a critical mass which make it timely for a new rapprochement of these disciplines.

A real-time, large scale, leaky-integrate-and-fire neural network processor realized using FPGA is presented. This has been designed, as part of a collaborative project, to investigate and implement biologically plausible models of the rodent vibrissae based somatosensory system to control a robot. An emphasis has been made on hard real-time performance of the processor, as it is to be used as part of a feedback control system. This has led to a revision of some of the established modelling protocols used in other hardware spiking neural network processors. The underlying neuron model has the ability to model synaptic noise and inter-neural propagation delays to provide a greater degree of biological plausibility. The processor has been demonstrated modelling real neural circuitry in real-time, independent of the underlying neural network activity.

We have proposed that the basal ganglia act as the central switching mechanism for the action selection system of the vertebrate brain. Simulation of our functional model of basal ganglia demonstrated that their output was consistent with this action selection hypothesis. Here we extend this model by incorporating anatomically-inspired local inhibitory axon collateral networks into two basal ganglia nuclei (globus pallidus and substantia nigra pars reticulata). Through simulation it is demonstrated that the basal ganglia's ability to function as a selection mechanism is impaired by the individual addition of the collateral networks but slightly improved when they co-exist. Therefore, we predict the existence of local axon collaterals in the entopeduncular nucleus because of its functional equivalence with the substantia nigra pars reticulata. We conclude that the action selection hypothesis is supported by the continued functioning of the basal ganglia model as a switching mechanism following appropriate anatomically-inspired additions.