Search

Previous research has shown that rats reared in simple/impoverished environments demonstrate greater repetitive responding for sensory reinforcers ( e.g. , light onset). Moreover, the brains of these rats are abnormally developed, compared to brains of rats reared in more complex/enriched environments. Repetitive behaviors are commonly observed in individuals with developmental disorders. Some of these repetitive behaviors could be maintained by the reinforcing effects of the sensory stimulation that they produce. Therefore, rearing rats in impoverished conditions may provide an animal model for certain repetitive behaviors associated with developmental disorders. We hypothesize that in rats reared in simple/impoverished environments, the normal habituation process to sensory reinforcers is impaired, resulting in high levels of repetitive behaviors. We tested the hypothesis using an operant sensory reinforcement paradigm in rats reared in simple/impoverished (IC), standard laboratory (SC), and complex/enrichened conditions (EC, treatments including postnatal handling and environmental enrichment). Results show that the within-session habituation of the reinforcer effectiveness of light onset was slower in the IC and SC rats than in the EC rats. A dishabituation challenge indicated that within-session decline of responses was due to habituation and not motor fatigue or sensory adaptation. In conclusion, rearing rats in simple/impoverished environments, and comparing them to rats reared in more complex/enriched environments, may constitute a useful approach for studying certain repetitive behaviors associated with developmental disorders. [ABSTRACT FROM AUTHOR]

The hierarchical structure of human and animal behavior has been of critical interest in neuroscience for many years. Yet understanding the neural processes that give rise to such structure remains an open challenge. In recent research, a new perspective on hierarchical behavior has begun to take shape, inspired by ideas from machine learning, and in particular the framework of hierarchical reinforcement learning. Hierarchical reinforcement learning builds on traditional reinforcement learning mechanisms, extending them to accommodate temporally extended behaviors or subroutines. The resulting computational paradigm has begun to influence both theoretical and empirical work in neuroscience, conceptually aligning the study of hierarchical behavior with research on other aspects of learning and decision making, and giving rise to some thought-provoking new findings. [Copyright &y& Elsevier]

Abstract: Background: Light onset can be both a sensory reinforcer (SR) with intrinsic reinforcing properties, and a conditioned reinforcer (CR) which predicts a biologically important reinforcer. Stimulant drugs, such as methamphetamine (METH), may increase the reinforcing effectiveness of CRs by enhancing the predictive properties of the CR. In contrast, METH-induced increases in the reinforcing effectiveness of SRs, are mediated by the immediate sensory consequences of the light. Methods: The effects of novelty (on SRs) and METH (on both CRs and SRs) were tested. Experiment 1: rats were pre-exposed to 5s light and water pairings presented according to a variable-time (VT) 2min schedule or unpaired water and light presented according to independent, concurrent VT 2min schedules. Experiment 2: rats were pre-exposed to 5s light presented according to a VT 2min schedule, or no stimuli. In both experiments, the pre-exposure phase was followed by a test phase in which 5s light onset was made response-contingent on a variable-interval (VI) 2min schedule and the effects of METH (0.5mg/kg) were determined. Results: Novel light onset was a more effective reinforcer than familiar light onset. METH increased the absolute rate of responding without increasing the relative frequency of responding for both CRs and SRs. Conclusion: Novelty plays a role in determining the reinforcing effectiveness of SRs. The results are consistent with the interpretation that METH-induced increases in reinforcer effectiveness of CRs and SRs may be mediated by immediate sensory consequences, rather than prediction. [Copyright &y& Elsevier]

Abstract: The term “sensory reinforcer” has been used to refer to sensory stimuli (e.g. light onset) that are primary reinforcers in order to differentiate them from other more biologically important primary reinforcers (e.g. food and water). Acquisition of snout poke responding for a visual stimulus (5s light onset) with fixed ratio 1 (FR 1), variable-interval 1min (VI 1min), or variable-interval 6min (VI 6min) schedules of reinforcement was tested in three groups of rats (n =8/group). The VI 6min schedule of reinforcement produced a higher response rate than the FR 1 or VI 1min schedules of visual stimulus reinforcement. One explanation for greater responding on the VI 6min schedule relative to the FR 1 and VI 1min schedules is that the reinforcing effectiveness of light onset habituated more rapidly in the FR 1 and VI 1min groups as compared to the VI 6min group. The inverse relationship between response rate and the rate of visual stimulus reinforcement is opposite to results from studies with biologically important reinforcers which indicate a positive relationship between response and reinforcement rate. Rapid habituation of reinforcing effectiveness may be a fundamental characteristic of sensory reinforcers that differentiates them from biologically important reinforcers, which are required to maintain homeostatic balance. [Copyright &y& Elsevier]

Abstract: Background: The human personality trait of sensation seeking (SS) indicates an attraction to novel sensations and experiences, and is associated with greater likelihood of drug abuse. In rodents, locomotor activity in a novel environment (Loco) has been found to predict drug self-administration (SA), and has been hypothesized to be a translational model of human SS. Previously, we reported (Gancarz et al., 2011 ) that high responder (HR) animals responded more than low responder (LR) animals to produce a response contingent light onset. The primary goal of this paper was a detailed analysis of the association between Loco and light contingent responding in a large sample of rats (n =93). Methods: Male rats were pre-exposed to dark operant test chambers for ten 30min sessions and baseline levels of responding (snout poking) were determined. The pre-exposure phase was followed by 6 sessions during which active responding produced a visual sensory reinforcer (VSR; 5s light onset) according to a variable interval 1min schedule of reinforcement. After completion of the VSR phase, Loco was tested. Results: The activating effects (total responding) of light were associated with Loco, but the response guiding effects (proportion of active responding) of the light were not. In addition, HR rats habituated more slowly in both the VSR and Loco tests than LR rats. Conclusions: These data indicate that VSR measures aspects of the rodent''s response to novel sensations and experiences that are not detected by Loco. These data provide some evidence for the use of light reinforcement as an animal model of SS. [Copyright &y& Elsevier]

Abstract: There is evidence that visual stimuli used to signal drug delivery in self-administration procedures have primary reinforcing properties, and that drugs of abuse enhance the reinforcing properties of such stimuli. Here, we explored the relationships between locomotor activity, responding for a visual stimulus, and self-administration of methamphetamine (METH). Rats were classified as high or low responders based on activity levels in a novel locomotor chamber and were subsequently tested for responding to produce a visual stimulus followed by self-administration of a low dose of METH (0.025mg/kg/infusion) paired with the visual stimulus. High responder rats responded more for the visual stimulus than low responder rats indicating that the visual stimulus was reinforcing and that operant responding for a visual stimulus has commonalities with locomotor activity in a novel environment. Similarly, high responder rats responded more for METH paired with a visual stimulus than low responder rats. Because of the reinforcing properties of the visual stimulus, it was not possible to determine if the rats were responding to produce the visual stimulus, METH or the combination. We speculate that responding to produce sensory reinforcers may be a measure of sensation seeking. These results indicate that visual stimuli have unconditioned reinforcing effects which may have a significant role in acquisition of drug self-administration, a role that is not yet well understood. [Copyright &y& Elsevier]

Abstract: The brain is able to perform actions based on an adequate internal representation of the world, where task-irrelevant features are ignored and incomplete sensory data are estimated. Traditionally, it is assumed that such abstract state representations are obtained purely from the statistics of sensory input for example by unsupervised learning methods. However, more recent findings suggest an influence of the dopaminergic system, which can be modeled by a reinforcement learning approach. Standard reinforcement learning algorithms act on a single layer network connecting the state space to the action space. Here, we involve in a feature detection stage and a memory layer, which together, construct the state space for a learning agent. The memory layer consists of the state activation at the previous time step as well as the previously chosen action. We present a temporal difference based learning rule for training the weights from these additional inputs to the state layer. As a result, the performance of the network is maintained both, in the presence of task-irrelevant features, and at randomly occurring time steps during which the input is invisible. Interestingly, a goal-directed forward model emerges from the memory weights, which only covers the state–action pairs that are relevant to the task. The model presents a link between reinforcement learning, feature detection and forward models and may help to explain how reward systems recruit cortical circuits for goal-directed feature detection and prediction. [Copyright &y& Elsevier]

Abstract: Stimuli associated with nicotine (NIC) can acquire new meaning via Pavlovian conditioning. If a stimulus is associated with the primary reinforcing effects of NIC, the new conditional properties of the stimulus should make it a more valuable reinforcer (i.e., increase the motivation to obtain the stimulus), and this value should be based, in part, on the strength or intensity of the primary reinforcer (i.e., NIC dose). The purpose of the present study was to investigate whether NIC-conditioned reinforcement increased motivation to obtain non-NIC stimuli, as reflected by performance on a progressive ratio (PR) reinforcement schedule, and whether this increased motivation was systematically related to NIC dose. Two Paired groups were allowed to nose-poke for NIC (0.03 or 0.09mg/kg/infusion, IV) accompanied by 15-s illumination of a stimulus light (conditional stimulus or CS). Two Unpaired groups (0.03 or 0.09mg/kg/infusion) could also make a nose-poke response for the CS; however their NIC infusions were controlled by the Paired group (i.e., yoked design). A fifth group (CS-Only) was allowed to nose-poke for CS presentations and saline infusions. After 29 conditioning sessions the nose-poke operant was prevented by obscuring the receptacle and the CS (accompanied by saline infusion for all groups) was made contingent upon a novel operant response (lever press). During the acquisition of this novel response, each CS/saline infusion earned increased the number of responses required to earn the next CS/saline infusion. Pairings with the primary reinforcing effects of NIC resulted the acquisition of a novel response for the CS. Motivation to obtain the CS depended on salience (dose) of the primary reinforcement (NIC). [Copyright &y& Elsevier]

In Experiment 1, analogue functional analyses were conducted to identify the functions of stereotypy for six students with multiple profound disabilities. Results indicated that stereotypy (a) occurred across conditions, (b) occurred primarily when alone, or (c) occurred during all sessions except in the Control condition. Experiment 2 analyzed stereotypy while masking visual, auditory, or tactile sensory consequences. Results showed that stereotypy was maintained by visual stimulation, tactile stimulation, or was undifferentiated across conditions. In Experiment 3, we showed that stereotypy could be reduced by providing competing sensory stimulation. In Experiment 4, stereotypy that was undifferentiated in Experiment 1 was analyzed using a concurrent operants procedure. Results showed that stereotypy was not multiply determined, but occurred to produce visual sensory stimulation. Our findings are discussed in terms of the sensory and social reinforcers that maintain stereotypy, assessment procedures used to identify those reinforcers, and the interpretation of assessment results. [Copyright &y& Elsevier]

The effects of bilateral excitotoxic lesions of the pedunculopontine tegmental nucleus (PPTg) on sucrose intake were examined in three experiments. First, in tests of conditioned place preference using 20% sucrose as the reinforcer, it was shown that lesioned rats, regardless of whether they were food deprived or non-deprived, formed normal place preferences and showed normal amounts of locomotion. However, consumption of 20% sucrose in the pairing trials was increased in the deprived PPTg lesioned rats compared to their matched controls. A second experiment showed that sucrose consumption in the home cage was increased in both deprived and non-deprived PPTg lesioned rats, but only when the concentration of sucrose was greater than 12%: below this there were no differences in intake between the lesioned and control rats. In a third home cage experiment, it was again shown that non-deprived PPTg lesioned rats increased their consumption of 20% sucrose compared to controls. PPTg lesioned rats concomitantly reduced their intake of lab chow such that overall energy intake remained the same as that of control rats. These data are taken to suggest (i) that bilateral excitotoxic lesions of the PPTg increase consumption of sucrose selectively in conditions of high motivational excitement; (ii) that the perception of the rewarding value of 20% sucrose, as judged by place preference, is not affected by these lesions; and (iii) that PPTg lesioned rats are able to adjust their energy intake to accommodate increased sucrose loads. These data are consistent with the hypothesis that bilateral excitotoxic lesions of the PPTg do not affect energy balance regulation or judgment of the hedonic value of sucrose, but that they do affect the control of responding in the face of high levels of motivational excitement. [Copyright &y& Elsevier]