Search

A variety of animals sometimes engage in a form of maladaptive decision-making characterized by repeatedly choosing an option providing food-predictive stimuli even though they earn less food for doing so. The temporal information-theoretic model suggests that such suboptimal choice depends on competition between the bits of temporal information conveyed by food-predictive stimuli (which encourages suboptimal choice) and the rate of food delivery (which encourages optimal choice). The model assumes that competition between these two sources of control is based on the ratio of the delay to food (Df) and the delay to food-predictive stimuli (Ds) at the choice point (i.e., Df/Ds). Research with both rats and pigeons suggests that temporal information outcompetes the rate of food delivery, thereby generating suboptimal choice, when the delay to food (Df) is sufficiently long. Limited data with pigeons, and none with rats, suggests that the rate of food delivery outcompetes temporal information, thereby generating optimal choice, when the delay to food-predictive stimuli (Ds) is sufficiently long. The present experiment sought to clarify whether longer delays to food-predictive stimuli decrease suboptimal choice in rats. We found that while longer delays to food (Df) increased suboptimal choice in rats, longer delays to food-predictive stimuli (Ds) did not decrease suboptimal choice. These results suggest a potential difference between rats and pigeons in the manner in which food-predictive stimuli and food itself compete to control choice. In terms of the temporal information-theoretic model, competition between temporal information and the rate of food delivery in rats appears to be influenced only by the delay to food at the choice point. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

The dorsolateral striatum (DLS) is involved in learning and executing procedural actions. Cell ensembles in the DLS, but not the dorsomedial striatum (DMS), exhibit a burst of firing at the start of a well-learned action sequence (“task-bracketing”). However, it is currently unclear what information is contained in these bursts. Some theories suggest that these bursts should represent the procedural action sequence itself (that they should be about future action chains), whereas others suggest that they should contain representations of the current state of the world, taking into account primarily past information. In addition, the DLS local field potential shows transient bursts of power in the 50 Hz range (γ50) around the time a learned action sequence is initiated. However, it is currently unknown how bursts of activity in DLS cell ensembles and bursts of γ50 power in the DLS local field potential are related to each other. We found that DLS bursts at lap initiation in rats represented recently experienced reward locations more than future procedural actions, indicating that task initiation DLS bursts contain primarily retrospective, rather than prospective, information to guide procedural actions. Furthermore, representations of past reward locations increased during periods of increased γ50 power in the DLS. There was no evidence of task initiation bursts, increased γ50 power, or retrospective reward location information in the neighboring dorsomedial striatum. These data support a role for the DLS in model-free theories of procedural decision-making over planned action-chain theories, suggesting that procedural actions derive from representations of the current and recent past. Significance Statement: While it is well-established that the dorsolateral striatum (DLS) plays a critical role in procedural decision-making, open questions remain about the kinds of representations contained in DLS ensemble activity that guide procedural actions. We found that DLS, but not DMS, cell ensembles contained nonlocal representations of past reward locations that appear moments before task initiation DLS bursts. These retrospective representations were temporally linked to a rise in γ50 power that also preceded the characteristic DLS burst at task initiation. The strength of both γ50 power and retrospective representations grew with experience. These results support models of procedural decision-making based on associations between available actions and the current state of the world over models based on planning over action-chains.

Numerous examples in the decision-making literature demonstrate that animals sometimes make choices that are not in their long-term best interest. One particular example finds pigeons preferring a low-probability alternative in lieu of a high-probability alternative, referred to as suboptimal choice. Although there is ample evidence that pigeons engage in such suboptimal choice, there is currently weak evidence (at best) that rats also do so. Cunningham and Shahan's (2018) temporal information-theoretic model suggests that suboptimal choice in pigeons arises when (1) the low-probability alternative provides stimuli that convey more temporal information than stimuli associated with the high-probability alternative and (2) when the delay to food is much longer relative to the delay to temporally informative signals at the choice point. The latter condition plays the important role of biasing decision making to be governed by the relative temporal information conveyed by stimuli rather than the relative rate of food delivery. The present experiment explored the possibility that rats will engage in suboptimal choice if the delay to food at the choice point is sufficiently long, as the temporal information-theoretic model suggests. Rats were given a choice between a suboptimal alternative providing food 20% of the time and an optimal alternative providing food 50% of the time. The suboptimal alternative provided stimuli that differentially signaled choice outcomes whereas the optimal alternative did not. The postchoice delay was manipulated across conditions and ranged from 10 s to 50 s. As with previous research, rats did not engage in suboptimal choice when the postchoice delay was 10 s. However, once the delay was at least 30 s, rats engaged in suboptimal choice. These results are consistent with the temporal information-theoretic model of suboptimal choice and suggest that rats and pigeons likely do not differ in the decision-making processes involved in the suboptimal choice procedure. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Delivery of alternative reinforcers in the presence of stimuli previously associated with reinforcement for target behavior increases the susceptibility of target behavior to relapse. To explore contingencies that might mitigate this counter-therapeutic effect, we trained pigeons on a procedure that entailed extinction of previously reinforced target-key pecking, access to a distinct stimulus context contingently on refraining from target behavior (differential-reinforcement-of-other-behavior; DRO), and reinforcement of alternative-key pecks (differential-reinforcement of alternative behavior; DRA) in that context. This DRO-DRA treatment was compared with standard DRA in successive conditions, counterbalanced across pigeons. Target behavior extinguished more rapidly in the Standard-DRA condition. When alternative reinforcement was discontinued, however, there was less resurgence after DRO-DRA than after Standard DRA. In a third condition, the DRO contingency was suspended so that the former DRA stimuli were not presented (DRO-NAC), and resurgence was greater than in the Standard-DRA and DRO-DRA conditions. Reinstatement produced by response-independent reinforcers was small and similar across conditions. Subsequent reacquisition of target-key pecking under baseline reinforcement conditions was faster following DRO-NAC than Standard-DRA or DRO-DRA. These findings suggest that DRO-DRA might serve as a useful method in clinical settings for reducing problem behavior while minimizing the threat of posttreatment relapse.

The idea that stimuli might function as conditioned reinforcers because of the information they convey about primary reinforcers has a long history in the study of learning. However, formal application of information theory to conditioned reinforcement has been largely abandoned in modern theorizing because of its failures with respect to observing behavior. In this paper we show how recent advances in the application of information theory to Pavlovian conditioning offer a novel approach to conditioned reinforcement. The critical feature of this approach is that calculations of information are based on reductions of uncertainty about expected time to primary reinforcement signaled by a conditioned reinforcer. Using this approach, we show that previous failures of information theory with observing behavior can be remedied, and that the resulting framework produces predictions similar to Delay Reduction Theory in both observing-response and concurrent-chains procedures. We suggest that the similarity of these predictions might offer an analytically grounded reason for why Delay Reduction Theory has been a successful theory of conditioned reinforcement. Finally, we suggest that the approach provides a formal basis for the assertion that conditioned reinforcement results from Pavlovian conditioning and may provide an integrative approach encompassing both domains.

Suboptimal choice refers to preference for an alternative offering a low probability of food (suboptimal alternative) over an alternative offering a higher probability of food (optimal alternative). Numerous studies have found that stimuli signaling probabilistic food play a critical role in the development and maintenance of suboptimal choice. However, there is still much debate about how to characterize how these stimuli influence suboptimal choice. There is substantial evidence that the temporal information conveyed by a food-predictive signal governs its function as both a Pavlovian conditioned stimulus and as an instrumental conditioned reinforcer. Thus, we explore the possibility that food-predictive signals influence suboptimal choice via the temporal information they convey. Application of this temporal information-theoretic approach to suboptimal choice provides a formal, quantitative framework that describes how food-predictive signals influence suboptimal choice in a manner consistent with related phenomena in Pavlovian conditioning and conditioned reinforcement. Our reanalysis of previous data on suboptimal choice suggests that, generally speaking, preference in the suboptimal choice procedure tracks relative temporal information conveyed by food-predictive signals for the suboptimal and optimal alternatives. The model suggests that suboptimal choice develops when the food-predictive signal for the suboptimal alternative conveys more temporal information than that for the optimal alternative. Finally, incorporating a role for competition between temporal information provided by food-predictive signals and relative primary reinforcement rate provides a reasonable account of existing data on suboptimal choice. (PsycINFO Database Record

Delivery of alternative reinforcers in the presence of stimuli previously associated with reinforcement for target behavior increases the susceptibility of target behavior to relapse. To explore contingencies that might mitigate this counter-therapeutic effect, we trained pigeons on a procedure that entailed extinction of previously reinforced target-key pecking, access to a distinct stimulus context contingently on refraining from target behavior (differential-reinforcement-of-other-behavior; DRO), and reinforcement of alternative-key pecks (differential-reinforcement of alternative behavior; DRA) in that context. This DRO-DRA treatment was compared with standard DRA in successive conditions, counterbalanced across pigeons. Target behavior extinguished more rapidly in the Standard-DRA condition. When alternative reinforcement was discontinued, however, there was less resurgence after DRO-DRA than after Standard DRA. In a third condition, the DRO contingency was suspended so that the former DRA stimuli were not presented (DRO-NAC), and resurgence was greater than in the Standard-DRA and DRO-DRA conditions. Reinstatement produced by response-independent reinforcers was small and similar across conditions. Subsequent reacquisition of target-key pecking under baseline reinforcement conditions was faster following DRO-NAC than Standard-DRA or DRO-DRA. These findings suggest that DRO-DRA might serve as a useful method in clinical settings for reducing problem behavior while minimizing the threat of posttreatment relapse.

We review quantitative accounts of behavioral momentum theory (BMT), its application to clinical treatment, and its extension to post-intervention relapse of target behavior. We suggest that its extension can account for relapse using reinstatement and renewal models, but that its application to resurgence is flawed both conceptually and in its failure to account for recent data. We propose that the enhanced persistence of target behavior engendered by alternative reinforcers is limited to their concurrent availability within a distinctive stimulus context. However, a failure to find effects of stimulus-correlated reinforcer rates in a Pavlovian-to-Instrumental Transfer (PIT) paradigm challenges even a straightforward Pavlovian account of alternative reinforcer effects. BMT has been valuable in understanding basic research findings and in guiding clinical applications and accounting for their data, but alternatives are needed that can account more effectively for resurgence while encompassing basic data on resistance to change as well as other forms of relapse.

Whereas intertemporal choice procedures are a common method for examining impulsive choice in nonhuman subjects, the apparatus used to implement this procedure varies across studies. The purpose of the present study was to compare impulsive choice between a two-lever chamber and a T-maze. In Experiment 1, rats chose between a smaller, immediate reinforcer and a larger, delayed reinforcer, first in a two-lever chamber and then in a T-maze. Delay to the larger reinforcer changed in an ascending and descending order (0-32 s) across sessions. Experiment 2 examined the same between-apparatus comparison but under steady-state conditions with the delay fixed at 32 s. In Experiment 1, choice for the larger, delayed reinforcer was generally higher in the T-maze than in the two-lever chamber. Similarly in Experiment 2, steady-state choice for the larger, delayed reinforcer was higher in the T-maze. Choice for the 32-s delayed reinforcer was also greater in Experiment 2 than in Experiment 1, suggesting that extended exposure to the delay is required for the T-maze to yield reliable impulsive choice data. While the reasons for the between-apparatus discrepancies are at present unknown, results from both experiments clearly demonstrate that the apparatus matters when assessing overall level and reliability of impulsive choice data.

Although patients with the carcinoid syndrome may have a variety of manifestations which involve the skin, gastrointestinal carcinoids rarely metastasize to the skin or subcutaneous tissue. The literature is reviewed regarding this occurrence, and an additional case is presented. Four of the six patients described had a primary tumor in the small intestine. This is not unexpected since this area has the highest rate of metastasis of carcinoid tumors.