Search

When pigeons are given a choice between 50% signaled reinforcement and 100% reinforcement they typically do not choose optimally, sometimes even preferring 50% reinforcement. Smith and Zentall (J Exp Psychol Anim Behav Process 42:212-220, 2016) proposed that choice depends primarily on the predictive value of the signal for reinforcement associated with each alternative (both 100% reinforcement) and not the frequency of the signal for reinforcement (50% vs. 100%). With extended training, however, Case and Zentall (Behav Process, 2018) found that pigeons actually show a reliable preference for the 50% reinforcement alternative. They suggested that contrast between the expected outcome at the time of choice (50% reinforcement) and the value of the signal for reinforcement (100% reinforcement) is the mechanism responsible for the preference for the suboptimal alternative (for the optimal alternative there should be no contrast). In the present research, we tested the contrast hypothesis by increasing the probability of reinforcement for choice of the suboptimal alternative to 75%, thereby reducing the contrast between expected and obtained reinforcement and found a reduced preference for the suboptimal alternative. That is, increasing the probability of reinforcement for choice of the suboptimal alternative decreased suboptimal choice. Thus, preference for the suboptimal alternative appears to result from two mechanisms: (1) the value of the signal for reinforcement that follows choice of the alternative and (2) positive contrast between the expected and obtained probability of reinforcement. We compared this interpretation with other hypotheses.

The ephemeral-reward task involves providing subjects a choice between two distinctive stimuli, A and B, each containing an identical reward. If A is chosen, the food associated with A is obtained ...

The failure of certain species to learn a particular task while others learn it easily can help identify the learning mechanisms involved. In the ephemeral reward task, animals are given a choice between two distinctive stimuli, A and B, each containing an identical bit of food. If they choose A they get the food on A and the trial is over. If they choose B they get the food on B and they are allowed to get the food on A before the trial is over. Thus, it is optimal to choose B. Although cleaner fish (wrasse) and parrots acquire the optimal response easily, several primate species do not. Furthermore, pigeons and rats also appear to be unable learn to choose optimally. The failure of primates, pigeons, and rats to learn this task and the ease with which cleaner fish and parrots learn it raises important questions about the learning mechanisms involved in those differences. To account for these paradoxical findings, we proposed that certain species may have difficulty with this task because they tend to respond impulsively to the initial choice which has similar immediate outcomes and they do not associate the choice and reinforcement with the second reinforcement. To test this hypothesis, we temporally separated the initial choice from the first reinforcement by imposing a 20-s delay between the choice and its outcome. Under these conditions both pigeons and rats gradually acquired the optimal choice response. We suggest that impulsive choice may make it difficult to acquire certain tasks and imposing a delay between choice and outcome may decrease impulsivity and allow for closer to optimal task performance.

Animals choose suboptimally when provided with cues that signal whether reinforcement is coming or not. For example, pigeons do not prefer an alternative that always provides them with a signal for reinforcement over an alternative that provides them with a signal for reinforcement only half of the time and a signal for the absence of reinforcement the rest of the time. In the present research, we tested the hypothesis that if the results of the choice are delayed, pigeons will choose less suboptimally. We tested this hypothesis by forcing pigeons to wait following their choice, requiring them to complete a fixed-interval 20-s schedule prior to receiving the signals for reinforcement. In Experiment 1, we gave the pigeons a choice between (a) a 50% chance of receiving a signal for reinforcement or a 50% chance of receiving a signal for the absence of reinforcement and (b) a 100% chance of receiving a signal for reinforcement. When the signal for reinforcement was delayed, most of the pigeons chose optimally. When it was not delayed, most of the pigeons chose suboptimally. In Experiment 2, we gave the pigeons a choice between (a) a 25% chance of receiving a signal for reinforcement or a 75% chance of receiving a signal for nonreinforcement and (b) a 100% chance of receiving an unreliable signal for reinforcement (predicting reinforcement 75% of the time). When the signal was not delayed, the pigeons showed a strong tendency to choose suboptimally but they chose suboptimally much less when the signal was delayed.

Procrastination is the tendency to put off initiation or completion of a task. Although people are typically known to procrastinate, recent research suggests that they sometimes "pre-crastinate" by initiating a task sooner than they need to (Rosenbaum et al. in Psychological Science, 25(7), 1487-1496, 2014). A similar finding of precrastination was reported by Wasserman and Brzykcy (Psychonomic Bulletin & Review, 22, 1130-1134, 2015) with pigeons using a somewhat different procedure. In the present experiment, we used a procedure with pigeons that was more similar to the procedure used by Rosenbaum et al. Pigeons were given a choice between two sequences of events (concurrent chains). Choice of the procrastination chain resulted in color A, which 15-s later would change to color B and 5-s later resulted in reinforcement. Choice of the precrastination chain resulted in color C, which 5-s later would change to color D and 15-s later resulted in reinforcement. Thus, both chains led to reinforcement after 20 s. Results indicated that the pigeons procrastinated. That is, they preferred the 15-5 chain over the 5-15 chain. The results are consistent with Fantino's (Journal of the Experimental Analysis of Behavior, 12, 723-730, 1969) delay reduction theory, which posits that stimuli that signal a reduction in the delay to reinforcement, such as the 5-s stimulus that occurred immediately prior to reinforcement, serve as strong conditioned reinforcers and should be preferred. In support of this theory, the pigeons pecked most at the 5-s stimulus that led immediately to reinforcement, indicating that it had become a strong conditioned reinforcer. The results suggest that delay reduction theory, a theory that emphasizes the attraction to stimuli that predict reinforcement with a short delay, also may contribute to human procrastination behavior because when task completion comes just before the deadline, it may become a stronger conditioned reinforcer than if task completion comes earlier.

The ephemeral reward task provides a subject with a choice between two alternatives A and B. If it chooses alternative A, reinforcement follows and the trial is over. If it chooses alternative B, reinforcement follows but the subject can also respond to alternative A which is followed by a second reinforcement. Thus, it would be optimal to choose alternative B. Surprisingly, Salwiczek et al. (PLoS One 7:e49068, 2012. doi: 10.1371/journal.pone.00490682012 ) reported that adult fish (cleaner wrasse) mastered this task within 100 trials, whereas monkeys and apes had great difficulty with it. The authors attributed the species differences to ecological differences in the species foraging experiences. However, Pepperberg and Hartsfield (J Comp Psychol 128:298–306, 2014) found that parrots too learned this task easily. We have found that with a similar task pigeons are not able to learn to choose optimally within 400 trials (Zentall et al. in J Comp Psychol 130:138–144, 2016). In Experiment 1 of the present study, we found that rats did not learn to choose optimally in 840 trials; however, in Experiment 2 we added a prior commitment to the initial choice by increasing delay to reinforcement for the choice response from a single lever press to the first lever press after 20 s (FI20 s). In a comparable amount of training to Experiment 1, the rats learned to choose optimally. Although the use of a prior commitment increases the delay to reinforcement, it appears to reduce impulsive responding which in turn leads to optimal choice.

It has been assumed that when pigeons learn how to match to sample, they learn simple stimulus-response chains but not the concept of sameness. However, transfer to novel stimuli has been influenced by pigeons’ tendency to be neophobic. We trained pigeons on matching ( n = 7) and mismatching ( n = 8) with colors as samples and, with each sample, one color as the nonmatching comparison. We then replaced either the matching or the nonmatching stimulus with a familiar stimulus never presented with that sample. Results suggest that for both matching and mismatching, pigeons locate the stimulus that matches the sample: If the task involves matching, they chose it; if it involves mismatching, they avoid it. Thus, the concept of sameness is the basis for correct choice with both tasks. This finding suggests that sameness is a basic concept that does not have to be learned and may have evolved in many species, including humans.

Prior research has found that pigeons are indifferent between an option that always provides a signal for reinforcement and an alternative that provides a signal for reinforcement only 50% of the time (and a signal for the absence of reinforcement 50% of the time). This suboptimal choice suggests that the frequency of the signal for reinforcement plays virtually no role and choice depends only on the predictive value of the signal for reinforcement associated with each alternative. In the present research we tested the hypothesis that if there are two or three signals for reinforcement associated with the suboptimal alternative but each occurs only 25% or 17% of the time, respectively, pigeons would show a greater preference for the suboptimal alternative. Although we found that increasing the number of signals for reinforcement associated with the suboptimal alternative did not increase the preference for the suboptimal alternative (relative to a single signal for reinforcement) extended training on this task resulted in a significant preference for the suboptimal alternative by both groups. This result suggests that contrast between the expected outcome at the time of choice (50% reinforcement) and the value of the signal for reinforcement (100% reinforcement) is also responsible for choice of the suboptimal alternative.

In the Monty Hall dilemma, humans are initially given a choice among three alternatives, one of which has a hidden prize. After they have chosen, but before revealing whether they have won the prize, subjects are shown that one of the remaining alternatives does not have the prize, and they are asked whether they want to stay with their original choice or switch to the remaining alternative. Switching results in obtaining the prize two thirds of the time, but even after considerable training, humans fail to consistently adopt the optimal strategy of switching. Pigeons, however, show closer-to-optimal switching performance with this task. One of the reasons that humans choose suboptimally is their mistaken assumption that with two alternatives, the probabilities of winning the prize are the same for staying and switching, and staying may be preferred because of a sense of endowment (ownership of the initial response). When we tried to produce an endowment effect in pigeons by requiring 20 pecks (rather than one peck) for the initial choice, it actually resulted in faster acquisition of the switching response. In the present research with pigeons, we examined the finding from human research that subjects are more likely to switch if they are not responsible for making the initial choice (another approach to the endowment effect). Inconsistent with the findings with humans, we found that when the initial choice was made for the pigeons, they actually showed less of a tendency to switch than did pigeons that made the initial choice themselves.

Pigeons show relatively poor memory when trained on 0-s delayed matching and tested with longer delays. We hypothesized that one reason for the effect of delay may be a loss of association between sample responding and reinforcement. To test this hypothesis, we compared the effect of standard delays inserted between the offset of the sample and the onset of the comparison stimuli with similar delays inserted between the comparison response and reinforcement. We also manipulated whether the delay was dark or lit because there is some evidence that lit delays are disruptive but filled intervals may also help bridge delays. In Experiment 1, pigeons were trained with a 0-s delay and were tested with longer delays. When both delays and delay illumination were novel, we found no effect of delay of reinforcement and only a small effect of delay illumination. In Experiment 2, to eliminate the effect of the novelty of the delay and delay illumination, pigeons were trained with delays from the start. Results indicated once again that delay of reinforcement had little effect on matching accuracy. However, in Experiment 2, delay illumination produced a general decline in matching accuracy and when it preceded comparison choice its effect increased with increasing delay. The results indicate that the rate of forgetting seen in the typical delayed matching experiment is not due to delay of reinforcement for attention to the sample. The results also confirm that lighting the delay after the sample is detrimental to delayed matching. Surprisingly perhaps, lighting the delay after the comparison response is also detrimental to delayed matching. Finally, it was concluded that a more accurate assessment of the rate of forgetting in delayed matching can be obtained by training with variable delays from the outset.

When humans are asked to judge the value of a set of objects of excellent quality, they often give this set higher value than those same objects with the addition of some of lesser quality. This is an example of the affect heuristic, often referred to as the less-is-more effect. Monkeys and dogs, too, have shown this suboptimal effect. But in the present experiments, normally hungry pigeons chose optimally: a preferred food plus a less--preferred food over a more-preferred food alone. In Experiment 2, however, pigeons on a less-restricted diet showed the suboptimal less-is-more effect. Choice on control trials indicated that the effect did not result from the novelty of two food items versus one. The effect in the less-food-restricted pigeons appears to result from the devaluation of the combination of the food items by the presence of the less-preferred food item. The reversal of the effect under greater food restriction may occur because, as motivation increases, the value of the less-preferred food increases faster than the value of the more-preferred food, thus decreasing the difference in value between the two foods.

Slot machines are among the most popular forms of commercial gambling, and the high frequency of losses that come close to winning – near hits – in this game appears to contribute to its popularity. In the present experiment we tested if pigeons, similarly to humans, prefer an alternative that provides near-hit outcomes in a slot-machine-like task. The pigeons received series of three stimuli, one every two seconds: if the three stimuli matched, food was delivered (a win); if they did not match, food was not delivered (a loss). We gave pigeons a choice between two options that provided food with the same probability but they differed in the sequence of stimuli on loss trials. For the near-hit alternative the non-matching stimulus was the third one (defined as a near hit). For the clear-loss alternative the non-matching stimulus was the second one. We found that the pigeons preferred the clear-loss alternative, that is, they preferred to be given information about the outcome sooner. This result is consistent with prior research on suboptimal choice with pigeons that emphasizes the role of information in choice but is inconsistent with the results of research with humans.

Prior commitment has been found to facilitate choice of a larger later reward (e.g., healthy living) and avoid the impulsive choice of the smaller immediate reward (e.g., smoking, drug taking). In this research with pigeons, we investigated the ephemeral choice task in which pigeons are given a choice between two alternatives, A and B, with similar reinforcement provided for each; however, if they choose A, they can also choose B, whereas if they choose B, A is removed. Thus, choosing A gives them two rewards, whereas choosing B gives them only one. Paradoxically, pigeons actually show a preference for B, the suboptimal alternative. We tested the hypothesis that pigeons made suboptimal choices because they were impulsive. To reduce impulsivity, we required the pigeons to make their initial choice 20 s before receiving the first reward. We found that requiring the pigeons to make a prior commitment encouraged them to choose optimally. The control group, for which the reward was provided immediately following initial choice, continued to choose suboptimally. The results confirm that requiring animals to make a prior commitment can facilitate the development of optimal choice. Furthermore, they may help explain why, without prior commitment, impulsive species, such as primates and pigeons have difficulty with this task, whereas presumably less impulsive species, such as wrasse fish and under some conditions parrots, are able to choose optimally even without prior commitment.

Recent research has examined a task in which choice of 1 alternative A provides reinforcement and in addition, allows access to alternative B that also provides reinforcement. However, although initial choice of B also provides reinforcement, it does not also allow access to A. Thus, optimal performance would be to always choose A. Curiously, Salwiczek et al. (2012) reported that adult wrasse (cleaner) fish mastered this task within 50 trials, whereas monkeys and apes had great difficulty with it. The authors attributed the species differences to ecological differences in the species foraging experiences. However, Pepperberg and Hartsfield (2014) found that parrots too learned this task. In Experiment 1, using the manual presentation of stimuli, we found that pigeons actually showed a reliable preference for B, the suboptimal alternative. In Experiment 2, we replicated the suboptimal preference using an automated version of the task. We hypothesized that the pigeons may have been basing their preference on the frequency of reinforcement associated with each alternative (initially, all trials ended with choice of B, whereas only half of the trials involved choice of A). In Experiment 3, we tested the hypothesis that the pigeons' preference was influenced by the frequency of reinforcements associated with A and B. Thus, when the pigeon chose A, we replaced B with C, so reinforcement occurred to B only when they chose it first. With this procedure we found that B was no longer preferred over A. Thus, the data supported our hypothesis. (PsycINFO Database Record

Pigeons show suboptimal choice on a gambling-like task similar to that shown by humans. Humans also show a preference for gambles in which there are near hits (losses that come close to winning). In the present research, we asked if pigeons would show a preference for alternatives with near-hit-like trials. In Experiment 1, we included an alternative that presented a near hit, in which a stimulus associated with reinforcement (a presumed conditioned reinforcer) changed to a stimulus associated with the absence of reinforcement (a presumed conditioned inhibitor). The pigeons tended to avoid this alternative. In Experiment 2, we varied the duration of the presumed conditioned reinforcer (2 vs. 8 s) that changed to a presumed conditioned inhibitor (8 vs. 2 s) and found that the longer the conditioned reinforcer was presented, the more the pigeons avoided it. In Experiment 3, the near-hit alternative involved an ambiguous stimulus for 8 s that changed to a presumed conditioned reinforcer (or a presumed conditioned inhibitor) for 2 s, but the pigeons still avoided it. In Experiment 4, we controlled for the duration of the conditioned reinforcer by presenting it first for 2 s followed by the ambiguous stimulus for 8 s. Once again, the pigeons avoided the alternative with the near-hit trials. In all 4 experiments, the pigeons tended to avoid alternatives that provided near-hit-like trials. We concluded that humans may be attracted to near-hit trials because near-hit trials give them the illusion of control, whereas this does not appear to be a factor for pigeons.