Search

This paper reports on a systematic review of artificial intelligence applications in education (AIEd) with a special focus on automated essay scoring (AES) systems. AES systems may provide enormous time-savings, especially for large-scale distance teaching institutions with massive numbers of students, by reducing marking and freeing up teachers’ resources for individual feedback and personal support of distance learners. After an introduction on how AES systems function, a review corpus of published articles between 2007 and 2021 is synthetised to evaluate critical discussions and research trends in AES. Articles in the corpus generally evaluated either the accuracy of AES systems or the experience of users, and include implementation of AES systems in various settings (i.e., higher education, K-12, and large-scale assessments). Despite the opportunities that AES might afford for educational institutions, many questions related to the feasibility and validity of AES systems, their implementation, and the associated ethical issues are still unanswered. The findings of this research provide a solid foundation for this discussion.

Stimuli that provide information about likely future reinforcers tend to shift behavior, provided a reliable relation between the stimulus and the reinforcer can be discriminated. Stimuli that are apparently more reliable exert greater control over behavior. We asked how the subjective value (measured in terms of preference) of reinforcers associated with stimuli influences stimulus control. Five pigeons worked on a concurrent chains procedure in which half of all trials ended in a smaller reinforcer sooner, and the other half in a larger reinforcer later. In Signaled trials, the color and flash duration on the keys in the initial link signaled the outcome of the trial. In Conflicting probe trials, the color and the flash duration signaled conflicting information about the outcome of the trial. Choice in Signaled trials shifted toward the signaled outcome, but was never exclusive. In Conflicting probe trials, control was divided idiosyncratically between the 2 stimulus dimensions, but still favored the outcome with the higher subjective value. Thus, stimulus control depends not only on the perceived reliability of stimuli, but also on the subjective value of the outcome.

When short-term memory is assessed in the delayed matching-to-sample (DMTS) procedure, performance is better when cues signal larger reinforcer magnitudes or higher reinforcer probabilities for correct responding. Previous studies demonstrating signaled-magnitude or signaled-probability effects presented cues for a prolonged period during the sample stimulus and/or retention interval. The present study asked whether a signaled-probability effect would occur with brief post-sample cues that signaled the presence or absence of reinforcement. Five pigeons responded in a DMTS task in which sample stimuli were sometimes followed by a 0.5-s cue signaling that reinforcers would either be available or not available in the current trial, and the retention interval varied from 0.5 s to 20 s. A reliable signaled-probability effect was found when reinforcers were arranged independently and for all correct responses, whereas a smaller, less systematic effect was found when reinforcers were arranged dependently and probabilistically. These findings highlight the importance of reinforcement contingencies and contingency discriminability in remembering, and add to the evidence showing that cues signaling differential reinforcement in DMTS may affect processes during the retention interval and comparison phase, rather than attention to the sample stimulus.

Resurgence is defined as the recurrence of a previously reinforced and then extinguished target response when reducing or eliminating a more recently reinforced alternative response. In experiments with children and pigeons, we evaluated patterns of resurgence across and within sessions through decreases in reinforcer availability by challenging alternative responding with extinction and progressive-ratio schedules. In Phase 1, we reinforced only target responding. In Phase 2, we extinguished target responding while reinforcing an alternative response. Finally, Phase 3 assessed resurgence by (a) extinguishing alternative responding versus (b) introducing a progressive-ratio schedule of reinforcement for alternative responding. In both children and pigeons, resurgence of target responding occurred in both conditions but generally was greater when assessed during extinction than with progressive ratios. Importantly, within-session patterns of resurgence did not differ between testing with progressive ratios and extinction. Resurgence with progressive ratios tended to be greater with longer durations between reinforcers but we observed similar findings with only simulated reinforcers during extinction testing. Therefore, the present investigation reveals that the events contributing to instances of resurgence remain to be understood, and presents an approach from which to examine variables influencing within-session patterns of resurgence.

In two laboratory experiments, we examined whether stimuli paired with alternative reinforcers could mitigate resurgence of a previously reinforced target response with pigeons (Experiment 1) and children diagnosed with Autism Spectrum Disorder (Experiment 2). In Phase 1, we arranged food reinforcement according to a variable-ratio schedule for engaging in a target response. In Phase 2, we arranged extinction for target responding and differentially reinforced alternative responding according to a fixed-ratio schedule, with every alternative-reinforcer delivery paired with a change in keylight color (Experiment 1) or automated verbal (praise) statement (Experiment 2). In Phase 3, we assessed resurgence during extinction of target and alternative responding in the presence versus absence of continued presentation of the paired stimulus. Despite variation across sessions, resurgence on average was lower when continuing to present the paired stimuli in all pigeons and children while maintenance of alternative responding did not differ between assessments. These findings indicate that stimuli paired with alternative reinforcement can modestly decrease resurgence, but further examination of their efficacy and a better understanding of the underlying processes are necessary before they can be recommended for clinical use in reducing resurgence of clinically relevant problem behavior.

Differential-reinforcement treatments reduce target problem behavior in the short term but at the expense of making it more persistent long term. Basic and translational research based on behavioral momentum theory suggests that combining features of stimuli governing an alternative response with the stimuli governing target responding could make target responding less persistent. However, changes to the alternative stimulus context when combining alternative and target stimuli could diminish the effectiveness of the alternative stimulus in reducing target responding. In an animal model with pigeons, the present study reinforced responding in the presence of target and alternative stimuli. When combining the alternative and target stimuli during extinction, we altered the alternative stimulus through changes in line orientation. We found that (1) combining alternative and target stimuli in extinction more effectively decreased target responding than presenting the target stimulus on its own; (2) combining these stimuli was more effective in decreasing target responding trained with lower reinforcement rates; and (3) changing the alternative stimulus reduced its effectiveness when it was combined with the target stimulus. Therefore, changing alternative stimuli (e.g., therapist, clinical setting) during behavioral treatments that combine alternative and target stimuli could reduce the effectiveness of those treatments in disrupting problem behavior.

The baseline rate of a reinforced target response decreases with the availability of response-independent sources of alternative reinforcement; however, resistance to disruption and relapse increases. Because many behavioral treatments for problem behavior include response-dependent reinforcement of alternative behavior, the present study assessed whether response-dependent alternative reinforcement also decreases baseline response rates but increases resistance to extinction and relapse. We reinforced target responding at equal rates across two components of a multiple schedule with pigeons. We compared resistance to extinction and relapse via reinstatement of (1) a target response trained concurrently with a reinforced alternative response in one component with (2) a target response trained either concurrently or in separate components from the alternative response across conditions. Target response rates trained alone in baseline were higher but resistance to extinction and relapse via reinstatement tests were greater after training concurrently with the alternative response. In another assessment, training target and alternative responding together, but separating them during extinction and reinstatement tests, produced equal resistance to extinction and relapse. Together, these findings are consistent with behavioral momentum theory-operant response-reinforcer relations determined baseline response rates but Pavlovian stimulus-reinforcer relations established during training determined resistance to extinction and relapse. These findings imply that reinforcing alternative behavior to treat problem behavior could initially reduce rates but increase persistence.

It is widely assumed that reinforcers are biologically relevant stimuli, or stimuli that have been associated with biologically relevant stimuli. However, brief, arbitrary stimuli have also been reported to have reinforcement-like effects, despite being unrelated to biologically relevant stimuli like food. The present study explored the potential reinforcement-like effects of brief stimuli across 5 experiments. In Experiments 1 through 4, pigeon subjects responded for food reinforcement and brief stimulus presentations in a 2-component multiple schedule. Neither baseline response rates nor resistance to change during disruption tests were systematically greater in a component with versus without brief stimulus presentations. Increasing the rate and duration of brief stimulus presentations in Experiment 4 did not reveal reinforcement-like effects when compared directly with food. In Experiment 5, pigeons chose between independent terminal links in a concurrent-chains procedure. Across conditions, varying the location, duration, and rate of brief stimulus presentations in the terminal links had no systematic effects on preference. In contrast, varying rates of food reinforcers resulted in large and reliable shifts in preference. Therefore, the present study found no systematic evidence that brief stimuli unrelated to food reliably increase response rates, resistance to change, or preference. These data demonstrate the value of systematic replication, and a behavioral momentum approach to assessing potential reinforcement-like effects.

Resurgence is the recurrence of a previously reinforced and then extinguished behavior induced by the extinction of another more recently reinforced behavior. Resurgence provides insight into behavioral processes relevant to treatment relapse of a range of problem behaviors. Resurgence is typically studied across three phases: (1) reinforcement of a target response, (2) extinction of the target and concurrent reinforcement of an alternative response, and (3) extinction of the alternative response, resulting in the recurrence of target responding. Because each phase typically occurs successively and spans multiple sessions, extended time frames separate the training and resurgence of target responding. This study assessed resurgence more dynamically and throughout ongoing training in 6 pigeons. Baseline entailed 50-s trials of a free-operant psychophysical procedure, resembling Phases 1 and 2 of typical resurgence procedures. During the first 25 s, we reinforced target (left-key) responding but not alternative (right-key) responding; contingencies reversed during the second 25 s. Target and alternative responding followed the baseline reinforcement contingencies, with alternative responding replacing target responding across the 50 s. We observed resurgence of target responding during signaled and unsignaled probes that extended trial durations an additional 100 s in extinction. Furthermore, resurgence was greater and/or sooner when probes were signaled, suggesting an important role of discriminating transitions to extinction in resurgence. The data were well described by an extension of a stimulus-control model of discrimination that assumes resurgence is the result of generalization of obtained reinforcers across space and time. Therefore, the present findings introduce novel methods and quantitative analyses for assessing behavioral processes underlying resurgence.

When reinforcer availability on one alternative of a concurrent schedule is signaled by a discriminative stimulus, responding on that alternative decreases. We investigated how the correlation between signal presentation and reinforcement (signal reliability) affects choice between signaled and unsignaled alternatives. Six pigeons responded in a concurrent schedule, in which reinforcers on one alternative were signaled by a key-color change. Across conditions, the probability of reinforcement following signal presentation varied (the probability in its absence was the complement). As signal reliability increased, response rates and latencies following signal onset on the signaled alternative decreased, whereas responding on the unsignaled alternative remained unchanged. Because the signal did not alter overall reinforcer rates, these findings are consistent with matching theories and research suggesting that responding on one alternative of a concurrent schedule depends on reinforcer, but not response, rates on other alternatives. However, these findings are inconsistent with others demonstrating concomitant changes in responding on signaled and unsignaled alternatives. We consider whether a response-competition account of concurrent performance can explain these discrepancies, and suggest avenues for future studies to investigate the mechanisms underlying effects of signaled reinforcement in concurrent schedules.

Resurgence is defined as the recurrence of a previously reinforced and then extinguished target response when reducing or eliminating a more recently reinforced alternative response. In experiments with children and pigeons, we evaluated patterns of resurgence across and within sessions through decreases in reinforcer availability by challenging alternative responding with extinction and progressive-ratio schedules. In Phase 1, we reinforced only target responding. In Phase 2, we extinguished target responding while reinforcing an alternative response. Finally, Phase 3 assessed resurgence by (a) extinguishing alternative responding versus (b) introducing a progressive-ratio schedule of reinforcement for alternative responding. In both children and pigeons, resurgence of target responding occurred in both conditions but generally was greater when assessed during extinction than with progressive ratios. Importantly, within-session patterns of resurgence did not differ between testing with progressive ratios and extinction. Resurgence with progressive ratios tended to be greater with longer durations between reinforcers but we observed similar findings with only simulated reinforcers during extinction testing. Therefore, the present investigation reveals that the events contributing to instances of resurgence remain to be understood, and presents an approach from which to examine variables influencing within-session patterns of resurgence.

Reinforcing an alternative response in the presence of the stimuli governing a target response increases resistance to extinction of target responding, relative to training target responding on its own. Conversely, training alternative and target responses in the presence of different stimuli and combining those stimuli only decreases resistance to extinction of target responding, relative to target responding on its own. The present study assessed how different methods of combining discriminative stimuli influence resistance to extinction of responding in pigeons. As in previous studies, combining stimuli across different keys only decreased resistance to extinction of target responding relative to target responding on its own. In comparison, combining stimuli on the same key initially increased resistance to extinction of target responding, but repeated tests resulted in similar levels of responding as target responding with stimuli combined on separate keys. Moreover, greater overall reinforcement rates produced greater resistance to extinction with both methods of combining stimuli, consistent with behavioral momentum theory. These findings reveal several behavioral processes influence the outcome of combining stimuli--including perceptual processes, discriminative control by contingencies, response competition, and behavioral momentum.

Differential-reinforcement treatments reduce target problem behavior in the short term but at the expense of making it more persistent long term. Basic and translational research based on behavioral momentum theory suggests that combining features of stimuli governing an alternative response with the stimuli governing target responding could make target responding less persistent. However, changes to the alternative stimulus context when combining alternative and target stimuli could diminish the effectiveness of the alternative stimulus in reducing target responding. In an animal model with pigeons, the present study reinforced responding in the presence of target and alternative stimuli. When combining the alternative and target stimuli during extinction, we altered the alternative stimulus through changes in line orientation. We found that (1) combining alternative and target stimuli in extinction more effectively decreased target responding than presenting the target stimulus on its own; (2) combining these stimuli was more effective in decreasing target responding trained with lower reinforcement rates; and (3) changing the alternative stimulus reduced its effectiveness when it was combined with the target stimulus. Therefore, changing alternative stimuli (e.g., therapist, clinical setting) during behavioral treatments that combine alternative and target stimuli could reduce the effectiveness of those treatments in disrupting problem behavior.

Greater rates of intermittent reinforcement in the presence of discriminative stimuli generally produce greater resistance to extinction, consistent with predictions of behavioral momentum theory. Other studies reveal more rapid extinction with higher rates of reinforcers - the partial reinforcement extinction effect. Further, repeated extinction often produces more rapid decreases in operant responding due to learning a discrimination between training and extinction contingencies. The present study examined extinction repeatedly with training with different rates of intermittent reinforcement in a multiple schedule. We assessed whether repeated extinction would reverse the pattern of greater resistance to extinction with greater reinforcer rates. Counter to this prediction, resistance to extinction was consistently greater across twelve assessments of training followed by six successive sessions of extinction. Moreover, patterns of responding during extinction resembled those observed during satiation tests, which should not alter discrimination processes with repeated testing. These findings join others suggesting operant responding in extinction can be durable across repeated tests.

The baseline rate of a reinforced target response decreases with the availability of response-independent sources of alternative reinforcement; however, resistance to disruption and relapse increases. Because many behavioral treatments for problem behavior include response-dependent reinforcement of alternative behavior, the present study assessed whether response-dependent alternative reinforcement also decreases baseline response rates but increases resistance to extinction and relapse. We reinforced target responding at equal rates across two components of a multiple schedule with pigeons. We compared resistance to extinction and relapse via reinstatement of (1) a target response trained concurrently with a reinforced alternative response in one component with (2) a target response trained either concurrently or in separate components from the alternative response across conditions. Target response rates trained alone in baseline were higher but resistance to extinction and relapse via reinstatement tests were greater after training concurrently with the alternative response. In another assessment, training target and alternative responding together, but separating them during extinction and reinstatement tests, produced equal resistance to extinction and relapse. Together, these findings are consistent with behavioral momentum theory-operant response-reinforcer relations determined baseline response rates but Pavlovian stimulus-reinforcer relations established during training determined resistance to extinction and relapse. These findings imply that reinforcing alternative behavior to treat problem behavior could initially reduce rates but increase persistence.

Reinforcing an alternative response in the same context as a target response reduces the rate of occurrence but increases the persistence of that target response. Applied researchers who use such techniques to decrease the rate of a target problem behavior risk inadvertently increasing the persistence of the same problem behavior. Behavioral momentum theory asserts that the increased persistence is a function of the alternative reinforcement enhancing the Pavlovian relation between the target stimulus context and reinforcement. A method showing promise for reducing the persistence-enhancing effects of alternative reinforcement is to train the alternative response in a separate stimulus context before combining with the target stimulus in extinction. The present study replicated previous findings using pigeons by showing that combining an “alternative” richer VI schedule (96 reinforcers/hr) with a “target” leaner VI schedule (24 reinforcers/hr) reduced resistance to extinction of target responding compared with concurrent training of the alternative and target responses (totaling 120 reinforcers/hr). We also found less relapse with a reinstatement procedure following extinction with separate-context training, supporting previous findings that training conditions similarly influence both resistance to extinction and relapse. Finally, combining the alternative stimulus context was less disruptive to target responding previously trained in the concurrent schedule, relative to combining with the target response trained alone. Overall, the present findings suggest the technique of combining stimulus contexts associated with alternative responses with those associated with target responses disrupts target responding. Furthermore, the effectiveness of this disruption is a function of training context of reinforcement for target responding, consistent with assertions of behavioral momentum theory.

Differential reinforcement of alternative behavior (DRA) is a treatment designed to eliminate problem behavior by reinforcing an alternative behavior at a higher rate. Availability of alternative reinforcement may be signaled, as with Functional Communication Training, or unsignaled. Whether or not alternative reinforcement is signaled could influence both the rate and persistence of problem behavior. The present study investigated whether signaling the availability of alternative reinforcement affects the rate and persistence of a concurrently available target response with pigeons. Three components of a multiple concurrent schedule arranged equal reinforcement rates for target responding. Two of the components also arranged equal reinforcement rates for an alternative response. In one DRA component, a discrete stimulus signaled the availability of response-contingent alternative reinforcement by changing the keylight color upon reinforcement availability. In the other DRA component, availability of alternative reinforcement was not signaled. Target responding was most persistent in the unsignaled DRA component when disrupted by satiation, free food presented between components, and extinction, relative to the signaled DRA and control components. These findings suggest the discrete stimulus functionally separated the availability of alternative reinforcement from the discriminative stimuli governing target responding. These findings provide a novel avenue to explore in translational research assessing whether signaling the availability of alternative reinforcement with DRA treatments reduces the persistence of problem behavior.

Studies of behavioral momentum reveal that reinforcing an alternative response in the presence of a target response reduces the rate of target responding but increases its persistence, relative to training the target response on its own. Because of the parallels between these studies and differential-reinforcement techniques to reduce problem behavior in clinical settings, alternative techniques to reduce problem behavior without enhancing its persistence are being explored. One potential solution is to train an alternative response in a separate stimulus context from problem behavior before combining the alternative stimulus with the target stimulus. The present study assessed how differences in reinforcement contingencies and rate for alternative responding influenced resistance to extinction of target responding when combining alternative and target stimuli in pigeons. Across three experiments, alternative stimuli signaling a response-reinforcer dependency and greater reinforcer rates more effectively decreased the persistence of target responding when combining alternative and target stimuli within the same extinction tests, but not when compared across separate extinction tests. Overall, these findings reveal that differences in competition between alternative and target responding produced by contingencies of alternative reinforcement could influence the effectiveness of treating problem behavior through combining stimulus contexts.

Reinforcing an alternative response in the presence of the stimuli governing a target response increases resistance to extinction of target responding, relative to training target responding on its own. Conversely, training alternative and target responses in the presence of different stimuli and combining those stimuli only decreases resistance to extinction of target responding, relative to target responding on its own. The present study assessed how different methods of combining discriminative stimuli influence resistance to extinction of responding in pigeons. As in previous studies, combining stimuli across different keys only decreased resistance to extinction of target responding relative to target responding on its own. In comparison, combining stimuli on the same key initially increased resistance to extinction of target responding, but repeated tests resulted in similar levels of responding as target responding with stimuli combined on separate keys. Moreover, greater overall reinforcement rates produced greater resistance to extinction with both methods of combining stimuli, consistent with behavioral momentum theory. These findings reveal several behavioral processes influence the outcome of combining stimuli--including perceptual processes, discriminative control by contingencies, response competition, and behavioral momentum.