Search

Lehrkräfte sind in einem herausfordernden Beruf tätig, der sehr zufriedenstellend, aber auch belastend sein kann. Vor dem Hintergrund des komplexen und spezifischen beruflichen Anforderungsprofils von Lehrkräften erweist sich die Methode der Kollegialen Fallberatung als geeignetes Instrument zur Professionalisierung sowie zur Gesundheitsförderung im Kontext Schule. Kollegiale Fallberatung kann hinsichtlich der Reflexion, Bearbeitung und Bewältigung beruflicher Handlungsprobleme entlastend und impulsgebend wirken. Auch aufgrund ihrer einfachen Erlern- und Handhabbarkeit erfährt die Kollegiale Fallberatung zunehmend Bedeutung im Lehramtsstudium. Bislang liegen jedoch nur wenige Daten zur Effektivität Kollegialer Fallberatung in den verschiedenen Phasen der Lehrkräftebildung vor. Insbesondere mangelt es an Daten, die Aufschluss über die Prozess- und Ergebnisqualität geben können. An dieses Desiderat anknüpfend kontrastiert der vorliegende Beitrag die Wahrnehmung von Lehramtsstudierenden bezüglich der beruflichen Nutzbarkeit (Selbsteinschätzung via Fragebogen) mit der tatsächlichen Qualität Kollegialer Fallberatung (Fremdeinschätzung via Beobachtung). Dafür wurden im Rahmen eines Kompakttrainings „Gesunde Schule“ an der Friedrich-Schiller-Universität Jena drei Kollegiale Fallberatungen durchgeführt und videografiert. Die Fragebogendaten zeigen, dass die Studierenden den kollegialen Austausch und die gemeinsame Reflexion pädagogischer Handlungsprobleme sowie die Atmosphäre in den Kollegialen Fallberatungen als positiv bewerten. Aus Sicht der Fallgeber*innen schaffen die durchgeführten Kollegialen Fallberatungen jedoch nur in eingeschränkter Weise Sicherheit für zukünftige Situationen. Die Analyse der Videografien mittels Direct Behavior Rating zeigt, dass insbesondere die Qualität der Hypothesen unzureichend ist, um dem Anspruch der multiperspektivischen und bedarfsgerechten Auseinandersetzung mit einem Praxisproblem gerecht werden zu können. Zudem werden Rollendiffusionen, v.a. bei den Moderator*innen, sichtbar. Die Ergebnisse werden vor dem Hintergrund zukünftiger Einsatzmöglichkeiten Kollegialer Fallberatung und damit verbundener Trainingsbedarfe im Lehramtsstudium diskutiert. Abschließend werden mögliche Forschungsperspektiven skizziert.

Direct Behavior Rating (DBR) has been proposed as a promising approach to assess student behavioral progress in classroom settings. The current study examines raters and occasions as well as their interactions as sources of error in the use of DBR–Multi-Item Scales (DBR-MIS) and DBR–Single-Item Scales (DBR-SIS) for academically engaged (AE) and disruptive behavior (DB). Furthermore, the stability of scores across two school subjects (i.e., German language, mathematics) was examined. A total of 20 students and two teachers in an inclusive elementary school classroom participated in the study. Generalizability study results suggest that little variance in DBR scores was attributable to the facets of raters or occasions, but the interactions between persons with raters and occasions accounted for a large portion of variance. Variance attributable to these interactions was slightly higher for DBR-MIS than for DBR-SIS ratings. Decision studies revealed that dependable measurements of AE could be achieved within 1.5 weeks of daily ratings for absolute decision-making. Findings regarding DB were mixed. Differences across both school subjects were found indicating that DBR ratings that are obtained in one academic setting may not necessarily be exchangeable with ratings obtained in another. Overall, the results demonstrate the utility of DBR for behavioral progress-monitoring purposes. [ABSTRACT FROM AUTHOR]

Direct Behavior Rating (DBR) is a tool designed for the assessment of behavioral changes over time. Unlike methods for summative evaluations, the development of progress monitoring tools requires evaluation of sensitivity to change. The present study aimed to evaluate this psychometric feature of five newly developed DBR Multi-Item Scales (DBR-MIS). Teachers identified students with behaviors interfering with their learning or the learning of others and implemented a Daily Report Card (DRC) intervention in the classroom settings for two months. The analyses were performed on 31 AB single case studies. Change metrics were calculated at an individual level by using Tau-U A vs. B + trend B and Hedges' g and at a scale-level by using Mixed Effect Meta-Analysis, Hierarchical Linear Models (HLMs), and Between-Case Standardized Mean Difference (BC-SMD). HLMs were estimated considering both fixed and random effects of intervention and linear trend within the intervention phase. The results supported sensitivity to change for three DBR-MIS (i.e., Academic Engagement, Organizational Skills, and Disruptive Behavior), and the relative magnitudes were consistent across the metrics. Sensitivity to change of DBR-MIS Interpersonal Skills received moderate support. Conversely, empirical evidence was not provided for sensitivity to change of DBR-MIS Oppositional Behavior. Particular emphasis was placed on the intervention trend in that responses to behavioral interventions might occur gradually or require consistency over time in order to be observed by raters. Implications for the use of the new DBR-MIS in the context of progress monitoring of social-emotional behaviors are discussed. [ABSTRACT FROM AUTHOR]

The primary goal of this study was to assess the treatment sensitivity of four newly developed Direct Behavior Rating–Multi-Item Scales (DBR-MIS) that assess the domains of academic engagement, disruptive behavior, organizational skills, and oppositional behavior in the context of a Daily Report Card (DRC) intervention. To achieve this goal, we first evaluated the integrity and effectiveness of the DRC intervention in this sample. Participants included six elementary school teachers, each of whom delivered a DRC intervention with one student from their classroom, while completing DBR-MIS ratings on a daily basis for 2 months. Results confirmed the effectiveness of the DRC intervention (all DRC target behaviors demonstrated improvement, with at least half demonstrating improvement that was moderate to large in magnitude) and revealed a positive relationship between DRC implementation integrity and student outcomes. We found strong evidence for the treatment sensitivity of the DBR-MIS assessing academic engagement, disruptive behavior, and organizational skills. Results for the treatment sensitivity of the DBR-MIS oppositional scale were inconclusive. Implications for progress monitoring using the recently developed DBR-MIS are discussed. [ABSTRACT FROM AUTHOR]

Although there is a growing body of evidence to support the use of direct behavior rating (DBR) as a formative behavioral assessment method in school-based problem-solving models, this work has centered largely on the assessment of attending and problem behaviors and on the use of single-item DBR scales. The primary purpose of the current study was to report on the development and initial validation of teacher-completed multi-item DBR scales (DBR-MIS) designed to expand upon prior DBR research to assess three constructs representing behaviors that are widely considered to support student academic achievement (academic engagement, interpersonal skills, and study skills), only one of which (academic engagement) has been the target of DBR assessment methods. Development of the scales involved a content validity study utilizing feedback from a panel of consumers (teachers, parents, and school administrators) and a panel of researchers with expertise in school-based behavioral assessment. Results of an exploratory factor analysis of ratings of students completed by teachers in Grades K through 3 (N = 307) supported a one-factor solution for each of the aforementioned constructs. Favorable internal consistency was found for each scale. [ABSTRACT FROM AUTHOR]

Due to their promise as a feasible tool for evaluating the effects of school-based interventions, Direct Behavior Ratings (DBR) have received much research attention over the past 2 decades. Although DBR methodology has demonstrated much promise, favorable psychometric characteristics only have been demonstrated for tools measuring a small number of constructs. Likewise, although a variety of methods of DBR have been proposed, most extant studies have focused on the use of single-item methods. The present study examined the dependability of four methods of formative behavioral assessment (i.e., single-item and multi-item ratings administered either daily [DBR] or weekly [formative behavior rating measures or FBRM]) across eight psychological constructs (i.e., interpersonal skills, academic engagement, organizational skills, disruptive behavior, oppositional behavior, interpersonal conflict, anxious depressed, and social withdrawal). School-based professionals (N = 91; i.e., teachers, paraprofessionals, and intervention specialists) each rated one student across all eight constructs after being assigned to one of the four assessment conditions. Dependability estimates varied substantially across methods and constructs (range = 0.75–0.96), although findings of the present study support the use of the broad set of formative assessment tools evaluated. [ABSTRACT FROM AUTHOR]

Reliable and valid data form the foundation for evidence-based practices, yet surprisingly few studies on school-based behavioral assessments have been conducted which implemented one of the most fundamental approaches to construct validation, the multitrait-multimethod matrix (MTMM). To this end, the current study examined the reliability and validity of data derived from three commonly utilized school-based behavioral assessment methods: Direct Behavior Rating – Single Item Scales, systematic direct observations, and behavior rating scales on three common constructs of interest: academically engaged, disruptive, and respectful behavior. Further, this study included data from different sources including student self-report, teacher report, and external observers. A total of 831 students in grades 3–8 and 129 teachers served as participants. Data were analyzed using bivariate correlations of the MTMM, as well as single and multi-level structural equation modeling. Results suggested the presence of strong methods effects for all the assessment methods utilized, as well as significant relations between constructs of interest. Implications for practice and future research are discussed. [ABSTRACT FROM AUTHOR]

Direct behavior ratings have been identified as a practical and feasible alternative to direct observation of behavior for monitoring behavioral progress. Despite the evidence of usability, there have been calls for further examination of direct behavior ratings using different behaviors and scales. To this end, we examined the ratings of schoolwide behavioral expectations by teachers and outside observers, as well as direct observation of academic engagement by outside observers in upper elementary school language arts classrooms. Across sessions, correlations between raters (range r = .82–.91) and between measures (range r = .64–.98) were moderate to high; significant differences between raters existed when rating students identified with negative behavior (n = 12) as opposed to positive behavior (n = 12). Findings are discussed in terms of implications for research and practice, limitations of the study, and the necessity for further research and development. [ABSTRACT FROM AUTHOR]

This study examined the treatment sensitivity of Direct Behavior Rating–Single Item Scales (DBR-SIS) in response to an evidence-based intervention delivered in a single-case, multiple-baseline design. DBR-SIS was used as a formative assessment in conjunction with a frequently used intervention in schools, a Daily Behavior Report Card (DRC). The intervention and concurrent assessment were conducted by five teachers in a rural Midwestern elementary school with five male students displaying mild to moderate behavioral challenges in the classroom. Study findings indicated DBR-SIS displays appropriate treatment sensitivity following intervention implementation. Agreement in the documentation of response and nonresponse to intervention implementation between DBR-SIS and systematic direct observation (SDO) data was evident across visual and empirical analyses. In addition, through a multiple-baseline design, this study documented negligible to no change in student behavior following implementation of a DRC in an applied classroom setting. These findings support previous calls for continued examination of the forms and components of DRC employed in schools. Finally, the study found educators rated the use of a combined DRC intervention and progress monitoring with DBR-SIS as favorable. [ABSTRACT FROM AUTHOR]

Direct behavior rating (DBR) represents a feasible method for monitoring student behavior in the classroom; however, limited work to date has focused on the use of multi-item scales. The purposes of the study were to examine the (a) dependability of data obtained from a multi-item DBR designed to assess peer conflict and (b) treatment sensitivity of Direct Behavior Rating Multi-Item Scales (DBR-MIS) constructed using factor-derived and individualized methods. Analyses were performed using teacher ratings of 65 students (53 boys, 12 girls) between 6 and 12 years old. Results of decision studies indicated that an acceptable criterion of dependability (ϕ > .70) for low-stakes, intraindividual decision making could be achieved using a three-item scale across eight occasions, a four- or five-item scale across four occasions, or a six-item scale across three occasions. Subsequent analyses verified that a six-item DBR demonstrated acceptable treatment sensitivity when ratings were conducted on 3 days during baseline and 3 days during treatment with methylphenidate. Implications for practice and future research are discussed. [ABSTRACT FROM AUTHOR]

Direct Behavior Rating (DBR) as a behavioral progress monitoring tool can be designed as longitudinal assessment with only short intervals between measurement points. The reliability of these instruments has been mostly evaluated in observational studies with small samples based on generalizability theory. However, for a standardized use in the pedagogical field, a larger and broader sample is required in order to assess measurement invariance between different participant groups and over time. Therefore, we constructed a DBR, the Questionnaire for Monitoring Behavior in Schools (QMBS) with multiple items to measure the occurrence of specific externalizing and internalizing student classroom behaviors on a Likert scale (1 = never to 7 = always). In a pilot study, two trained raters observed 16 primary education students and rated the student behavior over all items with a satisfactory reliability. In the main study, 108 regular primary school students, 97 regular secondary students, and 14 students in a clinical setting were rated daily over one week (five measurement points). Item response theory (IRT) analyses confirmed the technical adequacy of the instrument and latent growth models demonstrated the instrument’s stability over time. Further development of the instrument and study designs to implement DBRs is discussed.

Direct behavior ratings (DBRs) have been proposed as an efficient method to assess student behavior in the classroom due to their relative ease of administration compared to alternative methods like systematic direct observation. DBRs are considered low-inference assessments of behavior because they are designed to be completed immediately following a specified observation period of student behavior; however, in practice it is common for teachers and other respondents to delay completion of a DBR until they are reminded to do so. It is unclear what effect, if any, this latency between observation and DBR completion has on rater accuracy. Thus, the purpose of this study was to examine the effect of completion latency on accuracy in an analogue setting. Two-hundred forty-one undergraduate students (83.8% female) with a mean age of 21 participated across eight groups and were asked to complete an electronic DBR immediately after watching a video of student behavior or after a predetermined delay of 5 minutes, 15 minutes, 30 minutes, 1 hour, 2 hours, 4 hours, or 6 hours. A one-way analysis of variance revealed that there was no statistically significant relationship between completion latency and DBR accuracy, F(7, 233) = .959, p = .46, η2 = .028. [ABSTRACT FROM AUTHOR]

Accurate and rapid identification of students displaying behavioral problems requires instrumentation that is user friendly and reliable. The purpose of the study was to evaluate a multi-item direct behavior rating scale called the Direct Behavior Rating-Multiple Item Scale (DBR–MIS) for disruptive behavior to determine the number of administrations necessary to achieve a dependable result. In addition, a study was conducted to determine to what degree the DBR-MIS correlated with a brief rating scale of attention deficit hyperactivity disorder behaviors. Participants were Kindergarten students (n= 18) who were observed and rated over nine 5-min observation periods by 2 raters with minimal training. Results indicated that the DBR-MIS provided reliable estimates of disruptive behavior by the seventh or eighth rating occasion. Correlations between the DBR-MIS and the attention deficit hyperactivity disorder rating scale were weak to moderate. Overall, the DBR-MIS for disruptive behavior efficiently achieved dependable results when screening groups of Kindergarten students in a public school setting. [ABSTRACT FROM AUTHOR]

The implementation of multi-tiered systems in schools necessitates the use of screening assessments which produce valid and reliable data to identify students in need of tiered supports. Data derived from these screening assessments may be evaluated according to their classification accuracy, or the degree to which cut scores correctly identify individuals as “at-risk” or “not-at-risk.” The current study examined the performance of mean scores derived from over 1700 students in Grades 1, 2, 4, 5, 7, and 8 using Direct Behavior Rating—Single Item Scales. Students were rated across three time points (Fall, Winter, Spring) by their teachers in three areas: (a) academically engaged behavior, (b) disruptive behavior, and (c) respectful behavior. Classification accuracy indices and comparisons among behaviors were derived using Receiver Operating Characteristic (ROC) curve analyses, partial area under the curve (pAUC) tests, and bootstrapping methods to evaluate the degree to which mean behavior ratings accurately identified students who demonstrated elevated behavioral symptomology on the Behavioral and Emotional Screening System. Results indicated that optimal cut-scores for mean behavior ratings and a composite rating demonstrated high levels of specificity, sensitivity, and negative predictive value, with sensitivity point estimates for optimal cut-scores exceeding .70 for individual behaviors and .75 for composite scores across grade groups and time points. [ABSTRACT FROM AUTHOR]

To date, rater accuracy when using Direct Behavior Rating (DBR) has been evaluated by comparing DBR-derived data to scores yielded through systematic direct observation. The purpose of this study was to evaluate an alternative method for establishing comparison scores using expert-completed DBR alongside best practices in consensus building exercises, to evaluate the accuracy of ratings. Standard procedures for obtaining expert data were established and implemented across two sites. Agreement indices and comparison scores were derived. Findings indicate that the expert consensus building sessions resulted in high agreement between expert raters, lending support for this alternative method for identifying comparison scores for behavioral data. [ABSTRACT FROM AUTHOR]

Multitiered frameworks of service delivery have traditionally underserved students with mental health needs. Whereas research has supported the assessment and intervention of social and academic behavior across tiers, evidence is limited with regard to mental health concerns including internalizing behaviors (e.g., anxiety and depression). In particular, there is a notable shortage of brief anxiety assessment tools to be used for progress monitoring purposes. Moreover, traditional omnibus rating scale approaches may fail to capture contextually dependent anxiety. The purpose of the present investigation is to examine the sensitivity to change and concurrent validity of Direct Behavior Ratings (DBR; Chafouleas, Riley-Tillman, & Christ, 2009; Chafouleas, Riley-Tillman, & Sugai, 2007) of anxiety and traditional rating scales in measuring academic anxiety directly before, during, and after a potentially anxiety provoking stimulus. Research was conducted with 115 undergraduate students in a Southeastern university. Results indicated significant relationships between DBRs and pre- and postmeasures of anxiety. Change metrics suggested an overall lack of correspondence between DBR and the criterion measure, with DBR scales detecting greater change both across the testing situation and participants. The use of DBR for anxiety is considered within a multitiered, problem-solving framework. Feasibility and limitations associated with implementation are discussed. [ABSTRACT FROM AUTHOR]

The purpose of this study was to provide initial evaluation of a web-based training module on rating accuracy when using Direct Behavior Rating (DBR). Components of the training module included (a) an overview familiarizing users with assessing student behavior with this method, (b) modeling that includes frame-of-reference training, and (c) multiple opportunities to practice and receive immediate corrective feedback. Participants included 90 undergraduate students assigned to one of six sessions (three experimental and three control). Rating accuracy served as the outcome measure defined as the difference between the rater score and a comparison derived from an expert DBR or systematic direct observation (SDO) score. Rating targets included academically engaged, disruptive, and respectful behavior. Completion of the DBR training module generally yielded ratings that more closely compared with the scores obtained via DBR experts and SDO, yet specific results were mixed across type of rating (i.e., behavior target and duration) and comparison (i.e., DBR expert and SDO). Limitations, future research directions, and implications for practice are discussed. [ABSTRACT FROM AUTHOR]

Direct Behavior Rating (DBR) is a repeatable and efficient method of behavior assessment that is used to document teacher perceptions of student behavior in the classroom. Time-series data can be graphically plotted and visually analyzed to evaluate patterns of behavior or intervention effects. This study evaluated the decision accuracy of novice raters who were presented with single-phase graphical plots of DBR data. Three behaviors (i.e., academically engaged, disruptive, and respectful) and three graphical trends (i.e., positive, no trend, and negative) were analyzed by 27 graduate and five undergraduate participants who had minimal visual analysis experience. All graphs were unique, with data points arranged to form one of three "true" trends. Raters correctly classified graphs with positive, no, and negative trends an average of 76, 98, and 67% of instances. The generalized linear mixed model was used to handle significance tests for the categorical data. Results indicate that accuracy was influenced by the trend direction, with the most accurate ratings in the no trend condition. Despite the significant effect for trend direction, the current study provides empirical evidence for accuracy of DBR trends and interpretations. Novice raters and visual analysts yielded accurate decisions regarding the trend of plotted data for student behavior. [ABSTRACT FROM AUTHOR]

Abstract: The purpose of this study was to evaluate the utility of Direct Behavior Rating Single Item Scale (DBR–SIS) targets of disruptive, engaged, and respectful behavior within school-based universal screening. Participants included 31 first-, 25 fourth-, and 23 seventh-grade teachers and their 1108 students, sampled from 13 schools across three geographic locations (northeast, southeast, and midwest). Each teacher rated approximately 15 of their students across three measures, including DBR–SIS, the Behavioral and Emotional Screening System (Kamphaus & Reynolds, 2007), and the Student Risk Screening Scale (Drummond, 1994). Moderate to high bivariate correlations and area under the curve statistics supported concurrent validity and diagnostic accuracy of DBR–SIS. Receiver operating characteristic curve analyses indicated that although respectful behavior cut scores recommended for screening remained constant across grade levels, cut scores varied for disruptive behavior and academic engaged behavior. Specific cut scores for first grade included 2 or less for disruptive behavior, 7 or greater for academically engaged behavior, and 9 or greater for respectful behavior. In fourth and seventh grades, cut scores changed to 1 or less for disruptive behavior and 8 or greater for academically engaged behavior, and remained the same for respectful behavior. Findings indicated that disruptive behavior was particularly appropriate for use in screening at first grade, whereas academically engaged behavior was most appropriate at both fourth and seventh grades. Each set of cut scores was associated with acceptable sensitivity (.79–.87), specificity (.71–.82), and negative predictive power (.94–.96), but low positive predictive power (.43–.44). DBR–SIS multiple gating procedures, through which students were only considered at risk overall if they exceeded cut scores on 2 or more DBR–SIS targets, were also determined acceptable in first and seventh grades, as the use of both disruptive behavior and academically engaged behavior in defining risk yielded acceptable conditional probability indices. Overall, the current findings are consistent with previous research, yielding further support for the DBR–SIS as a universal screener. Limitations, implications for practice, and directions for future research are discussed. [Copyright &y& Elsevier]

The purpose of this study was to extend evaluation of rater accuracy using Direct Behavior Rating–Single-Item Scales (DBR-SIS). Extension of prior research was accomplished through use of criterion ratings derived from both systematic direct observation and expert DBR-SIS scores, and also through control of the durations over which the target behavior was displayed in each clip. Rating targets included academically engaged, disruptive, and respectful behavior. Undergraduate participants (n = 113) viewed video clips of classroom instruction and subsequently rated target students’ behavior using DBR-SIS. Results indicated that both rater bias and rater error were present to some degree based on the behavior target, wording, and duration displayed, with greater bias found for the target involving respectful behavior. In addition, positive wording for academically engaged behavior resulted in improved accuracy, yet negative wording resulted in improved accuracy for disruptive and possibly respectful behavior. Implications for research and practice are discussed. [ABSTRACT FROM AUTHOR]

Abstract: The purpose of this study was to investigate how Direct Behavior Rating Single Item Scales (DBR-SIS) involving targets of academically engaged, disruptive, and respectful behaviors function in school-based screening assessment. Participants included 831 students in kindergarten through eighth grades who attended schools in the northeastern United States. Teachers provided behavior ratings for a sample of students in their classrooms on the DBR-SIS, the Behavioral and Emotional Screening System (Kamphaus & Reynolds, 2007), and the Student Risk Screening Scale (Drummond, 1994). Given variations in rating procedures to accommodate scheduling differences across grades, analysis was conducted separately for elementary school and middle school grade levels. Results suggested that the recommended cut scores, the combination of behavior targets, and the resulting conditional probability indices varied depending on grade level grouping (lower elementary, upper elementary, middle). For example, for the lower elementary grade level grouping, a combination of disruptive behavior (cut score=2) and academically engaged behavior (cut score=8) was considered to offer the best balance among indices of diagnostic accuracy, whereas a cut score of 1 for disruptive behavior and 8 for academically engaged behavior were recommended for the upper elementary school grade level grouping and cut scores of 1 and 9, respectively, were suggested for middle school grade level grouping. Generally, DBR-SIS cut scores considered optimal for screening using single or combined targets including academically engaged behavior and disruptive behavior by offering a reasonable balance of indices for sensitivity (.51–.90), specificity (.47–.83), negative predictive power (.94–.98), and positive predictive power (.14–.41). The single target of respectful behavior performed poorly across all grade level groups, and performance of DBR-SIS targets was relatively better in the elementary school than middle school grade level groups. Overall, results supported that disruptive behavior is highly important in evaluating risk status in lower grade levels and that academically engaged behavior becomes more pertinent as students reach higher grade levels. Limitations, future directions, and implications are discussed. [Copyright &y& Elsevier]

Abstract: This study examined the impact of various components of rater training on the accuracy of rating behavior using Direct Behavior Rating-Single Item Scales (DBR-SIS). Specifically, the addition of frame-of-reference and rater error training components to a standard package involving an overview and then modeling, practice, and feedback was investigated. In addition, amount of exposure to the direct training component (i.e., number of practice and feedback opportunities) was evaluated, and the rates at which behavior was displayed were carefully manipulated to control for and evaluate training impact by target and rate of behavior. The sample consisted of undergraduate students assigned to one of 6 possible conditions. Overall findings suggested that completion of a training package did result in enhanced accuracy when using DBR-SIS to rate academic engagement and disruption. However, results also supported that the most comprehensive package of DBR training may not always result in greater improvements over a standard package involving direct training. In general, a more intensive training package appeared beneficial at improving ratings for targets that had previously been difficult to rate accurately (e.g., medium rate disruptive behavior). Limitations and implications for future research are discussed. [Copyright &y& Elsevier]

This study presents an evaluation of the diagnostic accuracy and concurrent validity of Direct Behavior Rating Single Item Scales for use in school-based behavior screening of second-grade students. Results indicated that each behavior target was a moderately to highly accurate predictor of behavioral risk. Optimal universal screening cut scores were also identified for each scale, with results supporting reduced false positive rates through the simultaneous use of multiple scales. [ABSTRACT FROM AUTHOR]

In this study, evaluation of direct behavior rating (DBR) occurred with regard to two primary areas: (a) accuracy of ratings with varied instrumentation (anchoring: proportional or absolute) and procedures (observation length: 5 min, 10 min, or 20 min) and (b) one-week test—retest reliability. Participants viewed video clips of a typical third grade student and then used single-item DBR scales to rate disruptive and academically engaged behavior. Overall, ratings tended to overestimate the actual occurrence of behavior. Although ratings of academic engagement were not affected by duration of the observation, ratings of disruptive behavior were, as the longer the duration, the more the ratings of disruptive behavior were overestimated. In addition, the longer the student was disruptive, the greater the overestimation effect. Results further revealed that anchoring the DBR scale as proportional versus absolute number of minutes did not affect rating accuracy. Finally, test—retest analyses revealed low to moderate consistency across time points for 10-min and 20-min observations, with increased consistency as the number of raters or number of ratings increased (e.g., four 5-min vs. one 20-min). Overall, results contribute to the technical evaluation of DBR as a behavior assessment method and provide preliminary information regarding the influence of duration of an observation period on DBR data. [ABSTRACT FROM AUTHOR]

Abstract: A total of 4 raters, including 2 teachers and 2 research assistants, used Direct Behavior Rating Single Item Scales (DBR-SIS) to measure the academic engagement and disruptive behavior of 7 middle school students across multiple occasions. Generalizability study results for the full model revealed modest to large magnitudes of variance associated with persons (students), occasions of measurement (day), and associated interactions. However, an unexpectedly low proportion of the variance in DBR data was attributable to the facet of rater, as well as a negligible variance component for the facet of rating occasion nested within day (10-min interval within a class period). Results of a reduced model and subsequent decision studies specific to individual rater and rater type (research assistant and teacher) suggested degree of reliability-like estimates differed substantially depending on rater. Overall, findings supported previous recommendations that in the absence of estimates of rater reliability and firm recommendations regarding rater training, ratings obtained from DBR-SIS, and subsequent analyses, be conducted within rater. Additionally, results suggested that when selecting a teacher rater, the person most likely to substantially interact with target students during the specified observation period may be the best choice. [Copyright &y& Elsevier]

High-quality formative assessment data are critical to the successful application of any problem-solving model (e.g., response to intervention). Formative data available for a wide variety of outcomes (academic, behavior) and targets (individual, class, school) facilitate effective decisions about needed intervention supports and responsiveness to those supports. The purpose of the current case study is to provide preliminary examination of direct behavior rating methods in class-wide assessment of engagement. A class-wide intervention is applied in a single-case design (B-A-B-A), and both systematic direct observation and direct behavior rating are used to evaluate effects. Results indicate that class-wide direct behavior rating data are consistent with systematic direct observation across phases, suggesting that in this case study, direct behavior rating data are sensitive to classroom-level intervention effects. Implications for future research are discussed. [ABSTRACT FROM AUTHOR]

In this study, preliminary evidence of the potential for Direct Behavior Rating (DBR) in screening assessment of school social behavior is provided through evaluation of the concurrent validity of DBR with a commonly used criterion measure. The teacher-completed form of the Social Skills Rating System (SSRS) was selected as the criterion measure given similarity to DBR in both broad construct of interest and intended use. Classroom teachers completed DBRs of academic engagement and disruptive behavior twice daily, along with fall and spring completion of the SSRS. Overall results support a moderate to strong association between the methods, with stronger correlations in the fall assessment period. In addition, evidence of teacher acceptability for DBR is provided. Along with specific recommendations for future research, implications are discussed regarding selection and interpretation of behavior assessment methods for screening purposes. [ABSTRACT FROM AUTHOR]

Generalizability theory is used to examine the impact of scaling gradients on a single-item Direct Behavior Rating (DBR). A DBR refers to a type of rating scale used to efficiently record target behavior(s) following an observation occasion. Variance components associated with scale gradients are estimated using a random effects design for persons (p) by raters (r) by occasions (o). Data from 106 undergraduate student participants are used in the analysis. Each participant viewed and rated video clips of six elementary-aged students who were engaged in a difficult task. Participant ratings are collected three times for each of two behaviors within three scale gradient conditions (6-, 10-, 14-point scale). Scale gradient does not substantially contribute to the magnitude of observed score variances. In contrast, the largest proportions of variance are attributed to rater and error across all scale gradient conditions. Implications, limitations, and future research considerations are discussed. [ABSTRACT FROM AUTHOR]