Search

Although originally conceived of as a marriage of direct behavioral observation and indirect behavior rating scales, recent research has indicated that Direct Behavior Ratings (DBRs) are affected by rater idiosyncrasies (rater effects) similar to other indirect forms of behavioral assessment. Most of this research has been conducted using generalizability theory (GT), yet another approach, many-facet Rasch measurement (MFRM), has recently been utilized to illuminate the previously opaque nature of these rater idiosyncrasies. The purpose of this study was to utilize both approaches (GT and MFRM) to consider rater effects with 126 second- through fifth-grade students who were rated on two DBR-Multi-Item Scales by four raters (22 of these ratings were fully crossed). Results indicated the presence of rater effects and revealed nuances about their nature, including showing differences across construct domains, identifying items that are potentially more susceptible to rater effects than others, and isolating specific raters who appear to have been more susceptible to rater effects than other raters. These findings further indicate the indirect nature of DBRs and offer potential avenues for addressing and ameliorating rater effects in research and practice. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Rater-mediated assessments, such as teacher behavior rating scales, measure student behavior indirectly through the lens of a rater. As a result, scores from rater-mediated assessments can be influenced by rater effects— individual differences in rater perspectives, attitudes, beliefs, and interpretation of rating scale items. Rater effects are a fundamental aspect of all rater-mediated assessments. However, traditional approaches to evaluate rater effects (i.e., classical test theory, generalizability theory, and multilevel modeling) merely estimate how much score variability is due to the rater. These approaches, while informative, do not offer a solution to the problem. In contrast, Many-facet Rasch measurement (MFRM) approaches estimate and control for rater effects in rater-mediated assessments so that scores are adjusted to account for rater variability. Thus, MFRM offers unique insights into individual- and group-level rater effects that can be used to inform a solution. The resultant purpose of this paper is to introduce MFRM, discuss its advantages for evaluating rater effects in rater-mediated assessments, and demonstrate its use through an applied example.

The current study examined (a) the associations of COVID-19 stress with anxiety and depression and whether those associations differed by gender or grade level, and (b) how different coping strateg...

Behavior rating scales represent one of the most commonly used types of assessments in school psychology. Yet, they suffer from a fundamental limitation: They are an indirect methodology influenced...

The assessment of culturally and linguistically diverse (CLD) students, including students who are learning English as a second language (English language learners, ELL), is a challenge for many sc...

The Classroom Assessment Scoring System (CLASS; Pianta et al., 2008) is a popular measure of teacher-child interactions. Despite its prominence, CLASS scores have fairly weak relations with various child outcomes (e.g., Zaslow et al., 2010). One potential reason for these findings could be systematic differences in observer severity. As such, the purpose of this study was to explore the scope and impact of rater effects on CLASS scores with a sample of 77 teachers who were rated by 13 observers. Results indicated significant rater effects across all three CLASS domains. Adjusting for these effects, however, did not improve relations between CLASS scores and child outcomes. Implications for the CLASS and related assessments are discussed.

Reading depends on the integration of both cognitive and motivational factors, yet reading motivation has received little attention in the research literature. The purpose of the present study was to evaluate the psychometric properties of the Baylor Revision of the Motivation to Read Survey (B-MRS; Watkins & Browning, 2015) with a sample of 731 students in Grades 6-8. The results of exploratory and confirmatory factor analyses supported the oblique two-factor structure of the B-MRS, which was invariant across gender, and regression analyses indicated that gender and grade were not significant predictors of reading motivation. These results add to the growing body of literature providing preliminary support for the B-MRS. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Emerging evidence indicates that mindfulness-based interventions are effective treatments for children and adolescents with ADHD. However, many clinicians are unaware of the application of mindfuln...

The COVID-19 pandemic and resulting stay-at-home orders created a need for assessing elementary, middle, and high school students’ experienced stressors associated with the coronavirus situation. I...

The State–Trait Inventory of Cognitive and Somatic Anxiety (STICSA) is a commonly administered self-report instrument of state–trait cognitive and somatic anxiety. Extant research has consistently supported the intended oblique two-factor scoring structure for the STICSA. However, this model assumes that population-level data have (or approximate) a simple structure and that item-level variance is unidimensional. These assumptions may not be tenable and have unintended consequences for STICSA subscore interpretation. Consequently, we tested these assumptions by fitting confirmatory and exploratory structural equation models to STICSA scores for a diverse sample of college students enrolled at a large Southwestern university in the United States ( n = 635). Results indicated that cognitive and somatic factors are not equally robust and that STICSA items appear to measure a nonnegligible mixture of both latent cognitive and somatic anxiety. It is recommended that future research use exploratory structural equation model in tandem with CFA to directly model data complexity.

This systematic review examines the literature on the effectiveness of child-centered play therapy (CCPT) for youths who have experienced traumatic events. Two independent reviewers conducted the search procedures, as well as all data extraction and coding. Seven peer-reviewed articles reporting treatment outcomes were included in the review. As the focus of the review was on CCPT, treatment methods were similar across the seven articles (e.g., use of similar materials). There was also some consistency in findings regarding outcomes related to internalizing problems, self-concept, and self-competence. However, there was significant variability in the outcome measures used to evaluate effects, as well as limitations regarding the study methods that impact the overall conclusions regarding the use of CCPT to treat children that have experienced traumatic events. Treatment recommendations and suggestions for future research are discussed.

Individuals with autism spectrum disorder (ASD) are hypothesized to exhibit relative strengths in verbal and non-verbal reasoning and weaknesses in working memory and speed of information processing. The purpose of the present investigation was to determine the degree to which this cognitive profile as measured by the Wechsler Intelligence Scale for Children—Fourth Edition (WISC-IV; Wechsler 2003a) cognitive proficiency index (CPI; measure of working memory and processing speed) and general ability index (GAI; measure of verbal and non-verbal reasoning) could accurately distinguish between a referred sample of 79 school-aged students diagnosed with ASD and two non-clinical comparison groups: (a) 2200 children in the WISC-IV standardization sample and (b) 216 school-aged students referred for psychoeducational testing whose school-based evaluations did not result in a diagnosis. Results indicated that the ASD sample exhibited significantly lower mean scores on the CPI when compared to the two control groups. However, diagnostic utility statistics indicated that a randomly selected participant from the ASD subgroup would exhibit a larger difference between the GAI and CPI than a randomly selected participant from the two control groups 51.9–66.0% of the time. Consequently, the GAI > CPI cognitive score profile exhibits low diagnostic accuracy for individuals with ASD. Psychologists who work in applied settings are cautioned against using group trends to guide decision-making for individual clients.

As a result of the upcoming Federal reauthorization of the Individuals With Disabilities Education Improvement Act (IDEA), practitioners and researchers have begun vigorously debating what constitutes evidence-based assessment for the identification of specific learning disability (SLD). This debate has resulted in strong support for a method that appraises an individual’s profile of cognitive test scores for the purposes of determining cognitive processing strengths and weaknesses, commonly referred to as patterns of strengths and weaknesses (PSW). Following the Fuchs and Deshler model, questions regarding the psychometric and conceptual integrity of the PSW model are addressed. Despite the strong claims made by many PSW proponents, the findings by this review demonstrate the need for additional information to determine whether PSW is a viable alternative to existing eligibility models and worthy for large scale adoption for SLD identification. Implications for public policy and future SLD research are also discussed.

The Culture-Language Interpretive Matrix (C-LIM) was developed to help practitioners determine the validity of test scores obtained from students who are culturally and linguistically different from the normative group of a test. The present study used an idiographic approach to investigate the diagnostic utility of the C-LIM for the Wechsler Intelligence Scales for Children-Fourth Edition (WISC-IV) for distinguishing between a referred sample of 69 English language learners, 79 English-speakers diagnosed with Autism, and 216 English speakers referred for a special education evaluation. Results indicated that the WISC-IV C-LIM differentiated between these groups of students at chance rates. Evidence from the previous studies along with the results of the current study does not support the use of the C-LIM for making decisions about individuals in applied practice.

The Culture-Language Interpretive Matrix (C-LIM) was developed by Flanagan, Ortiz, and Alfonso (2013) to evaluate the extent to which developmental language proficiency and acculturative learning o...

The purpose of the present investigation was to conduct a meta-analysis of the literature on examiner errors for the Wechsler scales of intelligence. Results indicate that a mean of 99.7% of protocols contained at least 1 examiner error when studies that included a failure to record examinee responses as an error were combined and a mean of 41.2% of protocols contained at least 1 examiner error when studies that ignored errors of omission were combined. Furthermore, graduate student examiners were significantly more likely to make at least 1 error on Wechsler intelligence test protocols than psychologists. However, psychologists made significantly more errors per protocol than graduate student examiners regardless of the inclusion or exclusion of failure to record examinee responses as errors. On average, 73.1% of Full-Scale IQ (FSIQ) scores changed as a result of examiner errors, whereas 15.8%-77.3% of scores on the Verbal Comprehension Index (VCI), Perceptual Reasoning Index (PRI), Working Memory Index (WMI), and Processing Speed Index changed as a result of examiner errors. In addition, results suggest that examiners tend to overestimate FSIQ scores and underestimate VCI scores. However, no strong pattern emerged for the PRI and WMI. It can be concluded that examiner errors occur frequently and impact index and FSIQ scores. Consequently, current estimates for the standard error of measurement of popular IQ tests may not adequately capture the variance due to the examiner.

The structural validity of the Wechsler Intelligence Scale for Children–Fourth Edition (WISC-IV) was evaluated using confirmatory factor analysis for a clinical sample of 1,537 students diagnosed with specific learning disabilities (SLD) by school psychologists in two large southwestern school districts. Results indicated that a bifactor model consisting of four first-order domain specific factors and a general intelligence breadth factor fit the data best. Consequently, the structural validity of the WISC-IV for students with SLD was supported by the results of the present study. The general intelligence factor contributed the most information, accounting for 48% of the common variance. Given this structure, it was recommended that score interpretation should emphasize the Full-Scale IQ score because of the marginal contributions of the first-order domain-specific factors and their low precision of measurement independent of the general factor.