Search

The present study examined the structure of the NEPSY-II within the norming sample using exploratory factor analysis. For the 3–4-year-old group, our results were conceptually uninterpretable. As a result, a unidimensional model was retained by default as a remedy to local fit issues. For the 7–12-year-old group, our analysis supported some aspects of the NEPSY-II conceptual domains in the form of a six-factor model that yielded the best fit to the data. While variance partitioning results indicate that the majority of NEPSY-II subtests at ages 7–12 contain adequate specificity to be interpreted in isolation, caution is suggested for interpreting the Social Perception subtests; in particular, given the inability to locate that latent dimension in either of the analyses conducted. Implications for the clinical interpretation of the instrument moving forward are discussed.

AbstractAlthough the field of school psychology has made progress toward the use of tests and assessment practices with empirical support over the past 20 years, many school psychology practitioners still engage in what can be described as low-value value assessment practices that lack compelling scientific support potentially taking time and resources away from practices that have a demonstrated evidence-base. Why do school psychologists engage in questionable assessment and interpretive practices despite decades of discrediting scientific evidence? This article critically examines several plausible explanations for the perpetuation of low-value practices in school psychology assessment. It also underscores the importance of critical thinking when evaluating assessment and interpretation practices, and discusses practical recommendations to assist in advancing evidence-based assessment in school psychology training and practice as the field progresses well-into the 21st century.Impact StatementMany school psychologists engage in assessment practices that lack compelling scientific support potentially taking time, resources, and energy away from more effective practices. This article critically reviews reasons why these questionable assessment practices persist long after discrediting scientific evidence has been aptly presented. Recommendations are offered to promote the use of evidence-basedpractices and discourage the use of assessment methods lacking compelling empirical support in training and clinical practice.

The Wechsler Intelligence Scale for Children’s (WISC) factorial\theoretical structure has undergone numerous substantive changes since it was first developed, and each of these changes has subsequently been questioned by assessment experts. Given remaining questions about the structure of the latest revision, the WISC-V, the present study used three different exploratory factor analytic techniques to investigate the factor structure of the 10 primary subtests in a large clinical sample (N= 5359). Results revealed that the WISC-V likely contains four factors (e.g., Gc, Gwm, Gs, and either Gv (via exploratory bifactor analysis) or a complexly determined perceptual reasoning factor (via the Schmid-Leiman procedure and oblique/higher-order factor analysis)), notthe five factors proposed by the test publisher. Variance apportionment and omega estimates indicate that only secondary interpretive emphasis should be placed upon group factors, but only when there is structural validity support. This study suggests that the WISC-V measures four, not five, factors, although the four-factor configuration may be different than previously reported in the technical literature.

AbstractIQ tests provide numerous scores, but valid interpretation of those scores is dependent on how precisely each score reflects its intended construct and whether it provides unique information independent of other constructs. Thus, IQ scores must be evaluated for their reliability and dimensionality to determine their psychometric utility. As a tutorial, the Wechsler Intelligence Scale for Children–Fifth Edition (WISC-V) scores were evaluated and it was demonstrated that the WISC-V is multidimensional, but only the Full-Scale IQ (FSIQ) was found to be sufficiently reliable for clinical use. WISC-V group factorswere not well defined and WISC-V index (i.e., factor) scoreswere contaminated with variance from other constructs and insufficiently reliable for clinical decisions. Clinicians were encouraged to go beyond structural goodness of fit and evaluate IQ test scores in terms of their reliability and ability to provide information that is not available from the general ability score as well their predictive and treatment validity. Software was provided to assist in that evaluation.Impact StatementIQ tests provide numerous scores, but valid interpretation of those scores is dependent on how precisely each score reflects its intended construct and whether it provides unique information independent of other constructs. Thus, IQ scores must be evaluated for their reliability and dimensionality to determine their psychometric utility. This article describes an evidence-based approach that clinicians can employ to assess the psychometric utility of IQ scores, supplies software tools to assist in that analysis, and provides a tutorial example using Wechsler Intelligence Scale for Children–Fifth Edition (WISC-V) scores.

AbstractIt is often assumed that children with learning disabilities (LD) exhibit unique profiles of ability scores that reflect idiosyncratic cognitive strengths and weaknesses. Interpretation of cognitive ability profiles initially focused on visual inspection of subtest scores followed by statistical comparisons to identify significant cognitive strengths and weaknesses. However, subsequent research demonstrated that these subtest profiles lacked sufficient reliability, validity, and diagnostic utility. Profile research typically utilized variable-centered methods, but person-centered methods might be more appropriate. The present study utilized latent profile analysis (LPA), a person-centered method that is model-based and flexible, with 1,830 school-identified students with LD and 2,200 simulated normative participants. Four broad ability score profiles distinguished by level rather than shape emerged. Thus, this latent mixture model analysis found no mixture of subpopulations, suggesting that WISC-IV score variation was due to underlying continuous latent factors rather than a typology unique to LD.Supplemental data for this article is available online at https://doi.org/10.1080/2372966X.2021.1919923Impact StatementIt is often assumed that children with learning disabilities (LD) exhibit unique profiles of ability test scores that reflect idiosyncratic cognitive strengths and weaknesses. However, this person-centered analysis found no profile of ability scores distinctive of children with LD. These results suggest that cognitive test scores may vary due to underlying continuous latent factors rather than a typology unique to LD.

AbstractThis study investigated the stability of Wechsler Intelligence Scale for Children–Fifth Edition (WISC-V) scores for 225 children and adolescents from an outpatient neuropsychological clinic across, on average, a 2.6 year test-retest interval. WISC-V mean scores were relatively constant but subtest stability score coefficients were all below 0.80 (M = 0.66) and only the Verbal Comprehension Index (VCI), Visual Spatial Index (VSI), and omnibus Full Scale IQ (FSIQ) stability coefficients exceeded 0.80. Neither intraindividual subtest difference scores nor intraindividual composite difference scores were stable across time (M = 0.26 and 0.36, respectively). Rare and unusual subtest and composite score differences as well as subtest and index scatter at initial testing were unlikely to be repeated at retest (kappa = 0.03 to 0.49). It was concluded that VCI, VSI, and FSIQ scores might be sufficiently stable to support normative comparisons but that none of the intraindividual (i.e. idiographic, ipsative, or person-relative) measures were stable enough for confident clinical decision making.

The goal of the present study was to investigate the factor structure of the Bullying Participant Behaviors Questionnaire in an independent elementary school sample. The BPBQ is a self-report inventory that purports to measure participation or experiences in five bullying roles: perpetrator, target, assistant, defender, and outsider. The current sample included 683 primarily White 8–11-year old youth from three elementary schools in the Midwest (46% male students). Analyses generally supported the item assignments to the BPBQ five-factor model (Perpetrator, Assistant, Outsider, Target, Defender). Confirmatory factor analysis revealed that the best fitting model consisted of the two general dimensions (Pro-perpetrator, Pro-target) and five group factors: Perpetrator, Assistant, Outsider, Target, and Defender, as was observed with an independent sample of middle school students. There is general support for the factor structure of the BPBQ, but it is particularly useful if interested in the broader Pro-perpetrator and Pro-target dimensions.

ABSTRACTThe Bullying Participant Behavior Questionnaire is a 50 item self-report survey designed to measure engagement in five bullying roles: bully, victim, assistant to the bully, defender of the victim, and outsider. The goal of the current study was to examine the higher-order factor structure of the BPBQ in a large middle school sample of 784 primarily White sixth- through eighth-grade students from the Midwest region of the United States. Analyses suggested that the Victim, Defender, and Outsider items loaded on their theoretically consistent factors and the Bully and Assistant items all loaded on a single factor (Bully/Assistant). Some factor correlations were moderate and suggested the presence of higher-order factor(s). A second-order EFA showed there appear to be two higher-order factors. Factor 1 is a combination of the Bully/Assistant and Outsider dimensions while Factor 2 is a combination of Defender and Victim dimensions. Confirmatory factor analysis indicated that the best fitting model consisted of the two general factors. There was a Pro-bully factor consisting of the Bully, Assistant, and Outsider group factors and a Pro-victim factor consisting of the Victim and Defender factors. There is support for the use of the BPBQ, but additional refinement of the items is needed.

ABSTRACTThis study utilized confirmatory factor analyses to examine the latent factor structure of the Wechsler Intelligence Scale for Children–Fourth Edition, Italian adaptation (WISC–IV Italian) standardization sample. One through five, oblique first-order factor models and higher-order as well as bifactor models were examined and compared using CFA. The bifactor model provided the best explanation of the scale’s factor structure. Across all models, general intelligence accounted for the largest amount of explained common variance, with group factors accounting for nonsignificant and trivial amounts of explained common variance. Omega-hierarchical subscale coefficients indicated that unit-weighted composites that would be generated by group factors would contain miniscule unique variance and therefore be of little clinical utility. Results are consistent with numerous empirical studies examining other European adaptations of the instrument and are at odds with the model put forward by the publisher. Clinicians who use the WISC–IV Italian adaptation are warned against attempting to interpret factor index scores independent of general intelligence.

ABSTRACTThe factor structure of the Wechsler Intelligence Scale for Children–Fourth Edition, Italian adaptation (WISC-IV Itaian; Orsini, Pezzuti, & Picone, 2012; Wechsler, 2012) standardization sample was examined with exploratory factor analytic methods (EFA) not included in the Technical Manual. Principal-axis extraction followed by oblique rotation using five-, four-, three-, and two-factor solutions all produced relatively similar results. None of the extraction criteria supported the retention of four factors, as suggested by the WISC–IV Italian Technical Manual. However, when the four-factor structure was subjected to second-order factor analysis and transformed with the Schmid and Leiman (1957) orthogonalization procedure, the hierarchical gfactor accounted for large portions of total and common variance, while the four first-order group factors accounted for small portions of total and common variance, rendering interpretation at the factor index level of questionable value. Clinicians who use the WISC-IV Italian should recognize the strong measurement of general intelligence yielded by the scale and clinical interpretation should avoid the overinterpretation of factor index scores that conflate group factor variance with general intelligence variance.

ABSTRACTThe reliability and factorial validity of the Wechsler Intelligence Scale for Children–Fifth Edition: Canadian (WISC-VCDN) was investigated. The higher-order model preferred by Wechsler (2014b) contained five group factors but lacked discriminant validity. An alternative bifactor model with four group factors and one general factor, akin to the traditional Wechsler model, exhibited the best global fit. The general factor accounted for 33.8% of the total variance and 67.6% of the common variance, but none of the group factors accounted for substantial portions of variance. All together, the general and group factors accounted for 50% of the total variance. Omega reliability coefficients demonstrated that reliable variance of WISC-VCDNfactor index scores was primarily due to the general factor, not the group factors. It was concluded that the cumulative weight of reliability and validity evidence suggests that psychologists should focus their interpretive efforts at the general factor level and exercise extreme caution when using group factor scores to make decisions about individuals.

ABSTRACTThe present study examined the factor structure of the Wechsler Intelligence Scale for Children–Fourth Edition, Spanish (WISC–IV Spanish, Wechsler, 2005a) with normative sample participants aged 6–16 years (N = 500) using confirmatory factor analytic techniques not reported in the WISC–IV Spanish Manual (Wechsler, 2005b). For the 10 core subtest configuration, 1 through 4, first-order factor models, andhigher-order versus bifactor models were compared using confirmatory factor analyses. The correlated four-factor Wechsler model provided good fit to these data, but the bifactor model showed statistically significant improvement over the higher-order model and correlated four-factor model. For the 14 core andsupplemental subtest configuration, an alternative five-factor model based upon Cattell-Horn-Carroll (CHC; as per Weiss, et al., 2013b) configuration was also estimated. Results indicated that for the 14 subtest configuration, the alternative CHC model was preferred to the four-factor Wechsler model and the bifactor version of the CHC model also fit these data best. Across both configurations, variance apportionment and model-based reliability estimates illustrate well the dominance of the general intelligence factor when compared to the influence of the various combinations of group factors. Implications for clinical interpretation and the anticipated revision of the measurement instrument are discussed.

A Correction to this paper has been published: https://doi.org/10.1007/s40688-021-00374-5

Replication of the Adjustment Scales for Children and Adolescents (ASCA) core syndrome factor structure with a sample of 124 randomly selected Hispanic/Latino youths is reported. The six ASCA core syndromes produced the identical two-factor solution observed in other samples. Principal-axis exploratory factor analysis using multiple factor extraction criteria and varimax, direct oblimin, and promax rotations produced nearly identical factor structure coefficients. Consistent with earlier studies, the ASCA was observed to measure two independent dimensions of youth psychopathology (Overactivity and Underactivity) that are similar to the conduct problems/ externalizing and withdrawal/internalizing dimensions typically found in the child psychopathology assessment literature.

The present study reports on the replication of the core syndrome factor structure of the Adjustment Scales for Children and Adolescents (ASCA) for a sample of 229 Native American Indian (Yavapai Apache) children and adolescents from rural north-central Arizona. The six ASCA core syndromes produced the identical two-factor solution as the standardization sample, an independent sample, and a sample of Native American Indians (Ojibwe) from north-central Minnesota. Principal-axis analysis using multiple criteria for the number of factors to extract and retain was used with varimax, direct oblimin, and promax rotations producing identical results and nearly identical factor structure coefficients. As with earlier studies, it was concluded that the ASCA measures two independent global dimensions of youth psychopathology (Overactivity and Underactivity) that are similar to the conduct problems/externalizing and withdrawal/internalizing dimensions commonly found in the child psychopathology assessment literature.

This study reports on the replication of the four-factor structure and the internal consistency of the Learning Behaviors Scale (LBS) with an independent sample of 241 students in Grades 1 through 7. Internal consistency estimates were as high or higher than those obtained with the standardization sample. Substantial replication of the four LBS factors (Competence Motivation, Attitude Toward Learning, Attention/Persistence, and Strategy/Flexibility) was found. Most items in the study were associated with the identical factor found with the standardization sample. Coefficients of congruence tested the factorial invariance and resulted in “good” to “excellent” matches to the exploratory factor analysis results based on the LBS standardization sample. Furthermore, three of the five LBS items that cross-loaded in this study were also items that cross-loaded in the standardization sample and did so on the same factors. Substantial evidence was found for the internal consistency and construct (factorial) validity of the LBS.

Although often applied in practice, clinically based cognitive subtest profile analysis has failed to achieve empirical support. Nonlinear multivariate subtest profile analysis may have benefits over clinically based techniques, but the psychometric properties of these methods must be studied prior to their implementation and interpretation. The current study posed the following question: Is Wechsler Intelligence Scale for Children–Third Edition (WISC-III) cluster membership based on nonlinear multivariate subtest profile analysis stable over a 3-year period? Membership stability to a subtest taxonomy, including constancy of displaying an unusual profile, was based on data from 585 students. General (.39) and partial (.26 to .51) kappa coefficients either failed to reach statistical significance or indicated poor classification stability, with the exception of two profile types. It was concluded that, with these two possible exceptions, profile-type membership to an empirically derived WISC-III subtest taxonomy should not be used in interpretation or educational decision making.

The present study reports data supporting the construct validity of the Kaufman Brief Intelligence Test (K-BIT; Kaufman & Kaufman, 1990), the Wechsler Intelligence Scale for Children-Third Edition (WISC-III; Wechsler, 1991), and the Adjustment Scales for Children and Adolescents (ASCA; McDermott, Marston, & Stott, 1993) through convergent and discriminant comparisons in a sample of 207 students receiving special education evaluations. Results were as hypothesized, with high and statistically significant correlations between the K-BIT and WISCIII, supporting convergent validity. Moderate and statistically significant correlations were obtained between the two intelligence measures (K-BIT and WISC-III) and measures of academic achievement (WIAT, WIAT-II, WJ-R ACH, or WJ-3 ACH) at levels typical of ability-achievement correlations. Correlations between the two intelligence measures (K-BIT and WISC-III) and the ASCA, a measure of child psychopathology, were low to near zero, supporting discriminant validity. Further discriminant evidence of construct validity was provided by the low to near zero correlations between the ASCA and the measures of academic achievement.

This study examined the construct validity of the Adjustment Scales for Children and Adolescents (ASCA; McDermott, Marston, & Stott, 1993). Distinct group differences and discriminative validity (Youngstrom, Findling, Danielson, & Calabrese, 2001) were examined with a sample of 106 students ranging from grade 1 through 6 (53 met the DSM-IV/DISCIVcriteria for ADHD, and 53 were members of a randomly selected and matched control group). Statistically significant group differences were observed with large effect sizes. Further, the ASCA demonstrated near perfect discrimination of the two groups and all diagnostic efficiency statistics were highly supportive in differentiating students meeting DISC-IV/DSM-IVcriteria for ADHD from a random and matched control group

This article reports on an investigation of the interrater agreement on the Adjustment Scales for Children and Adolescents (ASCA) syndromic profile classifications. Teaching professionals (N = 29) who shared the same classroom for a minimum of one hour per day provided independent ratings of the same child on the ASCA. Results indicated that statistically significant interrater agreement was achieved across all 22 syndromic profile classification levels. However, good clinical significance was obtained only for the three- and two-level broad classifications while fair clinical significance was obtained for the five-level broad classifications. Thus, there was adequate interrater agreement for ASCA classifications that reflected broad levels of adjustment and maladjustment but inadequate agreement for specific behavioral types of adjustment and maladjustment. Additional research with much larger sample sizes is needed to better determine the agreement for the 22 syndrome profiles.

Convergent and divergent (construct) validity of the Adjustment Scales for Children and Adolescents (ASCA) and the Preschool and Kindergarten Behavior Scales (PKBS) is reported. With a random sample of 154 five- and six-year-old children rated by 16 classroom teachers in a Midwestern state, convergent evidence of construct validity was provided for the PKBS Externalizing Problems scale and the ASCA Overactivity syndrome. Divergent evidence of construct validity was provided for the PKBS Externalizing Problems scale and ASCA Underactivity syndrome. Convergent and divergent evidence of construct validity for the PKBS Internalizing Problems scale and ASCA Overactivity and Underactivity syndromes was mixed. Specific scale comparisons found the ASCA to result in significantly higher mean ratings than the PKBS in 9 of 10 specific comparisons, and the effect sizes 112 were moderate to large.

AbstractLong-term stability of the Wechsler Intelligence Scale for Children-Third Edition (WISC-III) was investigated for children with specific learning disability (SLD), serious emotional disability (SED), and mental retardation (MR). Participants were 522 students from 33 states twice evaluated for special education eligibility over a mean test-retest interval of 2.87 years. There were no differential effects of disability groups on long-term stability coefficients. Stability coefficients for Full Scale IQ (FSIQ) scores were acceptable for all three disability groups. Of the global IQ and Index scores, only the Freedom from Distractability Index, Processing Speed Index, and Verbal IQ-Performance IQ discrepancy score stability coefficients were inadequate. Subtest stability was also inadequate. Mean changes from first testing to second testing for IQ and Index scores were not significant and the two significant subtest changes were not clinically meaningful due to small effect sizes. Individual change scores revealed that only the FSIQ was sufficiently stable for use with individual students with SLD, SED, or MR. Results extended those of Canivez and Watkins (1998, 1999) supporting long-term stability for the WISC-III.

Interobserver agreement of children's problem behavior was assessed using two samples of special education students ages 5 to 18 years. The first sample had observers from the same setting (N= 71); the second sample (N= 182) had observers from different settings with no concurrent observation. Regular and special education teachers and aides completed the Adjustment Scales for Children and Adolescents (ASCA; McDermott, Marston, & Stott, 1993). Inter- and intraclass correlations were generally significant for both samples, with some exceptions. Substantial interobserver agreement was found for the same-setting sample; however, agreement coefficients were lower for the different-setting sample and some level effects were noted. Overall, interobserver agreement for the ASCA was supported in common settings, but rating variability was evident across classrooms and appears indicative of contextual influences on behavior.

AbstractThis study used both exploratory (EFA) and confirmatory factor analyses (CFA) to examine the factor structure of the WISC-III among White and Black students from the WISC-III standardization sample and a sample of 348 Black students referred for psychological evaluation. Results of the EFA provided evidence of a large first principal factor as well as the expected Verbal and Performance components across all three groups. Empirical support for the Freedom from Distractibility dimension was provided only from the confirmatory factor analyses. Although the four factor confirmatory model exhibited the best overall statistical fit, inspection of specific factor loadings revealed anomalies with the third and fourth factors, especially for the Referred Black sample. Implications for school psychologists are presented and recommendations for future research are provided.

Long-term stability of the Wechsler Intelligence Scale for Children-Third Edition was investigated separately across gender, race/ethnicity, and age subgroups. Participants were 642 students from 33 states evaluated twice for special education eligibility over a mean test-retest interval of 2.83 years. Gender, race/ethnicity, and age produced few or no differential effects on long-term stability coefficients. Most of the demographic subgroup stability coefficients for VIQ, PIQ FSIQ VCI, and POI scores demonstrated satisfactory long-term stability. However, stability coefficients for FDI, PSI, and VIQ-PIQ discrepancy scores were not adequate. Mean differences from first testing to second testing were either not statistically significant or not clinically meaningful for all groups, except Hispanic/Latino youths. Analysis of individual change scores indicated that only the FSIQ was sufficiently stable for use with individual students. Results extended those of Canivez and Watkins (1998), supporting long-term stability for the WISC-III among most demographic subgroups studied.

This study examined the first-order factor structure of the Wechsler Intelligence Scale for Children-Third Edition (WISC-III; Wechsler, 1991) on three samples of children previously diagnosed with a handicapping condition. Five alternative factor models were compared through confirmatory factor analysis. Previous factor analytic studies that focused on the WISC-III's four-factor solution employed all 13 subtests in their analyses, despite the fact that only 12 subtests are combined during clinical evaluations to obtain scores on the four factors. This study investigated the WISC-III factor structure by considering the 12 subtests that actually combine to yield index scores on the dimensions of Verbal Comprehension, Perceptual Organization, Processing Speed, and Freedom from Distractibility. Results support a four-factor solution for children with disabilities.

The present study examined the concurrent validity and diagnostic efficiency of the Kaufman Brief Intelligence Test (K-BIT). Seventy-five students with learning disability were administered the K-BIT, Wechsler Intelligence Scale for Children-Third Edition (WISC-III), and Woodcock-Johnson-Revised Tests of Achievement (WJ-R ACH) as part of their triennial reevaluation. Correlations between the K-BIT and the WISC-III ranged from .18 (r2= .03) to .82 (r2= .67), M, =.62 (Mr2=.38). High levels of agreement were obtained between the K-BIT and WISC-III in identifying severe achievement-ability discrepancies on the WJ-R ACH. The K-BIT appears to be a promising general intellectual screening instrument when more comprehensive assessment is not possible or needed.

Concurrent validity of the Kaufman Brief Intelligence Test (K-BIT) with a sample of elementary- and middle-school students referred for multidisciplinary evaluations in a public school setting is presented. All correlations between the K-BIT and the Wechsler Intelligence Scale for Children–Third Edition (WISC-III) were significant. Correlations ranged from .36 (r2= .10) to .87 (r2= .75), Mr= .71 (Mr2= .50). K-BIT Vocabulary-Matrices discrepancy scores accounted for a significant but small proportion (13%) of the variability in WISC-III VIQ-PIQ discrepancies, but kappa (k) coefficients for these discrepancies indicated that agreement was generally no better than chance. The K-BIT appears to be a promising general intellectual screening instrument when more comprehensive assessment is not possible or needed, but interpretation is best left at the IQ Composite level for the present time.

Standardized behavior rating scales and checklists offer unobtrusive evaluations of students' behavior in natural social environments. This study investigated the interrater agreement of the Adjustment Scales for Children and Adolescents (ASCA), a behavior rating scale used in school settings. Participants were 71 students enrolled in a variety of special programs who were rated by 29 observers in 24 classrooms. Resulting interrater reliability coefficients were substantial, and level differences, although significant, were not clinically meaningful. It was concluded that the ASCA produced acceptable levels of interrater agreement when educational professionals and paraprofessionals observed exceptional students within a common environment.