Search

The reliable change index has been used to evaluate the significance of individual change in health-related quality of life. We estimate reliable change for two measures (physical function and emotional distress) in the Patient-Reported Outcomes Measurement Information System (PROMIS®) 29-item health-related quality of life measure (PROMIS-29 v2.1). Using two waves of data collected 3 months apart in a longitudinal observational study of chronic low back pain and chronic neck pain patients receiving chiropractic care, and simulations, we compare estimates of reliable change from classical test theory fixed standard errors with item response theory standard errors from the graded response model. We find that unless true change in the PROMIS physical function and emotional distress scales is substantial, classical test theory estimates of significant individual change are much more optimistic than estimates of change based on item response theory. Supplementary Information The online version supplementary material available at 10.1007/s11336-021-09774-1.

Objective.The National Neuropsychology Network (NNN) is a multicenter clinical research initiative funded by the National Institute of Mental Health (NIMH; R01 MH118514) to facilitate neuropsychology’s transition to contemporary psychometric assessment methods with resultant improvement in test validation and assessment efficiency.Method:The NNN includes four clinical research sites (Emory University; Medical College of Wisconsin; University of California, Los Angeles (UCLA); University of Florida) and Pearson Clinical Assessment. Pearson Q-interactive (Q-i) is used for data capture for Pearson published tests; web-based data capture tools programmed by UCLA, which serves as the Coordinating Center, are employed for remaining measures.Results:NNN is acquiring item-level data from 500–10,000 patients across 47 widely used Neuropsychology (NP) tests and sharing these data via the NIMH Data Archive. Modern psychometric methods (e.g., item response theory) will specify the constructs measured by different tests and determine their positive/negative predictive power regarding diagnostic outcomes and relationships to other clinical, historical, and demographic factors. The Structured History Protocol for NP (SHiP-NP) helps standardize acquisition of relevant history and self-report data.Conclusions:NNN is a proof-of-principle collaboration: by addressing logistical challenges, NNN aims to engage other clinics to create a national and ultimately an international network. The mature NNN will provide mechanisms for data aggregation enabling shared analysis and collaborative research. NNN promises ultimately to enable robust diagnostic inferences about neuropsychological test patterns and to promote the validation of novel adaptive assessment strategies that will be more efficient, more precise, and more sensitive to clinical contexts and individual/cultural differences.

As part of a scale development project, we fit a nominal response item response theory model to responses to the Health Care Engagement Measure (HEM). When using the original 5-point response format, categories were not ordered as intended for six of the 23 items. For the remaining, the category boundary discrimination between Categories 0 ( not at all true) and 1 ( a little bit true) was only weakly discriminating, suggesting uninformative categories. When the lowest two categories were collapsed, psychometric properties improved greatly. Category boundary discriminations within items, however, varied significantly. Specifically, higher response category distinctions, such as responding 3 ( very true) versus 2 ( mostly true) were considerably more discriminating than lower response category distinctions. Implications for HEM scoring and for improving measurement precision at lower levels of the construct are presented as is the unique role of the nominal response model in category analysis.

Post-traumatic stress disorder (PTSD) can develop after exposure to traumatic events and severely impacts the quality of life. PTSD is frequently comorbid with substance use disorders, with alcoholism being particularly common. However, not everyone who experiences trauma develops PTSD and the factors that render individuals susceptible or resilient to the effects of stress are unknown although gender appears to play an important role. Rodent models of stress exposure such as stress-enhanced fear learning (SEFL) recapitulate some aspects of PTSD symptomology, making them an invaluable tool for studying this disorder. This study examined whether exposure to a modified version of the SEFL procedure (4 footshocks instead of the standard 15 over 90min) would reveal both susceptible and resilient subjects. Following stress exposure, distinct susceptible and resilient groups emerged that differed in fear learning and anxiety-related behavior as well as voluntary alcohol intake. Some aspects of stress susceptibility manifested differently in males compared to females, with susceptibility associated with increased alcohol intake in males and increased baseline anxiety in females.

One of the most widely used measures of psychosis-related symptoms and characteristics is the 74-item Schizotypal Personality Questionnaire (SPQ). Using multidimensional Item Response Theory (bifactor 2-parameter model), we calibrated SPQ items in a sample of 375 youths aged 9–24 years and constructed a fully functional computerized adaptive form of the SPQ on an open-source platform for public use. To assess validity, we used the above parameters to simulate CAT sessions in a separate validation sample (N = 100) using three test-length-based stopping rules: 8 items, 16 items, and 32 items. Those scores were then compared to full-form and SPQ-Brief scores on their abilities to predict psychosis or clinical risk status. Areas under the receiver operating characteristic curves indicated mediocre predictive ability, but did not differ among any of the forms, even when only eight adaptive items were administered. The Youden index for the 16-item adaptive version was higher than that for the 22-item SPQ-Brief. Classification accuracy for the full SPQ was 73% compared to 66% for the both the SPQ-Brief and adaptive versions (average of three stopping rules). The SPQ-CAT shows promise as a much shorter but valid assessment of schizotypy which can save time with minimal loss of information.

Author(s): Anderson, Ariana E; Marder, Stephen; Reise, Steven P; Savitz, Adam; Salvadore, Giacomo; Fu, Dong Jing; Li, Qingqin; Turkoz, Ibrahim; Han, Carol; Bilder, Robert M | Abstract: Objective:Common genetic variation spans schizophrenia, schizoaffective and bipolar disorders, but historically, these syndromes have been distinguished categorically. A symptom dimension shared across these syndromes, if such a general factor exists, might provide a clearer target for understanding and treating mental illnesses that share core biological bases. Method:We tested the hypothesis that a bifactor model of the Positive and Negative Syndrome Scale (PANSS), containing 1 general factor and 5 specific factors (positive, negative, disorganized, excited, anxiety), explains the cross-diagnostic structure of symptoms better than the traditional 5-factor model, and examined the extent to which a general factor reflects the overall severity of symptoms spanning diagnoses in 5094 total patients with a diagnosis of schizophrenia, schizoaffective, and bipolar disorder. Results:The bifactor model provided superior fit across diagnoses, and was closer to the "true" model, compared to the traditional 5-factor model (Vuong test; P l .001). The general factor included high loadings on 28 of the 30 PANSS items, omitting symptoms associated with the excitement and anxiety/depression domains. The general factor had highest total loadings on symptoms that are often associated with the positive and disorganization syndromes, but there were also substantial loadings on the negative syndrome thus leading to the interpretation of this factor as reflecting generalized psychosis. Conclusions:A bifactor model derived from the PANSS can provide a stronger framework for measuring cross-diagnostic psychopathology than a 5-factor model, and includes a generalized psychosis dimension shared at least across schizophrenia, schizoaffective, and bipolar disorder.

Background About 70% of Latinos, 5 years old or older, in the United States speak Spanish at home. Measurement equivalence of the PROMIS® pain interference (PI) item bank by language of administration (English versus Spanish) has not been evaluated.

Traditional "paper-and-pencil" testing is imprecise in measuring speed and hence limited in assessing performance efficiency, but computerized testing permits precision in measuring itemwise response time. We present a method of scoring performance efficiency (combining information from accuracy and speed) at the item level. Using a community sample of 9,498 youths age 8-21, we calculated item-level efficiency scores on 4 neurocognitive tests, and compared the concurrent, convergent, discriminant, and predictive validity of these scores with simple averaging of standardized speed and accuracy-summed scores. Concurrent validity was measured by the scores' abilities to distinguish men from women and their correlations with age; convergent and discriminant validity were measured by correlations with other scores inside and outside of their neurocognitive domains; predictive validity was measured by correlations with brain volume in regions associated with the specific neurocognitive abilities. Results provide support for the ability of itemwise efficiency scoring to detect signals as strong as those detected by standard efficiency scoring methods. We find no evidence of superior validity of the itemwise scores over traditional scores, but point out several advantages of the former. The itemwise efficiency scoring method shows promise as an alternative to standard efficiency scoring methods, with overall moderate support from tests of 4 different types of validity. This method allows the use of existing item analysis methods and provides the convenient ability to adjust the overall emphasis of accuracy versus speed in the efficiency score, thus adjusting the scoring to the real-world demands the test is aiming to fulfill. (PsycINFO Database Record

Quantitative models of mental illness, such as the Hierarchical Taxonomy of Psychopathology (HiTOP), aim to optimize clinical assessment, which conventionally follows categorical diagnostic rubrics. The evidence base for these models is robust, but also uniform; available data come mostly from structured diagnostic interviews in nationally representative samples. It remains to be seen whether HiTOP adequately reflects mental illness as evaluated in routine clinical care, where diagnosis is often unsystematic and incomplete, relative to controlled research conditions. To test the generalizability of a quantitative nosology to real-world assessment contexts, we fit the HiTOP model to diagnoses in a large sample (N = 25,002) of treatment-seeking university students who were seen by health professionals in everyday practice. We then examined the criterion validity of model components in relation to clinically relevant outcomes (i.e., suicide attempts, self-injury, and binge drinking). Three related structures fit the data well: a correlated-factor model with internalizing, externalizing, and eating pathology dimensions; a higher-order model that added a general factor of psychopathology that spanned these 3 first-order factors; and a bifactor model that partitioned diagnostic (co)variance across a general factor and 3 orthogonal group factors. The first-order factors had expected patterns of criterion validity, and the general factor was a strong predictor of suicidality and self-injury, paralleling past research. Bifactor model group factors had interpretative problems, however. Across models, categorical diagnoses consistently offered minimal incremental validity relative to the transdiagnostic factors. We conclude that HiTOP is ecologically valid-explaining comorbidity patterns among diagnoses assigned "in the field"-and is poised to enhance clinical assessment and decision-making in routine care. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

OBJECTIVE: This article reviews current approaches to neuropsychological assessment, identifies opportunities for development of new methods using modern psychometric theory and advances in technology, and suggests a transition path that promotes application of novel methods without sacrificing validity. METHODS: Theoretical/state-of-the-art review. CONCLUSIONS: Clinical neuropsychological assessment today does not reflect advances in neuroscience, modern psychometrics, or technology. Major opportunities for improving practice include both psychometric and technological strategies. Modern psychometric approaches including item response theory (IRT) enable: linking procedures that can place different measures on common scales; adaptive testing algorithms that can dramatically increase efficiency of assessment; examination of differential item functioning (DIF) to detect measures that behave differently in different groups; and person fit statistics to detect aberrant patterns of responding of high value for performance validity testing. Opportunities to introduce novel technologies include computerized adaptive testing, Web-based assessment, healthcare- and bio-informatics strategies, mobile platforms, wearables, and the “internet-of-things.” To overcome inertia in current practices, new methods must satisfy requirements for back-compatibility with legacy instrumentation, enabling us to leverage the wealth of validity data already accrued for classic procedures. A path to achieve these goals involves creation of a global network to aggregate item-level data into a shared repository that will enable modern psychometric analyses to refine existing methods, and serve as a platform to evolve novel assessment strategies, which over time can revolutionize neuropsychological assessment practices world-wide.

The recent incorporation of genomics into clinical neuroscience has generated an unprecedented need for behavioral assessment of domains that can be linked to brain systems and serve as “biomarkers” of psychopathology (Insel & Cuthbert, 2009). The scale of such studies requires efficient computerized testing of a broad range of abilities using tests that have been validated with functional neuroimaging. Such “neurobehavioral probes” (Gur, Erwin and Gur, 1992) have been assembled into a computerized neurocognitive battery (CNB) in which these tasks have been adapted to assure adequate psychometric properties, such as reliability and validity (Gur et al., 2001, 2010) and its linkage to brain systems (Roalf et al., 2013). The CNB has been applied in large-scale genomic studies (Aliyu et al., 2006; Greenwood et al., 2007; Gur et al., 2007a,b; Almasy et al., 2008), treatment research (Gur et al., 2001b; Grant et al., 2013), and the military (Thomas et al., 2013). Although its individual tests’ scores have been largely reliable (and their inferences valid), the overall latent structure of the battery has not been sufficiently evaluated. Using confirmatory and exploratory factor analysis, we aimed to examine the factor structure of the CNB, which is of paramount importance in justifying how the instrument is scored and interpreted. The CNB currently comprises fourteen tests grouped into five domains of neurobehavioral function. These broad domains were selected because they represent well-established brain systems, and each test in the battery was used in functional neuroimaging studies to probe aspects of these domains (Roalf et al., 2013). There are three tests in each of four domains measuring “Executive Control,” “Episodic Memory,” “Complex Cognition,” and “Social Cognition” as summarized in Table 1. A fifth domain (Sensorimotor speed) was measured by two tests, but the current analysis examined accuracy and these tests only provide speed measures and were therefore not included. Here we examined the factorial structure of the CNB in a sample of over 9,000 individuals age 8 to 21 who were administered the CNB as part of their participation in the Philadelphia Neurodevelopmental Cohort (PNC; Gur et al., 2012, 2013). Table 1 Domain-Specific scales of the Computerized Neurocognitive Battery, by neurobehavioral function

Although the Positive and Negative Syndrome Scale (PANSS) was developed for use in schizophrenia (SZ), antipsychotic drug trials use the PANSS to measure symptom change also for bipolar (BP) and schizoaffective (SA) disorder, extending beyond its original indications. If the dimensions measured by the PANSS are different across diagnoses, then the same score change for the same drug condition may have different meanings depending on which group is being studied. Here, we evaluated whether the factor structure in the PANSS was consistent across schizophrenia (n = 3647), bipolar disorder (n = 858), and schizoaffective disorder (n = 592). Along with congruency coefficients, Hancock's H, and Jaccard indices, we used target rotations and statistical tests of invariance based on confirmatory factor models. We found the five symptom dimensions measured by the 30-item PANSS did not generalize well to schizoaffective and bipolar disorders. A model based on an 18-item version of the PANSS generalized better across SZ and BP groups, but significant problems remained in generalizing some of the factors to the SA sample. Schizophrenia and bipolar disorder showed greater similarity in factor structure than did schizophrenia and schizoaffective disorder. The Anxiety/Depression factor was the most consistent across disorders, while the Positive factor was the least consistent.

It is generally assumed that the latent trait is normally distributed in the population when estimating logistic item response theory (IRT) model parameters. This assumption requires that the latent trait be fully continuous and the population homogenous (i.e., not a mixture). When this normality assumption is violated, models are misspecified, and item and person parameter estimates are inaccurate. When normality cannot be assumed, it might be appropriate to consider alternative modeling approaches: (a) a zero-inflated mixture, (b) a log-logistic, (c) a Ramsay curve, or (d) a heteroskedastic-skew model. The first 2 models were developed to address modeling problems associated with so-called quasi-continuous or unipolar constructs, which apply only to a subset of the population, or are meaningful at one end of the continuum only. The second 2 models were developed to address non-normal latent trait distributions and violations of homogeneity of error variance, respectively. To introduce these alternative IRT models and illustrate their strengths and weaknesses, we performed real data application comparing results to those from a graded response model. We review both statistical and theoretical challenges in applying these models and choosing among them. Future applications of these and other alternative models (e.g., unfolding, diffusion) are needed to advance understanding about model choice in particular situations.

OBJECTIVES: To describe the development and initial psychometric properties of the observer-rated Global Therapist Competence Scale for Youth Psychosocial Treatment (G-COMP) in the context of cognitive-behavioral treatment (CBT) for youth anxiety disorders. METHOD: Independent coders rated 744 sessions from a sample of 68 youth (M age = 10.56 years) using the G-COMP and observational instruments of alliance, involvement, CBT adherence, CBT competence. RESULTS: Inter-rater reliability coefficients (ICCs [2,2]) were greater than .60 for the five G-COMP domain scores. G-COMP scores yielded small to medium correlations with instruments of the child-therapist alliance (rs = .17 - .44) and youth involvement in treatment (rs = .08 - .53), and medium to large correlations with instruments of CBT competence and adherence (rs = .26 - .63). Therapists in the research setting were rated higher compared to newly trained therapists in community clinics. CONCLUSIONS: Preliminary reliability and validity of the G-COMP scores are promising, but future research is needed with non-CBT samples.

Schizophrenia is marked by a global cognitive impairment that contributes significantly to chronic disability and unemployment. As new treatments are developed for cognition in schizophrenia, clinicians require easily administered instruments to assess cognition. We previously developed a very brief cognitive battery (Bell et al., 2005). The Brief Cognitive Assessment Tool for Schizophrenia (B-CATS) was developed specifically to provide clinicians with a way to assess cognition in their patients with schizophrenia. Here, we report the results of a validity study comparing B-CATS to a larger neurocognitive battery, the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) battery.Outpatients with schizophrenia (N=91) were administered the B-CATS and the non-overlapping tests of the MATRICS battery at two time points separated by 1month. They were also administered the UCSD Performance-based Skills Assessment-Brief (UPSA-B), a measure of functional capacity.The B-CATS has an administration time of approximately 10min. It demonstrates good test-retest reliability and internal consistency. It correlates 0.76 (p0.01) with the MATRICS battery. The shorter B-CATS and the MATRICS battery correlate with the UPSA-B at 0.50 and 0.58 respectively.A 10-minute version of the B-CATS correlates highly with the "gold standard" neurocognitive battery that has an administration time of over 60min. Both measures correlate moderately with a measure of functional capacity. This brief battery was designed to allow clinicians to monitor cognitive change and better inform treatment decisions.

© The Author(s) 2014 The current research evaluated a bifactor model for the Disgust Emotion Scale (DES) in three samples: N = 1,318 nonclinical participants, N = 152 clinic-referred patients, and N = 352 nonclinical participants. The primary goals were to (a) use bifactor modeling to examine the latent structure of the DES and in turn (b) evaluate whether the DES should be scored as a unidimensional scale or whether subscales should also be interpreted. Results suggested that a bifactor model fit the DES data well and that all DES items were strongly influenced by a general disgust proneness dimension and by five content dimensions. Moreover, model-based reliability analyses suggested that scoring a general disgust dimension is justified despite the confirmed multidimensional structure. However, subscales were found to be unreliable after controlling for the general disgust factor with the potential exception of the Mutilation/Death and Animals subscale. Subsequent analysis also showed that only the general disgust factor robustly predicted an obsessive-compulsive disorder symptom latent factor—a clinical condition closely related to disgust proneness; latent variables representing DES domains displayed weak relations with an obsessive-compulsive disorder factor above and beyond the general disgust factor. Implications for better understanding the structure of DES responses and its use in clinical research are discussed.

The psychometric properties of 4 paradigms adapted from the social neuroscience literature were evaluated to determine their suitability for use in clinical trials of schizophrenia. This 2-site study (University of California, Los Angeles and University of North Carolina) included 173 clinically stable schizophrenia outpatients and 88 healthy controls. The social cognition battery was administered twice to the schizophrenia group (baseline, 4-week retest) and once to the control group. The 4 paradigms included 2 that assess perception of nonverbal social and action cues (basic biological motion and emotion in biological motion) and 2 that involve higher level inferences about self and others’ mental states (self-referential memory and empathic accuracy). Each paradigm was evaluated on (1) patient vs healthy control group differences, (2) test-retest reliability, (3) utility as a repeated measure, and (4) tolerability. Of the 4 paradigms, empathic accuracy demonstrated the strongest characteristics, including large between-group differences, adequate test-retest reliability (.72), negligible practice effects, and good tolerability ratings. The other paradigms showed weaker psychometric characteristics in their current forms. These findings highlight challenges in adapting social neuroscience paradigms for use in clinical trials.

Despite much debate regarding the structure and measurement of impulsivity, research on this topic continues to thrive. There are many self-report measures of impulsivity and constructs related to impulse-control; however, the Barratt Impulsiveness Scale Version 11 (BIS-11; Patton, Stanford, & Barratt, 1995) appears to be the gold-standard self-report instrument in this domain, and it has played a major role in research (see Ireland & Archer, 2008; Stanford et al., 2009). Numerous recent studies have used the BIS-11 to explore the social consequences and behavioral correlates of individual differences in impulsivity (Carlson, Johnson, & Jacobs, 2010; Kjome et al., 2010; Piero, 2010; Sweitzer, Allen, & Kaut, 2008) as well as the biological and genetic origins of impulsivity (Benko et al., 2010; Kaladjian, Jeanningros, Azorin, Anton, & Mazzola-Pomietto, 2010; Lee et al., 2009; Stoltenberg & Nag, 2010). Despite the extensive use of the BIS-11, psychometric research on its internal structure has been relatively scant, especially in large community samples. The primary goal of the present investigation, therefore, was to apply exploratory and confirmatory factor analyses to BIS-11 item response data collected in a community sample (N = 691). Our objective was to test specific, well-known theories of the BIS-11structure, which form the basis of the aggregate scale and subscale scores that are currently used in research (Patton et al., 1995; Stanford et al., 2009). Specifically, we used exploratory and confirmatory unidimensional and bifactor modeling to evaluate the degree to which the BIS-11 total score reflects a single common latent trait of impulsivity. We also used exploratory and confirmatory factor analyses to evaluate the theory that BIS-11 responses reflect six correlated first-order constructs (attention, motor, self-control, planfullness, cognitive complexity, perseverance, cognitive instability), which in turn, form three second-order factors (attention, motor, and non-planning). Prior to further description of these analyses, the next section reviews the development of the BIS-11.

Confirmatory factor analytic studies of psychological measures showing item responses to be multidimensional do not provide sufficient guidance for applied work. Demonstrating that item response data are multifactorial in this way does not necessarily (a) mean that a total scale score is an inadequate indicator of the intended construct, (b) demand creating and scoring subscales, or (c) require specifying a multidimensional measurement model in research using structural equation modeling (SEM). To better inform these important decisions, more fine-grained psychometric analyses are necessary. We describe 3 established, but seldom used, psychometric approaches that address 4 distinct questions: (a) To what degree do total scale scores reflect reliable variation on a single construct? (b) Is the scoring and reporting of subscale scores justified? (c) If justified, how much reliable variance do subscale scores provide after controlling for a general factor? and (d) Can multidimensional item response data be represented by a unidimensional measurement model in SEM, or are multidimensional measurement models (e.g., second-order, bifactor) necessary to achieve unbiased structural coefficients? In the discussion, we provide guidance for applied researchers on how best to interpret the results from applying these methods and review their limitations.

Item response theory (IRT) was separately applied to parent- and teacher-rated symptoms of attention-deficit/hyperactivity disorder (ADHD) from a pooled sample of 526 six- to twelve-year-old children with and without ADHD. The dimensional structure ADHD was first examined using confirmatory factor analyses, including the bifactor model. A general ADHD factor and two group factors, representing inattentive and hyperactive/impulsive dimensions, optimally fit the data. Using the graded response model, we estimated discrimination and location parameters and information functions for all 18 symptoms of ADHD. Parent- and teacher-rated symptoms demonstrated adequate discrimination and location values, although these estimates varied substantially. For parent ratings, the test information curve peaked between −2 and +2 SD, suggesting that ADHD symptoms exhibited excellent overall reliability at measuring children in the low to moderate range of the general ADHD factor, but not in the extreme ranges. Similar results emerged for teacher ratings, in which the peak range of measurement precision was from −1.40 to 1.90 SD. Several symptoms were comparatively more informative than others; for example, is often easily distracted (“Distracted”) was the most informative parent- and teacher-rated symptom across the latent trait continuum. Clinical implications for the assessment of ADHD as well as relevant considerations for future revisions to diagnostic criteria are discussed.

While attributional style is regarded as a core domain of social cognition, questions persist about the psychometric characteristics of measures used to assess it. One widely used assessment of attributional style is the Ambiguous Intentions Hostility Questionnaire (AIHQ). Two limitations of the AIHQ include (1) a possible restricted range resulting from too few and too homogenous item scenarios, and (2) use of rater scores that are cumbersome and time-consuming to score and have unknown incremental validity. The present study evaluated the psychometric properties of the AIHQ while concurrently testing changes aiming to improve the scale, in particular expansion of the number of self-report items and removal of the rater-scored items. One hundred sixty individuals diagnosed with schizophrenia and 58 healthy controls completed the full AIHQ along with measures of symptoms, functioning, and verbal intelligence. The AIHQ – particularly the self-reported blame score – demonstrated adequate internal consistency, test-retest reliability, and distinguished patients from controls. It also was significantly related to clinically-rated hostility and suspiciousness symptoms, and correlated with functional capacity even after controlling for verbal intelligence. Incremental validity analyses suggested that a higher number of self-report items strengthens relationships to outcomes in a manner that justifies this expansion, while rater-scored items had mixed results in providing additional information beyond self-report in the AIHQ.

Although reverse-worded items have often been incorporated in scale construction to minimize the effects of acquiescent reporting biases, some researchers have more recently begun questioning this approach and wondering whether the advantages associated with incorporating reverse-worded items is worth the complexities that they bring to measures (e.g., Brown, 2003 ; Marsh, 1996 ). In this study, we used item response theory (IRT) to determine whether there is statistical justification to eliminate the reverse-worded items (e.g., "I have lots of friends") from the Loneliness Questionnaire (LQ; Asher, Hymel,Renshaw, 1984) and retain only the non-reverse-worded items (e.g., "I'm lonely") to inform the provision of a shortened LQ version. Using a large sample of children (Grades 2-7; n = 6,784) and adolescents (Grades 8-12; n = 4,941), we examined the psychometric properties of the 24-item LQ and found support for retaining the 9 non-reverse-worded LQ items to make up a shortened measure of loneliness in youth. We found that the non-reverse-worded items were associated with superior psychometric properties relative to the reverse-worded items with respect to reliability and IRT parameters (e.g., discrimination and item information). A 3-point Likert-type scale was also found to be more suitable for measuring loneliness across both children and adolescents compared to the original 5-point scale. The relative contributions of reverse-worded and non-reverse-worded items in scale development for youth instruments are also discussed.

To obtain Food and Drug Administration approval for the treatment of cognitive impairments associated with schizophrenia, a drug will need to demonstrate benefits beyond those that may be documented on objective cognitive tests. Interview-based measures of cognition such as the Cognitive Assessment Interview (CAI) are candidate coprimary outcome measures.Psychiatrically stable schizophrenia outpatients (n=150) were studied using the CAI to obtain information about cognitive functioning from both the patient and an informant. Patients also received objective assessments of neurocognition, functional capacity, functional outcome, and symptoms, at baseline and 1 month later.The CAI had good internal consistency (Cronbach's alpha=.92) and good test-retest reliability (r=.83). The CAI was moderately correlated with objective neurocognitive test scores (r's=-.39 to -.41) and moderately correlated with social functioning (r=-.38), work functioning (r=-.48), and overall functional outcome (r=-.49). The correlations of CAI scores with external validity indicators did not differ significantly by source of information (patient alone ratings were valid). Overall functional outcome correlated more strongly with patient CAI scores (r=-.50) than with objective neurocognitive test scores (r=.29) or functional capacity (r=.29).Field testing of the CAI produced reliable ratings of cognitive functioning that were correlated with functional outcome. Patient ratings alone yielded scores with reliability and validity values appropriate for use in clinical trials. The CAI appears to provide useful complementary information and possesses practical advantages for rating cognitive functioning including an interview-based method of administration, brief assessment time (15 min for the patient assessment), little or no practice effects, and ease of scoring.

The Positive and Negative Affect Schedule for Children (PANAS-C/P; child and parent versions) yield positive affect (PA) and negative affect (NA) scales that are clinically useful for identifying youth with anxiety and mood problems. Despite the advantages that item response theory (IRT) offers relative to classical test theory with respect to shortening test instruments, no studies to date have applied IRT methodology to the PANAS-C/P scales. In the present study, we thus applied IRT methodology using a school-based development sample (child sample: N = 799; parent sample: N = 553) and developed a shortened 5-item PA scale (joyful, cheerful, happy, lively, proud) and a 5-item NA scale (miserable, mad, afraid, scared, sad) for the sake of simultaneously increasing the assessment efficiency of the PANAS-C/P scales while improving the psychometric properties of the scales. The reduced PA and NA child scales classified relevant diagnostic groups in a separate clinic-referred validation sample (N = 662) just as well as the original PANAS-C child scales and may be used to help identify youth with internalizing disorders in need of mental health services.

Using a school-based (N = 1,060) and clinic-referred (N = 303) youth sample, the authors developed a 25-item shortened version of the Revised Child Anxiety and Depression Scale (RCADS) using Schmid-Leiman exploratory bifactor analysis to reduce client burden and administration time and thus improve the transportability characteristics of this youth anxiety and depression measure. Results revealed that all anxiety items primarily reflected a single "broad anxiety" dimension, which informed the development of a reduced 15-item Anxiety Total scale. Although specific DSM-oriented anxiety subscales were not included in this version, the items comprising the Anxiety Total scale were evenly pulled from the 5 anxiety-related content domains from the original RCADS. The resultant 15-item Anxiety Total scale evidenced significant correspondence with anxiety diagnostic groups based on structured clinical interviews. The scores from the 10-item Depression Total scale (retained from the original version) were also associated with acceptable reliability in the clinic-referred and school-based samples (α = .80 and .79, respectively); this is in contrast to the alternate 5-item shortened RCADS Depression Total scale previously developed by Muris, Meesters, and Schouten (2002), which evidenced depression scores of unacceptable reliability (α = .63). The shortened RCADS developed in the present study thus balances efficiency, breadth, and scale score reliability in a way that is potentially useful for repeated measurement in clinical settings as well as wide-scale screenings that assess anxiety and depressive problems. These future applications are discussed, as are recommendations for continued use of exploratory bifactor modeling in scale development.

article i nfo Progress in the development of new pharmacological and psychosocial treatments for the negative symptoms of schizophrenia is impeded by limitations of available assessment instruments. The multi-site Collaboration to Advance Negative Symptom Assessment in Schizophrenia (CANSAS) was established to develop and validate a new clinical rating scale using a transparent, iterative, and data-driven process. The Clinical Assessment Interview for Negative Symptoms (CAINS) was designed to address limitations of existing measures and assess consensus-based sub-domains, including asociality, avolition, anhedonia, affective blunting, and alogia. The structure and psychometric properties of the CAINS were evaluated in a sample of 281 schizophrenia and schizoaffective outpatients at four sites. Converging structural analyses indicated that the scale was comprised of two moderately correlated factors — one reflecting experiential impairments (diminished motivation and enjoyment of social, vocational, and recreational activities) and one reflecting expressive impairments (diminished non-verbal and verbal communication). Item-level analyses revealed generally good distributional properties, inter-rater agreement, discriminating anchor points, and preliminary convergent and discriminant validity. Results indicate that the CAINS is a promising new measure for quantifying negative symptoms in clinical neuroscience and treatment studies. Results guided item modification or deletion, and the reliability and validity of the revised, shorter version of the CAINS is in the final phase of development within the CANSAS project. Published by Elsevier B.V.

This study explored the application of latent variable measurement models to the Social Anhedonia Scale (SAS; Eckblad, Chapman, Chapman, & Mishlove, 1982), a widely used and influential measure in schizophrenia-related research. Specifically, we applied unidimensional and bifactor item response theory (IRT) models to data from a community sample of young adults (n = 2,227). Ordinal factor analyses revealed that identifying a coherent latent structure in the 40-item SAS data was challenging due to (a) the presence of multiple small content clusters (e.g., doublets); (b) modest relations between those clusters, which, in turn, implies a general factor of only modest strength; (c) items that shared little variance with the majority of items; and (d) cross-loadings in bifactor solutions. Consequently, we conclude that SAS responses cannot be modeled accurately by either unidimensional or bifactor IRT models. Although the application of a bifactor model to a reduced 17-item set met with better success, significant psychometric and substantive problems remained. Results highlight the challenges of applying latent variable models to scales that were not originally designed to fit these models.

Attention deficit hyperactivity disorder (ADHD) is a disorder characterized by attentional difficulties. Mindfulness is a receptive attention to present experience. Both ADHD and mindfulness are associated with attention and personality. This study tests whether individuals with ADHD have lower mindfulness scores than controls and, if true, whether personality contributes to these differences. One hundred and five adults (half with ADHD) were assessed for mindfulness, using the Kentucky Inventory of Mindfulness Skills, and personality, using the Tridimensional Character Inventory. Individuals with ADHD report themselves as less mindful than non-ADHD controls and more novelty-seeking, less self-directed, and more self-transcendent. Mindfulness is negatively associated with ADHD and positively associated with self-directedness and self-transcendence. Analyses of subscales of mindfulness suggest that ADHD is associated most with the "Acting in Awareness" dimension, perhaps because of shared items reflecting attentional variability. The current findings support that a large portion of variability in trait mindfulness can be explained by ADHD status and personality traits of self-directedness and self-transcendence. It further suggests that interventions that increase mindfulness might improve symptoms of ADHD and increase self-directedness and/or self-transcendence.

The purpose of this study was to apply a set of rarely reported psychometric indices that, nevertheless, are important to consider when evaluating psychological measures. All can be derived from a standardized loading matrix in a confirmatory bifactor model: omega reliability coefficients, factor determinacy, construct replicability, explained common variance, and percentage of uncontaminated correlations. We calculated these indices and extended the findings of 50 recent bifactor model estimation studies published in psychopathology, personality, and assessment journals. These bifactor derived indices (most not presented in the articles) provided a clearer and more complete picture of the psychometric properties of the assessment instruments. We reached 2 firm conclusions. First, although all measures had been tagged "multidimensional," unit-weighted total scores overwhelmingly reflected variance due to a single latent variable. Second, unit-weighted subscale scores often have ambiguous interpretations because their variance mostly reflects the general, not the specific, trait. Finally, we review the implications of our evaluations and consider the limits of inferences drawn from a bifactor modeling approach.

© 2015 American Psychological Association.The measurement of treatment adherence (a component of treatment integrity defined as the extent to which a treatment is delivered as intended) is a critical element in treatment evaluation research. This article presents initial psychometric data for scores on the Cognitive-Behavioral Therapy Adherence Scale for Youth Anxiety (CBAY-A), an observational measure designed to be sensitive to common practice elements found in individual cognitive- behavioral therapy (ICBT) for youth anxiety. Therapy sessions (N = 954) from 1 efficacy and 1 effectiveness study of ICBT for youth anxiety were independently rated by 2 coders. Interrater reliability (as gauged by intraclass correlation coefficients) for the item scores averaged 0.77 (SD = 0.15; range 48 to 80). The CBAY-A item and scale (skills, model, total) scores demonstrated evidence of convergent and discriminant validity with an observational measure of therapeutic interventions and an observational measure of the alliance. The CBAY-A item and scale scores also discriminated between therapists delivering ICBT in research and practice settings and therapists delivering nonmanualized usual clinical care. We discuss the importance of replicating these psychometric findings in different samples and highlight possible application of an adherence measure in testing integrity-outcome relations.

© 2015 APA, all rights reserved). Visuospatial processing is a commonly assessed neurocognitive domain with deficits linked to dysfunction in right posterior regions of the brain. With the growth of large-scale clinical research studies, there is an increased need for efficient and scalable assessments of neurocognition, including visuospatial processing. The purpose of the current study was to use a novel method that combines item response theory (IRT) and computerized adaptive testing (CAT) approaches to create an abbreviated form of the computerized Penn Line Orientation Test (PLOT). The 24-item PLOT was administered to 8,498 youths (aged 8-21 years) as part of the Philadelphia Neurodevelopmental Cohort study and, by Web-based data collection, in an independent sample of 4,593 adults from Great Britain as part of a TV documentary. IRT-based CAT simulations were used to select the best PLOT items for an abbreviated form by performing separate simulations in each group and choosing only items that were selected as useful (i.e., high item discrimination and in the appropriate difficulty range) in at least 1 of the simulations. Fifteen items were chosen for the final, short form of the PLOT, indicating substantial agreement among the models in how they evaluated each item's usefulness. Moreover, this abbreviated version performed comparably to the full version in tests of sensitivity to age and sex effects. This abbreviated version of the PLOT cuts administration time by 50% without detectable loss of information, which points to its feasibility for large-scale clinical and genomic studies. (PsycINFO Database Record

Research on empathy in schizophrenia has relied on dated self-report scales that do not conform to contemporary social neuroscience models of empathy. The current study evaluated the structure and correlates of the recently-developed Questionnaire of Cognitive and Affective Empathy (QCAE) in schizophrenia. This measure, whose structure and validity was established in healthy individuals, includes separate scales to assess the two main components of empathy: Cognitive Empathy (assessed by two subscales) and Affective Empathy (assessed by three subscales). Stable outpatients with schizophrenia (n = 145) and healthy individuals (n = 45) completed the QCAE, alternative measures of empathy, and assessments of clinical symptoms, neurocognition, and functional outcome. Exploratory and confirmatory factor analyses provided consistent support for a two-factor solution in the schizophrenia group, justifying the use of separate cognitive and affective empathy scales in this population. However, one of the three Affective Empathy subscales was not psychometrically sound and was excluded from further analyses. Patients reported significantly lower Cognitive Empathy but higher Affective Empathy than controls. Among patients, the QCAE scales showed significant correlations with an alternative self-report empathy scale, but not with performance on an empathic accuracy task. The QCAE Cognitive Empathy subscales also showed significant, though modest, correlations with negative symptoms and functional outcome. These findings indicate that structure of self-reported empathy is similar in people with schizophrenia and healthy subjects, and can be meaningfully compared between groups. They also contribute to emerging evidence that some aspects of empathy may be intact or hyper-responsive in schizophrenia.

A major barrier to developing treatments for negative symptoms has been measurement concerns with existing assessment tools. Fulfilling the top recommendation of the National Institute of Mental Health's Consensus Development Conference on Negative Symptoms, the Clinical Assessment Interview for Negative Symptoms (CAINS) was developed using an iterative, empirical approach, and includes items assessing motivation, pleasure, and emotion expression. The authors employed multiple analytic techniques to develop the CAINS and here provide final development and validation results.The CAINS structure, interrater agreement, test-retest reliability, and convergent and discriminant validity were assessed in a large and diverse sample of 162 outpatients with schizophrenia or schizoaffective disorder recruited from four sites.Three items with poor psychometric properties were removed, resulting in a 13-item CAINS. The CAINS factor structure was replicated, demonstrating two modestly correlated scales: expression (four items) and motivation/pleasure (nine items). The scales demonstrated good internal consistency, test-retest stability, and interrater agreement. Strong convergent validity was demonstrated by linkages with other negative symptom measures, self-report scales of sociality, pleasure, and motivation, and coded facial expressions. Discriminant validity was shown by independence from depression, medication side effects, and cognition. Notably, the CAINS scales were related to real-world vocational, independent living, and social/familial functioning.The CAINS is an empirically developed and evaluated measure of negative symptoms. Findings indicate that the CAINS is brief yet comprehensive and employable across a wide range of research and clinical contexts.

The construct validity of the Psychopathy Q-sort (PQS; Reise & Oliver, 1994) was investigated by correlating it with an array of observer-based and self-report personality measures in a sample of 350 men and women assessed at the Institute of Personality Assessment and Research (IPAR). High discriminant validity of the PQS was indicated by a pattern of significant correlations with self-report measures of the Cluster B personality disorder scales of the Diagnostic and Statistical Manual of Mental Disorders (3rd ed., rev.; American Psychiatric Association, 1980) and the absence of correlations with scales belonging to Clusters A and C of the manual. The PQS also correlated positively with the California Psychological Inventory measures of social poise and assurance and correlated negatively with measures of normative control of impulse. Gender differences in psychopathy were explored using Adjective Check List (Gough & Heilbrun, 1983) ratings provided by IPAR staff observers and by 76 pairs of spouses.

This study sought to resolve the dimensionality of the Loneliness Questionnaire (LQ; Asher, Hymel, & Renshaw, 1984) by applying recommended confirmatory factor analytic procedures that control for method effects (Brown, 2003). This study was needed given that inconsistent findings have been reported recently regarding the structure of this instrument (Bagner, Storch, & Roberti, 2004) and all models to date have not accounted for method effects due to the non-reversed-worded and reversed-worded items of this instrument. Using a large sample of youth in Grades 2 through 12 (N = 11,725), we compared the previously reported 1- and 2-factor models with a newly posited 1-factor model that incorporated correlated error terms to account for method effects. We found that the 1-factor model that included correlated error terms fit the data best, and that this factor structure evidenced measurement invariance across boys and girls in childhood, but not in adolescence. The meaning of the LQ indicators was also consistent for boys across development, but evidenced differences for girls in childhood versus adolescence. More generally, it was demonstrated that modeling method effects is vital to accurately understanding the dimensionality of loneliness when reversed-worded and non-reversed-worded items are used as indicators. The measurement and clinical implications of these findings are discussed.

The authors report on the development and calibration of item banks for depression, anxiety, and anger as part of the Patient-Reported Outcomes Measurement Information System (PROMIS®). Comprehensive literature searches yielded an initial bank of 1,404 items from 305 instruments. After qualitative item analysis (including focus groups and cognitive interviewing), 168 items (56 for each construct) were written in a first person, past tense format with a 7-day time frame and five response options reflecting frequency. The calibration sample included nearly 15,000 respondents. Final banks of 28, 29, and 29 items were calibrated for depression, anxiety, and anger, respectively, using item response theory. Test information curves showed that the PROMIS item banks provided more information than conventional measures in a range of severity from approximately −1 to +3 standard deviations (with higher scores indicating greater distress). Short forms consisting of seven to eight items provided information comparable to legacy measures containing more items.

A psychometric analysis of 2 interview-based measures of cognitive deficits was conducted: the 21-item Clinical Global Impression of Cognition in Schizophrenia (CGI-CogS; Ventura et al., 2008), and the 20-item Schizophrenia Cognition Rating Scale (SCoRS; Keefe et al., 2006), which were administered on 2 occasions to a sample of people with schizophrenia. Traditional psychometrics, bifactor analysis, and item response theory methods were used to explore item functioning and dimensionality and to compare instruments. Despite containing similar item content, responses to the CGI-CogS demonstrated superior psychometric properties (e.g., higher item intercorrelations, better spread of ratings across response categories) relative to the SCoRS. The authors argue that these differences arise mainly from the differential use of prompts and how the items are phrased and scored. Bifactor analysis demonstrated that although both measures capture a broad range of cognitive functioning (e.g., working memory, social cognition), the common variance on each is overwhelmingly explained by a single general factor. Item response theory analyses of the combined pool of 41 items showed that measurement precision is peaked in the mild to moderate range of cognitive impairment. Finally, simulated adaptive testing revealed that only about 10 to 12 items are necessary to achieve latent trait level estimates with reasonably small standard errors for most individuals. This suggests that these interview-based measures of cognitive deficits could be shortened without loss of measurement precision.

The application of psychological measures often results in item response data that arguably are consistent with both unidimensional (a single common factor) and multidimensional latent structures (typically caused by parcels of items that tap similar content domains). As such, structural ambiguity leads to seemingly endless "confirmatory" factor analytic studies in which the research question is whether scale scores can be interpreted as reflecting variation on a single trait. An alternative to the more commonly observed unidimensional, correlated traits, or second-order representations of a measure's latent structure is a bifactor model. Bifactor structures, however, are not well understood in the personality assessment community and thus rarely are applied. To address this, herein we (a) describe issues that arise in conceptualizing and modeling multidimensionality, (b) describe exploratory (including Schmid-Leiman [Schmid & Leiman, 1957] and target bifactor rotations) and confirmatory bifactor modeling, (c) differentiate between bifactor and second-order models, and (d) suggest contexts where bifactor analysis is particularly valuable (e.g., for evaluating the plausibility of subscales, determining the extent to which scores reflect a single variable even when the data are multidimensional, and evaluating the feasibility of applying a unidimensional item response theory (IRT) measurement model). We emphasize that the determination of dimensionality is a related but distinct question from either determining the extent to which scores reflect a single individual difference variable or determining the effect of multidimensionality on IRT item parameter estimates. Indeed, we suggest that in many contexts, multidimensional data can yield interpretable scale scores and be appropriately fitted to unidimensional IRT models.

In this review, we examine studies that use item response theory (IRT) to explore the psychometric properties of clinical measures. Next, we consider how IRT has been used in clinical research for: scale linking, computerized adaptive testing, and differential item functioning analysis. Finally, we consider the scale properties of IRT trait scores. We conclude that there are notable differences between cognitive and clinical measures that have relevance for IRT modeling. Future research should be directed toward a better understanding of the metric of the latent trait and the psychological processes that lead to individual differences in item response behaviors.

An instrument's sensitivity to detect individual-level change is an important consideration for both psychometric and clinical researchers. In this article, we develop a cognitive problems measure and evaluate its sensitivity to detect change from an item response theory (IRT) perspective. After illustrating assumption checking and model fit assessment, we detail 4 features of IRT modeling: (a) the scale information curve and its relation to the bandwidth of measurement precision, (b) the scale response curve and how it is used to link the latent trait metric with the raw score metric, (c) content-based versus norm-based score referencing, and (d) the level of measurement of the latent trait scale. We conclude that IRT offers an informative, alternative framework for understanding an instrument's psychometric properties and recommend that IRT analyses be considered prior to investigations of change, growth, or the effectiveness of clinical interventions.

The factor analysis of repeated measures psychiatric data presents interesting challenges for researchers in terms of identifying the latent structure of an assessment instrument. Specifically, repeated measures contain both within and between individual sources of variance. Although a number of techniques exist for separating out these 2 sources of variance, all are problematic. Recently, researchers have proposed that exploratory multilevel factor analysis (MFA) be used to appropriately analyze the latent structure of repeated measures data. The chief objective of this report is to provide a didactic step-by-step guide on how MFA may be applied to psychiatric data. In the discussion, we describe difficulties associated with MFA and consider challenges in factor analyzing life event appraisals in psychiatric samples.

Item response theory (IRT) methods are used by large testing firms, state agencies, and school districts to construct, analyze, and score most major aptitude, achievement, proficiency, entrance, and professional licensure exams. Personality assessment, in contrast, has not generally adopted these more powerful, modern psychometric techniques. We evaluate the possible role of IRT in the personality domain by highlighting key areas in which IRT and traditional methods differ. Although we conclude that IRT has a significant role to play in future personality measurement, there are many systemic and technical barriers to its routine application.

This study asks, how well does an item response theory (IRT) based computerized adaptive NEO PI-R work? To explore this question, real-data simulations (N = 1,059) were used to evaluate a maximum information item selection computerized adaptive test (CAT) algorithm. Findings indicated satisfactory recovery of full-scale facet scores with the administration of around four items per facet scale. Thus, the NEO PI-R could be reduced in half with little loss in precision by CAT administration. However, results also indicated that the CAT algorithm was not necessary. We found that for many scales, administering the "best" four items per facet scale would have produced similar results. In the conclusion, we discuss the future of computerized personality assessment and describe the role IRT methods might play in such assessments.

The primary purposes of the present study were to use the Q-sort method to develop a measure of alexithymia and to locate the construct within a two-dimensional (ego-control and ego-resiliency) model of personality. Thirteen professional judges described the characteristics of the alexithymic personality with the 100-item California Q-set. Scores from the sorts were aggregated to form the Alexithymia Prototype, which had a Spearman-Brown reliability of 0.99. Alexithymic people were described as having difficulties experiencing and expressing emotion, lacking imagination, and being literal, socially conforming, and utilitarian; they lack insight, are humorless, and experience meaninglessness; and anxiety and tension find outlet in bodily symptoms. This description is consistent, for the most part, with modern formulations of the alexithymia construct. In the language of the two-dimensional personality model, alexithymic individuals appear to be overcontrolling and lacking ego-resiliency (i.e., constricted, anxious, rigid, and withdrawn). We, therefore, compared the Alexithymia Prototype with two independently developed prototypes, Overcontrol and Ego-Resiliency. The Q-correlations between alexithymia and overcontrol and between alexithymia and ego-resiliency were 0.45 and -0.70, respectively. Although item analyses confirmed moderate overlap between alexithymia and overcontrol and considerable overlap between alexithymia and lacking ego-resiliency (ego-brittle), item differences suggest that alexithymia, indeed, is a unique personality construct.

The 20-item Toronto Alexithymia Scale (TAS-20) purportedly measures 3 intercorrelated dimensions of the alexithymia construct: (a) difficulties identifying feelings (DIF), (b) difficulties describing feelings (DDF), and (c) externally oriented thinking (EOT). The primary purpose of this study was to evaluate the factor structure of the TAS-20 with full-information item factor analysis in 2 diverse samples: medical students (n = 219) and psychiatric (psychoactive substance dependent-abusing) inpatients (n = 204). In each group, we estimated the parameters of 1-, 2-, 3-, and 4-factor models. None of the various factor solutions resulted in clean, simple structures in either sample. In the student sample, the data were best represented (although not well) by a 3-dimensional model: DIF, DDF, and EOT. In the psychiatric sample, however, the 3-dimensional solution was quite different; DIF and DDF formed 1 "emotional awareness deficits" factor, and EOT split into 2 unrelated factors. We concluded that alexithymia, as measured by the TAS-20, is multidimensional and not well-represented by a global severity score. Our recommendations include checking the factor structure of the TAS-20 when practicable, computing subscale scores, and exercising caution in interpreting TAS-20 EOT scores, particularly in psychoactive substance dependent-abusing inpatient samples.