Search

The reliable change index has been used to evaluate the significance of individual change in health-related quality of life. We estimate reliable change for two measures (physical function and emotional distress) in the Patient-Reported Outcomes Measurement Information System (PROMIS®) 29-item health-related quality of life measure (PROMIS-29 v2.1). Using two waves of data collected 3 months apart in a longitudinal observational study of chronic low back pain and chronic neck pain patients receiving chiropractic care, and simulations, we compare estimates of reliable change from classical test theory fixed standard errors with item response theory standard errors from the graded response model. We find that unless true change in the PROMIS physical function and emotional distress scales is substantial, classical test theory estimates of significant individual change are much more optimistic than estimates of change based on item response theory. Supplementary Information The online version supplementary material available at 10.1007/s11336-021-09774-1.

Objective.The National Neuropsychology Network (NNN) is a multicenter clinical research initiative funded by the National Institute of Mental Health (NIMH; R01 MH118514) to facilitate neuropsychology’s transition to contemporary psychometric assessment methods with resultant improvement in test validation and assessment efficiency.Method:The NNN includes four clinical research sites (Emory University; Medical College of Wisconsin; University of California, Los Angeles (UCLA); University of Florida) and Pearson Clinical Assessment. Pearson Q-interactive (Q-i) is used for data capture for Pearson published tests; web-based data capture tools programmed by UCLA, which serves as the Coordinating Center, are employed for remaining measures.Results:NNN is acquiring item-level data from 500–10,000 patients across 47 widely used Neuropsychology (NP) tests and sharing these data via the NIMH Data Archive. Modern psychometric methods (e.g., item response theory) will specify the constructs measured by different tests and determine their positive/negative predictive power regarding diagnostic outcomes and relationships to other clinical, historical, and demographic factors. The Structured History Protocol for NP (SHiP-NP) helps standardize acquisition of relevant history and self-report data.Conclusions:NNN is a proof-of-principle collaboration: by addressing logistical challenges, NNN aims to engage other clinics to create a national and ultimately an international network. The mature NNN will provide mechanisms for data aggregation enabling shared analysis and collaborative research. NNN promises ultimately to enable robust diagnostic inferences about neuropsychological test patterns and to promote the validation of novel adaptive assessment strategies that will be more efficient, more precise, and more sensitive to clinical contexts and individual/cultural differences.

Major obstacles to data harmonization in neuropsychology include lack of consensus about what constructs and tests are most important and invariant across healthy and clinical populations. This study addressed these challenges using data from the National Neuropsychology Network (NNN).Data were obtained from 5,000 NNN participants and Pearson standardization samples. Analyses included variables from four instruments: Wechsler Adult Intelligence Scale, 4th Edition (WAIS-IV); Wechsler Memory Scale, 4th Edition (WMS-IV); California Verbal Learning Test, 3rd Edition (CVLT3); and Delis-Kaplan Executive Function System (D-KEFS). We used confirmatory factor analysis to evaluate models suggested by prior work and examined fit statistics and measurement invariance across samples. We examined relations of factor scores to demographic and clinical characteristics.For each instrument, we identified four first-order and one second-order factor. Optimal models in patients generally paralleled the best-fitting models in the standardization samples, including task-specific factors. Analysis of the NNN data prompted specification of a Recognition-Familiarity factor on the WMS-IV and an Inhibition-Switching factor on the D-KEFS. Analyses showed strong to strict factorial invariance across samples with expected differences in factor means and variances. The Recognition-Familiarity factor correlated with age more strongly in NNN than in the standardization sample.Factor models derived from healthy groups generally fit well in patients. NNN data helped identify novel Recognition-Familiarity and Inhibition-Switching factors that were also invariant across samples and may be clinically useful. The findings support efforts to identify evidence-based and optimally efficient measurements of neuropsychological constructs that are valid across groups. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Large-scale studies spanning diverse project sites, populations, languages, and measurements are increasingly important to relate psychological to biological variables. National and international consortia already are collecting and executing mega-analyses on aggregated data from individuals, with different measures on each person. In this research, we show that Asparouhov and Muthén’s alignment method can be adapted to align data from disparate item sets and response formats. We argue that with these adaptations, the alignment method is well suited for combining data across multiple sites even when they use different measurement instruments. The approach is illustrated using data from the Whole Genome Sequencing in Psychiatric Disorders consortium and a real-data-based simulation is used to verify accurate parameter recovery. Factor alignment appears to increase precision of measurement and validity of scores with respect to external criteria. The resulting parameter estimates may further inform development of more effective and efficient methods to assess the same constructs in prospectively designed studies.

As part of a scale development project, we fit a nominal response item response theory model to responses to the Health Care Engagement Measure (HEM). When using the original 5-point response format, categories were not ordered as intended for six of the 23 items. For the remaining, the category boundary discrimination between Categories 0 ( not at all true) and 1 ( a little bit true) was only weakly discriminating, suggesting uninformative categories. When the lowest two categories were collapsed, psychometric properties improved greatly. Category boundary discriminations within items, however, varied significantly. Specifically, higher response category distinctions, such as responding 3 ( very true) versus 2 ( mostly true) were considerably more discriminating than lower response category distinctions. Implications for HEM scoring and for improving measurement precision at lower levels of the construct are presented as is the unique role of the nominal response model in category analysis.

Item response theory (IRT) model applications extend well beyond cognitive ability testing, and various patient-reported outcomes (PRO) measures are among the more prominent examples. PRO (and like) constructs differ from cognitive ability constructs in many ways, and these differences have model fitting implications. With a few notable exceptions, however, most IRT applications to PRO constructs rely on traditional IRT models, such as the graded response model. We review some notable differences between cognitive and PRO constructs and how these differences can present challenges for traditional IRT model applications. We then apply two models (the traditional graded response model and an alternative log-logistic model) to depression measure data drawn from the Patient-Reported Outcomes Measurement Information System project. We do not claim that one model is “a better fit” or more “valid” than the other; rather, we show that the log-logistic model may be more consistent with the construct of depression as a unipolar phenomenon. Clearly, the graded response and log-logistic models can lead to different conclusions about the psychometrics of an instrument and the scaling of individual differences. We underscore, too, that, in general, explorations of which model may be more appropriate cannot be decided only by fit index comparisons; these decisions may require the integration of psychometrics with theory and research findings on the construct of interest. Supplementary Information The online version supplementary material available at 10.1007/s11336-021-09802-0.

PURPOSE: Healthcare engagement is a core measurement target for efforts to improve healthcare systems. This construct is broadly defined as the extent to which healthcare services represent collaborative partnerships with patients. Previous qualitative work operationalized healthcare engagement as generalized self-efficacy in four related subdomains: self-management, collaborative communication, health information use, and healthcare navigation. Building on this work, our objective was to establish a healthcare engagement instrument that is sufficiently unidimensional to yield a single score.METHOD: We conducted cognitive interviews followed by a nation-wide mail survey of US Veteran Administration (VA) healthcare users. Data were collected on 49 candidate healthcare engagement items, as well as measures of self-efficacy for managing symptoms, provider communication, and perceived access. Items were subjected to exploratory bifactor, statistical learning, and IRT analyses.RESULTS: Cognitive interviews were completed by 56 patients and 9552 VA healthcare users with chronic conditions completed the mail survey. Participants were mostly white and male but with sizable minority participation. Psychometric analyses and content considerations reduced the item pool to 23 items, which demonstrated a strong general factor (OmegaH of .89). IRT analyses revealed a high level of reliability across the trait range and little DIF across groups. Most health information use items were removed during analyses, suggesting a more independent role for this domain.CONCLUSION: We provide quantitative evidence for a relatively unidimensional measure of healthcare engagement. Despite developed with VA healthcare users, the measure is intended for general use. Future work includes short-form development and validation with other patient groups.

Post-traumatic stress disorder (PTSD) can develop after exposure to traumatic events and severely impacts the quality of life. PTSD is frequently comorbid with substance use disorders, with alcoholism being particularly common. However, not everyone who experiences trauma develops PTSD and the factors that render individuals susceptible or resilient to the effects of stress are unknown although gender appears to play an important role. Rodent models of stress exposure such as stress-enhanced fear learning (SEFL) recapitulate some aspects of PTSD symptomology, making them an invaluable tool for studying this disorder. This study examined whether exposure to a modified version of the SEFL procedure (4 footshocks instead of the standard 15 over 90min) would reveal both susceptible and resilient subjects. Following stress exposure, distinct susceptible and resilient groups emerged that differed in fear learning and anxiety-related behavior as well as voluntary alcohol intake. Some aspects of stress susceptibility manifested differently in males compared to females, with susceptibility associated with increased alcohol intake in males and increased baseline anxiety in females.

One of the most widely used measures of psychosis-related symptoms and characteristics is the 74-item Schizotypal Personality Questionnaire (SPQ). Using multidimensional Item Response Theory (bifactor 2-parameter model), we calibrated SPQ items in a sample of 375 youths aged 9–24 years and constructed a fully functional computerized adaptive form of the SPQ on an open-source platform for public use. To assess validity, we used the above parameters to simulate CAT sessions in a separate validation sample (N = 100) using three test-length-based stopping rules: 8 items, 16 items, and 32 items. Those scores were then compared to full-form and SPQ-Brief scores on their abilities to predict psychosis or clinical risk status. Areas under the receiver operating characteristic curves indicated mediocre predictive ability, but did not differ among any of the forms, even when only eight adaptive items were administered. The Youden index for the 16-item adaptive version was higher than that for the 22-item SPQ-Brief. Classification accuracy for the full SPQ was 73% compared to 66% for the both the SPQ-Brief and adaptive versions (average of three stopping rules). The SPQ-CAT shows promise as a much shorter but valid assessment of schizotypy which can save time with minimal loss of information.

Author(s): Anderson, Ariana E; Marder, Stephen; Reise, Steven P; Savitz, Adam; Salvadore, Giacomo; Fu, Dong Jing; Li, Qingqin; Turkoz, Ibrahim; Han, Carol; Bilder, Robert M | Abstract: Objective:Common genetic variation spans schizophrenia, schizoaffective and bipolar disorders, but historically, these syndromes have been distinguished categorically. A symptom dimension shared across these syndromes, if such a general factor exists, might provide a clearer target for understanding and treating mental illnesses that share core biological bases. Method:We tested the hypothesis that a bifactor model of the Positive and Negative Syndrome Scale (PANSS), containing 1 general factor and 5 specific factors (positive, negative, disorganized, excited, anxiety), explains the cross-diagnostic structure of symptoms better than the traditional 5-factor model, and examined the extent to which a general factor reflects the overall severity of symptoms spanning diagnoses in 5094 total patients with a diagnosis of schizophrenia, schizoaffective, and bipolar disorder. Results:The bifactor model provided superior fit across diagnoses, and was closer to the "true" model, compared to the traditional 5-factor model (Vuong test; P l .001). The general factor included high loadings on 28 of the 30 PANSS items, omitting symptoms associated with the excitement and anxiety/depression domains. The general factor had highest total loadings on symptoms that are often associated with the positive and disorganization syndromes, but there were also substantial loadings on the negative syndrome thus leading to the interpretation of this factor as reflecting generalized psychosis. Conclusions:A bifactor model derived from the PANSS can provide a stronger framework for measuring cross-diagnostic psychopathology than a 5-factor model, and includes a generalized psychosis dimension shared at least across schizophrenia, schizoaffective, and bipolar disorder.

Background About 70% of Latinos, 5 years old or older, in the United States speak Spanish at home. Measurement equivalence of the PROMIS® pain interference (PI) item bank by language of administration (English versus Spanish) has not been evaluated.

Traditional "paper-and-pencil" testing is imprecise in measuring speed and hence limited in assessing performance efficiency, but computerized testing permits precision in measuring itemwise response time. We present a method of scoring performance efficiency (combining information from accuracy and speed) at the item level. Using a community sample of 9,498 youths age 8-21, we calculated item-level efficiency scores on 4 neurocognitive tests, and compared the concurrent, convergent, discriminant, and predictive validity of these scores with simple averaging of standardized speed and accuracy-summed scores. Concurrent validity was measured by the scores' abilities to distinguish men from women and their correlations with age; convergent and discriminant validity were measured by correlations with other scores inside and outside of their neurocognitive domains; predictive validity was measured by correlations with brain volume in regions associated with the specific neurocognitive abilities. Results provide support for the ability of itemwise efficiency scoring to detect signals as strong as those detected by standard efficiency scoring methods. We find no evidence of superior validity of the itemwise scores over traditional scores, but point out several advantages of the former. The itemwise efficiency scoring method shows promise as an alternative to standard efficiency scoring methods, with overall moderate support from tests of 4 different types of validity. This method allows the use of existing item analysis methods and provides the convenient ability to adjust the overall emphasis of accuracy versus speed in the efficiency score, thus adjusting the scoring to the real-world demands the test is aiming to fulfill. (PsycINFO Database Record

Analytic bifactor rotations (Jennrich & Bentler, 2011, 2012) have been recently developed and made generally available, but are not well understood. The Jennrich-Bentler analytic bifactor rotations (bi-quartimin and bi-geomin) are an alternative to, and arguably an improvement upon, the less technically sophisticated Schmid-Leiman orthogonalization (Schmid & Leiman, 1957). We review the technical details that underlie the Schmid-Leiman and Jennrich-Bentler bifactor rotations, using simulated data structures to illustrate important features and limitations. For the Schmid-Leiman, we review the problem of inaccurate parameter estimates caused by the linear dependencies, sometimes called “proportionality constraints,” that are required to expand a p correlated factors solution into a (p+1) (bi)factor space. We also review the complexities involved when the data depart from perfect cluster structure (e.g., item cross-loading on group factors). For the Jennrich-Bentler rotations, we describe problems in parameter estimation caused by departures from perfect cluster structure. In addition, we illustrate the related problems of: (a) solutions that are not invariant under different starting values (i.e., local minima problems); and, (b) group factors collapsing onto the general factor. Recommendations are made for substantive researchers including examining all local minima and applying multiple exploratory techniques in an effort to identify an accurate model.

OBJECTIVE: This article reviews current approaches to neuropsychological assessment, identifies opportunities for development of new methods using modern psychometric theory and advances in technology, and suggests a transition path that promotes application of novel methods without sacrificing validity. METHODS: Theoretical/state-of-the-art review. CONCLUSIONS: Clinical neuropsychological assessment today does not reflect advances in neuroscience, modern psychometrics, or technology. Major opportunities for improving practice include both psychometric and technological strategies. Modern psychometric approaches including item response theory (IRT) enable: linking procedures that can place different measures on common scales; adaptive testing algorithms that can dramatically increase efficiency of assessment; examination of differential item functioning (DIF) to detect measures that behave differently in different groups; and person fit statistics to detect aberrant patterns of responding of high value for performance validity testing. Opportunities to introduce novel technologies include computerized adaptive testing, Web-based assessment, healthcare- and bio-informatics strategies, mobile platforms, wearables, and the “internet-of-things.” To overcome inertia in current practices, new methods must satisfy requirements for back-compatibility with legacy instrumentation, enabling us to leverage the wealth of validity data already accrued for classic procedures. A path to achieve these goals involves creation of a global network to aggregate item-level data into a shared repository that will enable modern psychometric analyses to refine existing methods, and serve as a platform to evolve novel assessment strategies, which over time can revolutionize neuropsychological assessment practices world-wide.

The recent incorporation of genomics into clinical neuroscience has generated an unprecedented need for behavioral assessment of domains that can be linked to brain systems and serve as “biomarkers” of psychopathology (Insel & Cuthbert, 2009). The scale of such studies requires efficient computerized testing of a broad range of abilities using tests that have been validated with functional neuroimaging. Such “neurobehavioral probes” (Gur, Erwin and Gur, 1992) have been assembled into a computerized neurocognitive battery (CNB) in which these tasks have been adapted to assure adequate psychometric properties, such as reliability and validity (Gur et al., 2001, 2010) and its linkage to brain systems (Roalf et al., 2013). The CNB has been applied in large-scale genomic studies (Aliyu et al., 2006; Greenwood et al., 2007; Gur et al., 2007a,b; Almasy et al., 2008), treatment research (Gur et al., 2001b; Grant et al., 2013), and the military (Thomas et al., 2013). Although its individual tests’ scores have been largely reliable (and their inferences valid), the overall latent structure of the battery has not been sufficiently evaluated. Using confirmatory and exploratory factor analysis, we aimed to examine the factor structure of the CNB, which is of paramount importance in justifying how the instrument is scored and interpreted. The CNB currently comprises fourteen tests grouped into five domains of neurobehavioral function. These broad domains were selected because they represent well-established brain systems, and each test in the battery was used in functional neuroimaging studies to probe aspects of these domains (Roalf et al., 2013). There are three tests in each of four domains measuring “Executive Control,” “Episodic Memory,” “Complex Cognition,” and “Social Cognition” as summarized in Table 1. A fifth domain (Sensorimotor speed) was measured by two tests, but the current analysis examined accuracy and these tests only provide speed measures and were therefore not included. Here we examined the factorial structure of the CNB in a sample of over 9,000 individuals age 8 to 21 who were administered the CNB as part of their participation in the Philadelphia Neurodevelopmental Cohort (PNC; Gur et al., 2012, 2013). Table 1 Domain-Specific scales of the Computerized Neurocognitive Battery, by neurobehavioral function

© 2015, Routledge. All rights reserved. In structural equation modeling (SEM), researchers need to evaluate whether item response data, which are often multidimensional, can be modeled with a unidimensional measurement model without seriously biasing the parameter estimates. This issue is commonly addressed through testing the fit of a unidimensional model specification, a strategy previously determined to be problematic. As an alternative to the use of fit indexes, we considered the utility of a statistical tool that was expressly designed to assess the degree of departure from unidimensionality in a data set. Specifically, we evaluated the ability of the DETECT “essential unidimensionality” index to predict the bias in parameter estimates that results from misspecifying a unidimensional model when the data are multidimensional. We generated multidimensional data from bifactor structures that varied in general factor strength, number of group factors, and items per group factor; a unidimensional measurement model was then fit and parameter bias recorded. Although DETECT index values were generally predictive of parameter bias, in many cases, the degree of bias was small even though DETECT indicated significant multidimensionality. Thus we do not recommend the stand-alone use of DETECT benchmark values to either accept or reject a unidimensional measurement model. However, when DETECT was used in combination with additional indexes of general factor strength and group factor structure, parameter bias was highly predictable. Recommendations for judging the severity of potential model misspecifications in practice are provided.

Several studies have indicated that bi–factor models fit a broad range of psychometric data better than alternative multidimensional models such as second–order models, e.g Rodriguez, Reise and Haviland (2016), Gignac (2016), and Carnivez (2016). Murray and Johnson (2013) and Gignac (2016) argue that this phenomenon is partially due to un–modeled complexities (e.g. un–modeled cross-factor loadings) that induce a bias in standard statistical measures that favors bi–factor models over second–order models. We extend the Murray and Johnson simulation studies to show how the ability to distinguish second–order and bi–factor models diminishes as the amount of un–modeled complexity increases. By using theorems about rank constraints on the covariance matrix to find sub–models of measurement models that have less un–modeled complexity, we are able to reduce the statistical bias in favor of bi–factor models; this allows researchers to reliably distinguish between bi-factor and second-order models.

Although the Positive and Negative Syndrome Scale (PANSS) was developed for use in schizophrenia (SZ), antipsychotic drug trials use the PANSS to measure symptom change also for bipolar (BP) and schizoaffective (SA) disorder, extending beyond its original indications. If the dimensions measured by the PANSS are different across diagnoses, then the same score change for the same drug condition may have different meanings depending on which group is being studied. Here, we evaluated whether the factor structure in the PANSS was consistent across schizophrenia (n = 3647), bipolar disorder (n = 858), and schizoaffective disorder (n = 592). Along with congruency coefficients, Hancock's H, and Jaccard indices, we used target rotations and statistical tests of invariance based on confirmatory factor models. We found the five symptom dimensions measured by the 30-item PANSS did not generalize well to schizoaffective and bipolar disorders. A model based on an 18-item version of the PANSS generalized better across SZ and BP groups, but significant problems remained in generalizing some of the factors to the SA sample. Schizophrenia and bipolar disorder showed greater similarity in factor structure than did schizophrenia and schizoaffective disorder. The Anxiety/Depression factor was the most consistent across disorders, while the Positive factor was the least consistent.

It is generally assumed that the latent trait is normally distributed in the population when estimating logistic item response theory (IRT) model parameters. This assumption requires that the latent trait be fully continuous and the population homogenous (i.e., not a mixture). When this normality assumption is violated, models are misspecified, and item and person parameter estimates are inaccurate. When normality cannot be assumed, it might be appropriate to consider alternative modeling approaches: (a) a zero-inflated mixture, (b) a log-logistic, (c) a Ramsay curve, or (d) a heteroskedastic-skew model. The first 2 models were developed to address modeling problems associated with so-called quasi-continuous or unipolar constructs, which apply only to a subset of the population, or are meaningful at one end of the continuum only. The second 2 models were developed to address non-normal latent trait distributions and violations of homogeneity of error variance, respectively. To introduce these alternative IRT models and illustrate their strengths and weaknesses, we performed real data application comparing results to those from a graded response model. We review both statistical and theoretical challenges in applying these models and choosing among them. Future applications of these and other alternative models (e.g., unfolding, diffusion) are needed to advance understanding about model choice in particular situations.

OBJECTIVES: To describe the development and initial psychometric properties of the observer-rated Global Therapist Competence Scale for Youth Psychosocial Treatment (G-COMP) in the context of cognitive-behavioral treatment (CBT) for youth anxiety disorders. METHOD: Independent coders rated 744 sessions from a sample of 68 youth (M age = 10.56 years) using the G-COMP and observational instruments of alliance, involvement, CBT adherence, CBT competence. RESULTS: Inter-rater reliability coefficients (ICCs [2,2]) were greater than .60 for the five G-COMP domain scores. G-COMP scores yielded small to medium correlations with instruments of the child-therapist alliance (rs = .17 - .44) and youth involvement in treatment (rs = .08 - .53), and medium to large correlations with instruments of CBT competence and adherence (rs = .26 - .63). Therapists in the research setting were rated higher compared to newly trained therapists in community clinics. CONCLUSIONS: Preliminary reliability and validity of the G-COMP scores are promising, but future research is needed with non-CBT samples.

BackgroundThe NIH-sponsored Patient-Reported Outcomes Measurement Information System (PROMIS) Gastrointestinal (GI) Symptoms scales were developed to assess patients' GI symptoms in clinical settings.AimsTo assess responsiveness to change and provide minimally important difference (MID) estimates for the PROMIS GI Symptoms scales.MethodsA sample of 256 GI outpatients self-administered the eight PROMIS GI Symptoms scales (gastroesophageal reflux, disrupted swallowing, diarrhea, bowel incontinence/soilage, nausea and vomiting, constipation, belly pain, and gas/bloating/flatulence) at two visits. Patient self-reported and physician-reported assessments of the subjects' overall GI condition were employed as change anchors. In addition, we prospectively assessed change at both visits using a GI-symptom anchor, the Gastrointestinal Symptom Rating Scale (GSRS). Responsiveness to change was assessed using F-statistics. The minimally changed group was those somewhat better or somewhat worse on the retrospective anchors and changing by one category on the modified GSRS (e.g., from slight to mild discomfort to moderate to moderately severe discomfort).ResultsResponsiveness to change was statistically significant for 6 of 8 PROMIS scales using the self-report GI anchor, 3 of 8 scales using the physician-reported anchor, and 5 of 5 scales using the corresponding GSRS scales as anchors. The MID estimates for scales for improvement and worsening were about 0.5-0.6 SD using the GSRS anchor and generally larger in magnitude than the change for the "about the same" group.ConclusionsThe responsiveness and MID estimates provided here for the PROMIS GI Symptoms scales can aid in scale score interpretation in clinical trials and observational studies.

© The Author(s) 2014 The current research evaluated a bifactor model for the Disgust Emotion Scale (DES) in three samples: N = 1,318 nonclinical participants, N = 152 clinic-referred patients, and N = 352 nonclinical participants. The primary goals were to (a) use bifactor modeling to examine the latent structure of the DES and in turn (b) evaluate whether the DES should be scored as a unidimensional scale or whether subscales should also be interpreted. Results suggested that a bifactor model fit the DES data well and that all DES items were strongly influenced by a general disgust proneness dimension and by five content dimensions. Moreover, model-based reliability analyses suggested that scoring a general disgust dimension is justified despite the confirmed multidimensional structure. However, subscales were found to be unreliable after controlling for the general disgust factor with the potential exception of the Mutilation/Death and Animals subscale. Subsequent analysis also showed that only the general disgust factor robustly predicted an obsessive-compulsive disorder symptom latent factor—a clinical condition closely related to disgust proneness; latent variables representing DES domains displayed weak relations with an obsessive-compulsive disorder factor above and beyond the general disgust factor. Implications for better understanding the structure of DES responses and its use in clinical research are discussed.

The psychometric properties of 4 paradigms adapted from the social neuroscience literature were evaluated to determine their suitability for use in clinical trials of schizophrenia. This 2-site study (University of California, Los Angeles and University of North Carolina) included 173 clinically stable schizophrenia outpatients and 88 healthy controls. The social cognition battery was administered twice to the schizophrenia group (baseline, 4-week retest) and once to the control group. The 4 paradigms included 2 that assess perception of nonverbal social and action cues (basic biological motion and emotion in biological motion) and 2 that involve higher level inferences about self and others’ mental states (self-referential memory and empathic accuracy). Each paradigm was evaluated on (1) patient vs healthy control group differences, (2) test-retest reliability, (3) utility as a repeated measure, and (4) tolerability. Of the 4 paradigms, empathic accuracy demonstrated the strongest characteristics, including large between-group differences, adequate test-retest reliability (.72), negligible practice effects, and good tolerability ratings. The other paradigms showed weaker psychometric characteristics in their current forms. These findings highlight challenges in adapting social neuroscience paradigms for use in clinical trials.

It is unknown whether measures adapted from social neuroscience linked to specific neural systems will demonstrate relationships to external variables. Four paradigms adapted from social neuroscience were administered to 173 clinically stable outpatients with schizophrenia to determine their relationships to functionally meaningful variables and to investigate their incremental validity beyond standard measures of social and nonsocial cognition. The 4 paradigms included 2 that assess perception of nonverbal social and action cues (basic biological motion and emotion in biological motion) and 2 that involve higher level inferences about self and others’ mental states (self- referential memory and empathic accuracy). Overall, social neuroscience paradigms showed significant relationships to functional capacity but weak relationships to community functioning; the paradigms also showed weak correlations to clinical symptoms. Evidence for incremental validity beyond standard measures of social and nonsocial cognition was mixed with additional predictive power shown for functional capacity but not community functioning. Of the newly adapted paradigms, the empathic accuracy task had the broadest external validity. These results underscore the difficulty of translating developments from neuroscience into clinically useful tasks with functional significance.

Despite much debate regarding the structure and measurement of impulsivity, research on this topic continues to thrive. There are many self-report measures of impulsivity and constructs related to impulse-control; however, the Barratt Impulsiveness Scale Version 11 (BIS-11; Patton, Stanford, & Barratt, 1995) appears to be the gold-standard self-report instrument in this domain, and it has played a major role in research (see Ireland & Archer, 2008; Stanford et al., 2009). Numerous recent studies have used the BIS-11 to explore the social consequences and behavioral correlates of individual differences in impulsivity (Carlson, Johnson, & Jacobs, 2010; Kjome et al., 2010; Piero, 2010; Sweitzer, Allen, & Kaut, 2008) as well as the biological and genetic origins of impulsivity (Benko et al., 2010; Kaladjian, Jeanningros, Azorin, Anton, & Mazzola-Pomietto, 2010; Lee et al., 2009; Stoltenberg & Nag, 2010). Despite the extensive use of the BIS-11, psychometric research on its internal structure has been relatively scant, especially in large community samples. The primary goal of the present investigation, therefore, was to apply exploratory and confirmatory factor analyses to BIS-11 item response data collected in a community sample (N = 691). Our objective was to test specific, well-known theories of the BIS-11structure, which form the basis of the aggregate scale and subscale scores that are currently used in research (Patton et al., 1995; Stanford et al., 2009). Specifically, we used exploratory and confirmatory unidimensional and bifactor modeling to evaluate the degree to which the BIS-11 total score reflects a single common latent trait of impulsivity. We also used exploratory and confirmatory factor analyses to evaluate the theory that BIS-11 responses reflect six correlated first-order constructs (attention, motor, self-control, planfullness, cognitive complexity, perseverance, cognitive instability), which in turn, form three second-order factors (attention, motor, and non-planning). Prior to further description of these analyses, the next section reviews the development of the BIS-11.

About 70% of Latinos, 5 years old or older, in the United States speak Spanish at home. Measurement equivalence of the PROMISA sample of 527 adult Spanish-speaking Latinos completed the Spanish version of the 41-item PROMISOne of the 41 items in the Spanish version of the PROMISEnglish- and Spanish-speaking subjects with the same level of pain interference responded differently to 1 of the 41 items in the PROMIS

Item response theory (IRT) was separately applied to parent- and teacher-rated symptoms of attention-deficit/hyperactivity disorder (ADHD) from a pooled sample of 526 six- to twelve-year-old children with and without ADHD. The dimensional structure ADHD was first examined using confirmatory factor analyses, including the bifactor model. A general ADHD factor and two group factors, representing inattentive and hyperactive/impulsive dimensions, optimally fit the data. Using the graded response model, we estimated discrimination and location parameters and information functions for all 18 symptoms of ADHD. Parent- and teacher-rated symptoms demonstrated adequate discrimination and location values, although these estimates varied substantially. For parent ratings, the test information curve peaked between −2 and +2 SD, suggesting that ADHD symptoms exhibited excellent overall reliability at measuring children in the low to moderate range of the general ADHD factor, but not in the extreme ranges. Similar results emerged for teacher ratings, in which the peak range of measurement precision was from −1.40 to 1.90 SD. Several symptoms were comparatively more informative than others; for example, is often easily distracted (“Distracted”) was the most informative parent- and teacher-rated symptom across the latent trait continuum. Clinical implications for the assessment of ADHD as well as relevant considerations for future revisions to diagnostic criteria are discussed.

Although the structure of the Rosenberg Self-Esteem Scale (RSES; Rosenberg, 1965) has been exhaustively evaluated, questions regarding dimensionality and direction of wording effects continue to be debated. To shed new light on these issues, we ask: (1) for what percentage of individuals is a unidimensional model adequate, (2) what additional percentage of individuals can be modeled with multidimensional specifications, and (3) what percentage of individuals respond so inconsistently that they cannot be well modeled? To estimate these percentages, we applied iteratively reweighted least squares (IRLS; Yuan & Bentler, 2000) to examine the structure of the RSES in a large, publicly available dataset. A distance measure, ds, reflecting a distance between a response pattern and an estimated model, was used for case weighting. We found that a bifactor model provided the best overall model fit, with one general factor and two wording-related group factors. But, based on dr values, a distance measure based on individual residuals, we concluded that approximately 86% of cases were adequately modeled through a unidimensional structure, and only an additional 3% required a bifactor model. Roughly 11% of cases were judged as “unmodelable” due to their significant residuals in all models considered. Finally, analysis of ds revealed that some, but not all, of the superior fit of the bifactor model is owed to that model’s ability to better accommodate implausible and possibly invalid response patterns, and not necessarily because it better accounts for the effects of direction of wording.

Bifactor latent structures were introduced over 70 years ago, but only recently has bifactor modeling been rediscovered as an effective approach to modeling construct-relevant multidimensionality in a set of ordered categorical item responses. I begin by describing the Schmid-Leiman bifactor procedure (Schmid & Leiman, 1957), and highlight its relations with correlated-factors and second-order exploratory factor models. After describing limitations of the Schmid-Leiman, two newer methods of exploratory bifactor modeling are considered, namely, analytic bifactor (Jennrich & Bentler, 2011) and target bifactor rotations (Reise, Moore, & Maydeu-Olivares, 2011). In section two, I discuss limited and full-information estimation approaches to confirmatory bifactor models that have emerged from the item response theory and factor analysis traditions, respectively. Comparison of the confirmatory bifactor model to alternative nested confirmatory models and establishing parameter invariance for the general factor also are discussed. In the final section, important applications of bifactor models are reviewed. These applications demonstrate that bifactor modeling potentially provides a solid foundation for conceptualizing psychological constructs, constructing measures, and evaluating a measure's psychometric properties. However, some applications of the bifactor model may be limited due to its restrictive assumptions.

To obtain Food and Drug Administration approval for the treatment of cognitive impairments associated with schizophrenia, a drug will need to demonstrate benefits beyond those that may be documented on objective cognitive tests. Interview-based measures of cognition such as the Cognitive Assessment Interview (CAI) are candidate coprimary outcome measures.Psychiatrically stable schizophrenia outpatients (n=150) were studied using the CAI to obtain information about cognitive functioning from both the patient and an informant. Patients also received objective assessments of neurocognition, functional capacity, functional outcome, and symptoms, at baseline and 1 month later.The CAI had good internal consistency (Cronbach's alpha=.92) and good test-retest reliability (r=.83). The CAI was moderately correlated with objective neurocognitive test scores (r's=-.39 to -.41) and moderately correlated with social functioning (r=-.38), work functioning (r=-.48), and overall functional outcome (r=-.49). The correlations of CAI scores with external validity indicators did not differ significantly by source of information (patient alone ratings were valid). Overall functional outcome correlated more strongly with patient CAI scores (r=-.50) than with objective neurocognitive test scores (r=.29) or functional capacity (r=.29).Field testing of the CAI produced reliable ratings of cognitive functioning that were correlated with functional outcome. Patient ratings alone yielded scores with reliability and validity values appropriate for use in clinical trials. The CAI appears to provide useful complementary information and possesses practical advantages for rating cognitive functioning including an interview-based method of administration, brief assessment time (15 min for the patient assessment), little or no practice effects, and ease of scoring.

Using a school-based (N = 1,060) and clinic-referred (N = 303) youth sample, the authors developed a 25-item shortened version of the Revised Child Anxiety and Depression Scale (RCADS) using Schmid-Leiman exploratory bifactor analysis to reduce client burden and administration time and thus improve the transportability characteristics of this youth anxiety and depression measure. Results revealed that all anxiety items primarily reflected a single "broad anxiety" dimension, which informed the development of a reduced 15-item Anxiety Total scale. Although specific DSM-oriented anxiety subscales were not included in this version, the items comprising the Anxiety Total scale were evenly pulled from the 5 anxiety-related content domains from the original RCADS. The resultant 15-item Anxiety Total scale evidenced significant correspondence with anxiety diagnostic groups based on structured clinical interviews. The scores from the 10-item Depression Total scale (retained from the original version) were also associated with acceptable reliability in the clinic-referred and school-based samples (α = .80 and .79, respectively); this is in contrast to the alternate 5-item shortened RCADS Depression Total scale previously developed by Muris, Meesters, and Schouten (2002), which evidenced depression scores of unacceptable reliability (α = .63). The shortened RCADS developed in the present study thus balances efficiency, breadth, and scale score reliability in a way that is potentially useful for repeated measurement in clinical settings as well as wide-scale screenings that assess anxiety and depressive problems. These future applications are discussed, as are recommendations for continued use of exploratory bifactor modeling in scale development.

In 2002, the NIH launched the 'Roadmap for Medical Research'. The Patient-Reported Outcomes Measurement Information System (PROMIS(®)) is one of the Roadmap's key aspects. To create the next generation of patient-reported outcome measures, PROMIS utilizes item response theory (IRT) and computerized adaptive testing. In 2009, the NIH funded the second wave of PROMIS studies (PROMIS II). PROMIS II studies continue PROMIS's agenda, but also include new features, including longitudinal analyses and more sociodemographically diverse samples. PROMIS II also includes increased emphasis on pediatric populations and evaluation of PROMIS item banks for clinical research and population science. These aspects bring new psychometric challenges. To address this, investigators associated with PROMIS gathered at the Third Psychometric Summit in September 2010 to identify, describe and discuss pressing psychometric issues and new developments in the field, as well as make analytic recommendations for PROMIS. The summit addressed five general themes: linking, differential item functioning, dimensionality, IRT models for longitudinal applications and new IRT software. In this article, we review the discussions and presentations that occurred at the Third PROMIS Psychometric Summit.

article i nfo Progress in the development of new pharmacological and psychosocial treatments for the negative symptoms of schizophrenia is impeded by limitations of available assessment instruments. The multi-site Collaboration to Advance Negative Symptom Assessment in Schizophrenia (CANSAS) was established to develop and validate a new clinical rating scale using a transparent, iterative, and data-driven process. The Clinical Assessment Interview for Negative Symptoms (CAINS) was designed to address limitations of existing measures and assess consensus-based sub-domains, including asociality, avolition, anhedonia, affective blunting, and alogia. The structure and psychometric properties of the CAINS were evaluated in a sample of 281 schizophrenia and schizoaffective outpatients at four sites. Converging structural analyses indicated that the scale was comprised of two moderately correlated factors — one reflecting experiential impairments (diminished motivation and enjoyment of social, vocational, and recreational activities) and one reflecting expressive impairments (diminished non-verbal and verbal communication). Item-level analyses revealed generally good distributional properties, inter-rater agreement, discriminating anchor points, and preliminary convergent and discriminant validity. Results indicate that the CAINS is a promising new measure for quantifying negative symptoms in clinical neuroscience and treatment studies. Results guided item modification or deletion, and the reliability and validity of the revised, shorter version of the CAINS is in the final phase of development within the CANSAS project. Published by Elsevier B.V.

This study explored the application of latent variable measurement models to the Social Anhedonia Scale (SAS; Eckblad, Chapman, Chapman, & Mishlove, 1982), a widely used and influential measure in schizophrenia-related research. Specifically, we applied unidimensional and bifactor item response theory (IRT) models to data from a community sample of young adults (n = 2,227). Ordinal factor analyses revealed that identifying a coherent latent structure in the 40-item SAS data was challenging due to (a) the presence of multiple small content clusters (e.g., doublets); (b) modest relations between those clusters, which, in turn, implies a general factor of only modest strength; (c) items that shared little variance with the majority of items; and (d) cross-loadings in bifactor solutions. Consequently, we conclude that SAS responses cannot be modeled accurately by either unidimensional or bifactor IRT models. Although the application of a bifactor model to a reduced 17-item set met with better success, significant psychometric and substantive problems remained. Results highlight the challenges of applying latent variable models to scales that were not originally designed to fit these models.

Background Practical, reliable “real world” measures of cognition are needed to supplement neurocognitive performance data to evaluate possible efficacy of new drugs targeting cognitive deficits associated with schizophrenia. Because interview-based measures of cognition offer one possible approach, data from the MATRICS initiative (n = 176) were used to examine the psychometric properties of the Schizophrenia Cognition Rating Scale (SCoRS) and the Clinical Global Impression of Cognition in Schizophrenia (CGI-CogS). Method We used classical test theory methods and item response theory to derive the 10-item Cognitive Assessment Interview (CAI) from the SCoRS and CGI-CogS (“parent instruments”). Sources of information for CAI ratings included the patient and an informant. Validity analyses examined the relationship between the CAI and objective measures of cognitive functioning, intermediate measures of cognition, and functional outcome. Results The rater's score from the newly derived CAI (10 items) correlate highly (r = .87) with those from the combined set of the SCoRS and CGI-CogS (41 items). Both the patient (r = .82) and the informant (r = .95) data were highly correlated with the rater's score. The CAI was modestly correlated with objectively measured neurocognition (r = −.32), functional capacity (r = −.44), and functional outcome (r = −.32), which was comparable to the parent instruments. Conclusions The CAI allows for expert judgment in evaluating a patient's cognitive functioning and was modestly correlated with neurocognitive functioning, functional capacity, and functional outcome. The CAI is a brief, repeatable, and potentially valuable tool for rating cognition in schizophrenia patients who are participating in clinical trials.

Short-form patient-reported outcome measures are popular because they minimize patient burden. We assessed the efficiency of static short forms and computer adaptive testing (CAT) using data from the Patient-Reported Outcomes Measurement Information System (PROMIS) project.We evaluated the 28-item PROMIS depressive symptoms bank. We used post hoc simulations based on the PROMIS calibration sample to compare several short-form selection strategies and the PROMIS CAT to the total item bank score.Compared with full-bank scores, all short forms and CAT produced highly correlated scores, but CAT outperformed each static short form in almost all criteria. However, short-form selection strategies performed only marginally worse than CAT. The performance gap observed in static forms was reduced by using a two-stage branching test format.Using several polytomous items in a calibrated unidimensional bank to measure depressive symptoms yielded a CAT that provided marginally superior efficiency compared to static short forms. The efficiency of a two-stage semi-adaptive testing strategy was so close to CAT that it warrants further consideration and study.

Attention deficit hyperactivity disorder (ADHD) is a disorder characterized by attentional difficulties. Mindfulness is a receptive attention to present experience. Both ADHD and mindfulness are associated with attention and personality. This study tests whether individuals with ADHD have lower mindfulness scores than controls and, if true, whether personality contributes to these differences. One hundred and five adults (half with ADHD) were assessed for mindfulness, using the Kentucky Inventory of Mindfulness Skills, and personality, using the Tridimensional Character Inventory. Individuals with ADHD report themselves as less mindful than non-ADHD controls and more novelty-seeking, less self-directed, and more self-transcendent. Mindfulness is negatively associated with ADHD and positively associated with self-directedness and self-transcendence. Analyses of subscales of mindfulness suggest that ADHD is associated most with the "Acting in Awareness" dimension, perhaps because of shared items reflecting attentional variability. The current findings support that a large portion of variability in trait mindfulness can be explained by ADHD status and personality traits of self-directedness and self-transcendence. It further suggests that interventions that increase mindfulness might improve symptoms of ADHD and increase self-directedness and/or self-transcendence.

Cognitive impairment in schizophrenia is often severe, enduring, and contributes significantly to chronic disability. But clinicians have difficulty in assessing cognition due to a lack of brief instruments. We evaluated whether a brief battery of cognitive tests derived from larger batteries could generate a summary score representing global cognitive function. Using data from 3 previously published trials, we calculated the corrected item-total correlations (CITCs) or the correlation of each test with the battery total score. We computed the proportion of variance that each test shares with the global score excluding that test (R(t)(2)=CITC(2)) and the variance explained per minute of administration time for each test (R(t)(2)/min). The 3 tests with the highest R(t)(2)/min were selected for the brief battery. The composite score from the trail making test B, category fluency, and digit symbol correlated .86 with the global score of the larger battery in 2 of the studies and correlated between .73 and .82 with the total battery scores excluding these 3 tests. A Brief Cognitive Assessment Tool for Schizophrenia (B-CATS) using the above 3 tests can be administered in 10-11 min. The full batteries of the larger studies have administration times ranging from 90 to 210 min. Given prior research suggesting that a single factor of global cognition best explains the pattern of cognitive deficit in schizophrenia, an instrument like B-CATS can provide clinicians with meaningful data regarding their patients' cognitive function. It can also serve researchers who want an estimate of global cognitive function without requiring a full neuropsychological battery.

The aims of this paper are to present findings related to differential item functioning (DIF) in the Patient Reported Outcome Measurement Information System (PROMIS) depression item bank, and to discuss potential threats to the validity of results from studies of DIF. The 32 depression items studied were modified from several widely used instruments. DIF analyses of gender, age and education were performed using a sample of 735 individuals recruited by a survey polling firm. DIF hypotheses were generated by asking content experts to indicate whether or not they expected DIF to be present, and the direction of the DIF with respect to the studied comparison groups. Primary analyses were conducted using the graded item response model (for polytomous, ordered response category data) with likelihood ratio tests of DIF, accompanied by magnitude measures. Sensitivity analyses were performed using other item response models and approaches to DIF detection. Despite some caveats, the items that are recommended for exclusion or for separate calibration were "I felt like crying" and "I had trouble enjoying things that I used to enjoy." The item, "I felt I had no energy," was also flagged as evidencing DIF, and recommended for additional review. On the one hand, false DIF detection (Type 1 error) was controlled to the extent possible by ensuring model fit and purification. On the other hand, power for DIF detection might have been compromised by several factors, including sparse data and small sample sizes. Nonetheless, practical and not just statistical significance should be considered. In this case the overall magnitude and impact of DIF was small for the groups studied, although impact was relatively large for some individuals.

We propose the application of a bifactor model for exploring the dimensional structure of an item response matrix, and for handling multidimensionality.We argue that a bifactor analysis can complement traditional dimensionality investigations by: (a) providing an evaluation of the distortion that may occur when unidimensional models are fit to multidimensional data, (b) allowing researchers to examine the utility of forming subscales, and, (c) providing an alternative to non-hierarchical multidimensional models for scaling individual differences.To demonstrate our arguments, we use responses (N = 1,000 Medicaid recipients) to 16 items in the Consumer Assessment of Healthcare Providers and Systems (CAHPS2.0) survey.Exploratory and confirmatory factor analytic and item response theory models (unidimensional, multidimensional, and bifactor) were estimated.CAHPS items are consistent with both unidimensional and multidimensional solutions. However, the bifactor model revealed that the overwhelming majority of common variance was due to a general factor. After controlling for the general factor, subscales provided little measurement precision.The bifactor model provides a valuable tool for exploring dimensionality related questions. In the Discussion, we describe contexts where a bifactor analysis is most productively used, and we contrast bifactor with multidimensional IRT models (MIRT). We also describe implications of bifactor models for IRT applications, and raise some limitations.

Background: The construction and evaluation of item banks to measure unidimensional constructs of health-related quality of life (HRQOL) is a fundamental objective of the Patient-Reported Outcomes Measurement Information System (PROMIS) project. Objectives: Item banks will be used as the foundation for developing short-form instruments and enabling computerized adaptive testing. The PROMIS Steering Committee selected 5 HRQOL domains for initial focus: physical functioning, fatigue, pain, emotional distress, and social role participation. This report provides an overview of the methods used in the PROMIS item analyses and proposed calibration of item banks. Analyses: Analyses include evaluation of data quality (eg, logic and range checking, spread of response distribution within an item), descriptive statistics (eg, frequencies, means), item response theory model assumptions (unidimensionality, local independence, monotonicity), model fit, differential item functioning, and item calibration for banking. Recommendations: Summarized are key analytic issues; recommendations are provided for future evaluations of item banks in HRQOL assessment.

© 2015 American Psychological Association.The measurement of treatment adherence (a component of treatment integrity defined as the extent to which a treatment is delivered as intended) is a critical element in treatment evaluation research. This article presents initial psychometric data for scores on the Cognitive-Behavioral Therapy Adherence Scale for Youth Anxiety (CBAY-A), an observational measure designed to be sensitive to common practice elements found in individual cognitive- behavioral therapy (ICBT) for youth anxiety. Therapy sessions (N = 954) from 1 efficacy and 1 effectiveness study of ICBT for youth anxiety were independently rated by 2 coders. Interrater reliability (as gauged by intraclass correlation coefficients) for the item scores averaged 0.77 (SD = 0.15; range 48 to 80). The CBAY-A item and scale (skills, model, total) scores demonstrated evidence of convergent and discriminant validity with an observational measure of therapeutic interventions and an observational measure of the alliance. The CBAY-A item and scale scores also discriminated between therapists delivering ICBT in research and practice settings and therapists delivering nonmanualized usual clinical care. We discuss the importance of replicating these psychometric findings in different samples and highlight possible application of an adherence measure in testing integrity-outcome relations.

© 2015 APA, all rights reserved). Visuospatial processing is a commonly assessed neurocognitive domain with deficits linked to dysfunction in right posterior regions of the brain. With the growth of large-scale clinical research studies, there is an increased need for efficient and scalable assessments of neurocognition, including visuospatial processing. The purpose of the current study was to use a novel method that combines item response theory (IRT) and computerized adaptive testing (CAT) approaches to create an abbreviated form of the computerized Penn Line Orientation Test (PLOT). The 24-item PLOT was administered to 8,498 youths (aged 8-21 years) as part of the Philadelphia Neurodevelopmental Cohort study and, by Web-based data collection, in an independent sample of 4,593 adults from Great Britain as part of a TV documentary. IRT-based CAT simulations were used to select the best PLOT items for an abbreviated form by performing separate simulations in each group and choosing only items that were selected as useful (i.e., high item discrimination and in the appropriate difficulty range) in at least 1 of the simulations. Fifteen items were chosen for the final, short form of the PLOT, indicating substantial agreement among the models in how they evaluated each item's usefulness. Moreover, this abbreviated version performed comparably to the full version in tests of sensitivity to age and sex effects. This abbreviated version of the PLOT cuts administration time by 50% without detectable loss of information, which points to its feasibility for large-scale clinical and genomic studies. (PsycINFO Database Record