Search

Although satisfying friendships are crucial for well-being throughout adulthood, measures of friendship satisfaction have been limited by: (1) item content relevant to children only, (2) a focus on single relationships rather than the friendship network, and (3) disagreement about the number of dimensions necessary to capture the construct. To overcome these limitations, we assembled an item pool from a number of existing measures, created additional items drawn from research on friendships, and then examined the structure and psychometric properties of those items in two online surveys of over 2000 respondents each. Factor analyses consistently identified two correlated factors—closeness and socializing—but bi-factor modeling revealed that scores on both subscales load strongly on a general factor, suggesting that the multifaceted content can be scored efficiently as a unidimensional composite. Analyses using item response theory (IRT) supported the creation of a reliable 14-item instrument that demonstrated adequate convergent and predictive validity. Thus, the Friendship Network Satisfaction (FNS) Scale is a psychometrically sound tool to advance research on friendships across the lifespan.

The reliable change index has been used to evaluate the significance of individual change in health-related quality of life. We estimate reliable change for two measures (physical function and emotional distress) in the Patient-Reported Outcomes Measurement Information System (PROMIS®) 29-item health-related quality of life measure (PROMIS-29 v2.1). Using two waves of data collected 3 months apart in a longitudinal observational study of chronic low back pain and chronic neck pain patients receiving chiropractic care, and simulations, we compare estimates of reliable change from classical test theory fixed standard errors with item response theory standard errors from the graded response model. We find that unless true change in the PROMIS physical function and emotional distress scales is substantial, classical test theory estimates of significant individual change are much more optimistic than estimates of change based on item response theory. Supplementary Information The online version supplementary material available at 10.1007/s11336-021-09774-1.

Objective.The National Neuropsychology Network (NNN) is a multicenter clinical research initiative funded by the National Institute of Mental Health (NIMH; R01 MH118514) to facilitate neuropsychology’s transition to contemporary psychometric assessment methods with resultant improvement in test validation and assessment efficiency.Method:The NNN includes four clinical research sites (Emory University; Medical College of Wisconsin; University of California, Los Angeles (UCLA); University of Florida) and Pearson Clinical Assessment. Pearson Q-interactive (Q-i) is used for data capture for Pearson published tests; web-based data capture tools programmed by UCLA, which serves as the Coordinating Center, are employed for remaining measures.Results:NNN is acquiring item-level data from 500–10,000 patients across 47 widely used Neuropsychology (NP) tests and sharing these data via the NIMH Data Archive. Modern psychometric methods (e.g., item response theory) will specify the constructs measured by different tests and determine their positive/negative predictive power regarding diagnostic outcomes and relationships to other clinical, historical, and demographic factors. The Structured History Protocol for NP (SHiP-NP) helps standardize acquisition of relevant history and self-report data.Conclusions:NNN is a proof-of-principle collaboration: by addressing logistical challenges, NNN aims to engage other clinics to create a national and ultimately an international network. The mature NNN will provide mechanisms for data aggregation enabling shared analysis and collaborative research. NNN promises ultimately to enable robust diagnostic inferences about neuropsychological test patterns and to promote the validation of novel adaptive assessment strategies that will be more efficient, more precise, and more sensitive to clinical contexts and individual/cultural differences.

Major obstacles to data harmonization in neuropsychology include lack of consensus about what constructs and tests are most important and invariant across healthy and clinical populations. This study addressed these challenges using data from the National Neuropsychology Network (NNN).Data were obtained from 5,000 NNN participants and Pearson standardization samples. Analyses included variables from four instruments: Wechsler Adult Intelligence Scale, 4th Edition (WAIS-IV); Wechsler Memory Scale, 4th Edition (WMS-IV); California Verbal Learning Test, 3rd Edition (CVLT3); and Delis-Kaplan Executive Function System (D-KEFS). We used confirmatory factor analysis to evaluate models suggested by prior work and examined fit statistics and measurement invariance across samples. We examined relations of factor scores to demographic and clinical characteristics.For each instrument, we identified four first-order and one second-order factor. Optimal models in patients generally paralleled the best-fitting models in the standardization samples, including task-specific factors. Analysis of the NNN data prompted specification of a Recognition-Familiarity factor on the WMS-IV and an Inhibition-Switching factor on the D-KEFS. Analyses showed strong to strict factorial invariance across samples with expected differences in factor means and variances. The Recognition-Familiarity factor correlated with age more strongly in NNN than in the standardization sample.Factor models derived from healthy groups generally fit well in patients. NNN data helped identify novel Recognition-Familiarity and Inhibition-Switching factors that were also invariant across samples and may be clinically useful. The findings support efforts to identify evidence-based and optimally efficient measurements of neuropsychological constructs that are valid across groups. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Large-scale studies spanning diverse project sites, populations, languages, and measurements are increasingly important to relate psychological to biological variables. National and international consortia already are collecting and executing mega-analyses on aggregated data from individuals, with different measures on each person. In this research, we show that Asparouhov and Muthén’s alignment method can be adapted to align data from disparate item sets and response formats. We argue that with these adaptations, the alignment method is well suited for combining data across multiple sites even when they use different measurement instruments. The approach is illustrated using data from the Whole Genome Sequencing in Psychiatric Disorders consortium and a real-data-based simulation is used to verify accurate parameter recovery. Factor alignment appears to increase precision of measurement and validity of scores with respect to external criteria. The resulting parameter estimates may further inform development of more effective and efficient methods to assess the same constructs in prospectively designed studies.

As part of a scale development project, we fit a nominal response item response theory model to responses to the Health Care Engagement Measure (HEM). When using the original 5-point response format, categories were not ordered as intended for six of the 23 items. For the remaining, the category boundary discrimination between Categories 0 ( not at all true) and 1 ( a little bit true) was only weakly discriminating, suggesting uninformative categories. When the lowest two categories were collapsed, psychometric properties improved greatly. Category boundary discriminations within items, however, varied significantly. Specifically, higher response category distinctions, such as responding 3 ( very true) versus 2 ( mostly true) were considerably more discriminating than lower response category distinctions. Implications for HEM scoring and for improving measurement precision at lower levels of the construct are presented as is the unique role of the nominal response model in category analysis.

Item response theory (IRT) model applications extend well beyond cognitive ability testing, and various patient-reported outcomes (PRO) measures are among the more prominent examples. PRO (and like) constructs differ from cognitive ability constructs in many ways, and these differences have model fitting implications. With a few notable exceptions, however, most IRT applications to PRO constructs rely on traditional IRT models, such as the graded response model. We review some notable differences between cognitive and PRO constructs and how these differences can present challenges for traditional IRT model applications. We then apply two models (the traditional graded response model and an alternative log-logistic model) to depression measure data drawn from the Patient-Reported Outcomes Measurement Information System project. We do not claim that one model is “a better fit” or more “valid” than the other; rather, we show that the log-logistic model may be more consistent with the construct of depression as a unipolar phenomenon. Clearly, the graded response and log-logistic models can lead to different conclusions about the psychometrics of an instrument and the scaling of individual differences. We underscore, too, that, in general, explorations of which model may be more appropriate cannot be decided only by fit index comparisons; these decisions may require the integration of psychometrics with theory and research findings on the construct of interest. Supplementary Information The online version supplementary material available at 10.1007/s11336-021-09802-0.

PURPOSE: Healthcare engagement is a core measurement target for efforts to improve healthcare systems. This construct is broadly defined as the extent to which healthcare services represent collaborative partnerships with patients. Previous qualitative work operationalized healthcare engagement as generalized self-efficacy in four related subdomains: self-management, collaborative communication, health information use, and healthcare navigation. Building on this work, our objective was to establish a healthcare engagement instrument that is sufficiently unidimensional to yield a single score.METHOD: We conducted cognitive interviews followed by a nation-wide mail survey of US Veteran Administration (VA) healthcare users. Data were collected on 49 candidate healthcare engagement items, as well as measures of self-efficacy for managing symptoms, provider communication, and perceived access. Items were subjected to exploratory bifactor, statistical learning, and IRT analyses.RESULTS: Cognitive interviews were completed by 56 patients and 9552 VA healthcare users with chronic conditions completed the mail survey. Participants were mostly white and male but with sizable minority participation. Psychometric analyses and content considerations reduced the item pool to 23 items, which demonstrated a strong general factor (OmegaH of .89). IRT analyses revealed a high level of reliability across the trait range and little DIF across groups. Most health information use items were removed during analyses, suggesting a more independent role for this domain.CONCLUSION: We provide quantitative evidence for a relatively unidimensional measure of healthcare engagement. Despite developed with VA healthcare users, the measure is intended for general use. Future work includes short-form development and validation with other patient groups.

Post-traumatic stress disorder (PTSD) can develop after exposure to traumatic events and severely impacts the quality of life. PTSD is frequently comorbid with substance use disorders, with alcoholism being particularly common. However, not everyone who experiences trauma develops PTSD and the factors that render individuals susceptible or resilient to the effects of stress are unknown although gender appears to play an important role. Rodent models of stress exposure such as stress-enhanced fear learning (SEFL) recapitulate some aspects of PTSD symptomology, making them an invaluable tool for studying this disorder. This study examined whether exposure to a modified version of the SEFL procedure (4 footshocks instead of the standard 15 over 90min) would reveal both susceptible and resilient subjects. Following stress exposure, distinct susceptible and resilient groups emerged that differed in fear learning and anxiety-related behavior as well as voluntary alcohol intake. Some aspects of stress susceptibility manifested differently in males compared to females, with susceptibility associated with increased alcohol intake in males and increased baseline anxiety in females.

One of the most widely used measures of psychosis-related symptoms and characteristics is the 74-item Schizotypal Personality Questionnaire (SPQ). Using multidimensional Item Response Theory (bifactor 2-parameter model), we calibrated SPQ items in a sample of 375 youths aged 9–24 years and constructed a fully functional computerized adaptive form of the SPQ on an open-source platform for public use. To assess validity, we used the above parameters to simulate CAT sessions in a separate validation sample (N = 100) using three test-length-based stopping rules: 8 items, 16 items, and 32 items. Those scores were then compared to full-form and SPQ-Brief scores on their abilities to predict psychosis or clinical risk status. Areas under the receiver operating characteristic curves indicated mediocre predictive ability, but did not differ among any of the forms, even when only eight adaptive items were administered. The Youden index for the 16-item adaptive version was higher than that for the 22-item SPQ-Brief. Classification accuracy for the full SPQ was 73% compared to 66% for the both the SPQ-Brief and adaptive versions (average of three stopping rules). The SPQ-CAT shows promise as a much shorter but valid assessment of schizotypy which can save time with minimal loss of information.

Author(s): Anderson, Ariana E; Marder, Stephen; Reise, Steven P; Savitz, Adam; Salvadore, Giacomo; Fu, Dong Jing; Li, Qingqin; Turkoz, Ibrahim; Han, Carol; Bilder, Robert M | Abstract: Objective:Common genetic variation spans schizophrenia, schizoaffective and bipolar disorders, but historically, these syndromes have been distinguished categorically. A symptom dimension shared across these syndromes, if such a general factor exists, might provide a clearer target for understanding and treating mental illnesses that share core biological bases. Method:We tested the hypothesis that a bifactor model of the Positive and Negative Syndrome Scale (PANSS), containing 1 general factor and 5 specific factors (positive, negative, disorganized, excited, anxiety), explains the cross-diagnostic structure of symptoms better than the traditional 5-factor model, and examined the extent to which a general factor reflects the overall severity of symptoms spanning diagnoses in 5094 total patients with a diagnosis of schizophrenia, schizoaffective, and bipolar disorder. Results:The bifactor model provided superior fit across diagnoses, and was closer to the "true" model, compared to the traditional 5-factor model (Vuong test; P l .001). The general factor included high loadings on 28 of the 30 PANSS items, omitting symptoms associated with the excitement and anxiety/depression domains. The general factor had highest total loadings on symptoms that are often associated with the positive and disorganization syndromes, but there were also substantial loadings on the negative syndrome thus leading to the interpretation of this factor as reflecting generalized psychosis. Conclusions:A bifactor model derived from the PANSS can provide a stronger framework for measuring cross-diagnostic psychopathology than a 5-factor model, and includes a generalized psychosis dimension shared at least across schizophrenia, schizoaffective, and bipolar disorder.

In structural equation models, outliers could result in inaccurate parameter estimates and misleading fit statistics when using traditional methods. To robustly estimate structural equation models,...

Mahalanobis distance (M-distance) case diagnostics are a useful tool for assessing response pattern inconsistency in factor analysis; however, the derivations of these statistics assume continuous ...

Previous studies (Gignac, 2016; Murray & Johnson, 2013) have explored conditions for which accurate statistical comparison of a second-order and bifactor model is challenging, if not impossible. However, a mathematical basis for this problem has not been offered. We show that a second-order model implies a unique set of tetrad constraints (Bollen & Ting, 1993, 1998, 2000) that the bifactor model does not, and that the two models are distinguishable to the degree that these unique tetrad constraints are violated. Simulated population matrices, and mathematical proofs, are used to demonstrate that: (a) when a second-order model with cross-loadings or correlated residuals is true, fitting a pure (misspecified) second-order model leads to violation of the tetrad constraints, which in turn leads to the chi-square and Bayesian Information Criterion favoring a (misspecified) bifactor model, and (b) a true bifactor model can be identified only when the tetrad constraints of the second-order model are violated, which is mainly a function of the proportionality of loadings. Three model-comparison approaches are offered for applied researchers.

Background About 70% of Latinos, 5 years old or older, in the United States speak Spanish at home. Measurement equivalence of the PROMIS® pain interference (PI) item bank by language of administration (English versus Spanish) has not been evaluated.

Traditional "paper-and-pencil" testing is imprecise in measuring speed and hence limited in assessing performance efficiency, but computerized testing permits precision in measuring itemwise response time. We present a method of scoring performance efficiency (combining information from accuracy and speed) at the item level. Using a community sample of 9,498 youths age 8-21, we calculated item-level efficiency scores on 4 neurocognitive tests, and compared the concurrent, convergent, discriminant, and predictive validity of these scores with simple averaging of standardized speed and accuracy-summed scores. Concurrent validity was measured by the scores' abilities to distinguish men from women and their correlations with age; convergent and discriminant validity were measured by correlations with other scores inside and outside of their neurocognitive domains; predictive validity was measured by correlations with brain volume in regions associated with the specific neurocognitive abilities. Results provide support for the ability of itemwise efficiency scoring to detect signals as strong as those detected by standard efficiency scoring methods. We find no evidence of superior validity of the itemwise scores over traditional scores, but point out several advantages of the former. The itemwise efficiency scoring method shows promise as an alternative to standard efficiency scoring methods, with overall moderate support from tests of 4 different types of validity. This method allows the use of existing item analysis methods and provides the convenient ability to adjust the overall emphasis of accuracy versus speed in the efficiency score, thus adjusting the scoring to the real-world demands the test is aiming to fulfill. (PsycINFO Database Record

Attachment has recently been proposed as a key developmental construct in psychosis, in particular with respect to interpersonal functioning and social cognition. The current study examined the latent structure of the self-report Psychosis Attachment Measure (PAM) and its relationship to lower-level perceptual and higher-order inferential social cognitive processes. The PAM was administered to 138 psychiatrically stable outpatients with schizophrenia alongside a battery of symptom, social cognitive, and functional measures. PAM responses were analyzed using latent variable measurement models, which did not yield evidence of the coherent two-dimensional structure predicted by previous literature. A unidimensional subscale comprising 6 of the 16 original PAM items possessed the strongest psychometric properties. This subscale was generally uncorrelated with social cognitive measures and showed weak correlations with some symptoms measures and with community functioning. These results suggest that either the PAM may not measure attachment in psychosis or it may measure only attachment anxiety but demonstrate little construct validity in this population. These results tell a cautionary tale regarding making theoretical inferences on the basis of measures without coherent latent structure. Attachment measures with stronger psychometric properties will help clarify putative relationships between attachment and social cognitive processes in psychosis.

Analytic bifactor rotations (Jennrich & Bentler, 2011, 2012) have been recently developed and made generally available, but are not well understood. The Jennrich-Bentler analytic bifactor rotations (bi-quartimin and bi-geomin) are an alternative to, and arguably an improvement upon, the less technically sophisticated Schmid-Leiman orthogonalization (Schmid & Leiman, 1957). We review the technical details that underlie the Schmid-Leiman and Jennrich-Bentler bifactor rotations, using simulated data structures to illustrate important features and limitations. For the Schmid-Leiman, we review the problem of inaccurate parameter estimates caused by the linear dependencies, sometimes called “proportionality constraints,” that are required to expand a p correlated factors solution into a (p+1) (bi)factor space. We also review the complexities involved when the data depart from perfect cluster structure (e.g., item cross-loading on group factors). For the Jennrich-Bentler rotations, we describe problems in parameter estimation caused by departures from perfect cluster structure. In addition, we illustrate the related problems of: (a) solutions that are not invariant under different starting values (i.e., local minima problems); and, (b) group factors collapsing onto the general factor. Recommendations are made for substantive researchers including examining all local minima and applying multiple exploratory techniques in an effort to identify an accurate model.

Item response theory (IRT) measurement models are now commonly used in educational, psychological, and health-outcomes measurement, but their impact in the evaluation of measures of psychiatric constructs remains limited. Herein we present two, somewhat contradictory, theses. The first is that, when skillfully applied, IRT has much to offer psychiatric measurement in terms of scale development, psychometric analysis, and scoring. The second argument, however, is that psychiatric measurement presents some unique challenges to the application of IRT – challenges that may not be easily addressed by application of conventional IRT models and methods. These challenges include, but are not limited to, the modeling of conceptually narrow constructs and their associated limited item pools, and unipolar constructs where the expected latent trait distribution is highly skewed.

Quantitative models of mental illness, such as the Hierarchical Taxonomy of Psychopathology (HiTOP), aim to optimize clinical assessment, which conventionally follows categorical diagnostic rubrics. The evidence base for these models is robust, but also uniform; available data come mostly from structured diagnostic interviews in nationally representative samples. It remains to be seen whether HiTOP adequately reflects mental illness as evaluated in routine clinical care, where diagnosis is often unsystematic and incomplete, relative to controlled research conditions. To test the generalizability of a quantitative nosology to real-world assessment contexts, we fit the HiTOP model to diagnoses in a large sample (N = 25,002) of treatment-seeking university students who were seen by health professionals in everyday practice. We then examined the criterion validity of model components in relation to clinically relevant outcomes (i.e., suicide attempts, self-injury, and binge drinking). Three related structures fit the data well: a correlated-factor model with internalizing, externalizing, and eating pathology dimensions; a higher-order model that added a general factor of psychopathology that spanned these 3 first-order factors; and a bifactor model that partitioned diagnostic (co)variance across a general factor and 3 orthogonal group factors. The first-order factors had expected patterns of criterion validity, and the general factor was a strong predictor of suicidality and self-injury, paralleling past research. Bifactor model group factors had interpretative problems, however. Across models, categorical diagnoses consistently offered minimal incremental validity relative to the transdiagnostic factors. We conclude that HiTOP is ecologically valid-explaining comorbidity patterns among diagnoses assigned "in the field"-and is poised to enhance clinical assessment and decision-making in routine care. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

OBJECTIVE: This article reviews current approaches to neuropsychological assessment, identifies opportunities for development of new methods using modern psychometric theory and advances in technology, and suggests a transition path that promotes application of novel methods without sacrificing validity. METHODS: Theoretical/state-of-the-art review. CONCLUSIONS: Clinical neuropsychological assessment today does not reflect advances in neuroscience, modern psychometrics, or technology. Major opportunities for improving practice include both psychometric and technological strategies. Modern psychometric approaches including item response theory (IRT) enable: linking procedures that can place different measures on common scales; adaptive testing algorithms that can dramatically increase efficiency of assessment; examination of differential item functioning (DIF) to detect measures that behave differently in different groups; and person fit statistics to detect aberrant patterns of responding of high value for performance validity testing. Opportunities to introduce novel technologies include computerized adaptive testing, Web-based assessment, healthcare- and bio-informatics strategies, mobile platforms, wearables, and the “internet-of-things.” To overcome inertia in current practices, new methods must satisfy requirements for back-compatibility with legacy instrumentation, enabling us to leverage the wealth of validity data already accrued for classic procedures. A path to achieve these goals involves creation of a global network to aggregate item-level data into a shared repository that will enable modern psychometric analyses to refine existing methods, and serve as a platform to evolve novel assessment strategies, which over time can revolutionize neuropsychological assessment practices world-wide.

The recent incorporation of genomics into clinical neuroscience has generated an unprecedented need for behavioral assessment of domains that can be linked to brain systems and serve as “biomarkers” of psychopathology (Insel & Cuthbert, 2009). The scale of such studies requires efficient computerized testing of a broad range of abilities using tests that have been validated with functional neuroimaging. Such “neurobehavioral probes” (Gur, Erwin and Gur, 1992) have been assembled into a computerized neurocognitive battery (CNB) in which these tasks have been adapted to assure adequate psychometric properties, such as reliability and validity (Gur et al., 2001, 2010) and its linkage to brain systems (Roalf et al., 2013). The CNB has been applied in large-scale genomic studies (Aliyu et al., 2006; Greenwood et al., 2007; Gur et al., 2007a,b; Almasy et al., 2008), treatment research (Gur et al., 2001b; Grant et al., 2013), and the military (Thomas et al., 2013). Although its individual tests’ scores have been largely reliable (and their inferences valid), the overall latent structure of the battery has not been sufficiently evaluated. Using confirmatory and exploratory factor analysis, we aimed to examine the factor structure of the CNB, which is of paramount importance in justifying how the instrument is scored and interpreted. The CNB currently comprises fourteen tests grouped into five domains of neurobehavioral function. These broad domains were selected because they represent well-established brain systems, and each test in the battery was used in functional neuroimaging studies to probe aspects of these domains (Roalf et al., 2013). There are three tests in each of four domains measuring “Executive Control,” “Episodic Memory,” “Complex Cognition,” and “Social Cognition” as summarized in Table 1. A fifth domain (Sensorimotor speed) was measured by two tests, but the current analysis examined accuracy and these tests only provide speed measures and were therefore not included. Here we examined the factorial structure of the CNB in a sample of over 9,000 individuals age 8 to 21 who were administered the CNB as part of their participation in the Philadelphia Neurodevelopmental Cohort (PNC; Gur et al., 2012, 2013). Table 1 Domain-Specific scales of the Computerized Neurocognitive Battery, by neurobehavioral function

© 2015, Routledge. All rights reserved. In structural equation modeling (SEM), researchers need to evaluate whether item response data, which are often multidimensional, can be modeled with a unidimensional measurement model without seriously biasing the parameter estimates. This issue is commonly addressed through testing the fit of a unidimensional model specification, a strategy previously determined to be problematic. As an alternative to the use of fit indexes, we considered the utility of a statistical tool that was expressly designed to assess the degree of departure from unidimensionality in a data set. Specifically, we evaluated the ability of the DETECT “essential unidimensionality” index to predict the bias in parameter estimates that results from misspecifying a unidimensional model when the data are multidimensional. We generated multidimensional data from bifactor structures that varied in general factor strength, number of group factors, and items per group factor; a unidimensional measurement model was then fit and parameter bias recorded. Although DETECT index values were generally predictive of parameter bias, in many cases, the degree of bias was small even though DETECT indicated significant multidimensionality. Thus we do not recommend the stand-alone use of DETECT benchmark values to either accept or reject a unidimensional measurement model. However, when DETECT was used in combination with additional indexes of general factor strength and group factor structure, parameter bias was highly predictable. Recommendations for judging the severity of potential model misspecifications in practice are provided.

Several studies have indicated that bi–factor models fit a broad range of psychometric data better than alternative multidimensional models such as second–order models, e.g Rodriguez, Reise and Haviland (2016), Gignac (2016), and Carnivez (2016). Murray and Johnson (2013) and Gignac (2016) argue that this phenomenon is partially due to un–modeled complexities (e.g. un–modeled cross-factor loadings) that induce a bias in standard statistical measures that favors bi–factor models over second–order models. We extend the Murray and Johnson simulation studies to show how the ability to distinguish second–order and bi–factor models diminishes as the amount of un–modeled complexity increases. By using theorems about rank constraints on the covariance matrix to find sub–models of measurement models that have less un–modeled complexity, we are able to reduce the statistical bias in favor of bi–factor models; this allows researchers to reliably distinguish between bi-factor and second-order models.

Although the Positive and Negative Syndrome Scale (PANSS) was developed for use in schizophrenia (SZ), antipsychotic drug trials use the PANSS to measure symptom change also for bipolar (BP) and schizoaffective (SA) disorder, extending beyond its original indications. If the dimensions measured by the PANSS are different across diagnoses, then the same score change for the same drug condition may have different meanings depending on which group is being studied. Here, we evaluated whether the factor structure in the PANSS was consistent across schizophrenia (n = 3647), bipolar disorder (n = 858), and schizoaffective disorder (n = 592). Along with congruency coefficients, Hancock's H, and Jaccard indices, we used target rotations and statistical tests of invariance based on confirmatory factor models. We found the five symptom dimensions measured by the 30-item PANSS did not generalize well to schizoaffective and bipolar disorders. A model based on an 18-item version of the PANSS generalized better across SZ and BP groups, but significant problems remained in generalizing some of the factors to the SA sample. Schizophrenia and bipolar disorder showed greater similarity in factor structure than did schizophrenia and schizoaffective disorder. The Anxiety/Depression factor was the most consistent across disorders, while the Positive factor was the least consistent.

It is generally assumed that the latent trait is normally distributed in the population when estimating logistic item response theory (IRT) model parameters. This assumption requires that the latent trait be fully continuous and the population homogenous (i.e., not a mixture). When this normality assumption is violated, models are misspecified, and item and person parameter estimates are inaccurate. When normality cannot be assumed, it might be appropriate to consider alternative modeling approaches: (a) a zero-inflated mixture, (b) a log-logistic, (c) a Ramsay curve, or (d) a heteroskedastic-skew model. The first 2 models were developed to address modeling problems associated with so-called quasi-continuous or unipolar constructs, which apply only to a subset of the population, or are meaningful at one end of the continuum only. The second 2 models were developed to address non-normal latent trait distributions and violations of homogeneity of error variance, respectively. To introduce these alternative IRT models and illustrate their strengths and weaknesses, we performed real data application comparing results to those from a graded response model. We review both statistical and theoretical challenges in applying these models and choosing among them. Future applications of these and other alternative models (e.g., unfolding, diffusion) are needed to advance understanding about model choice in particular situations.

OBJECTIVES: To describe the development and initial psychometric properties of the observer-rated Global Therapist Competence Scale for Youth Psychosocial Treatment (G-COMP) in the context of cognitive-behavioral treatment (CBT) for youth anxiety disorders. METHOD: Independent coders rated 744 sessions from a sample of 68 youth (M age = 10.56 years) using the G-COMP and observational instruments of alliance, involvement, CBT adherence, CBT competence. RESULTS: Inter-rater reliability coefficients (ICCs [2,2]) were greater than .60 for the five G-COMP domain scores. G-COMP scores yielded small to medium correlations with instruments of the child-therapist alliance (rs = .17 - .44) and youth involvement in treatment (rs = .08 - .53), and medium to large correlations with instruments of CBT competence and adherence (rs = .26 - .63). Therapists in the research setting were rated higher compared to newly trained therapists in community clinics. CONCLUSIONS: Preliminary reliability and validity of the G-COMP scores are promising, but future research is needed with non-CBT samples.

BackgroundThe NIH-sponsored Patient-Reported Outcomes Measurement Information System (PROMIS) Gastrointestinal (GI) Symptoms scales were developed to assess patients' GI symptoms in clinical settings.AimsTo assess responsiveness to change and provide minimally important difference (MID) estimates for the PROMIS GI Symptoms scales.MethodsA sample of 256 GI outpatients self-administered the eight PROMIS GI Symptoms scales (gastroesophageal reflux, disrupted swallowing, diarrhea, bowel incontinence/soilage, nausea and vomiting, constipation, belly pain, and gas/bloating/flatulence) at two visits. Patient self-reported and physician-reported assessments of the subjects' overall GI condition were employed as change anchors. In addition, we prospectively assessed change at both visits using a GI-symptom anchor, the Gastrointestinal Symptom Rating Scale (GSRS). Responsiveness to change was assessed using F-statistics. The minimally changed group was those somewhat better or somewhat worse on the retrospective anchors and changing by one category on the modified GSRS (e.g., from slight to mild discomfort to moderate to moderately severe discomfort).ResultsResponsiveness to change was statistically significant for 6 of 8 PROMIS scales using the self-report GI anchor, 3 of 8 scales using the physician-reported anchor, and 5 of 5 scales using the corresponding GSRS scales as anchors. The MID estimates for scales for improvement and worsening were about 0.5-0.6 SD using the GSRS anchor and generally larger in magnitude than the change for the "about the same" group.ConclusionsThe responsiveness and MID estimates provided here for the PROMIS GI Symptoms scales can aid in scale score interpretation in clinical trials and observational studies.

Schizophrenia is marked by a global cognitive impairment that contributes significantly to chronic disability and unemployment. As new treatments are developed for cognition in schizophrenia, clinicians require easily administered instruments to assess cognition. We previously developed a very brief cognitive battery (Bell et al., 2005). The Brief Cognitive Assessment Tool for Schizophrenia (B-CATS) was developed specifically to provide clinicians with a way to assess cognition in their patients with schizophrenia. Here, we report the results of a validity study comparing B-CATS to a larger neurocognitive battery, the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) battery.Outpatients with schizophrenia (N=91) were administered the B-CATS and the non-overlapping tests of the MATRICS battery at two time points separated by 1month. They were also administered the UCSD Performance-based Skills Assessment-Brief (UPSA-B), a measure of functional capacity.The B-CATS has an administration time of approximately 10min. It demonstrates good test-retest reliability and internal consistency. It correlates 0.76 (p0.01) with the MATRICS battery. The shorter B-CATS and the MATRICS battery correlate with the UPSA-B at 0.50 and 0.58 respectively.A 10-minute version of the B-CATS correlates highly with the "gold standard" neurocognitive battery that has an administration time of over 60min. Both measures correlate moderately with a measure of functional capacity. This brief battery was designed to allow clinicians to monitor cognitive change and better inform treatment decisions.

© The Author(s) 2014 The current research evaluated a bifactor model for the Disgust Emotion Scale (DES) in three samples: N = 1,318 nonclinical participants, N = 152 clinic-referred patients, and N = 352 nonclinical participants. The primary goals were to (a) use bifactor modeling to examine the latent structure of the DES and in turn (b) evaluate whether the DES should be scored as a unidimensional scale or whether subscales should also be interpreted. Results suggested that a bifactor model fit the DES data well and that all DES items were strongly influenced by a general disgust proneness dimension and by five content dimensions. Moreover, model-based reliability analyses suggested that scoring a general disgust dimension is justified despite the confirmed multidimensional structure. However, subscales were found to be unreliable after controlling for the general disgust factor with the potential exception of the Mutilation/Death and Animals subscale. Subsequent analysis also showed that only the general disgust factor robustly predicted an obsessive-compulsive disorder symptom latent factor—a clinical condition closely related to disgust proneness; latent variables representing DES domains displayed weak relations with an obsessive-compulsive disorder factor above and beyond the general disgust factor. Implications for better understanding the structure of DES responses and its use in clinical research are discussed.

The psychometric properties of 4 paradigms adapted from the social neuroscience literature were evaluated to determine their suitability for use in clinical trials of schizophrenia. This 2-site study (University of California, Los Angeles and University of North Carolina) included 173 clinically stable schizophrenia outpatients and 88 healthy controls. The social cognition battery was administered twice to the schizophrenia group (baseline, 4-week retest) and once to the control group. The 4 paradigms included 2 that assess perception of nonverbal social and action cues (basic biological motion and emotion in biological motion) and 2 that involve higher level inferences about self and others’ mental states (self-referential memory and empathic accuracy). Each paradigm was evaluated on (1) patient vs healthy control group differences, (2) test-retest reliability, (3) utility as a repeated measure, and (4) tolerability. Of the 4 paradigms, empathic accuracy demonstrated the strongest characteristics, including large between-group differences, adequate test-retest reliability (.72), negligible practice effects, and good tolerability ratings. The other paradigms showed weaker psychometric characteristics in their current forms. These findings highlight challenges in adapting social neuroscience paradigms for use in clinical trials.

It is unknown whether measures adapted from social neuroscience linked to specific neural systems will demonstrate relationships to external variables. Four paradigms adapted from social neuroscience were administered to 173 clinically stable outpatients with schizophrenia to determine their relationships to functionally meaningful variables and to investigate their incremental validity beyond standard measures of social and nonsocial cognition. The 4 paradigms included 2 that assess perception of nonverbal social and action cues (basic biological motion and emotion in biological motion) and 2 that involve higher level inferences about self and others’ mental states (self- referential memory and empathic accuracy). Overall, social neuroscience paradigms showed significant relationships to functional capacity but weak relationships to community functioning; the paradigms also showed weak correlations to clinical symptoms. Evidence for incremental validity beyond standard measures of social and nonsocial cognition was mixed with additional predictive power shown for functional capacity but not community functioning. Of the newly adapted paradigms, the empathic accuracy task had the broadest external validity. These results underscore the difficulty of translating developments from neuroscience into clinically useful tasks with functional significance.

Despite much debate regarding the structure and measurement of impulsivity, research on this topic continues to thrive. There are many self-report measures of impulsivity and constructs related to impulse-control; however, the Barratt Impulsiveness Scale Version 11 (BIS-11; Patton, Stanford, & Barratt, 1995) appears to be the gold-standard self-report instrument in this domain, and it has played a major role in research (see Ireland & Archer, 2008; Stanford et al., 2009). Numerous recent studies have used the BIS-11 to explore the social consequences and behavioral correlates of individual differences in impulsivity (Carlson, Johnson, & Jacobs, 2010; Kjome et al., 2010; Piero, 2010; Sweitzer, Allen, & Kaut, 2008) as well as the biological and genetic origins of impulsivity (Benko et al., 2010; Kaladjian, Jeanningros, Azorin, Anton, & Mazzola-Pomietto, 2010; Lee et al., 2009; Stoltenberg & Nag, 2010). Despite the extensive use of the BIS-11, psychometric research on its internal structure has been relatively scant, especially in large community samples. The primary goal of the present investigation, therefore, was to apply exploratory and confirmatory factor analyses to BIS-11 item response data collected in a community sample (N = 691). Our objective was to test specific, well-known theories of the BIS-11structure, which form the basis of the aggregate scale and subscale scores that are currently used in research (Patton et al., 1995; Stanford et al., 2009). Specifically, we used exploratory and confirmatory unidimensional and bifactor modeling to evaluate the degree to which the BIS-11 total score reflects a single common latent trait of impulsivity. We also used exploratory and confirmatory factor analyses to evaluate the theory that BIS-11 responses reflect six correlated first-order constructs (attention, motor, self-control, planfullness, cognitive complexity, perseverance, cognitive instability), which in turn, form three second-order factors (attention, motor, and non-planning). Prior to further description of these analyses, the next section reviews the development of the BIS-11.

Confirmatory factor analytic studies of psychological measures showing item responses to be multidimensional do not provide sufficient guidance for applied work. Demonstrating that item response data are multifactorial in this way does not necessarily (a) mean that a total scale score is an inadequate indicator of the intended construct, (b) demand creating and scoring subscales, or (c) require specifying a multidimensional measurement model in research using structural equation modeling (SEM). To better inform these important decisions, more fine-grained psychometric analyses are necessary. We describe 3 established, but seldom used, psychometric approaches that address 4 distinct questions: (a) To what degree do total scale scores reflect reliable variation on a single construct? (b) Is the scoring and reporting of subscale scores justified? (c) If justified, how much reliable variance do subscale scores provide after controlling for a general factor? and (d) Can multidimensional item response data be represented by a unidimensional measurement model in SEM, or are multidimensional measurement models (e.g., second-order, bifactor) necessary to achieve unbiased structural coefficients? In the discussion, we provide guidance for applied researchers on how best to interpret the results from applying these methods and review their limitations.

About 70% of Latinos, 5 years old or older, in the United States speak Spanish at home. Measurement equivalence of the PROMISA sample of 527 adult Spanish-speaking Latinos completed the Spanish version of the 41-item PROMISOne of the 41 items in the Spanish version of the PROMISEnglish- and Spanish-speaking subjects with the same level of pain interference responded differently to 1 of the 41 items in the PROMIS

Item response theory (IRT) was separately applied to parent- and teacher-rated symptoms of attention-deficit/hyperactivity disorder (ADHD) from a pooled sample of 526 six- to twelve-year-old children with and without ADHD. The dimensional structure ADHD was first examined using confirmatory factor analyses, including the bifactor model. A general ADHD factor and two group factors, representing inattentive and hyperactive/impulsive dimensions, optimally fit the data. Using the graded response model, we estimated discrimination and location parameters and information functions for all 18 symptoms of ADHD. Parent- and teacher-rated symptoms demonstrated adequate discrimination and location values, although these estimates varied substantially. For parent ratings, the test information curve peaked between −2 and +2 SD, suggesting that ADHD symptoms exhibited excellent overall reliability at measuring children in the low to moderate range of the general ADHD factor, but not in the extreme ranges. Similar results emerged for teacher ratings, in which the peak range of measurement precision was from −1.40 to 1.90 SD. Several symptoms were comparatively more informative than others; for example, is often easily distracted (“Distracted”) was the most informative parent- and teacher-rated symptom across the latent trait continuum. Clinical implications for the assessment of ADHD as well as relevant considerations for future revisions to diagnostic criteria are discussed.

Although the structure of the Rosenberg Self-Esteem Scale (RSES; Rosenberg, 1965) has been exhaustively evaluated, questions regarding dimensionality and direction of wording effects continue to be debated. To shed new light on these issues, we ask: (1) for what percentage of individuals is a unidimensional model adequate, (2) what additional percentage of individuals can be modeled with multidimensional specifications, and (3) what percentage of individuals respond so inconsistently that they cannot be well modeled? To estimate these percentages, we applied iteratively reweighted least squares (IRLS; Yuan & Bentler, 2000) to examine the structure of the RSES in a large, publicly available dataset. A distance measure, ds, reflecting a distance between a response pattern and an estimated model, was used for case weighting. We found that a bifactor model provided the best overall model fit, with one general factor and two wording-related group factors. But, based on dr values, a distance measure based on individual residuals, we concluded that approximately 86% of cases were adequately modeled through a unidimensional structure, and only an additional 3% required a bifactor model. Roughly 11% of cases were judged as “unmodelable” due to their significant residuals in all models considered. Finally, analysis of ds revealed that some, but not all, of the superior fit of the bifactor model is owed to that model’s ability to better accommodate implausible and possibly invalid response patterns, and not necessarily because it better accounts for the effects of direction of wording.

While attributional style is regarded as a core domain of social cognition, questions persist about the psychometric characteristics of measures used to assess it. One widely used assessment of attributional style is the Ambiguous Intentions Hostility Questionnaire (AIHQ). Two limitations of the AIHQ include (1) a possible restricted range resulting from too few and too homogenous item scenarios, and (2) use of rater scores that are cumbersome and time-consuming to score and have unknown incremental validity. The present study evaluated the psychometric properties of the AIHQ while concurrently testing changes aiming to improve the scale, in particular expansion of the number of self-report items and removal of the rater-scored items. One hundred sixty individuals diagnosed with schizophrenia and 58 healthy controls completed the full AIHQ along with measures of symptoms, functioning, and verbal intelligence. The AIHQ – particularly the self-reported blame score – demonstrated adequate internal consistency, test-retest reliability, and distinguished patients from controls. It also was significantly related to clinically-rated hostility and suspiciousness symptoms, and correlated with functional capacity even after controlling for verbal intelligence. Incremental validity analyses suggested that a higher number of self-report items strengthens relationships to outcomes in a manner that justifies this expansion, while rater-scored items had mixed results in providing additional information beyond self-report in the AIHQ.

Bifactor latent structures were introduced over 70 years ago, but only recently has bifactor modeling been rediscovered as an effective approach to modeling construct-relevant multidimensionality in a set of ordered categorical item responses. I begin by describing the Schmid-Leiman bifactor procedure (Schmid & Leiman, 1957), and highlight its relations with correlated-factors and second-order exploratory factor models. After describing limitations of the Schmid-Leiman, two newer methods of exploratory bifactor modeling are considered, namely, analytic bifactor (Jennrich & Bentler, 2011) and target bifactor rotations (Reise, Moore, & Maydeu-Olivares, 2011). In section two, I discuss limited and full-information estimation approaches to confirmatory bifactor models that have emerged from the item response theory and factor analysis traditions, respectively. Comparison of the confirmatory bifactor model to alternative nested confirmatory models and establishing parameter invariance for the general factor also are discussed. In the final section, important applications of bifactor models are reviewed. These applications demonstrate that bifactor modeling potentially provides a solid foundation for conceptualizing psychological constructs, constructing measures, and evaluating a measure's psychometric properties. However, some applications of the bifactor model may be limited due to its restrictive assumptions.