Search

Objective: Executive functioning encompasses interactive cognitive processes such as planning, organization, set-shifting, inhibition, self-monitoring, working memory, and initiating and sustaining motor and mental activity. Researchers therefore typically assess executive functioning with multiple tests, each yielding multiple scores. A single composite score of executive functioning, which summarizes deficits across a battery of tests, would be useful in research and clinical trials. This study examines multiple candidate composite scores of executive functioning using tests from the Delis-Kaplan Executive Function System (D-KEFS)., Method: Participants were 875 adults between the ages of 20 and 89 years from the D-KEFS standardization sample. Seven Total Achievement scores were used from three tests (i.e., Trail Making, Verbal Fluency, and Color-Word Interference) to form eight composite scores that were compared based on their psychometric properties and association with intelligence (IQ)., Results: The distributions of most composite scores were mildly to severely skewed, and some had a pronounced ceiling effect. The composite scores all showed a medium positive correlation with IQ. The composite scores were highly intercorrelated in the total sample and in four IQ subgroups (i.e., IQ <89, 90-99, 100-109, 110+), with some being so highly correlated that they appear redundant., Conclusions: This study is part of a larger research program developing a cognition endpoint for research and clinical trials with sound psychometric properties and utility across discrepant test batteries. Future research is needed to examine the reliability and ecological validity of these composite scores., (© The Author(s) 2020. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permission@oup.com.)

Objective: To assess agreement between four brief computerized neurocognitive assessment tools (CNTs), ANAM, CogState, CNS Vital Signs, and ImPACT, by comparing rates of low scores., Methods: Four hundred and six US Army service members (SMs) with and without acute mild traumatic brain injury completed two randomly assigned CNTs with order of administration also randomly assigned. We performed a base rate analysis for each CNT to determine the proportions of SMs in the control and mTBI groups who had various numbers of scores that were 1.0+, 1.5+, and 2.0+ standard deviations below the normative mean. We used these results to identify a hierarchy of low score levels ranging from poorest to least poor performance. We then compared the agreement between every low score level from each CNT pair administered to the SMs., Results: More SMs in the mTBI group had low scores on all CNTs than SMs in the control group. As performance worsened, the association with mTBI became stronger for all CNTs. Most if not all SMs who performed at the worst level on any given CNT also had low scores on the other CNTs they completed but not necessarily at an equally low level., Conclusion: These results suggest that all of the CNTs we examined are broadly similar but still retain some psychometric differences that need to be better understood. Furthermore, the base rates of low scores we present could themselves be useful to clinicians and researchers as a guide for interpreting results from the CNTs., (© The Author(s) 2019. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.)

Objective: The Automated Neuropsychological Assessment Metrics (Version 4) Traumatic Brain Injury Military (ANAM4 TBI-MIL) is commonly administered among U.S. service members both pre-deployment and following TBI. The current study used the ANAM4 TBI-MIL to develop a cognition summary score for TBI research and clinical trials, comparing eight composite scores based on their distributions and sensitivity/specificity when differentiating between service members with and without mild TBI (MTBI)., Method: Male service members with MTBI (n = 56; Mdn = 11 days-since-injury) or no self-reported TBI history (n = 733) completed eight ANAM4 TBI-MIL tests. Their throughput scores (correct responses/minute) were used to calculate eight composite scores: the overall test battery mean (OTBM); global deficit score (GDS); neuropsychological deficit score-weighted (NDS-W); low score composite (LSC); number of scores <50th, ≤16th percentile, or ≤5th percentile; and the ANAM Composite Score (ACS)., Results: The OTBM and ACS were normally distributed. Other composites had skewed, zero-inflated distributions (62.9% had GDS = 0). All composites differed significantly between participants with and without MTBI (p < .001), with deficit scores showing the largest effect sizes (d = 1.32-1.47). The Area Under the Curve (AUC) was lowest for number of scores ≤5th percentile (AUC = 0.653) and highest for the LSC, OTBM, ACS, and NDS-W (AUC = 0.709-0.713)., Conclusions: The ANAM4 TBI-MIL has no well-validated composite score. The current study examined multiple candidate composite scores, finding that deficit scores showed larger group differences than the OTBM, but similar AUC values. The deficit scores were highly correlated. Future studies are needed to determine whether these scores show less redundancy among participants with more severe TBIs., (© The Author(s) 2019. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.)

Objective: Computerized neurocognitive assessment tools (NCATS) are often used as a screening tool to identify cognitive deficits after mild traumatic brain injury (mTBI). However, differing methodology across studies renders it difficult to identify a consensus regarding the validity of NCATs. Thus, studies where multiple NCATs are administered in the same sample using the same methodology are warranted., Method: We investigated the validity of four NCATs: the ANAM4, CNS-VS, CogState, and ImPACT. Two NCATs were randomly assigned and a battery of traditional neuropsychological (NP) tests administered to healthy control active duty service members (n = 272) and to service members within 7 days of an mTBI (n = 231). Analyses included correlations between NCAT and the NP test scores to investigate convergent and discriminant validity, and regression analyses to identify the unique variance in NCAT and NP scores attributed to group status. Effect sizes (Cohen's f2) were calculated to guide interpretation of data., Results: Only 37 (0.6%) of the 5,655 correlations calculated between NCATs and NP tests are large (i.e. r ≥ 0.50). The majority of correlations are small (i.e. 0.30 > r ≥ 0.10), with no clear patterns suggestive of convergent or discriminant validity between the NCATs and NP tests. Though there are statistically significant group differences across most NCAT and NP test scores, the unique variance accounted for by group status is minimal (i.e. semipartial R2 ≤ 0.033, 0.024, 0.062, and 0.011 for ANAM4, CNS-VS, CogState, and ImPACT, respectively), with effect sizes indicating small to no meaningful effect., Conclusion: Though the results are not overly promising for the validity of the four NCATs we investigated, traditional methods of investigating psychometric properties may not be appropriate for computerized tests. We offer several conceptual and methodological considerations for future studies regarding the validity of NCATs., (© The Author 2017. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.)

Computerized neurocognitive assessment tools (NCATs) have become a common way to assess postconcussion symptoms. As there is increasing research directly comparing multiple NCATs to each other, it is important to consider the impact that order of test administration may have on the integrity of the results. This study investigates the impact of administration order in a study of four different NCATs; Automated Neuropsychological Assessment Metrics (ANAM4), CNS Vital Signs (CNS-VS), CogState, and Immediate Post-Concussion Assessment and Cognitive Test (ImPACT). A total of 272 healthy active duty Service Members were enrolled into this study. All participants were randomly assigned to take two of the four NCATs with order of administration counterbalanced. Analyses attempted to investigate the effect of administration order alone (e.g., Time 1 versus Time 2), the effect of administration order combined with the impact of the specific NCAT received at Time 1, and only the impact of the Time 1 NCAT on Time 2 score variability. Specifically, independent samples t tests were used to compare Time 1 and Time 2 scores within each NCAT. Additional t tests compared Time 1 to Time 2 scores with Time 2 scores grouped by the NCAT received at Time 1. One-way analysis of variance (ANOVA) was used to compare only an NCAT's Time 2 scores grouped by the NCAT received at Time 1. Cohen's d effect sizes were calculated for all comparisons. The results from this study revealed statistically significant order effects for CogState and CNS-VS, though with effect sizes generally indicating minimum practical value, and marginal or absent order effects for ANAM4 and ImPACT with no clinically meaningful implications. Despite finding minimal order effects, clinicians should be mindful of the impact of administering multiple NCATs in a single session. Future studies should continue to be designed to minimize the potential effect of test administration order.

Cognitive impairment is a core clinical feature of traumatic brain injury (TBI). After TBI, cognition is a key determinant of post-injury productivity, outcome, and quality of life. As a final common pathway of diverse molecular and microstructural TBI mechanisms, cognition is an ideal endpoint in clinical trials involving many candidate drugs and nonpharmacological interventions. Cognition can be reliably measured with performance-based neuropsychological tests that have greater granularity than crude rating scales, such as the Glasgow Outcome Scale-Extended, which remain the standard for clinical trials. Remarkably, however, there is no well-defined, widely accepted, and validated cognition endpoint for TBI clinical trials. A single cognition endpoint that has excellent measurement precision across a wide functional range and is sensitive to the detection of small improvements (and declines) in cognitive functioning would enhance the power and precision of TBI clinical trials and accelerate drug development research. We outline methodologies for deriving a cognition composite score and a research program for validation. Finally, we discuss regulatory issues and the limitations of a cognition endpoint., Competing Interests: Author Disclosure Statement GLI has been reimbursed by the government, professional scientific bodies, and commercial organizations for discussing or presenting research relating to MTBI and sport-related concussion at meetings, scientific conferences, and symposiums. He has a clinical practice in forensic neuropsychology involving individuals who have sustained mild TBIs. He has received honorariums for serving on research panels that provide scientific peer review of programs. He is a co-investigator, collaborator, or consultant on grants relating to mild TBI funded by several organizations. He has received grant funding from pharmaceutical companies to do psychometric research using neuropsychological tests. He has received research support from neuropsychological test publishing companies in the past (not in the past 5 years). For the remaining authors, no competing financial interests exist. NDS receives salary support from a Vancouver Coastal Health Research Institute Clinician-Scientist Career Development Award. GLI acknowledges support from the INTRuST Posttraumatic Stress Disorder and Traumatic Brain Injury Clinical Consortium funded by the Department of Defense Psychological Health/Traumatic Brain Injury Research Program (X81XWH-07-CC-CSDoD), Harvard Integrated Program to Protect and Improve the Health of NFLPA Members, and the Mooney-Reed Charitable Foundation. GTM acknowledges support from NIH U01 NS086090-01, DOD USAMRAA W81XWH-13-1-0441, DOD W81XWH-14-2-0176, and One Mind. This work is related to the TBI Endpoints Development Initiative and a grant entitled Development and Validation of a Cognition Endpoint for Traumatic Brain Injury Clinical Trials.

Base rates of low ANAM4 TBI-MIL scores were calculated in a convenience sample of 733 healthy male active duty soldiers using available military reference values for the following cutoffs: ≤2nd percentile (2 SDs), ≤5th percentile, <10th percentile, and <16th percentile (1 SD). Rates of low scores were also calculated in 56 active duty male soldiers who sustained an mTBI an average of 23 days (SD = 36.1) prior. 22.0% of the healthy sample and 51.8% of the mTBI sample had two or more scores below 1 SD (i.e., 16th percentile). 18.8% of the healthy sample and 44.6% of the mTBI sample had one or more scores ≤5th percentile. Rates of low scores in the healthy sample were influenced by cutoffs and race/ethnicity. Importantly, some healthy soldiers obtain at least one low score on ANAM4. These base rate analyses can improve the methodology for interpreting ANAM4 performance in clinical practice and research., (© The Author 2014. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.)

Computerized neurocognitive assessment tools (NCATs) are increasingly used for baseline and post-concussion assessments. To date, NCATs have not demonstrated strong test-retest reliabilities. Most studies have used non-military populations and different methodologies, complicating the determination of the utility of NCATs in military populations. The test-retest reliability of four NCATs (Automated Neuropsychological Assessment Metrics 4 [ANAM4], CNS-Vital Signs, CogState, and Immediate Post-Concussion Assessment and Cognitive Test [ImPACT]) was investigated in a healthy active duty military sample. Four hundred and nineteen Service Members were randomly assigned to take one NCAT and 215 returned after approximately 30 days for retest. Participants deemed to have inadequate effort during one or both testing sessions, according to the NCATs scoring algorithms, were removed from analyses. Each NCAT had at least one reliability score (intraclass correlation) in the "adequate" range (.70-.79), only ImPACT had one score considered "high" (.80-.89), and no scores met "very high" criteria (.90-.99). However, overall test-retest reliabilities in four NCATs in a military sample are consistent with reliabilities reported in the literature and are lower than desired for clinical decision-making.

This exploratory study was conducted to increase understanding of neuropsychological test performance in those with blast-related mild traumatic brain injury (mTBI). The two variables of interest for their impact on test performance were presence of mTBI symptoms and history of posttraumatic stress disorder (PTSD). Forty-five soldiers postblast mTBI, 27 with enduring mTBI symptoms and 18 without, completed a series of neuropsychological tests. Seventeen of the 45 met criteria for PTSD. The Paced Auditory Serial Addition Test (Frencham, Fox, & Mayberry, 2005; Spreen & Strauss, 1998) was the primary outcome measure. Two-sided, 2-sample t tests were used to compare scores between groups of interest. Presence of mTBI symptoms did not impact test performance. In addition, no significant differences between soldiers with and without PTSD were identified. Standard neuropsychological assessment may not increase understanding about impairment associated with mTBI symptoms. Further research in this area is indicated., (Copyright 2010 APA, all rights reserved)