Digital Cognitive Assessment: Results from the TestMyBrain NIMH Research Domain Criteria (RDoC) Field Test Battery Report

Digital technology has become a major target area for the development of assessments that can be deployed through mobile devices, across large cohorts, and in naturalistic environments. Here, we summarize the results of a report commissioned by the National Institute of Mental Health (HHSN271201700776P) to evaluate mobile assessments of cognition and their appropriateness for deployment in a field test battery. Using data from over 100,000 participants tested through our digital research platform, TestMyBrain.org , we analyze the appropriateness of 25 standard tests of cognition and information processing for field test battery use. Measures are evaluated in terms of their psychometric properties, validity, engagement, and sensitivity to variations in device hardware and software. We also define a minimum duration for acceptable reliability (minDAR) across all 25 tests, operationalized as the duration a test needs to be to achieve an internal reliability of at least 0.7 for primary outcome measures. We note that many tests adapted from experimental approaches, particularly those involving aspects of positive and negative valence, need further development to achieve acceptable length and reliability (based on very high minDAR values, e.g. 180 minutes vs 3 minutes for threat biases in memory vs. memory alone). Device variability also presents a confound for reaction time tests (e.g. iOS vs Android Cohen’s d = 0.4 for simple reaction time, p < 0.001). Areas of focus for development of such measures are described. Digital cognitive assessment is a promising methodology for large sample studies at relatively low cost. There are notable areas where further research and development work is needed, however, to fully realize the potential for cognitive phenotyping at scale.