A Mixing of Paper and Electronic Trail-Making Test for Cognitive Impairment with Automatic Analysis: Development and Validation Study (Preprint)

BACKGROUND Computer-aided detection used in screening and diagnostic cognitive impairment becomes increasingly the focus of attention, which allows objective, ecologically valid, and convenient assessment. Among that, sensor technology based on digital devices is potential method for detection. v OBJECTIVE The aim of this study was to develop and validate a novel trail-making test (TMT) based on a mixing of paper and electronic devices. METHODS In this research, a community-dwelling sample of older adults (n = 297) was included, classified as cognitively normal control (NC, n = 100), diagnosed with mild cognitive impairment (MCI, n = 98) or Alzheimer's disease (AD, n = 99). We used an electromagnetic tablet to record each participant's hand-painted strokes. Meanwhile, a sheet of A4 paper was placed on top of the tablet to keep the traditional interaction style for subjects who are not familiar or comfortable with electronic devices, e.g., touchscreen. In this way, all participants were instructed to perform the TMT-square and circle (TMT-S&C). Furthermore, we developed an efficient and interpretable cognitive impairment screening model to automated analyse cognitive impairment level, which was dependent on demographic, time-related, pressure-related, jerk-related, and template-related features. Among that, the novel template-based features were based on vector quantization (VQ) algorithm. Firstly, the model singled out a candidate trajectory as the standard answer (template) from the normal group. Then the distance between the recorded trajectories and the reference was computed as the important evaluation index. To verify the effectiveness of our method, we compared the performance of well-trained machine learning model using the extracted evaluation index with conventional demographic characters and time related features. In addition, the well-trained model also was validated on follow-ups’ data (NC: 38, MCI: 32, and AD: 22). RESULTS We compared five candidate machine learning methods, screened out the random forest as the ideal model with the best performance (Accuracy: 0.777 for NC vs. MCI, 0.929 for NC vs. AD, and 0.805 for AD vs. MCI). Meanwhile, the well-trained classifier achieved better performance than the conventional assessment method and high stability and accuracy on follow-ups’ data. CONCLUSIONS The study demonstrated that the mixing of paper and electronic TMT produced more reliable and valid data for evaluating subjects' cognitive impairment compared with conventional paper-based assessment. CLINICALTRIAL The study was approved by Medical Ethics Committee of Shanghai Tongji Hospital (ChiCTR2000039550)