0102 Performance Evaluation of a 24-hour Sleep-Wake State Classifier Derived from Research-Grade Actigraphy

Introduction Wrist-worn research-grade actigraphy devices are commonly used to identify sleep and wakefulness in freely-living people. However, common existing algorithms were developed primarily to classify sleep-wake within a defined in-bed period with PSG, and exhibit relatively high sensitivity (accuracy on sleep epochs) but relatively low specificity (accuracy on wake epochs). This classification imbalance results in the algorithms performing poorly when attempting to classify data that does not have a predefined sleep period, such as over a 24-hour interval. Here, we develop a 24-hour actigraphy classifier to overcome limitations in specificity (accuracy on wake epochs), which typically afflict in-bed focused algorithms. Methods Four datasets scored via either PSG or direct observation of daytime wakefulness were combined (n=52 participants of mean age 49.8yrs, age range 19 - 86; 52% male; 221 total days/nights). Actigraphy (counts) and PSG (RPSGT-staged epochs) were temporally aligned. A model was trained to transform a time-series actigraphy counts to a time series of sleep-wake classifications, using the TensorFlow library for Python. 5-fold cross-validation was used to train and evaluate the model. Classification performance was compared to the output of the Spectrum device (Philips-Respironics) using the Oakley algorithm with a wake threshold of ‘medium’. Results The developed classifier was compared to the Spectrum classifications across the 24-hour intervals. The developed classifier had higher accuracy (95.4% vs. 76.8%), higher specificity (95.9% vs. 68.9%) and higher balanced-accuracy (95.2% vs. 81.6%) relative to the Spectrum classifications, each assessed via paired-sample t-test. Sensitivity did not statistically differ (94.5% vs. 94.4%). Conclusion The model trained and evaluated on 24-hour data outperformed the existing algorithm output in terms of overall accuracy, specificity, and balanced accuracy, while sensitivity did not significantly differ. A model trained on 24-hour data may be more appropriate for analyses of freely living people, or older populations where napping is more common. Developing an accurate 24-hour sleep/wake classifier fosters new opportunities to evaluate sleep patterns in the absence of self-reports or assumptions about time in bed. Support (If Any) UL1TR002014, NSF#1622766, R43/44-AG056250