Evaluation of an accelerometer-based monitor for detecting bed net use and human entry/exit using a machine learning algorithm

Abstract Background Distribution of long-lasting insecticidal bed nets (LLINs) is one of the main control strategies for malaria. Improving malaria prevention programmes requires understanding usage patterns in households receiving LLINs, but there are limits to what standard cross-sectional surveys of self-reported LLIN use can provide. This study was designed to assess the performance of an accelerometer-based approach for measuring a range of LLIN use behaviours as a proof of concept for more granular LLIN-use monitoring over longer time periods. Methods This study was carried out under controlled conditions from May to July 2018 in Liverpool, UK. A single accelerometer was affixed to the side panel of an LLIN and participants carried out five LLIN use behaviours: (1) unfurling a net; (2) entering an unfurled net; (3) lying still as if sleeping; (4) exiting from under a net; and, (5) folding up a net. The randomForest package in R, a supervised non-linear classification algorithm, was used to train models on 20-s epochs of tagged accelerometer data. Models were compared in a validation dataset using overall accuracy, sensitivity and specificity, receiver operating curves and the area under the curve (AUC). Results The five-category model had overall accuracy of 82.9% in the validation dataset, a sensitivity of 0.681 for entering a net, 0.632 for exiting, 0.733 for net down, and 0.800 for net up. A simplified four-category model, combining entering/exiting a net into one category had accuracy of 94.8%, and increased sensitivity for net down (0.756) and net up (0.829). A further simplified three-category model, identifying sleeping, net up, and a combined net down/enter/exit category had accuracy of 96.2% (483/502), with an AUC of 0.997 for net down and 0.987 for net up. Models for detecting entering/exiting by adults were significantly more accurate than for children (87.8% vs 70.0%; p