Neural Network Based Luminance Variation Resistant Remote-Photoplethysmography for Driver’s Heart Rate Monitoring

The driver's physiological status has enormous value to public traffic safety and cannot be ignored nowadays. In addition, heart rate (HR) is one of the most important indicators of human's health status. When detecting the driver's HR, using traditional contact-type devices might bring about the driver's distraction or discomfort. On the contrary, the remote photoplethysmography (rPPG) technique is a better way to monitor a driver's HR in vehicle applications simply by using a web-camera without interfering the driver. Most of the rPPG studies intended to reduce the interference caused by facial motion or luminance changes in the indoor or controlled scenario, but there are relatively fewer discussions on outdoor scenarios. Consequently, the purpose of this paper is to enhance the rPPG technique to make it suitable for the outdoor driving scenarios and for monitoring the driver's HR in different weather conditions, including daytime and nighttime. We first utilize artificial neural network (ANN) and train multiple personalized ANN models for each driver. For predicting the drivers' HR beat more precisely, we propose the approach, adaptive neural network model selection (ANNMS), which adaptively selects a personalized ANN model based on different noise conditions. Our algorithm eliminates the effect of noises caused by the variations of facial luminance in eight outdoor driving scenarios. The proposed driver's HR beat monitoring system has been evaluated against the state-of-the-art rPPG techniques that are Chrominance signal-based (CHRO) and k-nearest neighbors-based (kNN) algorithms. Compared with the CHRO and kNN algorithms, the ANNMS reduces the mean absolute error from 14.71 bpm (CHRO) and 9.91 (kNN) to 4.51 bpm (ANNMS) and enhances the success-rate-10, the probability in which the absolute error is below 10 bpm, from 44.1% (CHRO) and 56.3% (kNN) to 91.5% (ANNMS).