Search

This article presents a deep-learning-based two-stream network to estimate remote Photoplethysmogram (rPPG) signal and hence derive the heart rate (HR) from an RGB facial video. Our proposed network employs temporal modulation blocks (TMBs) to efficiently extract temporal dependencies and spatial attention blocks on a mean frame to learn spatial features. Our TMBs are composed of two subblocks that can simultaneously learn overall and channelwise spatiotemporal features, which are pivotal for the task. Data augmentation (DA) in training and multiple redundant estimations for noise removal in testing were also designed to make the training more effective and the inference more robust. Experimental results show that the proposed temporal shift-channelwise spatio-temporal network (TS-CST Net) has reached competitive and even superior performances among the state-of-the-art (SOTA) methods on four popular datasets, showcasing our network’s learning capability.

Eldering caring is important in today’s aging society, especially that accident anticipation/prevention plays an important role. In this paper, a novel approach to preventing elderly accidents based on a very low-resolution thermopile sensor array (TPA) (only $32\times 32$ pixels) is proposed for prediction of bed-exit event that might lead to elderly falls in home caring. Low-resolution TPA sensor, capable of collecting far infrared energy, ensures cost-effective monitoring, no interference with user’s daily life, and most importantly privacy-preservation. Since most of the fall accidents occur when the elderly attempts to get off the bed without assistance, it is thus the focus of this paper to monitor his/her posture and action via TPA image sensor and then predict that an action of getting off the bed will occur in a near future (e.g., $S$ seconds later). Our system can raise an alarm to the caregivers so that they can intervene and offer the necessary assistance. A deep-learning model based on CNN-RNN (Convolutional neural network-Recurrent neural network) architecture was designed which is capable of predicting the elderly bed-exit intention by $S =5.78$ seconds in advance of the action onset at an accuracy of 99.37% according to our dataset evaluation. Our system is also suitable for on-line real-time operation which will be helpful to elderly caring in our society.