Search

Human activity recognition (HAR) based on wearable devices has found wide applications in fitness, health care, etc. Given the personalized wearing styles of such devices and distinctive motion patterns, the activities of daily living normally vary from person to person in terms of strength, amplitude, speed, category, etc. The specialization of a universal HAR model to a specific subject without experiencing catastrophic forgetting is a significant challenge. In this paper, we propose a novel incremental learning method, namely, an adaptive coefficient-based kernelized and regularized network (KeRNet-AC), for personalized activity recognition. During the adaptation stage of the model training process, KeRNet-AC consistently monitors the probable ill-conditioned degree of the generated solution, which we believe is strongly correlated with the catastrophic forgetting problem, and automatically makes the solution well conditioned. To reduce the computational complexity of KeRNet-AC, we also introduce an active data selection principle into KeRNet-AC. This variation is called A-KeRNet-AC. To evaluate the performance of KeRNet-AC and A-KeRNet-AC, we conduct extensive experiments on five public activity datasets. The experimental results demonstrate that KeRNet-AC outperforms related state-of-the-art methods in most cases and that A-KeRNet-AC can quickly perform model training and activity prediction. Moreover, the performance of the proposed methods steadily improves during the adaptation stage and ultimately converges without degradation, demonstrating the strong potential of KeRNet-AC and A-KeRNet-AC for personalized activity recognition.

Sensor-based activity recognition seeks the profound high-level knowledge about human activities from multitudes of low-level sensor readings. Conventional pattern recognition approaches have made tremendous progress in the past years. However, those methods often heavily rely on heuristic hand-crafted feature extraction, which could hinder their generalization performance. Additionally, existing methods are undermined for unsupervised and incremental learning tasks. Recently, the recent advancement of deep learning makes it possible to perform automatic high-level feature extraction thus achieves promising performance in many areas. Since then, deep learning based methods have been widely adopted for the sensor-based activity recognition tasks. This paper surveys the recent advance of deep learning based sensor-based activity recognition. We summarize existing literature from three aspects: sensor modality, deep model, and application. We also present detailed insights on existing work and propose grand challenges for future research., 10 pages, 2 figures, and 5 tables; submitted to Pattern Recognition Letters (second revision)

User profiles are widely used in the age of big data. However, generating and releasing user profiles may cause serious privacy leakage, since a large number of personal data are collected and analyzed. In this paper, we propose a differentially private user profile construction method DP-UserPro, which is composed of DP-CLIQUE and privately top-k tags selection. DP-CLIQUE is a differentially private high dimensional data cluster algorithm based on CLIQUE. The multidimensional tag space is divided into cells, Laplace noises are added into the count value of each cell. Based on the breadth-first-search, the largest connected dense cells are clustered into a cluster. Then a privately top-k tags selection approach is proposed based on the score function of each tag, to select the most important k tags which can represent the characteristics of the cluster. Privacy and utility of DP-UserPro are theoretically analyzed and experimentally evaluated in the last. Comparison experiments are carried out with Tag Suppression algorithm on two real datasets, to measure the False Negative Rate (FNR) and precision. The results show that DP-UserPro outperforms Tag Suppression by 62.5% in the best case and 14.25% in the worst case on FNR, and DP-UserPro is about 21.1% better on precision than that of Tag Suppression, in average.

Changes in the motor pattern have been shown to be useful advanced indicators of cognitive disorders, such as Parkinson's disease (PD) and cerebral small vessel disease (SVD). It would be highly advantageous to tap into data containing people's motor patterns from motion sensing devices to analyze subtle changes in cognitive abilities, thereby providing personalized interventions before the actual onset of such conditions. However, this goal is very challenging due to two main technical problems: 1) the size of data labeled by doctors is small, and 2) the available data tends to be highly imbalanced (the vast majority tend to be from normal subjects with only a small fraction from subjects with cognitive disorder). In order to effectively deal with these challenges to infer cognitive wellness from motor patterns with high accuracy, we propose the MOtor-Cognitive Analytics (MOCA) framework. The proposed MOCA first uses the random oversampling iterative random forest based feature selection method to reduce the feature space dimensionality and avoid overfitting, and then adds a bias in the optimization problem of weighted extreme learning machine to achieve good generalization ability in handling imbalanced small-sampling dataset. Experimental results on two real-world datasets including SVD and stroke patients show that MOCA can effectively reduce the rate of misdiagnosis and significantly outperform state-of-the-art methods in inferring people's cognitive capabilities. This work opens up opportunities for population-level pre-screening using motion sensing devices and can inform current discussions on reforming the health-care infrastructure. NRF (Natl Research Foundation, S’pore) MOH (Min. of Health, S’pore) Accepted version

Automatic activity recognition is an active research topic which aims to identify human activities automatically. A significant challenge is to recognize new activities effectively. In this paper, we propose an effective class incremental learning method, named Class Incremental Random Forests (CIRF), to enable existing activity recognition models to identify new activities. We design a separating axis theorem based splitting strategy to insert internal nodes and adopt Gini index or information gain to split leaves of the decision tree in the random forests (RF). With these two strategies, both inserting new nodes and splitting leaves are allowed in the incremental learning phase. We evaluate our method on three UCI public activity datasets and compare with other state-of-the-art methods. Experimental results show that the proposed incremental learning method converges to the performance of batch learning methods (RF and extremely randomized trees). Compared with other state-of-the-art methods, it is able to recognize new class data continuously with a better performance.

Currently available traditional as well as videogame-based cognitive assessment techniques are inappropriate due to several reasons. This paper presents a novel psychosocial game suite, BrainStorm,...

Miscellaneous mini-wearable devices (Jawbone Up, Apple Watch, Google Glass, et al.) have emerged in recent years to recognize the user’s activities of daily living (ADLs) such as walking, running, climbing and bicycling. To better suits a target user, a generic activity recognition (AR) model inside the wearable devices requires to adapt itself according to the user’s personality in terms of wearing styles and so on. In this paper, an online kernelized and regularized extreme learning machine (OKRELM) is proposed for wearable-based activity recognition. A small-scale but important subset of every incoming data chunk is chosen to go through the update stage during the online sequential learning. Therefore, OKRELM is a lightweight incremental learning model with less time consumption during the update and prediction phase, a robust and effective classifier compared with the batch learning scheme. The performance of OKRELM is evaluated and compared with several related approaches on a UCI online available AR dataset and experimental results show the efficiency and effectiveness of OKRELM.

Recognizing activities of daily living is an important research topic for health monitoring and elderly care. However, most existing activity recognition models only work with static and pre-defined sensor configurations. Enabling an existing activity recognition model to adapt to the emergence of new sensors in a dynamic environment is a significant challenge. In this paper, we propose a novel feature incremental learning method, namely the Feature Incremental Random Forest (FIRF), to improve the performance of an existing model with a small amount of data on newly appeared features. It consists of two important components – 1) a mutual information based diversity generation strategy (MIDGS) and 2) a feature incremental tree growing mechanism (FITGM). MIDGS enhances the internal diversity of random forests, while FITGM improves the accuracy of individual decision trees. To evaluate the performance of FIRF, we conduct extensive experiments on three well-known public datasets for activity recognition. Experimental results demonstrate that FIRF is significantly more accurate and efficient compared with other state-of-the-art methods. It has the potential to allow the dynamic exploitation of new sensors in changing environments.

In activity recognition, it is often expensive and time-consuming to acquire sufficient activity labels. To solve this problem, transfer learning leverages the labeled samples from the source domain to annotate the target domain which has few or none labels. Existing approaches typically consider learning a global domain shift while ignoring the intra-affinity between classes, which will hinder the performance of the algorithms. In this paper, we propose a novel and general cross-domain learning framework that can exploit the intra-affinity of classes to perform intra-class knowledge transfer. The proposed framework, referred to as Stratified Transfer Learning (STL), can dramatically improve the classification accuracy for cross-domain activity recognition. Specifically, STL first obtains pseudo labels for the target domain via majority voting technique. Then, it performs intra-class knowledge transfer iteratively to transform both domains into the same subspaces. Finally, the labels of target domain are obtained via the second annotation. To evaluate the performance of STL, we conduct comprehensive experiments on three large public activity recognition datasets~(i.e. OPPORTUNITY, PAMAP2, and UCI DSADS), which demonstrates that STL significantly outperforms other state-of-the-art methods w.r.t. classification accuracy (improvement of 7.68%). Furthermore, we extensively investigate the performance of STL across different degrees of similarities and activity levels between domains. And we also discuss the potential of STL in other pervasive computing applications to provide empirical experience for future research., 10 pages; accepted by IEEE PerCom 2018; full paper. (camera-ready version)

Sleep plays a vital role in good health and well-being throughout human life. A great deal of studies have been done to detect sleep stages. Most of the current sleep monitoring systems are invasive to users, e.g. requiring users to wear a device during sleep. In this paper, we use microphone to detect sleep stages including deep sleep, light sleep and rapid eye movement (REM), and propose a convolutional neural network using spectrogram as input. This paper's contribution mainly concentrates on the following two aspects: First, microphone is unobtrusive for sleep detection. Second, we build the mapping between acoustic signal and sleep stages with little manual intervention to extract features. Performance of the proposed method is validated on a realistic environmental dataset containing 52 nights of 5 participants. Experimental results show that the accuracy of sleep stages detection is superior to the representative off-the-shelf applications. Besides, we propose to utilize the attention maps to visualize acoustic data to better understand the relationship between sound and sleep stages. Experimental results show that the model has the ability to effectively reduce noises in classification by ignoring the high-frequency sounds and white noises.

Wireless LANs, especially WiFi, have been pervasively deployed and have fostered myriad wireless communication services and ubiquitous computing applications. Indoor localization is an essential modules of these applications. A primary concern in designing scenario-tailored application is to obtain precise estimated distance combatting with harsh indoor wireless signal propagation issues, particularly multipath effect. The conventional propagation model based on received signal strength indicator(RSSI) is easily affected by temporal and spatial fluctuation due to the multipath effect, which leads to most of the distance estimation errors in current indoor localization systems. Intuitively, these positions in weak multipath effect(WME) conditions, which are slightly affected by multipath effect, perform better under free space propagation model. Therefore, the ability to distinguish weak multipath effect, which is slightly affected by multipath effect is a key enabler for accurate distance estimation. Enabling such capabilities on commercial WiFi infrastructure, however, is difficult due to the coarse multipath resolution with the MAC layer RSSI. In this paper, we propose a universal precise distance estimation scheme based on weak multipath effect identification, leveraging the channel state information(CSI) from the PHY layer. In our distance estimation system, we first select positions which are identified as weak multipath effect conditions. Then we build a free space propagation model with RSSI to estimate distance between the transmitter and the receiver, choosing these selected positions. Experimental results demonstrate that choosing positions in weak multipath effect conditions can effectively improve the accuracy of distance estimation in a variety of typical indoor environments.

Falls threaten the health and life of the elders heavily because they lead to injuries or even death. Therefore, a reliable monitoring and alarm mechanism is desperately in need to guarantee the quality of elders' life. In this paper, we propose a multistage machine learning method to perform fall detection and solve the false alarm and missing alarm problem in traditional fall detection methods. Our proposed method consists of three stages: 1) threshold filtering, 2) ELM classifier, and 3) orientation-based filtering. Our method utilizes a high-precision triaxial accelerometer to collect the relevant information. After filtered by our three-stage method, the signal can be determined whether it is a fall or not. Experimental results demonstrate that: different from the traditional state-of-art methods with a single machine learning classifier, our method can greatly reduce the missing alarm and false alarm rate on the premise of high accuracy for all detection.

Various mini-wearable devices have emerged in the past few years to recognize activities of daily living for users. Wearable devices are normally designed to be miniature and portable. Models running on the devices inevitably face following challenges: low-computational-complexity, lightweight and high-accuracy. In order to meet these requirements, a novel powerful activity recognition model named b-COELM is proposed in this paper. b-COELM retains the superiorities (low-computational-complexity, lightweight) of Proximal Support Vector Machine, and extends the powerful generalization ability of Extreme Learning Machine in multi-class classification problems. Experimental results show the efficiency and effectiveness of b-COELM for recognizing activities of daily living.

Active learning approach has been integrated with support vector machine or other machine-learning techniques in many areas. However, the challenge is: Unlabeled instances are often abundant or easy to obtain, but their labels are expensive and time-consuming to get in general. In spite of this, most existing methods cannot guarantee the usefulness of each query in learning a new classifier. In this paper, we propose a new active learning approach of selecting the most informative query for annotation. Unlabeled instance, which is nearest to the support vector machine’s hyperplane learnt from both the unlabeled instance itself and all labeled instances, is selected as the query for annotation. Merits of these queries in learning a new optimal hyperplane have been assured before they are annotated and put into the training set. Experimental results on several UCI datasets have shown the efficiency of our approach.

In this paper, we describe the results of a controlled experiment measuring everyday movement activity through a novel recognition prototype named AIR. AIR measures distance from the feet using infrared (IR) sensors. We tested this approach for recognizing six prevalent activities: standing stationary, walking, running, walking in place, going upstairs, and going downstairs and compared results to other commonly used approaches. Our results show that AIR obtains much higher accuracy in recognizing activity than approaches that rely primarily on accelerometers. Moreover, AIR has good generalization ability when applying recognition model to new users.

Miscellaneous mini-wearable devices (e.g. wristbands, smartwatches, armbands) have emerged in our life, capable of recognizing activities of daily living, monitoring health information, so on. Conventional activity recognition (AR) models deployed inside these devices are generic classifiers learned offline from abundant data. Transferring generic model to user-oriented model requires time-consuming human effort for annotations. To solve this problem, we propose SS-ARTMAP-AR, a self-supervised incremental learning AR model updated from surrounding information such as Bluetooth, Wi-Fi, GPS, GSM data without user's annotation effort. Experimental results show that SS-ARTMAP-AR can gradually adapt individual users, become more incremental intelligence.

Accurate indoor localization is very important for various kinds of Location Based Services (LBS). For most of traditional approaches, the location estimation problems assume the availability of a vast amount of labeled calibrated data, which requires a great deal of manual effort. Previous researches cannot deal with this problem in both calibration reduction, location accuracy. In this paper, we propose a heterogeneous data driven manifold regularization model known as HeterMan to calibration-effort reduction for tracking a mobile node in a wireless sensor network. With the constraint of user heading orientation, we build a mapping function between signal space, physical space with extremely less labeled data, a large amount of unlabeled data. Experimental results show that we can achieve high accuracy with extremely less calibration effort comparing with previous methods. Furthermore, our method can reduce computation complexity, time consumption by parallel processing while maintaining high accuracy.

By utilizing captured characteristics of surrounding contexts through widely used Bluetooth sensor, user-centric social contexts can be effectively sensed and discovered by dynamic Bluetooth information. At present, state-of-the-art approaches for building classifiers can basically recognize limited classes trained in the learning phase; however, due to the complex diversity of social contextual behavior, the built classifier seldom deals with newly appeared contexts, which results in degrading the recognition performance greatly. To address this problem, we propose, an OSELM (online sequential extreme learning machine) based class incremental learning method for continuous and unobtrusive sensing new classes of social contexts from dynamic Bluetooth data alone. We integrate fuzzy clustering technique and OSELM to discover and recognize social contextual behaviors by real-world Bluetooth sensor data. Experimental results show that our method can automatically cope with incremental classes of social contexts that appear unpredictably in the real-world. Further, our proposed method have the effective recognition capability for both original known classes and newly appeared unknown classes, respectively.

Benefit from widely used Bluetooth sensor, user surrounding contexts can be availably recognized leveraging Bluetooth data. Most existing studies seldom deal with newly extended surrounding contexts which results in degrading the recognition performance, in that the built classifier just basically recognizes limited classes learned in training phase. This paper proposes a fuzzy class incremental learning method based on OSELM, named FCI-ELM, for continuously recognizing extended classes of contexts. The encouraging results of our experiments show that FCI-ELM can automatically and continuously recognize newly discovered classes of contexts in the real-world, and the model can keep the same recognition ability by class incremental learning approach.

For the elderly people, timely detecting the fall accident is very critical to receive the first aid. In order to achieve high detection accuracy and low false-alarm rate at the same time, we propose a multimodal wearables based fall detecting and monitoring method leveraging two-stage weighted extreme learning machine. Experimental results show that our method is able to effectively implement on miniaturized wearable devices, and compared to state-of-the-art ELM classifier, we can also obtain higher detection accuracy and lower false-alarm rate simultaneously, which enables various kinds of mHealth applications in large-scale population, especially for the elderly people’s healthcare in the field of fall detection.

User-centric ambient social contexts can be effectively captured by dynamic bluetooth data. However, conventional approaches for training classifiers struggle with social contexts that are incrementally constructed and continuously discovered in everyday environments. Incremental social contexts can confuse a classifier because it assumes that the number and composition of context classes is fixed throughout training and inference phases. To address this challenge we propose ContextSense, an ELM-based learning method for continuous and unobtrusive discovery of new social contexts incrementally from dynamic bluetooth data. Experimental results show that ContextSense can automatically cope with "incremental social context" classes that appear unpredictably in the real-world.

Recent years, miscellaneous mini-wearable devices (e.g. wristbands, wristwatches, armbands) have emerged in our lives to recognize daily activities for the users. Owning to the limitations of hardware, Activity Recognition (AR) models running inside the device are bound to certain challenges, such as processing power, storage capability and battery life. This paper proposes an activity recognition system by considering three limitations above, and a model generation framework to construct AR models which are lightweight in different phases in model generation.

Recent years, a variety of infrastructure-mediated sensing methods have been proposed to recognize activities of daily living (ADLs). However, due to the inconvenience such as high-cost, difficult-to-install, intrusive and applicable to the house with specific architectures alone, existing water-use activity recognition methods cannot be widely used into people's houses. In this paper, a single-point infrastructure-mediated sensing technique is proposed for water-use activity recognition. A single 3-axis accelerometer sensor is attached to the surface of the main water pipe in the house to detect and collect the vibration signals of the main water pipe. These signal data are then processed through six modules in the proposed activity recognition system. Four classes of water-use activities (Bathing, Flushing toilet, Cooking and Washing) are classified by the system and experimental results show that our system can recognize about 70.37% water-use activities.