Search

There are approaches that successfully recognize activities of daily living by using a trained classifier on feature vectors created from binary sensor data. Although these approaches have been successful, there are still open issues such as the evaluation of multiple temporal windows, ensembles of classifiers or unbalanced classes which need to be addressed in order to improve the performance of the real-time activity recognition process. In this paper, we present a methodology for Real-Time Activity Recognition based on the diverse fields of Machine Learning, including Fuzzy Logic and Recurrent Neural Networks. The methodology uses a long-term and short-term representation of binary-sensor activations based on Fuzzy Temporal Windows. The paper proposes an ensemble of activity-based classifiers for the purposes of balanced training, where each classifier in the ensemble is a Long Short-Term Memory. The approach was evaluated using two binary-sensor datasets of daily living activities and benchmarked against previous approaches based on the combination of sensor activation features.

This paper proposes a deep convolutional neural network for HAR using smartphone sensors.Experiments show that the proposed method derives relevant and more complex features.The method achieved an almost perfect classification on moving activities.It outperforms other state-of-the-art data mining techniques in HAR. Human activities are inherently translation invariant and hierarchical. Human activity recognition (HAR), a field that has garnered a lot of attention in recent years due to its high demand in various application domains, makes use of time-series sensor data to infer activities. In this paper, a deep convolutional neural network (convnet) is proposed to perform efficient and effective HAR using smartphone sensors by exploiting the inherent characteristics of activities and 1D time-series signals, at the same time providing a way to automatically and data-adaptively extract robust features from raw data. Experiments show that convnets indeed derive relevant and more complex features with every additional layer, although difference of feature complexity level decreases with every additional layer. A wider time span of temporal local correlation can be exploited (1?9-1?14) and a low pooling size (1?2-1?3) is shown to be beneficial. Convnets also achieved an almost perfect classification on moving activities, especially very similar ones which were previously perceived to be very difficult to classify. Lastly, convnets outperform other state-of-the-art data mining techniques in HAR for the benchmark dataset collected from 30 volunteer subjects, achieving an overall performance of 94.79% on the test set with raw sensor data, and 95.75% with additional information of temporal fast Fourier transform of the HAR data set.

Smartphone sensing capabilities have enabled human activity recognition (HAR) solutions to better understand human behavior through computational techniques. However, such solutions have suffered from scalability problems due to the high consumption of computational resources (e.g. memory and processing) and the difficulty of acting in real time due to not observing data evolution over time. These problems occur because the HAR solutions for smartphones have been solved through offline learning with models limited by a data history. The disadvantage of this approach is that human activities constantly change over time and are strongly influenced by the physical environment and user profile. To overcome such problem, this paper proposes a novel low-cost learning algorithm called NOHAR (NOvelty discrete data stream for Human Activity Recognition), focused on continuous flow of data analysis. NOHAR is an online classification algorithm based on symbolic data generated by a discretization process using algorithms as SAX and SFA. The advantages of these algorithms are their abilities to compress and reduce the dimensionality of data. In addition, this paper proposes a new framework called DISTAR (DIscrete STream learning for Activity Recognition) focused on the data streaming analysis. Its goals include standardizing the development of online algorithms for symbolic data. Experimental results using three databases show that NOHAR is on average 33 times faster than the state of the art and can reduce memory consumption by an average of 99.97%.

Sensor-based activity recognition (AR) is a core problem with the research domain of smart environments. It has, however, the potential to provide solutions to address the problems associated with the growing size and ageing profile of the global population. The work presented within this paper focuses on the extended belief rule-based system (EBRBS), which offered promising performance compared with popular benchmark AR models and exhibited a high robustness in the situation of sensor failure. Nevertheless, efficiency remains one of the major issues to be improved for determining and updating the extended belief rule base (EBRB) within the EBRBS. This is critical for further utilizing the EBRBS in AR situations within dynamic smart environments. An eigendecomposition-based sensor selection method is firstly proposed to select an effective subset of sensors and to also enable efficient implementation to facilitate online AR. A novel domain division-based rule generation method is also proposed to generate and update an EBRB efficiently when new sensor data are available or when some sensors should be included or excluded in the EBRB. The combination of these two methods leads to an enhanced EBRBS, called online updating EBRBS. Two datasets (in a balanced class situation) obtained from simulation and actual environments are studied to provide detailed experimental analysis as a preliminary study and basis to handle further the imbalanced situation of real AR. The experimental results demonstrate an enhanced performance of the online updating EBRBS compared with the original EBRBS and some benchmark AR models, in terms of efficiency and effectiveness.

We demonstrate that different people perform activities differently.Combine AdaBoost with LDA to build general activity model with minimum labelled data.Hybrid AdaBoost with HMM&CRF for temporal regulatization of human activities.Use publicly available datasets to validate the proposed methods. Activity recognition has been a hot topic for decades, from the scientific research to the development of off-the-shelf commercial products. Since people perform the activities differently, to avoid overfitting, building a general model with activity data of various users is required before the deployment for personal use. However, annotating a large amount of activity data is expensive and time-consuming. In this paper, we build a general model for activity recognition with a limited amount of labelled data. We combine Latent Dirichlet Allocation (LDA) and AdaBoost to jointly train a general activity model with partially labelled data. After that, when AdaBoost is used for online prediction, we combine it with graphical models (such as HMM and CRF) to exploit the temporal information in human activities to smooth out the accidental misclassifications. Experiments with publicly available datasets show that we are able to obtain the accuracy of more than 90% with 1% labelled data.

Since years, the number of seniors increases while, at the same time, we observe a diminution of the potential support ratio. In order to overcome this limitation, solutions emerged, such as smart homes and wearable devices. Smart homes integrate sensors, actuators, and artificial intelligence to assist seniors in their everyday life. One of the objectives is to recognize the activities of everyday life. This recognition aims to provide the right assistance at the right moment and gives some autonomy to seniors. However, it is a complex task (a significant quantity of different sensors, hardware implementation), and the number of solutions (combinations between approaches, for example, video-based HAR and wearable sensors-based HAR) that exist is important. In this paper, we propose to perform the activity recognition from three ultra-wideband (UWB) radars, deep learning models, and a voting system. Also, all the experiments have been conducted in a real apartment and are composed of 15 different activities. The presented solution is simple compared to the literature since we exploit only one type of sensor. Finally, we obtained promising results with our approach. Indeed, the classification rate reaches 90% and more in some cases.

Discovering underlying patterns for predicting future actions from spatio-temporal human activity information is a fundamental component of research related to the development of expert systems in human activity recognition and assistive robotics. Current research focuses on classification or learning representations of activities for various applications. However, not much attention is given to the pattern discovery of activities which have a major role in the prediction of unseen actions. This paper proposes a novel Adaptive Segmentation and Sequence Learning (ASSL) framework which aims at segmenting unlabelled observations of human activities from extracted 3D joint information. Learning from these obtained segments provides information about the underlying patterns of activity sequences needed in predicting subsequent actions. In the proposed method, the temporal accumulated motion energy of body parts in an activity is utilised in the segmentation process to obtain key actions from unlabelled activity sequences since body parts show changes in acceleration and deceleration during an activity. Based on the segments obtained, the temporal sequence of transitions across activity segments are learned by employing a Long Short-Term Memory Recurrent Neural Network. This ASSL technique has been evaluated using both an experimental human activity dataset and a public activity dataset, and achieved a better performance when compared with other techniques including an Auto-regressive Integrated Moving Average, Support Vector Regression and Gaussian Mixture Regression Models in learning to predict patterns of activity sequences.

Mobile activity recognition is an effective approach to understanding human context in real time. Existing methods based on supervised learning that require a large amount of training samples for building activity recognition models. The collection of labeled training samples is a boring process and most users are reluctant to get involved. Crowdsourcing is a simple and potential approach to collecting the training samples and building accurate activity recognizers. Since different people usually have different physical features and behavior patterns, an accurate activity recognition model cannot be constructed directly from the training samples collected by crowdsourcing. In this paper, we have proposed a Mixture Expert Model for Activity Recognition (MEMAR) based on feedback and crowdsourcing samples. The proposed model can continuously discover the difference between user activity and crowdsourcing samples. Then we update activity recognition models with the discovered differences. A mobile can correctly utilize crowdsourcing samples for recognition model construction with MEMAR and can also track and recognize mobile users’ behavior dynamics. The experiments based on a smartphone dataset verify the validity of MEMAR. We believe MEMAR provides a basis for context-aware mobile applications.

Self-track the physical activity in order to monitor physical and mental disorders.Transparent and interpretable human activity model.Data acquisition by means of current smartphones placed wherever the user wants.An avatar communicates results in an emotional way according to the user needs.Experimentation presented to show the potential and applicability of the tool. The level of physical activity is one of the most important parameters to monitor the evolution of some physical and mental disorders, e.g., obesity or depression. Its continuous monitoring provides an objective and quantitative measurement tool to be applied by therapists when developing patient-specific treatments. Most of existing activity recognition systems have been developed using multiple accelerometers strapped to the subject's extremities. In this paper, we design an interpretable and transparent model able to describe the basic physical activity and the energy requirements of a person, by capturing the accelerations of its smartphone placed wherever he/she wants. In addition, the use of virtual agents that interact and communicate relevant information in an emotional way, increases the effectiveness and acceptance of the Human Computer Interfaces. Thus, we develop a computational application that combines activity recognition techniques with affective computing ones. We include first experimental results that demonstrate the potential and applicability of this tool.

We present an assistive system for guiding cognitively-impaired people in performing daily activities.The system is based on electrical signal analysis with a single sensor.The system is able to detect cognitive errors and send prompts accordingly.The system has been implemented, deployed and tested in real home environment.The simulation of real-case scenarios has shown promising results. Recently, sensors and actuators have quickly spread throughout our everyday life. These devices are robust, cheap, accessible, connected to the Internet, etc. With the growing needs in terms of human and medical resources to help cognitively-impaired people to remain at home, researchers are investing in new ways to exploit this technology with artificial intelligence, in order to build expert systems to assist the residents in their daily activities. Several systems have been proposed in the last few years, mostly based on binary sensors, cameras and other sensors such as Radio-frequency identification (RFID) tags. Cameras are very intrusive, binary sensors (such as movement detectors) give only basic information, and other types of sensors (such as RFID) need complex deployment. In this context, this paper presents a new assistive expert system based on electric device identification to address the problem of guidance and supervision in the performance of activities for people with cognitive disorders living in a smart home. This system is solely based on a single power analyzer placed in the electric panel. We propose an algorithmic approach used to recognize erratic behaviors related to cognitive deficits and provides cues to guide the person in the completion of an ongoing task. This is achieved through load signatures study of appliances represented by three features (active power (P), reactive power (Q) and line-to-neutral), which allows to determine the errors committed by the resident. We implemented this system within a genuine smart-home prototype equipped with household appliances used by the patient during his morning routines. Different multimedia prompting devices (iPad, screen, speakers, etc.) were used. We tested the system with real-case scenarios modeled from former clinical trials, allowing demonstration of accuracy and effectiveness of our system in assisting a cognitively-impaired resident in the completion of daily activities.

A new approach for the recognition of human activity using silhouette is proposed.The effectiveness of the proposed approach is measured using various classifiers.A new hybrid classification model is proposed to boost the recognition accuracy.The minimum classification error is achieved through a hybrid classification model. The aim of this paper is to present a new approach for human activity recognition in a video sequence by exploiting the key poses of the human silhouettes, and constructing a new classification model. The spatio-temporal shape variations of the human silhouettes are represented by dividing the key poses of the silhouettes into a fixed number of grids and cells, which leads to a noise free depiction. The computation of parameters of grids and cells leads to modeling of feature vectors. This computation of parameters of grids and cells is further arranged in such a manner so as to preserve the time sequence of the silhouettes. To classify, these feature vectors, a hybrid classification model is proposed based upon the comparative study of Linear Discriminant Analysis (LDA), K-Nearest Neighbor (K-NN) and Support Vector Machine (SVM) classifier. The proposed hybrid classification model is a combination of SVM and 1-NN model and termed as 'SVM-NN'. The effectiveness of the proposed approach of activity representation and classification model is tested over three public data sets i.e. Weizmann, KTH, and Ballet Movement. The comparative analysis shows that the proposed method is superior in terms of recognition accuracy to similar state-of-the-art methods.

We propose an efficient frequent activity patterns mining in smart environments.We build an accurate activity classifier based on the mined frequent patterns.We distinguish overlapped activities with global and local weights of sensor events.We use publicly available dataset of smart environments to validate our methods. In the past decades, activity recognition has aroused a great interest for the research groups majoring in context-awareness computing and human behaviours monitoring. However, the correlations between the activities and their frequent patterns have never been directly addressed by traditional activity recognition techniques. As a result, activities that trigger the same set of sensors are difficult to differentiate, even though they present different patterns such as different frequencies of the sensor events. In this paper, we propose an efficient association rule mining technique to find the association rules between the activities and their frequent patterns, and build an activity classifier based on these association rules. We also address the classification of overlapped activities by incorporating the global and local weight of the patterns. The experiment results using publicly available dataset demonstrate that our method is able to achieve better performance than traditional recognition methods such as Decision Tree, Naive Bayesian and HMM. Comparison studies show that the proposed association rule mining method is efficient, and we can further improve the activity recognition accuracy by considering global and local weight of frequent patterns of activities.

Skin segmentation is one of the most important tasks for human activity recognition, video monitoring, face detection, hand gesture recognition, content-based detection, adult content filtering, human tracking, and robotic surgeries. Skin segmentation in ideal situations is easy to accomplish because it is with similar backgrounds. However, the skin segmentation in non-ideal situations is complicated due to difficult background illuminations, the presence of skin-like pixels, and environmental changes. The current studies are handling the mentioned challenges by adding the preprocessing stages in their methods, which increases the overall cost of the system. In addition, prevailing segmentation studies have ignored black skin and mainly focused on white skin for their experiments. To deal with skin segmentation in challenging environments irrespective of skin color, and to eliminate the cost of the preprocessing, this paper proposes an outer residual skip connection-based deep convolutional neural network (OR-Skip-Net) which innovatively empowers the features by transferring the direct edge information from the initial layer to the end of the network. Experiments were performed on the following eight open datasets for skin segmentation in different environments: hand gesture recognition dataset, event detection dataset, laboratoire d'informatique en image et systemes d'information dataset, in-house dataset, UT-interaction dataset, augmented multi-party interaction dataset, Pratheepan dataset, and black skin people dataset. In addition, two other experiments were performed for gland segmentation from colon cancer histology images for the diagnosis of colorectal cancer using the Warwick-QU dataset and for iris segmentation using the Noisy Iris Challenge Evaluation - Part II dataset to explore the possibility of applying our method to different applications. Experimental results showed that the proposed OR-Skip-Net outperformed existing methods in terms of skin, gland, and iris segmentation accuracies.

International audience; Human Activity Recognition (HAR) is an important research issue for pervasive computing that aims to identify human activities in smart homes. In the literature, most reasoning approaches for HAR are based on centralized approach where a central system is responsible for processing and reasoning about sensor data in order to recognize activities. Since sensor data are distributed, heterogeneous, and dynamic (i.e., whose characteristics are varying over time) in the smart home, reasoning process on these data for HAR needs to be distributed over a group of heterogeneous, autonomous and interacting entities in order to be more efficient. This paper proposes a main contribution, the DCR approach, a fully Distributed Collaborative Reasoning multi-agent approach where agents, with diverse classifiers, observe sensor data, make local predictions, communicate and collaborate to identify current activities. Then, an improved version of the DCR approach is proposed, the DCR-OL approach, a distributed Online Learning approach where learning agents learns from their collaborations to improve their own performance in activity recognition. Finally, we test our approaches by performing an evaluation study on Aruba dataset, that indicates an enhancement in terms of accuracy, F-measure and G-mean metrics compared to the centralized approach and also compared to a distributed approach existing in the literature.

Despite the increasing developments on human activity recognition using wearable technology, there are still many open challenges in spotting and recognising sporadic gestures. As opposed to activities, which exhibit continuous behaviour, the difficulty of spotting gestures lies in their rather sparse nature. This paper proposes a novel solution to spot and recognise a set of similar eating and drinking gestures from continuous inertial data streams. First, potential segments containing an eating or a drinking gesture are found using a Crossings-based Adaptive Segmentation Technique (CAST). Second, further to the long-established range of features employed in previous human activities recognition research work, a gesture discrepancy measure is proposed to improve the classification performance of the system. At the final step, a range of state-of-the-art classification models is employed for evaluation. Various conclusions can be drawn from the results obtained. First, given the 100% recall achieved at the segmentation step, the CAST can be considered a reliable segmentation technique for spotting drinking and eating gestures which may be employed in future gesture spotting work. Second, the addition of gesture discrepancy as a feature descriptor consistently improves the classification performance of the system. Third, the reliability of the food and drink intake monitoring approach proposed in this work finds support on the out-performance of previous similar work.

Few-shot learning is a technique to learn a model with a very small amount of labeled training data by transferring knowledge from relevant tasks. In this paper, a few-shot learning method for wearable sensor based human activity recognition, which is a technique that seeks high-level human activity knowledge from low-level sensor inputs, is proposed. Due to the high costs to obtain human generated activity data and the ubiquitous similarities between activity modes, it can be more efficient to borrow information from existing activity recognition models than to collect more data to train a new model from scratch when only a few data are available for model training. The proposed few-shot human activity recognition method leverages a deep learning model for feature extraction and classification while knowledge transfer is performed in the manner of model parameter transfer. In order to alleviate negative transfer, a metric is proposed to measure cross-domain class-wise relevance so that knowledge of higher relevance is assigned larger weights during knowledge transfer. Promising results in extensive experiments show the advantages of the proposed approach.

To provide more sophisticated healthcare services, it is necessary to collect the precise information on a patient. One impressive area of study to obtain meaningful information is human activity recognition, which has proceeded through the use of supervised learning techniques in recent decades. Previous studies, however, have suffered from generating a training dataset and extending the number of activities to be recognized. In this paper, to find out a new approach that avoids these problems, we propose unsupervised learning methods for human activity recognition, with sensor data collected from smartphone sensors even when the number of activities is unknown. Experiment results show that the mixture of Gaussian exactly distinguishes those activities when the number of activities k is known, while hierarchical clustering or DBSCAN achieve above 90% accuracy by obtaining k based on Calinski–Harabasz index, or by choosing appropriate values for ɛ and MinPts when k is unknown. We believe that the results of our approach provide a way of automatically selecting an appropriate value of k at which the accuracy is maximized for activity recognition, without the generation of training datasets by hand.

Human activity recognition systems are currently implemented by hundreds of applications and, in recent years, several technology manufacturers have introduced new wearable devices for this purpose. Battery consumption constitutes a critical point in these systems since most are provided with a rechargeable battery. In this paper, by using discrete techniques based on the Ameva algorithm, an innovative approach for human activity recognition systems on mobile devices is presented. Furthermore, unlike other systems in current use, this proposal enables recognition of high granularity activities by using accelerometer sensors. Hence, the accuracy of activity recognition systems can be increased without sacrificing efficiency. A comparative is carried out between the proposed approach and an approach based on the well-known neural networks.