Search

With sensors built into smartphones and wearable devices, Human Activity Recognition (HAR) makes it possible to identify regular activities. One of the challenges is to access the motion information from activity data and extract the features from this information. A general model trained without motion features may perform poorly while recognizing the contextual patterns in human activities. In this paper, we proposed the mHAR (motion HAR) model for recognizing static, gradual, and dynamic motion activity to address this challenge. We examine the impact of the proposed model against the general non-convolutional and convolutional recurrent networks in terms of accuracy rates and F1 score. As an experimental dataset, the publicly accessible WISDM was employed. Additionally, the findings of the experiments demonstrate that, in contrast to the unbalanced class problem, motion features play a significant role in activity recognition by achieving accuracy rates of 92.29% and an average F1 score of 83.33%. [ABSTRACT FROM AUTHOR]

Human Activity Recognition (HAR) using wearable Internet of Things (IoT) devices represents a well investigated researched field encompassing various application domains. Many current approaches rely on cloud-based methodologies for gathering data from diverse users, resulting in the creation of extensive training datasets. Although this strategy facilitates the application of powerful Machine Learning (ML) techniques, it raises significant privacy concerns, which can become particularly severe given the sensitivity of HAR data. Moreover, the labeling process can be extremely time-consuming and even more challenging for IoT wearable devices due to the absence of efficient input systems. In this paper, we address both aforementioned challenges by designing, implementing, and validating edge-based Human Activity Recognition (HAR) systems that operate on resource-constrained IoT devices, which relies on the utilization of Self-Organizing Maps (SOM) for activity detection. We incorporate a feature selection process before training to reduce data dimensionality and, consequently, the SOM size, aligning with the resource limitations of wearable IoT devices. Additionally, we explore the application of Federated Learning (FL) techniques for HAR tasks, enabling new users to leverage SOM models trained by others on their respective datasets. Our federated Extreme Edge (EE)-aware HAR system is implemented on a wearable IoT device and rigorously tested against state-of-the-art and experimental datasets. The results demonstrate that our C++-based SOM implementation achieves a consistent reduction in model size compared to state-of-the-art approaches. Furthermore, our findings highlight the effectiveness of the FL-based approach in overcoming personalized training challenges, particularly in onboarding scenarios. [ABSTRACT FROM AUTHOR]

Human Activity Recognition (HAR) and fall detection, as applications within the field of biomedical signal processing, are increasingly pivotal in enhancing patient care, preventive healthcare, and rehabilitation. Fall therapy is one of the most expensive treatments, usually taking a long time to complete. A single fall accident might result in serious injuries, long-term incapacity, or even death. As a result, a reliable and cost-effective fall detection system is essential. Wearable sensors have received wide attention due to their availability and capability to capture different human motions. Thus, in the current study, we develop a comprehensive HAR system for multi-classification tasks to recognize several human actions, such as walking, sitting, standing, falling, and others. At the same time, a binary classification of this model is developed to recognize fall and non-fall actions, which can be used to track elderly actions and send an alert in case of falling to do necessary rescue actions. The developed system is built using a Parallel Convolutional Neural Network and Transformer-based architecture (PCNN-Transformer). PCNN-Transformer benefits from the parallel architecture and the residual mapping mechanism to learn temporal feature representations from the sensors' data. The CNN blocks are aligned in parallel alongside several Transformer-based encoders, followed by a concatenation operation to sum up the extracted features from the input data (sensors data). Moreover, the CNN blocks implement a residual mapping mechanism to reduce the model complexity and training time. The proposed model is tested using several open-source datasets: SisFall, UniMib-SHAR, and MobiAct. It recorded high accuracy rates compared to several deep learning models. For instance, in the binary classification (fall detection), the proposed model achieved an average accuracy of 99.95%, 98.68%, and 99.71% for SisFall, UniMib-SHAR, and MobiAct, respectively. • Propose a comprehensive HAR and fall detection model called PCNN-Transformer based on smartphone sensors. • PCNN-Transformer implements a parallel structure using CNN blocks and Transformer encoders. • Apply residual mapping mechanism as a feature extractor using deep CNN to expose relevant features. • Implement extensive evaluation experiments using three public HAR datasets with extensive comparisons. [ABSTRACT FROM AUTHOR]

Gesture recognition using WiFi is vital for human–computer interaction, smart homes, smart spaces, and elderly care. WiFi signals are non-stationary, sensitive to the environment, and traditional pattern recognition-based HAR methods incur exorbitant training and deployment costs due to their reliance on the quality and quantity of training data. While there has been extensive research on enhancing HAR techniques, the efficiency of the system is still limited by the scarcity of training data and identity-action feature coupling. This research focuses on attaining gesture recognition across multiple users with a limited number of samples. Our findings show that the conventional data augmentation methods are incapable of facilitating the system in attaining sample diversity. As a result, we propose a training-free augmentation strategy as a means to provide adequate training data. Unlike the conventional data enhancement approach, this scheme aims to develop data processing methods that differentiate between various samples in practical applications. Consequently, it enhances data pertinent to specific HAR assignments effectively. To effectively extract action features in WiFi samples, an unsupervised c ross- u ser d omain s ample g eneration (CUDSG) model is proposed. This model generates virtual gesture samples for new user domains by decoupling and recombining gesture and identity features. This extends the sensing boundary of the system to new user domains without requiring a significant number of users to participate. Model performance was evaluated using various classifiers, such as SVM, KNN and CNN. The results demonstrate a significant improvement in average classification accuracy from 57.3% to 98.4%. This indicates that CUDSG is a highly effective tool for enhancing the performance of existing gesture recognition techniques. • A deep learning framework is harness to decouple the fusion features in wireless sensing. • A low-cost, training-free data augmentation scheme is proposed. • The experimental results show that our method can effectively improve the performance of existing gesture recognition method. [ABSTRACT FROM AUTHOR]

A significant quantity of research work has been completed to recognize human activities. The majority of the proposed learning algorithms have treated the activity data as the role of fuel in vehicles. The capacity of the learning algorithms to recognize the activity patterns can be improved by the adequate availability of activity data. In this paper, we introduced the FLAAP (Finding and Learning the Associated Activity Patterns) activity dataset and data acquired by using the smartphone (accelerometer and gyroscope) sensors placed at the waist of the subjects while performing the activities. This dataset contains the record of ten activities performed by eight distinct subjects. Between February 1st and May 31st, 2022, millions of raw sensor activity data samples were captured constantly at 100Hz sampling rates. The Human Activity Recognition (HAR) datasets, which keep a record of such activities and report associations in activity patterns (mostly used for recognizing the Activities of Daily Living (ADL)), were lacking. This paucity is addressed by the FLAAP dataset which can be useful in finding the associated patterns in ADL. The obtained experimental findings demonstrate that the learning algorithm Random Forest (RF), which was used, has recognized the activities with around 77.22% accuracy. The applied RF learning algorithm on the FLAAP dataset provides the research gap for the researchers in developing more delicate learning models for enhancing recognition rates. Furthermore, the research community could be particularly interested in examining the learning performance of algorithms while using various data pre-processing techniques, transferring knowledge to target domains, and other techniques. [ABSTRACT FROM AUTHOR]

AM-FM decomposition techniques have been successfully used for extracting significative features from a large variety of signals, helping realtime signal monitoring and pattern recognition, since they represent signals as a simultaneous composition of amplitude modulation and frequency modulation, where the carriers, amplitude envelopes, and the instantaneous frequencies are the features to be estimated. Human activities often involve repetitive movements, such as in running or cycling, where sinusoidal AM-FM decompositions of signals have already demonstrated to be useful to extract compact features to aid monitoring, classification, or detection. In this work we thus present the challenges and results of implementing the iterated coherent Hilbert decomposition (ICHD), a particularly effective algorithm to obtain an AM-FM decomposition, within a resource-constrained and low-power ARM Cortex-M4 microcontroller that is present in a wearable sensor we developed. We apply ICHD to the gyroscope data acquired from an inertial measurement unit (IMU) that is present in the sensor. Optimizing the implementation allowed us to achieve real-time performance using less then 16 % of the available CPU time, while consuming only about 5.4 mW of power, which results in a run-time of over 7 days using a small 250 mAh rechargeable cell. [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) based on multimodal wearable motion sensors is a valuable technology, and gymnastics action recognition is one of its important application scenarios. However, this field faces challenges such as excessive sensor node deployment, hindering effective utilization of time-series sensor data, and low accuracy in action recognition. To address these issues, this paper introduces a gymnastics action recognition system comprising a hardware setup and an action recognition framework that combines a time series data imaging method with a novel three-channel convolutional model ViTGS. Firstly, a cost-effective single-node sensor system equipped with a 6-axis IMU is devised to efficiently capture gymnastics action data. Secondly, three one-dimensional time series data image encoding methods, Gramian Angular Field (GAF), Recurrence Plot (RP), and Markov Transition Field (MTF), are employed to capture the temporal features, phase space features, and state transition features of the data respectively. Finally, the ViTGS model is proposed, which uses the ViT_B/16 network to extract the features of each channel image, and uses Gated Fusion Network (GFN) to realize the effective fusion of different channel features and uses SVM as the recognition layer to further improve the classification accuracy. The experimental results show that using single-node (left wrist) sensor data, an accuracy of 98.56% is obtained on the acceleration dataset, and an accuracy of 99.04% is obtained on the angular velocity dataset. The method of this paper is not only applicable to gymnastics action recognition but also can be generalized to other human action recognition fields. [Display omitted] [ABSTRACT FROM AUTHOR]

In today's healthcare applications, the use of mobile technologies brings together physicians and patients for intelligent and automatic monitoring of daily clinical activities, remote life assistants, and preventive care, especially for the elderly and those under medical control. As smartphones become an important part of our everyday life, they are ever more employed in human activities recognition (HAR) including the monitoring of personal health care and wellbeing. However, HAR is complex and it is important to use the best technology and learn about human activity using machine learning. The purpose of this paper is to develop a HAR system based on the smartphone sensors' data using Bagging and Adaboost ensemble classifiers. The experimental results for the HAR data have been evaluated after performing different data mining techniques. For each subject, the total classification accuracy, the F-measure, and the ROC area were calculated. Adaboost ensemble classifiers algorithm improved significantly the performance of smartphone-based HAR, combined with SVM, it reached 97.44% accuracy compared to the rest of the classifiers. The proposed algorithm of Adaboost SVM can lead to an accurate HAR for elderly and disabled patients who need continuous care as well as it is a tool that supports the decisions of all medical practitioners. [ABSTRACT FROM AUTHOR]

Extracting representative features to recognize human activities through the use of wearables is an area of on-going research. While hand-crafted features and machine learning (ML) techniques have been sufficiently well investigated in the past, the use of deep learning (DL) techniques is the current trend. Specifically, Convolutional Neural Networks (CNNs), Long Short Term Memory Networks (LSTMs), and hybrid models have been investigated. We propose a novel hybrid network architecture to recognize human activities through the use of wearable motion sensors and DL techniques. The LSTM and the 2D CNN branches of the model that run in parallel receive the raw signals and their spectrograms, respectively. We concatenate the features extracted at each branch and use them for activity recognition. We compare the classification performance of the proposed network with three single and three hybrid commonly used network architectures: 1D CNN, 2D CNN, LSTM, standard 1D CNN-LSTM, 1D CNN-LSTM proposed by Ordóñez and Roggen, and an alternative 1D CNN-LSTM model. We tune the hyper-parameters of six of the models using Bayesian optimization and test the models on two publicly available datasets. The comparison between the seven networks is based on four performance metrics and complexity measures. Because of the stochastic nature of DL algorithms, we provide the average values and standard deviations of the performance metrics over ten repetitions of each experiment. The proposed 2D CNN-LSTM architecture achieves the highest average accuracies of 95.66% and 92.95% on the two datasets, which are, respectively, 2.45% and 3.18% above those of the 2D CNN model that ranks the second. This improvement is a consequence of the proposed model enabling the extraction of a broader range of complementary features that comprehensively represent human activities. We evaluate the complexities of the networks in terms of the total number of parameters, model size, training/testing time, and the number of floating point operations (FLOPs). We also compare the results of the proposed network with those of recent related work that use the same datasets. [Display omitted] • We propose a new hybrid deep learning (DL) model with two parallel branches. • The two branches (LSTM & 2D-CNN) receive raw data and their spectrograms as input. • We implement and compare the performance & complexity of seven DL models altogether. • DL models process wearable motion sensor data for daily/sports activity recognition. • Proposed hybrid model exhibits superior performance metrics & acceptable complexity. [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) enables the recognition of the activities of daily living using signals from motion sensors integrated into mobile and wearable devices. One of the challenges is the uniqueness of each individual with his/her different characteristics. A general model trained without user data may perform poorly on specific users. Another challenge is running deep learning (DL) models on mobile and wearable devices due to their limited resources. In this paper, to cope with these two challenges, we use transfer learning to build personalized models and model compression for running DL algorithms. We examine the impact of different DL architectures, the number of layers to be fine-tuned, the amount of user training data, and the transfer to new datasets on the performance of HAR. We compare the performance of the transferred models with general and user-specific models in terms of F1 score, training and inference time. [Display omitted] • We use transfer learning (TL) to build personalized models for Human Activity Recognition (HAR). • We compress DL models using quantization to be deployed on mobile devices with limited resources. • We examine the impact of different architectures and layers to fine-tune on the performance of TL. • TL is compared with a general and user-specific model. • Using TL results in higher F1 scores and less running time. • Even if new labels are introduced, TL performs better, with similar positions and classes. • When the models are compressed, the inference time reduces with no compromise in accuracy. [ABSTRACT FROM AUTHOR]

Dependable and accurate monitoring of elderly at home becomes crucial to limit both the costs and human efforts of following up elderly for establishing a healthy care system. Human Activity Recognition (HAR) tools, based on sensors installed in smart homes, will become an important tool to provide useful information to the caregiver when something happens in the house of an elderly and care is required. The current available detection tools either exist out of interpretable knowledge-driven techniques or scalable data-driven ones. In this paper, a hybrid methodology that combines both approaches is designed and evaluated to Track Activities by Linking Knowledge (TALK). Both sensor data and their link to the relevant domain knowledge about where those sensors are installed, the performed activities that occur, and how the household is constructed, are generalized in a specific knowledge graph (KG) structure to represent continuous events. The interpretable knowledge graph embedding technique Instance Neighboring using Knowledge (INK) is then used to transform these events inside the KG to a tabular format, which can be used by any traditional machine learning classifier to create a HAR tool. The TALK methodology is evaluated on two HAR datasets and shows (a) that TALK outperforms both traditional automated data-driven as well as knowledge-driven techniques in terms of predictive performance, and (b) how TALK can be easily used in a more out of lab environment. All these results and the interpretable aspects show that TALK can become an important tool to monitor elderly in their homes efficiently, effectively and with less intrusive techniques. [ABSTRACT FROM AUTHOR]

Human Activity Recognition (HAR) remains a challenging issue that requires to be resolved. Utilizing images, smart phones, or sensors, HAR could be done. Recent studies have explored HAR utilizing Unmanned Aerial Vehicle (UAV) videos. But, owing to numerous restrictions correlated to the platform, HAR from videos captured by drones remains a challenge. To resolve this issue, a novel mechanism for HAR from UAV videos utilizing the Diminutive Multi-Dimensional Locality Coding based Convolutional Neural Network (DMLC-CNN) model is proposed. Primarily, the input video is converted into frames; also, Bounding Boxes (BBs) are produced for humans. For attaining the number of clusters grounded on human presence, the BB is given to the Robust Lucrative Sensitive Amalgam K-Nearest Neighbor (RoLSA-KNN) clustering. After that, utilizing the LR-fit-centric Superior Photogrammetric Silhouette (LR-SPS) segmentation, the humans' silhouettes are segmented. The attained output is divided into patches; then, it is converted into stacked layer frames. Afterward, these frames are converted into a 3D kernel. The Diminutive Multi-Dimensional Hyper Graph (DMHG) extracted the features; then, the Locality-constrained Orthonormal Feature Space Coding (LOFSC) Algorithm encoded the extracted features. Lastly, for recognizing Human Activity (HA), DMLC-CNN is wielded. The proposed framework is contrasted with the conventional techniques. After experimental evaluation, the proposed mechanism was found to be more efficient in HAR. • A novel framework for HAR utilizing a DMLC-CNN model grounded on UAV videos is proposed. • The experimental evaluation has been executed. • then, the techniques' performances are assessed concerning various metrics. • The outcomes exhibited that the proposed technique achieves better accuracy (94.50%). • Moreover, the proposed model consumed less time of 36,017 ms for clustering. [ABSTRACT FROM AUTHOR]

Human Activity Recognition (HAR) is very useful for the clinical applications, and many machine learning algorithms have been successfully implemented to achieve high-performance results. Although handcrafted feature extraction techniques were used in the past, Artificial Neural Network (ANN) is now more popular. In this work, a model has been proposed called Gated Recurrent Unit-Inception (GRU-INC) model has been proposed, which is an Inception-Attention based approach using Gated Recurrent Unit (GRU) that effectively makes use of the temporal and spatial information of the time-series data. The proposed model achieved an F1-score of 96.27%, 90.05%, 90.30%, 99.12%, and 95.99% on the publicly available datasets such as, UCI-HAR, OPPORTUNITY, PAMAP2, WISDM, and Daphnet, respectively. GRU along with Attention Mechanism (AM) was utilized for the temporal part, and Inception module along with Convolutional Block Attention Module (CBAM) was exploited for the spatial part of the model. The proposed architecture was evaluated against state-of-the-art models and similar works. It has been proved that the GRU-INC model has a higher recognition rate as well as lower computational cost. Thus our framework could be applicable in activity associated clinical and rehabilitation applications. • A GRU-Inception based deep learning model to identify human activities. • Attention mechanism is incorporated with GRU to improve temporal feature extraction. • Inception modules along with a CBAM block further highlights the spatial features. • The model is relatively wider rather than a deep structure thus reducing complexity. [ABSTRACT FROM AUTHOR]

• Deep adaptation transfer learning combined with OS-ELM to handle cross domain HAR. • GAP and SE structure in CNN facilitate feature extraction and adapt to input data. • DANN and DDC have different effects on cross-person and cross-position transfer. • OS-ELM classifier improves HAR accuracy with a few annotated data in target domain. Deep learning (DL) has been evolving to a prevalent method in human activity recognition (HAR). However, the performance of wearable sensor based HAR models decline significantly when training data come from different persons or sensor positions, and a time-consuming data annotation is indispensible to cater for the big-data driven DL models. In this paper we proposed a fast and robust hybrid model to handle the transfer issues of wearable sensor based HAR between different persons (cross-person) and different positions (cross-position) with just a few annotated data in target domain. The model consists of three parts: (1) A convolutional neural network (CNN) with global average pooling layer to facilitate the extraction of advanced common features in source domain and target domain; (2) A domain adaptive neural network with a gradient reversal layer (DANN) and deep domain confusion network with an adaptive layer (DDC) to reduce domain shift caused by the change of persons and sensor positions; (3) An adaptive classifier based on online sequential extreme learning machine (OS-ELM) to achieve fast and accurate classification with a few annotated data in target domain. Experimental results on four public datasets verified the superiority of the proposed hybrid model over standard CNN and deep transfer learning models in adapting the classifier to new sensor locations and subjects quickly, where the HAR accuracy can be improved by at least 12% for cross-person transfer and 20% for cross-position transfer, respectively. [ABSTRACT FROM AUTHOR]

The aim of this paper is to present a new approach for human activity recognition in a video sequence by exploiting the key poses of the human silhouettes, and constructing a new classification model. The spatio-temporal shape variations of the human silhouettes are represented by dividing the key poses of the silhouettes into a fixed number of grids and cells, which leads to a noise free depiction. The computation of parameters of grids and cells leads to modeling of feature vectors. This computation of parameters of grids and cells is further arranged in such a manner so as to preserve the time sequence of the silhouettes. To classify, these feature vectors, a hybrid classification model is proposed based upon the comparative study of Linear Discriminant Analysis (LDA), K-Nearest Neighbor (K-NN) and Support Vector Machine (SVM) classifier. The proposed hybrid classification model is a combination of SVM and 1-NN model and termed as ‘SVM–NN’. The effectiveness of the proposed approach of activity representation and classification model is tested over three public data sets i.e. Weizmann, KTH, and Ballet Movement. The comparative analysis shows that the proposed method is superior in terms of recognition accuracy to similar state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Nowadays, people use smart devices everywhere and for different applications such as healthcare. The Internet of Healthcare Things (IoHT) generates enormous amounts of data daily, which need exploitation and analysis to help healthcare professionals make decisions and provide a fast diagnosis. Human Activity Recognition (HAR) has received more attention due to its importance in elderly care, lifestyle improvement, and IoT systems. This paper presents a novel HAR system based on optimizing two algorithms: convolutional neural network (CNN) and the recently proposed optimization algorithm, Arithmetic Optimization Algorithm (AOA), to boost the HAR performance with fewer resources. The proposed CNN is applied to learn and extract features from input data where a modified AOA algorithm, called Binary AOA (BAOA), is used to select the most optimal features. Finally, the support vector machine (SVM) is adopted to classify the selected feature based on different activities. We evaluate the proposed HAR model with three different public datasets, UCI-HAR, WISDM-HAR, and KU-HAR datasets. Moreover, we compare the feature selection method, BAOA, to various optimization algorithms using several evaluation measures, and we found that BAOA recorded the best performance. Furthermore, we compare the proposed model to several existing HAR methods. The outcomes confirmed the competitive performance of the proposed model, which achieved 95.23%, 99.5%, and 96.8% for UCI-HAR, WISDM-HAR, and KU-HAR datasets, respectively. • Propose a new HAR system using the advantages of Metaheuristic and deep learning. • Develop a feature extraction method using deep CNN to expose relevant features. • Propose a feature selection method based on Binary Athematic Optimization Algorithm • Implement extensive evaluation experiment using three public HAR datasets. [ABSTRACT FROM AUTHOR]

• A four-layer edge optimized and user-personalized framework for life-logging human activities is proposed. • A lightweight edge intelligence module requiring low computation is designed, which reduces data transmission to the cloud. • A novel Max Score Pooling (MSP) algorithm based on ensembled metaheuristic algorithms is developed for the user-specific parameter selection. Additionally, it makes the framework resilient to certain sensor failures. • The MSP optimized Decision Tree classifier is developed for real-time activity recognition in the Spark environment. • Experimental evaluation demonstrates the outperformance of the proposed model with existing ones. The fostered use of smart wearables for lifelogging daily activities has fuelled massive data generation. Lack of personalization, massive network traffic, increased latency, and high vulnerability to missing and noisy data are the significant impediments that existing frameworks face. This paper proposes a user-personalized and edge-optimized four-layer framework for lifelogging activities to address these impediments. A lightweight Edge Intelligence (EI) module with low computation requirements is designed to reduce data transmission to the cloud, lowering energy consumption. A novel Max Score Pooling (MSP) algorithm based on ensembled metaheuristic algorithms is proposed to provide a user-specific and optimized set of features. MSP optimized Decision Tree (MSP-DT) classifier is developed for real-time activity recognition in the Spark environment. The classifier's performance is calibrated regularly, making the framework resilient to sensor failure. Experiments demonstrate that the proposed framework can recognize 12 physical activities of different subjects with a mean accuracy of 97.67% and 47.66% reduction in transmitted data. [Display omitted] [ABSTRACT FROM AUTHOR]