Search

Analyzing physical activities through wearable devices is a promising research area for improving health assessment. This research focuses on the development of an affordable and real-time Human Activity Recognition (HAR) system designed to operate on low-performance microcontrollers. The system utilizes data from a body-worn accelerometer to recognize and classify human activities, providing a cost-effective, easy-to-use, and highly accurate solution. A key challenge addressed in this study is the execution of efficient motion recognition within a resource-constrained environment. The system employs a Random Forest (RF) classifier, which outperforms Gradient Boosting Decision Trees (GBDT), Support Vector Machines (SVM), and K-Nearest Neighbors (KNN) in terms of accuracy and computational efficiency. The proposed features Average absolute deviation (AAD), Standard deviation (STD), Interquartile range (IQR), Range, and Root mean square (RMS). The research has conducted numerous experiments and comparisons to establish optimal parameters for ensuring system effectiveness, including setting a sampling frequency of 50 Hz and selecting an 8-s window size with a 40% overlap between windows. Validation was conducted on both the WISDM public dataset and a self-collected dataset, focusing on five fundamental daily activities: Standing, Sitting, Jogging, Walking, and Walking the stairs. The results demonstrated high recognition accuracy, with the system achieving 96.7% on the WISDM dataset and 97.13% on the collected dataset. This research confirms the feasibility of deploying HAR systems on low-performance microcontrollers and highlights the system’s potential applications in patient support, rehabilitation, and elderly care. [ABSTRACT FROM AUTHOR]

Activity recognition combined with artificial intelligence is a vital area of research, ranging across diverse domains, from sports and healthcare to smart homes. In the industrial domain, and the manual assembly lines, the emphasis shifts to human–machine interaction and thus to human activity recognition (HAR) within complex operational environments. Developing models and methods that can reliably and efficiently identify human activities, traditionally just categorized as either simple or complex activities, remains a key challenge in the field. Limitations of the existing methods and approaches include their inability to consider the contextual complexities associated with the performed activities. Our approach to address this challenge is to create different levels of activity abstractions, which allow for a more nuanced comprehension of activities and define their underlying patterns. Specifically, we propose a new hierarchical taxonomy for human activity abstraction levels based on the context of the performed activities that can be used in HAR. The proposed hierarchy consists of five levels, namely atomic, micro, meso, macro, and mega. We compare this taxonomy with other approaches that divide activities into simple and complex categories as well as other similar classification schemes and provide real-world examples in different applications to demonstrate its efficacy. Regarding advanced technologies like artificial intelligence, our study aims to guide and optimize industrial assembly procedures, particularly in uncontrolled non-laboratory environments, by shaping workflows to enable structured data analysis and highlighting correlations across various levels throughout the assembly progression. In addition, it establishes effective communication and shared understanding between researchers and industry professionals while also providing them with the essential resources to facilitate the development of systems, sensors, and algorithms for custom industrial use cases that adapt to the level of abstraction. [ABSTRACT FROM AUTHOR]

Nowadays, for controlling crime, surveillance cameras are typically installed in all public places to ensure urban safety and security. However, automating Human Activity Recognition (HAR) using computer vision techniques faces several challenges such as lowlighting, complex spatiotemporal features, clutter backgrounds, and inefficient utilization of surveillance system resources. Existing attempts in HAR designed straightforward networks by analyzing either spatial or motion patterns resulting in limited performance while the dual streams methods are entirely based on Convolutional Neural Networks (CNN) that are inadequate to learning the long-range temporal information for HAR. To overcome the above-mentioned challenges, this paper proposes an optimized dual stream framework for HAR which mainly consists of three steps. First, a shots segmentation module is introduced in the proposed framework to efficiently utilize the surveillance system resources by enhancing the lowlight video stream and then it detects salient video frames that consist of human. This module is trained on our own challenging Lowlight Human Surveillance Dataset (LHSD) which consists of both normal and different levels of lowlighting data to recognize humans in complex uncertain environments. Next, to learn HAR from both contextual and motion information, a dual stream approach is used in the feature extraction. In the first stream, it freezes the learned weights of the backbone Vision Transformer (ViT) B-16 model to select the discriminative contextual information. In the second stream, ViT features are then fused with the intermediate encoder layers of FlowNet2 model for optical flow to extract a robust motion feature vector. Finally, a two stream Parallel Bidirectional Long Short-Term Memory (PBiLSTM) is proposed for sequence learning to capture the global semantics of activities, followed by Dual Stream Multi-Head Attention (DSMHA) with a late fusion strategy to optimize the huge features vector for accurate HAR. To assess the strength of the proposed framework, extensive empirical results are conducted on real-world surveillance scenarios and various benchmark HAR datasets that achieve 78.6285%, 96.0151%, and 98.875% accuracies on HMDB51, UCF101, and YouTube Action, respectively. Our results show that the proposed strategy outperforms State-of-the-Art (SOTA) methods. The proposed framework gives superior performance in HAR, providing accurate and reliable recognition of human activities in surveillance systems. [ABSTRACT FROM AUTHOR]

Due to its low cost and privacy protection, Channel-State-Information (CSI)-based activity detection has gained interest recently. However, to achieve high accuracy, which is challenging in practice, a significant number of training samples are required. To address the issues of the small sample size and cross-scenario in neural network training, this paper proposes a WiFi human activity-recognition system based on transfer learning using an antenna array: Wi-AR. First, the Intel5300 network card collects CSI signal measurements through an antenna array and processes them with a low-pass filter to reduce noise. Then, a threshold-based sliding window method is applied to extract the signal of independent activities, which is further transformed into time–frequency diagrams. Finally, the produced diagrams are used as input to a pretrained ResNet18 to recognize human activities. The proposed Wi-AR was evaluated using a dataset collected in three different room layouts. The testing results showed that the suggested Wi-AR recognizes human activities with a consistent accuracy of about 94%, outperforming the other conventional convolutional neural network approach. [ABSTRACT FROM AUTHOR]

Recognizing the activities being performed on a map is crucial for adaptive map design based on user context. Despite eye tracking (ET) demonstrating potential in recognizing map activities and electroencephalography (EEG) measuring map users' cognitive load, no studies have yet combined ET and EEG for recognition of the user's activity on maps. Our study collected participants' ET and EEG data during four types of map activities. After feature extraction and selection, we trained LightGBM (light Gradient-Boosting Machine) to classify these activities, and achieved 88.0% accuracy when combining ET and EEG features in the entire map usage trial, which is higher than using ET (85.9%) or EEG (53.9%) alone. Acceptable recognition accuracy could also be achieved with the early time windows (73.1% when using the first 3 seconds). Saccade features of ET were the most important for differentiating map activities, indicating selective map content for different tasks. Our findings demonstrate the feasibility and advantages of combining ET and EEG for activity recognition in map use. The results not only improve our understanding of visual patterns and cognitive processes in map use, but also enable the design of adaptive maps that can automatically adapt to the activities a map user is performing. [ABSTRACT FROM AUTHOR]

Mobile fitness applications provide the opportunity to show users real-time feedback on their current fitness activity. For such applications, it is essential to accurately track the user's current fitness activity using available mobile sensors, such as inertial measurement units (IMUs). Convolutional neural networks (CNNs) have been shown to produce strong results in different time series classification tasks, including the recognition of daily living activities. However, fitness activities can present unique challenges to the human activity recognition task (HAR), including greater similarity between individual activities and fewer available data for model training. In this paper, we evaluate the applicability of CNNs to the fitness activity recognition task (FAR) using IMU data and determine the impact of input data size and sensor count on performance. For this purpose, we adapted three existing CNN architectures to the FAR task and designed a fourth CNN variant, which we call the scaling fully convolutional network (Scaling-FCN). We designed a preprocessing pipeline and recorded a running exercise data set with 20 participants, in which we evaluated the respective recognition performances of the four networks, comparing them with three traditional machine learning (ML) methods commonly used in HAR. Although CNN architectures achieve at least 94% test accuracy in all scenarios, two traditional ML architectures surpass them in the default scenario, with support vector machines (SVMs) achieving 99.00 ± 0.34% test accuracy. The removal of all sensors except one foot sensor reduced the performance of traditional ML architectures but improved the performance of CNN architectures on our data set, with our Scaling-FCN reaching the highest accuracy of 99.86 ± 0.11% on the test set. Our results suggest that CNNs are generally well suited for fitness activity recognition, and noticeable performance improvements can be achieved if sensors are dropped selectively, although traditional ML architectures can still compete with or even surpass CNNs when favorable input data are utilized. [ABSTRACT FROM AUTHOR]

Participatory exposure research, which tracks behaviour and assesses exposure to stressors like air pollution, traditionally relies on time-activity diaries. This study introduces a novel approach, employing machine learning (ML) to empower laypersons in human activity recognition (HAR), aiming to reduce dependence on manual recording by leveraging data from wearable sensors. Recognising complex activities such as smoking and cooking presents unique challenges due to specific environmental conditions. In this research, we combined wearable environment/ambient and wrist-worn activity/biometric sensors for complex activity recognition in an urban stressor exposure study, measuring parameters like particulate matter concentrations, temperature, and humidity. Two groups, Group H (88 individuals) and Group M (18 individuals), wore the devices and manually logged their activities hourly and minutely, respectively. Prioritising accessibility and inclusivity, we selected three classification algorithms: k-nearest neighbours (IBk), decision trees (J48), and random forests (RF), based on: (1) proven efficacy in existing literature, (2) understandability and transparency for laypersons, (3) availability on user-friendly platforms like WEKA, and (4) efficiency on basic devices such as office laptops or smartphones. Accuracy improved with finer temporal resolution and detailed activity categories. However, when compared to other published human activity recognition research, our accuracy rates, particularly for less complex activities, were not as competitive. Misclassifications were higher for vague activities (resting, playing), while well-defined activities (smoking, cooking, running) had few errors. Including environmental sensor data increased accuracy for all activities, especially playing, smoking, and running. Future work should consider exploring other explainable algorithms available on diverse tools and platforms. Our findings underscore ML's potential in exposure studies, emphasising its adaptability and significance for laypersons while also highlighting areas for improvement. [ABSTRACT FROM AUTHOR]

A social issue that leads to the safety and security of people is the network that monitors the safety of people living alone in remote areas. Various research and developments of indoor human activity recognition using IoT sensor nodes have been conducted in recent years. In this research, we focus on turning on and off indoor lighting and propose an activity recognition system at home using an illuminance sensor. An IoT sensor node with a built‐in illuminance sensor is attached to the wall near the indoor lighting equipment. The living activities of people in the room are estimated from changes in the illuminance. We assume a significant change in the illuminance of the indoor lighting is observed at least once a day at the sampling interval of the IoT sensor node. In that case, it is possible to estimate the operation of the indoor lighting due to daily activities and to confirm the safety of people living alone in remote areas. The activity recognition system was evaluated for a total of 380 days, excluding the missing period. The estimation of indoor lighting manipulation by daily activities was accurate for 376 days. Precision, Recall, and F‐measure score values, which are evaluation indices for activity estimation, were 94.9%, 98.2%, and 96.5%, respectively. [ABSTRACT FROM AUTHOR]

The past decade has seen an increased interest in human activity recognition based on sensor data. Most often, the sensor data come unannotated, creating the need for fast labelling methods. For assessing the quality of the labelling, an appropriate performance measure has to be chosen. Our main contribution is a novel post-processing method for activity recognition. It improves the accuracy of the classification methods by correcting for unrealistic short activities in the estimate. We also propose a new performance measure, the Locally Time-Shifted Measure (LTS measure), which addresses uncertainty in the times of state changes. The effectiveness of the post-processing method is evaluated, using the novel LTS measure, on the basis of a simulated dataset and a real application on sensor data from football. The simulation study is also used to discuss the choice of the parameters of the post-processing method and the LTS measure. [ABSTRACT FROM AUTHOR]

By observing the actions taken by operators, it is possible to determine the risk level of a work task. One method for achieving this is the recognition of human activity using biosignals and inertial measurements provided to a machine learning algorithm performing such recognition. The aim of this research is to propose a method to automatically recognize physical exertion and reduce noise as much as possible towards the automation of the Job Strain Index (JSI) assessment by using a motion capture wearable device (MindRove armband) and training a quadratic support vector machine (QSVM) model, which is responsible for predicting the exertion depending on the patterns identified. The highest accuracy of the QSVM model was 95.7%, which was achieved by filtering the data, removing outliers and offsets, and performing zero calibration; in addition, EMG signals were normalized. It was determined that, given the job strain index's purpose, physical exertion detection is crucial to computing its intensity in future work. [ABSTRACT FROM AUTHOR]

Human activity recognition is an application of machine learning with the aim of identifying activities from the gathered activity raw data acquired by different sensors. In medicine, human gait is commonly analyzed by doctors to detect abnormalities and determine possible treatments for the patient. Monitoring the patient's activity is paramount in evaluating the treatment's evolution. This type of classification is still not enough precise, which may lead to unfavorable reactions and responses. A novel methodology that reduces the complexity of extracting features from multimodal sensors is proposed to improve human activity classification based on accelerometer data. A sliding window technique is used to demarcate the first dominant spectral amplitude, decreasing dimensionality and improving feature extraction. In this work, we compared several state-of-art machine learning classifiers evaluated on the HuGaDB dataset and validated on our dataset. Several configurations to reduce features and training time were analyzed using multimodal sensors: all-axis spectrum, single-axis spectrum, and sensor reduction. [ABSTRACT FROM AUTHOR]

In recent years, human activity monitoring and recognition have gained importance in providing valuable information to improve the quality of life. A lack of activity can cause health problems including falling, depression, and decreased mobility. Continuous activity monitoring can be useful to prevent progressive health problems. With this purpose, this study presents a wireless smart insole with four force-sensitive resistors (FSRs) that monitor foot contact states during activities for both indoor and outdoor use. The designed insole is a compact solution and provides walking comfort with a slim and flexible structure. Moreover, the inertial measurement unit (IMU) sensors designed in our previous study were used to collect 3-axis accelerometer and 3-axis gyroscope outputs. Smart insoles were located in the shoe sole for both right and left feet, and two IMU sensors were attached to the thigh area of each leg. The sensor outputs were collected and recorded from forty healthy volunteers for eight different gait-based activities including walking uphill and descending stairs. The obtained datasets were separated into three categories; foot contact states, the combination of acceleration and gyroscope outputs, and a set of all sensor outputs. The dataset for each category was separately fed into deep learning algorithms, namely, convolutional long–short-term memory neural networks. The performance of each neural network for each category type was examined. The results show that the neural network using only foot contact states presents 90.1% accuracy and provides better performance than the combination of acceleration and gyroscope datasets for activity recognition. Moreover, the neural network presents the best results with 93.4% accuracy using a combination of all the data compared with the other two categories. [ABSTRACT FROM AUTHOR]

Electroencephalography (EEG) is a non-invasive method employed to discern human behaviors by monitoring the neurological responses during cognitive and motor tasks. Machine learning (ML) represents a promising tool for the recognition of human activities (HAR), and eXplainable artificial intelligence (XAI) can elucidate the role of EEG features in ML-based HAR models. The primary objective of this investigation is to investigate the feasibility of an EEG-based ML model for categorizing everyday activities, such as resting, motor, and cognitive tasks, and interpreting models clinically through XAI techniques to explicate the EEG features that contribute the most to different HAR states. The study involved an examination of 75 healthy individuals with no prior diagnosis of neurological disorders. EEG recordings were obtained during the resting state, as well as two motor control states (walking and working tasks), and a cognition state (reading task). Electrodes were placed in specific regions of the brain, including the frontal, central, temporal, and occipital lobes (Fz, C1, C2, T7, T8, Oz). Several ML models were trained using EEG data for activity recognition and LIME (Local Interpretable Model-Agnostic Explanations) was employed for interpreting clinically the most influential EEG spectral features in HAR models. The classification results of the HAR models, particularly the Random Forest and Gradient Boosting models, demonstrated outstanding performances in distinguishing the analyzed human activities. The ML models exhibited alignment with EEG spectral bands in the recognition of human activity, a finding supported by the XAI explanations. To sum up, incorporating eXplainable Artificial Intelligence (XAI) into Human Activity Recognition (HAR) studies may improve activity monitoring for patient recovery, motor imagery, the healthcare metaverse, and clinical virtual reality settings. [ABSTRACT FROM AUTHOR]

Ubiquitous computing has been a green research area that has managed to attract and sustain the attention of researchers for some time now. As ubiquitous computing applications, human activity recognition and localization have also been popularly worked on. These applications are used in healthcare monitoring, behavior analysis, personal safety, and entertainment. A robust model has been proposed in this article that works over IoT data extracted from smartphone and smartwatch sensors to recognize the activities performed by the user and, in the meantime, classify the location at which the human performed that particular activity. The system starts by denoising the input signal using a second-order Butterworth filter and then uses a hamming window to divide the signal into small data chunks. Multiple stacked windows are generated using three windows per stack, which, in turn, prove helpful in producing more reliable features. The stacked data are then transferred to two parallel feature extraction blocks, i.e., human activity recognition and human localization. The respective features are extracted for both modules that reinforce the system's accuracy. A recursive feature elimination is applied to the features of both categories independently to select the most informative ones among them. After the feature selection, a genetic algorithm is used to generate ten different generations of each feature vector for data augmentation purposes, which directly impacts the system's performance. Finally, a deep neural decision forest is trained for classifying the activity and the subject's location while working on both of these attributes in parallel. For the evaluation and testing of the proposed system, two openly accessible benchmark datasets, the ExtraSensory dataset and the Sussex-Huawei Locomotion dataset, were used. The system outperformed the available state-of-the-art systems by recognizing human activities with an accuracy of 88.25% and classifying the location with an accuracy of 90.63% over the ExtraSensory dataset, while, for the Sussex-Huawei Locomotion dataset, the respective results were 96.00% and 90.50% accurate. [ABSTRACT FROM AUTHOR]

Recognition of human activities is part of smart healthcare applications. In this context, the detection of human activities is an area that has been studied for many years. In these studies, various motion sensors placed in the body are utilized. With the increase in the usage of smart devices, smartphones, and smartwatches have become the constant equipment of these studies thanks to their internal sensors. Sometimes abnormal data are included in data sets due to the way the data were collected and for reasons arising from the sensors. For this reason, it becomes important to detect outlier data. In this study, step counter and heart rate sensors were used in addition to an accelerometer and gyroscope in order to detect human activities. Afterward, the outliers were detected and cleared with a Mahalanobis distance-based approach. With the aim of achieving a better classification performance, machine learning methods were used by strengthening them with ensemble learning methods. The obtained results showed that step counter, heart rate sensors, and ensemble learning methods positively affect the success of the classification. In addition, it was found that the Mahalanobis distance-based outlier detection method increased the classification accuracy significantly. [ABSTRACT FROM AUTHOR]

Sensor-based human activity recognition aims to recognize the activities performed by people with the sensor readings. Most of existing works in this area rely on supervised classification algorithms, and can only recognize activities covered by the training data. Whereas, in many practical applications, while performing activity recognition, not only the activities covered by the training data, but also some previously unseen activities need to be recognized. In this paper, we study the problem of generalized zero-shot activity recognition. In this problem, the activities that need to be recognized contain both the activities covered by the training data and the previously unseen activities. We firstly give a formulation of this problem, and then propose an embedding-based method to address it. In this method, an embedding-compatibility model is learned. When performing activity recognition, the learned model and the calibrated stacking mechanism are employed. Extensive experiments on publicly available datasets demonstrate the effectiveness of our method. [ABSTRACT FROM AUTHOR]

The increasing use of Wearable devices opens up the use of a wide range of applications. Using different models, these devices can be of great use in Human Activity Recognition (HAR), where the main goal is to process information obtained from sensors located in them, especially in eHealth. The high volume of data collected by various smart devices in contemporary ML scenarios, leads to higher processing consumption and in many cases results in compromised privacy. These shortcomings could be overcome by using Federated Learning (FL), a learning paradigm that allows for decentralized training of models such that user’s personal data does not need to ever leave their devices, which substantially reduces to possibility of a breach. This paper analyses the behaviour and performances of FL when applied to the context of HAR. The obtained results show that FL can achieve comparable performances to those of centralized Deep Learning, while facilitating improved data privacy and diversity, as well as fostering real-time continuous learning.

Activity recognition using data collected with smart devices such as mobile and wearable sensors has become a critical component of many emerging applications ranging from behavioral medicine to gaming. However, an unprecedented increase in the diversity of smart devices in the internet-of-things era has limited the adoption of activity recognition models for use across different devices. This lack of cross-domain adaptation is particularly notable across sensors of different modalities where the mapping of the sensor data in the traditional feature level is highly challenging. To address this challenge, we propose ActiLabel, a combinatorial framework that learns structural similarities among the events that occur in a target domain and those of a source domain and identifies an optimal mapping between the two domains at their structural level. The structural similarities are captured through a graph model, referred to as the dependency graph, which abstracts details of activity patterns in low-level signal and feature space. The activity labels are then autonomously learned in the target domain by finding an optimal tiered mapping between the dependency graphs. We carry out an extensive set of experiments on three large datasets collected with wearable sensors involving human subjects. The results demonstrate the superiority of ActiLabel over state-of-the-art transfer learning and deep learning methods. In particular, ActiLabel outperforms such algorithms by average F1-scores of 36.3 % , 32.7 % , and 9.1 % for cross-modality, cross-location, and cross-subject activity recognition, respectively. [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) is essential for the development of robots to assist humans in daily activities. HAR is required to be accurate, fast and suitable for low-cost wearable devices to ensure portable and safe assistance. Current computational methods can achieve accurate recognition results but tend to be computationally expensive, making them unsuitable for the development of wearable robots in terms of speed and processing power. This paper proposes a light-weight architecture for recognition of activities using five inertial measurement units and four goniometers attached to the lower limb. First, a systematic extraction of time-domain features from wearable sensor data is performed. Second, a small high-speed artificial neural network and line search method for cost function optimization are used for activity recognition. The proposed method is systematically validated using a large dataset composed of wearable sensor data from seven activities (sitting, standing, walking, stair ascent/descent, ramp ascent/descent) associated with eight healthy subjects. The accuracy and speed results are compared against methods commonly used for activity recognition including deep neural networks, convolutional neural networks, long short-term memory and convolutional–long short-term memory hybrid networks. The experiments demonstrate that the light-weight architecture can achieve a high recognition accuracy of 98.60%, 93.10% and 84.77% for seen data from seen subjects, unseen data from seen subjects and unseen data from unseen subjects, respectively, and an inference time of 85 μ s. The results show that the proposed approach can perform accurate and fast activity recognition with a reduced computational complexity suitable for the development of portable assistive devices. [ABSTRACT FROM AUTHOR]

RF sensing offers an unobtrusive, user-friendly, and privacy-preserving method for detecting accidental falls and recognizing human activities. Contemporary RF-based HAR systems generally employ a single monostatic radar to recognize human activities. However, a single monostatic radar cannot detect the motion of a target, e.g., a moving person, orthogonal to the boresight axis of the radar. Owing to this inherent physical limitation, a single monostatic radar fails to efficiently recognize orientation-independent human activities. In this work, we present a complementary RF sensing approach that overcomes the limitation of existing single monostatic radar-based HAR systems to robustly recognize orientation-independent human activities and falls. Our approach used a distributed mmWave MIMO radar system that was set up as two separate monostatic radars placed orthogonal to each other in an indoor environment. These two radars illuminated the moving person from two different aspect angles and consequently produced two time-variant micro-Doppler signatures. We first computed the mean Doppler shifts (MDSs) from the micro-Doppler signatures and then extracted statistical and time- and frequency-domain features. We adopted feature-level fusion techniques to fuse the extracted features and a support vector machine to classify orientation-independent human activities. To evaluate our approach, we used an orientation-independent human activity dataset, which was collected from six volunteers. The dataset consisted of more than 1350 activity trials of five different activities that were performed in different orientations. The proposed complementary RF sensing approach achieved an overall classification accuracy ranging from 98.31 to 98.54%. It overcame the inherent limitations of a conventional single monostatic radar-based HAR and outperformed it by 6%. [ABSTRACT FROM AUTHOR]

Human-Computer Interaction (HCI) is a sub-area within computer science focused on the study of the communication between people (users) and computers and the evaluation, implementation, and design of user interfaces for computer systems. HCI has accomplished effective incorporation of the human factors and software engineering of computing systems through the methods and concepts of cognitive science. Usability is an aspect of HCI dedicated to guaranteeing that human-computer communication is, amongst other things, efficient, effective, and sustaining for the user. Simultaneously, Human activity recognition (HAR) aim is to identify actions from a sequence of observations on the activities of subjects and the environmental conditions. The vision-based HAR study is the basis of several applications involving health care, HCI, and video surveillance. This article develops a Fire Hawk Optimizer with Deep Learning Enabled Activity Recognition (FHODL-AR) on HCI driven usability. In the presented FHODLAR technique, the input images are investigated for the identification of different human activities. For feature extraction, a modified SqueezeNet model is introduced by the inclusion of few bypass connections to the SqueezeNet among Fire modules. Besides, the FHO algorithm is utilized as a hyperparameter optimization algorithm, which in turn boosts the classification performance. To detect and categorize different kinds of activities, probabilistic neural network (PNN) classifier is applied. The experimental validation of the FHODL-AR technique is tested using benchmark datasets, and the outcomes reported the improvements of the FHODLAR technique over other recent approaches. [ABSTRACT FROM AUTHOR]

Vision-based human activity recognition (HAR) has emerged as one of the essential research areas in video analytics. Over the last decade, numerous advanced deep learning algorithms have been introduced to recognize complex human actions from video streams. These deep learning algorithms have shown impressive performance for the video analytics task. However, these newly introduced methods either exclusively focus on model performance or the effectiveness of these models in terms of computational efficiency, resulting in a biased trade-off between robustness and computational efficiency in their proposed methods to deal with challenging HAR problem. To enhance both the accuracy and computational efficiency, this paper presents a computationally efficient yet generic spatial–temporal cascaded framework that exploits the deep discriminative spatial and temporal features for HAR. For efficient representation of human actions, we propose an efficient dual attentional convolutional neural network (DA-CNN) architecture that leverages a unified channel–spatial attention mechanism to extract human-centric salient features in video frames. The dual channel–spatial attention layers together with the convolutional layers learn to be more selective in the spatial receptive fields having objects within the feature maps. The extracted discriminative salient features are then forwarded to a stacked bi-directional gated recurrent unit (Bi-GRU) for long-term temporal modeling and recognition of human actions using both forward and backward pass gradient learning. Extensive experiments are conducted on three publicly available human action datasets, where the obtained results verify the effectiveness of our proposed framework (DA-CNN+Bi-GRU) over the state-of-the-art methods in terms of model accuracy and inference runtime across each dataset. Experimental results show that the DA-CNN+Bi-GRU framework attains an improvement in execution time up to 167× in terms of frames per second as compared to most of the contemporary action-recognition methods. [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) has attracted widespread attention in areas such as human–computer interaction, work performance management, and healthcare. Owing to advantages such as continuous monitoring, reduced cost of deployment, and ease of privacy protection, wearable-sensor-based HAR is preferred over the traditional approach of using external sensors. In this study, the influence of different combinations of seven body-worn accelerometer positions on the classification of 23 complex daily activities was examined. A conventional machine learning model, namely, RandomForest (RF), and two deep-learning (DL) models, convolutional neural network (CNN)-long short-time memory (LSTM) and CNN transformer, were used to understand the impact of using different models on the classification performance. The results showed a strong correlation between the classification models regarding the combinations of sensor positions and classification performance (F1-score). Additionally, the combination of the four sensors from the left and right wrists, right upper arm, and right thigh was determined to be the best. This study also showed that, owing to feature calculation, the RF model took a longer processing time than the DL-based models and that the CNN-LSTM model would be preferable to RF if plenty of data were available for training it. The results can provide a reference for application designers in choosing appropriate combinations of sensor positions based on requirements for wearability and classification performance. [ABSTRACT FROM AUTHOR]

Health monitoring, rehabilitation, and fitness are just a few domains where human activity recognition can be applied. In this study, a deep learning approach has been proposed to recognise ambulation and fitness activities from data collected by five participants using smart insoles. Smart insoles, consisting of pressure and inertial sensors, allowed for seamless data collection while minimising user discomfort, laying the baseline for the development of a monitoring and/or rehabilitation system for everyday life. The key objective has been to enhance the deep learning model performance through several techniques, including data segmentation with overlapping technique (2 s with 50% overlap), signal down-sampling by averaging contiguous samples, and a cost-sensitive re-weighting strategy for the loss function for handling the imbalanced dataset. The proposed solution achieved an Accuracy and F1-Score of 98.56% and 98.57%, respectively. The Sitting activities obtained the highest degree of recognition, closely followed by the Spinning Bike class, but fitness activities were recognised at a higher rate than ambulation activities. A comparative analysis was carried out both to determine the impact that pre-processing had on the proposed core architecture and to compare the proposed solution with existing state-of-the-art solutions. The results, in addition to demonstrating how deep learning solutions outperformed those of shallow machine learning, showed that in our solution the use of data pre-processing increased performance by about 2%, optimising the handling of the imbalanced dataset and allowing a relatively simple network to outperform more complex networks, reducing the computational impact required for such applications. [ABSTRACT FROM AUTHOR]

This paper proposes a novel approach to activity recognition where videos are compressed using video coding to generate feature vectors based on compression variables. We propose to eliminate the temporal domain of feature vectors by computing the mean and standard deviation of each variable across all video frames. Thus, each video is represented by a single feature vector of 67 variables. As for the motion vectors, we eliminated their temporal domain by projecting their phases using PCA, thus representing each video by a single feature vector with a length equal to the number of frames in a video. Consequently, complex classifiers such as LSTM can be avoided and classical machine learning techniques can be used instead. Experimental results on the JHMDB dataset resulted in average classification accuracies of 68.8% and 74.2% when using the projected phases of motion vectors and video coding feature variables, respectively. The advantage of the proposed solution is the use of FVs with low dimensionality and simple machine learning techniques. [ABSTRACT FROM AUTHOR]

Human activity recognition based on generated sensor data plays a major role in a large number of applications such as healthcare monitoring and surveillance system. Yet, accurately recognizing human activities is still challenging and active research due to people's tendency to perform daily activities in a different and multitasking way. Existing approaches based on the recurrent setting for human activity recognition have some issues, such as the inability to process data parallelly, the requirement for more memory and high computational cost albeit they achieved reasonable results. Convolutional Neural Network processes data parallelly, but, it breaks the ordering of input data, which is significant to build an effective model for human activity recognition. To overcome these challenges, this study proposes causal convolution based on performers-attention and supervised contrastive learning to entirely forego recurrent architectures, efficiently maintain the ordering of human daily activities and focus more on important timesteps of the sensors' data. Supervised contrastive learning is integrated to learn a discriminative representation of human activities and enhance predictive performance. The proposed network is extensively evaluated for human activities using multiple datasets including wearable sensor data and smart home environments data. The experiments on three wearable sensor datasets and five smart home public datasets of human activities reveal that our proposed network achieves better results and reduces the training time compared with the existing state-of-the-art methods and basic temporal models. [ABSTRACT FROM AUTHOR]

With the development of deep learning, numerous models have been proposed for human activity recognition to achieve state‐of‐the‐art recognition on wearable sensor data. Despite the improved accuracy achieved by previous deep learning models, activity recognition remains a challenge. This challenge is often attributed to the complexity of some specific activity patterns. Existing deep learning models proposed to address this have often recorded high overall recognition accuracy, while low recall and precision are often recorded on some individual activities due to the complexity of their patterns. Some existing models that have focused on tackling these issues are always bulky and complex. Since most embedded systems have resource constraints in terms of their processor, memory and battery capacity, it is paramount to propose efficient lightweight activity recognition models that require limited resources consumption, and still capable of achieving state‐of‐the‐art recognition of activities, with high individual recall and precision. This research proposes a high performance, low footprint deep learning model with a squeeze and excitation block to address this challenge. The squeeze and excitation block consist of a global average‐pooling layer and two fully connected layers, which were placed to extract the flattened features in the model, with best‐fit reduction ratios in the squeeze and excitation block. The squeeze and excitation block served as channel‐wise attention, which adjusted the weight of each channel to build more robust representations, which enabled our network to become more responsive to essential features while suppressing less important ones. By using the best‐fit reduction ratio in the squeeze and excitation block, the parameters of the fully connected layer were reduced, which helped the model increase responsiveness to essential features. Experiments on three publicly available datasets (PAMAP2, WISDM, and UCI‐HAR) showed that the proposed model outperformed existing state‐of‐the‐art with fewer parameters and increased the recall and precision of some individual activities compared to the baseline, and the existing models. [ABSTRACT FROM AUTHOR]

As in manufacturing with its Industry 4.0 transformation, the enormous potential of artificial intelligence (AI) is also being recognized in the construction industry. Specifically, the equipment-intensive construction industry can benefit from using AI. AI applications can leverage the data recorded by the numerous sensors on machines and mirror them in a digital twin. Analyzing the digital twin can help optimize processes on the construction site and increase productivity. We present a case from special foundation engineering: the machine production of bored piles. We introduce a hierarchical classification for activity recognition and apply a hybrid deep learning model based on convolutional and recurrent neural networks. Then, based on the results from the activity detection, we use discrete-event simulation to predict construction progress. We highlight the difficulty of defining the appropriate modeling granularity. While activity detection requires equipment movement, simulation requires knowledge of the production flow. Therefore, we present a flow-based production model that can be captured in a modularized process catalog. Overall, this paper aims to illustrate modeling using digital-twin technologies to increase construction process improvement in practice. [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) has the potential to significantly impact applications such as health monitoring, context-aware systems, transportation, robotics, and smart cities. Because of the prevalence of wireless devices, the Wi-Fi-based approach has attracted a lot of attention among other existing methods such as sensor-based and vision-based HAR. Wi-Fi devices can be used to distinguish between daily activities such as "walking," "running," and "sleeping," which affect Wi-Fi signal propagation. This paper proposes a Deep Learning method for HAR tasks that makes use of channel state information (CSI). We convert the CSI data to RGB images and classify the activity recognition using a 2D-Convolutional Neural Network (CNN). We evaluate the performance of the proposed method on two publicly available datasets for CSI data. Our experiments show that converting data into RGB images improves performance and accuracy compared to our previous method by at least 5%. [ABSTRACT FROM AUTHOR]

Remote health monitoring plays a significant role in research areas related to medicine, neurology, rehabilitation, and robotic systems. These applications include Human Activity Recognition (HAR) using wearable sensors, signal processing, mathematical methods, and machine learning to improve the accuracy of remote health monitoring systems. To improve the detection and accuracy of human activity recognition, we create a novel method to reduce the complexities of extracting features using the HuGaDB dataset. Our model extracts power spectra; due to the high dimensionality of features, sliding windows techniques are used to determine frequency bandwidth automatically, where an improved QRS algorithm selects the first dominant spectrum amplitude. In addition, the bandwidth algorithm has been used to reduce the dimensionality of data, remove redundant dimensions, and improve feature extraction. In this work, we have considered widely used machine learning classifiers. Our proposed method was evaluated using the accelerometer angles spectrum installed in six parts of the body and then reducing the bandwidth to know the evolution. Our approach attains an accuracy rate of 95.1% in the HuGaDB dataset with 70% of bandwidth, outperforming others in the human activity recognition accuracy. [ABSTRACT FROM AUTHOR]

Activity recognition in unmanned aerial vehicle (UAV) surveillance is addressed in various computer vision applications such as image retrieval, pose estimation, object detection, object detection in videos, object detection in still images, object detection in video frames, face recognition, and video action recognition. In the UAV-based surveillance technology, video segments captured from aerial vehicles make it challenging to recognize and distinguish human behavior. In this research, to recognize a single and multi-human activity using aerial data, a hybrid model of histogram of oriented gradient (HOG), mask-regional convolutional neural network (Mask-RCNN), and bidirectional long short-term memory (Bi-LSTM) is employed. The HOG algorithm extracts patterns, Mask-RCNN extracts feature maps from the raw aerial image data, and the Bi-LSTM network exploits the temporal relationship between the frames for the underlying action in the scene. This Bi-LSTM network reduces the error rate to the greatest extent due to its bidirectional process. This novel architecture generates enhanced segmentation by utilizing the histogram gradient-based instance segmentation and improves the accuracy of classifying human activities using the Bi-LSTM approach. Experimental outcomes demonstrate that the proposed model outperforms the other state-of-the-art models and has achieved 99.25% accuracy on the YouTube-Aerial dataset. [ABSTRACT FROM AUTHOR]

Laying hen activities in modern intensive housing systems can dramatically influence the policies needed for the optimal management of such systems. Intermittent monitoring of different behaviors during daytime cannot provide a good overview, since daily behaviors are not equally distributed over the day. This paper investigates the application of deep learning technology in the automatic recognition of laying hen behaviors equipped with body-worn inertial measurement unit (IMU) modules in poultry systems. Motivated by the human activity recognition literature, a sophisticated preprocessing method is tailored on the time-series data of IMU, transforming it into the form of so-called activity images to be recognized by the deep learning models. The diverse range of behaviors a laying hen can exhibit are categorized into three classes: low-, medium-, and high-intensity activities, and various recognition models are trained to recognize these behaviors in real-time. Several ablation studies are conducted to assess the efficacy and robustness of the developed models against variations and limitations common for an in situ practical implementation. Overall, the best trained model on the full-feature acquired data achieves a mean accuracy of almost 100%, where the whole process of inference by the model takes less than 30 milliseconds. The results suggest that the application of deep learning technology for activity recognition of individual hens has the potential to accurately aid successful management of modern poultry systems. [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) is a complex and multifaceted problem. The research community has reported numerous approaches to perform HAR. Along with HAR approaches, various surveys have revealed HAR trends in various environments and applications. HAR is linked to a variety of technology-dependent daily life systems, such as human–computer interaction systems, security surveillance, video surveillance, healthcare surveillance, robotics, content-based information retrieval, and monitoring systems. Because of technological advancements, HAR trends change quickly and necessitate an up-to-date and broader perspective. This study offers an HAR taxonomy, which includes online/offline HAR, multimodal/unimodal HAR, handcrafted feature-based, and learning-based approaches. This study attempts to present the multidisciplinary nature of HAR, such as application areas, activity types, task complexities, benchmark datasets, and/methods. This research includes a comparative analysis of state-of-the-art HAR methods and a discussion of popular datasets. The selected studies have been categorized using taxonomy, and different attributes such as activity complexity, dataset size, and recognition rate have been used for their analysis. The comparative analysis of HAR approaches has also helped to highlight domain challenges and open research directions for HAR researchers to follow. [ABSTRACT FROM AUTHOR]

The electromyogram (EMG) is a waveform representation of the action potential generated by muscle cells using electrodes. EMG acquired using surface electrodes is called surface EMG (sEMG), and it is the acquisition of muscle action potentials transmitted by volume conduction from the skin. Surface electrodes require disposable conductive gel or adhesive tape to be attached to the skin, which is costly to run, and the tape is hard on the skin when it is removed. Muscle activity can be evaluated by acquiring muscle potentials and analyzing quantitative, temporal, and frequency factors. It is also possible to evaluate muscle fatigue because the frequency of the EMG becomes lower as the muscle becomes fatigued. Research on human activity recognition from EMG signals has been actively conducted and applied to systems that support arm and hand functions. This paper proposes a method for recognizing the muscle activity state of the arm using pulse wave data (PPG: Photoplethysmography) and a method for estimating EMG using pulse wave data. This paper assumes that the PPG sensor is worn on the user's wrist to measure the heart rate. The user also attaches an elastic band to the upper arm, and when the user exerts a force on the arm, the muscles of the upper arm contract. The arteries are then constricted, and the pulse wave measured at the wrist becomes weak. From the change in the pulse wave, the muscle activity of the arm can be recognized and the number of action potentials of the muscle can be estimated. From the evaluation experiment with five subjects, three types of muscle activity were recognized with 80+%, and EMG was estimated with approximately 20% error rate. [ABSTRACT FROM AUTHOR]

Recently, the scientific community has placed great emphasis on the recognition of human activity, especially in the area of health and care for the elderly. There are already practical applications of activity recognition and unusual conditions that use body sensors such as wrist-worn devices or neck pendants. These relatively simple devices may be prone to errors, might be uncomfortable to wear, might be forgotten or not worn, and are unable to detect more subtle conditions such as incorrect postures. Therefore, other proposed methods are based on the use of images and videos to carry out human activity recognition, even in open spaces and with multiple people. However, the resulting increase in the size and complexity involved when using image data requires the use of the most recent advanced machine learning and deep learning techniques. This paper presents an approach based on deep learning with attention to the recognition of activities from multiple frames. Feature extraction is performed by estimating the pose of the human skeleton, and classification is performed using a neural network based on Bidirectional Encoder Representation of Transformers (BERT). This algorithm was trained with the UP-Fall public dataset, generating more balanced artificial data with a Generative Adversarial Neural network (GAN), and evaluated with real data, outperforming the results of other activity recognition methods using the same dataset. [ABSTRACT FROM AUTHOR]

Most industrial workplaces involving robots and other apparatus operate behind the fences to remove defects, hazards, or casualties. Recent advancements in machine learning can enable robots to co-operate with human co-workers while retaining safety, flexibility, and robustness. This article focuses on the computation model, which provides a collaborative environment through intuitive and adaptive human–robot interaction (HRI). In essence, one layer of the model can be expressed as a set of useful information utilized by an intelligent agent. Within this construction, a vision-sensing modality can be broken down into multiple layers. The authors propose a human-skeleton-based trainable model for the recognition of spatiotemporal human worker activity using LSTM networks, which can achieve a training accuracy of 91.365%, based on the InHARD dataset. Together with the training results, results related to aspects of the simulation environment and future improvements of the system are discussed. By combining human worker upper body positions with actions, the perceptual potential of the system is increased, and human–robot collaboration becomes context-aware. Based on the acquired information, the intelligent agent gains the ability to adapt its behavior according to its dynamic and stochastic surroundings. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (CNNs) achieve state-of-the-art performance in wearable human activity recognition (HAR), which has become a new research trend in ubiquitous computing scenario. Increasing network depth or width can further improve accuracy. However, in order to obtain the optimal HAR performance on mobile platform, it has to consider a reasonable tradeoff between recognition accuracy and resource consumption. Improving the performance of CNNs without increasing memory and computational burden is more beneficial for HAR. In this article, we first propose a new CNN that uses hierarchical-split (HS) idea for a large variety of HAR tasks, which is able to enhance multiscale feature representation ability via capturing a wider range of receptive fields of human activities within one feature layer. Experiments conducted on benchmarks demonstrate that the proposed HS module is an impressive alternative to baseline models with similar model complexity, and can achieve higher recognition performance (e.g., 97.28%, 93.75%, 99.02%, and 79.02% classification accuracies) on UCI-HAR, PAMAP2, WISDM, and UNIMIB-SHAR. Extensive ablation studies are performed to evaluate the effect of the variations of receptive fields on classification performance. Finally, we demonstrate that multiscale receptive fields can help to learn more discriminative features (achieving 94.10% SOTA accuracy) in weakly labeled HAR dataset. [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) extensively uses wearable inertial sensors since this data source provides the most information for non-visual datasets' time series. HAR research has advanced significantly in recent years due to the proliferation of wearable devices with sensors. To improve recognition performance, HAR researchers have extensively investigated other sources of biosignals, such as a photoplethysmograph (PPG), for this task. PPG sensors measure the rate at which blood flows through the body, and this rate is regulated by the heart's pumping action, which constantly occurs throughout the body. Even though detecting body movement and gestures was not initially the primary purpose of PPG signals, we propose an innovative method for extracting relevant features from the PPG signal and use deep learning (DL) to predict physical activities. To accomplish the purpose of our study, we developed a deep residual network referred to as PPG-NeXt, designed based on convolutional operation, shortcut connections, and aggregated multi-branch transformation to efficiently identify different types of daily life activities from the raw PPG signal. The proposed model achieved more than 90% prediction F1-score from experimental results using only PPG data on the three benchmark datasets. Moreover, our results indicate that combining PPG and acceleration signals can enhance activity recognition. Although, both biosignals—electrocardiography (ECG) and PPG—can differentiate between stationary activities (such as sitting) and non-stationary activities (such as cycling and walking) with a level of success that is considered sufficient. Overall, our results propose that combining features from the ECG signal can be helpful in situations where pure tri-axial acceleration (3D-ACC) models have trouble differentiating between activities with relative motion (e.g., walking, stair climbing) but significant differences in their heart rate signatures. [ABSTRACT FROM AUTHOR]

Mobile ubiquitous computing has not only enriched human comfort but also has a deep impact on changing standards of human daily life. Modern inventions can be even more automated by using the Internet of Things (IoT) and Artificial Intelligence (AI). Mobile devices, body area networks, and embedded computing systems allow healthcare providers to continuously assess and monitor their patients but also bring privacy concerns. This paper proposes a smartphone-based end-to-end novel framework named PP-SPA for privacy-preserved Human Activity Recognition (HAR) and real-time activity functioning support using the smartphone-based virtual personal assistant. PP-SPA helps to improve the routine life functioning of the Cognitive Impaired individuals. PP-SPA uses a highly accurate machine learning model that takes input from smartphone sensors (i.e., accelerometer, gyroscope, magnetometer, and GPS) for accurate HAR and uses a digital diary to recommend real-time support for improvement in individual's health. PP-SPA yields a proficient accuracy of 90% with the Hoeffding Tree and Logistic Regression algorithm which endeavors reasonable models in terms of uncertainty. [ABSTRACT FROM AUTHOR]

Federated Learning (FL) has recently received significant interest, thanks to its capability of protecting data privacy. However, existing FL paradigms yield unsatisfactory performance for a wide class of human activity recognition (HAR) applications, since they are oblivious to the intrinsic relationship between data of different users. We propose ClusterFL, a clustering-based federated learning system that can provide high model accuracy and low communication overhead for HAR applications. ClusterFL features a novel clustered multitask federated learning framework that minimizes the empirical training loss of multiple learned models while automatically capturing the intrinsic clustering relationship among the nodes. We theoretically prove the convergence of proposed FL framework for non-convex and strongly convex models and provide the guidance on selection of hyper-parameters for achieving such convergence. Based on the learned cluster relationship, ClusterFL can efficiently drop the nodes that converge slower or have little correlations with others in each cluster, significantly speeding up the convergence while maintaining the accuracy performance. We evaluate the performance of ClusterFL on an NVIDIA edge testbed using four new HAR datasets collected from 145 users. The results show that ClusterFL outperforms several state-of-the-art FL paradigms in terms of overall accuracy and can save more than 50% communication overhead. [ABSTRACT FROM AUTHOR]

Human gait activity recognition is an emerging field of motion analysis that can be applied in various application domains. One of the most attractive applications includes monitoring of gait disorder patients, tracking their disease progression and the modification/evaluation of drugs. This paper proposes a robust, wearable gait motion data acquisition system that allows either the classification of recorded gait data into desirable activities or the identification of common risk factors, thus enhancing the subject's quality of life. Gait motion information was acquired using accelerometers and gyroscopes mounted on the lower limbs, where the sensors were exposed to inertial forces during gait. Additionally, leg muscle activity was measured using strain gauge sensors. As a matter of fact, we wanted to identify different gait activities within each gait recording by utilizing Machine Learning algorithms. In line with this, various Machine Learning methods were tested and compared to establish the best-performing algorithm for the classification of the recorded gait information. The combination of attention-based convolutional and recurrent neural networks algorithms outperformed the other tested algorithms and was individually tested further on the datasets of five subjects and delivered the following averaged results of classification: 98.9% accuracy, 96.8% precision, 97.8% sensitivity, 99.1% specificity and 97.3% F1-score. Moreover, the algorithm's robustness was also verified with the successful detection of freezing gait episodes in a Parkinson's disease patient. The results of this study indicate a feasible gait event classification method capable of complete algorithm personalization. [ABSTRACT FROM AUTHOR]

Identification of human activity with decent precision is a challenging task in the field of computer vision, especially when applying for surveillance purpose. A rule-based classifier method is proposed in this paper, which is capable of recognizing a view-invariant multiple human activity recognition in real time. A single Kinect sensor is used for the input of RGB-D data in real time. Initially, a skeleton-tracking algorithm is applied. After tracking the skeletons, activities are recognized from each individually tracked skeleton independently. Different rules are defined to recognize discrete skeleton positions and classify a particular order of multiple postures into activities. During the experimentation, we examine about 14 activities and found that the proposed method is robust and efficient concerning multiple views, scaling and phase variation activities during different realistic acts. A self-generated dataset in the controlled environment is used for the experiment. About 2 min of data was collected. Data from two different males were collected for multiple human activities. Experimental results show that the proposed method is flexible and efficient for multiple view activities as well as scale and phase variation activities. It provides a detection accuracy of 98%. [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) applications have received much attention due to their necessary implementations in various domains, including Industry 5.0 applications such as smart homes, e-health, and various Internet of Things applications. Deep learning (DL) techniques have shown impressive performance in different classification tasks, including HAR. Accordingly, in this article, we develop a comprehensive HAR system based on a novel DL architecture called Multi-ResAtt (multilevel residual network with attention). This model incorporates initial blocks and residual modules aligned in parallel. Multi-ResAtt learns data representations on the inertial measurement units level. Multi-ResAtt integrates a recurrent neural network with attention to extract time-series features and perform activity recognition. We consider complex human activities collected from wearable sensors to evaluate the Multi-ResAtt using three public datasets, Opportunity; UniMiB-SHAR; and PAMAP2. Additionally, we compared the proposed Multi-ResAtt to several DL models and existing HAR systems, and it achieved significant performance. [ABSTRACT FROM AUTHOR]

Human action acknowledgment is an abundant and significant area for machine learning-based researchers due to the level of accuracy in identifying human actions. Due to the rapid growth of technologies in the machine and deep learning techniques, wireless sensors, handy Internet of Things (IoT) devices, and Wireless Fidelity (Wi-Fi), the activity recognition process is made effective with higher accuracy. By using those booming technologies and preserving the privacy of the test person we propose a novel human action recognition model that uses the channel state information (CSI) from Wi-Fi and the most prominent machine learning model, CNN with LSTM. Initially, CSI is introduced, the changes in CSI signals are assessed, and the obtained data samples are made as input to the CNN-LSTM model. To make the recognition more accurate, we also incorporated Kalman filters for noise removal and smoothed the data sample. Furthermore, we have used an image segmentation procedure to identify the initial and end times of all the activities considered and to fragment the image obtained, which is further fed as input to the CNN-LSTM model. Getting a dataset for the experiment is a herculean task. Hence a self-collected dataset is used to assess, or model proposed. Finally, the results obtained are verified and validated for their correctness with appropriate machine learning metrics and parameters like accuracy, F1 score, etc. Our proposed model affords the accuracy of 98.96% for all the considered activities. The model can adapt itself even for a minimum sampling rate and subcarriers found in the test bed. [ABSTRACT FROM AUTHOR]

Using Wi-Fi signals to sense target activity is a promising study field, accounting for convenience concerns. However, it remains challenging to recognize target activity as a way of high-precision and stability due to the multi-path effect in Wi-Fi signals. In this paper, we propose a robust framework named WiPD, for accurate activity recognition based on Wi-Fi phase difference data. Firstly, a novel feature representation mechanism named visualized spectrum matrix (VSM) for Wi-Fi activity recognition is proposed. VSM is generated by performing a Short Time Fourier Transform operation on Wi-Fi phase difference data. Then, we design a neural network with the input type of VSM, namely, WiPD-Net, in which the activity features are extracted by both four convolutional neural network submodules and two WiPD-Block submodules. Experiment results show that our proposed WiPD-Net outperforms the existing baselines on our dataset and one public dataset. In particular, WiPD-Net can reach up to an accuracy of 99.80%, and achieve a good migration performance among five Wi-Fi environments. [ABSTRACT FROM AUTHOR]

Ambient Intelligence is an emerging technology that allows for reasoning of activities that are carried out in a smart environment. Smart sensors play a vital role in activity recognition tasks. Though a great number of sensors provide more pinpointed information about where and when the interaction in the environment occurred, addition of sensors also results in higher power consumption and greater cost constraints. The number of training samples used for activity recognition tasks also increases and becomes computationally expensive. Thus, to overcome these constraints sensor optimization in smart environment is carried out. Mapping of sensors to a user's activity becomes tedious when more than one activity is carried out at the same timestamp. Unambiguous sensor-activity relationship is derived with the help of spatial and temporal context represented by ontology model that captures real time sensor data. Since there exits spatial noise during activity annotation process, it is eliminated with help of pointwise mutual information score. Spatial and temporal data of an activity helps sensor optimization in multi resident living with concurrent activity occurrence. Inactive sensors are removed using mutual information score and redundant sensors are contextually paired based on where an activity happens. By doing so, only unique set of sensors are paired every time even when concurrent activity occurs. The quality of features derived is tested by testing the accuracy of the model by activity recognition task using Random Forest algorithm. Nearly 24% of sensors were identified as inactive in the smart environment. The accuracy of Human Activity Recognition (HAR) task remained constant even after removing the inactive sensors. Addition of spatial context helped to improve HAR with increased accuracy. Contextual pairing of redundant sensors resulted in minimal drop in the model's accuracy. [ABSTRACT FROM AUTHOR]

Ubiquitous mobile sensors on human activity recognition pose the threat of leaking personal information that is implicitly contained within the time-series sensor signals and can be extracted by attackers. Existing protective methods only support specific sensitive attributes and require massive relevant sensitive ground truth for training, which is unfavourable to users. To fill this gap, we propose a novel data transformation framework for prohibiting the leakage of sensitive information from sensor data. The proposed framework transforms raw sensor data into a new format, where the sensitive information is hidden and the desired information (e.g., human activities) is retained. Training can be conducted without using any personal information as ground truth. Meanwhile, multiple attributes of sensitive information (e.g., age, gender) can be collectively hidden through a one-time transformation. The experimental results on two multimodal sensor-based human activity datasets manifest the feasibility of the presented framework in hiding users’ sensitive information (inference MAE increases $\sim$ ∼ 2 times and inference accuracy degrades $\sim$ ∼ 50%) without degrading the usability of the data for activity recognition (only $\sim$ ∼ 2% accuracy degradation). [ABSTRACT FROM AUTHOR]

Human activity recognition is a key to a lot of applications such as healthcare and smart home. In this study, we provide a comprehensive survey on recent advances and challenges in human activity recognition (HAR) with deep learning. Although there are many surveys on HAR, they focused mainly on the taxonomy of HAR and reviewed the state-of-the-art HAR systems implemented with conventional machine learning methods. Recently, several works have also been done on reviewing studies that use deep models for HAR, whereas these works cover few deep models and their variants. There is still a need for a comprehensive and in-depth survey on HAR with recently developed deep learning methods. [ABSTRACT FROM AUTHOR]