Search

Increasing number of ubiquitous devices are being used in the medical field to collect patient information. Those connected sensors can potentially be exploited by third parties who want to misuse personal information and compromise the security, which could ultimately result even in patient death. This paper addresses the security concerns in eHealth networks and suggests a new approach to dealing with anomalies. In particular we propose a concept for safe in-hospital learning from internet of health things (IoHT) device data while securing the network traffic with a collaboratively trained anomaly detection system using federated learning. That way, real time traffic anomaly detection is achieved, while maintaining collaboration between hospitals and keeping local data secure and private. Since not only the network metadata, but also the actual medical data is relevant to anomaly detection, we propose to use differential privacy (DP) for providing formal guarantees of the privacy spending accumulated during the federated learning.

An important target for machine learning research is obtaining unbiased results, which require addressing bias that might be present in the data as well as the methodology. This is of utmost importance in medical applications of machine learning, where trained models should be unbiased so as to result in systems that are widely applicable, reliable and fair. Since bias can sometimes be introduced through the data itself, in this paper we investigate the presence of ethnoracial bias in patients’ clinical data. We focus primarily on vital signs and demographic information and classify patient ethnoraces in subsets of two from the three ethnoracial groups (African Americans, Caucasians, and Hispanics). Our results show that ethnorace can be identified in two out of three patients, setting the initial base for further investigation of the complex issue of ehtnoracial bias.

Understanding people’s eating habits plays a crucial role in interventions promoting a healthy lifestyle. This requires objective measurement of the time at which a meal takes place, the duration of the meal, and what the individual eats. Smartwatches and similar wrist-worn devices are an emerging technology that offers the possibility of practical and real-time eating monitoring in an unobtrusive, accessible, and affordable way. To this end, we present a novel approach for the detection of eating segments with a wrist-worn device and fusion of deep and classical machine learning. It integrates a novel data selection method to create the training dataset, and a method that incorporates knowledge from raw and virtual sensor modalities for training with highly imbalanced datasets. The proposed method was evaluated using data from 12 subjects recorded in the wild, without any restriction about the type of meals that could be consumed, the cutlery used for the meal, or the location where the meal took place. The recordings consist of data from accelerometer and gyroscope sensors. The experiments show that our method for detection of eating segments achieves precision of 0.85, recall of 0.81, and F1-score of 0.82 in a person-independent manner. The results obtained in this study indicate that reliable eating detection using in the wild recorded data is possible with the use of wearable sensors on the wrist.

Wireless network (radio) virtualization and its synergy with ML/AI-based technologies is a novel concept that can efficiently address problems of legacy networks, such as flash crowds. This paper discusses the integration aspects of intelligence-based technologies with Sate-of-the-Art end-to-end reconfigurable, flexible and scalable network architecture, capable of handling demands in flash crowd scenarios. The presented results, demonstrate that advanced solutions based on ML can significantly improve the network proactivity and adaptivity by reliably predicting flash crowd scenarios. The results also show that in case of low dataset fidelity, conventional statistical models are a more suitable option.

Human Activity Recognition (HAR) from wearable sensors has gained significant attention in the last few decades, largely because of the potential healthcare benefits. For many years, HAR was done using classical machine learning approaches that require extraction of features. With the resurgence of deep learning, a major shift happened and at the moment, HAR researchers are mainly investigating different kinds of deep neural networks. However, deep learning comes with the challenge of having access to large amounts of labeled examples, which in the field of HAR is considered an expensive task, both in terms of time and effort. Another challenge is the fact that the training and testing data in HAR can be different due to the personal preferences of different people when performing the same activity. In order to try and mitigate these problems, in this paper we explore transfer learning, a paradigm for transferring knowledge from a source domain, to another related target domain. More specifically, we explore the effects of transferring knowledge between two open-source datasets, the Opportunity and JSI-FOS datasets, using weight-transfer for the DeepConvLSTM architecture. We also explore the performance of this transfer at different amounts of labeled data from the target domain. The experiments showed that it is beneficial to transfer the weights of fewer layers, and that deep transfer learning can perform better than a domain-specific deep end-to-end model in specific circumstances. Finally, we show that deep transfer learning is a viable alternative to classical machine learning approaches as it produces comparable results and does not require feature extraction.

Human falls are common source of injury among the elderly, because often the elderly person is injured and cannot call for help. In the literature this is addressed by various fall-detection systems, of which most common are the ones that use wearable sensors. This paper describes the winning method developed for the Challenge Up: Multimodal Fall Detection competition. It is a multi-sensor data-fusion machine-learning method that recognizes human activities and falls using 5 wearable inertial sensors: accelerometers and gyroscopes. The method was evaluated on a dataset collected by 12 subjects of which 3 were used as a test-data for the challenge. In order to optimally adapt the method to the 3 test subjects, we performed an unsupervised similarity search—that finds the three most similar users to the three users in the test data. This helped us to tune the method and its parameters to the 3 most similar users as the ones used for the test. The internal evaluation on the 9 users showed that with this optimized configuration the method achieves 98% accuracy. During the final evaluation for the challenge, our method achieved the highest results (82.5% F1-score, and 98% accuracy) and won the competition.

We present the concept of an Intelligent Assistant Carer system for the elderly, designed to help with active aging and to facilitate the interactions with carers. The system is modular, allowing the users to choose the appropriate functions according to their needs, and is built on an open platform in order to make it compatible with third-party products and services. Currently, the system consists of a wearable device (a smartwatch) and an internet portal that manages the data and takes care of the interactions between the user, the carers, and the support services. We present in detail one of the modules, i.e., fall detection, and the results of a pilot study for the system on 150 users over the course of 3 months.

In this paper we summarize the contributions of participants to the Sussex-Huawei Transportation-Locomotion (SHL) Recognition Challenge organized at the HASCA Workshop of Ubi-Comp 2019. The goal of this machine learning/data science challenge is to recognize eight locomotion and transportation activities (Still, Walk, Run, Bike, Bus, Car, Train, Subway) from the inertial sensor data of a smartphone in a placement independent manner. The training data is collected with smartphones placed at three body positions (Torso, Bag and Hips), while the testing data is collected with a smartphone placed at another body position (Hand). We introduce the dataset used in the challenge and the protocol for the competition. We present a meta-analysis of the contributions from 14 submissions, their approaches, the software tools used, computational cost and the achieved results. Overall, three submissions achieved F1 scores between 70% and 80% five with F1 scores between 60% and 70%, five between between 50% and 60%, and one below 50%, with a latency of a maximum of 5 seconds.

Transportation and locomotion mode recognition from multimodal smartphone sensors is useful for providing just-in-time context-aware assistance. However, the field is currently held back by the lack of standardized datasets, recognition tasks, and evaluation criteria. Currently, the recognition methods are often tested on the ad hoc datasets acquired for one-off recognition problems and with different choices of sensors. This prevents a systematic comparative evaluation of methods within and across research groups. Our goal is to address these issues by: 1) introducing a publicly available, large-scale dataset for transportation and locomotion mode recognition from multimodal smartphone sensors; 2) suggesting 12 reference recognition scenarios, which are a superset of the tasks we identified in the related work; 3) suggesting relevant combinations of sensors to use based on energy considerations among accelerometer, gyroscope, magnetometer, and global positioning system sensors; and 4) defining precise evaluation criteria, including training and testing sets, evaluation measures, and user-independent and sensor-placement independent evaluations. Based on this, we report a systematic study of the relevance of statistical and frequency features based on the information theoretical criteria to inform recognition systems. We then systematically report the reference performance obtained on all the identified recognition scenarios using a machine-learning recognition pipeline. The extent of this analysis and the clear definition of the recognition tasks enable future researchers to evaluate their own methods in a comparable manner, thus contributing to further advances in the field. The dataset and the code are available online. http://www.shl-dataset.org/.

The paper presents a Machine Learning (ML) approach to household Electrical Energy (EE) consumption prediction. It includes: data preprocessing, feature engineering, learning a classification model, and experimental evaluation on one of the largest datasets for household EE consumption – DataPort dataset. Beside the features extracted on the historical EE consumption, we additionally analyze weather and contextual-calendar related features. We believe that the combination of multiple sources of data (calendar, weather, historical EE consumption) provides more information to the model in order to learn better performing model. The experimental results showed that in all the cases the ML algorithms outperform the baselines, with the best performing the XGBoost - achieved 0.69 RMSE score, 0.41 MAE score and 0.67 R2 score which is significantly better than the best performing baseline model (the value from 24 h ago). Additionally, the results show that the largest errors are made for the weekends, which was expected due to the irregularities in the schedule - trips, vacations, etc.

The Sussex-Huawei Transportation-Locomotion (SHL) Recognition Challenge 2018 aims to recognize eight transportation activities (Still, Walk, Run, Bike, Bus, Car, Train, Subway) from the inertial and pressure sensor data of a smartphone. In this chapter, we, as part of competition organizing team, present reference recognition performance obtained by applying various classical and deep-learning classifiers to the testing dataset. The classical classifiers include naive Bayes, decision tree, random forest, K-nearest neighbours and support vector machine, while the deep-learning classifiers include fully-connected and convolutional deep neural networks. We feed different types of input to the classifier, including hand-crafted features, raw sensor data in the time domain, and in the frequency domain. We additionally employ a post-processing scheme, which smoothens the predictions in order and improves the recognition performance. Results show that convolutional neural network operating on frequency-domain raw data achieves the best performance among all the classifiers. Finally, we achieve a benchmark result with F1 score 92.9%, which is comparable to the best result from the team that won the competition (achieving F1 score 93.9%). The competition dataset and the benchmark implementation is made available online (http://www.shl-dataset.org/).

Neural text generation is the process of a training neural network to generate a human understandable text (poem, story, article). Recurrent Neural Networks and Long-Short Term Memory are powerful sequence models that are suitable for this kind of task. In this paper, we have developed two types of language models, one generating news articles and the other generating poems in Macedonian language. We developed and tested several different model architectures, among which we also tried transfer-learning model, since text generation requires a lot of processing time. As evaluation metric we used ROUGE-N metric (Recall-Oriented Understudy for Gisting Evaluation), where the generated text was tested against a reference text written by an expert. The results showed that even though the generate text had flaws, it was human understandable, and it was consistent throughout the sentences. To the best of our knowledge this is a first attempt in automatic text generation (poems and articles) in Macedonian language using Deep Learning.

Multiple Contexts Ensemble (MCE) method to estimate the human energy expenditure (EE).MCE outperforms conventional regression approaches, ensembles and BodyMedia EE device.MCE provides better accuracy than using only the activity of the user as the context.MCE is independent of the machine learning algorithm, thus any algorithm can be used. Monitoring human energy expenditure (EE) is important in many health and sports applications, since the energy expenditure directly reflects the intensity of physical activity. The actual energy expenditure is unpractical to measure; therefore, it is often estimated from the physical activity measured with accelerometers and other sensors. Previous studies have demonstrated that using a person's activity as the context in which the EE is estimated, and using multiple sensors, improves the estimation. In this study, we go a step further by proposing a context-based reasoning method that uses multiple contexts provided by multiple sensors. The proposed Multiple Contexts Ensemble (MCE) approach first extracts multiple features from the sensor data. Each feature is used as a context for which multiple regression models are built using the remaining features as training data: for each value of the context feature, a regression model is trained on a subset of the dataset with that value. When evaluating a data sample, the models corresponding to the context (feature) values in the evaluated sample are assembled into an ensemble of regression models that estimates the EE of the user. Experiments showed that the MCE method outperforms (in terms of lower root means squared error and lower mean absolute error): (i) five single-regression approaches (linear and non-linear); (ii) two ensemble approaches: Bagging and Random subspace; (iii) an approach that uses artificial neural networks trained on accelerometer-data only; and (iv) BodyMedia (a state-of-the-art commercial EE-estimation device).

In this paper we, as part of the Sussex-Huawei Locomotion-Transportation (SHL) Recognition Challenge organizing team, present reference recognition performance obtained by applying various classical and deep-learning classifiers to the testing dataset. We aim to recognize eight modes of transportation (Still, Walk, Run, Bike, Bus, Car, Train, Subway) from smartphone inertial sensors: accelerometer, gyroscope and magnetometer. The classical classifiers include naive Bayesian, decision tree, random forest, K-nearest neighbour and support vector machine, while the deep-learning classifiers include fully-connected and convolutional deep neural networks. We feed different types of input to the classifier, including hand-crafted features, raw sensor data in the time domain, and in the frequency domain. We employ a post-processing scheme to improve the recognition performance. Results show that convolutional neural network operating on frequency-domain raw data achieves the best performance among all the classifiers.

We explain how to obtain a highly versatile and precisely annotated dataset for the multimodal locomotion of mobile users. After presenting the experimental setup, data management challenges and potential applications, we conclude with the best practices for assuring data quality and reducing loss. The dataset currently comprises 7 months of measurements, collected by smartphone's sensors and a body-worn camera, while the 3 participants used 8 different modes of transportation. It comprises 950 GB of sensor data, which corresponds to 750 hours of labelled data. The obtained data will be useful for a wide range of research questions related to activity recognition, and will be made available to the community1.

Ensuring the validity and usability of activity recognition approaches requires agreement on a set of standard evaluation methods. Due to the diversity of the sensors and other hardware employed, however, designing, implementing, and accepting standard tests is a difficult task. This article presents an initiative to evaluate activity recognition systems: a living-lab evaluation established through the annual Evaluating Ambient Assisted Living Systems through Competitive Benchmarking--Activity Recognition (EvAAL-AR) competition. In the EvAAL-AR, each team brings its own activity-recognition system; all systems are evaluated live on the same activity scenario performed by an actor. The evaluation criteria attempt to capture practical usability: recognition accuracy, user acceptance, recognition delay, installation complexity, and interoperability with ambient assisted living systems. Here, the authors discuss the competition and the competing systems, focusing on the system that achieved the best recognition accuracy, and the system that was evaluated as the best overall. The authors also discuss lessons learned from the competition and ideas for future development of the competition and of the activity recognition field in general.

We present a novel unsupervised approach, UnADevs, for discovering activity clusters corresponding to periodic and stationary activities in streaming sensor data. Such activities usually last for some time, which is exploited by our method; it includes mechanisms to regulate sensitivity to brief outliers and can discover multiple clusters overlapping in time to better deal with deviations from nominal behaviour. The method was evaluated on two activity datasets containing large number of activities (14 and 33 respectively) against online agglomerative clustering and DBSCAN. In a multi-criteria evaluation, our approach achieved significantly better performance on majority of the measures, with the advantages that: (i) it does not require to specify the number of clusters beforehand (it is open ended); (ii) it is online and can find clusters in real time; (iii) it has constant time complexity; (iv) and it is memory efficient as it does not keep the data samples in memory. Overall, it has managed to discover 616 of the total 717 activities. Because it discovers clusters of activities in real time, it is ideal to work alongside an active learning system.

Affect recognition is an important task in ubiquitous computing, in particular in health and human-computer interaction. In the former, it contributes to the timely detection and treatment of emotional and mental disorders, and in the latter, it enables indigenous interaction and enhanced user experience. We present an inter-domain study for affect recognition on seven different datasets, recorded with six different sensors, three different sensor placements, 211 subjects and nearly 1000 hours of labelled data. The datasets are processed and translated into a common spectro-temporal space. The data represented in the common spectro-temporal space is used to train a deep neural network (DNN) for arousal recognition that benefits from the large amounts of data even when the data are heterogeneous (i.e., different sensors and different datasets). The DNN approach outperforms the classical machine-learning approaches in six out of seven datasets.

Human activity recognition is a basic building block in numerous healthcare systems, mainly because the ability to understand the user's situation and context. This paper presents a solution to the general problem with evaluation of human activity recognition systems, i.e., an activity recognition system may perform perfectly in controlled lab experiments, but significantly worse once applied to more realistic conditions. The solution is presented through the practical experience gained with the creation of our RAReFall activity recognition system. Although the system was awarded the first place at the EvAAL-AR live competition, the recognition accuracy at the competition was significantly lower compared to the controlled lab experiments performed just before the competition. To overcome the encountered problem we developed an automatic calibration method, which solves the encountered problem by adapting and re-calibrating the accelerometer data in real-time while the user is performing everyday activities. The method increased the overall accuracy for 8 percentage points and for 51 percentage points for the sitting activity.

This paper presents an approach to detecting perceived stress in students using data collected with smartphones. The goal is to develop a machine-learning model that can unobtrusively detect the stress level in students using data from several smartphone sources: accelerometers, audio recorder, GPS, Wi-Fi, call log and light sensor. From these, features were constructed describing the students' deviation from usual behaviour. As ground truth, we used the data obtained from stress level questionnaires with three possible stress levels: "Not stressed", "Slightly stressed" and "Stressed". Several machine learning approaches were tested: a general models for all the students, models for cluster of similar students, and student-specific models. Our findings show that the perceived stress is highly subjective and that only person-specific models are substantially better than the baseline.

GUIDL (Graphical User Interface Description Language) system has been proposed as an effective aiding technology that enables inclusion of visually impaired computer users into activities of graphical user interface design. GUIDL system implements many aiding concepts that make creation of graphical user interfaces suitable for visually impaired. In order to make GUIDL system even more accessible to visually impaired a GUIDL IA (Intelligent Assistant) is proposed as a mean of providing a more suitable and simpler way of getting help about various parts of GUIDL system. GUIDL IA enables visually impaired to ask questions in the form of natural language compared to browsing through instruction manual which makes getting the right information easier and quicker. GUIDL IA also enables visually impaired to hear the answer through integrated text-to-speech synthesizer that enables more natural pronunciation of given answers. In this paper GUIDL IA is presented and discussed along with the results about its effectives.

Monitoring human energy expenditure is important in many health and sport applications, since the energy expenditure directly reflects the level of physical activity. The actual energy expenditure is unpractical to measure; hence, the field aims at estimating it by measuring the physical activity with accelerometers and other sensors. Current advanced estimators use a context-dependent approach in which a different regression model is invoked for different activities of the user. In this paper, we go a step further and use multiple contexts corresponding to multiple sensors, resulting in an ensemble of models for energy expenditure estimation. This provides a multi-view perspective, which leads to a better estimation of the energy. The proposed method was experimentally evaluated on a comprehensive set of activities where it outperformed the current state-of-the-art.

Ambient assisted living (AAL) systems need to understand the user’s situation, which makes activity recognition an important component. Falls are one of the most critical problems of the elderly, so AAL systems often incorporate fall detection. We present an activity recognition (AR) and fall detection (FD) system aiming to provide robust real-time performance. It uses two wearable accelerometers, since this is probably the most mature technology for such purpose. For the AR, we developed an architecture that combines rules to recognize postures, which ensures that the behavior of the system is predictable and robust, and classifiers trained with machine learning algorithms, which provide maximum accuracy in the cases that cannot be handled by the rules. For the FD, rules are used that take into account high accelerations associated with falls and the recognized horizontal orientation (e.g., falling is often followed by lying). The system was tested on a dataset containing a wide range of activities, two different types of falls and two events easily mistaken for falls. The F-measure of the AR was 99 %, even though it was never tested on the same persons it was trained on. The F-measure of the FD was 78 % due to the difficulty of the events to be recognized and the need for real-time performance, which made it impossible to rely on the recognition of long lying after a fall.

This paper presents a context-aware, multi-agent system called “Confidence” that helps elderly people remain independent longer by detecting falls and unusual movement, which may indicate a health problem. The system combines state-of-the-art sensor technologies and four groups of agents providing a reliable, robust, flexible monitoring system. It can call for help in case of an emergency, and issue warnings if unusual behavior is detected. The first group gathers data from the location and inertial sensors and suppresses noise. The second group reconstructs the position and activity of a person and detects the context. The third group assesses the person's condition in the environment and reacts to critical situations such as falls. The fourth group detects unusual behavior as an indicator of a potential health problem. The system was successfully tested on a scenario consisting of events that were difficult to recognize as falls, as well as in a scenario consisting of normal days and days when the person was ill. It was also demonstrated live several times, with excellent performance in complex situations.

One of the essential tasks of healthcare and smart-living systems is to recognize the current activity of a particular user. Such activity recognition (AR) is demanding when only limited sensors are used, such as accelerometers. Given a small number of accelerometers, intelligent AR systems often use simple architectures, either general or specific for their AR. In this paper, a system for AR named TriLAR is presented. TriLAR has an AR-specific architecture consisting of three layers: (i) a bottom layer, where an arbitrary number of AR methods can be used to recognize the current activity; (ii) a middle layer, where the predictions from the bottom-layer methods are inputs for a hierarchical structure that combines domain knowledge and meta-classification; and (iii) a top layer, where a hidden Markov model is used to correct spurious transitions between the recognized activities from the middle layer. The middle layer has a hierarchical, three-level structure. First, a meta-classifier is used to make the initial separation between the most distinct activities. Second, domain knowledge in the form of rules is used to differentiate between the remaining activities, recognizing those of interest (i.e., static activities). Third, another meta-classifier deals with the remaining activities. In this way, each activity is recognized by the method best suited to it, leaving unrecognized activities to the next method. This architecture was tested on a dataset recorded using ten volunteers who acted out a complex, real-life scenario while wearing accelerometers placed on the chest, thigh, and ankle. The results show that TriLAR successfully recognized elementary activities using one or two sensors and significantly outperformed three standard, single-layer methods with all sensor placements.

To further extend the applicability of wearable sensors, methods for accurately extracting subtle psychological information from the sensor data are required. However, accessing subjective information in everyday life, such as cognitive load, remains challenging. To bring consensus on methods for cognitive load monitoring, a machine learning challenge is organized. The participants developed machine learning methods for cognitive load classification using wrist-worn physiological sensors’ data, namely heart rate, R-R intervals, skin conductance, and skin temperature. The data from subjects solving cognitive tasks of varying difficulty is used for the challenge. This article presents a systematic comparison and multi-strategic performance evaluation of the thirteen methods submitted to this challenge. A systematic comparison of preprocessing techniques, classification algorithms, and implementation techniques is presented. Performance variations for different task difficulty levels, different subjects, and different experiment periods are evaluated. The results indicate that the most robust methods used multimodal sensor data, classical classification approaches such as decision trees and support vector machines or their ensembles, and Bayesian hyperparameter optimization for hyperparameter tuning. The most accurate models used handcrafted features that are further selected using sequential backward floating search and evaluated using stratified person-aware cross-validation strategy. Moreover, the results indicated better classification performance for specific test subjects, the tasks with the highest difficulty, and in some cases, the time elapsed since the start of the experiment. This dependency is likely due to model overfitting or due to the subjective nature of the psychophysiological process. The intersubject variability in responses is challenging to be captured through objective binary labels for cognitive load, thereby warranting more sophisticated annotation approaches.