Search

The Wearables for Epilepsy And Research (WEAR) International Study Group identified a set of methodology standards to guide research on wearable devices for seizure detection. We formed an international consortium of experts from clinical research, engineering, computer science, and data analytics at the beginning of 2020. The study protocols and practical experience acquired during the development of wearable research studies were discussed and analyzed during bi-weekly virtual meetings to highlight commonalities, strengths, and weaknesses, and to formulate recommendations. Seven major essential components of the experimental design were identified, and recommendations were formulated about: (1) description of study aims, (2) policies and agreements, (3) study population, (4) data collection and technical infrastructure, (5) devices, (6) reporting results, and (7) data sharing. Introducing a framework of methodology standards promotes optimal, accurate, and consistent data collection. It also guarantees that studies are generalizable and comparable, and that results can be replicated, validated, and shared.

Day to day interactions with wearable and pervasive systems lead to collected data that capture various aspects of human behavior and enable machine learning algorithms to extract extensive information about users. We discuss privacy risk awareness, and ways to preserve privacy and integrate it in current frameworks.

The International Symposium on Wearable Computers (ISWC) has been the leading research venue for wearable technology research since 1997. In 2019, the 23rd ISWC was held in London, U.K., from Sep. 9th–13th. Following on the last eight years of successful collaboration, ISWC was colocated with the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp).

Reports on major events and exhibitions that were part of the 2018 ACM International Symposium on Wearable Computers (ISWC) in Singapore in October 2018.

The synchronization of wearable devices in distributed, multi-device systems is a persistent challenge. Particularly machine learning approaches suffer from the devices’ inaccurate clock sources and unmatched time. While the online synchronization based on radio transmission is energy-intensive, offline approaches originated in activity recognition suffer from inaccurate motion patterns. In recent years, intra-body communication emerged as a promising technique that uses the human body as a limited and hence more efficient medium. Due to the absence of commercial platforms, applications are rare and underinvestigated. To boost their development and to enable the precise synchronization, we introduce IBSync and propose to repurpose the ECG sensor in commercial wearable devices to detect artificial signals induced into the skin. The short-time Fourier transform and Pearson’s normalized cross-correlation are used to detect, precisely locate, and assign synchronization landmarks within the measurements. Based on a total of 105 min of recordings, we evaluated the concept and demonstrate its general feasibility with a promising accuracy of 0.203 ± 1.633 samples (1.587 ± 12.755 ms) in typical proximity to the transmitter.

Recent studies in Human Activity Recognition (HAR) have shown that Deep Learning methods are able to outperform classical Machine Learning algorithms. One popular Deep Learning architecture in HAR is the DeepConvLSTM. In this paper we propose to alter the DeepConvLSTM architecture to employ a 1-layered instead of a 2-layered LSTM. We validate our architecture change on 5 publicly available HAR datasets by comparing the predictive performance with and without the change employing varying hidden units within the LSTM layer(s). Results show that across all datasets, our architecture consistently improves on the original one: Recognition performance increases up to 11.7% for the F1-score, and our architecture significantly decreases the amount of learnable parameters. This improvement over DeepConvLSTM decreases training time by as much as 48%. Our results stand in contrast to the belief that one needs at least a 2-layered LSTM when dealing with sequential data. Based on our results we argue that said claim might not be applicable to sensor-based HAR., 6 pages, 2 figures, accepted at ISWC 21: International Symposium on Wearable Computer, Sept, 2021

Federated learning is a knowledge transmission and training process that occurs in turn between user models on edge devices and the training model in the central server. Due to privacy policies and concerns and heterogeneous data, this is a widespread requirement in federated learning applications. In this work, we use knowledge-based methods, and in particular case-based reasoning (CBR), to develop a wearable, explainable artificial intelligence (xAI) framework. CBR is a problem-solving AI approach for knowledge representation and manipulation, which considers successful solutions of past conditions that are likely to serve as candidate solutions for a requested problem. It enables federated learning when each user owns not only his/her private data, but also uniquely designed cases. New generated cases can be compared to the knowledge base and the recommendations enable the user to communicate better with the whole system. It improves users’ task performance and increases user acceptability when they need explanations to understand why and how AI algorithms arrive at these optimal solutions.

Today’s maintenance and renewal planning in transformer stations of energy distribution networks is mainly based on expert knowledge, experience gained from historical data as well as the knowledge gathered from regular on-site inspections. This approach is already reaching its limits due to insufficient databases and almost no information about the stations’ condition being gathered between inspection intervals. A condition-based strategy that requires more maintenance for equipment with a high probability of failure is needed. Great potential is promised by intelligent sensor-based diagnostics, where objective comparability can be achieved by condition monitoring of the station fleet. Cost-effective micro-electromechanical (MEMS)-bases sensor systems promise to provide a suitable solution for network operators and enable a widespread use. In our paper, we present a MEMS-based sensor system, that can be used to gain information about network transparency, station safety as well as maintenance and renewal planning. Moreover, we propose an intelligent measurement scheme which adaptively selects relevant data and avoids unneeded redundancy (Smart Data instead of Big Data).

Most off-the-shelf wearable devices do not provide reliable synchronization interfaces, causing multi-device sensing and machine learning approaches, e.g. for activity recognition, still to suffer from inaccurate clock sources and unmatched time. Instead of using active online synchronization techniques, such as those based on bidirectional wireless communication, we propose in this work to use the human heartbeat as a reference signal that is continuously and ubiquitously available throughout the entire body surface. We introduce PulSync, a novel approach that enables the alignment of sensor data across multiple devices utilizing the unique fingerprint-like character of the heart rate variability interval function. In an evaluation on a dataset from 25 subjects, we demonstrate the reliable alignment of independent ECG recordings with a mean accuracy of $-0.71\pm 3.44$ samples, respectively $-2.86\pm 11.43$ ms at 250 Hz sampling rate.

Inertial measurement units, such as those embedded in many smartwatches, enable to track the orientation of the body joint to which the wearable is attached. Tracking the user's entire body would thus require many wearables, which tends to be difficult to achieve in daily life. This paper presents a calibration-free method to match the data stream from a body-worn Inertial Measurement Unit, to any body joint estimates that are recognized and located from a camera in the environment. This allows networked cameras to associate detected body joints from nearby humans with an orientation stream from a nearby wearable; This wearable's orientation stream can be transformed and complemented with its user's full body pose estimates. Results show that in multi-user environments where users try to perform actions synchronously, out of 42 joint candidates, our calibration-free method obtains a perfect match within 83 to 155 samples (2.8 to 5.2 seconds), depending on the scenario. This would facilitate a seamless combination of wearable- and vision-based tracking for a robust, full-body user tracking.

Previous studies have shown that available benchmark datasets from the field of Human Activity Recognition are of limited use for Deep Learning applications. This can be traced back to issues in the quality, the scope, as well as in the variability of the datasets. These limitations often lead to overfitting of networks and thus to results that are only conditionally generalizable. One way to counteract this problem is to extend the data by using data augmentation techniques. This paper presents an algorithm and compares two augmentation strategies: (1) user-wise augmentation and (2) fold-wise augmentation to extend the size of a dataset here shown on the PAMAP2 dataset, with an arbitrary number of synthetic samples. These synthesized data resemble the user- and activity-specific characteristics and fit seamlessly into the dataset. They are created by a recurrent Generative Adversarial Network, with both the generator and discriminator modeled by a set of LSTM cells to produce the synthetic time-series data. In our evaluation, we trained four DeepConvLSTM models with supervised learning, three times with a LOSO cross-validation: one baseline model and two times with additional data but different augmentation strategies, as well as one model without cross-validation that monitors the synthesized data quality. The compared augmentation strategies demonstrate the impact as well as the generalized nature of the augmented data. By increasing the size of the dataset by factor 5, we improved the F1-Score by 11.0% with strategy (1) and 5.1% with strategy (2).

Current medical imaging uses MRI or CT images to diagnose tissue injuries. In addition to this classic procedure, there are also alternative technologies that have advantages against MRI or CT. These include electrical impedance tomography (EIT). With the help of EIT it is possible to obtain an initial screening of the body quickly and without a lot of hardware. Classical software-based methods of imaging reconstruction use a linear back projection or iterative approaches, such as Gauss-Newton algorithm. This paper introduces innovative approaches of artificial intelligence (AI) for imaging. For this purpose, extensive AI-based simulations with a Generative Adversial Network (GAN) are performed and the approaches are transferred to a gelatine phantom and to the human body within a small study.

Human-Computer Interaction (HCI) research has been focussing on the design of new interaction techniques and the understanding of people and the way they interact with computing devices and new technologies. The ways in which the work is performed with these interactive technologies has arguably been less of a focus. This workshop aims at addressing this specific aspect of Human-Computer Interaction in the control rooms domain. Control rooms are crucial elements of safety-critical infrastructures (e.g., crisis management, emergency medical services, fire services, power supply, or traffic management). They have been studied in terms of Human-Computer Interaction with respect to routine and emergency operations, human-machine task allocation, interaction design and evaluation approaches for more than 30 years. However, they are dynamic and evolving environments with, for instance, the gradual introduction of higher levels of automation/autonomy. While state of the art control rooms are still characterized by stationary workstations with several smaller screens and large wall-mounted displays, introducing mobile and wearable devices as well as IoT solutions could enable more flexible and cooperative ways of working. The workshop aims at understanding how recent technologies in HCI could change the way control rooms are designed, engineered and operated. This workshop is organized by the IFIP WG 13.5 on Human Error, Resilience, Reliability and Safety in System Development.

Photoplethysmography is an optical measurement principle which is present in most modern wearable devices such as fitness trackers and smartwatches. As the analysis of physiological signals requires reliable but energy-efficient algorithms, suitable datasets are essential for their development, evaluation, and benchmark. A broad variety of clinical datasets is available with recordings from medical pulse oximeters which traditionally apply transmission mode photoplethysmography at the fingertip or earlobe. However, only few publicly available datasets utilize recent reflective mode sensors which are typically worn at the wrist and whose signals show different characteristics. Moreover, the recordings are often advertised as raw, but then turn out to be preprocessed and filtered while the applied parameters are not stated. In this way, the heart rate and its variability can be extracted, but interesting secondary information from the non-stationary signal is often lost. Consequently, the test of novel signal processing approaches for wearable devices usually implies the gathering of own or the use of inappropriate data.In this paper, we present a multi-varied method to analyze the suitability and applicability of presumably raw photoplethysmography signals. We present an analytical tool which applies 7 decision metrics to characterize 10 publicly available datasets with a focus on less or ideally unfiltered, raw signals. Besides the review, we finally provide a guideline for future datasets, to suit to and to be applicable in digital signal processing, to support the development and evaluation of algorithms for resource-limited wearable devices.

Many current human-robot interactive systems tend to use accurate and fast - but also costly - actuators and tracking systems to establish working prototypes that are safe to use and deploy for user studies. This paper presents an embedded framework to build a desktop space for human-robot interaction, using an open-source robot arm, as well as two RGB cameras connected to a Raspberry Pi-based controller that allow a fast yet low-cost object tracking and manipulation in 3D. We show in our evaluations that this facilitates prototyping a number of systems in which user and robot arm can commonly interact with physical objects.

Transfer Learning is becoming increasingly important to the Human Activity Recognition community, as it enables algorithms to reuse what has already been learned from models. It promises shortened training times and increased classification results for new datasets and activity classes. However, the question of what exactly is transferred is not dealt with in detail in many of the recent publications, and it is furthermore often difficult to reproduce the presented results. Therefore we would like to contribute with this paper to the understanding of transfer learning for sensor-based human activity recognition. In our experiment use weight transfer to transfer models between two datasets, as well as between sensors from the same dataset. As source- and target- datasets PAMAP2 and Skoda Mini Checkpoint are used. The utilized network architecture is based on a DeepConvLSTM. The result of our investigation shows that transfer learning has to be considered in a very differentiated way, since the desired positive effects by applying the method depend very much on the data and also on the architecture used.

Depth imaging has, through recent technological advances, become ubiquitous as products become smaller, more affordable, and more precise. Depth cameras have also emerged as a promising modality for activity recognition as they allow detection of users&rsquo, body joints and postures. Increased resolutions have now enabled a novel use of depth cameras that facilitate more fine-grained activity descriptors: The remote detection of a person&rsquo, s breathing by picking up the small distance changes from the user&rsquo, s chest over time. We propose in this work a novel method to model chest elevation to robustly monitor a user&rsquo, s respiration, whenever users are sitting or standing, and facing the camera. The method is robust to users occasionally blocking their torso region and is able to provide meaningful breathing features to allow classification in activity recognition tasks. We illustrate that with this method, with specific activities such as paced-breathing meditating, performing breathing exercises, or post-exercise recovery, our model delivers a breathing accuracy that matches that of a commercial respiration chest monitor belt. Results show that the breathing rate can be detected with our method at an accuracy of 92 to 97% from a distance of two metres, outperforming state-of-the-art depth imagining methods especially for non-sedentary persons, and allowing separation of activities in respiration-derived features space.

With wearable computing research recently passing the 20-year mark, this survey looks back at how the field developed and explores where it’s headed. According to the authors, wearable computing is entering its most exciting phase yet, as it transitions from demonstrations to the creation of sustained markets and industries, which in turn should drive future research and innovation.

This proceedings presents the papers from Urb-IoT 2018 - 3rd EAI International Conference on IoT in Urban Space, which took place in Guimarães, Portugal on 21-22 November 2018. The conference aims to explore the emerging dynamics within the scope of the Internet of Things (IoT) and the new science of cities.The papers discuss fusion of heterogeneous urban sources, understanding urban data using machine learning and mining techniques, urban analytics, urban IoT infrastructures, crowd sourcing techniques, incentification and gamification, urban mobility and intelligent transportation systems, real time urban information systems, and more. The proceedings discuss innovative technologies that navigate industry and connectivity sectors in transportation, utility, public safety, healthcare, and education. The authors also discuss the increasing deployments of IoT technologies and the rise of the so-called 'Sensored Cities'‚ which are opening up new avenues of research opportunities towards that future.

Epilepsy seizure detection with wearable devices is an emerging research field. As opposed to the gold standard, consisting of simultaneous video and EEG monitoring of patients, wearables have the advantage that they put a lower burden on epilepsy patients. We report on the first stages in a research effort that is dedicated to the development of a multimodal seizure detection system specifically for focal onset epileptic seizures. By in-depth analysis of data from three in-hospital patients with each having six to nine seizures recorded, we show that such seizures can manifest very differently and thus significantly impact classification. Using a Random Forest model on a rich set of features, we have obtained overall precision and recall scores of up to 0.92 and 0.72 respectively. These results show that the approach has validity, but we identify the type of focal seizure to be a critical factor for the classification performance.

Wearable activity recognition research needs benchmark data, which rely heavily on synchronizing and annotating the inertial sensor data, in order to validate the activity classifiers. Such validation studies become challenging when recording outside the lab, over longer stretches of time. This paper presents a method that uses an inconspicuous, ear-worn device that allows the wearer to annotate his or her activities as the recording takes place. Since the ear-worn device has integrated inertial sensors, we use cross-correlation over all wearable inertial signals to propagate the annotations over all sensor streams. In a feasibility study with 7 participants performing 6 different physical activities, we show that our algorithm is able to synchronize signals between sensors worn on the body using cross-correlation, typically within a second. A comfort rating scale study has shown that attachment is critical. Button presses can thus define markers in synchronized activity data, resulting in a fast, comfortable, and reliable annotation method.

Affective computing aims to detect a person's affective state (e.g. emotion) based on observables. The link between affective states and biophysical data, collected in lab settings, has been established successfully. However, the number of realistic studies targeting affect detection in the wild is still limited. In this paper we present an exploratory field study, using physiological data of 11 healthy subjects. We aim to classify arousal, State-Trait Anxiety Inventory (STAI), stress, and valence self-reports, utilizing feature-based and convolutional neural network (CNN) methods. In addition, we extend the CNNs to multi-task CNNs, classifying all labels of interest simultaneously. Comparing the F1 score averaged over the different tasks and classifiers the CNNs reach an 1.8% higher score than the classical methods. However, the F1 scores barely exceed 45%. In the light of these results, we discuss pitfalls and challenges for physiology-based affective computing in the wild.

Signal segmentation is a generic task in many time series applications. We propose approaching it via Bayesian changepoint algorithms, i.e., by assigning segments between changepoints. When successive signals show a recurrent change-point pattern, estimating changepoint recurrence is beneficial for two reasons: While recurrent changepoints yield more robust signal segment estimates, non-recurrent changepoints bear valuable information for unsupervised anomaly detection.This study introduces the changepoint recurrence distribution (CPRD) as an empirical estimate of the recurrent behavior of observed changepoints. Two generic methods for incorporating the estimated CPRD into the process of assessing recurrence of future changepoints are suggested. The knowledge of non-recurrent changepoints arising from one of these methods allows additional unsupervised anomaly detection.The quality both of changepoint recurrence estimation via CPRD and of changepoint-related signal segmentation and un-supervised anomaly detection are verified in a proof-of-concept study for two exemplary machine tool monitoring tasks.

Hardware accelerators for compute intensive algorithms such as convolutional neural networks benefit from number representations with reduced precision. In this paper, we evaluate and extend a number representation based on power-of-two quantization enabling bit-shift-based processing of multiplications. We found that weights of a neural network can either be represented by a single 4 bit power-of-two value or with two 4 bit values depending on accuracy requirements. We evaluate the classification accuracy of VGG-16 and ResNet50 on the ImageNet dataset with weights represented in our novel number format. To include a more complex task, we additionally evaluate the format on two networks for semantic segmentation. In addition, we design a novel processing element based on bit-shifts which is configurable in terms of throughput (4 bit mode) and accuracy (8 bit mode). We evaluate this processing element in an FPGA implementation of a dedicated accelerator for neural networks incorporating a 32-by-64 processing array running at 250 MHz with 1 TOp/s peak throughput in 8 bit mode. The accelerator is capable of processing regular convolutional layers and dilated convolutions in combination with pooling and upsampling. For a semantic segmentation network with 108.5 GOp/frame, our FPGA implementation achieves a throughput of 7.0 FPS in the 8 bit accurate mode and upto 11.2 FPS in the 4 bit mode corresponding to 760.1 GOp/s and 1,218 GOp/s effective throughput, respectively. Finally, we compare the novel design to classical multiplier-based approaches in terms of FPGA utilization and power consumption. Our novel multiply-accumulate engines designed for the optimized number representation uses 9 % less logical elements while allowing double throughput compared to a classical implementation. Moreover, a measurement shows 25 % reduction of power consumption at same throughput. Therefore, our flexible design offers a solution to the trade-off between energy efficiency, accuracy, and high throughput.

In this study, we consider the problem of detecting process-related anomalies for machine tools. The similar shape of successive sensor signals, which arises due to the same process step sequence applied to each workpiece, suggests extracting shape-related features. In recent years, shapelets dominated the field of shape-related features. Unfortunately, they involve a high computational burden due to hyperparameter optimization.We introduce alternative shape-related features relying on abrupt signal changes (changepoints) reflecting the changes of process steps. During normal operation, changepoints follow a highly recurrent pattern, i.e., appear at similar locations. Thus, being able to distinguish regular, recurrent from abnormal, non-recurrent changepoints allows detecting process anomalies.For changepoint recurrence estimation, we extend the Bayesian Online Changepoint Detection (BOCPD) method. The extension allows distinguishing normal and abnormal changepoints relying on empirical estimates of the changepoint recurrence distribution. Subsequently, changepoint-related features are introduced and compared to shapelets and wavelet-based features in a case study comprising real-world machine tool data.Qualitative results verify changepoint locations being comparable to shapelet locations found by the FLAG shapelet approach. Furthermore, quantitative results suggest superior classification performance both to shapelets and wavelet-based features.

Photoplethysmography (PPG)-based continuous heart rate monitoring is essential in a number of domains, e.g., for healthcare or fitness applications. Recently, methods based on time-frequency spectra emerged to address the challenges of motion artefact compensation. However, existing approaches are highly parametrised and optimised for specific scenarios of small, public datasets. We address this fragmentation by contributing research into the robustness and generalisation capabilities of PPG-based heart rate estimation approaches. First, we introduce a novel large-scale dataset (called PPG-DaLiA), including a wide range of activities performed under close to real-life conditions. Second, we extend a state-of-the-art algorithm, significantly improving its performance on several datasets. Third, we introduce deep learning to this domain, and investigate various convolutional neural network architectures. Our end-to-end learning approach takes the time-frequency spectra of synchronised PPG- and accelerometer-signals as input, and provides the estimated heart rate as output. Finally, we compare the novel deep learning approach to classical methods, performing evaluation on four public datasets. We show that on large datasets the deep learning model significantly outperforms other methods: The mean absolute error could be reduced by 31 % on the new dataset PPG-DaLiA, and by 21 % on the dataset WESAD.

Wearable computing has a huge potential to shape the way we interact with mobile devices in the future. Interaction with mobile devices is still mainly limited to visual output and tactile finger-based input. Despite the visions of next-generation mobile interaction, the hand-held form factor hinders new interaction techniques becoming commonplace. In contrast, wearable devices and sensors are intended for more continuous and close-to-body use. This makes it possible to design novel wearable-augmented mobile interaction methods – both explicit and implicit. For example, the EEG signal from a wearable breast strap could be used to identify user status and change the device state accordingly (implicit) and the optical tracking with a head-mounted camera could be used to recognize gestural input (explicit). In this paper, the authors outline the design space for how the existing and envisioned wearable devices and sensors could augment mobile interaction techniques. Based on designs and discussions in a recently organized workshop on the topic as well as other related work, the authors present an overview of this design space and highlight some use cases that underline the potential therein.

Background Hand tremor typically has a negative impact on a person’s ability to complete many common daily activities. Previous research has investigated how to quantify hand tremor with smartphones and wearable sensors, mainly under controlled data collection conditions. Solutions for daily real-life settings remain largely underexplored. Objective Our objective was to monitor and assess hand tremor severity in patients with Parkinson disease (PD), and to better understand the effects of PD medications in a naturalistic environment. Methods Using the Welch method, we generated periodograms of accelerometer data and computed signal features to compare patients with varying degrees of PD symptoms. Results We introduced and empirically evaluated the tremor intensity parameter (TIP), an accelerometer-based metric to quantify hand tremor severity in PD using smartphones. There was a statistically significant correlation between the TIP and self-assessed Unified Parkinson Disease Rating Scale (UPDRS) II tremor scores (Kendall rank correlation test: z=30.521, P Conclusions Our work demonstrates the potential use of smartphone inertial sensors as a systematic symptom severity assessment mechanism to monitor PD symptoms and to assess medication effectiveness remotely. Our smartphone-based monitoring app may also be relevant for other conditions where hand tremor is a prevalent symptom.

PPG-based continuous heart rate estimation is challenging due to the effects of physical activity. Recently, methods based on time-frequency spectra emerged to compensate motion artefacts. However, existing approaches are highly parametrised and optimised for specific scenarios. In this paper, we first argue that cross-validation schemes should be adapted to this topic, and show that the generalisation capabilities of current approaches are limited. We then introduce deep learning, specifically CNN-models, to this domain. We investigate different CNN-architectures (e.g. the number of convolutional layers, applying batch normalisation, or ensemble prediction), and report insights based on our systematic evaluation on two publicly available datasets. Finally, we show that our CNN-based approach performs comparably to classical methods.

Affect recognition aims to detect a person's affective state based on observables, with the goal to e.g. improve human-computer interaction. Long-term stress is known to have severe implications on wellbeing, which call for continuous and automated stress monitoring systems. However, the affective computing community lacks commonly used standard datasets for wearable stress detection which a) provide multimodal high-quality data, and b) include multiple affective states. Therefore, we introduce WESAD, a new publicly available dataset for wearable stress and affect detection. This multimodal dataset features physiological and motion data, recorded from both a wrist- and a chest-worn device, of 15 subjects during a lab study. The following sensor modalities are included: blood volume pulse, electrocardiogram, electrodermal activity, electromyogram, respiration, body temperature, and three-axis acceleration. Moreover, the dataset bridges the gap between previous lab studies on stress and emotions, by containing three different affective states (neutral, stress, amusement). In addition, self-reports of the subjects, which were obtained using several established questionnaires, are contained in the dataset. Furthermore, a benchmark is created on the dataset, using well-known features and standard machine learning methods. Considering the three-class classification problem ( baseline vs. stress vs. amusement ), we achieved classification accuracies of up to 80%,. In the binary case ( stress vs. non-stress ), accuracies of up to 93%, were reached. Finally, we provide a detailed analysis and comparison of the two device locations ( chest vs. wrist ) as well as the different sensor modalities.

Depth cameras have been known to be capable of picking up the small changes in distance from users' torsos, to estimate respiration rate. Several studies have shown that under certain conditions, the respiration rate from a non-mobile user facing the camera can be accurately estimated from parts of the depth data. It is however to date not clear, what factors might hinder the application of this technology in any setting, what areas of the torso need to be observed, and how readings are affected for persons at larger distances from the RGB-D camera. In this paper, we present a benchmark dataset that consists of the point cloud data from a depth camera, which monitors 7 volunteers at variable distances, for variable methods to pin-point the person's torso, and at variable breathing rates. Our findings show that the respiration signal's signal-to-noise ratio becomes debilitating as the distance to the person approaches 4 metres, and that bigger windows over the person's chest work particularly well. The sampling rate of the depth camera was also found to impact the signal's quality significantly.

Today's demands regarding workpiece quality in cutting machine tool processing require automated monitoring of both machine condition and the cutting process. Currently, best-performing monitoring approaches rely on high-frequency acoustic emission (AE) sensor data and definition of advanced features, which involve complex computations. This approach is challenging for machine monitoring via embedded sensor systems with constrained computational power and energy budget. To cope with constrained energy, we rely on data recording with microelectromechanical system (MEMS) vibration sensors, which rely on lower-frequency sampling. To clarify whether these lower-frequency signals bear information for typical machine monitoring prediction tasks, we evaluate data for the most generic machine monitoring task of tool condition monitoring (TCM). To cope with computational complexity of advanced features, we introduce two intelligent preprocessing algorithms. First, we split non-stationary signals of recurrent structure into similar segments. Then, we identify most discriminative spectral differences in the segmented signals that allow for best separation of classes for the given TCM task. Subsequent feature extraction only in most relevant signal segments and spectral regions enables high expressiveness even for simple features. Extensive evaluation of the outlined approach on multiple data sets of different combinations of cutting machine tools, tool types and workpieces confirms its sensibility. Intelligent preprocessing enables reliable identification of stationary segments and most discriminative frequency bands. With subsequent extraction of simple but tailor-made features in these spectral-temporal regions of interest (Rols), TCM typically framed as multi feature classification problem can be converted to a single feature threshold comparison problem with an average F1 score of 97.89%.

This article focuses on the use of data gloves for human-computer interaction concepts, where external sensors cannot always fully observe the user’s hand. A good concept hereby allows to intuitively switch the interaction context on demand by using different hand gestures. The recognition of various, possibly complex hand gestures, however, introduces unintentional overhead to the system. Consequently, we present a data glove prototype comprising a glove-embedded gesture classifier utilizing data from Inertial Measurement Units (IMUs) in the fingertips. In an extensive set of experiments with 57 participants, our system was tested with 22 hand gestures, all taken from the French Sign Language (LSF) alphabet. Results show that our system is capable of detecting the LSF alphabet with a mean accuracy score of 92% and an F1 score of 91%, using complementary filter with a gyroscope-to-accelerometer ratio of 93%. Our approach has also been compared to the local fusion algorithm on an IMU motion sensor, showing faster settling times and less delays after gesture changes. Real-time performance of the recognition is shown to occur within 63 milliseconds, allowing fluent use of the gestures via Bluetooth-connected systems.

Capturing and digitizing all nuances during presentations is notoriously difficult. At best, digital slides tend to be combined with audio, while video footage of the presenter’s body language often turns out to be either too sensitive, occluded, or hard to achieve for common lighting conditions. If presentations require capturing what is written on the whiteboard, more expensive setups are usually needed. In this paper, we present an approach that complements the data from a wrist-worn inertial sensor with depth camera footage, to obtain an accurate posture representation of the presenter. A wearable inertial measurement unit complements the depth footage by providing more accurate arm rotations and wrist postures when the depth images are occluded, whereas the depth images provide an accurate full-body posture for indoor environments. In an experiment with 10 volunteers, we show that posture estimates from depth images and inertial sensors complement each other well, resulting in far less occlusions and tracking of the wrist with an accuracy that supports capturing sketches.

Authoring protocols for manual tasks such as following recipes, manufacturing processes or laboratory experiments requires significant effort. This paper presents a system that estimates individual procedure transitions from the user’s physical movement and gestures recorded with inertial motion sensors. Combined with egocentric or external video recordings, this facilitates efficient review and annotation of video databases. We investigate different clustering algorithms on wearable inertial sensor data recorded on par with video data, to automatically create transition marks between task steps. The goal is to match these marks to the transitions given in a description of the workflow, thus creating navigation cues to browse video repositories of manual work. To evaluate the performance of unsupervised algorithms, the automatically-generated marks are compared to human expert-created labels on two publicly-available datasets. Additionally, we tested the approach on a novel dataset in a manufacturing lab environment, describing an existing sequential manufacturing process. The results from selected clustering methods are also compared to some supervised methods.

An ever-growing range of wireless sensors for medical monitoring has shown that there is significant interest in monitoring patients in their everyday surroundings. It however remains a challenge to merge information from several wireless sensors and applications are commonly built from scratch. This paper presents a middleware targeted for medical applications on smartphone-like platforms that relies on an event-based design to enable flexible coupling with changing sets of wireless sensor units, while posing only a minor overhead on the resources and battery capacity of the interconnected devices. We illustrate the requirements for such middleware with three different healthcare applications that were deployed with our middleware solution, and characterize the performance with energy consumption, overhead caused for the smartphone, and processing time under real-world circumstances. Results show that with sensing-intensive applications, our solution only minimally impacts the phone's resources, with an added CPU utilization of 3% and a memory usage under 7 MB. Furthermore, for a minimum message delivery ratio of 99.9%, up to 12 sensor readings per second are guaranteed to be handled, regardless of the number of applications using our middleware.

Estimating human pose can be done with a variety of technologies. Models are typically derived from a motion capturing system that aims at generating a skeleton model as accurate and fast as possible. To date, a large variety of solutions is available to obtain capture data, often either from the environment, such as from depth cameras, or person-based, such as by body-worn inertial sensors. There are, however, few frameworks that are able to combine different types of human motion data into one environment. To this end, we present a modular and open architecture that allows specifically to integrate different capture sources and different filters to capture, process, and fuse these, in real time, into a human posture model. We show how our proposed solution leads to a fast and flexible way to deal with multiple motion capture modalities, and how such a framework can be used for editing and combining postures.

Interpreting datasets containing inertial data (acceleration, rate-of-turn, magnetic flux) requires a description of the datasets itself. Often this description is unstructured, stored as a convention or simply not available anymore. In this note, we argue that each modality exhibits particular statistical properties, which allows to reconstruct it solely from the sensor's data. To investigate this, tri-axial inertial sensor data from five publicly available datasets were analysed. Three statistical properties: mode, kurtosis, and number of modes are shown to be sufficient for classification - assuming the sampling rate and sample format are known, and that both acceleration and magnetometer data is present. While those assumption hold, 98% of all 1003 data points were correctly classified.

Traditional intra-body communication approaches mostly rely on either fixed cable joints embedded in clothing, or on wireless radio transmission that tends to reach beyond the body. Situated between these approaches is body-coupled communication, a promising yet less-explored method that transmits information across the user's skin. We propose a novel body-coupled communication approach that simplifies the physical layer of data transmission via capacitive coupling between wearable systems with conductive fabrics: This layer provides a stable reference potential for the feedback path in proximity to the attached wearables on the human body, to cancel the erratic dependency on the environmental ground, and to increase the communications' reliability. Evaluation of our prototype shows significant increases in signal quality, due to reduced attenuation and noise. Requirements on hardware and, subsequently, energy consumption, cost, and implementation effort are reduced as well.

Users share a lot of personal information with friends, family members, and colleagues via social networks. Surprisingly, some users choose to share their sleeping patterns, perhaps both for awareness as well as a sense of connection to others. Indeed, sharing basic sleep data, whether a person has gone to bed or waking up, informs others about not just one's sleeping routines but also indicates physical state, and reflects a sense of wellness. We present Somnometer, a social alarm clock for mobile phones that helps users to capture and share their sleep patterns. While the sleep rating is obtained from explicit user input, the sleep duration is estimated based on monitoring a user's interactions with the app. Observing that many individuals currently utilize their mobile phone as an alarm clock revealed behavioral patterns that we were able to leverage when designing the app. We assess whether it is possible to reliably monitor one's sleep duration using such apps. We further investigate whether providing users with the ability to track their sleep behavior over a long time period can empower them to engage in healthier sleep habits. We hypothesize that sharing sleep information with social networks impacts awareness and connectedness among friends. The result from a controlled study reveals that it is feasible to monitor a user's sleep duration based just on her interactions with an alarm clock app on the mobile phone. The results from both an in-the-wild study and a controlled experiment suggest that providing a way for users to track their sleep behaviors increased user awareness of sleep patterns and induced healthier habits. However, we also found that, given the current broadcast nature of existing social networks, users were concerned with sharing their sleep patterns indiscriminately.

Public health trends are currently monitored and diagnosed based on large studies that often rely on pen-and-paper data methods that tend to require a large collection campaign. With the pervasiveness of smart-phones and -watches throughout the general population, we argue in this paper that such devices and their built-in sensors can be used to capture such data more accurately with less of an effort. We present a system that targets a pan-European and harmonised architecture, using smartphones and wrist-worn activity loggers to enable the collection of data to estimate sedentary behavior and physical activity, plus the consumption of sugar-sweetened beverages. We report on a unified pilot study across three countries and four cities (with different languages, locale formats, and data security and privacy laws) in which 83 volunteers were asked to log beverages consumption along with a series of surveys and longitudinal accelerometer data. Our system is evaluated in terms of compliance, obtained data, and first analyses.

Robot systems largely depend on embedded systems to operate. The interfaces of those embedded systems, e.g. motor controllers or laser scanners, are often vendor-specific and therefore require a component that translates from/to the Robot Operating System (ROS) Middleware interface. In this work we present an implementation and evaluation of a ROS Middleware client based on the Contiki operating systems, which is suitable for constrained embedded devices, like wireless sensor nodes. We show that in-buffer processing of ROS messages without relying on dynamic memory allocation is possible. That message contents can be accessed conveniently via well-known concepts of the C language (structs) with negligible processing overhead compared to a C++-based client. And that the message-passing middle- ware concept of ROS fits nicely in Contiki's event-based nature. Furthermore, in order for an environment enriched with wireless sensor network to help robots in navigating, understanding and manipulating environments a direct integration is mandatory.

Deriving more detailed insights into one's ecological footprint is a premise to reduce one's individual environmental impact. Personal water consumption contributes significantly to this impact, but remains hard to quantify individually unless digital meters are installed. In this paper, we present a dual-sensing approach to retro-fit common water meters with a wireless sensor unit that is able to capture an individual's water usage, and digitally forward it over the home's WiFi network. Utilizing active infrared distance sensing or sensing magnetic flux, it is possible to measure water consumption with an accuracy below 0.1l on commonly installed meters. With a continuous power consumption (assuming a daily water consumption of 2 hours) of less than 20 mW, the system can be provide real-time feedback to home-owners, office workers and people sharing such a retro-fitted water supply.

Current motion sensors in wearable devices are primarily used for simple orientation and motion sensing. They provide however signals related to more complex and subtle human behaviours which will enable next-generation human-oriented computing in scenarios of high societal value. This requires large scale human activity corpuses and improved methods to recognise activities and their context. This workshop deals with the challenges of designing reproducible experimental setups, running large-scale dataset collection campaigns, designing robust activity and context recognition methods and evaluating systems in the real world. As a special topic, we wish to reflect on the challenges and approaches to recognise activities outside of a pre-defined set to achieve an open-ended activity and context awareness. Following the success of previous years, this workshop is the place to share experiences on human activity corpus and their applications and to discuss the future of activity sensing, in particular towards open-ended contextual intelligence.

Exchanging data as comma-separated values (CSV) is slow, cumbersome and error-prone. Especially for time-series data, which is common in Activity Recognition, synchronizing several independently recorded sensors is challenging. Adding second level evidence, like video recordings from multiple angles and time-coded annotations, further complicates the matter of curating such data. A possible alternative is to make use of standardized multi-media formats. Sensor data can be encoded in audio format, and time-coded information, like annotations, as sub-titles. Video data can be added easily. All this media can be merged into a single container file, which makes the issue of synchronization explicit. The incurred performance overhead by this encoding is shown to be negligible and compression can be applied to optimize storage and transmission overhead.

Detecting the task at hand can often be improved when it is also known what object the user is holding. Several sensing modalities have been suggested to identify handheld objects, from wrist-worn RFID readers to cameras. A critical obstacle to using such sensors, however, is that they tend to be too power hungry for continuous usage. This paper proposes a system that detects grasping using first inertial sensors and then Electromyography (EMG) on the forearm, to then selectively activate the object identification sensors. This three-tiered approach would therefore only attempt to identify in-hand objects once it is known a grasping has occurred. Our experiments show that high recall can be obtained for grasp detection, 95% on average across participants, with the grasping of lighter and smaller objects clearly being more difficult.

In the participatory sensing model, humans may serve as opportunistic sensors and flexible actuators while also consuming sensing services. Integrating humans into sensing systems has the potential to increase scale and reduce costs. However, contemporary participatory sensing software provides poor consideration of user dynamism, which includes: mobility across networks, mobility across devices and context-awareness. To address these limitations we propose the User Component and User Bindings. The former represents the user as a first class reconfigurable element of evolving and shared participatory sensing platforms. The latter allows the middleware to support multiple communications channels including Online Social Networks (OSN) to connect users with sensing applications. Our approach increases user participation, reduces out-of-context interactions and only consumes a limited amount of energy by sharing context information between applications. We support these claims by evaluating our approach on a two weeks experiment in which three participants take part in three concurrent participatory applications.

In the last decades, mobile phones have turned into sensor-rich devices that use different built-in sensors such as accelerometers or gyroscopes. The sensors have enriched the interaction possibilities, allowing, for example, gestural interaction. With the prevalence of wearable devices and peripherals, such as fitness bracelets and breast straps, the input and output possibilities can be further extended with both new sensors and actuators. Current applications could benefit from them, and entirely new applications could be designed. The design space for new applications needs to be identified, which will again drive advances in mobile and wearable computing. This workshop sets focus on wearable devices as means to enrich smartphones and their interaction capabilities. We will discuss the new design space and generate ideas of new applications. Furthermore, we will provide sensors and actuators allowing the participants to implement rapid prototypes of their novel application ideas.

In this paper, we will provide a typology of sensor-based activity recognition and interaction, which we call wearable activity recognition. The typology will focus on a conceptual level regarding the relation between persons and computing systems. Two paradigms, first the activity based seamless and obtrusive interaction and second activity-tracking for reflection, are seen as predominant. The conceptual approach will lead to the key term of this technology research, which is currently underexposed in a wider and conceptual understanding: human action/activity. Modeling human action as a topic for human-computer interaction (HCI) in general exists since its beginning. We will apply two classic theories which are influential in the HCI research to the application of wearable activity recognition. It is both a survey and a critical reflection on these concepts. As a further goal of our approach, we argue for the relevance and the benefits this typology can have. Beside practical consequences, a typology of the human-computer relation and the discussion of the key term activity can be a medium for exchange which other disciplines. Especially when applications become more serious, for example in health care, a typology including a wider mutual understanding can be useful for cooperations with non-technical practitioners e.g. doctors or psychologists.

Experimentation in Wet Laboratories requires tracking and identification of small containers like test tubes, flasks, and bottles. The current practise involves colored adhesive markers, waterproof hand-writing, QR- and Barcodes, or RFID-Tags. These markers are often not self-descriptive and require a lookup table on paper or some digitally stored counterpart. Furthermore they are subject to harsh environmental conditions (e.g. samples are kept in a freezer), and can be hard to share with other lab workers for lack of a consistent annotation systems. Increasing their durability, as well as providing a central tracking system for these containers, is therefore of great interest. In this paper we present a system for the implicit tracking of RFID-augmented containers with a wrist-worn reader unit, and a voice-interaction scheme based on a head-mounted display.