Search

Recent studies in Human Activity Recognition (HAR) have shown that Deep Learning methods are able to outperform classical Machine Learning algorithms. One popular Deep Learning architecture in HAR is the DeepConvLSTM. In this paper we propose to alter the DeepConvLSTM architecture to employ a 1-layered instead of a 2-layered LSTM. We validate our architecture change on 5 publicly available HAR datasets by comparing the predictive performance with and without the change employing varying hidden units within the LSTM layer(s). Results show that across all datasets, our architecture consistently improves on the original one: Recognition performance increases up to 11.7% for the F1-score, and our architecture significantly decreases the amount of learnable parameters. This improvement over DeepConvLSTM decreases training time by as much as 48%. Our results stand in contrast to the belief that one needs at least a 2-layered LSTM when dealing with sequential data. Based on our results we argue that said claim might not be applicable to sensor-based HAR., 6 pages, 2 figures, accepted at ISWC 21: International Symposium on Wearable Computer, Sept, 2021

Federated learning is a knowledge transmission and training process that occurs in turn between user models on edge devices and the training model in the central server. Due to privacy policies and concerns and heterogeneous data, this is a widespread requirement in federated learning applications. In this work, we use knowledge-based methods, and in particular case-based reasoning (CBR), to develop a wearable, explainable artificial intelligence (xAI) framework. CBR is a problem-solving AI approach for knowledge representation and manipulation, which considers successful solutions of past conditions that are likely to serve as candidate solutions for a requested problem. It enables federated learning when each user owns not only his/her private data, but also uniquely designed cases. New generated cases can be compared to the knowledge base and the recommendations enable the user to communicate better with the whole system. It improves users’ task performance and increases user acceptability when they need explanations to understand why and how AI algorithms arrive at these optimal solutions.

Today’s maintenance and renewal planning in transformer stations of energy distribution networks is mainly based on expert knowledge, experience gained from historical data as well as the knowledge gathered from regular on-site inspections. This approach is already reaching its limits due to insufficient databases and almost no information about the stations’ condition being gathered between inspection intervals. A condition-based strategy that requires more maintenance for equipment with a high probability of failure is needed. Great potential is promised by intelligent sensor-based diagnostics, where objective comparability can be achieved by condition monitoring of the station fleet. Cost-effective micro-electromechanical (MEMS)-bases sensor systems promise to provide a suitable solution for network operators and enable a widespread use. In our paper, we present a MEMS-based sensor system, that can be used to gain information about network transparency, station safety as well as maintenance and renewal planning. Moreover, we propose an intelligent measurement scheme which adaptively selects relevant data and avoids unneeded redundancy (Smart Data instead of Big Data).

BACKGROUND Video electroencephalography recordings, routinely used in epilepsy monitoring units, are the gold standard for monitoring epileptic seizures. However, monitoring is also needed in the day-to-day lives of people with epilepsy, where video electroencephalography is not feasible. Wearables could fill this gap by providing patients with an accurate log of their seizures. OBJECTIVE Although there are already systems available that provide promising results for the detection of tonic-clonic seizures (TCSs), research in this area is often limited to detection from 1 biosignal modality or only during the night when the patient is in bed. The aim of this study is to provide evidence that supervised machine learning can detect TCSs from multimodal data in a new data set during daytime and nighttime. METHODS An extensive data set of biosignals from a multimodal watch worn by people with epilepsy was recorded during their stay in the epilepsy monitoring unit at 2 European clinical sites. From a larger data set of 243 enrolled participants, those who had data recorded during TCSs were selected, amounting to 10 participants with 21 TCSs. Accelerometry and electrodermal activity recorded by the wearable device were used for analysis, and seizure manifestation was annotated in detail by clinical experts. Ten accelerometry and 3 electrodermal activity features were calculated for sliding windows of variable size across the data. A gradient tree boosting algorithm was used for seizure detection, and the optimal parameter combination was determined in a leave-one-participant-out cross-validation on a training set of 10 seizures from 8 participants. The model was then evaluated on an out-of-sample test set of 11 seizures from the remaining 2 participants. To assess specificity, we additionally analyzed data from up to 29 participants without TCSs during the model evaluation. RESULTS In the leave-one-participant-out cross-validation, the model optimized for sensitivity could detect all 10 seizures with a false alarm rate of 0.46 per day in 17.3 days of data. In a test set of 11 out-of-sample TCSs, amounting to 8.3 days of data, the model could detect 10 seizures and produced no false positives. Increasing the test set to include data from 28 more participants without additional TCSs resulted in a false alarm rate of 0.19 per day in 78 days of wearable data. CONCLUSIONS We show that a gradient tree boosting machine can robustly detect TCSs from multimodal wearable data in an original data set and that even with very limited training data, supervised machine learning can achieve a high sensitivity and low false-positive rate. This methodology may offer a promising way to approach wearable-based nonconvulsive seizure detection.

BACKGROUND Hand tremor typically has a negative impact on a person’s ability to complete many common daily activities. Previous research has investigated how to quantify hand tremor with smartphones and wearable sensors, mainly under controlled data collection conditions. Solutions for daily real-life settings remain largely underexplored. OBJECTIVE Our objective was to monitor and assess hand tremor severity in patients with Parkinson disease (PD), and to better understand the effects of PD medications in a naturalistic environment. METHODS Using the Welch method, we generated periodograms of accelerometer data and computed signal features to compare patients with varying degrees of PD symptoms. RESULTS We introduced and empirically evaluated the tremor intensity parameter (TIP), an accelerometer-based metric to quantify hand tremor severity in PD using smartphones. There was a statistically significant correlation between the TIP and self-assessed Unified Parkinson Disease Rating Scale (UPDRS) II tremor scores (Kendall rank correlation test: z=30.521, PP CONCLUSIONS Our work demonstrates the potential use of smartphone inertial sensors as a systematic symptom severity assessment mechanism to monitor PD symptoms and to assess medication effectiveness remotely. Our smartphone-based monitoring app may also be relevant for other conditions where hand tremor is a prevalent symptom.

This proceedings presents the papers from Urb-IoT 2018 - 3rd EAI International Conference on IoT in Urban Space, which took place in Guimarães, Portugal on 21-22 November 2018. The conference aims to explore the emerging dynamics within the scope of the Internet of Things (IoT) and the new science of cities.The papers discuss fusion of heterogeneous urban sources, understanding urban data using machine learning and mining techniques, urban analytics, urban IoT infrastructures, crowd sourcing techniques, incentification and gamification, urban mobility and intelligent transportation systems, real time urban information systems, and more. The proceedings discuss innovative technologies that navigate industry and connectivity sectors in transportation, utility, public safety, healthcare, and education. The authors also discuss the increasing deployments of IoT technologies and the rise of the so-called 'Sensored Cities'‚ which are opening up new avenues of research opportunities towards that future.

Affect recognition is an interdisciplinary research field bringing together researchers from natural and social sciences. Affect recognition research aims to detect the affective state of a person based on observables, with the goal to, for example, provide reasoning for the person’s decision making or to support mental wellbeing (e.g., stress monitoring). Recently, beside of approaches based on audio, visual or text information, solutions relying on wearable sensors as observables, recording mainly physiological and inertial parameters, have received increasing attention. Wearable systems enable an ideal platform for long-term affect recognition applications due to their rich functionality and form factor, while providing valuable insights during everyday life through integrated sensors. However, existing literature surveys lack a comprehensive overview of state-of-the-art research in wearable-based affect recognition. Therefore, the aim of this paper is to provide a broad overview and in-depth understanding of the theoretical background, methods and best practices of wearable affect and stress recognition. Following a summary of different psychological models, we detail the influence of affective states on the human physiology and the sensors commonly employed to measure physiological changes. Then, we outline lab protocols eliciting affective states and provide guidelines for ground truth generation in field studies. We also describe the standard data processing chain and review common approaches related to the preprocessing, feature extraction and classification steps. By providing a comprehensive summary of the state-of-the-art and guidelines to various aspects, we would like to enable other researchers in the field to conduct and evaluate user studies and develop wearable systems.

Photoplethysmography (PPG)-based continuous heart rate monitoring is essential in a number of domains, e.g., for healthcare or fitness applications. Recently, methods based on time-frequency spectra emerged to address the challenges of motion artefact compensation. However, existing approaches are highly parametrised and optimised for specific scenarios of small, public datasets. We address this fragmentation by contributing research into the robustness and generalisation capabilities of PPG-based heart rate estimation approaches. First, we introduce a novel large-scale dataset (called PPG-DaLiA), including a wide range of activities performed under close to real-life conditions. Second, we extend a state-of-the-art algorithm, significantly improving its performance on several datasets. Third, we introduce deep learning to this domain, and investigate various convolutional neural network architectures. Our end-to-end learning approach takes the time-frequency spectra of synchronised PPG- and accelerometer-signals as input, and provides the estimated heart rate as output. Finally, we compare the novel deep learning approach to classical methods, performing evaluation on four public datasets. We show that on large datasets the deep learning model significantly outperforms other methods: The mean absolute error could be reduced by 31 % on the new dataset PPG-DaLiA, and by 21 % on the dataset WESAD.

Background Hand tremor typically has a negative impact on a person’s ability to complete many common daily activities. Previous research has investigated how to quantify hand tremor with smartphones and wearable sensors, mainly under controlled data collection conditions. Solutions for daily real-life settings remain largely underexplored. Objective Our objective was to monitor and assess hand tremor severity in patients with Parkinson disease (PD), and to better understand the effects of PD medications in a naturalistic environment. Methods Using the Welch method, we generated periodograms of accelerometer data and computed signal features to compare patients with varying degrees of PD symptoms. Results We introduced and empirically evaluated the tremor intensity parameter (TIP), an accelerometer-based metric to quantify hand tremor severity in PD using smartphones. There was a statistically significant correlation between the TIP and self-assessed Unified Parkinson Disease Rating Scale (UPDRS) II tremor scores (Kendall rank correlation test: z=30.521, P Conclusions Our work demonstrates the potential use of smartphone inertial sensors as a systematic symptom severity assessment mechanism to monitor PD symptoms and to assess medication effectiveness remotely. Our smartphone-based monitoring app may also be relevant for other conditions where hand tremor is a prevalent symptom.

Every benchmark dataset that contains inertial data (acceleration, rate-of-turn, magnetic flux) requires a thorough description of the datasets itself. This description tends often to be unstructured, and supplied to researchers as a conventional description, and in many cases crucial details are not available anymore. In this chapter, we argue that each sensor modality exhibits particular statistical properties that allow to reconstruct the modality solely from the sensor data itself. In order to investigate this, tri-axial inertial sensor data from five publicly available datasets are analysed. We found that in particular three statistical properties, the mode, the kurtosis, and the number of modes tend to be sufficient for classification of sensor modality—requiring as the only assumption that the sampling rate and sample format are known, and the fact that that acceleration and magnetometer data is present in the dataset. With those assumption in place, we found that \(98\%\) of all 1003 data points were successfully classified.

In this paper, we introduce a research project on investigating the relation of computers and humans in the field of wearable activity recognition. We use an interdisciplinary approach, combining general philosophical assumptions on the mediating character of technology with the current computer science design practice. Wearable activity recognition is about computer systems which automatically detect human actions. Of special relevance for our research project are applications using wearable activity recognition for self-tracking and self-reflection, for instance by tracking personal activity data like sports. We assume that activity recognition is providing a new perspective on human actions, this perspective is mediated by the recognition process, which includes the recognition models and algorithms chosen by the designer, and the visualization to the user. We analyze this mediating character with two concepts which are both based on phenomenological thoughts namely first Peter-Paul Verbeek’s theory on human-technology relations and second the ideas of embodied interaction. Embedded in the concepts is a direction which leads to the role of technical design in how technology mediates. Regarding this direction, we discuss two case studies, both in the possible using practice of self-tracking and the design practice. This paper ends with prospects towards a better design, how the technologies should be designed to support self-reflection in a valuable and responsible way.

Authoring protocols for manual tasks such as following recipes, manufacturing processes or laboratory experiments requires significant effort. This paper presents a system that estimates individual procedure transitions from the user’s physical movement and gestures recorded with inertial motion sensors. Combined with egocentric or external video recordings, this facilitates efficient review and annotation of video databases. We investigate different clustering algorithms on wearable inertial sensor data recorded on par with video data, to automatically create transition marks between task steps. The goal is to match these marks to the transitions given in a description of the workflow, thus creating navigation cues to browse video repositories of manual work. To evaluate the performance of unsupervised algorithms, the automatically-generated marks are compared to human expert-created labels on two publicly-available datasets. Additionally, we tested the approach on a novel dataset in a manufacturing lab environment, describing an existing sequential manufacturing process. The results from selected clustering methods are also compared to some supervised methods.

Wearable computing has a huge potential to shape the way we interact with mobile devices in the future. Interaction with mobile devices is still mainly limited to visual output and tactile finger-based input. Despite the visions of next-generation mobile interaction, the hand-held form factor hinders new interaction techniques becoming commonplace. In contrast, wearable devices and sensors are intended for more continuous and close-to-body use. This makes it possible to design novel wearable-augmented mobile interaction methods – both explicit and implicit. For example, the EEG signal from a wearable breast strap could be used to identify user status and change the device state accordingly (implicit) and the optical tracking with a head-mounted camera could be used to recognize gestural input (explicit). In this paper, the authors outline the design space for how the existing and envisioned wearable devices and sensors could augment mobile interaction techniques. Based on designs and discussions in a recently organized workshop on the topic as well as other related work, the authors present an overview of this design space and highlight some use cases that underline the potential therein.

Deriving more detailed insights into one's ecological footprint is a premise to reduce one's individual environmental impact. Personal water consumption contributes significantly to this impact, but remains hard to quantify individually unless digital meters are installed. In this paper, we present a dual-sensing approach to retro-fit common water meters with a wireless sensor unit that is able to capture an individual's water usage, and digitally forward it over the home's WiFi network. Utilizing active infrared distance sensing or sensing magnetic flux, it is possible to measure water consumption with an accuracy below 0.1l on commonly installed meters. With a continuous power consumption (assuming a daily water consumption of 2 hours) of less than 20 mW, the system can be provide real-time feedback to home-owners, office workers and people sharing such a retro-fitted water supply.

Wet Laboratories are highly dynamic, shared environments full of tubes, racks, compounds, and dedicated machinery. The recording of experiments, despite the fact that several ubiquitous computing systems have been suggested in the past decades, still relies predominantly on hand-written notes. Similarly, the information retrieval capabilities inside a laboratory are limited to traditional computing interfaces, which due to safety regulations are sometimes not usable at all. In this paper, Google Glass is combined with a wrist-worn gesture sensor to support Wetlab experimenters. Taking "in-situ" documentation while an experiment is performed, as well as contextualizing the protocol at hand can be implemented on top of the proposed system. After an analysis of current practices and needs through a series of explorative deployments in wet labs, we motivate the need for a wearable hands-free system, and introduce our specific design to guide experimenters. Finally, using a study with 22 participants evaluating the system on a benchmark DNA extraction experiment, we explore the use of gesture recognition for enabling the system to track where the user might be in the experiment.

In the participatory sensing model, humans may serve as opportunistic sensors and flexible actuators while also consuming sensing services. Integrating humans into sensing systems has the potential to increase scale and reduce costs. However, contemporary participatory sensing software provides poor consideration of user dynamism, which includes: mobility across networks, mobility across devices and context-awareness. To address these limitations we propose the User Component and User Bindings. The former represents the user as a first class reconfigurable element of evolving and shared participatory sensing platforms. The latter allows the middleware to support multiple communications channels including Online Social Networks (OSN) to connect users with sensing applications. Our approach increases user participation, reduces out-of-context interactions and only consumes a limited amount of energy by sharing context information between applications. We support these claims by evaluating our approach on a two weeks experiment in which three participants take part in three concurrent participatory applications.

In the last decades, mobile phones have turned into sensor-rich devices that use different built-in sensors such as accelerometers or gyroscopes. The sensors have enriched the interaction possibilities, allowing, for example, gestural interaction. With the prevalence of wearable devices and peripherals, such as fitness bracelets and breast straps, the input and output possibilities can be further extended with both new sensors and actuators. Current applications could benefit from them, and entirely new applications could be designed. The design space for new applications needs to be identified, which will again drive advances in mobile and wearable computing. This workshop sets focus on wearable devices as means to enrich smartphones and their interaction capabilities. We will discuss the new design space and generate ideas of new applications. Furthermore, we will provide sensors and actuators allowing the participants to implement rapid prototypes of their novel application ideas.

In this paper, we will provide a typology of sensor-based activity recognition and interaction, which we call wearable activity recognition. The typology will focus on a conceptual level regarding the relation between persons and computing systems. Two paradigms, first the activity based seamless and obtrusive interaction and second activity-tracking for reflection, are seen as predominant. The conceptual approach will lead to the key term of this technology research, which is currently underexposed in a wider and conceptual understanding: human action/activity. Modeling human action as a topic for human-computer interaction (HCI) in general exists since its beginning. We will apply two classic theories which are influential in the HCI research to the application of wearable activity recognition. It is both a survey and a critical reflection on these concepts. As a further goal of our approach, we argue for the relevance and the benefits this typology can have. Beside practical consequences, a typology of the human-computer relation and the discussion of the key term activity can be a medium for exchange which other disciplines. Especially when applications become more serious, for example in health care, a typology including a wider mutual understanding can be useful for cooperations with non-technical practitioners e.g. doctors or psychologists.

A key task in Wireless Sensor Networks (WSNs) is to deliver specific information about a spatial phenomenon of interest. However, in WSNs the operating conditions and/or user requirements are often desired to be evolvable, whether driven by changes of the monitored parameters or WSN properties. To this end, few sensor nodes sample the phenomenon and transmit the acquired samples, typically multihop, to the application through a gateway called sink. Accurately representing the physical phenomenon and reliably, timely delivering the user required information comes at the cost of higher energy as additional messages are required. This work proposes a tunable co-design for network optimization to avoid under or over provision of information and interaction of the attributes and their effects on each other. We validate the approach viability through analytical modeling, simulations for a range of requirements.

The advent of a range of wearable products for monitoring one's healthcare and fitness has pushed decades of research into the market over the past years. These units record motion and detect common physical activities to assist the wearer in monitoring fitness, general state of health, and sleeping trends. Most of the detection algorithms on board of these devices however are closed-source and the devices do not allow the recording of raw inertial data. This paper presents a project that, faced by these limitations of commercial wearable products, set out to create an open-source recording platform for activity recognition research that (1) is sufficiently power-efficient, and (2) remains small and comfortable enough to wear, to be able to record raw inertial data for extended periods of time. We study especially, via high-resolution power profiling, several trade-offs present in the choice for the basic hardware components of our prototype, and contribute with three key design areas that have had a significant impact on our prototype design.

The experience sampling methodology is a well known tool in psychology to asses a subject's condition. Regularly or whenever an important event happens the subject stops whatever he is currently involved in and jots down his current perceptions, experience, and activities, which in turn form the basis of these diary studies. Such methods are also widely in use for gathering labelled data for wearable long-term activity recognition, where subjects are asked to note conducted activities. We present the design of a personal electronic diary for daily activities, including user interfaces on a PC, Smartphone, and Google Glass. A 23-participant structured in-field study covering seven different activities highlights the difference of mobile touch interaction and ubiquitous voice recognition for tracking activities.

Head mounted displays (HMDs) can be used as an guidance system for manual assembling tasks: contrary to using a Tablet-PC, instructions are always shown in the field of view while hands are kept free for the task. This is believed to be one of the major advantage of using HMDs. In the study reported here, performance, visual fatigue, and subjective strain was measured in a dual task paradigm. Participants were asked to follow a toy car assembly instructions while monitoring a virtual gauge. Both tasks had to be executed in parallel either while wearing Google Glass or using a Tablet-PC. Results show slower performance on the HMD but no difference in subjective strain.

The Robot Operating System (ROS) is a popular middleware that eases the design, implementation, and maintenance of robot systems. In particular, ROS enables the integration of a large number of heterogeneous devices in a single system. To allow these devices to communicate and cooperate, ROS requires device-specific interfaces to be available. This restricts the number of devices that can effectively be integrated in a ROS-based system. In this work we present the design, implementation, and evaluation of a ROS middleware client that allows to integrate constrained devices like wireless sensor nodes in a ROS-based system. Wireless sensor nodes embedded in the environment in which a robot system is operating can indeed help robots in navigating and interacting with the environment. The client has been implemented for devices running the Contiki operating system but its design can be readily extended to other systems like, e.g., TinyOS. Our evaluation shows that: in-buffer processing of ROS messages without relying on dynamic memory allocation is possible; message contents can be accessed conveniently using well-known concepts of the C language (structs) with negligible processing overhead with respect to a C++-based client; and that ROS’ message-passing abstraction facilitates the integration of devices running event-based systems like Contiki.

The Seventh International IEEE Workshop on Practical Issues in Building Sensor Network Applications (SenseApp 2012) was held in Clearwater, Florida, USA, in conjunction with the 37th IEEE Conference on Local Computer Networks (LCN 2012).

The World Health Organization calls tobacco use the single most preventable cause of premature death, presenting both a personal health risk and an increased load on public healthcare systems. However, smoking cessation is often hindered by the low perceivability of health risks and unawareness of habits in day-to-day life, and effective smoking cessation systems, besides personal counseling, are still to be improved. This demo presents the design and implementation of two instrumented lighters that can be used to track a smokers' personal consumption habits. A Gas lighter and a USB lighter which have been outfitted with a micro-controller, storage unit and real-time clock. Both lighters store the day-of-time whenever they are used to light up a cigarette. This information can later be retrieved by the user for personal consumption statistics like most common time-of-day of consumption, total number of smoked cigarettes, daily consumed cigarettes etc. The presented prototypes allow the continous tracking of smoking behaviour over the course of several days.

This book constitutes the refereed proceedings of the AmI 2011 Workshops, held in Amsterdam, The Netherlands, in November 2011. The papers are organized in topical sections on aesthetic intelligence: designing smart and beautiful architectural spaces; ambient intelligence in future lighting systems; interactive human behavior analysis in open or public spaces; user interaction methods for elderly, people with dementia; empowering and integrating senior citizens with virtual coaching; integration of AMI and AAL platforms in the future internet (FI) platform initiative; ambient gaming; human behavior understanding: inducing behavioral change; privacy, trust and interaction in the internet of things; doctoral colloquium.

With activity sensors becoming smaller and more power efficient by the day, wearable activity sensors that anyone could wear just as easily as a wristwatch have become a feasible concept. We present a small lightweight module, called Porcupine, which aims explicitly at continuously monitoring human activities as long as possible, and as fine-grained as possible. The main focus in this work is not so much the hardware, which uses omni-present and relatively cheap accelerometer technology, but the algorithms that analyze the sensor data and predict what physical activity the wearer is performing. We present results from the latest experiments with our prototypes, and show some scenarios in which such a fine-grained actigraph can be put to use. We also discuss the important application of the porcupine technology in the clinical monitoring of patients with Bipolar disorder and other psychiatric disorders where activity monitoring is clinically important.

The integration of digital components into clothing is becoming an increasingly important segment in wearable computing research. The first indications for this trend are the incorporation of existing mobile technologies, such as personal digital assistants (PDAs) or mobile phones, into jackets via flexible textile circuits. In the long term, other components could also be envisioned that are embedded in apparel, using a flexible bus-type network that links all the devices together. This paper introduces a functioning prototype of such a flexible network that not only allows communication between wearable components, but is also able to supply power to them. We propose an arrangement of layered textiles as opposed to the more traditional routed circuitry layout, which results in a novel approach towards the concept of a flexible clothing network.

Current input technologies for wearable computers are difficult to use and learn and can be unreliable. Physical interfaces offer an alternative to traditional input methods. In this paper we propose that developing a well-designed physical interface requires an exploration of the psychological idea of affordance. We present our findings from a design study in which we explore the natural affordance of a cube and suggest possible requirements for the design of graspable cubeshaped physical interfaces as alternative rich-action input device. We expect that such a framework will enhance the precision and usability of devices for wearable and mobile computing.