Search

© 2010-2012 IEEE. While accurately predicting mood and wellbeing could have a number of important clinical benefits, traditional machine learning (ML) methods frequently yield low performance in this domain. We posit that this is because a one-size-fits-all machine learning model is inherently ill-suited to predicting outcomes like mood and stress, which vary greatly due to individual differences. Therefore, we employ Multitask Learning (MTL) techniques to train personalized ML models which are customized to the needs of each individual, but still leverage data from across the population. Three formulations of MTL are compared: i) MTL deep neural networks, which share several hidden layers but have final layers unique to each task; ii) Multi-task Multi-Kernel learning, which feeds information across tasks through kernel weights on feature types; and iii) a Hierarchical Bayesian model in which tasks share a common Dirichlet Process prior. We offer the code for this work in open source. These techniques are investigated in the context of predicting future mood, stress, and health using data collected from surveys, wearable sensors, smartphone logs, and the weather. Empirical results demonstrate that using MTL to account for individual differences provides large performance improvements over traditional machine learning methods and provides personalized, actionable insights.

© 2017 IEEE. To accomplish forecasting of mood in real-world situations, affective computing systems need to collect and learn from multimodal data collected over weeks or months of daily use. Such systems are likely to encounter frequent data loss, e.g. when a phone loses location access, or when a sensor is recharging. Lost data can handicap classifiers trained with all modalities present in the data. This paper describes a new technique for handling missing multimodal data using a specialized denoising autoencoder: The Multimodal Autoencoder (MMAE). Empirical results from over 200 participants and 5500 days of data demonstrate that the MMAE is able to predict the feature values from multiple missing modalities more accurately than reconstruction methods such as principal components analysis (PCA). We discuss several practical benefits of the MMAE's encoding and show that it can provide robust mood prediction even when up to three quarters of the data sources are lost., NIH (Grant R01GM105018)

© 2020 IEEE. We examine the problem of forecasting tomorrow morning's three self-reported levels (on scales from 0 to 100) of stressed-calm, sad-happy, and sick-healthy based on physiological data (skin conductance, skin temperature, and acceleration) from a sensor worn on the wrist from 10am-5pm today. We train automated forecasting regression algorithms using Random Forests and compare their performance over two sets of data: workers consisting of 490 days of weekday data from 39 employees at a high-tech company in Japan and students consisting of 3,841 days of weekday data from 201 New England USA college students. Mean absolute errors on held-out test data achieved 10.8, 13.5, and 14.4 for the estimated levels of mood, stress, and health respectively of office workers, and 17.8, 20.3, and 20.4 for the mood, stress, and health respectively of students. Overall the two groups reported comparable stress and mood scores, while employees reported slightly poorer health, and engaged in significantly lower levels of physical activity as measured by accelerometers. We further examine differences in population features and how systems trained on each population performed when tested on the other., National Institute of Health (Grant R01GM105018), National Science Foundation (Grant 1840167)

© ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2018. We designed, developed, and evaluated a novel system, QuantifyMe, for novice self-experimenters to conduct proper-methodology single-case self-experiments in an automated and scientific manner using their smartphones. In this work we evaluate its use with four different kinds of personalized investigations, examining how variables such as sleep duration and regularity, activity, and leisure time affect personal happiness, stress, productivity, and sleep efficiency. We describe lessons learned developing the system, and recommendations for its improvement, as well as its potential for enabling personalized insights to be scientifically evaluated in many individuals.

This exploratory study examined the effects of varying g-forces, including feelings of weightlessness, on an individual's physiology during parabolic flight. Specifically, we collected heart rate, accelerometer, and skin conductance measurements from 16 flyers aboard a parabolic flight using wearable, wireless sensors. The biosignals were then correlated to participant reports of nausea, anxiety, and excitement during periods of altered g-forces. Using linear mixed-effects models, we found that (1) heart rate was positively correlated to individuals' self-reported highest/lowest periods of both anxiety and excitement, and (2) bilateral skin conductance asymmetry was positively correlated to individuals' self-reported highest/lowest periods of nausea.

Life stress is a well-established risk factor for a variety of mental and physical health problems, including anxiety disorders, depression, chronic pain, heart disease, asthma, autoimmune diseases, and neurodegenerative disorders. The purpose of this article is to describe emerging approaches for assessing stress using speech, which we do by reviewing the methodological advantages of these digital health tools, and the validation, ethical, and privacy issues raised by these technologies. As we describe, it is now possible to assess stress via the speech signal using smartphones and smart speakers that employ software programs and artificial intelligence to analyze several features of speech and speech acoustics, including pitch, jitter, energy, rate, and length and number of pauses. Because these digital devices are ubiquitous, we can now assess individuals’ stress levels in real time in almost any natural environment in which people speak. These technologies thus have great potential for advancing digital health initiatives that involve continuously monitoring changes in psychosocial functioning and disease risk over time. However, speech-based indices of stress have yet to be well-validated against stress biomarkers (e.g., cortisol, cytokines) that predict disease risk. In addition, acquiring speech samples raises the possibility that conversations intended to be private could one day be made public; moreover, obtaining real-time psychosocial risk information prompts ethical questions regarding how these data should be used for medical, commercial, and personal purposes. Although assessing stress using speech thus has enormous potential, there are critical validation, privacy, and ethical issues that must be addressed. Keywords: Life stress; voice; speech; privacy; ethics; digital health, Brain and Behavior Research Foundation (Grant NARSAD 23958)

Smartphones and wearable sensors have enabled unprecedented data collection, with many products now providing feedback to users about recommended step counts or sleep durations. However, these recommendations do not provide personalized insights that have been shown to be best suited for a specific individual. A scientific way to find individualized recommendations and causal links is to conduct experi ments using single-case experimental design; however, properly designed single-case experiments are not easy to conduct on oneself. We designed, developed, and evaluated a novel platform, QuantifyMe, for novice self-experimenters to conduct proper-methodology single-case self-experiments in an automated and scientific manner using their smartphones. We provide software for the platform that we used (available for free on GitHub), which provides the methodological elements to run many kinds of customized studies. In this work, we evaluate its use with four different kinds of personalized investigations, examining how variables such as sleep duration and regularity, activity, and leisure time affect personal happiness, stress, productivity, and sleep efficiency. We conducted a six-week pilot study (N = 13) to evaluate QuantifyMe. We describe the lessons learned developing the platform and recommendations for its improvement, as well as its potential for enabling personalized insights to be scientifically evaluated in many individuals, reducing the high administrative cost for advancing human health and wellbeing. Keywords: single-case experimental design; mobile health; wearable sensors; self-experiment; self-tracking

Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2016., Cataloged from PDF version of thesis., Includes bibliographical references (pages 97-100)., Electrodermal activity (EDA) refers to the electrical activity measured on and under the surface of the skin and has been used to study sleep, stress, and mood. While gathering this signal was once confined to the laboratory, it can now be acquired in ambulatory studies through commercially available wearable sensors. In this thesis, we model and analyze electrodermal response (EDR) events (1-5 second peaks in the EDA signal) during sleep in an ambulatory study. In particular, we describe an EDR event detection algorithm and extract shape features from these events to discuss the difference in shape between sleep and wake. We also describe an automatic artifact detection algorithm that we use on over 100,000 hours of EDA data we have collected in the 30-day SNAPSHOT Study from 164 participants. Finally, we model the detected EDR events as a point process using a state-space generalized linear model. We identify a significant influence of recent EDR event history on current EDR event likelihood across different participants. We also use this model to analyze EDR event rates during different periods of the night., by Sara Ann Taylor., S.M.

Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2016., Cataloged from PDF version of thesis., Includes bibliographical references (pages 97-100)., Electrodermal activity (EDA) refers to the electrical activity measured on and under the surface of the skin and has been used to study sleep, stress, and mood. While gathering this signal was once confined to the laboratory, it can now be acquired in ambulatory studies through commercially available wearable sensors. In this thesis, we model and analyze electrodermal response (EDR) events (1-5 second peaks in the EDA signal) during sleep in an ambulatory study. In particular, we describe an EDR event detection algorithm and extract shape features from these events to discuss the difference in shape between sleep and wake. We also describe an automatic artifact detection algorithm that we use on over 100,000 hours of EDA data we have collected in the 30-day SNAPSHOT Study from 164 participants. Finally, we model the detected EDR events as a point process using a state-space generalized linear model. We identify a significant influence of recent EDR event history on current EDR event likelihood across different participants. We also use this model to analyze EDR event rates during different periods of the night., by Sara Ann Taylor., S.M.

Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2016., Cataloged from PDF version of thesis., Includes bibliographical references (pages 97-100)., Electrodermal activity (EDA) refers to the electrical activity measured on and under the surface of the skin and has been used to study sleep, stress, and mood. While gathering this signal was once confined to the laboratory, it can now be acquired in ambulatory studies through commercially available wearable sensors. In this thesis, we model and analyze electrodermal response (EDR) events (1-5 second peaks in the EDA signal) during sleep in an ambulatory study. In particular, we describe an EDR event detection algorithm and extract shape features from these events to discuss the difference in shape between sleep and wake. We also describe an automatic artifact detection algorithm that we use on over 100,000 hours of EDA data we have collected in the 30-day SNAPSHOT Study from 164 participants. Finally, we model the detected EDR events as a point process using a state-space generalized linear model. We identify a significant influence of recent EDR event history on current EDR event likelihood across different participants. We also use this model to analyze EDR event rates during different periods of the night., by Sara Ann Taylor., S.M.

Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2016., Cataloged from PDF version of thesis., Includes bibliographical references (pages 97-100)., Electrodermal activity (EDA) refers to the electrical activity measured on and under the surface of the skin and has been used to study sleep, stress, and mood. While gathering this signal was once confined to the laboratory, it can now be acquired in ambulatory studies through commercially available wearable sensors. In this thesis, we model and analyze electrodermal response (EDR) events (1-5 second peaks in the EDA signal) during sleep in an ambulatory study. In particular, we describe an EDR event detection algorithm and extract shape features from these events to discuss the difference in shape between sleep and wake. We also describe an automatic artifact detection algorithm that we use on over 100,000 hours of EDA data we have collected in the 30-day SNAPSHOT Study from 164 participants. Finally, we model the detected EDR events as a point process using a state-space generalized linear model. We identify a significant influence of recent EDR event history on current EDR event likelihood across different participants. We also use this model to analyze EDR event rates during different periods of the night., by Sara Ann Taylor., S.M.

Electrodermal activity (EDA) recording is a powerful, widely used tool for monitoring psychological or physiological arousal. However, analysis of EDA is hampered by its sensitivity to motion artifacts. We propose a method for removing motion artifacts from EDA, measured as skin conductance (SC), using a stationary wavelet transform (SWT). We modeled the wavelet coefficients as a Gaussian mixture distribution corresponding to the underlying skin conductance level (SCL) and skin conductance responses (SCRs). The goodness-of-fit of the model was validated on ambulatory SC data. We evaluated the proposed method in comparison with three previous approaches. Our method achieved a greater reduction of artifacts while retaining motion-artifact-free data.

In order to model students' happiness, we apply machine learning methods to data collected from undergrad students monitored over the course of one month each. The data collected include physiological signals, location, smartphone logs, and survey responses to behavioral questions. Each day, participants reported their wellbeing on measures including stress, health, and happiness. Because of the relationship between happiness and depression, modeling happiness may help us to detect individuals who are at risk of depression and guide interventions to help them. We are also interested in how behavioral factors (such as sleep and social activity) affect happiness positively and negatively. A variety of machine learning and feature selection techniques are compared, including Gaussian Mixture Models and ensemble classification. We achieve 70% classification accuracy of self-reported happiness on held-out test data., MIT Media Lab Consortium, Robert Wood Johnson Foundation (Wellbeing Initiative), National Institutes of Health (U.S.) (Grant R01GM105018), Samsung (Firm), Natural Sciences and Engineering Research Council of Canada

To filter noise or detect features within physiological signals, it is often effective to encode expert knowledge into a model such as a machine learning classifier. However, training such a model can require much effort on the part of the researcher; this often takes the form of manually labeling portions of signal needed to represent the concept being trained. Active learning is a technique for reducing human effort by developing a classifier that can intelligently select the most relevant data samples and ask for labels for only those samples, in an iterative process. In this paper we demonstrate that active learning can reduce the labeling effort required of researchers by as much as 84% for our application, while offering equivalent or even slightly improved machine learning performance., MIT Media Lab Consortium, Robert Wood Johnson Foundation

What can wearable sensors and usage of smart phones tell us about academic performance, self-reported sleep quality, stress and mental health condition? To answer this question, we collected extensive subjective and objective data using mobile phones, surveys, and wearable sensors worn day and night from 66 participants, for 30 days each, totaling 1,980 days of data. We analyzed daily and monthly behavioral and physiological patterns and identified factors that affect academic performance (GPA), Pittsburg Sleep Quality Index (PSQI) score, perceived stress scale (PSS), and mental health composite score (MCS) from SF-12, using these month-long data. We also examined how accurately the collected data classified the participants into groups of high/low GPA, good/poor sleep quality, high/low self-reported stress, high/low MCS using feature selection and machine learning techniques. We found associations among PSQI, PSS, MCS, and GPA and personality types. Classification accuracies using the objective data from wearable sensors and mobile phones ranged from 67-92%.

We collected and analyzed subjective and objective data using surveys and wearable sensors worn day and night from 68 participants for ~30 days each, to address questions related to the relationships among sleep duration, sleep irregularity, self-reported Happy-Sad mood and other daily behavioral factors in college students. We analyzed this behavioral and physiological data to (i) identify factors that classified the participants into Happy-Sad mood using support vector machines (SVMs); and (ii) analyze how accurately sleep duration and sleep regularity for the past 1-5 days classified morning Happy-Sad mood. We found statistically significant associations amongst Sad mood and poor health-related factors. Behavioral factors including the frequency of negative social interactions, and negative emails, and total academic activity hours showed the best performance in separating the Happy-Sad mood groups. Sleep regularity and sleep duration predicted daily Happy-Sad mood with 65-80% accuracy. The number of nights giving the best prediction of Happy-Sad mood varied for different individuals., MIT Media Lab Consortium, Samsung (Firm), National Institutes of Health (U.S.) (NIH grant R01GM105018), National Institutes of Health (U.S.) (NIH grant T32HL007901), National Institutes of Health (U.S.) (NIH grant K99HL119618), National Institutes of Health (U.S.) (NIH grant K24HL105664)

Recently, wearable devices have allowed for long term, ambulatory measurement of electrodermal activity (EDA). Despite the fact that ambulatory recording can be noisy, and recording artifacts can easily be mistaken for a physiological response during analysis, to date there is no automatic method for detecting artifacts. This paper describes the development of a machine learning algorithm for automatically detecting EDA artifacts, and provides an empirical evaluation of classification performance. We have encoded our results into a freely available web-based tool for artifact and peak detection., MIT Media Lab Consortium, Samsung (Firm), National Institutes of Health (U.S.) (NIH grant R01GM105018), Natural Sciences and Engineering Research Council of Canada, Seventh Framework Programme (European Commission) (People Programme (Marie Curie Actions), FP7/2007-2013/ under REA grant agreement #327702)

Background Previous research has shown that gratitude positively influences psychological wellbeing and physical health. Grateful people are reported to feel more optimistic and happy, to better mitigate aversive experiences, and to have stronger interpersonal bonds. Gratitude interventions have been shown to result in improved sleep, more frequent exercise and stronger cardiovascular and immune systems. These findings call for the development of technologies that would inspire gratitude. This paper presents a novel system designed toward this end. Methods We leverage pervasive technologies to naturally embed inspiration to express gratitude in everyday life. Novel to this work, mobile sensor data is utilized to infer optimal moments for stimulating contextually relevant thankfulness and appreciation. Sporadic mood measurements are inventively obtained through the smartphone lock screen, investigating their interplay with grateful expressions. Both momentary thankful emotion and dispositional gratitude are measured. To evaluate our system, we ran two rounds of randomized control trials (RCT), including a pilot study (N = 15, 2 weeks) and a main study (N = 27, 5 weeks). Studies’ participants were provided with a newly developed smartphone app through which they were asked to express gratitude; the app displayed inspirational content to only the intervention group, while measuring contextual cues for all users. Results In both rounds of the RCT, the intervention was associated with improved thankful behavior. Significant increase was observed in multiple facets of practicing gratitude in the intervention groups. The average frequency of practicing thankfulness increased by more than 120 %, comparing the baseline weeks with the intervention weeks of the main study. In contrast, the control group of the same study exhibited a decrease of 90 % in the frequency of thankful expressions. In the course of the study’s 5 weeks, increases in dispositional gratitude and in psych, Robert Wood Johnson Foundation, MIT Media Lab Consortium