Search

© 2016 Authors. In this work we introduce an experimental methodology and open-sourced web application designed to promote behavior change and wellbeing within a workplace environment. Colleagues are paired together and asked to design a Behavioral Intervention Technology (BIT) uniquely customized to suit their partner's behavior change goal. We present the results of a preliminary evaluation of the experiment, and suggest ways to further improve and expand our experiment design., Robert Wood Johnson Foundation, Natural Sciences and Engineering Research Council Canada, Massachusetts Institute of Technology. Media Laboratory. Advancing Wellbeing Initiative & consortium

© 2016 Authors. In this work we introduce an experimental methodology and open-sourced web application designed to promote behavior change and wellbeing within a workplace environment. Colleagues are paired together and asked to design a Behavioral Intervention Technology (BIT) uniquely customized to suit their partner's behavior change goal. We present the results of a preliminary evaluation of the experiment, and suggest ways to further improve and expand our experiment design.

© 2010-2012 IEEE. While accurately predicting mood and wellbeing could have a number of important clinical benefits, traditional machine learning (ML) methods frequently yield low performance in this domain. We posit that this is because a one-size-fits-all machine learning model is inherently ill-suited to predicting outcomes like mood and stress, which vary greatly due to individual differences. Therefore, we employ Multitask Learning (MTL) techniques to train personalized ML models which are customized to the needs of each individual, but still leverage data from across the population. Three formulations of MTL are compared: i) MTL deep neural networks, which share several hidden layers but have final layers unique to each task; ii) Multi-task Multi-Kernel learning, which feeds information across tasks through kernel weights on feature types; and iii) a Hierarchical Bayesian model in which tasks share a common Dirichlet Process prior. We offer the code for this work in open source. These techniques are investigated in the context of predicting future mood, stress, and health using data collected from surveys, wearable sensors, smartphone logs, and the weather. Empirical results demonstrate that using MTL to account for individual differences provides large performance improvements over traditional machine learning methods and provides personalized, actionable insights.

© 2017 IEEE. To accomplish forecasting of mood in real-world situations, affective computing systems need to collect and learn from multimodal data collected over weeks or months of daily use. Such systems are likely to encounter frequent data loss, e.g. when a phone loses location access, or when a sensor is recharging. Lost data can handicap classifiers trained with all modalities present in the data. This paper describes a new technique for handling missing multimodal data using a specialized denoising autoencoder: The Multimodal Autoencoder (MMAE). Empirical results from over 200 participants and 5500 days of data demonstrate that the MMAE is able to predict the feature values from multiple missing modalities more accurately than reconstruction methods such as principal components analysis (PCA). We discuss several practical benefits of the MMAE's encoding and show that it can provide robust mood prediction even when up to three quarters of the data sources are lost.

© Springer International Publishing AG 2016. This paper investigates how an intelligent agent could be designed to both predict whether it is bonding with its user, and convey appropriate facial expression and body language responses to foster bonding. Video and Kinect recordings are collected from a series of naturalistic conversations, and a reliable measure of bonding is adapted and verified. A qualitative and quantitative analysis is conducted to determine the non-verbal cues that characterize both high and low bonding conversations. We then train a deep neural network classifier using one minute segments of facial expression and body language data, and show that it is able to accurately predict bonding in novel conversations.

Thesis: Ph. D., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, February, 2020, Cataloged from student-submitted PDF of thesis. "February 2020.", Includes bibliographical references (pages 309-342)., Social learning is a crucial component of human intelligence, allowing us to rapidly adapt to new scenarios, learn new tasks, and communicate knowledge that can be built on by others. This dissertation argues that the ability of artificial intelligence to learn, adapt, and generalize to new environments can be enhanced by mechanisms that allow for social learning. I propose several novel deep- and reinforcement-learning methods that improve the social and affective capabilities of artificial intelligence (AI), through social learning both from humans and from other AI agents. First, I show how AI agents can learn from the causal influence of their actions on other agents, leading to enhanced coordination and communication in multi-agent reinforcement learning. Second, I investigate learning socially from humans, using non-verbal and implicit affective signals such as facial expressions and sentiment., This ability to optimize for human satisfaction through sensing implicit social cues can enhance human-AI interaction, and guide AI systems to take actions aligned with human preferences. Learning from human interaction with reinforcement learning, however, may require dealing with sparse, off-policy data, without the ability to explore online in the environment - a situation that is inherent to safety-critical, real-world systems that must be tested before being deployed. I present several techniques that enable learning effectively in this challenging setting. Experiments deploying these models to interact with humans reveal that learning from implicit, affective signals is more effective than relying on humans to provide manual labels of their preferences, a task that is cumbersome and time-consuming. However, learning from humans' affective cues requires recognizing them first., In the third part of this thesis, I present several machine learning methods for automatically interpreting human data and recognizing affective and social signals such as stress, happiness, and conversational rapport. I show that personalizing such models using multi-task learning achieves large performance gains in predicting highly individualistic outcomes like human happiness. Together, these techniques create a framework for building socially and emotionally intelligent AI agents that can flexibly learn from each other and from humans., by Natasha Jaques., Ph. D., Ph.D. Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences

This exploratory study examined the effects of varying g-forces, including feelings of weightlessness, on an individual's physiology during parabolic flight. Specifically, we collected heart rate, accelerometer, and skin conductance measurements from 16 flyers aboard a parabolic flight using wearable, wireless sensors. The biosignals were then correlated to participant reports of nausea, anxiety, and excitement during periods of altered g-forces. Using linear mixed-effects models, we found that (1) heart rate was positively correlated to individuals' self-reported highest/lowest periods of both anxiety and excitement, and (2) bilateral skin conductance asymmetry was positively correlated to individuals' self-reported highest/lowest periods of nausea.

Electrodermal activity (EDA) recording is a powerful, widely used tool for monitoring psychological or physiological arousal. However, analysis of EDA is hampered by its sensitivity to motion artifacts. We propose a method for removing motion artifacts from EDA, measured as skin conductance (SC), using a stationary wavelet transform (SWT). We modeled the wavelet coefficients as a Gaussian mixture distribution corresponding to the underlying skin conductance level (SCL) and skin conductance responses (SCRs). The goodness-of-fit of the model was validated on ambulatory SC data. We evaluated the proposed method in comparison with three previous approaches. Our method achieved a greater reduction of artifacts while retaining motion-artifact-free data.

This paper presents a system prototype designed to capture naturally occurring instances of positive emotion during the course of normal interaction with a computer. A facial expression recognition algorithm is applied to images captured with the user's webcam. When the user smiles, both a photo and a screenshot are recorded and saved to the user's profile for later review. Based on positive psychology research, we hypothesize that the act of reviewing content that led to smiles will improve positive affect, and consequently, overall wellbeing. We conducted a preliminary user study to test this hypothesis, as well as to gather feedback on the initial design., Robert Wood Johnson Foundation (Wellbeing Initiative)

In order to model students' happiness, we apply machine learning methods to data collected from undergrad students monitored over the course of one month each. The data collected include physiological signals, location, smartphone logs, and survey responses to behavioral questions. Each day, participants reported their wellbeing on measures including stress, health, and happiness. Because of the relationship between happiness and depression, modeling happiness may help us to detect individuals who are at risk of depression and guide interventions to help them. We are also interested in how behavioral factors (such as sleep and social activity) affect happiness positively and negatively. A variety of machine learning and feature selection techniques are compared, including Gaussian Mixture Models and ensemble classification. We achieve 70% classification accuracy of self-reported happiness on held-out test data., MIT Media Lab Consortium, Robert Wood Johnson Foundation (Wellbeing Initiative), National Institutes of Health (U.S.) (Grant R01GM105018), Samsung (Firm), Natural Sciences and Engineering Research Council of Canada

To filter noise or detect features within physiological signals, it is often effective to encode expert knowledge into a model such as a machine learning classifier. However, training such a model can require much effort on the part of the researcher; this often takes the form of manually labeling portions of signal needed to represent the concept being trained. Active learning is a technique for reducing human effort by developing a classifier that can intelligently select the most relevant data samples and ask for labels for only those samples, in an iterative process. In this paper we demonstrate that active learning can reduce the labeling effort required of researchers by as much as 84% for our application, while offering equivalent or even slightly improved machine learning performance., MIT Media Lab Consortium, Robert Wood Johnson Foundation

What can wearable sensors and usage of smart phones tell us about academic performance, self-reported sleep quality, stress and mental health condition? To answer this question, we collected extensive subjective and objective data using mobile phones, surveys, and wearable sensors worn day and night from 66 participants, for 30 days each, totaling 1,980 days of data. We analyzed daily and monthly behavioral and physiological patterns and identified factors that affect academic performance (GPA), Pittsburg Sleep Quality Index (PSQI) score, perceived stress scale (PSS), and mental health composite score (MCS) from SF-12, using these month-long data. We also examined how accurately the collected data classified the participants into groups of high/low GPA, good/poor sleep quality, high/low self-reported stress, high/low MCS using feature selection and machine learning techniques. We found associations among PSQI, PSS, MCS, and GPA and personality types. Classification accuracies using the objective data from wearable sensors and mobile phones ranged from 67-92%.

We collected and analyzed subjective and objective data using surveys and wearable sensors worn day and night from 68 participants for ~30 days each, to address questions related to the relationships among sleep duration, sleep irregularity, self-reported Happy-Sad mood and other daily behavioral factors in college students. We analyzed this behavioral and physiological data to (i) identify factors that classified the participants into Happy-Sad mood using support vector machines (SVMs); and (ii) analyze how accurately sleep duration and sleep regularity for the past 1-5 days classified morning Happy-Sad mood. We found statistically significant associations amongst Sad mood and poor health-related factors. Behavioral factors including the frequency of negative social interactions, and negative emails, and total academic activity hours showed the best performance in separating the Happy-Sad mood groups. Sleep regularity and sleep duration predicted daily Happy-Sad mood with 65-80% accuracy. The number of nights giving the best prediction of Happy-Sad mood varied for different individuals., MIT Media Lab Consortium, Samsung (Firm), National Institutes of Health (U.S.) (NIH grant R01GM105018), National Institutes of Health (U.S.) (NIH grant T32HL007901), National Institutes of Health (U.S.) (NIH grant K99HL119618), National Institutes of Health (U.S.) (NIH grant K24HL105664)

Recently, wearable devices have allowed for long term, ambulatory measurement of electrodermal activity (EDA). Despite the fact that ambulatory recording can be noisy, and recording artifacts can easily be mistaken for a physiological response during analysis, to date there is no automatic method for detecting artifacts. This paper describes the development of a machine learning algorithm for automatically detecting EDA artifacts, and provides an empirical evaluation of classification performance. We have encoded our results into a freely available web-based tool for artifact and peak detection., MIT Media Lab Consortium, Samsung (Firm), National Institutes of Health (U.S.) (NIH grant R01GM105018), Natural Sciences and Engineering Research Council of Canada, Seventh Framework Programme (European Commission) (People Programme (Marie Curie Actions), FP7/2007-2013/ under REA grant agreement #327702)