Search

In this work, we describe a system for classifying activities in first-person video using a fuzzy inference system. Our fuzzy inference system is built on top of traditional object-and motion-based video features and provides a description of activities in terms of multiple fuzzy output variables. We demonstrate the application of the fuzzy system on a well known dataset of unscripted first person videos to classify actions into four categories. Comparing the results to other supervised learning techniques and the state-of-the-art, we find that our fuzzy system outperforms alternatives. Further, the fuzzy outputs have the potential to be much more descriptive than conventional classifiers due to their ability to handle uncertainty and produce explainable results.

Recent studies show that physiological data can detect changes in mental effort, making way for the development of wearable sensors to monitor mental effort in school, work, and at home. We have yet to explore how such a device would work with a single participant over an extended time duration. We used a longitudinal case study design with ~38 h of data to explore the efficacy of electrodermal activity, skin temperature, and heart rate for classifying mental effort. We utilized a 2-state Markov switching regression model to understand the efficacy of these physiological measures for predicting self-reported mental effort during logged activities. On average, a model with state-dependent relationships predicted within one unit of reported mental effort (training RMSE = 0.4, testing RMSE = 0.7). This automated sensing of mental effort can have applications in various domains including student engagement detection and cognitive state assessment in drivers, pilots, and caregivers.

COVID-19 pandemic has adversely and disproportionately impacted people suffering from mental health issues and substance use problems. This has been exacerbated by social isolation during the pandemic and the social stigma associated with mental health and substance use disorders, making people reluctant to share their struggles and seek help. Due to the anonymity and privacy they provide, social media emerged as a convenient medium for people to share their experiences about their day to day struggles. Reddit is a well-recognized social media platform that provides focused and structured forums called subreddits, that users subscribe to and discuss their experiences with others. Temporal assessment of the topical correlation between social media postings about mental health/substance use and postings about Coronavirus is crucial to better understand public sentiment on the pandemic and its evolving impact, especially related to vulnerable populations. In this study, we conduct a longitudinal topical analysis of postings between subreddits r/depression, r/Anxiety, r/SuicideWatch, and r/Coronavirus, and postings between subreddits r/opiates, r/OpiatesRecovery, r/addiction, and r/Coronavirus from January 2020 - October 2020. Our results show a high topical correlation between postings in r/depression and r/Coronavirus in September 2020. Further, the topical correlation between postings on substance use disorders and Coronavirus fluctuates, showing the highest correlation in August 2020. By monitoring these trends from platforms such as Reddit, epidemiologists, and mental health professionals can gain insights into the challenges faced by communities for targeted interventions.

Sickle cell disease (SCD) is a red blood cell disorder complicated by lifelong issues with pain. Management of SCD related pain is particularly challenging due to its subjective nature. Hence, the development of an objective automatic pain assessment method is critical to pain management in SCD. In this work, we developed a continuous pain assessment model using physiological and body movement sensor signals collected from a wearable wrist-worn device. Specifically, we implemented ensemble feature selection methods to select robust and stable features extracted from wearable data for better understanding of pain. Our experiments showed that the stability of feature selection methods could be substantially increased by using the ensemble approach. Since different ensemble feature selection methods prefer varying feature subsets for pain estimation, we further utilized stacked generalization to maximize the information usage contained in the selected features from different methods. Using this approach, our best performing model obtained the root-mean-square error of 1.526 and the Pearson correlation of 0.618 for continuous pain assessment. This indicates that subjective pain scores can be estimated using objective wearable sensor data with high precision.

Recent advances in natural language processing have enabled automation of a wide range of tasks, including machine translation, named entity recognition, and sentiment analysis. Automated summarization of documents, or groups of documents, however, has remained elusive, with many efforts limited to extraction of keywords, key phrases, or key sentences. Accurate abstractive summarization has yet to be achieved due to the inherent difficulty of the problem, and limited availability of training data. In this paper, we propose a topic-centric unsupervised multi-document summarization framework to generate extractive and abstractive summaries for groups of scientific articles across 20 Fields of Study (FoS) in Microsoft Academic Graph (MAG) and news articles from DUC-2004 Task 2. The proposed algorithm generates an abstractive summary by developing salient language unit selection and text generation techniques. Our approach matches the state-of-the-art when evaluated on automated extractive evaluation metrics and performs better for abstractive summarization on five human evaluation metrics (entailment, coherence, conciseness, readability, and grammar). We achieve a kappa score of 0.68 between two co-author linguists who evaluated our results. We plan to publicly share MAG-20, a human-validated gold standard dataset of topic-clustered research articles and their summaries to promote research in abstractive summarization., Comment: 6 pages, 6 Figures, 8 Tables. Accepted at IEEE Big Data 2020 (https://bigdataieee.org/BigData2020/AcceptedPapers.html)

Pain in sickle cell disease (SCD) is often associated with increased morbidity, mortality, and high healthcare costs. The standard method for predicting the absence, presence, and intensity of pain has long been self-report. However, medical providers struggle to manage patients based on subjective pain reports correctly and pain medications often lead to further difficulties in patient communication as they may cause sedation and sleepiness. Recent studies have shown that objective physiological measures can predict subjective self-reported pain scores for inpatient visits using machine learning (ML) techniques. In this study, we evaluate the generalizability of ML techniques to data collected from 50 patients over an extended period across three types of hospital visits (i.e., inpatient, outpatient and outpatient evaluation). We compare five classification algorithms for various pain intensity levels at both intra-individual (within each patient) and inter-individual (between patients) level. While all the tested classifiers perform much better than chance, a Decision Tree (DT) model performs best at predicting pain on an 11-point severity scale (from 0-10) with an accuracy of 0.728 at an inter-individual level and 0.653 at an intra-individual level. The accuracy of DT significantly improves to 0.941 on a 2-point rating scale (i.e., no/mild pain: 0-5, severe pain: 6-10) at an intra-individual level. Our experimental results demonstrate that ML techniques can provide an objective and quantitative evaluation of pain intensity levels for all three types of hospital visits., Comment: Accepted for presentation at the FIRST WORKSHOP ON COMPUTATIONAL & AFFECTIVE INTELLIGENCE IN HEALTHCARE APPLICATIONS (VULNERABLE POPULATIONS) In Conjunction with the International Conference on Pattern Recognition (ICPR) 2021

Pain management is a crucial part in Sickle Cell Disease treatment. Accurate pain assessment is the first stage in pain management. However, pain is a subjective response and hard to assess via objective approaches. In this paper, we proposed a system to map objective physiological measures to subjective self-reported pain scores using machine learning techniques. Using Multinomial Logistic Regression and data from 40 patients, we were able to predict patients’ pain scores on an 11-point rating scale with an average accuracy of 0.578 at the intra-individual level, and an accuracy of 0.429 at the inter-individual level. With a condensed 4-point rating scale, the accuracy at the inter-individual level was further improved to 0.681. Overall, we presented a preliminary machine learning model that can predict pain scores in SCD patients with promising results. To our knowledge, such a system has not been proposed earlier within the SCD or pain domains by exploiting machine learning concepts within the clinical framework.

The amount of Internet of Things (IoT) data is growing rapidly. Although there is a growing understanding of the quality of such data at the device and network level, important challenges in interpreting and evaluating the quality at informational and application levels remain to be explored. This article discusses some of these challenges and solutions of IoT systems at the different OSI layers to understand the factors affecting the quality of the overall system. With the help of two IoT-enabled digital health applications, the authors investigate the role of semantics in measuring the data quality of the system, as well as integrating multimodal data for clinical decision support. They also discuss the extension of IoT to the Internet of Everything by including human-in-the-loop to enhance the system accuracy. This paradigm shift through the confluence of sensors and data analytics can lead to accelerated innovation in applications by overcoming the limitations of the current systems, leading to unprecedented opportunities in healthcare.

A feasibility study was conducted to investigate the use of a wearable gait analysis system for classifying gait speed using a low-cost wearable camera in a semi-structured indoor setting. Data were collected from 19 participants who wore the system during indoor walk sequences at varying self-determined speeds (slow, medium, and fast). Gait parameters using this system were compared with parameters obtained from a vest comprising of a single triaxial accelerometer and from a marker-based optical motion-capture system. Computer-vision techniques and signal processing methods were used to generate frequency-domain gait parameters from each gait-recording device, and those parameters were analysed to determine the effectiveness of the different measurement systems in discriminating gait speed. Results indicate that the authors’ low-cost, portable, vision-based system can be effectively used for in-home gait analysis.

We use self-report and electrodermal activity (EDA) wearable sensor data from 77 nights of sleep of six participants to test the efficacy of EDA data for sleep monitoring. We used factor analysis to find latent factors in the EDA data, and used causal model search to find the most probable graphical model accounting for self-reported sleep efficiency (SE), sleep quality (SQ), and the latent factors in the EDA data. Structural equation modeling was used to confirm fit of the extracted graph to the data. Based on the generated graph, logistic regression and na&iuml, ve Bayes models were used to test the efficacy of the EDA data in predicting SE and SQ. Six EDA features extracted from the total signal over a night&rsquo, s sleep could be explained by two latent factors, EDA Magnitude and EDA Storms. EDA Magnitude performed as a strong predictor for SE to aid detection of substantial changes in time asleep. The performance of EDA Magnitude and SE in classifying SQ demonstrates promise for using a wearable sensor for sleep monitoring. However, our data suggest that obtaining a more accurate sensor-based measure of SE will be necessary before smaller changes in SQ can be detected from EDA sensor data alone.

Provides a unique and robust solution to the extremely challenging task of ADL modeling.Incorporates scene information to build ADL models.In the absence of manually labeled surfaces, can still generate high-level activity state summaries.We provide a dataset for the computer vision community that is described in this manuscript. We describe a novel technique to combine motion data with scene information to capture activity characteristics of older adults using a single Microsoft Kinect depth sensor. Specifically, we describe a method to learn activities of daily living (ADLs) and instrumental ADLs (IADLs) in order to study the behavior patterns of older adults to detect health changes. To learn the ADLs, we incorporate scene information to provide contextual information to build our activity model. The strength of our algorithm lies in its generalizability to model different ADLs while adding more information to the model as we instantiate ADLs from learned activity states. We validate our results in a controlled environment and compare it with another widely accepted classifier, the hidden Markov model (HMM) and its variations. We also test our system on depth data collected in a dynamic unstructured environment at TigerPlace, an independent living facility for older adults. An in-home activity monitoring system would benefit from our algorithm to alert healthcare providers of significant temporal changes in ADL behavior patterns of frail older adults for fall risk, cognitive impairment, and other health changes.

Automated tracking of physical fitness has sparked a health revolution by allowing individuals to track their own physical activity and health in real time. This concept is beginning to be applied to tracking of cognitive load. It is well known that activity in the brain can be measured through changes in the body&rsquo, s physiology, but current real-time measures tend to be unimodal and invasive. We therefore propose the concept of a wearable educational fitness (EduFit) tracker. We use machine learning with physiological data to understand how to develop a wearable device that tracks cognitive load accurately in real time. In an initial study, we found that body temperature, skin conductance, and heart rate were able to distinguish between (i) a problem solving activity (high cognitive load), (ii) a leisure activity (moderate cognitive load), and (iii) daydreaming (low cognitive load) with high accuracy in the test dataset. In a second study, we found that these physiological features can be used to predict accurately user-reported mental focus in the test dataset, even when relatively small numbers of training data were used. We explain how these findings inform the development and implementation of a wearable device for temporal tracking and logging a user&rsquo, s learning activities and cognitive load.

Early hospital mortality prediction is critical as intensivists strive to make efficient medical decisions about the severely ill patients staying in intensive care units. As a result, various methods have been developed to address this problem based on clinical records. However, some of the laboratory test results are time-consuming and need to be processed. In this paper, we propose a novel method to predict mortality using features extracted from the heart signals of patients within the first hour of ICU admission. In order to predict the risk, quantitative features have been computed based on the heart rate signals of ICU patients. Each signal is described in terms of 12 statistical and signal-based features. The extracted features are fed into eight classifiers: decision tree, linear discriminant, logistic regression, support vector machine (SVM), random forest, boosted trees, Gaussian SVM, and K-nearest neighborhood (K-NN). To derive insight into the performance of the proposed method, several experiments have been conducted using the well-known clinical dataset named Medical Information Mart for Intensive Care III (MIMIC-III). The experimental results demonstrate the capability of the proposed method in terms of precision, recall, F1-score, and area under the receiver operating characteristic curve (AUC). The decision tree classifier satisfies both accuracy and interpretability better than the other classifiers, producing an F1-score and AUC equal to 0.91 and 0.93, respectively. It indicates that heart rate signals can be used for predicting mortality in patients in the ICU, achieving a comparable performance with existing predictions that rely on high dimensional features from clinical records which need to be processed and may contain missing information., 11 pages, 5 figures, preprint of accepted paper in IEEE&ACM CHASE 2018 and published in Smart Health journal

We present an approach for activity state recognition implemented on data collected from various sensors—standard web cameras under normal illumination, web cameras using infrared lighting, and the inexpensive Microsoft Kinect camera system. Sensors such as the Kinect ensure that activity segmentation is possible during the daytime as well as night. This is especially useful for activity monitoring of older adults since falls are more prevalent at night than during the day. This paper is an application of fuzzy set techniques to a new domain. The approach described herein is capable of accurately detecting several different activity states related to fall detection and fall risk assessment including sitting, being upright, and being on the floor to ensure that elderly residents get the help they need quickly in case of emergencies and ultimately to help prevent such emergencies.

We present the results of analyzing gait motion in first-person video taken from a commercially available wearable camera embedded in a pair of glasses. The video is analyzed with three different computer vision methods to extract motion vectors from different gait sequences from four individuals for comparison against a manually annotated ground truth dataset. Using a combination of signal processing and computer vision techniques, gait features are extracted to identify the walking pace of the individual wearing the camera as well as validated using the ground truth dataset. Our preliminary results indicate that the extraction of activity from the video in a controlled setting shows strong promise of being utilized in different activity monitoring applications such as in the eldercare environment, as well as for monitoring chronic healthcare conditions.

We present an approach for patient activity recognition in hospital rooms using depth data collected using a Kinect sensor. Depth sensors such as the Kinect ensure that activity segmentation is possible during day time as well as night while addressing the privacy concerns of patients. It also provides a technique to remotely monitor patients in a non-intrusive manner. An existing fall detection algorithm is currently generating fall alerts in several rooms in the University of Missouri Hospital (MUH). In this paper we describe a technique to reduce false alerts such as pillows falling off the bed or equipment movement. We do so by detecting the presence of the patient in the bed for the times when the fall alert is generated. We test our algorithm on 96 hours obtained in two hospital rooms from MUH.

This paper presents an approach to designing a strategy for the game of Geister using the three main research areas of computational intelligence. We use a goal-based fuzzy inference system to evaluate the utility of possible actions and a neural network to estimate unobservable features (the true natures of the opponent ghosts). Finally, we develop a coevolutionary algorithm to learn the parameters of the strategy. The resulting autonomous gameplay agent was entered in a global competition sponsored by the IEEE Computational Intelligence Society and finished second among eight participating teams.

Complex activities such as instrumental activities of daily living (IADLs) can be identified by creating a hierarchical model of fuzzy rules. In this work, we present a framework to model a specific IADL - "making the bed". For this activity recognition, the need for a three level Fuzzy Inference System (FIS) model is shown. Simple features such as bounding box parameters were extracted from the foreground images and combined with 3D features extracted from the Kinect depth data. This was then fed as input to the three layered FIS for further analysis. Data collected from several participants were tested and evaluated. Such a framework can be used to model several other IADLS as well as basic activities of daily living (ADLs). Analysis of ADLs can be used to compare daily patterns in older adults to measure changes in behavior. This can then be used to predict health changes to assist older adults in leading independent lifestyles for longer time periods.

We present algorithms to segment the activities of sitting and standing, and identify the regions of sit-to-stand (STS) transitions in a given image sequence. As a means of fall risk assessment, we propose methods to measure STS time using the 3-D modeling of a human body in voxel space as well as ellipse fitting algorithms and image features to capture orientation of the body. The proposed algorithms were tested on ten older adults with ages ranging from 83 to 97. Two techniques in combination yielded the best results, namely the voxel height in conjunction with the ellipse fit. Accurate STS time was computed on various STSs and verified using a marker-based motion capture system. This application can be used as part of a continuous video monitoring system in the homes of older adults and can provide valuable information to help detect fall risk and enable early interventions.

We present a unique occlusion and foreground overlap detection technique from depth sensor data using a fuzzy rule-based system. Features such as bounding box parameters and skeletonization were extracted from the foreground images and then input to the Fuzzy Inference System. Overlap and occlusion confidence measures were taken for each frame in the image sequence and compared against the extracted ground truth. This technique can help filter out occluded regions in the image sequence which, in an Eldercare environment, can then be used to compute accurate estimates of fall risk parameters such as stride time, stride length, and walking speed on a daily basis in in order to monitor the well-being of older adults in an ambient assisted living facility.

As a part of our passive fall risk assessment research in home environments, we present a method to identify older residents using features extracted from their gait information from a single depth camera. Depth images have been collected continuously for about eight months from several apartments at a senior housing facility. Shape descriptors such as bounding box information and image moments were extracted from silhouettes of the depth images. The features were then clustered using Possibilistic C Means for resident identification. This technology will allow researchers and health professionals to gather more information on the individual residents by filtering out data belonging to non-residents. Gait related information belonging exclusively to the older residents can then be gathered. The data can potentially help detect changes in gait patterns which can be used to analyze fall risk for elderly residents by passively observing them in their home environments.

The ability to rise from a chair is an important parameter to assess the balance deficits of a person. In particular, this can be an indication of risk for falling in elderly persons. Our goal is automated assessment of fall risk using video data. Towards this goal, we present a simple yet effective method of detecting transition, i.e. sit-to-stand and stand-to-sit, from image frames using fuzzy clustering methods on image moments. The technique described in this paper is shown to be robust even in the presence of noise and has been tested on several data sequences using different subjects yielding promising results.