Search

Data-driven approaches for remote detection of Parkinson's Disease and its motor symptoms have proliferated in recent years, owing to the potential clinical benefits of early diagnosis. The holy grail of such approaches is the free-living scenario, in which data are collected continuously and unobtrusively during every day life. However, obtaining fine-grained ground-truth and remaining unobtrusive is a contradiction and therefore, the problem is usually addressed via multiple-instance learning. Yet for large scale studies, obtaining even the necessary coarse ground-truth is not trivial, as a complete neurological evaluation is required. In contrast, large scale collection of data without any ground-truth is much easier. Nevertheless, utilizing unlabelled data in a multiple-instance setting is not straightforward, as the topic has received very little research attention. Here we try to fill this gap by introducing a new method for combining semi-supervised with multiple-instance learning. Our approach builds on the Virtual Adversarial Training principle, a state-of-the-art approach for regular semi-supervised learning, which we adapt and modify appropriately for the multiple-instance setting. We first establish the validity of the proposed approach through proof-of-concept experiments on synthetic problems generated from two well-known benchmark datasets. We then move on to the actual task of detecting PD tremor from hand acceleration signals collected in-the-wild, but in the presence of additional completely unlabelled data. We show that by leveraging the unlabelled data of 454 subjects we can achieve large performance gains (up to 9% increase in F1-score) in per-subject tremor detection for a cohort of 45 subjects with known tremor ground-truth., Comment: A. Papadopoulos and A. Delopoulos, "Leveraging Unlabelled Data in Multiple-Instance Learning Problems for Improved Detection of Parkinsonian Tremor in Free-Living Conditions," in IEEE Journal of Biomedical and Health Informatics, doi: 10.1109/JBHI.2023.3267095

Heart murmurs are abnormal sounds present in heartbeats, caused by turbulent blood flow through the heart. The PhysioNet 2022 challenge targets automatic detection of murmur from audio recordings of the heart and automatic detection of normal vs. abnormal clinical outcome. The recordings are captured from multiple locations around the heart. Our participation investigates the effectiveness of selfsupervised learning for murmur detection. We train the layers of a backbone CNN in a self-supervised way with data from both this year's and the 2016 challenge. We use two different augmentations on each training sample, and normalized temperature-scaled cross-entropy loss. We experiment with different augmentations to learn effective phonocardiogram representations. To build the final detectors we train two classification heads, one for each challenge task. We present evaluation results for all combinations of the available augmentations, and for our multipleaugmentation approach. Our team's, Listen2YourHeart, SSL murmur detection classifier received a weighted accuracy score of 0.737 (ranked 13th out of 40 teams) and an outcome identification challenge cost score of 11946 (ranked 7th out of 39 teams) on the hidden test set., Comment: To be published in the proceedings of CinC 2022 (https://cinc.org/). This is a preprint version of the final paper

The importance of automated and objective monitoring of dietary behavior is becoming increasingly accepted. The advancements in sensor technology along with recent achievements in machine-learning--based signal-processing algorithms have enabled the development of dietary monitoring solutions that yield highly accurate results. A common bottleneck for developing and training machine learning algorithms is obtaining labeled data for training supervised algorithms, and in particular ground truth annotations. Manual ground truth annotation is laborious, cumbersome, can sometimes introduce errors, and is sometimes impossible in free-living data collection. As a result, there is a need to decrease the labeled data required for training. Additionally, unlabeled data, gathered in-the-wild from existing wearables (such as Bluetooth earbuds) can be used to train and fine-tune eating-detection models. In this work, we focus on training a feature extractor for audio signals captured by an in-ear microphone for the task of eating detection in a self-supervised way. We base our approach on the SimCLR method for image classification, proposed by Chen et al. from the domain of computer vision. Results are promising as our self-supervised method achieves similar results to supervised training alternatives, and its overall effectiveness is comparable to current state-of-the-art methods. Code is available at https://github.com/mug-auth/ssl-chewing ., 5 pages, IEEE EMBC 2021

While automatic tracking and measuring of our physical activity is a well established domain, not only in research but also in commercial products and every-day life-style, automatic measurement of eating behavior is significantly more limited. Despite the abundance of methods and algorithms that are available in bibliography, commercial solutions are mostly limited to digital logging applications for smart-phones. One factor that limits the adoption of such solutions is that they usually require specialized hardware or sensors. Based on this, we evaluate the potential for estimating the weight of consumed food (per bite) based only on the audio signal that is captured by commercial ear buds (Samsung Galaxy Buds). Specifically, we examine a combination of features (both audio and non-audio features) and trainable estimators (linear regression, support vector regression, and neural-network based estimators) and evaluate on an in-house dataset of 8 participants and 4 food types. Results indicate good potential for this approach: our best results yield mean absolute error of less than 1 g for 3 out of 4 food types when training food-specific models, and 2.1 g when training on all food types together, both of which improve over an existing literature approach., Comment: 4 pages, 2 figures, IEEE EMBC 2021

Heart Rate Variability is a significant indicator of the Autonomic Neural System's functioning, traditionally evaluated from electrocardiogram recordings. Photoplethysmography sensors, like electrocardiograph devices, track the heart's activity and have been widely popularized by their use in smart watches and fitness trackers. In this study we develop a deep learning based approach which is able to successfully estimate the patient's Root Mean Square of the Successive Differences, a common heart rate variability metric, from lower quality, less expensive photoplethysmography sensors under a wide range of conditions.

The structure of the cumulative food intake (CFI) curve has been associated with obesity and eating disorders. Scales that record the weight loss of a plate from which a subject eats food are used for capturing this curve; however, their measurements are contaminated by additive noise and are distorted by certain types of artifacts. This paper presents an algorithm for automatically processing continuous in-meal weight measurements in order to extract the clean CFI curve and in-meal eating indicators, such as total food intake and food intake rate. The algorithm relies on the representation of the weight-time series by a string of symbols that correspond to events such as bites or food additions. A context-free grammar is next used to model a meal as a sequence of such events. The selection of the most likely parse tree is finally used to determine the predicted eating sequence. The algorithm is evaluated on a dataset of 113 meals collected using the Mandometer, a scale that continuously samples plate weight during eating. We evaluate the effectiveness for seven indicators and for bite-instance detection. We compare our approach with three state-of-the-art algorithms, and achieve the lowest error rates for most indicators (24 g for total meal weight). The proposed algorithm extracts the parameters of the CFI curve automatically, eliminating the need for manual data processing, and thus facilitating large-scale studies of eating behavior.

Parkinson’s disease (PD) is a neurodegenerative disorder with both motor and non-motor symptoms. Despite the progressive nature of PD, early diagnosis, tracking the disease’s natural history and measuring the drug response are factors that play a major role in determining the quality of life of the affected individual. Apart from the common motor symptoms, i.e., tremor at rest, rigidity and bradykinesia, studies suggest that PD is associated with disturbances in eating behavior and energy intake. Specifically, PD is associated with drug-induced impulsive eating disorders such as binge eating, appetite-related non-motor issues such as weight loss and/or gain as well as dysphagia—factors that correlate with difficulties in completing day-to-day eating-related tasks. In this work we introduce Plate-to-Mouth (PtM), an indicator that relates with the time spent for the hand operating the utensil to transfer a quantity of food from the plate into the mouth during the course of a meal. We propose a two-step approach towards the objective calculation of PtM. Initially, we use the 3D acceleration and orientation velocity signals from an off-the-shelf smartwatch to detect the bite moments and upwards wrist micromovements that occur during a meal session. Afterwards, we process the upwards hand micromovements that appear prior to every detected bite during the meal in order to estimate the bite’s PtM duration. Finally, we use a density-based scheme to estimate the PtM durations distribution and form the in-meal eating behavior profile of the subject. In the results section, we provide validation for every step of the process independently, as well as showcase our findings using a total of three datasets, one collected in a controlled clinical setting using standardized meals (with a total of 28 meal sessions from 7 Healthy Controls (HC) and 21 PD patients) and two collected in-the-wild under free living conditions (37 meals from 4 HC/10 PD patients and 629 meals from 3 HC/3 PD patients, respectively). Experimental results reveal an Area Under the Curve (AUC) of 0.748 for the clinical dataset and 0.775/1.000 for the in-the-wild datasets towards the classification of in-meal eating behavior profiles to the PD or HC group. This is the first work that attempts to use wearable Inertial Measurement Unit (IMU) sensor data, collected both in clinical and in-the-wild settings, towards the extraction of an objective eating behavior indicator for PD.

Diabetic Retinopathy (DR) is a leading cause of vision loss globally. Yet despite its prevalence, the majority of affected people lack access to the specialized ophthalmologists and equipment required for assessing their condition. This can lead to delays in the start of treatment, thereby lowering their chances for a successful outcome. Machine learning systems that automatically detect the disease in eye fundus images have been proposed as a means of facilitating access to DR severity estimates for patients in remote regions or even for complementing the human expert's diagnosis. In this paper, we propose a machine learning system for the detection of referable DR in fundus images that is based on the paradigm of multiple-instance learning. By extracting local information from image patches and combining it efficiently through an attention mechanism, our system is able to achieve high classification accuracy. Moreover, it can highlight potential image regions where DR manifests through its characteristic lesions. We evaluate our approach on publicly available retinal image datasets, in which it exhibits near state-of-the-art performance, while also producing interpretable visualizations of its predictions., Comment: 11 pages

The recent advances in multimedia modeling with deep learning methods have significantly affected remote sensing applications, such as canopy height mapping. Estimating canopy height maps in large-scale is an important step towards sustainable ecosystem management. Apart from the standard height estimation method using LiDAR data, other airborne measurement techniques, such as very high-resolution passive airborne imaging, have also shown to provide accurate estimations. However, those methods suffer from high cost and cannot be used at large-scale nor frequently. In our study, we adopt a neural network architecture to estimate pixel-wise canopy height from cost-effective spaceborne imagery. A deep convolutional encoder-decoder network, based on the SegNet architecture together with skip connections, is trained to embed the multi-spectral pixels of a Sentinel-2 input image to height values via end-to-end learned texture features. Experimental results in a study area of 942 \(\mathrm{km}^2\) yield similar or better estimation accuracy resolution in comparison with a method based on costly airborne images as well as with another state-of-the-art deep learning approach based on spaceborne images.

BACKGROUND Obesity is a major public health problem globally and in Europe. The prevalence of childhood obesity is also soaring. Several parameters of the living environment are contributing to this increase, such as the density of fast food retailers, and thus, preventive health policies against childhood obesity must focus on the environment to which children are exposed. Currently, there are no systems in place to objectively measure the effect of living environment parameters on obesogenic behaviors and obesity. The H2020 project “BigO: Big Data Against Childhood Obesity” aims to tackle childhood obesity by creating new sources of evidence based on big data. OBJECTIVE This paper introduces the Obesity Prevention dashboard (OPdashboard), implemented in the context of BigO, which offers an interactive data platform for the exploration of objective obesity-related behaviors and local environments based on the data recorded using the BigO mHealth (mobile health) app. METHODS The OPdashboard, which can be accessed on the web, allows for (1) the real-time monitoring of children’s obesogenic behaviors in a city area, (2) the extraction of associations between these behaviors and the local environment, and (3) the evaluation of interventions over time. More than 3700 children from 33 schools and 2 clinics in 5 European cities have been monitored using a custom-made mobile app created to extract behavioral patterns by capturing accelerometer and geolocation data. Online databases were assessed in order to obtain a description of the environment. The dashboard’s functionality was evaluated during a focus group discussion with public health experts. RESULTS The preliminary association outcomes in 2 European cities, namely Thessaloniki, Greece, and Stockholm, Sweden, indicated a correlation between children’s eating and physical activity behaviors and the availability of food-related places or sports facilities close to schools. In addition, the OPdashboard was used to assess changes to children’s physical activity levels as a result of the health policies implemented to decelerate the COVID-19 outbreak. The preliminary outcomes of the analysis revealed that in urban areas the decrease in physical activity was statistically significant, while a slight increase was observed in the suburbs. These findings indicate the importance of the availability of open spaces for behavioral change in children. Discussions with public health experts outlined the dashboard’s potential to aid in a better understanding of the interplay between children’s obesogenic behaviors and the environment, and improvements were suggested. CONCLUSIONS Our analyses serve as an initial investigation using the OPdashboard. Additional factors must be incorporated in order to optimize its use and obtain a clearer understanding of the results. The unique big data that are available through the OPdashboard can lead to the implementation of models that are able to predict population behavior. The OPdashboard can be considered as a tool that will increase our understanding of the underlying factors in childhood obesity and inform the design of regional interventions both for prevention and treatment.

Background Obesity is a major public health problem globally and in Europe. The prevalence of childhood obesity is also soaring. Several parameters of the living environment are contributing to this increase, such as the density of fast food retailers, and thus, preventive health policies against childhood obesity must focus on the environment to which children are exposed. Currently, there are no systems in place to objectively measure the effect of living environment parameters on obesogenic behaviors and obesity. The H2020 project “BigO: Big Data Against Childhood Obesity” aims to tackle childhood obesity by creating new sources of evidence based on big data. Objective This paper introduces the Obesity Prevention dashboard (OPdashboard), implemented in the context of BigO, which offers an interactive data platform for the exploration of objective obesity-related behaviors and local environments based on the data recorded using the BigO mHealth (mobile health) app. Methods The OPdashboard, which can be accessed on the web, allows for (1) the real-time monitoring of children’s obesogenic behaviors in a city area, (2) the extraction of associations between these behaviors and the local environment, and (3) the evaluation of interventions over time. More than 3700 children from 33 schools and 2 clinics in 5 European cities have been monitored using a custom-made mobile app created to extract behavioral patterns by capturing accelerometer and geolocation data. Online databases were assessed in order to obtain a description of the environment. The dashboard’s functionality was evaluated during a focus group discussion with public health experts. Results The preliminary association outcomes in 2 European cities, namely Thessaloniki, Greece, and Stockholm, Sweden, indicated a correlation between children’s eating and physical activity behaviors and the availability of food-related places or sports facilities close to schools. In addition, the OPdashboard was used to assess changes to children’s physical activity levels as a result of the health policies implemented to decelerate the COVID-19 outbreak. The preliminary outcomes of the analysis revealed that in urban areas the decrease in physical activity was statistically significant, while a slight increase was observed in the suburbs. These findings indicate the importance of the availability of open spaces for behavioral change in children. Discussions with public health experts outlined the dashboard’s potential to aid in a better understanding of the interplay between children’s obesogenic behaviors and the environment, and improvements were suggested. Conclusions Our analyses serve as an initial investigation using the OPdashboard. Additional factors must be incorporated in order to optimize its use and obtain a clearer understanding of the results. The unique big data that are available through the OPdashboard can lead to the implementation of models that are able to predict population behavior. The OPdashboard can be considered as a tool that will increase our understanding of the underlying factors in childhood obesity and inform the design of regional interventions both for prevention and treatment.

Obesity is currently affecting very large portions of the global population. Effective prevention and treatment starts at the early age and requires objective knowledge of population-level behavior on the region/neighborhood scale. To this end, we present a system for extracting and collecting behavioral information on the individual-level objectively and automatically. The behavioral information is related to physical activity, types of visited places, and transportation mode used between them. The system employs indicator-extraction algorithms from the literature which we evaluate on publicly available datasets. The system has been developed and integrated in the context of the EU-funded BigO project that aims at preventing obesity in young populations., Comment: Accepted version to be published in 2020, 42nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Montreal, Canada

Obesity affects a rising percentage of the children and adolescent population, contributing to decreased quality of life and increased risk for comorbidities. Although the major causes of obesity are known, the obesogenic behaviors manifest as a result of complex interactions of the individual with the living environment. For this reason, addressing childhood obesity remains a challenging problem for public health authorities. The BigO project (https://bigoprogram.eu) relies on large-scale behavioral and environmental data collection to create tools that support policy making and intervention design. In this work, we propose a novel analysis approach for modeling the expected population behavior as a function of the local environment. We experimentally evaluate this approach in predicting the expected physical activity level in small geographic regions using urban environment characteristics. Experiments on data collected from 156 children and adolescents verify the potential of the proposed approach. Specifically, we train models that predict the physical activity level in a region, achieving 81% leave-one-out accuracy. In addition, we exploit the model predictions to automatically visualize heatmaps of the expected population behavior in areas of interest, from which we draw useful insights. Overall, the predictive models and the automatic heatmaps are promising tools in gaining direct perception for the spatial distribution of the population's behavior, with potential uses by public health authorities., Accepted version to be published in 2020, 42nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Montreal, Canada

Parkinson's disease (PD) is the second most common age-related neurodegenerative disorder after Alzheimer's disease, associated, among others, with motor symptoms such as resting tremor, rigidity and bradykinesia. At the same time, early diagnosis of PD is hindered by a high misdiagnosis rate and the subjective nature of the diagnosis process itself. Recent developments in mobile and wearable devices, such as smartphones and smartwatches, have allowed the automated detection and objective measurement of PD symptoms. In this paper we investigate the hypothesis that PD motor symptom degradation can be assessed by studying the in-meal behavior and modeling the food intake process. To achieve this, we use the inertial data from a commercial smartwatch to investigate the in-meal eating behavior of healthy controls and PD patients. In addition, we define and provide a methodology for calculating Plate-to-Mouth (PtM), an indicator that relates with the average time that the hand spends transferring food from the plate towards the mouth during the course of a meal. The presented experimental results, using our collected dataset of 28 participants (7 healthy controls and 21 PD patients), support our hypothesis. Results initially point out that PD patients have a higher PtM value than the healthy controls. Finally, using PtM we achieve a precision/recall/F1 of 0.882/0.714/0.789 towards classifying the meals from the PD patients and healthy controls.

Studying eating behaviours is important in the fields of eating disorders and obesity. However, the current methodologies of quantifying eating behaviour in a real-life setting are lacking, either in reliability (e.g. self-reports) or in scalability. In this descriptive study, we deployed previously evaluated laboratory-based methodologies in a Swedish high school, using the Mandometer®, together with video cameras and a dedicated mobile app in order to record eating behaviours in a sample of 41 students, 16–17 years old. Without disturbing the normal school life, we achieved a 97% data-retention rate, using methods fully accepted by the target population. The overall eating style of the students was similar across genders, with male students eating more than females, during lunches of similar lengths. While both groups took similar number of bites, males took larger bites across the meal. Interestingly, the recorded school lunches were as long as lunches recorded in a laboratory setting, which is...

Mittels moderner Technologien wie Smartphones können große Mengen an Daten kontinuierlich über einen längeren Zeitraum, objektiv und ohne aktives Zutun der Nutzer im alltäglichen Leben erhoben werden. Zur frühen Diagnosestellung des idiopathischen Parkinsonsyndroms als langsam progrediente, chronische Erkrankung mit einschleichendem Auftreten motorischer sowie nicht-motorischer Symptome ist dies ideal. Eine frühere Diagnosestellung ist eine Grundvoraussetzung zur Entwicklung neuromodulatorischer Therapien. Das iPrognosis-Konsortium (11 Organisationen mit drei medizinischen Zentren aus 6 EU-Ländern) entwickelt eine Android Smartphone App zur Erfassung von Verhalten und Verhaltensänderungen im Umgang mit dem Smartphone, welche Ausdruck motorischer und nicht-motorischer Symptome der Parkinsonerkrankung sein können.

Automated and objective monitoring of eating behavior has received the attention of both the research community and the industry over the past few years. In this paper we present a method for automatically detecting meals in free living conditions, using the inertial data (acceleration and orientation velocity) from commercially available smartwatches. The proposed method operates in two steps. In the first step we process the raw inertial signals using an End-to-End Neural Network with the purpose of detecting the bite events throughout the recording. During the next step, we process the resulting bite detections using signal processing algorithms to obtain the final meal start and end timestamp estimates. Evaluation results obtained from our Leave One Subject Out experiments using our publicly available FIC and FreeFIC datasets, exhibit encouraging results by achieving an F1/Average Jaccard Index of 0.894/0.804.

Obesity is a preventable disease that affects the health of a significant population percentage, reduces the life expectancy and encumbers the health care systems. The obesity epidemic is not caused by isolated factors, but it is the result of multiple behavioural patterns and complex interactions with the living environment. Therefore, in-depth understanding of the population behaviour is essential in order to create successful policies against obesity prevalence. To this end, the BigO system facilitates the collection, processing and modelling of behavioural data at population level to provide evidence for effective policy and interventions design. In this paper, we introduce the behaviour profiles mechanism of BigO that produces comprehensive models for the behavioural patterns of individuals, while maintaining high levels of privacy protection. We give examples for the proposed mechanism from real world data and we discuss usages for supporting various types of evidence-based policy design.

Obesity in childhood and adolescence represents a major health problem. Novel e-Health technologies have been developed in order to provide a comprehensive and personalized plan of action for the prevention and management of overweight and obesity in childhood and adolescence. We used information and communication technologies to develop a “National Registry for the Prevention and Management of Overweight and Obesity” in order to register online children and adolescents nationwide, and to guide pediatricians and general practitioners regarding the management of overweight or obese subjects. Furthermore, intelligent multi-level information systems and specialized artificial intelligence algorithms are being developed with a view to offering precision and personalized medical management to obese or overweight subjects. Moreover, the Big Data against Childhood Obesity platform records behavioral data objectively by using inertial sensors and Global Positioning System (GPS) and combines them with data of the environment, in order to assess the full contextual framework that is associated with increased body mass index (BMI). Finally, a computerized decision-support tool was developed to assist pediatric health care professionals in delivering personalized nutrition and lifestyle optimization advice to overweight or obese children and their families. These e-Health applications are expected to play an important role in the management of overweight and obesity in childhood and adolescence.

Parkinson’s Disease (PD) is a slowly evolving neurologicaldisease that affects about 1% of the population above 60years old, causing symptoms that are subtle at first, but whoseintensity increases as the disease progresses. Automated detectionof these symptoms could offer clues as to the early onset of thedisease, thus improving the expected clinical outcomes of thepatients via appropriately targeted interventions. This potentialhas led many researchers to develop methods that use widelyavailable sensors to measure and quantify the presence of PDsymptoms such as tremor, rigidity and braykinesia. However,most of these approaches operate under controlled settings,such as in lab or at home, thus limiting their applicabilityunder free-living conditions. In this work, we present a methodfor automatically identifying tremorous episodes related to PD,based on IMU signals captured via a smartphone device. Wepropose a Multiple-Instance Learning approach, wherein a subject is represented as an unordered bag of accelerometer signalsegments and a single, expert-provided, tremor annotation. Ourmethod combines deep feature learning with a learnable poolingstage that is able to identify key instances within the subjectbag, while still being trainable end-to-end. We validate our algorithmon a newly introduced dataset of 45 subjects, containingaccelerometer signals collected entirely in-the-wild. The good classification performance obtained in the conducted experimentssuggests that the proposed method can efficiently navigate the noisy environment of in-the-wild recordings.&nbsp

Objective: To provide clinically-corroborated evidence of the Parkinson’s disease (PD) diagnostic potential of machine learning-based approaches for motor symptoms severity inference via multimodal data, passively captured during the natural use of smartphones. Background: PD symptoms can be mild in the early stages and they usually go unnoticed, leaving the disease undiagnosed for years [1]. Subtle motor manifestations may start five to six years prior to PD clinical diagnosis and thereafter progress quickly [2]. Motor impairment affects daily activities and can severely impact patients’ quality over the course of the disease. Information derived from mobile electronic sensors can provide, via algorithmic transformation, objective and dense information of an individual’s motor status, allowing for frequent relevant symptoms early screening and subsequent monitoring. Methodology: We analyzed longitudinal recordings of tri-axial accelerometer, voice and keystroke timing data, captured passively from 70 PD patients and healthy controls (HC), in their daily life via the iPrognosis Android smartphone application, for relevant motor symptoms severity inference. The proposed methods for motor symptoms inference show promising PD diagnostic performance in our relatively small clinically-evaluated cohorts. Our results highlight the potential of evolving these methods into an objective PD screening/monitoring tool that could support clinical diagnosis, drug response assessment and decision-making. Passive capturing of the required input data further fosters evaluation of individuals’ natural behavior, as well as long-term adherence

Parkinson’s Disease (PD) is a neurodegenerativedisorder that manifests through slowly progressing symptoms,such as tremor, voice degradation and bradykinesia. Automateddetection of such symptoms has recently received much attentionby the research community, owing to the clinical benefits associated with the early diagnosis of the disease. Unfortunately,most of the approaches proposed so far, operate under a strictlylaboratory setting, thus limiting their potential applicability inreal world conditions. In this work, we present a method for automatically detecting tremorous episodes related to PD, based on acceleration signals. We propose to address the problemat hand, as a case of Multiple-Instance Learning, wherein asubject is represented as an unordered bag of signal segmentsand a single, expert-provided, ground-truth. We employ adeep learning approach that combines feature learning and alearnable pooling stage and is trainable end-to-end. Results ona newly introduced dataset of accelerometer signals collectedin-the-wild confirm the validity of the proposed approach.&nbsp

Certain patterns of eating behaviour during meal have been identified as risk factors for long-term abnormal eating development in healthy individuals and, eventually, can affect the body weight. To detect early signs of problematic eating behaviour, this paper proposes a novel method for building behaviour assessment models. The goal of the models is to predict whether the in-meal eating behaviour resembles patterns associated with obesity, eating disorders, or low-risk behaviours. The models are trained using meals recorded with a plate scale from a reference population and labels annotated by a domain expert. In addition, the domain expert assigned scores that characterise the degree of any exhibited abnormal patterns. To improve model effectiveness, we use the domain expert's scores to create training error regularisation weights that alter the importance of each training instance for its class during model training. The behaviour assessment models are based on the SVM algorithm and the fuzzy SVM algorithm for their instance-weighted variation. Experiments conducted on meals recorded from 120 individuals show that: (a) the proposed approach can produce effective models for eating behaviour classification (for individuals), or for ranking (for populations); and (b) the instance-weighted fuzzy SVM models achieve significant performance improvements, compared to the non-weighted, standard SVM models.

BigO (bigoprogram.eu) is an EU-funded project that collects objective evidence on the causes of obesity in local communities and helps public health authorities design effective counter obesity interventions. A novel technological platform is being built relying on mobile devices and sensors for data acquisition combined with big data analytics and visualization. During the 4 year project duration the BigO platform will be used by 9000 school and age-matched obese children and adolescents as sources for community data. Led by Aristotle University of Thessaloniki, the project brings together schools, health and clinical scientists, technology providers, personal health solutions businesses and mobile communication providers in Greece, Sweden, Ireland, Spain and the Netherlands.

Clickthrough data from image search engines provide a massive and continuously generated source of user feedback that can be used to model how the search engine users perceive the visual content. Image clickthrough data have been successfully used to build concept detectors without any manual annotation effort, although the generated annotations suffer from labeling errors. Previous research efforts therefore focused on modeling the sample uncertainty in order to improve concept detector effectiveness. In this paper, we study the problem in an online learning setting using streaming clickthrough data where each click is treated seperately when it becomes available; the concept detector model is therefore continuously updated without batch retraining. We argue that sample uncertainty can be incorporated in the online learning setting by exploiting the repetitions of incoming clicks at the classifier level, where these act as an implicit importance weighting mechanism. For online concept detector training we use the LASVM algorithm. The inferred weighting approximates the solution of batch trained concept detectors using weighted SVM variants that are known to achieve improved performance and high robustness to noise compared to the standard SVM. Furthermore, we evaluate methods for selecting negative samples using a small number of candidates sampled locally from the incoming stream of clicks. The selection criteria aim at drastically improving the performance and the convergence speed of the online concept detectors. To validate our arguments we conduct experiments for 30 concepts on the Clickture-Lite dataset. The experimental results demonstrate that: (a) the proposed online approach produces effective and noise resilient concept detectors that can take advantage of streaming clickthrough data and achieve performance that is equivalent to Fuzzy SVM concept detectors with sample weights and 78.6% improved compared to standard SVM concept detectors; and (b) the selection criteria speed up convergence and improve effectiveness compared to random negative sampling even for a small number of available clicks (up to 134% after 100 clicks). HighlightsOnline training of concept detectors on streaming clickthrough data for image retrieval.Continuous model update based on the stream of incoming clicks.Implicit importance weighting of training samples based on the number of clicks.Online concept detectors demonstrate high effectiveness and noise robustness.Sample selection strategies boost the performance and speedup convergence.

Overweight and obesity are both associated with in-meal eating parameters such as eating speed. Recently, the plethora of available wearable devices in the market ignited the interest of both the scientific community and the industry toward unobtrusive solutions for eating behavior monitoring. In this paper, we present an algorithm for automatically detecting the in-meal food intake cycles using the inertial signals (acceleration and orientation velocity) from an off-the-shelf smartwatch. We use five specific wrist micromovements to model the series of actions leading to and following an intake event (i.e., bite). Food intake detection is performed in two steps. In the first step, we process windows of raw sensor streams and estimate their micromovement probability distributions by means of a convolutional neural network. In the second step, we use a long short-term memory network to capture the temporal evolution and classify sequences of windows as food intake cycles. Evaluation is performed using a challenging dataset of 21 meals from 12 subjects. In our experiments, we compare the performance of our algorithm against three state-of-the-art approaches, where our approach achieves the highest F1 detection score (0.913 in the leave-one-subject-out experiment). The dataset used in the experiments is available at https://mug.ee.auth.gr/intake-cycle-detection/ .

The way we eat and what we eat, the way we move and the way we sleep significantly impact the risk of becoming obese. These aspects of behavior decompose into several personal behavioral elements including our food choices, eating place preferences, transportation choices, sleeping periods and duration etc. Most of these elements are highly correlated in a causal way with the conditions of our local urban, social, regulatory and economic environment. To this end, the H2020 project "BigO: Big Data Against Childhood Obesity" (http://bigoprogram.eu) aims to create new sources of evidence together with exploration tools, assisting the Public Health Authorities in their effort to tackle childhood obesity. In this paper, we present the technology-based methodology that has been developed in the context of BigO in order to: (a) objectively monitor a matrix of a population's obesogenic behavioral elements using commonly available wearable sensors (accelerometers, gyroscopes, GPS), embedded in smart phones and smart watches; (b) acquire information for the environment from open and online data sources; (c) provide aggregation mechanisms to correlate the population behaviors with the environmental characteristics; (d) ensure the privacy protection of the participating individuals; and (e) quantify the quality of the collected big data., This paper has been accepted for publication at the Statistical Journal of the International Association for Official Statistics, https://content.iospress.com/journals/statistical-journal-of-the-iaos

In this paper, we propose an end-to-end neural network (NN) architecture for detecting in-meal eating events (i.e., bites), using only a commercially available smartwatch. Our method combines convolutional and recurrent networks and is able to simultaneously learn intermediate data representations related to hand movements, as well as sequences of these movements that appear during eating. A promising F-score of 0.884 is achieved for detecting bites on a publicly available dataset with 10 subjects.

Automated monitoring and analysis of eating behaviour patterns, i.e., "how one eats", has recently received much attention by the research community, owing to the association of eating patterns with health-related problems and especially obesity and its comorbidities. In this work, we introduce an improved method for meal micro-structure analysis. Stepping on a previous methodology of ours that combines feature extraction, SVM micro-movement classification and LSTM sequence modelling, we propose a method to adapt a pretrained IMU-based food intake cycle detection model to a new subject, with the purpose of improving model performance for that subject. We split model training into two stages. First, the model is trained using standard supervised learning techniques. Then, an adaptation step is performed, where the model is fine-tuned on unlabeled samples of the target subject via semisupervised learning. Evaluation is performed on a publicly available dataset that was originally created and used in [1] and has been extended here to demonstrate the effect of the semisupervised approach, where the proposed method improves over the baseline method.

This article presents a novel approach to training classifiers for concept detection using tags and a variant of Support Vector Machine that enables the usage of training weights per sample. Combined with an appropriate tag weighting mechanism, more relevant samples play a more important role in the calibration of the final concept-detector model. We propose a complete, automated framework that (i) calculates relevance scores for each image-concept pair based on image tags, (ii) transforms the scores into relevance probabilities and automatically annotates each image according to this probability, (iii) transforms either the relevance scores or the probabilities into appropriate training weights and finally, (iv) incorporates the training weights and the visual features into a Fuzzy Support Vector Machine classifier to build the concept-detector model. The framework can be applied to online public collections, by gathering a large pool of diverse images, and using the calculated probability to select a training set and the associated training weights. To evaluate our argument, we experiment on two large annotated datasets. Experiments highlight the retrieval effectiveness of the proposed approach. Furthermore, experiments with various levels of annotation error show that using weights derived from tags significantly increases the robustness of the resulting concept detectors.

Video segmentation is a fundamental problem in computer vision and aims to extract meaningful entities from a video. One of the most useful cues in this quest is motion as is described by the trajectories of tracked points. In this paper we present a motion segmentation method attempting to address some of the major issues in the area. Namely, we propose an efficient framework where more complex motion models can be seamlessly integrated both maintaining computational tractability and not penalizing non translational motion. Moreover, we expose in depth the problem of object leakage due to occlusion and highlight that motion segmentation could be treated as a graph coloring problem. Our algorithm uses an approach based on graph theory and resolves occlusion cases in a robust manner. To endow our method with scalability, we follow the previously presented subsequence architecture and test it in a streaming setup. Extensive experiments demonstrate the flexibility and robustness of the method. The segmentation results are competitive compared to the state of the art. Display Omitted Each video is divided in sub-sequences.Motion models are extracted for each video sub-sequence.The model error distribution and the model ranking for each trajectory are used to correlate different trajectories.Occlusions cause segmentation leakages when sub-sequences are merged.To avoid this effect we model segmentation as a graph coloring problem.

Clickthrough data is a source of information that can be used for automatically building concept detectors for image retrieval. Previous studies, however, have shown that in many cases the resulting training sets suffer from severe label noise that has a significant impact in the SVM concept detector performance. This paper evaluates and proposes a set of strategies for automatically building effective concept detectors from clickthrough data. These strategies focus on: (1) automatic training set generation; (2) assignment of label confidence weights to the training samples and (3) using these weights at the classifier level to improve concept detector effectiveness. For training set selection and in order to assign weights to individual training samples three Information Retrieval (IR) models are examined: vector space models, BM25 and language models. Three SVM variants that take into account importance at the classifier level are evaluated and compared to the standard SVM: the Fuzzy SVM, the Power SVM, and the Bilateral-weighted Fuzzy SVM. Experiments conducted on the MM Grand Challenge dataset (consisting of 1M images and 82.3M unique clicks) for 40 concepts demonstrate that (1) on average, all weighted SVM variants are more effective than the standard SVM; (2) the vector space model produces the best training sets and best weights; (3) the Bilateral-weighted Fuzzy SVM produces the best results but is very sensitive to weight assignment and (4) the Fuzzy SVM is the most robust training approach for varying levels of label noise.

This paper introduces a family of local feature aggregation functions and a novel method to estimate their parameters, such that they generate optimal representations for classification (or any task that can be expressed as a cost function minimization problem). To achieve that, we compose the local feature aggregation function with the classifier cost function and we backpropagate the gradient of this cost function in order to update the local feature aggregation function parameters. Experiments on synthetic datasets indicate that our method discovers parameters that model the class-relevant information in addition to the local feature space. Further experiments on a variety of motion and visual descriptors, both on image and video datasets, show that our method outperforms other state-of-the-art local feature aggregation functions, such as Bag of Words, Fisher Vectors and VLAD, by a large margin., In Proceedings of the 25th European Signal Processing Conference (EUSIPCO 2017)

BACKGROUND The available methods for monitoring food intake—which for a great part rely on self-report—often provide biased and incomplete data. Currently, no good technological solutions are available. Hence, the SPLENDID eating detection sensor (an ear-worn device with an air microphone and a photoplethysmogram [PPG] sensor) was developed to enable complete and objective measurements of eating events. The technical performance of this device has been described before. To date, literature is lacking a description of how such a device is perceived and experienced by potential users. OBJECTIVE The objective of our study was to explore how potential users perceive and experience the SPLENDID eating detection sensor. METHODS Potential users evaluated the eating detection sensor at different stages of its development: (1) At the start, 12 health professionals (eg, dieticians, personal trainers) were interviewed and a focus group was held with 5 potential end users to find out their thoughts on the concept of the eating detection sensor. (2) Then, preliminary prototypes of the eating detection sensor were tested in a laboratory setting where 23 young adults reported their experiences. (3) Next, the first wearable version of the eating detection sensor was tested in a semicontrolled study where 22 young, overweight adults used the sensor on 2 separate days (from lunch till dinner) and reported their experiences. (4) The final version of the sensor was tested in a 4-week feasibility study by 20 young, overweight adults who reported their experiences. RESULTS Throughout all the development stages, most individuals were enthusiastic about the eating detection sensor. However, it was stressed multiple times that it was critical that the device be discreet and comfortable to wear for a longer period. In the final study, the eating detection sensor received an average grade of 3.7 for wearer comfort on a scale of 1 to 10. Moreover, experienced discomfort was the main reason for wearing the eating detection sensor CONCLUSIONS The SPLENDID eating detection sensor, which uses an air microphone and a PPG sensor, is a promising new device that can facilitate the collection of reliable food intake data, as shown by its technical potential. Potential users are enthusiastic, but to be successful wearer comfort and discreetness of the device need to be improved.

Weighted SVM (or fuzzy SVM) is the most widely used SVM variant owning its effectiveness to the use of instance weights. Proper selection of the instance weights can lead to increased generalization performance. In this work, we extend the span error bound theory to weighted SVM and we introduce effective hyperparameter selection methods for the weighted SVM algorithm. The significance of the presented work is that enables the application of span bound and span-rule with weighted SVM. The span bound is an upper bound of the leave-one-out error that can be calculated using a single trained SVM model. This is important since leave-one-out error is an almost unbiased estimator of the test error. Similarly, the span-rule gives the actual value of the leave-one-out error. Thus, one can apply span bound and span-rule as computationally lightweight alternatives of leave-one-out procedure for hyperparameter selection. The main theoretical contributions are: (a) we prove the necessary and sufficient condition for the existence of the span of a support vector in weighted SVM; and (b) we prove the extension of span bound and span-rule to weighted SVM. We experimentally evaluate the span bound and the span-rule for hyperparameter selection and we compare them with other methods that are applicable to weighted SVM: the $K$-fold cross-validation and the ${\xi}-{\alpha}$ bound. Experiments on 14 benchmark data sets and data sets with importance scores for the training instances show that: (a) the condition for the existence of span in weighted SVM is satisfied almost always; (b) the span-rule is the most effective method for weighted SVM hyperparameter selection; (c) the span-rule is the best predictor of the test error in the mean square error sense; and (d) the span-rule is efficient and, for certain problems, it can be calculated faster than $K$-fold cross-validation., Comment: 35 pages, 6 figures

(1) Background: Evidence-based policymaking requires data about the local population’s socioeconomic status (SES) at detailed geographical level, however, such information is often not available, or is too expensive to acquire. Researchers have proposed solutions to estimate SES indicators by analyzing Google Street View images, however, these methods are also resource-intensive, since they require large volumes of manually labeled training data. (2) Methods: We propose a methodology for automatically computing surrogate variables of SES indicators using street images of parked cars and deep multiple instance learning. Our approach does not require any manually created labels, apart from data already available by statistical authorities, while the entire pipeline for image acquisition, parked car detection, car classification, and surrogate variable computation is fully automated. The proposed surrogate variables are then used in linear regression models to estimate the target SES indicators. (3) Results: We implement and evaluate a model based on the proposed surrogate variable at 30 municipalities of varying SES in Greece. Our model has R 2 = 0.76 and a correlation coefficient of 0.874 with the true unemployment rate, while it achieves a mean absolute percentage error of 0.089 and mean absolute error of 1.87 on a held-out test set. Similar results are also obtained for other socioeconomic indicators, related to education level and occupational prestige. (4) Conclusions: The proposed methodology can be used to estimate SES indicators at the local level automatically, using images of parked cars detected via Google Street View, without the need for any manual labeling effort.

Detecting chewing sounds from a microphone placed inside the outer ear for eating behaviour monitoring still remains a challenging task. This is mainly due the difficulty in discriminating non-chewing sounds (e.g. speech or sounds caused by walking) from chews, as well as due to to the high variability of the chewing sounds of different food types. Most approaches rely on detecting distictive structures on the sound wave, or on extracting a set of features and using a classifier to detect chews. In this work, we propose to use feature-learning in the time domain with 1-dimensional convolutional neural networks for for chewing detection. We apply a network of convolutional layers followed by fully connected layers directly on windows of the audio samples to detect chewing activity, and then aggregate individual chews to eating events. Experimental results on a large, semi-free living dataset collected in the context of the SPLENDID project indicate high effectiveness, with an accuracy of 0.980 and F1 score of 0.883.

Automatic objective monitoring of eating behavior using inertial sensors is a research problem that has received a lot of attention recently, mainly due to the mass availability of IMUs and the evidence on the importance of quantifying and monitoring eating patterns. In this paper we propose a method for detecting food intake cycles during the course of a meal using a commercially available wristband. We first model micro-movements that are part of the intake cycle and then use HMMs to model the sequences of micro-movements leading to mouthfuls. Evaluation is carried out on an annotated dataset of 8 subjects where the proposed method achieves 0:78 precision and 0:77 recall. The evaluation dataset is publicly available at http://mug.ee.auth.gr/intake-cycle-detection/.

Monitoring of eating behavior using wearable technology is receiving increased attention, driven by the recent advances in wearable devices and mobile phones. One particularly interesting aspect of eating behavior is the monitoring of chewing activity and eating occurrences. There are several chewing sensor types and chewing detection algorithms proposed in the bibliography, however no datasets are publicly available to facilitate evaluation and further research. In this paper, we present a multi-modal dataset of over 60 hours of recordings from 14 participants in semi-free living conditions, collected in the context of the SPLENDID project. The dataset includes raw signals from a photoplethysmography (PPG) sensor and a 3D accelerometer, and a set of extracted features from audio recordings; detailed annotations and ground truth are also provided both at eating event level and at individual chew level. We also provide a baseline evaluation method, and introduce the 'challenge' of improving the baseline chewing detection algorithms. The dataset can be downloaded from http: //dx.doi.org/10.17026/dans-zxw-v8gy, and supplementary code can be downloaded from https://github. com/mug-auth/chewing-detection-challenge.git.

We present a novel method that evaluates the geometric consistency of putative point matches in weakly calibrated settings, i.e. when the epipolar geometry but not the camera calibration is known, using only the point coordinates as information. The main idea behind our approach is the fact that each point correspondence in our data belongs to one of two classes (inliers/outlier). The classification of each point match relies on the histogram of a quantity representing the difference between cross ratios derived from a construction involving 6-tuples of point matches. Neither constraints nor scenario dependent parameters/thresholds are needed. Even for few candidate point matches the ensemble of 6-tuples containing each of them turns to provide statistically reliable histograms that prove to discriminate between inliers and outliers. In fact, in most cases a random sampling among this population is sufficient. Nevertheless, the accuracy of the method is positively correlated to its sampling density leading to an accuracy versus resulting computational complexity trade-off. Theoretical analysis and experiments are given that show the consistent performance of the proposed classification method when applied in inlier/outlier discrimination. The achieved accuracy is favourably evaluated against established methods that employ geometric only information, i.e. those relying on the Sampson, the algebraic and the symmetric epipolar distances. Finally, we also present an application of our scheme in uncalibrated stereo inside a RANSAC framework and compare it to the same as above methods.

This paper presents a method for the recognition of the six basic facial expressions in images or in image sequences using landmark points. The proposed technique relies on the observation that the vectors formed by the landmark point coordinates belong to a different manifold for each of the expressions. In addition experimental measurements validate the hypothesis that each of these manifolds can be decomposed to a small number of linear subspaces of very low dimension. This yields a parameterization of the manifolds that allows for computing the distance of a feature vector from each subspace and consequently from each one of the six manifolds. Two alternative classifiers are next proposed that use the corresponding distances as input: the first one is based on the minimum distance from the manifolds, while the second one uses SVMs that are trained with the vector of all distances from each subspace. The proposed technique is tested for two scenarios, the subject-independent and the subject-dependent one. Extensive experiments for each scenario have been performed on two publicly available datasets yielding very satisfactory expression recognition accuracy.

Motion segmentation refers to the problem of separating the objects in a video sequence according to their motion. It is a fundamental problem of computer vision, since various systems focusing on the analysis of dynamic scenes include motion segmentation algorithms. In this paper we present a novel approach, where a video shot is temporally divided in successive and overlapping windows and motion segmentation is performed on each window respectively. This attribute renders the algorithm suitable even for long video sequences. In the last stage of the algorithm the segmentation results for every window are aggregated into a final segmentation. The presented algorithm can handle effectively asynchronous trajectories on each window even when they have no temporal intersection. The evaluation of the proposed algorithm on the Berkeley motion segmentation benchmark demonstrates its scalability and accuracy compared to the state of the art.

In this paper we present a research system that follows a semantic approach to facilitate medical association studies in the area of cervical cancer. Our system, named ASSIST and developed as an EU research project, assists in cervical cancer research by unifying multiple patient record repositories, physically located in different medical centers or hospitals. Semantic modeling of medical data and rules for inferring domain-specific information allow the system to (i) homogenize the information contained in the isolated repositories by translating it into the terms of a unified semantic representation, (ii) extract diagnostic information not explicitly stored in the individual repositories, and (iii) automate the process of evaluating medical hypotheses by performing case–control association studies, which is the ultimate goal of the system.

Unobtrusive analysis of eating behavior based on Inertial Measurement Unit (IMU) sensors (e.g. accelerometer) is a topic that has attracted the interest of both the industry and the research community over the past years. This work presents a method for detecting food intake moments that occur during a meal session using the accelerometer and gyroscope signals of an off-the-shelf smartwatch. We propose a two step approach. First, we model the hand micro-movements that take place while eating using an array of binary Support Vector Machines (SVMs); then the detection of intake moments is achieved by processing the sequence of SVM score vectors by a Long Short Term Memory (LSTM) network. Evaluation is performed on a publicly available dataset with 10 subjects, where the proposed method outperforms similar approaches by achieving an F1 score of 0.892.