Search

Abstract An interconnected system employing Kinect Azure and Fitbit Sense for continuous and non-intrusive data collection was used in the PRO-HOME protected discharge program, aiming at monitoring functional and clinical parameters in hospitalized older patients at different risks of frailty. The present study shows the findings on 30 older patients included in the PRO-HOME project. The Fitbit Sense recorded the mean daily and hourly number of steps, mean daily walked distance, and time spent inactive. Moreover, Kinect infrared camera captured gait speed and daily mean latero-lateral (body sway) and antero-posterior oscillations (lean-in). Patients underwent a standard Comprehensive Geriatric Assessment (CGA) to compute the Multidimensional Prognostic Index (MPI), including basic and instrumental activities of daily living (ADL, IADL), cognition (Short Portable Mental Status Questionnaire, SPMSQ) and nutrition, risk of pressure sores (Exton-Smith Scale, ESS), comorbidity, number of drugs and cohabitation status. Significant correlations between the mean hourly number of steps and MPI (p = 0.022), IADL (p = 0.013), SPMSQ (p = 0.006), ESS (p = 0.009), and both mean and maximum automated gait speed (p = 0.046 and p = 0.048) were found. Automated gait speed was also correlated with mean walked distance per day (p = 0.007) and lean-in (p = 0.047). Domotic technological monitoring through Fitbit Sense and Kinect Azure provides information on multidimensional frailty, including mobility and cognitive and functional status, in older people.

Abstract Backgroud Discharge planning from the hospital of frail older patients is an important step to avoid inappropriate long‐stay hospitalizations and to prevent the risks related to the prolonged hospitalization. In this frame, we developed an experimental trial—‘PRO‐HOME’, a multicomponent programme of interventions for multimorbid and polytreated hospitalized older patients. Aim The main aim of the study was to develop a protected discharge facility using a mini apartment equipped with advanced architectural and technological components to reduce the length of hospital stay of older participants (aged 65+ years old) admitted to the hospital for an acute event, deemed stable and dischargeable. Materials and Methods This is a pilot randomized controlled study, comparing 30 hospitalized participants included in a multidimensional, transitional care programme based on information and communication technologies to 30 patients in standard usual care until hospital discharge. Results We presented the study design of the PRO‐HOME programme, including architectural and technological components, the enrolment procedures, the components of the intervention that is physical activity, cognitive training and life‐style education and the evaluation method of the intervention based on the Comprehensive Geriatric Assessment to explore the changes in the individual domains that are target of the multicomponent intervention. Conclusions The final results will suggest whether the PRO‐HOME programme represents a useful and feasible intervention to reduce the length of hospital stay of multimorbid and polytreated hospitalized older patients and improve their physical and cognitive performances and overall quality of life. Patient or Public Contribution Due to the characteristics of the population of interest of the PRO‐HOME study, we involved in the study design and programme of the activities the participants enrolled in a previous smart home‐based project named MoDiPro carried‐out during a 3‐year period. The elderly participants from the local population involved were asked, by means of focus groups, for feedback on their experience in MoDiPro, and their suggestions were integrated into the design phase of the current PRO‐HOME project. The focus groups included open group interviews with a qualitative collection of the patients' feedback so that the participants could interact with each other.

Innovative applications of human movement analysis, for example, for mitigating/slowing down certain pathological conditions, have recently emerged from the modeling and automated measurement of full-body expressive midlevel individual and group movement qualities, at a higher complexity level than movement qualities derived directly from physical signals, still not characterizing any gesture in a specific way. More in general, the availability of automated analysis techniques of midlevel expressive movement qualities can contribute to interaction design incorporating body-based performance practices inspired by artistic theories in dance and music. This work investigates how such practices and techniques can support embodied interaction design by enabling automated measuring of cues of leadership, cohesion, and fluidity in full-body movement in group settings. In particular, the dance-inspired scientific approach, the data collection protocol, and the analysis techniques adopted for assessing movement qualities connected to leadership and cohesion within the group and fluidity of the dancers’ full-body movement are described. Finally, future developments of this research are outlined.

Music ensemble performance provides an ecologically valid context for investigating leadership dynamics in small group interactions. Musical texture, specifically the relative salience of simultaneously sounding ensemble parts, is a feature that can potentially alter leadership dynamics by introducing hierarchical relationships between individual parts. The present study extended previous work on quantifying interpersonal coupling in musical ensembles by examining the relationship between musical texture and leader-follower relations, operationalised as directionality of influence between co-performers' body motion in concert video recordings. It was hypothesised that the directionality of influence, indexed by Granger Causality, would be greater for 'homophonic' textures with a clear distinction between melody and accompaniment parts than for 'polyphonic' textures with less distinction between melody and accompaniment. This hypothesis was tested by using pose estimation algorithms to track instrumentalists' body movements in a string quartet and a clarinet quintet, and then applying Granger Causality analysis to their head motion to estimate directional influence between instrumentalist pairs for sections of the pieces that varied in texture. It was found that Granger Causality values were generally higher (indicating greater directionality of influence) for homophonic than polyphonic textures. Furthermore, considering melody and accompaniment instrument roles revealed more evidence for the melody instrument influencing accompanying instruments than vice versa, plus a high degree of directionality among accompanying instruments, in homophonic textures. These observed patterns of directional information flow in co-performer body motion are consistent with changing leader-follower relations depending on hierarchical relations between ensemble parts in terms of the relative salience of melodic material in the musical texture. The finding that automatic pose estimation can detect modulations of leadership dynamics in standard video recordings under naturalistic performance conditions has implications for investigating interpersonal coordination in large-scale music video datasets representing different cultural traditions, and for exploring nonverbal communication in group activities more generally.

IntroductionThe World Health Organization (WHO) acknowledges the presence of a significant body of research on the positive effects of the arts on health, considering a variety of factors including physical well-being, quality of life, and social and community impact. The model that underlies cultural welfare puts the performing arts, visual arts, and cultural heritage at the service of people personal and societal well-being. The potential connections between movements of the body and artistic content have been extensively studied over time, considering movement as a non-verbal language with a universal character.MethodsThis pilot study presents the results of the validation of an innovative multimodal system, the DanzArTe-Emotional Wellbeing Technology, designed to support active and participative experience of older people providing physical and cognitive activation through a full-body physical interaction with a traditional visual work of art of religious subject. DanzArTe supports a replicable treatment protocol for multidimensional frailty, administered through a low cost and scalable technological platform capable of generating real-time visual and auditory feedback (interactive sonification) from the automated analysis of individual as well as joint movement expressive qualities. The study involved 45 participants, 23 of whom participated in the DanzArTe program and 22 who were included in the control group.ResultsThe two groups were similar in terms of age (p = 0.465) and gender (p = 0.683). The results showed that the DanzArTe program had a positive impact on participants' self-perceived psychological health and well-being (Mean Psychological General Well-Being Index—Short T1 = 19.6 ± 4.3 Vs. T2 = 20.8 ± 4.9; p = 0.029). The same trend was not observed in the control group (p = 0.389).DiscussionThe findings suggest that such programs may have a significant impact particularly on the mental and social well-being of older adults and could be a valuable tool for promoting healthy aging and improving quality of life.

Human communication entails subtle non-verbal modes of expression, which can be analyzed quantitatively using computational approaches and thus support human sciences. In this paper we present huSync, a computational framework and system that utilizes trajectory information extracted using pose estimation algorithms from video sequences to quantify synchronization between individuals in small groups. The system is exploited to study interpersonal coordination in musical ensembles. Musicians communicate with each other through sounds and gestures, providing nonverbal cues that regulate interpersonal coordination. huSync was applied to recordings of concert performances by a professional instrumental ensemble playing two musical pieces. We examined effects of different aspects of musical structure (texture and phrase position) on interpersonal synchronization, which was quantified by computing phase locking values of head motion for all possible within-group pairs. Results indicate that interpersonal coupling was stronger for polyphonic textures (ambiguous leadership) than homophonic textures (clear melodic leader), and this difference was greater in early portions of phrases than endings (where coordination demands are highest). Results were cross-validated against an analysis of audio features, showing links between phase locking values and event density. This research produced a system, huSync, that can quantify synchronization in small groups and is sensitive to dynamic modulations of interpersonal coupling related to ambiguity in leadership and coordination demands, in standard video recordings of naturalistic human group interaction. huSync enabled a better understanding of the relationship between interpersonal coupling and musical structure, thus enhancing collaborations between human and computer scientists.

The Interpersonal Entrainment in Music Performance Data Collection (IEMPDC) comprises six related corpora of music research materials: Cuban Son & Salsa (CSS), European String Quartet (ESQ), Malian Jembe (MJ), North Indian Raga (NIR), Tunisian Stambeli (TS), and Uruguayan Candombe (UC). The core data for each corpus comprises media files and computationally extracted event onset timing data. Annotation of metrical structure and code used in the preparation of the collection is also shared. The collection is unprecedented in size and level of detail and represents a significant new resource for empirical and computational research in music. In this article we introduce the main features of the data collection and the methods used in its preparation. Details of technical validation procedures and notes on data visualization are available as Appendices. We also contextualize the collection in relation to developments in Open Science and Open Data, discussing important distinctions between the two related concepts.

Social behaviors rely on the coordination of multiple effectors within one’s own body as well as between the interacting bodies. However, little is known about how coupling at the interpersonal level impacts coordination among body parts at the intrapersonal level, especially in ecological, complex, situations. Here, we perturbed interpersonal sensorimotor communication in violin players of an orchestra and investigated how this impacted musicians’ intrapersonal movements coordination. More precisely, first section violinists were asked to turn their back to the conductor and to face the second section of violinists, who still faced the conductor. Motion capture of head and bow kinematics showed that altering the usual interpersonal coupling scheme increased intrapersonal coordination. Our perturbation also induced smaller yet more complex head movements, which spanned multiple, faster timescales that closely matched the metrical levels of the musical score. Importantly, perturbation differentially increased intrapersonal coordination across these timescales. We interpret this behavioral shift as a sensorimotor strategy that exploits periodical movements to effectively tune sensory processing in time and allows coping with the disruption in the interpersonal coupling scheme. As such, head movements, which are usually deemed to fulfill communicative functions, may possibly be adapted to help regulate own performance in time.

We use functional Magnetic Resonance Imaging (fMRI) to explore synchronized neural responses between observers of audiovisual presentation of a string quartet performance during free viewing. Audio presentation was accompanied by visual presentation of the string quartet as stick figures observed from a static viewpoint. Brain data from 18 musical novices were obtained during audiovisual presentation of a 116 s performance of the allegro of String Quartet, No. 14 in D minor by Schubert played by the ‘Quartetto di Cremona.’ These data were analyzed using intersubject correlation (ISC). Results showed extensive ISC in auditory and visual areas as well as parietal cortex, frontal cortex and subcortical areas including the medial geniculate and basal ganglia (putamen). These results from a single fixed viewpoint of multiple musicians are greater than previous reports of ISC from unstructured group activity but are broadly consistent with related research that used ISC to explore listening to music or watching solo dance. A feature analysis examining the relationship between brain activity and physical features of the auditory and visual signals yielded findings of a large proportion of activity related to auditory and visual processing, particularly in the superior temporal gyrus (STG) as well as midbrain areas. Motor areas were also involved, potentially as a result of watching motion from the stick figure display of musicians in the string quartet. These results reveal involvement of areas such as the putamen in processing complex musical performance and highlight the potential of using brief naturalistic stimuli to localize distinct brain areas and elucidate potential mechanisms underlying multisensory integration.

A multimodal dataset is presented, which has been collected for analyzing and measuring the quality of movement performed during sport activities. Martial arts (namely karate) are taken as test-beds for investigation. Karate encompasses predefined sequences of movements (“katas”) that can be carried out with different qualities, e.g., by performers at different skill levels (highly vs. poorly skilled).The experimental setup and method are described. The dataset is composed of motion capture (MoCap) data, synchronized with video and audio recordings, of several participants with different levels of experience. The raw MoCap data are independent of any particular post-processing algorithm and can be used for other research purposes. In the second part of the paper, a set of measures is proposed to evaluate a kata performance. They are based on the geometrical and kinematic features, such as posture correctness and synchronization between limbs. and were chosen according to karate experts’ opinion.

This paper addresses the use of a remote interactive platform to support home-based rehabilitation for children with motor and cognitive impairment. The interaction between user and platform is achieved on customizable full-body interactive serious games (exergames). These exergames perform real-time analysis of multimodal signals to quantify movement qualities and postural attitudes. Interactive sonification of movement is then applied for providing a real-time feedback based on “aesthetic resonance” and engagement of the children. The games also provide log file recordings therapists can use to assess the performance of the children and the effectiveness of the games. The platform allows the customization of the games to address the children’s needs. The platform is based on the EyesWeb XMI software, and the games are designed for home usage, based on Kinect for Xbox One and simple sensors including 3-axis accelerometers available in low-cost Android smartphones.

An emerging trend in interactive music performance consists of the audience directly participating in the performance by means of mobile devices. This is a step forward with respect to concepts like active listening and collaborative music making: non-expert members of an audience are enabled to directly participate in a creative activity such as the performance. This requires the availability of technologies for capturing and analysing in real-time the natural behaviour of the users/performers, with particular reference to non- verbal expressive and social behaviour. This paper presents a prototype of a non-verbal expressive and social search engine and active listening system, enabling two teams of non-expert users to act as performers. The performance consists of real-time sonic manipulation and mixing of music pieces selected according to features characterising performers’ movements captured by mobile devices. The system is described with specific reference to the SIEMPRE Podium Performance, a non-verbal socio-mobile music performance presented at the Art & ICT Exhibition that took place in Vilnius (LI) in November 2013.

Our understanding of the perceived actions of those around us includes an ability to segment this continuous stream of activity into discrete events. We studied naïve observers' abilities to segment a video of an unfamiliar dance style into events using a combination of behavioural, computational vision and brain imaging methods. A 386 s video of a solo Bharatanatyam dancer was used as the basis for the study. A computational analysis provided us with, for every video frame, a Motion Index (MI) quantifying the movement of the entire dancer. A behavioural analysis using 30 naïve observers provided us with the time points where observers were most likely to place an event boundary. These behavioural and computational data were used to interpret the brain activity of another 11 participants who viewed the dance video while in an MRI scanner. Results showed that the Motion Index predicted brain activity in a single cluster in the right hemisphere that was located close to the Extrastriate Body Area (EBA). Event boundaries in the video were related to extensive clusters of bilateral activity in the Inferior Occipital Gyrus which extended towards the posterior Superior Temporal Sulcus (pSTS). Event boundaries also activated a region in the right Inferior Frontal Gyrus. These results extend our understanding of how movement kinaesthetics modulate action interpretation.