Search

Previous studies suggest that engagement in musical activities may enhance well-being and impact social inclusion. However, unfortunately, people with physical disabilities cannot often use musical instruments or music production software due to accessibility issues. We propose Boris, an original conversational agent specific for people with a physical disability, to entertain, stimulate expressiveness, and promote communication. Boris enables (even inexperienced) users to compose songs through hands-free interaction by analyzing their vocalizations to obtain more than just their transcription: the system listens to the user even while humming a song and generates a melody by learning and reproducing their human voice patterns. Indeed, it exploits an artificial musical intelligence that can imitate the typically human cognitive skills to produce music using an advanced technique called abstract melody.

In the last years, researchers and therapists have pinpointed a number of critical aspects in current speech-language interventions. Several studies have explored the use of technology to overcome these barriers and to support speech-therapy in children with language impairments (e.g. DLD and ASD). In this paper, we propose a conceptual framework for designing linguistic activities (for assessment and training), based on advances in psycholinguistics. Moving from this theoretical framework, we identified a development process - from the UX design to coding of activities – which is based on a novel set of Design Patterns at multiple layers of abstraction. We then put this framework into practice by implementing these patterns into two technological solutions – tablet and robots – and performing an empirical study to evaluate their benefits. Our results, although still preliminary since they assessed only a tablet experimental condition, are promising for extending the identified Design Patterns to other technological solutions.

In literature a lot has been done to investigate the possible advantages of integrating the virtual and physical world, combining the benefits of both and improving the overall user experience. Plenty of tangible user interfaces were designed, to facilitate the development of cognitive skills, motor skills, social skills, or more concrete notions such as STEM concepts for children. Also, thanks to the flexibility provided by digital systems, it is recently growing a sub-research field that specifically addressed people with a cognitive disability. In this scenario, immersive multi-sensory environments are gaining more and more interest as a support tool for learning and therapeutic methods. In this paper, we present Smart Buckets, an interactive interface integrated into "Magika", a multi-sensory system; the result of co-design with therapists specialized in the treatment and education of children with a cognitive disability. The paper illustrates the multi-step co-design process used to produce this idea as a tailored support for their therapeutic activities. Finally, it highlights the importance of adding a new customizable tangible interaction combined within the multi-sensory experience of Magika system.

Interactive Multi-Sensory Environments (iMSEs) are room-sized interactive installations equipped with digitally enriched physical materials and ambient embedded devices. These items can sense users' presence, gestures, movements, and manipulation, and react by providing gentle stimulation (e.g., light, sound, projections, blowing bubbles, tactile feel, aromas) to different senses. Most of prior research on iMSEs investigates their use for persons with disabilities (e.g., autism). Our work focuses on the use of iMSEs in primary education contexts and for mixed groups of young students, i.e., children with and without disability. The paper describes the latest version of an iMSE called Magic Room that has been installed in two local schools. We report two empirical studies devoted to understand how the Magic Room could be used in inclusive educational settings, and to explore its potential benefits.

In this paper we present Do You Speak Holo?, an application for Microsoft HoloLens co-designed with linguistics experts that aims to facilitate the learning process of a foreign language for children and adolescents with language disorders (LD). The application includes several educational activities, aimed at improving both vocabulary and morpho-syntactic aspects of the language through the interaction with virtual content immersed in the real world. The application includes a virtual assistant, which acts as a virtual teacher and guides the user step by step in the comprehension and execution of the activities.

Music production software often has complex interfaces and needs the user to know the basic musical know-how. In this paper, we present a conversational agent that allows creating music in a simplified way through voice-based interaction. Indeed, our agent can be configured and customized with simple and natural voice commands. In addition, it has some typically human cognitive skills to produce music: it listens to the user while singing a song and generates a melody by discovering and copying the patterns of her/his human voice. Technologically, the system is empowered by Google Dialogflow for conversation management and uses an advanced technique called abstract melody for music production. This Musical and Conversational Artificial Intelligence is an actual innovation since it does not require any preliminary knowledge about music and, consequently, includes professionals, but also children, beginners, and people with physical disease.

Our research explores game play in Wearable Collaborative Virtual Environments (WCVEs) to provide new forms of treatment for persons with Neurodevelopmental Disorders (NDD) that complement traditional methods of intervention. We investigate this issue using the Social MatchUP (SMUP) application as a case study. SMUP provides a set of games in WCVEs that have been co-designed with NDD experts and aim at improving communication skills. In SMUP, multiple users wearing a Virtual Reality visor play together to accomplish collaborative tasks that take place in shared virtual environments and require talking to each other to be completed. The paper presents an exploratory empirical study devoted to evaluate the potential of SMUP games for persons with NDD to improve their speech-based conversational capability. We organized participants (24 persons with NDD) in 2 groups, one playing a game in SMUP and one playing a similar game in the real world, and assessed likability, usability, and a number of conversational performance metrics. Our results indicate that the game experience in SMUP was usable and enjoyable, and boosted higher conversational skills with respect to its counterpart in the physical setting.

Mixed Reality (MR) has been receiving increasing interest in the rehabilitation of people with Cognitive Disabilities. The power of MR in the context of therapies is the possibility to maintain a physical and psychological relationship with the surrounding environment while experiencing customized multimedia content and tasks that are appropriate for the needs of these users. The purpose of this work is to explore the potential of MR applications integrated with interactive speech-based Virtual Assistants. We present HoloLearn, an application for Microsoft HoloLens, which aims to help people with Cognitive Disabilities to improve their autonomy and learn simple activities of their everyday life (e.g., setting the table). The paper discusses the design features of the Virtual Assistant created and the results of a controlled study. The participants involved were 15 subjects with Cognitive Disabilities who used HoloLearn in two experimental conditions - with and without a Virtual Assistant. The results, although preliminary, indicate that enriching a MR experience with the presence of a speech-based Virtual Assistant would improve the user performance in the execution of the MR activities.

One of the biggest challenges in Group Therapy is to track each patient’s experience and feeling without him/her noticing. Altering the familiarity of the mutual support group routine may weaken the therapeutic efficacy of the intervention. It must be avoided the “Elephant in the room’s Effect”: everyone knows is being observed and acts consequently. Therapists struggle and spend years of training on developing the skills they need to “silently” monitor all patients at the same time. From our perspective, we wonder whether and how technology can be a support for therapists in such a challenging task. More precisely, how to provide them with a non-invasive support tool that is invisible to the end-users, but at the same time ever-present for the caregivers. Basically, we asked ourselves: Can we deceive “the Elephant in the room”? Therapists may benefit from automatic measures indicating how the participants perceive the session and gathering the participants’ feedback is one path to develop valuable mutual support interventions. Our work describes the design, development and assessment of a non-invasive tool to monitor a Group Session.

This research explores the anthropomorphic perception of Emoty by its target user. Emoty is a Conversational Agent specifically designed as an emotional facilitator and trainer for individuals with neurodevelopmental disorders (NDD). NDD is a group of conditions that are characterized by severe deficits in the cognitive, emotional, and motor areas and produce severe impairments in communication and social functioning. Our application promotes skills of emotion expression and recognition and was developed in cooperation with psychologists and therapists as a supporting tool for regular interventions. We conducted an empirical study with 19 people with NDD. We observed their behavior while interacting with the system and recorded the commentaries they made and the questions they asked when the session was over. Starting from this, we discovered a twofold nature of Emoty: for some aspects, it is perceived more like a machine, but for some others, it is more human-like. In this regard, we discussed some relevant points about gender, fallibility, interaction, and sensitivity of the agent, and we paved the ground towards a better understanding of the perception of people with NDD concerning Conversational Technology.

Magika is an interactive Multisensory Environment that enables new forms of playful interventions for children, especially those with Special Education Needs. Designed in cooperation with more than 30 specialists at local care centers and primary schools, Magika integrates digital worlds projected on the wall and the floor with a gamut of "smart" physical objects (toys, ambient lights, materials, and various connected appliances) to enable tactile, auditory, visual, and olfactory stimuli. The room is connected with an interface for educators that enables them to: control the level of stimuli and their progression; define and share a countless number of game-based learning activities; customize such activities to the evolving needs of each child. This paper describes Magika and discusses its potential benefits for play, education and inclusion.

The demo introduces Reflex, a phy-gital game for children and adolescents with Neuro-Developmental Disorder. The game, offered through a cross-platform application for smartphones and tablets, bridges the digital and the physical worlds by tracking, via a bottom-looking mirror positioned on the device camera, physical items placed on a table. This new interaction paradigm, the first pilot study and its adaptability to each user profile reveals an unexplored potential for learning.

The paper introduces Reflex, a mirrored camera mobile training application for persons with Neuro-Developmental Disorder (NDD). The game, offered through a cross-platform application for smart-phones and tablets, bridges the digital and the physical worlds by tracking, via a bottom-looking mirror positioned on the device camera, physical items placed on the table. This interaction paradigm defined as phygital, its co-designed features and the first pilot study reveal an unexplored potential for learning.

This paper presents Social MatchUP, a multiplayer Virtual Reality game for children with Neurodevelopmental Disorders (NDD). Shared virtual reality environments (SVREs) allow NDD children to interact in the same virtual space, but without the possible discomfort or fear caused by having a real person in front of them. Social MatchUP is a simple Concentration-like game, run on smartphones, where players should communicate to match up all the pairs of images they are given. Because every player can only interact with half of the pictures, but can see what his companion is doing, the game improves social and communication skills, and can be used also as a learning tool. A simple and easy-to-use customization tool was also developed to let therapists and teachers adapt the game context to the needs of the children they take care of.

Our research explores social robots as learning tools for persons with Neurodevelopmental Disorder (NDD). The paper reports an empirical study that investigates engagement as a prerequisite for any learning process of NDD subjects. The study involved 5 persons in this target group and three robots (two research products developed at our lab, and a commercial one), which were used in sequence during individual therapeutic sessions at a care center. The results enable us to compare the engagement effects of different social robots and improves our understanding of the behavior of persons with NDD during robotic experiences.

Due to recent technical developments, urban systems generate large and complex data sets. While visualizations have been used to make these accessible, often they are tailored to one specific group of users, typically the public or expert users. We present Touching Transport, an application that allows a diverse group of users to visually explore public transit data on a multi-touch tabletop. It provides multiple perspectives of the data and consists of three visualization modes conveying tempo-spatial patterns as map, time-series, and arc view. We exhibited our system publicly, and evaluated it in a lab study with three distinct user groups: citizens with knowledge of the local environment, experts in the domain of public transport, and non-experts with neither lo- cal nor domain knowledge. Our observations and evaluation results show we achieved our goals of both attracting visitors to explore the data while enabling gathering insights for both citizens and experts. We discuss the design considerations in developing our system, and describe our lessons learned in designing engaging tabletop visualizations. ispartof: pages:281-288 ispartof: International Working Conference on Advanced Visual Interfaces pages:281-288 ispartof: AVI'14 location:Como, Italy date:27 May - 30 May 2014 status: published

Navigation - browsing from one related piece of information to another - is an important means of accessing information for a large class of applications: hypermedia, object-oriented databases, multimedia databases, and even traditional systems. In this article, we discuss a variety of sophisticated navigation paradigms. We distinguish among free, guided tour, and history-based navigation. In addition, we discuss integrating pure browsing with query-based access. We discuss static and dynamic models of the various navigation patterns, highlighting the fact that defining an adequate browing semantics requires a data model and a runtime model, both of which are more complex than those commonly found in hypermedia or databases. We illustrate these concepts with examples from a real-life banking application.

Acute kidney injury (AKI) is an independent risk factor for mortality in critically ill patients whose epidemiology has been made unclear in the past by the use of different definitions across various studies. The RIFLE consensus definition has provided a unifying definition for AKI leading to large retrospective studies in different countries. The present study is a prospective observational multicenter study designed to prospectively evaluate all incident admissions in 10 Intensive Care Units (ICUs) in Italy and the relevant epidemiology of AKI. A simple user-friendly web-based data collection tool was created with the scope to serve for this study and to facilitate future multicenter collaborative efforts. We enrolled 601 consecutive patients into the study; 25 patients with End-Stage Renal Disease were excluded leaving 576 patients for analysis. The median age was 66 (IQR 53-76) years, 59.4% were male, while median SAPS II and APACHE II scores were 43 (IQR 35-54) and 18 (IQR 13-24), respectively. The most common diagnostic categories for ICU admission were: respiratory (27.4%), followed by neurologic (17%), trauma (14.4%), and cardiovascular (12.1%). Crude ICU and hospital mortality were 21.7% and median ICU length of stay was 5 days (IQR 3, 14). Of 576 patients, 246 patients (42.7%) had AKI within 24 hours of ICU admission while 133 developed new AKI later during their ICU stay. RIFLE-initial class was Risk in 205 patients (54.1%), Injury in 99 (26.1%) and Failure in 75 (19.8%). Progression of AKI to a worse RIFLE class was seen in 114 patients (30.8% of AKI patients). AKI patients were older, with higher frequency of common risk factors. 116 AKI patients (30.6%) fulfilled criteria for sepsis during their ICU stay, compared to 33 (16.7%) of non-AKI patients (P0.001). 48 patients (8.3%) were treated with renal replacement therapy (RRT) in the ICU. Patients were started on RRT a median of 2 (IQR 0-6) days after ICU admission. Among AKI patients, they were started on RRT a median of 1 (IQR 0-4) days after fulfilling criteria for AKI. Median duration of RRT was 5 (IQR 2-10) day. AKI patients had a higher crude ICU mortality (28.8% vs. non-AKI 8.1%, P0.001) and longer ICU length of stay (median 7 days vs. 3 days [non-AKI], P0.001). Crude ICU mortality and ICU length of stay increased with greater severity of AKI. Two hundred twenty five patients (59.4% of AKI patients) had complete recovery of renal function, with a SCr at time of ICU discharge which was ≤120% of baseline; an additional 51 AKI patients (13.5%) had partial renal recovery, while 103 (27.2%) had not recovered renal function at the time of death or ICU discharge. Septic patients had more severe AKI, and were more likely to receive RRT with less frequency of renal function recovery. Patients with sepsis had higher ICU mortality and longer ICU stay. The study confirms previous analyses describing RIFLE as an optimal classification system to stage AKI severity. AKI is indeed a deadly complication for ICU patients where the level of severity correlated with mortality and length of stay. The tool developed for data collection resulted user friendly and easy to implement. Some of its features including a RIFLE class alert system, may help the treating physician to collect systematically AKI data in the ICU and possibly may guide specific decision on the institution of renal replacement therapy.

Web-based hypermedia systems are becoming more and more sophisticated, new modeling requirements constantly arise, and design models must constantly evolve. Since design tools should complement models to support an efficient design process, model evolution raises a technological issue: Design tools must be modified when their underlying model changes. This is why the paper proposes a general approach to efficiently update design tools in response to model evolutions. The key ideas are: a) the description of a hypermedia model in terms of a general meta-model, powerful enough to express the semantics of current and future design constructs; b) the transformation of a hypermedia design tool into a meta-CASE tool, able to cope with model updates without requiring to be redefined and rebuilt from scratch. The approach is presented by discussing a case study, that is, the feasibility study to transform our design toolkit, Jweb, into a meta-CASE tool (Jweb3). This tool will support the latest version of our model (called W2000), and will easily evolve with the model it supports. We discuss the adoption of the OMG meta-modeling standards MOF and XMI as enabling technology, we present a sample of the representation of W2000 in terms of MOF, and we sketch the architecture of the under-implementation Jweb3.

Hypermedia applications combine the flexibility of navigation based-access to information, typical of hypertext, with the communication power of multiple media, typical of multimedia systems. By their very nature, hypermedia applications support multimode interacation, i.e., interaction based on a combination of multiple modalities that are induced by different media and different navigation paradigms. The potentially huge number of mode combinations in hypermedia can accommodate a large variety of user needs and tasks. Multimode interaction, however, is intrinsically complex for the users if several multimode paradigmas coexist within the same application. This paper discusses the concept of modal navigation as a technique that allows to achieve both simplicity in user interaction and flexibility in tuning navigation styles to specific needs of different categories of users. According to modal navigation, the semantics of navigation commands depends upon the current setting of modes. Various paradigms are discussed for modal navigation that take into account different degrees of user's control in the definition of mode configuration and mode resetting. The approach will be exemplified by discussing a real life hypermedia application under development at HOC in cooperation with the Poldi Pezzoli Museum in Milano.

Hypermedia design is the task of specifying representation structures, dynamic behaviour, navigation patterns, and lay-out features, of a hypermedia application. This paper describes a structured approach to hypermedia design, identifying the crucial steps in the design process. Our main goal is to provide a guideline to make hypermedia design more systematic and better organised, and to help mastering the complexity of hypermedia application development and improving consistency and usability of the resulting applications.

The Dexter Model defines the notion of atomic components and composite components, but it does not prescribe, nor it suggests, any particular structure for composite components. This paper proposes a specific type of composite component, called “collection”.A collection is a container holding several members. Collections can contain other collections (nested collections). Collections can be regarded as sets, but they can also have an inner structure. Collections can be created in several ways: manually, through queries, by operations on other collections, by exploiting links, etc.Collections introduce a navigational pattern, based on their structure, that is different from the standard node&link navigation.If active media are considered, collections allow the design and implementation of complex synchronisation strategies, difficult to obtain otherwise.The paper describes the motivations for using collections, their structure, their navigational capabilities and a number of possible authoring mechanisms. It also examines the interplay between standard navigation and collection navigation, possible synchronization strategies for collections, as well as the requirements for the definition of a runtime support (which could be used to extend the runtime layer of the Dexter Model).

From the experience gained in evaluating several existing Hypermedia POIs (Points of Information), and from our own development work in this field, we have identified some critical issues in designing this class of systems. This paper briefly presents these results, by outlining the major stages of POI design and by discussing their impact on the quality of the final system. These concepts are exemplified by presenting the design choices in HyperMilano, a full size hypermedia POI developed at the Multimedia Laboratory of Politecnico di Milano.