Search

People with dementia (PwD) have been conducting educational activities to live well with dementia. Appropriate CHI design is needed to support the lives of PwD. However, current prototypical systems are mostly designed for needs on the surface and do not approach hidden needs. To develop an system that PwD wants to use on a daily basis, we conducted a co-production practice with PwD focussing on outing their home. We collaborate with a group of early-onset dementia who wanted their opinions to be reflected in ICT. A participatory design workshop was undertaken to discuss the development of outing assistants. As a result, it was found that they did not want the system to be responsible for intervening in their disabilities, but to positively encourage their activities. To conduct a case study for outing assists, we focused on a person with dementia’s activity to verify whether it contributes to her life. The system was implemented as a voice and visual interaction system, designed to encourage a positive relationship between the supporter and her. Our results indicate that the system design to support PwD should produce positive interaction and should be adapted to the individual context.

In order to create air conditioning service scenarios adaptive to elderly people, we have conducted detailed situational analyses focusing on bodily feeling information. The practical experimental environment has introduced our constructed indoor interaction system to understand situations based on bodily feeling and to control air conditioners properly. The experiments consists of 7 sessions participated by 6 elderly people have accumulated 228 times of bodily feeling inputs. The detailed analyses have done utilizing our developed multimodal behavior observation tool equipped with the functions of behavior description from multiple viewpoints and flexible retrieval. The analysis results indicate the possibility of correspondence of the scenes of situational changes in conversation like topic transitions and turn-taking events or those of reading to the changes of bodily feeling. The multimodal bodily feeling analysis has produced valuable findings to enhance situation understanding.

Our goal is to represent commonsense knowledge as computational models, which are applied to spoken dialogue systems that realize smart man-machine communications by correctly understanding speakers' intentions and emotions. For this purpose, we have constructed a multimodal speech behavior corpus which includes metadata annotated from various viewpoints in everyday life. This paper describes a methodology of modeling of thinking processes in problem solving based on child development, by analyzing data of attention-catching.

We have developed a method for multimodal observation of infant development. In order to analyze development of problem solving skills by observing scenes of task achievement or communication with others, we have introduced a method for extracting detailed behavioral features expressed by gestures or eyes. We have realized an environment for recording behavior of the same infants continuously as multi-angle video. The environment has evolved into a practical infrastructure through the following four steps; (1) Establish an infant school and study the camera arrangement. (2) Obtain participants in the school who agree with the project purpose and start to hold regular classes. (3) Begin to construct a multimodal infant behavior corpus with considering observation methods. (4) Practice development process analyses using the corpus. We have constructed a support tool for observing a huge amount of video data which increases with age. The system has contributed to enrich the corpus with annotations from multimodal viewpoints about infant development. With a focus on the demonstrative expression as a fundamental human behavior, we have extracted 240 scenes from the video during 10 months and observed them. The analysis results have revealed interesting findings about the developmental changes in infants' gestures and eyes, and indicated the effectiveness of the proposed observation method.

We have developed a framework to understand situations and intentions of speakers focusing on the utterances of demonstratives. We aim at constructing a 'Multimodal Infant Behavior Corpus', which makes a valuable contribution to the elucidation of human commonsense knowledge and its acquisition mechanism. For this purpose, we have constructed environments for multimodal observation of infant behavior, in particular, environments for infant behavior recording; we have set up multiple cameras and microphones in the Cedar yurt. We have also developed a wearable speech recording device of high quality to capture infant utterances clearly. Moreover, we have developed a comment-collecting system which allows everyone to make comments easily from the multi-viewpoints. Those construction and developments make it possible to realize a framework for multimodal observation of infant behavior. Utilizing the multimodal environments, we propose a situation description model based on observation of demonstratives uttered by infants, since demonstratives appear frequently in their conversations and become a precious clue to understand situations. The proposed model, which represents the mental distances of speakers and listeners to objects on a general and simple model, enables us to predict speakers' next behavior. The consideration results enable us to conclude that the constructed environments lead to development and realization of human interaction models applicable to spoken dialog systems for elder people supporting.