A study of constructing a thinking process model based on multimodal behavior analysis

Our goal is to represent commonsense knowledge as computational models, which are applied to spoken dialogue systems that realize smart man-machine communications by correctly understanding speakers' intentions and emotions. For this purpose, we have constructed a multimodal speech behavior corpus which includes metadata annotated from various viewpoints in everyday life. This paper describes a methodology of modeling of thinking processes in problem solving based on child development, by analyzing data of attention-catching.