Cultural Background and Input Familiarity Influence Multisensory Emotion Perception.

Objectives: During multisensory emotion perception, the attention devoted to the visual versus the auditory modality (i.e., modality dominance) varies depending on the cultural background of the perceiver. In the present study, we examined (a) how cultural familiarity influences multisensory emotion perception in Eastern and Western cultures and (b) the underlying processes accounting for the cultural difference in modality dominance. Method: Native Mandarin speakers from China and native English speakers from the United States were presented with audiovisual emotional stimuli from their own culture (i.e., familiar) and from a different culture (i.e., unfamiliar) and asked to evaluate the emotion from one of the two modalities. Across modalities, the emotions were either the same (i.e., congruent, happy face, and happy voice) or different (i.e., incongruent, happy face, and sad voice). Results: When the input was in a familiar cultural context, American participants were more influenced by the visual modality, while Chinese participants were more influenced by the auditory modality. While both groups integrated the incongruent emotion from the irrelevant modality, only the American group integrated the congruent emotion from the irrelevant modality. When the input was in a less familiar cultural context, both groups showed increased visual dominance, but only the Chinese group simultaneously showed decreased auditory dominance. Conclusions: We conclude that cultural background and input familiarity interact to influence modality dominance during multisensory emotion perception. Public Significance Statement: The present study reveals that American participants were more influenced by facial expressions than vocal expressions, while Chinese participants were more influenced by vocal expressions than facial expressions during multisensory emotion perception. Recognizing these differences could facilitate communication and interactions between individuals from East Asian and Western cultures. [ABSTRACT FROM AUTHOR]