Affective Image Classification Based on User Eye Movement and EEG Experience Information.

To improve the quality of human–computer interaction, it is important for computers to be able to identify the images quickly that can trigger pleasant feelings and create a good user experience. To reach this goal, the affective classification of images is an important prerequisite. Prior research into affective image classification used image features as categorical measurable signals. However, these techniques failed to provide a good correlation between the signal properties of image features and the expected affective experience of the viewer, leaving the research unable to bridge the affective gap. To solve this problem, in this paper we propose a method of affective image classification based on information about the user's experience collected through measurable eye movements and electroencephalography (EEG) signals. This approach bridges the affective gap by establishing an association between experience information (measurable signal properties) and the expected affective experience of the viewer. First, we screened eye movement indexes and EEG indexes to provide an affective correlation index. Then we built an experience space based on participants' physiological data obtained using an eye tracker and EEG recorder simultaneously. Next, physiological experience data from the experience space were extracted, analyzed mathematically and normalized to obtain parametric physiological experience data. Using a multiple linear regression technique, we connected the participants' affective states and physiological experience data. We developed a quantitative mapping between the affective experience states and the sample images to acquire the classification of affective images. To demonstrate the feasibility of the proposed classification method, we conducted a study in which 16 abstract art paintings were classified as positive, negative or neutral based on subjects' physiological responses. The results showed that the proposed affective image classification method was accurate and successful in identifying images that can lead to a pleasurable experience. [ABSTRACT FROM AUTHOR]