1. Impact of Visual Design Elements and Principles in Human Electroencephalogram Brain Activity Assessed with Spectral Methods and Convolutional Neural Networks
- Author
-
Pablo Sánchez-Núñez, Gustavo Vaccaro, José Ignacio Peláez, Francisco E. Cabrera, Javier Escudero, [Cabrera,FE, Vaccaro,G, Peláez,JI] Department of Languages and Computer Sciences, School of Computer Science and Engineering, Universidad de Málaga, Málaga, Spain. [Cabrera,FE, Sánchez-Núñez,P, Peláez,JI] Centre for Applied Social Research (CISA), Ada Byron Research Building, Universidad de Málaga, Málaga, Spain. [Cabrera,FE, Peláez,JI] Instituto de Investigación Biomédica de Málaga (IBIMA), Málaga, Spain. [Sánchez-Núñez,P] Department of Audiovisual Communication and Advertising, Faculty of Communication Sciences, Universidad de Málaga, Málaga, Spain. [Escudero,J] School of Engineering, Institute for Digital Communications (IDCOM), The University of Edinburgh, Edinburgh, UK., and This research was partially supported by On the Move, an international mobility pro gramme organized by the Society of Spanish Researchers in the United Kingdom (SRUK) and CRUE Universidades Españolas. The Article Processing Charge (APC) was funded by the Programa Op erativo Fondo Europeo de Desarrollo Regional (FEDER) Andalucía 2014–2020 under Grant UMA 18-FEDERJA-148 and Plan Nacional de I+D+i del Ministerio de Ciencia e Innovación-Gobierno de España (2021-2024) under Grant PID2020-115673RB-100.
- Subjects
Visual perception ,Computer science ,Brain activity and meditation ,Humanities::Humanities::Music [Medical Subject Headings] ,Emotions ,Electroencephalography ,Biochemistry ,Convolutional neural network ,Analytical Chemistry ,Organisms::Eukaryota::Animals::Chordata::Vertebrates::Mammals::Primates::Haplorhini::Catarrhini::Hominidae::Humans [Medical Subject Headings] ,0302 clinical medicine ,Percepción visual ,Psychiatry and Psychology::Psychological Phenomena and Processes::Mental Processes::Perception::Visual Perception [Medical Subject Headings] ,EEG ,Instrumentation ,medicine.diagnostic_test ,05 social sciences ,Brain ,Atomic and Molecular Physics, and Optics ,emotion classification ,Emociones ,Visual Perception ,CNN ,Electroencefalografía ,visual features ,Cariotipificación espectral ,Emotion classification ,Red nerviosa ,visual perception ,Clasificación ,TP1-1185 ,050105 experimental psychology ,Article ,Analytical, Diagnostic and Therapeutic Techniques and Equipment::Diagnosis::Diagnostic Techniques and Procedures::Diagnostic Techniques, Neurological::Electroencephalography [Medical Subject Headings] ,03 medical and health sciences ,Rhythm ,medicine ,Pruebas neuropsicológicas ,Humans ,0501 psychology and cognitive sciences ,Visual communication ,Electrical and Electronic Engineering ,business.industry ,Deep learning ,Chemical technology ,visual design elements and principles (VDEPs) ,Pattern recognition ,Anatomy::Nervous System::Central Nervous System::Brain [Medical Subject Headings] ,spectral analysis ,Spectral Analysis ,visual attention ,Phenomena and Processes::Mathematical Concepts::Neural Networks (Computer) [Medical Subject Headings] ,Psychiatry and Psychology::Behavior and Behavior Mechanisms::Emotions [Medical Subject Headings] ,Artificial intelligence ,Neural Networks, Computer ,business ,030217 neurology & neurosurgery ,Music - Abstract
The visual design elements and principles (VDEPs) can trigger behavioural changes and emotions in the viewer, but their effects on brain activity are not clearly understood. In this paper, we explore the relationships between brain activity and colour (cold/warm), light (dark/bright), movement (fast/slow), and balance (symmetrical/asymmetrical) VDEPs. We used the public DEAP dataset with the electroencephalogram signals of 32 participants recorded while watching music videos. The characteristic VDEPs for each second of the videos were manually tagged for by a team of two visual communication experts. Results show that variations in the light/value, rhythm/movement, and balance in the music video sequences produce a statistically significant effect over the mean absolute power of the Delta, Theta, Alpha, Beta, and Gamma EEG bands (p <, 0.05). Furthermore, we trained a Convolutional Neural Network that successfully predicts the VDEP of a video fragment solely by the EEG signal of the viewer with an accuracy ranging from 0.7447 for Colour VDEP to 0.9685 for Movement VDEP. Our work shows evidence that VDEPs affect brain activity in a variety of distinguishable ways and that a deep learning classifier can infer visual VDEP properties of the videos from EEG activity.
- Published
- 2021
- Full Text
- View/download PDF