Multimodal fuzzy fusion for enhancing the motor-imagery-based brain computer interface

Brain–computer interface technologies, such as steady-state visually evoked potential, P300, and motor imagery are methods of communication between the human brain and the external devices. Motor imagery–based brain–computer interfaces are popular because they avoid unnecessary external stimulus. Although feature extraction methods have been illustrated in several machine intelligent systems in motor imagery-based brain–computer interface studies, the performance remains unsatisfactory. There is increasing interest in the use of the fuzzy integrals, the Choquet and Sugeno integrals, that are appropriate for use in applications in which fusion of data must consider possible data interactions. To enhance the classification accuracy of brain-computer interfaces, we adopted fuzzy integrals, after employing the classification method of traditional brain–computer interfaces, to consider possible links between the data. Subsequently, we proposed a novel classification framework called the multimodal fuzzy fusion-based brain-computer interface system. Ten volunteers performed a motor imagery-based brain-computer interface experiment, and we acquired electroencephalography signals simultaneously. The multimodal fuzzy fusion-based brain-computer interface system enhanced performance compared with traditional brain–computer interface systems. Furthermore, when using the motor imagery-relevant electroencephalography frequency alpha and beta bands for the input features, the system achieved the highest accuracy, up to 78.81% and 78.45% with the Choquet and Sugeno integrals, respectively. Herein, we present a novel concept for enhancing brain–computer interface systems that adopts fuzzy integrals, especially in the fusion for classifying brain–computer interface commands. This work was supported in part by the Australian Research Council (ARC) under discovery grant DP180100670 and DP180100656, and in part by the Spanish Ministry of Science under discovery grant TIN2016-77356-P(MINECO, FEDER, UE). This work was also particularly supported by the Ministry of Education through the SPROUT Project - Center for Intelligent Drug Systems and Smart Bio-devices (IDS2B) of National Chiao Tung University, Taiwan, and supported in part by the Ministry of Science and Technology (MOST), Taiwan, under Contract MOST 107-2221-E-009-150-.