Methodology of collection, recording and markup of biophysical multimodal data in the study of human psychoemotional states

Background and Objectives: Effective diagnostics of depression state using instrumental methods of biopotentials measurement is promising both as a tool for increasing the efficiency of work of a diagnostician and for application in automated hardware and software therapy complexes, including those with biofeedback, allowing to create neuroadaptive systems for correction of psychoemotional and cognitive problems of patients. The purpose of this paper is to analyze the requirements to the methodology of biophysical data collection, hardware and software for their primary processing on the basis of open datasets of psychoemotional state determination, to formulate a methodology for the formation of multimodal datasets for the study of mental states, their changes, suitable for use in machine learning algorithms, to describe a possible method of realization of these requirements in hardware and software complexes. Materials and Methods: Open datasets of depressed patients were selected to analyze the main characteristics of datasets characterizing mental states. To formulate the main requirements, publications on the features of multimodal data application for the diagnosis of depressive disorders were reviewed. Results: The result of the work are the requirements to multimodal biopotential data for the study of psychoemotional states, the methodology and functional concept of hardware-software complex for their registration, the synchronization and recording in annotated form. Conclusion: The example of depressive disorder shows the usefulness and possibility of recording multimodal, synchronized annotated data on the psycho-emotional state of the subject for research, diagnostic purposes and application as a training sample in machine learning algorithms. The proposed methodology and the concept of the hardware-software complex allow to level out the main disadvantages of multimodal systems of biopotentials registration realized in the form of separate blocks and to supplement the instrumental data with annotation by state labels and time labels.