Search

Recent studies have highlighted the possibility of using surface electromyographic (EMG) signals to develop human–computer interfaces that are also able to recognize complex motor tasks involving the hand as the handwriting of digits. However, the automatic recognition of words from EMG information has not yet been studied. The aim of this study is to investigate the feasibility of using combined forearm and wrist EMG probes for solving the handwriting recognition problem of 30 words with consolidated machine-learning techniques and aggregating state-of-the-art features extracted in the time and frequency domains. Six healthy subjects, three females and three males aged between 25 and 40 years, were recruited for the study. Two tests in pattern recognition were conducted to assess the possibility of classifying fine hand movements through EMG signals. The first test was designed to assess the feasibility of using consolidated myoelectric control technology with shallow machine-learning methods in the field of handwriting detection. The second test was implemented to assess if specific feature extraction schemes can guarantee high performances with limited complexity of the processing pipeline. Among support vector machine, linear discriminant analysis, and K-nearest neighbours (KNN), the last one showed the best classification performances in the 30-word classification problem, with a mean accuracy of 95% and 85% when using all the features and a specific feature set known as TDAR, respectively. The obtained results confirmed the validity of using combined wrist and forearm EMG data for intelligent handwriting recognition through pattern recognition approaches in real scenarios. [ABSTRACT FROM AUTHOR]

Emotion recognition plays an essential role in human–human interaction since it is a key to understanding the emotional states and reactions of human beings when they are subject to events and engagements in everyday life. Moving towards human–computer interaction, the study of emotions becomes fundamental because it is at the basis of the design of advanced systems to support a broad spectrum of application areas, including forensic, rehabilitative, educational, and many others. An effective method for discriminating emotions is based on ElectroEncephaloGraphy (EEG) data analysis, which is used as input for classification systems. Collecting brain signals on several channels and for a wide range of emotions produces cumbersome datasets that are hard to manage, transmit, and use in varied applications. In this context, the paper introduces the Empátheia system, which explores a different EEG representation by encoding EEG signals into images prior to their classification. In particular, the proposed system extracts spatio-temporal image encodings, or atlases, from EEG data through the Processing and transfeR of Interaction States and Mappings through Image-based eNcoding (PRISMIN) framework, thus obtaining a compact representation of the input signals. The atlases are then classified through the Empátheia architecture, which comprises branches based on convolutional, recurrent, and transformer models designed and tuned to capture the spatial and temporal aspects of emotions. Extensive experiments were conducted on the Shanghai Jiao Tong University (SJTU) Emotion EEG Dataset (SEED) public dataset, where the proposed system significantly reduced its size while retaining high performance. The results obtained highlight the effectiveness of the proposed approach and suggest new avenues for data representation in emotion recognition from EEG signals. [ABSTRACT FROM AUTHOR]

This article examines utopian elements in Thomas Stuber's 2018 film In den Gängen. Despite the neoliberal forces that constrain a group of supermarket shelf stackers in the former East Germany, the film's characters exhibit congenial bonds in the face of alienated work. Strikingly, their encounters with machines also convey an other-directedness and improvised conviviality, or energy that enables them to devise bonds in fleeting moments. As much as the film presumes a foreclosed-on socialist utopia, it also depicts a range of contemporary challenges to neoliberalism, including practices of commoning, non-normative forms of relationality, and human-machine interactions that gesture towards a posthumanist selfhood. At the same time, the three main characters' personal circumstances and histories necessarily impact their ability to benefit from these elements. Both age and gender play crucial roles in their individual fates, and not all of them thrive within their caring, other-directed community. Ultimately, In den Gängen suggests the possibility of mundane existences gesturing towards superlative futures. [ABSTRACT FROM AUTHOR]

Speaker identification has significant advantages for the field of human–computer interaction. Recently, many scholars have made contributions in this field and successfully created deep learning models for automatic speaker identification systems. However, most of the speech signal processing work is limited to English-only applications, despite numerous challenges with Arabic speech, particularly with the recitation of the Holy Quran, which is the Islamic holy book. In the light of these considerations, this study proposes a model for identifying the reciter of the Holy Quran using a dataset of 11,000 audio samples extracted from 20 Quran reciters. To enable feeding the audio samples' visual representation to the pre-trained models, the audio samples are converted from their original audio representation to visual representation using the Mel-Frequency Cepstrum Coefficients. Six pre-trained deep learning models are evaluated separately in the proposed model. The results from the test dataset reveal that the NASNetLarge model achieved the highest accuracy rate of 98.50% among the pre-trained models used in this study. [ABSTRACT FROM AUTHOR]

Motion recognition (MR)‐based somatosensory interaction technology, which interprets user movements as input instructions, presents a natural approach for promoting human‐computer interaction, a critical element for advancing metaverse applications. Herein, this work introduces a non‐intrusive muscle‐sensing wearable device, that in conjunction with machine learning, enables motion‐control‐based somatosensory interaction with metaverse avatars. To facilitate MR, the proposed device simultaneously detects muscle mechanical activities, including dynamic muscle shape changes and vibrational mechanomyogram signals, utilizing a flexible 16‐channel pressure sensor array (weighing ≈0.38 g). Leveraging the rich information from multiple channels, a recognition accuracy of ≈96.06% is achieved by classifying ten lower‐limb motions executed by ten human subjects. In addition, this work demonstrates the practical application of muscle‐sensing‐based somatosensory interaction, using the proposed wearable device, for enabling the real‐time control of avatars in a virtual space. This study provides an alternative approach to traditional rigid inertial measurement units and electromyography‐based methods for achieving accurate human motion capture, which can further broaden the applications of motion‐interactive wearable devices for the coming metaverse age. [ABSTRACT FROM AUTHOR]

Wearable flexible electronic strain sensor devices have gained significant attention in recent years due to their potential for detecting human motion in various scenarios. However, the development of strain sensors with high sensitivity across a wide range of strains remains a major challenge. We present herein a novel strain sensor based on a graded structure thermoplastic polyurethane (TPU)/carbon nanotube (CNT) composite yarn with significantly enhanced mechanical performance imparted by the designed structure. The twisted CNTs/TPU spiral yarn demonstrated a fracture elongation of up to 1066% while maintaining charge conductivity under high‐strain conditions. Moreover, it exhibited sensitive changes in resistance versus tensile strain, excellent repeatability, and stability. As a strain sensor, it achieved a gauge factor (GF) of 67.2 within a strain range below 50%, reaching 51.7 in a strain range exceeding 150%. With a fast response time of 0.12 s, it enabled accurate identification of movements in different body parts. These findings highlight the broad application potential of the designed spiral yarn strain sensor in areas such as human motion monitoring and human–computer interaction. Highlights: Prepares a flexible strain sensor with graded‐structure (CNT‐fiber‐yarn).Shows high sensitivity and a wide strain response rangeReveals the conductive model of strain sensor. [ABSTRACT FROM AUTHOR]

A new method based on a digital twin is proposed for fault diagnosis, in order to compensate for the shortcomings of the existing methods for fault diagnosis modeling, including the single fault type, low similarity, and poor visual effect of state monitoring. First, a fault diagnosis test platform is established to analyze faults under constant and variable speed conditions. Then, the obtained data are integrated into the Unity3D platform to realize online diagnosis and updated with real-time working status data. Finally, an industrial test of the digital twin model is conducted, allowing for its comparison with other advanced methods in order to verify its accuracy and application feasibility. It was found that the accuracy of the proposed method for the entire reducer was 99.5%, higher than that of other methods based on individual components (e.g., 93.5% for bearings, 96.3% for gear shafts, and 92.6% for shells). [ABSTRACT FROM AUTHOR]

The absence of some forms of non-verbal communication in virtual reality (VR) can make VR-based group discussions difficult even when a leader is assigned to each group to facilitate discussions. In this paper, we discuss if the sensor data from off-the-shelf VR devices can be used to detect opportunities for facilitating engaging discussions and support leaders in VR-based group discussions. To this end, we focus on the detection of suppressed speaking intention in VR-based group discussions by using personalized and general models. Our extensive analysis of experimental data reveals some factors that should be considered to enable effective feedback to leaders. In particular, our results show the benefits of combining the sensor data from leaders and low-engagement participants, and the usefulness of specific HMD sensor features. [ABSTRACT FROM AUTHOR]

This document is an introduction to a special issue of the International Journal of Human-Computer Interaction focused on innovations in human-computer interaction (HCI) in China. It provides a historical overview of HCI development in China, starting with the focus on ergonomic aspects of computing technology in the 1980s. The article highlights the growth of HCI in China, including the establishment of research and development centers by foreign corporations and the impact of HCI on various industries. The special issue includes articles on topics such as trustworthy AI, affective computing, human-robot interaction, HCI in autonomous vehicles and healthcare, and the application of VR/AR/MR technologies. It also explores the integration of HCI with traditional Chinese culture and the use of innovative technologies to enhance user performance and experience. The goal of the special issue is to showcase the contributions of Chinese HCI professionals and promote collaboration in addressing the challenges of intelligent technologies and human-AI collaboration. [Extracted from the article]

As human-computer interaction (HCI) technology becomes more and more integrated into our daily life, increasing attention has been drawn towards the interaction experience in addition to HCI efficiency. In the present study, we conducted a survey to explore context-specific emotional experience in HCI. Four hundred participants were recruited to report the frequency of their emotional experiences on 44 fine-grained emotion items in six representative HCI scenarios. Compared with six matched human-human interaction (HHI) scenarios used as control, the HCI scenarios were in general more frequently associated with negative emotions, and less frequently associated with positive emotions, especially when computer served as a tool for communication with other people. Furthermore, the 44 emotional experience items in HCI were summarized as five factors, representing low-arousal focused, positively engaged, emotionally empathetic, high-arousal negative and frustratingly confused. Our study presents a comprehensive overview of context-specific emotional experience in human-computer interactions and provides a framework for emotion evaluation in HCI applications. [ABSTRACT FROM AUTHOR]

Non-pharmacological treatments have gained significant attention in the field of cognitive impairment. Among them, human–computer interaction-based (HCI) methods have emerged as a promising approach due to their broad applicability and convenience in assessing symptoms associated with this progressively debilitating condition. However, existing rehabilitation training systems for cognitive impairment lack effective assessment methods to meet the diverse rehabilitation needs of users. In this article, we surveyed existing HCI-based cognitive rehabilitation training systems and analyzed their advantages and shortcomings. Drawing from the insights gained from these systems, we propose a novel Leap Motion-based building block training system that incorporates system software capable of generating highly realistic virtual scenes, with the added capability of user behavior detection using Kinect. We conducted user testing of this new system, comparing the performance of a representative cohort with mild cognitive impairment (MCI) (n = 9) to that of disease-free participants (n = 10). Additionally, we conducted ergonomic experiments to assess the system's performance in elderly people. The experimental results revealed significant differences between the MCI cohort and the control cohort. Specifically, the MCI cohort exhibited a reduced range of motion and longer task completion times compared to the control cohort. These findings have the potential to contribute to the differentiation of cognitive levels. In conclusion, our analysis of existing cognitive rehabilitation training systems provides valuable insights for researchers working on the development of future innovative cognitive rehabilitation training systems and enriches the non-pharmacological treatment models for cognitive impairment. Furthermore, the observed relationship between behavioral data, task completion times, and cognitive levels in older adults offers useful insights for the design of HCI-based approaches for diagnosing and assessing the treatment of MCI. [ABSTRACT FROM AUTHOR]

Sketching is a common way to depict design ideas freely. Augmented reality technology extends the human–computer interaction space from the screen to the real world, providing a new application space for 3D sketching. However, due to the difficulty of depth perception, it is difficult for sketching strokes to maintain good planar and spatial consistency, making drawing 3D sketches in AR challenging. In this article, we propose FMASketch—a method of drawing mid-air sketches similar to building blocks. The auxiliary surfaces can ensure that the strokes drawn by users can always stay on the same plane as drawing in 2D space. Gestures are used to stitch together multiple drawing planes, and the strokes on the planes form a 3D sketch, which is very similar to how 3D modeling is done on the desktop by drawing different views and assembling them. Experiments have verified the effectiveness of the auxiliary surface generation method and gesture operations in this article. The mid-air sketching method is easy to learn and provides a new idea for rapid modeling in AR scenarios. [ABSTRACT FROM AUTHOR]

Gamified and immersive educational interventions have proven to be powerful motivators to encourage participation in unattractive activities, and human–computer interaction (HCI) is crucial for the successful development of such educational interventions. However, existing interaction methods suffer from either visuo-tactile inconsistency or unfriendly interaction methods, resulting in an unenjoyable user experience. Hence, in this article, the visuo-tactile interaction design based on Norman's design principles (visibility, feedback, constraint, consistency, affordance, and mapping) and spatial augmented reality (SAR) for intuitive cultural education is proposed for the first time, and an application for Chinese chess, SARChess, is implemented as a case study. Players interact with tangible chess pieces rather than through a digital interface, and the visual output and tangible input are integrated seamlessly to achieve high-level visuo-tactile consistency. Furthermore, a mixed investigation method based on pretest–posttest design, subjective questionnaires, and theme-based interviews was designed and performed to evaluate the user experience, learning performance, and player motivation of SARChess from both objective and subjective perspectives. The results show that SARChess has a significantly better user experience, learning performance, and player motivation than conventional chess and mouse-interaction-based chess, and the unique gameplay and visual cues in SARChess make it more engaging to participants. Additionally, this study finds and summarizes some design implications for applying design principles to traditional cultural education applications. [ABSTRACT FROM AUTHOR]

Children with autism face challenges in areas like language and social skills, which hinder their ability to undergo regular fire training. Fire is one of the most common and dangerous disaster in real life, making it essential to provide children with appropriate prevention and response education. Virtual humans can offer diverse presentation forms and interact with children with autism, thus better stimulating their willingness to participate. To train the fire safety skills of children with autism, this paper proposes the application of highly realistic virtual humans in a fire education system, aiming to improve their fire safety skills. The results show that this approach effectively enhances the fire safety skills of children with autism. To enhance the realism of virtual humans in the fire education system, this paper improves the 3D pose estimation method and proposes a multi-physical factor pose estimation algorithm. By evaluating the Mean Penetration Error (MPE) and the Percentage Not Penetrated (PNP) it was shown that the pose estimation algorithm achieved higher accuracy with only 6.4% foot penetration. We counted the number of movements and the number of movements captured by the system for all participants in the fire training and showed that the system’s motion capture accuracy was over 90%. [ABSTRACT FROM AUTHOR]

Factors, such as whether a website is designed to be user-oriented beyond its mere visual design, its effectiveness and efficiency, its usability, and the organisation of the information it offers, have come to the fore once again after the Covid-19 pandemic. It has been evident that the link structure in a website, better known as the website's information architecture, helps the practitioners with identifying factors that affect the usability of a website. In this sense, practitioners must ensure that the information architecture supports the usage intentions of a websites' visitors to better serve and motivate them. However, in many cases, different types of users navigate websites that contain immense amounts of information, so understanding their needs is also important for practitioners. In parallel, this article addresses the problem that different visitors of a large-scale website will need to navigate through dense information to find the information they are looking for, and the information architecture of the website must support different user tasks for the website to be widely adopted. Thus, unlike previous studies, this article combines the principles of information architecture and the technology acceptance model to investigate the effect of information architecture on visitor's usage intentions. The work also guides practitioners in developing architectural strategies to better enable visitors to fulfil their objectives in the least amount of time. [ABSTRACT FROM AUTHOR]

Crane-lift planning is critical but challenging to ensure construction safety and efficiency for the installation of heavy components. It engages a group decision-making process that involves various stakeholders to exchange knowledge of crane capabilities, site characteristics, and operational feasibility. However, conventional planning methods based on drawings, animations, and standalone virtual prototypes are inefficient in enabling collaborative decision-making due to rigid visualization and inflexible interactions among different stakeholders. This paper aims to develop a collaborative heavy lift planning (Co-HeLP) system that engages crane suppliers, site managers, and crane operators in an immersive and shared environment for crane layout optimization and lifting simulation. Specifically, the system incorporates a multiuser networking unit to enable stakeholders to design crane-lift layouts concurrently and synchronize changes in real time for facilitating information-sharing and collaborative planning. In addition, a cross-platform virtual reality (VR) unit is developed to provide stakeholders with role-tailored interactions of crane allocation and first-person operation. Automatic feasibility checking is also formulated to assist stakeholders in appraising jib reach limits and loading capacities more efficiently. The system is demonstrated using a real-life modular building project that involves the installation of multiple bulky modules. To evaluate the system, two groups of industry practitioners from crane suppliers and main contractors conducted heavy lift planning using Co-HeLP, where their responses were collected using questionnaire-based surveys and interviews. Results show that the system is rated highly on automatability (0.77/1), collectiveness (0.81/1), and applicability (0.85/1), with the advantages of full-scale visualization and multirole interactions. This paper opens up new possibilities for stakeholders to jointly manage crane-lift constraints and build consensus on layout plans through a mutually interactive and role-independent shared environment. More broadly, the paper derives a generic multiuser VR mechanism to address fragmented coordination and inefficient knowledge transfer, thereby facilitating collective planning and management in construction. [ABSTRACT FROM AUTHOR]

The human-computer interaction process is a vital task in attaining artificial intelligence, especially for a person suffering from hearing or speaking disabilities. Recognizing actions more traditionally known as sign language is a common way for them to interact. Computer vision and Deep learning models are capable of understanding these actions and can simulate them to build up a sustainable learning process. This sign language mechanism will be helpful for both the persons with disabilities and the machines to unbound the gap to achieve intelligence. Therefore, in the proposed work, a real-time sign language system is introduced that is capable of identifying numbers ranging from 0 to 9. The database is acquired from the 8 different subjects respectively and processed to achieve approximately 200k amount of data. Further, a deep learning model named LSTM is used for sign recognition. The results were compared with different approaches and on distinct databases proving the supremacy of the proposed work with 91.50% accuracy. Collection of daily life useful signs and further improving the efficiency of the LSTM model is the research direction for future work. The code and data will be available at . [ABSTRACT FROM AUTHOR]

With the rapid development of 3D reconstruction, especially the emergence of algorithms such as NeRF and 3DGS, 3D reconstruction has become a popular research topic in recent years. 3D reconstruction technology provides crucial support for training extensive computer vision models and advancing the development of general artificial intelligence. With the development of deep learning and GPU technology, the demand for high-precision and high-efficiency 3D reconstruction information is increasing, especially in the fields of unmanned systems, human-computer interaction, virtual reality, and medicine. The rapid development of 3D reconstruction is becoming inevitable. This survey categorizes the various methods and technologies used in 3D reconstruction. It explores and classifies them based on three aspects: traditional static, dynamic, and machine learning. Furthermore, it compares and discusses these methods. At the end of the survey, which includes a detailed analysis of the trends and challenges in 3D reconstruction development, we aim to provide a comprehensive introduction for individuals who are currently engaged in or planning to conduct research on 3D reconstruction. Our goal is to help them gain a comprehensive understanding of the relevant knowledge related to 3D reconstruction. [ABSTRACT FROM AUTHOR]

Objective: Determining the efficacy of two trust repair strategies (apology and denial) for trust violations of an ethical nature by an autonomous teammate. Background: While ethics in human-AI interaction is extensively studied, little research has investigated how decisions with ethical implications impact trust and performance within human-AI teams and their subsequent repair. Method: Forty teams of two participants and one autonomous teammate completed three team missions within a synthetic task environment. The autonomous teammate made an ethical or unethical action during each mission, followed by an apology or denial. Measures of individual team trust, autonomous teammate trust, human teammate trust, perceived autonomous teammate ethicality, and team performance were taken. Results: Teams with unethical autonomous teammates had significantly lower trust in the team and trust in the autonomous teammate. Unethical autonomous teammates were also perceived as substantially more unethical. Neither trust repair strategy effectively restored trust after an ethical violation, and autonomous teammate ethicality was not related to the team score, but unethical autonomous teammates did have shorter times. Conclusion: Ethical violations significantly harm trust in the overall team and autonomous teammate but do not negatively impact team score. However, current trust repair strategies like apologies and denials appear ineffective in restoring trust after this type of violation. Application: This research highlights the need to develop trust repair strategies specific to human-AI teams and trust violations of an ethical nature. [ABSTRACT FROM AUTHOR]

Background: Cognitive load during AR use has been measured conventionally by performance tests and subjective rating. With the growing interest in physiological measurement using non‐invasive biometric sensors, unbiased real‐time detection of cognitive load in AR is expected. However, a range of sensors and parameters are used in various subject fields, and reported results are fragmented. Objectives: The aim of this review is to analyse systematically how physiological methods have been used to measure cognitive load and what the implications are for the future research on AR‐based tools. Methods: This paper took the systematic review approach. Through screening with 10 exclusion criteria, 23 studies, that contain 3 key elements: AR‐based intervention, cognitive state examination and physiological methods, were identified, analysed and synthesised. Results: Physiological methods in their current form require reference to provide meaningful interpretations and suggestions. Therefore, they are often combined with conventional methods. Many studies investigate the effect of wearable devices in comparison with non‐AR stimuli, which has been controversial, but detection of different causes of cognitive load are on the horizon. Eye‐tracking is the method most used and most consistent in the use of its parameters. Conclusions: A multi‐method approach combining two or more evaluation instruments is essential for the validation of users' cognitive state. In addition to the AR stimuli in question, having another independent variable such as task difficulty in experiment design is useful. Statistical approaches with more data input could help establish a reliable scale. The future research should attempt to dissociate cognitive load caused by different effects such as device, instruction, and other AR techniques as well as intrinsic and extraneous aspects, in a better experimental setup with multiple parameters. Lay Description: What is already known about this topic: The efficacy of AR‐based instructional tools can be evaluated using the concept of cognitive load.Cognitive load has been conventionally measured by self‐reporting and performance.There is a growing interest in physiological measurement of cognitive load with non‐invasive biometric sensors. What this paper adds: This paper systematically analyses how physiological methods are used to measure cognitive load in AR.It synthesises the fragmented results of AR studies conducted with a range of physiological methods. Implications for practice: Combining two or more evaluation instruments is essential for the validation of users' cognitive state.Eye‐tracking is the method most used, and fixation duration is the most consistent parameter.A statistical approach with more biometric data input could help establish a reliable cognitive load scale.Future research should attempt to separate cognitive load caused by different effects and different cognitive aspects. [ABSTRACT FROM AUTHOR]

• The road user experience is defined as the subjective perception of the road traffic environment that road users experience when participating in urban road traffic. • Car drivers' road user experience integrates the experience of static with perceptions of dynamic, and describes the overall road traffic environment experienced by car drivers. • Car Drivers' Road User Experience Scale comprises four factors: travel experience, road condition experience, road sign experience, and road network experience. • Car drivers' road user experience is closely related to the amount of tasks they experience while driving and traffic climate. • Investigation of such experience can avoid the limitations of the human–computer interaction model, and help to develop a virtuous cycle of driving experience improvement and road system optimization. Urban roads serve a wide variety of road users, and the content and level of urban road user experience is an important part of evaluating and improving the service quality of road traffic. The study aim was to increase understanding of the road user experience of car drivers by developing and validating the Car Drivers' Road User Experience Scale. Using web-based questionnaires, three studies were conducted with three independent samples of Chinese car drivers to develop and validate the measurement scale. The results showed that the scale comprises four factors: travel experience, road condition experience, road sign experience, and road network experience. This study explored the theory and characteristics of car drivers' road user experience and provided a new scientific measurement tool. This tool could provide new data to support strategies to improve driver road user experience and promote the development of the urban road traffic environment. [ABSTRACT FROM AUTHOR]

Occlusion or posture change of the face in natural scenes has typical asymmetry; however, an asymmetric face plays a key part in the lack of information available for facial expression recognition. To solve the problem of low accuracy of asymmetric facial expression recognition, this paper proposes a fusion of channel global features and a spatial local information expression recognition network called the "Channel–Spatial Integration Network" (CSINet). First, to extract the underlying detail information and deepen the network, the attention residual module with a redundant information filtering function is designed, and the backbone feature-extraction network is constituted by module stacking. Second, considering the loss of information in the local key area of face occlusion, the channel–spatial fusion structure is constructed, and the channel features and spatial features are combined to enhance the accuracy of occluded facial recognition. Finally, before the full connection layer, more local spatial information is embedded into the global channel information to capture the relationship between different channel–spatial targets, which improves the accuracy of feature expression. Experimental results on the natural scene facial expression data sets RAF-DB and FERPlus show that the recognition accuracies of the modeling approach proposed in this paper are 89.67% and 90.83%, which are 13.24% and 11.52% higher than that of the baseline network ResNet50, respectively. Compared with the latest facial expression recognition methods such as CVT, PACVT, etc., the method in this paper obtains better evaluation results of masked facial expression recognition, which provides certain theoretical and technical references for daily facial emotion analysis and human–computer interaction applications. [ABSTRACT FROM AUTHOR]

According to the computers are social actors (CASA) paradigm, a dominant theoretical framework for research on human–computer interaction, people treat computers as if they were people. Recent studies on human–machine communication (HMC) and human–artificial intelligence (AI) interaction, however, appear to focus on when and how people respond to machines differently than to human agents. To reconcile this apparent contradiction, this study reviews critically the two overarching theoretical explanations proposed and tested in each respective tradition, the mindlessness account and the machine heuristic. After elaborating on several conceptual and operational issues with each explanatory mechanism, an alternative theoretical model of HMC is proposed that integrates both research traditions and generates predictions that potentially deviate from the dual-process models. Lastly, it is discussed how recent developments in AI technology invite modifications to the current understanding of HMC and beyond. [ABSTRACT FROM AUTHOR]

Emotion recognition using the electroencephalogram (EEG) has garnered significant attention within the realm of human–computer interaction due to the wealth of genuine emotional data stored in EEG signals. However, traditional emotion recognition methods are deficient in mining the connection between multi-domain features and fitting their advantages. In this paper, we propose a novel capsule Transformer network based on a multi-domain feature for EEG-based emotion recognition, referred to as MES-CTNet. The model's core consists of a multichannel capsule neural network(CapsNet) embedded with ECA (Efficient Channel Attention) and SE (Squeeze and Excitation) blocks and a Transformer-based temporal coding layer. Firstly, a multi-domain feature map is constructed by combining the space–frequency–time characteristics of the multi-domain features as inputs to the model. Then, the local emotion features are extracted from the multi-domain feature maps by the improved CapsNet. Finally, the Transformer-based temporal coding layer is utilized to globally perceive the emotion feature information of the continuous time slices to obtain a final emotion state. The paper fully experimented on two standard datasets with different emotion labels, the DEAP and SEED datasets. On the DEAP dataset, MES-CTNet achieved an average accuracy of 98.31% in the valence dimension and 98.28% in the arousal dimension; it achieved 94.91% for the cross-session task on the SEED dataset, demonstrating superior performance compared to traditional EEG emotion recognition methods. The MES-CTNet method, utilizing a multi-domain feature map as proposed herein, offers a broader observation perspective for EEG-based emotion recognition. It significantly enhances the classification recognition rate, thereby holding considerable theoretical and practical value in the EEG emotion recognition domain. [ABSTRACT FROM AUTHOR]

Intelligent command and control has become a new style of modern warfare command, and the high dynamic, strong adversarial, and massive data battlefield environment has put forward higher requirements for command efficiency and task execution ability. The traditional human-computer interaction mode has low interaction efficiency and lacks a humancentered interaction design, which cannot fully utilize the effectiveness of underwater command and control. This article is based on intelligent human-machine interaction and principal component analysis to design an interaction module that can generate an adaptive interface for underwater command and control based on user and business needs. By building an underwater command and control intelligent simulation platform, it has been confirmed that adaptive intelligent human-machine interaction can simplify the command and control process, reduce interaction time, and improve combat efficiency, with strong application value. [ABSTRACT FROM AUTHOR]

Purpose: About 70% of all aircraft accidents are caused by human–machine interaction, thus identifying and quantifying performance shaping factors is a significant challenge in the study of human reliability. An information flow field model of human–machine interaction is put forward to help better pinpoint the factors influencing performance and to make up for the lack of a model of information flow and feedback processes in the aircraft cockpit. To enhance the efficacy of the human–machine interaction, this paper aims to examine the important coupling factors in the system using the findings of the simulation. Design/methodology/approach: The performance-shaping factors were retrieved from the model, which was created to thoroughly describe the information flow. The coupling degree between the performance shaping factors was calculated, and simulation and sensitivity analysis are based on system dynamics. Findings: The results show that the efficacy of human–computer interaction is significantly influenced by individual important factors and coupling factors. To decrease the frequency of accidents after seven hours, attention should be paid to these factors. Originality/value: The novelty of this work lies in proposing a theoretical model of cockpit information flow and using system dynamics to analyse the effect of the factors in the human–machine loop on human–machine efficacy. [ABSTRACT FROM AUTHOR]

Manufacturers perpetually adapt their systems to meet unforeseen events, new objectives, competition, and improved understanding of processes. In that human-directed work, models mediate an enduring relationship between production resources and engineers. Accommodating new understanding in the models controlling production can lead to more effective manufacturing. That work has previously been the province of quality programs such as Six Sigma, but is now fertile ground to study human-computer interaction about that enduring relationship mediated by models. Can AI augment human capability in the arcane work of formulating and refining models? This question is relevant to complex system engineering generally, not just manufacturing. In answering this question, this paper adapts Klein's flexecution for use in adaptable manufacturing systems. Theory flexecution, the methodical refinement of models, points to human-computer interactions that emphasize the roles of models, explanation, and machine agents that recognize the engineer's goals. This perspective article illustrates these ideas with an example of formulating models for production scheduling. [ABSTRACT FROM AUTHOR]

Eye gaze tracking has recently become indispensable for domains like virtual reality, augmented reality, human–computer interaction and advertisement. The commercial eye gaze tracking equipment is too expensive to be used by the masses. In this manuscript, a non-invasive, low-resolution ordinary camera-based system has been proposed for tracking eye gaze. The proposed system attained an accuracy of 84% without requiring any costly equipment or infrared light like other commercial tools. Eye gaze fixations data of 37 participants were recorded on a webpage and gaze patterns were plotted using heat maps. Further, a case analysis was done to validate the performance of the proposed system, where the user interface has been controlled using eye movements. [ABSTRACT FROM AUTHOR]

[Display omitted] Flexible pressure sensors with high sensitivity and wide sensing range are highly desired in e-skins and wearable electronics. However, there is always a trade-off between high sensitivity and broad sensing range for most flexible pressure sensors. Herein, an artificial wheat awn-like hierarchical structure is designed onto the dielectric layer of the iontronic pressure sensor, realizing both high sensitivity and broad working range. The sensor is constructed by sandwiching a wheat awn-like polyvinyl alcohol/H 3 PO 4 dielectric layer between two transparent electrodes of silver nanowires/thermoplastic polyurethane/ionic liquid. The obtained sensor exhibits a high precision of 1 Pa, a high sensitivity of 47.65 kPa−1 (1–200 Pa), a wide measurement range from 1 Pa to 238 kPa, short response/recovery time of 13 ms/12 ms, outstanding stability over 6000 cycles, as well as good transparency. Considering these excellent properties, the sensor shows promising potential in health monitoring, human–computer interaction, wearable electronics, etc. [ABSTRACT FROM AUTHOR]

This article uncovers how computers learned to recognize emotion from a neurological difference often conceived of as marked by a difficulty in recognizing it at all: autism. Developers of facial emotion recognition technologies have repeatedly mobilized autism to produce new kinds of knowledge at the interface of machine learning and human feeling, deploying it as a charismatic use-case, as a source for conceptual schemata for how emotion should be made computable, and as a testbed for development. The entanglement of emotion recognition and autism research shows how disability has been transformed into a rhetorical, conceptual, and material resource for the expansion of surveillance capitalism. This history has also underwritten a larger reconceptualization of emotional data, from a neglected variable in human–computer interaction to an exploitable corporate asset, a transformation that has allowed platform users' emotional lives to be mined for new forms of knowledge, value, and power. [ABSTRACT FROM AUTHOR]

Complexity intensifies when gesticulations span various scales. Traditional scale-invariant object recognition methods often falter when confronted with case-sensitive characters in the English alphabet. The literature underscores a notable gap, the absence of an open-source multi-scale un-instructional gesture database featuring a comprehensive dictionary. In response, we have created the NITS (gesture scale) database, which encompasses isolated mid-air gesticulations of ninety-five alphanumeric characters. In this research, we present a scale-centric framework that addresses three critical aspects: (1) detection of smaller gesture objects: our framework excels at detecting smaller gesture objects, such as a red color marker. (2) Removal of redundant self co-articulated strokes: we propose an effective approach to eliminate redundant self co-articulated strokes often present in gesture trajectories. (3) Scale-variant approach for recognition: to tackle the scale vs. size ambiguity in recognition, we introduce a novel scale-variant methodology. Our experimental results reveal a substantial improvement of approximately 16% compared to existing state-of-the-art recognition models for mid-air gesture recognition. These outcomes demonstrate that our proposed approach successfully emulates the perceptibility found in the human visual system, even when utilizing data from monophthalmic vision. Furthermore, our findings underscore the imperative need for comprehensive studies encompassing scale variations in gesture recognition. [ABSTRACT FROM AUTHOR]

Objective: We created and validated a scale to measure perceptions of system trustworthiness. Background: Several scales exist in the literature that attempt to assess trustworthiness of system referents. However, existing measures suffer from limitations in their development and validation. The current study sought to develop a scale based on theory and methodological rigor. Method: We conducted exploratory and confirmatory factor analyses on data from two online studies to develop the System Trustworthiness Scale (STS). Additional analyses explored the manipulation of the factors and assessed convergent and divergent validity. Results: The exploratory factor analyses resulted in a three-factor solution that represented the theoretical constructs of trustworthiness: performance, purpose, and process. Confirmatory factor analyses confirmed the three-factor solution. In addition, correlation and regression analyses demonstrated the scale's divergent and predictive validity. Conclusion: The STS is a psychometrically valid and predictive scale for assessing trustworthiness perceptions of system referents. Applications: The STS assesses trustworthiness perceptions of systems. Importantly, the scale differentiates performance, purpose, and process constructs and is adaptable to a variety of system referents. [ABSTRACT FROM AUTHOR]

• Demonstrates HCI-modelling's role in enhancing DHT design and regulation. • Evaluates UI components against NHS standards using Design Science Research. • Reveals technical feasibility of combining NHS CUI and Design System. • Quantifies usability issues in DHTs through HCI-modelling. • Proposes a robust methodology for standardizing DHT user interfaces. Digital Health Technologies (DHTs) have been shown to have variable usability as measured by efficiency, effectiveness and user satisfaction despite large-scale government projects to regulate and standardise user interface (UI) design. We hypothesised that Human-Computer Interaction (HCI) modelling could improve the methodology for DHT design and regulation, and support the creation of future evidence-based UI standards and guidelines for DHTs. Using a Design Science Research (DSR) framework, we developed novel UI components that adhered to existing standards and guidelines (combining the NHS Common User Interface (CUI) standard and the NHS Design System). We firstly evaluated the Patient Banner UI component for compliance with the two guidelines and then used HCI-modelling to evaluate the "Add New Patient" workflow to measure time to task completion and cognitive load. Combining the two guidelines to produce new UI elements is technically feasible for the Patient Banner and the Patient Name Input components. There are some inconsistencies between the NHS Design System and the NHS CUI when implementing the Patient Banner. HCI-modelling successfully quantified challenges adhering to the NHS CUI and the NHS Design system for the "Add New Patient" workflow. We successfully developed new design artefacts combing two major design guidelines for DHTs. By quantifying usability issues using HCI-modelling, we have demonstrated the feasibility of a methodology that combines HCI-modelling into a human-centred design (HCD) process could enable the development of standardised UI elements for DHTs that is more scientifically robust than HCD alone. Combining HCI-modelling and Human-Centred Design could improve scientific progress towards developing safer and more user-friendly DHTs. [ABSTRACT FROM AUTHOR]

Wearable sensors are widely used in medical applications and human–computer interaction because of their portability and powerful privacy. Human activity identification based on sensor data plays a vital role in these fields. Therefore, it is important to improve the recognition performance of different types of actions. Aiming at the problems of insufficient time-varying feature extraction and gradient explosion caused by too many network layers, a time convolution network recognition model with attention mechanism (TCN-Attention-HAR) was proposed. The model effectively recognizes and emphasizes the key feature information. The ability of extracting temporal features from TCN (temporal convolution network) is improved by using the appropriate size of the receiver domain. In addition, attention mechanisms are used to assign higher weights to important information, enabling models to learn and identify human activities more effectively. The performance of the Open Data Set (WISDM, PAMAP2 and USC-HAD) is improved by 1.13%, 1.83% and 0.51%, respectively, compared with other advanced models, these results clearly show that the network model presented in this paper has excellent recognition performance. In the knowledge distillation experiment, the parameters of student model are only about 0.1% of those of teacher model, and the accuracy of the model has been greatly improved, and in the WISDM data set, compared with the teacher's model, the accuracy is 0.14% higher. [ABSTRACT FROM AUTHOR]

In the context of the digital era, the factors influencing the cognitive load of the full ecological smart home on the elderly are mostly interconnected. Most existing studies have conducted single correlation analyses, ignoring the fact that cognitive load is the result among multiple interactions of multiple factors. Furthermore, the color, material and Finishing of the product design can also impact on the user’s perceptual needs. Therefore, exploring the grouping dynamics of cognitive load and users’ perceptual needs for color (C), material (M), and Finishing (F) of smart products can provide insights for inclusive design of smart homes. The article analyzes the asymmetric multiple concurrent causal effects of full ecological smart homes on the cognitive load of the elderly from a histological perspective using fuzzy set Qualitative Comparative Analysis (fsQCA) based on the four elements of Innovation Diffusion Theory. At the same time, principal component analysis and quantitative theory I class method are used to explore the quantitative relationship between color, material, Finishing and users’ perceptual imagery of the product. The results of the study showed that there were no necessary conditions leading to high or low cognitive load in the fsQCA analysis, indicating that the problem was the result of the interaction of multiple conditions, and the final analysis yielded three histological pathways leading to low cognitive load and one pathway leading to high load in older adults. Moreover, the study identifies the combination of colors, materials, and finishes that best represent user preferences. This study establishes a dialogue between theory, results, and cases in analyzing of the group dynamics of the impact of full ecological smart homes on the cognitive load of the elderly. It provides a theoretical basis for the development of digital inclusion enhancement strategies. [ABSTRACT FROM AUTHOR]

Artificial intelligence (AI) has great potential in ophthalmology. We investigated how ambiguous outputs from an AI diagnostic support system (AI-DSS) affected diagnostic responses from optometrists when assessing cases of suspected retinal disease. Thirty optometrists (15 more experienced, 15 less) assessed 30 clinical cases. For ten, participants saw an optical coherence tomography (OCT) scan, basic clinical information and retinal photography (‘no AI’). For another ten, they were also given AI-generated OCT-based probabilistic diagnoses (‘AI diagnosis’); and for ten, both AI-diagnosis and AI-generated OCT segmentations (‘AI diagnosis + segmentation’) were provided. Cases were matched across the three types of presentation and were selected to include 40% ambiguous and 20% incorrect AI outputs. Optometrist diagnostic agreement with the predefined reference standard was lowest for ‘AI diagnosis + segmentation’ (204/300, 68%) compared to ‘AI diagnosis’ (224/300, 75% p = 0.010), and ‘no Al’ (242/300, 81%, p = < 0.001). Agreement with AI diagnosis consistent with the reference standard decreased (174/210 vs 199/210, p = 0.003), but participants trusted the AI more (p = 0.029) with segmentations. Practitioner experience did not affect diagnostic responses (p = 0.24). More experienced participants were more confident (p = 0.012) and trusted the AI less (p = 0.038). Our findings also highlight issues around reference standard definition. [ABSTRACT FROM AUTHOR]

There is scope to enhance agricultural measurement and control systems user interactivity, which typically necessitates training for users to perform specific operations successfully. With the continuous development of natural language semantic processing technology, it has become essential to augment the user-friendliness of multifaceted control and query operations in the agricultural measurement and control sector, ultimately leading to reduced operation costs for users. The study aims to focus on command parsing. The proposed AMR-OPO semantic parsing framework is based on the natural language understanding method of Abstract Meaning Representation of Rooted Markup Graphs (AMR). It transforms the user’s natural language inputs into structured ternary (OPO) statements (operation-place-object) and converts the corresponding parameters of the user’s input commands. The framework subsequently sends the transformed commands to the relevant devices via the IoT gateway. To tackle the intricate task of parsing instructions, we developed a BERT-BiLSTM-ATT-CRF-OPO entity recognition model. This model can detect and extract entities from agricultural instructions, and precisely populate them into OPO statements. Our model shows exceptional accuracy in instruction parsing, with precision, recall, and F-value all measuring at 92.13%, 93.12%, and 92.76%, correspondingly. The findings from our experiment reveal outstanding and precise performance of our approach. It is anticipated that our algorithm will enhance the user experience offered by agricultural measurement and control systems, while also making them more user-friendly. [ABSTRACT FROM AUTHOR]

Modern materials with unique structures and functions are inspired by nature. However, nature's craftsmanship is beyond the reach of traditional handmade techniques. 3D printing technology creates the possibility of creating bionic structures. In this study, inspired by the ability of spiders to sense the capture of prey through the vibration of their webs, a spider‐web‐like, highly sensitive hydrogel sensor is designed and printed. The hydrogel is made of eco‐friendly green polyvinyl alcohol cross‐linked with microcrystalline cellulose. Surprisingly, the 3D‐printed hydrogel sensor can manipulate the sensitivity by changing the angle and strain range of the structure. Accordingly, the Gauge factor (GF) of the 3D‐printed hydrogel (θ = 60°) is up to 58.23, which is 23.9 times higher than that of the non‐3D‐printed hydrogel sensor (GF = 2.43). The GF of the 3D‐printed hydrogel at other angles still remained high at 24.74 (θ = 90°) and 12.31 (θ = 120°), respectively. It can be seen that the 3D‐printed hydrogel sensor has advanced performance. In addition, an intelligent gesture recognition system for human–computer interaction is constructed using 3D‐printed hydrogel sensors, which offers potential prospects for helping people communicate. [ABSTRACT FROM AUTHOR]

Chinese calligraphy is a symbol of cultural heritage; it is widely used in visual design (e.g., film posters and modern fashion) for its aesthetic value. In human-computer interaction, visual design complexity strongly influences users' preferences. Indeed, the relevance of visual complexity to aesthetic preference has been confirmed. However, the visual complexity of calligraphy artworks has seldom been investigated and the process of how visual complexity affects aesthetic preference remains vague. Therefore, in this work we proposed a computational method to evaluate complexity and conducted several perception studies. Results showed that layout features and calligraphy style (regular script, running script, and cursive script) affected visual complexity. The level of visual complexity (low, medium, and high) affected aesthetic preference but calligraphy style did not. Furthermore, Chinese calligraphy with moderate visual complexity evokes strong aesthetic preference. The present findings can help designers redesign websites and interfaces for high aesthetic preference and can provide insights for developing the theoretical design of calligraphic art for advanced interaction in cultural heritage, such as in interactive systems for teaching calligraphy based on aesthetic cognition. [ABSTRACT FROM AUTHOR]

Many forms of Intangible Culture Heritage (ICH) are fast disappearing. The Lantern Festival is an important part of Chinese tradition and a critical ICH of Taiwan. However, few young people attend ICH activities. Given the importance of ICH education, this study describes what participants learned after lantern exhibitions that combined Augmented Reality (AR), mobile websites, and Paper-based Exhibition Brochures. This study evaluated what a cohort of college students (n = 115) learned about ICH through experiments and questionnaires. The results show that the public understood ICH better through digital technology. All participants thought that applying AR to ICH was good for the public to gain more information on lantern culture, although it took a lot of time to learn the new technology. The results also encouraged the application of AR to engage in ICH education. Therefore, a large number of digital technologies that are popular among young people should be used as part of ICH education in order to improve their cultural awareness and learning motivation. [ABSTRACT FROM AUTHOR]

Background Objectives Methods Results Discussion Conclusion Stakeholders working on the COVID‐19 pandemic response needed access to evidence, requiring a systematic approach to identify and disseminate relevant research.Outline the stages of development of a COVID‐19 Literature Digest; demonstrate the impact the Digest had on decision‐making and knowledge gain; identify the lessons learned.A standardised process was developed to identify and select papers. The main sources for content were PubMed, bioRxiv and medRxiv. A shared EndNote library was used to deduplicate and organise papers. Three user surveys obtained feedback from subscribers to determine if the Digest remained valuable, and explore the benefits to individuals.40–60 papers were summarised each week. 211 Digests were produced from March 2020 to March 2022, with around 10,000 papers included altogether. Survey results suggest benefits of the Digest were gaining new knowledge, saving time and contributing to evidence‐based decision making.Digest procedures constantly evolved and were adapted in response to survey feedback. Lessons identified: learn from failure, communication is key, measure your impact, work collaboratively, reflect and be flexible.The Digest was successfully produced within the limits of available resource. The learning from this Digest will inform evidence monitoring, selection and dissemination for future health crises. [ABSTRACT FROM AUTHOR]

AbstractThe utilization of Empathy Maps in User Experience and Design Thinking is becoming commonplace. They can be seen as a valuable tool to help developers better understand the user’s requirements. During the interaction design, it is essential to have a thorough knowledge of what the user wants, which might be difficult to gather or portray to others. In this sense, Empathy Maps visually represent the user’s information, attitude, and behavior. Nevertheless, creating Empathy Maps can become tedious, as this usually requires the involvement of a group of people with expert knowledge, deeply analyzing the information elicited from the user. This is the reason why most Empathy Maps are created manually. A first literature analysis reports that there are still no specific tools to generate Empathy Maps automatically, although there are technologies to take on this challenge. In this paper, we present a tool for automatically generating an initial version of an Empathy Map that can help speed up the capture of explicit knowledge, and then it can be further worked on by the development team. The tool creates the Empathy Map from an audio interview, featuring audio transcription technology and natural language processing. More specifically, this paper includes the architecture, a proof of concept, and user testing, reporting numerical evidence and promising results to corroborate the proposal’s feasibility, reaching an overall accuracy of 86% at classification. [ABSTRACT FROM AUTHOR]

Autistic Children often struggle with social interaction and communication, studies have found that many of them prefer to interact with objects than people. However, there is a lack of research exploring the specific characteristics and factors involved in interactions within families with autistic children where objects are the center of the interaction. This paper describes the process and findings of a diary study exploring how young autistic children interact with their families through objects in natural scenarios. A one-week diary study was conducted with six families with young autistic children. Diary videos were recorded onsite and coded later according to a social interaction behavior scheme with corresponding diary entries. Qualitative data analysis was conducted to reveal possible patterns. Results revealed ongoing difficulties in establishing and maintaining family interaction and identified influential factors of object-centered family interaction. The most prevalent pattern observed was parents taking the lead in interactions, followed by the child's confirmation response. Remarkably, daily necessities emerged as potential physical mediums for enhancing family interactions, opening avenues for exploring tangible designs in human–computer interaction. These findings offer valuable implications for future research and the development of innovative designs that promote enriching interactions for autistic children and their families. [ABSTRACT FROM AUTHOR]

Abstract\nIMPLICATIONS FOR REHABILITATIONBackground: Knowledge-to-action gap exists in delivering physical activity (PA) to people with developmental disabilities via online platforms. Although web-based platforms have great potential in facilitating the delivery of PA for this target group, the lack of knowledge regarding web accessibility poses a challenge in accessing PA-related information online.Objective: This study evaluates the delivery of PA in terms of web accessibility. It also aims to identify barriers and facilitators in delivering PA knowledge to people with developmental disabilities online to improve web accessibility for the target user group.Methods: The study employs a concurrent nested design incorporating both quantitative (web usability questionnaire) and qualitative data (in-depth interviews). Fifteen pairs of individuals consisting of a person with developmental disabilities and a primary caregiver participated in the study, and three web-based platforms were selected for web accessibility tests and in-depth interviews.Results: The nested analysis provides a quantitative comparison of web accessibility and identifies barriers and facilitators of delivering PA for the target user group from the web accessibility perspective. Conclusion: The study findings could inform the development of accessible online platforms that distribute health-related knowledge to populations with developmental disabilities. Additionally, they could help enhance the design of other platforms intended for these populations.Online platforms have significant potential to improve the delivery of physical activity information to individuals with developmental disabilities.Barriers to accessing online platforms due to poor web usability impede the process of online knowledge translation to end-users.Web usability can be enhanced by implementing appropriate interventions, such as restructuring navigation and redesigning the user interface.Improving web usability will enhance the accessibility of physical activity knowledge for individuals with developmental disabilities, which consequently will positively impact their health.Online platforms have significant potential to improve the delivery of physical activity information to individuals with developmental disabilities.Barriers to accessing online platforms due to poor web usability impede the process of online knowledge translation to end-users.Web usability can be enhanced by implementing appropriate interventions, such as restructuring navigation and redesigning the user interface.Improving web usability will enhance the accessibility of physical activity knowledge for individuals with developmental disabilities, which consequently will positively impact their health. [ABSTRACT FROM AUTHOR]

Machine learning and speech emotion recognition are rapidly evolving fields, significantly impacting human-centered computing. Machine learning enables computers to learn from data and make predictions, while speech emotion recognition allows computers to identify and understand human emotions from speech. These technologies contribute to the creation of innovative human–computer interaction (HCI) applications. Deep learning algorithms, capable of learning high-level features directly from raw data, have given rise to new emotion recognition approaches employing models trained on advanced speech representations like spectrograms and time–frequency representations. This study introduces CNN and LSTM models with GWO optimization, aiming to determine optimal parameters for achieving enhanced accuracy within a specified parameter set. The proposed CNN and LSTM models with GWO optimization underwent performance testing on four diverse datasets—RAVDESS, SAVEE, TESS, and EMODB. The results indicated superior performance of the models compared to linear and kernelized SVM, with or without GWO optimizers. [ABSTRACT FROM AUTHOR]

Speech is one of the most important human bio-signals. However, only some speech production characteristics are fully understood, which are required for a successful speech-based Brain-Computer Interface (BCI). A proper brain-to-speech system that can generate the speech of full sentences intelligibly and naturally poses a great challenge. In our study, we used the SingleWordProduction-Dutch-iBIDS dataset, in which speech and intracranial stereotactic electroencephalography (sEEG) signals of the brain were recorded simultaneously during a single word production task. We apply deep neural networks (FC-DNN, 2D-CNN, and 3D-CNN) on the ten speakers’ data for sEEG-to-Mel spectrogram prediction. Next, we synthesize speech using the WaveGlow neural vocoder. Our objective and subjective evaluations have shown that the DNN-based approaches with neural vocoder outperform the baseline linear regression model using Griffin-Lim. The synthesized samples resemble the original speech but are still not intelligible, and the results are clearly speaker dependent. In the long term, speech-based BCI applications might be useful for the speaking impaired or those having neurological disorders. [ABSTRACT FROM AUTHOR]