Search

In this paper, we propose a 3D color picker for virtual reality (VR) systems. Using the proposed color picker, the RGB and HSV color spaces are displayed in 3D shape, and a color can be selected through a one-step interaction by pointing at a location inside the color space with a 3D pointing device. This process is very simple compared with the multistep color selection interactions required for existing VR color pickers. Since the interior of the color space is not visible, we also propose three types of user interfaces that show the interior of the color space. These interfaces either cut out a portion of the color space, show cross-sections of the color space, or show discrete samples placed in the color space. We conducted user experiments on the color selection time, color selection accuracy, and usability and proved the usefulness and effectiveness of the proposed VR color picker.

The rapid development and popularity of IoT technology has reshaped the way people interact with the real world. Many researchers have attempted to build natural language interfaces for IoT platforms, but have not produced much progress in parsing natural language commands that contain multiple operations and more complex logical structures. In this paper, we propose IoT-NLI, a natural language query and control interface for popular IoT platforms, which uses hierarchical semantic parsing algorithms and directed edge-tagged graph structures to efficiently parse natural language commands input by users, enabling them to perform multiple operations contained in one complex natural language command. Experiments in three domains, agriculture, industry, and smart home, show that IoT-NLI has excellent performance and reasonable response time. Finally, a IoT-NLI application was developed on the Android platform and integrated with the AliCloud platform. It enables users to query and control devices on Android phones through chat windows similar to instant messaging software.

Many fields of study still face the challenges inherent to the analysis of complex multidimensional datasets, such as the field of computational biology, whose research of infectious diseases must contend with large protein-protein interaction networks with thousands of genes that vary in expression values over time. In this paper, we explore the visualization of multivariate data through CroP, a data visualization tool with a coordinated multiple views framework where users can adapt the workspace to different problems through flexible panels. In particular, we focus on the visualization of relational and temporal data, the latter being represented through layouts that distort timelines to represent the fluctuations of values across complex datasets, creating visualizations that highlight significant events and patterns. Moreover, CroP provides various layouts and functionalities to not only highlight relationships between different variables, but also dig-down into discovered patterns in order to better understand their sources and their effects. These methods are demonstrated through multiple experiments with diverse multivariate datasets, with a focus on gene expression time-series datasets. In addition to a discussion of our results, we also validate CroP through model and interface tests performed with participants from both the fields of information visualization and computational biology.

Healthcare has evolved significantly over time, from traditional healthcare systems to cutting-edge medical technologies. As these technologies advance, researchers have become interested in their usability. The usefulness of healthcare websites helps to provide more precise medical information. A comprehensive review of the literature is required to identify usability features, techniques, and issues in healthcare websites over a specified time period. In this study, articles from the years 2017–2021 are reviewed from well-known digital libraries i.e, IEEE, ACM, and ScienceDirect that include papers from various conferences, magazines, books, and journals. Initially, the study found 10,512 titles based on the search string developed from the proposed research questions which were then further filtered down to a total of 55 papers. This systematic literature review (SLR) summarises and collects relevant data in response to pre-defined research questions. This analysis of existing research will help website designers and developers, in developing more user-friendly healthcare websites for the users. In the future, this SLR will help in determining the optimal solutions and developing a framework for the identified usability challenges and limitations. It also includes employing the usability evaluation tools discovered by researchers to identify and fix usability issues on websites.

People treat conversational agents as mere tools and not as human-like social actors. While there has been much research on human-like agents, few studies have approached the realization of a conversational agent as a social actor from the viewpoint of speech synthesis. This study investigated the effect of synthetic voice of conversational agent trained with spontaneous speech on human interactants. Specifically, we hypothesized that humans will exhibit more social responses when interacting with conversational agent that has a synthetic voice built on spontaneous speech. Typically, speech synthesizers are built on a speech corpus where voice professionals read a set of written sentences. The synthesized speech is clear as if a newscaster were reading the news or a voice actor were playing an anime character. However, this is quite different from spontaneous speech we speak in everyday conversation. Recent advances in speech synthesis enabled us to build a speech synthesizer on a spontaneous speech corpus, and to obtain a near-conversational synthesized speech with reasonable quality. By making use of these technologies, we examined whether humans produce more social responses to a spontaneously speaking conversational agent. We conducted a large-scale conversation experiment with a conversational agent whose utterances were synthesized with the model trained either with spontaneous speech or read speech. The result showed that the subjects who interacted with the agent whose utterances were synthesized from spontaneous speech tended to show shorter response time and a larger number of backchannels. The result of a questionnaire showed that subjects who interacted with the agent whose utterances were synthesized from spontaneous speech tended to rate their conversation with the agent as closer to a human conversation. These results suggest that speech synthesis built on spontaneous speech is essential to realize a conversational agent as a social actor.

Exergames and serious games, based on standard personal computers, mobile devices and gaming consoles or on novel immersive Virtual and Augmented Reality techniques, have become popular in the last few years and are now applied in various research fields, among which cognitive assessment and training of heterogeneous target populations. Moreover, the adoption of Web based solutions together with the integration of Artificial Intelligence and Machine Learning algorithms could bring countless advantages, both for the patients and the clinical personnel, as allowing the early detection of some pathological conditions, improving the efficacy and adherence to rehabilitation processes, through the personalisation of training sessions, and optimizing the allocation of resources by the healthcare system. The current work proposes a systematic survey of existing solutions in the field of cognitive assessment and training. We evaluate the visualization and interaction technologies commonly adopted and the measures taken to fulfil the need of the pathological target populations. Moreover, we analyze how implemented solutions are validated, i.e. the chosen experimental designs, data collection and analysis. Finally, we consider the availability of the applications and raw data to the large community of researchers and medical professionals and the actual application of proposed solutions in the standard clinical practice. Despite the potential of these technologies, research is still at an early stage. Although the recent release of accessible immersive virtual reality headsets and the increasing interest on vision-based techniques for tracking body and hands movements, many studies still rely on non-immersive virtual reality (67.2%), mainly mobile and personal computers, and standard gaming tools for interactions (41.5%). Finally, we highlight that although the interest of research community in this field is increasingly higher, the sharing of dataset (10.6%) and implemented applications (3.8%) should be promoted and the number of healthcare structures which have successfully introduced the new technological approaches in the treatment of their host patients is limited (10.2%).

Despite the utility of gaze gestures as an input method, there is a lack of guidelines available regarding how to design gaze gestures, what algorithms to use for gaze gesture recognition, and how these algorithms compare in terms of performance. To facilitate the development of applications that leverage gaze gestures, we have evaluated the performance of a combination of template-based and data-driven algorithms on two custom gesture sets that can map to user actions. Template-based algorithms had consistently high accuracies but the slowest runtimes, making them best for small gesture sets or accuracy-critical applications. Data-driven algorithms run much faster and scale better to larger gesture sets, but require more training data to achieve the accuracy of the template-based methods. The main takeaways for gesture set design are 1) gestures should have distinct forms even when performed imprecisely and 2) gestures should have clear key-points for the eyes to fixate onto.

In order to improve the intelligence and humanization of the home environment, smart air conditioners are used as the research objects. User needs using the theory of Human–Computer Interaction (HCI) are investigated and researched, and smart air conditioners are researched and practiced on human interaction design. Firstly, in order to establish a user model, the questions and needs of the target users are explored through the questionnaire survey method. Secondly, from the perspective of ergonomics, the hardware and software interaction interface of the smart air conditioner is analyzed. Finally, the intelligent air conditioner is designed for HCI according to user needs, and the unsupervised feature value extraction of the vibration measurement signal of the outdoor unit of the household air conditioner is carried out by using the stacked autoencoder neural network. Three preliminary options for the design of remote controllers for air conditioners are proposed, and the fuzzy evaluation method is utilized to analyze and evaluate the three options. The research results show that the comprehensive evaluation results of the three preliminary options of the remote-control design are 0.78, 0.77, and 0.8 respectively. Compared with Option 1 and Option 2, Option 3 has obvious advantages. The design of Option 3 is more prominent in terms of comfort, aesthetics, and rationality. Therefore, Option 3 is selected as the final design solution. Under different hidden layer numbers and node numbers, the classification accuracy rate changes in a convex function. When the number of layers is 3 and the number of nodes is 100, the classification accuracy rate is the highest. According to the needs of users, a specific interactive design analysis is carried out on the hardware design of home appliances from the perspective of ergonomics.

Students’ motivation in computer graphic courses is a challenge for educators. Computer courses entail abstract notions and algorithmic features, and these difficulties affect the students’ motivation. A problem-based learning serious game framework named Immersivio was designed. The researchers designed a quantitative study with a between subject design and implemented a treatment course using the problem-based serious game. The game was designed based on the principles of problem-based learning and serious games, i.e., higher order thinking, collaborative learning, and cognitive thinking and information visualization theory. A non-random sampling procedure (purposive sampling) was used in this quasi-experimental study and the participants formed to 2 groups, the experimental group (n = 24) and the control group (n = 26). MANOVA test results indicated that the experimental group participants who had experienced the Immersivio game were more motivated to learn computer graphics in terms of extrinsic motivation, intrinsic motivation, interest, attainment, cost, identification with academics, self-efficacy, and instrumentality. This study has implications for basic and advanced computer graphic lecturers and educators in this field.

Hearing-impaired people use sign language to express their thoughts and emotions and reinforce information delivered in daily conversations. Though they make a significant percentage of any population, the majority of people can’t interact with them due to limited or no knowledge of sign languages. Sign language recognition aims to detect the significant motions of the human body, especially hands, analyze them and understand them. Such systems may become life-saving when hearing-challenged people are in desperate situations like heart attacks, accidents, etc. In the present study, deep learning-based hand gesture recognition models are developed to accurately predict the emergency signs of Indian Sign Language (ISL). The dataset used contains the videos for eight different emergency situations. Several frames were extracted from the videos and are fed to three different models. Two models are designed for classification, while one is an object detection model, applied after annotating the frames. The first model consists of a three-dimensional convolutional neural network (3D CNN), while the second comprises of a pre-trained VGG-16 and a recurrent neural network with a long short-term memory (RNN-LSTM) scheme. The last model is based on YOLO (You Only Look Once) v5, an advanced object detection algorithm. The prediction accuracies of the classification models were 82% and 98%, respectively. YOLO based model outperformed the rest and achieved an impressive mean average precision of 99.6%.

The pandemic has accelerated the process of digitization in many fields, highlighting some critical issues that have slowed the development of digital technologies in many areas. One of these is undoubtedly that of skills assessment, which on a digital platform is a more streamlined, reliable and faster process and allows some individuals from disadvantaged groups to be able to carry out the exam more easily. We present some strategies for enabling online remote secure written exams and their implementation through the open-source system LibreEOL. The system can be used for university exams (in presence or from remote locations) and training courses of all types to evaluate the effectiveness of learning in itinere, i.e., before, during and after a course, to provide quantitative data on training effectiveness. We outline the architecture of the services underpinning the system and the logical organisation of the various user roles and content relating to questions and exam delivery. We describe the algorithms that have allowed us to develop an environment that appears to be an essential aid both for teachers and students in light of the questionnaires conducted on students. The system is successfully used in the EchemTest, promoted by the European Chemistry Thematic Network, and allows comparison of chemistry training outcomes in different European countries. Despite the importance, it has taken on in the last two years due to the pandemic, such a system makes it possible to carry out a valuable skills assessment.

The advances on the Internet and news media technologies suggest intelligent and personalised media interfaces in order to improve reading efficiency of news readers and enhance news disseminations. Cognitive load is one of factors that affect the understanding ability of news readers and therefore news dissemination. It is expected that the display elements in an intelligent user interface of news media could be automatically adjusted to modulate the audience’s perceived cognitive load level in order to improve the news reading efficiency. While the font style and font colour of news text are the main display elements in the user interface of news media, it is not clear how these elements affect perceived cognitive load of news readers. This paper investigates perceived cognitive load of news readers under different font style and font colour conditions. Experiments with the news text in Chinese as a case study found that the change of text font style of keywords resulted in the increase of the reader’s perceived cognitive load during reading news text under the low introduced cognitive load and blue colour displayed keywords. While under the high introduced cognitive load, the italic font and red colour of keywords text resulted in the decrease of the reader’s perceived cognitive load significantly during reading news text, which therefore improves the news reading efficiency. This paper is limited to the effects of two aspects of news text (font styles and colours) on user’s perceived cognitive load. The future work will focus on the investigation of effects of other factors such as news pictures and their colours, numbers, and locations on user’s perceived cognitive load.

This paper explores Null Island, a fictional place located at 0° latitude and 0° longitude in the WGS84 (World Geodetic System 1984) geographic coordinate system. Null Island is erroneously associated with large amounts of geographic data in a wide variety of location-based services, place databases, social media and web-based maps. Whereas it was originally considered a joke within the geospatial community, this article will demonstrate implications of its existence, both technological and social in nature, promoting Null Island as a fundamental issue of geographic information that requires more widespread awareness. The article summarizes error sources that lead to data being associated with Null Island. We identify four evolutionary phases which help explain how this fictional place evolved and established itself as an entity reaching beyond the geospatial profession to the point of being discovered by the visual arts and the general population. After providing an accurate account of data that can be found at (0, 0), geospatial, technological and social implications of Null Island are discussed. Guidelines to avoid misplacing data to Null Island are provided. Since data will likely continue to appear at this location, our contribution is aimed at academics, computing professionals and the general population to promote awareness of this error source.

Human intention is an internal, mental characterization for acquiring desired information. From interactive interfaces containing either textual or graphical information, intention to perceive desired information is subjective and strongly connected with eye gaze. In this work, we determine such intention by analyzing real-time eye gaze data with a low-cost regular webcam. We extracted unique features (e.g., Fixation Count, Eye Movement Ratio) from the eye gaze data of 31 participants to generate a dataset containing 124 samples of visual intention for perceiving textual or graphical information, labeled as either TEXT or IMAGE, having 48.39% and 51.61% distribution, respectively. Using this dataset, we analyzed 5 classifiers, including Support Vector Machine (SVM) (Accuracy: 92.19%). Using the trained SVM, we investigated the variation of visual intention among 30 participants, distributed in 3 age groups, and found out that young users were more leaned towards graphical contents whereas older adults felt more interested in textual ones. This finding suggests that real-time eye gaze data can be a potential source of identifying visual intention, analyzing which intention aware interactive interfaces can be designed and developed to facilitate human cognition.

Real-time hand gesture recognition plays a vital role in human-computer interaction (HCI). Recent radar-based hand gesture recognition methods have focused on achieving high classification accuracy using deep neural network (DNN)-based classifiers. However, the hand gesture recognition system should not only classify the gestures accurately but also detect out-of-distribution (OOD) samples to be used in real-world HCI scenarios with high reliability. Recognition systems without OOD detection capability misclassify unintended gestures in silence, especially in real-time scenarios. To tackle this problem, we propose a real-time hand gesture recognition system that can simultaneously classify hand gestures and detect OOD samples by using a Frequency Modulated Continuous Wave (FMCW) radar sensor. First, we design radar data processing technique and Transformer encoder-based classifier to achieve high classification accuracy. Second, the relative Mahalanobis distance (RMD)-based OOD detection method is adopted to increase the reliability of the proposed system. Finally, one in-distribution dataset and two OOD datasets are collected to verify the proposed system. The proposed system achieves a classification accuracy of 93.95% on the in-distribution dataset. We conduct the OOD detection experiments with two OOD datasets for which the proposed system reports AUROC values of 92.96% and 92.84%, respectively. Furthermore, the feasibility of the proposed system is certified through a real-time experimental demonstration.

Hand gesture recognition is essential to human-computer interaction as the most natural way of communicating. Furthermore, with the development of 3D hand pose estimation technology and the performance improvement of low-cost depth cameras, skeleton-based dynamic hand gesture recognition has received much attention. This paper proposes a novel multi-stream improved spatio-temporal graph convolutional network (MS-ISTGCN) for skeleton-based dynamic hand gesture recognition. We adopt an adaptive spatial graph convolution that can learn the relationship between distant hand joints and propose an extended temporal graph convolution with multiple dilation rates that can extract informative temporal features from short to long periods. Furthermore, we add a new attention layer consisting of effective spatio-temporal attention and channel attention between the spatial and temporal graph convolution layers to find and focus on key features. Finally, we propose a multi-stream structure that feeds multiple data modalities (i.e., joints, bones, and motions) as inputs to improve performance using the ensemble technique. Each of the three-stream networks is independently trained and fused to predict the final hand gesture. The performance of the proposed method is verified through extensive experiments with two widely used public dynamic hand gesture datasets: SHREC’17 Track and DHG-14/28. Our proposed method achieves the highest recognition accuracy in various gesture categories for both datasets compared with state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Interacting with computer applications using actions that are designed by end users themselves instead of pre-defined ones has advantages such as better memorability in some Human-Computer Interaction (HCI) scenarios. In this paper we propose a method for allowing users to use self-defined mid-air hand gestures as commands for HCI after they provide a few training samples for each gesture in front of a depth image sensor. The gesture detection and recognition algorithm is mainly based on pattern matching using 3 separate sets of features, which carry both finger-action and hand-motion information. An experiment in which each subject designed their own set of 8 gestures, provided about 5 samples for each, and then used them to play a game is conducted all in one sitting. During the experiment a recognition rate of 66.7% is achieved with a false positive ratio of 22.2%. Further analyses on the collected dataset shows that a higher recognition rate of up to about 85% can be achieved if more wrong detections were allowed. [ABSTRACT FROM AUTHOR]

In this work we propose a new and innovative process for the induction of emotional states, through visual stimulus based on interactive user interfaces (UI), to create a dataset of emotional physiological signals that can be used in the evaluation of the user experience (UX). Most existing datasets of emotional physiological signals, were generated based on images and videos, which are not useful for analysis of emotions in the evaluation process of the user experience, for this reason in this research, we propose a new emotional induction process that allows the creation of a physiological emotion dataset focused on the evaluation process of user experience. The population sample used for the generated emotional induction process is 15 users, 7 women and 8 men, which culminated in the creation of a dataset of 333 physiological signal files plus data from the SAM questionnaires and knowing the user. The information from these questionnaires was used to perform the statistical analysis of the data, which helped to determine the relationship that exists between the study variables. Some of the future activities planned are to increase the sample size of the datase, increase the repository of stimuli for digital products, etc. [ABSTRACT FROM AUTHOR]

Novel view synthesis of remote sensing (RS) scenes is of great significance for scene visualization, human–computer interaction, and various downstream applications. Despite the recent advances in computer graphics and photogrammetry technology, generating novel views is still challenging particularly for RS images due to its high complexity, view sparsity, and limited view-perspective variations. In this article, we propose a novel RS view synthesis method by leveraging the recent advances in implicit neural representations. Considering the overhead and far depth imaging of RS images, we represent the 3-D space by combining implicit multiplane images (ImMPI) representation and deep neural networks. The 3-D scene is reconstructed under a self-supervised optimization paradigm through a differentiable multiplane renderer with multiview input constraints. Images from any novel views thus can be freely rendered on the basis of the reconstructed model. As a by-product, the depth maps corresponding to the given viewpoint can be generated along with the rendering output. We refer to our method as ImMPIs. To further improve the view synthesis under sparse-view inputs, we explore the learning-based initialization of RS 3-D scenes and proposed a neural-network-based prior extractor to accelerate the optimization process. In addition, we propose a new dataset for RS novel view synthesis with multiview real-world Google Earth images. Extensive experiments demonstrate the superiority of the ImMPI over previous state-of-the-art methods in terms of reconstruction accuracy, visual fidelity, and time efficiency. Ablation experiments also suggest the effectiveness of our methodology design. [ABSTRACT FROM AUTHOR]

Gesture recognition has become increasingly important in human-computer interaction and can support different applications such as smart home, VR, and gaming. Traditional approaches usually rely on dedicated sensors that are worn by the user or cameras that require line of sight. In this paper, we present a fine-grained finger gesture recognition system by using commodity WiFi without requiring user to wear any sensors. Our system takes advantages of the fine-grained Channel State Information available from commodity WiFi devices and the prevalence of WiFi network infrastructures. It senses and identifies subtle movements of finger gestures by examining the unique patterns exhibited in the detailed CSI. We devise environmental noise removal mechanism to mitigate the effect of signal dynamic due to the environment changes. Moreover, we propose to capture the intrinsic gesture behavior to deal with individual diversity and gesture inconsistency. Lastly, we utilize multiple WiFi links and larger bandwidth at 5GHz to achieve finger gesture recognition under multi-user scenario. Our experimental evaluation in different environments demonstrates that our system can achieve over 90 percent recognition accuracy and is robust to both environment changes and individual diversity. Results also show that our system can provide accurate gesture recognition under different scenarios. [ABSTRACT FROM AUTHOR]

Hand gestures are a well-known and intuitive method of human-computer interaction. The majority of the research has concentrated on hand gesture recognition from the RGB images, however, little work has been done on recognition from videos. In addition, RGB cameras are not robust in varying lighting conditions. Motivated by this, we present the video based hand gestures recognition using the depth camera and a light weight convolutional neural network (CNN) model. We constructed a dataset and then used a light weight CNN model to detect and classify hand movements efficiently. We also examined the classification accuracy with a limited number of frames in a video gesture. We compare the depth camera’s video gesture recognition performance to that of the RGB camera. We evaluate the proposed model’s performance on edge computing devices and compare to benchmark models in terms of accuracy and inference time. The proposed model results in an accuracy of 99.48% on the RGB version of test dataset and 99.18% on the depth version of test dataset. Finally, we compare the accuracy of the proposed light weight CNN model with the state-of-the hand gesture classification models. [ABSTRACT FROM AUTHOR]

The thrill of scientific discovery, the excitement of engineering development, and the fresh thinking of design explorations were invigorating as we participated in the birth of a new discipline: Information Visualization. This discipline, based on graphical user interfaces with pointing devices, became possible as software matured, hardware sped up, and screen resolution improved. Driven by the concepts of direct manipulation and dynamic queries, we made interactive interfaces that empowered users and opened up new possibilities for the next generation of designers. We worked with professionals who had real problems and tested real users to get their feedback. Some projects failed and some papers never got published, but many of the new ideas found their way into widely used commercial products. Our great satisfaction is that our students have spread the community spirit of the Human-Computer Interaction Laboratory as they continue to make further contributions. [ABSTRACT FROM AUTHOR]

In this paper, we aim to explore the fine-grained perception ability of deep models for the newly proposed scene sketch semantic segmentation task. Scene sketches are abstract drawings containing multiple related objects. It plays a vital role in daily communication and human-computer interaction. The study has only recently started due to a main obstacle of the absence of large-scale datasets. The currently available dataset SketchyScene is composed of clip art-style edge maps, which lacks abstractness and diversity. To drive further research, we contribute two new large-scale datasets based on real hand-drawn object sketches. A general automatic scene sketch synthesis process is developed to assist with new dataset composition. Furthermore, we propose to enhancing local detail perception in deep models to realize accurate stroke-oriented scene sketch segmentation. Due to the inherent differences between hand-drawn sketches and natural images, extreme low-level local features of strokes are incorporated to improve detail discrimination. Stroke masks are also integrated into model training to guide the learning attention. Extensive experiments are conducted on three large-scale scene sketch datasets. Our method achieves state-of-the-art performance under four evaluation metrics and yields meaningful interpretability via visual analytics. [ABSTRACT FROM AUTHOR]

Thanks to the powerful capability of the feature extraction, deep learning has become a promising technology for an increasing number of researchers to decode movements from surface Electromyography (sEMG) signals. The mainstream methods of deep learning are based on Convolutional Neural Networks (CNN) or Recurrent Neural Networks (RNN). However, the sequential features of sEMG signals will be ignored in CNN-based approaches while the training of the neural networks is much time-consuming in RNN-based approaches. To solve these problems, a novel convolutional vision transformer (CviT) with stacking ensemble learning is proposed in this paper, which has great potential in the fusion of sequential and spatial features of sEMG signals with the parallel training. In NinaPro DB2, the proposed method achieves 80.02% with the window length of 200ms. In the subset of NinaPro DB2 (Exercise E1), the proposed method achieves 83.47% and 84.09% with the window length of 200ms and 300ms respectively. In the subsets of NinaPro DB5 (Exercise A, Exercise B), the proposed method achieves 76.83% and 73.23% respectively. The experimental results demonstrate that the proposed CviT has better performance than most current approaches. In addition, the successful application of Transformer in sEMG-based movements classification provides a significant reference for the application of Transformer in other biological signals. [ABSTRACT FROM AUTHOR]

The haptic sensation of low-frequency vibration plays a vital role in the music listening experience, but it can be enjoyed only in certain facilities and environments. Many haptic devices have been proposed to convey music audio-induced vibration for various situations. Such devices require powerful, low-frequency vibration output and transmission over a wide area. Making such devices small and user-friendly is difficult, hindering their popularity. To promote haptic devices for music listening, this paper describes a method for developing a practical device using motors and a thread and evaluates this method's effectiveness. The proposed necklace-type device is small (about 55 $\mathbf \;{\times }$ 58 $\mathbf {\times }$ 15 mm), lightweight (58.5 g), and easy to wear, making it suitable for use during everyday travel. In addition, it can transmit low-frequency (20 Hz) vibrations, whose amplitude exceeds airborne vibration in a nightclub, to a wide area across the chest and neck, with a total power consumption of approximately 2 W. Our proposed method will contribute to the development of practical and high-performance haptic devices for music listening. [ABSTRACT FROM AUTHOR]

Haptic feedback is critical in a broad range of human-machine/computer-interaction applications. However, the high cost and low portability/wearability of haptic devices remain unresolved issues, severely limiting the adoption of this otherwise promising technology. Electrotactile interfaces have the advantage of being more portable and wearable due to their reduced actuators’ size, as well as their lower power consumption and manufacturing cost. The applications of electrotactile feedback have been explored in human-computer interaction and human-machine-interaction for facilitating hand-based interactions in applications, such as prosthetics, virtual reality, robotic teleoperation, surface haptics, portable devices, and rehabilitation. This article presents a technological overview of electrotactile feedback, as well a systematic review and meta-analysis of its applications for hand-based interactions. We discuss the different electrotactile systems according to the type of application. We also discuss over a quantitative congregation of the findings, to offer a high-level overview into the state-of-art and suggest future directions. Electrotactile feedback systems showed increased portability/wearability, and they were successful in rendering and/or augmenting most tactile sensations, eliciting perceptual processes, and improving performance in many scenarios. However, knowledge gaps (e.g., embodiment), technical (e.g., recurrent calibration, electrodes’ durability) and methodological (e.g., sample size) drawbacks were detected, which should be addressed in future studies. [ABSTRACT FROM AUTHOR]

The touchless techniques in human-computer interaction (HCI) can effectively expand computer access capabilities for disabled people. This paper presents Touchless Head-Control (THC), an assistive system method for computer cursor control based on head pose captured with an RGB camera. Our work aimed to replace the standard cursor control using a device on the user’s head. The convolutional neural networks with predicted fine-grained feature maps and binned classification were applied to estimate the head pose angles. The mouse pointer or cursor is moved to actual locations on the screen based on head movement (yaw and pitch) and the center position of the face. Head tilt to the right or left (roll) to control the mouse button. In addition, the proposed method can be used to simulate the movement of the robot or joystick using the head to control objects within three degrees of freedom (DOF). Various participants were involved in the interaction design evaluation, in which target selection accuracy, travel time, and path efficiency were measured. This technology allows people with limited motor skills to easily control a PC cursor and 3D object orientation without the use of additional equipment or sensors. [ABSTRACT FROM AUTHOR]

Gesture recognition is the central enabler of human-computer interaction (HCI). In addition to the semantic information contained in gestures, gesture-based user identification can effortlessly enhance HCI system security. Recently, the Wi-Fi-integrated sensing and communication (ISAC) technology has shown great potential in a field hitherto occupied by computer vision and radar sensing. In this work, leveraging Wi-Fi sensing, we propose a system called WiGesID that achieves joint gesture recognition and human identification (JGRHI). The basic idea behind WiGesID is to identify personalized spatiotemporal dynamic patterns from the gestures of different users. Moreover, we develop an effective approach to recognize new categories of gestures and users by computing relation scores between the features of the new category samples and the support samples. To evaluate the performance, we implemented WiGesID and conducted extensive experiments. The results demonstrate that our system outperforms the state-of-the-art method for cross-domain sensing and accurately recognizes new categories, which promotes the use of this application of Wi-Fi sensing in HCI. [ABSTRACT FROM AUTHOR]

In this article, we introduce a next-generation annotation tool called NOVA for emotional behaviour analysis, which implements a workflow that interactively incorporates the ‘human in the loop’. A main aspect of NOVA is the possibility of applying semi-supervised active learning where Machine Learning techniques are used already during the annotation process by giving the possibility to pre-label data automatically. Furthermore, NOVA implements recent eXplainable AI (XAI) techniques to provide users with both, a confidence value of the automatically predicted annotations, as well as visual explanations. We investigate how such techniques can assist non-experts in terms of trust, perceived self-efficacy, cognitive workload as well as creating correct mental models about the system by conducting a user study with 53 participants. The results show that NOVA can easily be used by non-experts and lead to a high computer self-efficacy. Furthermore, the results indicate that XAI visualisations help users to create more correct mental models about the machine learning system compared to the baseline condition. Nevertheless, we suggest that explanations in the field of AI have to be more focused on user-needs as well as on the classification task and the model they want to explain. [ABSTRACT FROM AUTHOR]

Advances in human-computer interaction (HCI) technologies have granted sonographers and radiologists a much improved user experience when operating different ultrasound (US) machines. Continued HCI improvements in US would benefit from a systematic study of the HCI control logic used in this domain. Such a study has not been presented previously and is the subject of this paper. We surveyed sonographers to determine the most frequently used controls in US machines. We standardized the representation of the US machine HCI control logic by using the unified modelling language (UML). We used UML diagrams to analyze the HCI control logic of 10 different cart-based US machines from several major manufacturers, and we discovered that the control logic for the most frequently used functions are identical. While this control logic does not follow an established standard, it has been commonly adopted. Using the UML for the visualization and formulation of control logic, we can target logically optimal interactions (whose operation steps cannot be further reduced), e.g., adjustment of B-mode gain, frequency and depth, and can derive methods to simplify logically sub-optimal interactions, e.g., the pointing and selecting operation, as well as image measurements.Our study provides insights into existing HCI approaches used in US machines and establishes a rigorous UML-based framework for future US machine design to improve interoperability, efficiency and ease-of-use. [ABSTRACT FROM AUTHOR]

In recent years, radar-based hand gesture recognition (HGR) has attracted much attention in the field of human-computer interaction (HCI) due to its benefits of high recognition accuracy and independence from lighting conditions. Conventional deep learning (DL) based models for HGR rely on a large amount of labelled data for training to achieve a satisfactory recognition accuracy. However, it is usually time-consuming and labor-intensive for collecting large amounts of radar data and also difficult to cover all possible hand gesture classes. In this paper, we introduce meta-learning to address the few-shot learning problem for frequency modulated continuous wave (FMCW) radar based HGR. We propose a meta-learning network to learn a model that can quickly adapt to unseen hand gesture tasks with few training observations. In particular, we propose a 3D convolutional neural network (CNN) based dual-channel fusion network for feature extraction by exploiting the correlations of multiple features in radar echo signals to improve the recognition accuracy. In addition, we also develop a learnable relation module with neural networks as a non-linear classifier to measure the similarity between the samples of different hand gestures, which can be more applicable for the HGR task. Finally, we evaluate the performance of the proposed model by conducting experiments on an FMCW radar hardware system. Experimental results show that the proposed meta-learning model substantially enhances the recognition accuracy compared with the state-of-the-art methods including DL and meta-learning based models for FMCW radar-based HGR. [ABSTRACT FROM AUTHOR]

Vision-based detection of fingertips is useful for freehand Human-Computer Interaction (HCI)—especially in virtual, augmented, and mixed reality—to have a seamless experience. The estimation of fingertips position in an RGB image involves overcoming various challenges like occlusion, appearance ambiguities, etc. The general approach relies on a two-stage pipeline involving hand location and detection of fingertips for a single hand. This paper presents an effective single-stage Convolutional Neural Network (CNN) for the detection of fingertips of both hands. We use a set of reference points, referred to as pose particles, and train a CNN model end-to-end to find the N-nearest particles in the proximity of each fingertip. Moreover, the same CNN model is used to compute the position vector’s components with reference to these N-nearest neighbors. Finally, a fingertip position is estimated by computing the centroid of all the points given by these position vectors. With the proposed approach, it is possible to estimate the fingertips position for single or double hands. Moreover, there is no requirement for prior hand localization. We demonstrated the feasibility and effectiveness of the proposed methodology by performing experiments on three different datasets. [ABSTRACT FROM AUTHOR]

Visual question answering (VQA) for remote sensing scene has great potential in intelligent human–computer interaction system. Although VQA in computer vision has been widely researched, VQA for remote sensing data (RSVQA) is still in its infancy. There are two characteristics that need to be specially considered for the RSVQA task: 1) no object annotations are available in the RSVQA datasets, which makes it difficult for models to exploit informative region representation and 2) there are questions with clearly different difficulty levels for each image in the RSVQA task. Directly training a model with questions in a random order may confuse the model and limit the performance. To address these two problems, in this article, a multi-level visual feature learning method is proposed to jointly extract language-guided holistic and regional image features. Besides, a self-paced curriculum learning (SPCL)-based VQA model is developed to train networks with samples in an easy-to-hard way. To be more specific, a language-guided SPCL method with a soft weighting strategy is explored in this work. The proposed model is evaluated on three public datasets, and extensive experimental results show that the proposed RSVQA framework can achieve promising performance. Code will be available at https://gitlab.lrz.de/ai4eo/reasoning/VQA-easy2hard. [ABSTRACT FROM AUTHOR]

Conversational systems have recently attracted significant attention. Both the research community and industry believe that it will exert huge impact on human-computer interaction, and specifically, the IR/RecSys community has begun to explore Conversational Recommendation. In real-life scenarios, such systems are often urgently needed in helping users accomplishing different tasks under various situations. However, existing works still face several shortcomings: (1) Most efforts are largely confined in single task setting. They fall short of hands in handling tasks across domains. (2) Aside from soliciting user preference from dialogue history, a conversational recommender naturally has access to the back-end data structure which should be fully leveraged to yield good recommendations. In this paper, we thus present a Topic-guided Conversational Recommender (TCR) which is specifically designed for the multi-domain setting. It augments the sequence-to-sequence (seq2seq) models with a neural latent topic component to better guide the response generation. To better leverage the dialogue history and the back-end data structure, we adopt a graph convolutional network (GCN) to model the relationships between different recommendation candidates while also capture the match between candidates and the dialogue history. We then seamlessly combine these two parts with the idea of pointer networks. We perform extensive evaluation on a large-scale task-oriented multi-domain dialogue dataset and the results show that our method achieves superior performance as compared to a wide range of baselines. [ABSTRACT FROM AUTHOR]

Advancements in driver state detection and artificial intelligence allow for more and more user-centred and individual experiences. Intelligence and adaptivity in the vehicle context address the three main goals: Increasing safety, usability and empathy in vehicle systems. Adaptivity of systems can be evaluated by considering the technical system features, user-interface-related features and the actual user experience of the adaptive system. We provide an overview of classifications for adaptive systems including the Levels of Adaptive Sensitive Responses (LASR). The levels differentiate the input that a system considers in its operations to adapt to user groups or to the individual user. Along with that, we propose User Experience (UX) design guidelines applicable to the different levels. In an online survey, we varied LASR and one of the UX design guidelines, namely transparency. The within-subjects study showed that both, the levels and the variation of transparency, influenced the perception of intelligence, transparency and intuitive design. However, a significant proportion of users did not understand the difference between the two LASR versions, indicating that users build mental models of systems that imply more personal data usage than the system actually employs. The LASR framework allowed this differentiation to be revealed in system performance and user perception. More research is necessary to elaborate the correlation between levels of adaptivity, UX design, specific UX design guidelines and user experience measures. [ABSTRACT FROM AUTHOR]

Facial expression recognition is important for human–computer interaction and other applications. Several facial expression datasets have been published in recent decades and have enabled improvements in algorithms for classifying emotions. However, recognition of realistic expressions in real-world conditions is still challenging because of uncontrolled conditions, such as lighting, brightness, pose, and occlusion. In this paper, we propose a light attention embedding network based on the spatial attention mechanism (LAENet-SA), which can focus on locations in an image that are relevant to emotion. LAENet-SA allows a small number of attention modules to be embedded and can be constructed from typical convolutional neural networks. The performance of LAENet-SA on facial expression recognition has been validated on three facial expression datasets, including a lab-controlled dataset and two in-the-wild datasets. Experimental results show that LAENet-SA improved the performance on each dataset, compared with state-of-the-art methods, and achieved better generalization when tested on facial images with occlusion. [ABSTRACT FROM AUTHOR]

Graph query autocompletion (gQAC) generates a small list of ranked query suggestions during the graph query formulation process in a visual environment. The current state-of-the-art of gQAC provides suggestions that are formed by adding subgraph increments to arbitrary places of an existing (partial) user query. However, according to the research results on human-computer interaction (HCI), humans can only interact with a small number of recent software artifacts in hand. Hence, many of such suggestions could be irrelevant. In this paper, we present the GFocus framework that exploits a novel notion of user focus of graph query formulation (or simply focus). Intuitively, the focus is the subgraph that a user is working on. We formulate locality principles inspired by the HCI research to automatically identify and maintain the focus. We propose novel monotone submodular ranking functions for generating popular and comprehensive query suggestions only at the focus. In particular, the query suggestions of GFocus have high result counts (when they are used as queries) and maximally cover the possible suggestions at the focus. We propose efficient algorithms and an index for ranking the suggestions. Our results show that GFocus saves 12-32 percent more mouse clicks and is 35× more efficient than the state-of-the-art competitor. [ABSTRACT FROM AUTHOR]

Social touch is essential for our social interactions, communication, and well-being. It has been shown to reduce anxiety and loneliness; and is a key channel to transmit emotions for which words are not sufficient, such as love, sympathy, reassurance. However, direct physical contact is not always possible due to being remotely located, interacting in a virtual environment, or as a result of a health issue. Mediated social touch enables physical interactions, despite the distance, by transmitting the haptic cues that constitute social touch through devices. As this technology is fairly new, the users’ needs and their expectations on a device design and its features are unclear, as well as who would use this technology, and in which conditions. To better understand these aspects of mediated interaction, we conducted an online survey on 258 respondents located in the USA. Results give insights on the type of interactions and device features that the US population would like to use. [ABSTRACT FROM AUTHOR]

Development of haptic interfaces to enrich augmented and virtual reality with the sense of touch is the next frontier for technological advancement of these systems. Among available technologies, electrotactile stimulation enables design of high-density interfaces that can provide natural-like sensation of touch in interaction with virtual objects. The present study investigates the human perception of electrotactile sensations on fingertips, focusing on the sensation localization in function of the size and position of reference electrode. Ten healthy subjects participated in the study, with the task to mark the sensations elicited by stimulating the index fingertip using an 8-pad electrode. The test systematically explored several configurations of the active (position) and reference (position and size) electrode pads. The results indicated that there was a spreading of perceived sensations across the fingertip, but that they were mostly localized below the active pad. The position and size of the reference electrode were shown to affect the location of the perceived sensations, which can potentially be exploited as an additional parameter to modulate the feedback. The present study demonstrates that the fingertip is a promising target for the delivery of high-resolution feedback. [ABSTRACT FROM AUTHOR]

Natural language and visualization are being increasingly deployed together for supporting data analysis in different ways, from multimodal interaction to enriched data summaries and insights. Yet, researchers still lack systematic knowledge on how viewers verbalize their interpretations of visualizations, and how they interpret verbalizations of visualizations in such contexts. We describe two studies aimed at identifying characteristics of data and charts that are relevant in such tasks. The first study asks participants to verbalize what they see in scatterplots that depict various levels of correlations. The second study then asks participants to choose visualizations that match a given verbal description of correlation. We extract key concepts from responses, organize them in a taxonomy and analyze the categorized responses. We observe that participants use a wide range of vocabulary across all scatterplots, but particular concepts are preferred for higher levels of correlation. A comparison between the studies reveals the ambiguity of some of the concepts. We discuss how the results could inform the design of multimodal representations aligned with the data and analytical tasks, and present a research roadmap to deepen the understanding about visualizations and natural language. [ABSTRACT FROM AUTHOR]

We compare physical and virtual reality (VR) versions of simple data visualizations and explore how the addition of virtual annotation and filtering tools affects how viewers solve basic data analysis tasks. We report on two studies, inspired by previous examinations of data physicalizations. The first study examines differences in how viewers interact with physical hand-scale, virtual hand-scale, and virtual table-scale visualizations and the impact that the different forms had on viewer’s problem solving behavior. A second study examines how interactive annotation and filtering tools might support new modes of use that transcend the limitations of physical representations. Our results highlight challenges associated with virtual reality representations and hint at the potential of interactive annotation and filtering tools in VR visualizations. [ABSTRACT FROM AUTHOR]

Recently many researches have explored the potential of visual programming in robotics, the Internet of Things (IoT), and education. However, there is a lack of studies that analyze the recent evidence-based visual programming approaches that are applied in several domains. This study presents a systematic review to understand, compare, and reflect on recent visual programming approaches using twelve dimensions: visual programming classification, interaction style, target users, domain, platform, empirical evaluation type, test participants' type, number of test participants, test participants' programming skills, evaluation methods, evaluation measures, and accessibility of visual programming tools. The results show that most of the selected articles discussed tools that target IoT and education, while other fields such as data science, robotics are emerging. Further, most tools use abstractions to hide implementation details and use similar interaction styles. The predominant platforms for the tools are web and mobile, while desktop-based tools are on the decline. Only a few tools were evaluated with a formal experiment, whilst the remaining ones were evaluated with evaluation studies or informal feedback. Most tools were evaluated with students with little to no programming skills. There is a lack of emphasis on usability principles in the design stage of the tools. Additionally, only one of the tools was evaluated for expressiveness. Other areas for exploration include supporting end users throughout the life cycle of applications created with the tools, studying the impact of tutorials on improving learnability, and exploring the potential of machine learning to improve debugging solutions developed with visual programming.

With the increasing size and complexity of next-generation communication networks, it is critical to utilize interactive visualizations to support the monitoring, planning, and management of networks. Effectively visualizing large-scale networks is difficult with traditional methods because of the high link density and complex node relationships. Given the limited screen space, to assist Internet Service Provider's (ISP) network planning and management activities, investigating how to present ultra-large-scale network data efficiently is crucial. This paper presents a real-time interactive visualization system that combines the design strategies of progressive disclosure and multiple panels to elegantly visualize the large-scale networks and avoid the information-overload problem. The system also visualizes the configuration of the network elements and provides the network performance information, including the port-level Quality of Service (QoS) metrics. Furthermore, the system enables navigation through the port-level connection and provides different modes for multiple purposes.

Manipulating objects with the hand is a common task in Virtual Reality (VR). However, some issues can occur during these manipulations. Occlusion is one issue that happens when the virtual hand covers an object, and therefore the perception of that object is hindered. Semi-transparent hands could solve this problem. Another issue is the difficulty of delicate hand object manipulation. By using interpenetrable hands, turning off the physics of the hands, this difficulty might decrease. Still, there is a lack of research into how significant the impact is while using these methods. In this paper, with the designs of semi-transparent hands and interpenetrable hands, we present the results of our conducted user study focusing on the effect of semi-transparent and interpenetrable hands on hand object manipulation tasks in virtual reality. The user study includes a VR environment where participants are asked to perform tasks. These tasks are recorded in objective results of accuracy and speed. Afterwards, they fill out a questionnaire about their opinion on the difficulty while using the different methods in which the subjective findings are recorded. Additionally, we improved the semi-transparent hands by adding a feature that smoothly transitions the hand from opaque to semi-transparent. As an input device, we used the Leap Motion Controller (LMC) for this user study. However, any hand tracking sensor that tracks hands from the VR headset's point of view could be used. Semi-transparent and interpenetrable hands have shown significant improvement for precise manipulation, which was verified by user feedback from the questionnaire and the data from the tasks.

Inattentive driving is a key reason of road mishaps causing more deaths than speeding or drunk driving. Research efforts have been made to monitor drivers' attentional states and provide support to drivers. Both invasive and non-invasive methods have been applied to track driver's attentional states, but most of these methods either use exclusive equipment which are costly or use sensors that cause discomfort. In this paper, a vision-based scheme is proposed for monitoring the attentional states of the drivers. The system comprises four major modules-cue extraction and parameter estimation, state of attention estimation, monitoring and decision making, and level of attention estimation. The system estimates the attentional level and classifies the attentional states based on the percentage of eyelid closure over time (PERCLOS), the frequency of yawning and gaze direction. Various experiments were conducted with human participants to assess the performance of the suggested scheme, which demonstrates the system's effectiveness with 92% accuracy.

This paper introduces a new methodology aimed at comfort for the driver in-the-wild multimodal corpus creation for audio-visual speech recognition in driver monitoring systems. The presented methodology is universal and can be used for corpus recording for different languages. We present an analysis of speech recognition systems and voice interfaces for driver monitoring systems based on the analysis of both audio and video data. Multimodal speech recognition allows using audio data when video data are useless (e.g. at nighttime), as well as applying video data in acoustically noisy conditions (e.g., at highways). Our methodology identifies the main steps and requirements for multimodal corpus designing, including the development of a new framework for audio-visual corpus creation. We identify the main research questions related to the speech corpus creation task and discuss them in detail in this paper. We also consider some main cases of usage that require speech recognition in a vehicle cabin for interaction with a driver monitoring system. We also consider other important use cases when the system detects dangerous states of driver’s drowsiness and starts a question-answer game to prevent dangerous situations. At the end based on the proposed methodology, we developed a mobile application that allows us to record a corpus for the Russian language. We created RUSAVIC corpus using the developed mobile application that at the moment a unique audiovisual corpus for the Russian language that is recorded in-the-wild condition.

Galvanic taste stimulation (GTS) is a technique of electrical stimulation that induces, inhibits, and enhances human taste sensations. This work focuses on a GTS method that induces taste sensation using electrodes attached to the chin and to the back of the neck. The authors aim to establish an advanced GTS method that induces taste sensations at specific locations inside the buccal cavity, without attaching electrodes inside the mouth. Conventionally, the location in the buccal cavity where the taste sensation has been perceived was uncontrollable, and it has been used to induce taste sensations for the whole buccal cavity or the throat. We hypothesized that the position of the taste induction can be affected by the electrical potential distribution inside the mouth. Then, we conducted simulations and experiments to verify the hypothesis with regard to control over spatial selectivity. Specifically, we developed a novel GTS configuration using electrodes attached around the head. We verified that the developed multielectrode GTS configuration can manipulate the electrical potential distribution in the tongue, and that it can manipulate the position where the sensation of taste is induced. To the best of our understanding, this is the first report that realized induction of spatially selective taste sensations at a specific position in the buccal cavity.

Current language processing technologies allow the creation of conversational chatbot platforms. Even though artificial intelligence is still too immature to support satisfactory user experience in many mass market domains, conversational interfaces have found their way into ad hoc applications such as call centres and online shopping assistants. However, they have not been applied so far to social inclusion of elderly people, who are particularly vulnerable to the digital divide. Many of them relieve their loneliness with traditional media such as TV and radio, which are known to create a feeling of companionship. In this paper we present the EBER chatbot, designed to reduce the digital gap for the elderly. EBER reads news in the background and adapts its responses to the user’s mood. Its novelty lies in the concept of “intelligent radio”, according to which, instead of simplifying a digital information system to make it accessible to the elderly, a traditional channel they find familiar -background news- is augmented with interactions via voice dialogues. We make it possible by combining Artificial Intelligence Modelling Language, automatic Natural Language Generation and Sentiment Analysis. The system allows accessing digital content of interest by combining words extracted from user answers to chatbot questions with keywords extracted from the news items. This approach permits defining metrics of the abstraction capabilities of the users depending on a spatial representation of the word space. To prove the suitability of the proposed solution we present results of real experiments conducted with elderly people that provided valuable insights. Our approach was considered satisfactory during the tests and improved the information search capabilities of the participants.

Human–Computer natural language interaction is helpful to reduce the operation and maintenance cost of the SCADA system, and it is necessary to solve the complex natural language interface problem that supports data query and real-time control. According to the complexity of natural language instructions, a hierarchical classification of semantic parsing algorithm is adopted. Firstly, the KWECS method is used to classify the intent of natural language instruction, then the TF-IDF keyword extraction algorithm combined with the cosine similarity is used to structure the key-value of the classified natural language instructions which was used into SCADA control intermediate language and then formally converted into actual control or query instruction. If the analysis fails, the complex control and query instruction analysis are carried out according to the classification results, structuring instruction parsing based on dependency parsing and SQL natural language parsing based on deep learning are adopted respectively to implement real-time control interface and database query interface. Our experimental results show that the proposed hierarchical classification of natural language comprehensive query and control interface can better solve the problem of human-computer natural language interaction in the SCADA system, and the accuracy of intent recognition reaches 96.5%. In more detail, the accuracy, precision, recall, and F-score of instruction parsing reach 88.47%, 90.21%, 89.48%, and 89.72% respectively. Especially, it provides more convenient interactive means for industrial and agricultural information management and control.

Recently, electroencephalogram-based emotion recognition has become crucial in enabling the Human-Computer Interaction (HCI) system to become more intelligent. Due to the outstanding applications of emotion recognition, e.g., person-based decision making, mind-machine interfacing, cognitive interaction, affect detection, feeling detection, etc., emotion recognition has become successful in attracting the recent hype of AI-empowered research. Therefore, numerous studies have been conducted driven by a range of approaches, which demand a systematic review of methodologies used for this task with their feature sets and techniques. It will facilitate the beginners as guidance towards composing an effective emotion recognition system. In this article, we have conducted a rigorous review on the state-of-the-art emotion recognition systems, published in recent literature, and summarized some of the common emotion recognition steps with relevant definitions, theories, and analyses to provide key knowledge to develop a proper framework. Moreover, studies included here were dichotomized based on two categories: i) deep learning-based, and ii) shallow machine learning-based emotion recognition systems. The reviewed systems were compared based on methods, classifier, the number of classified emotions, accuracy, and dataset used. An informative comparison, recent research trends, and some recommendations are also provided for future research directions.