Search

This article discusses the need for artificial intelligence (AI) practitioners to shift their focus from a purely technical mindset toward a more human-centered approach. Technical experts are trained to consider the technical aspects of their work, which can cause them to overlook important socio–technical considerations and implications, resulting in a socio–technical gap in AI-based systems. Unhelpful practitioner cultures can lead to them “rejecting practices or downplaying the importance of values or the possible threats of ignoring them”. While efforts are being made to create ethical and more human-centered AI systems, there is a need for corresponding changes in the attitudes and perceptions of AI practitioners. Practitioners need to move away from a sole focus on compliance with responsible AI guidelines and regulations toward active reflection and empathy based on a true understanding of the profound effects their decisions can have on different stakeholders. However, one problematic barrier to beginning work on interventions that target practitioners’ mindsets and attitudes is the lack of a standardized method for evaluating or measuring the effectiveness of design interventions on their attitudes and perceptions. This article suggests the need for clearer metrics within the human–computer interaction (HCI) community for looking at practitioners’ attitudes toward socio–technical factors in AI design. [ABSTRACT FROM AUTHOR]

Background: Building a reputable network on Twitter is viewed as impactful in several scholarly disciplines, but little is known about the professional and interdisciplinary human-computer interaction (HCI) community. This study combined two approaches from scholarly communication and technical communication to capture the static and dynamic features of the HCI scholar Twitter network. Literature review: Related studies that described the scholarly reputation built through Twitter and social networking in the field of HCI were reviewed and discussed. Research questions: 1. In which countries are HCI scholars more likely to follow their peers in the same country? 2. What are the characteristics (country, reputation) and actions (reciprocity) of HCI scholars who are more likely to build HCI scholarly network profiles on Twitter? Research methodology: The network analysis method of the exponential random graph model (ERGM) was adopted to trace and visualize current follower networks on Twitter. Results and discussion: We found that 22.9% of HCI scholars use Twitter and that reciprocity and country of current employment best drive the Twitter connections of scholars. Characteristics of HCI scholars’ tie formation online are also illustrated and discussed. Implications for practice: This study contributes to field studies of professional networks by identifying the structural properties and factors that influence scholars’ search for professional development on Twitter. The empirical findings should be a helpful reference for HCI professional societies and individual scholars in operating online professional networks. [ABSTRACT FROM AUTHOR]

Contribution: This document presents a systematic bibliographic review that demonstrates the need to conduct research on how the user experience impacts the development of computational thinking. Background: In the field of computer science, computational thinking is defined as a method that enhances problem-solving skills, system design, and human behavior understanding. Over the last few decades, several tools have been proposed for the development of computational thinking skills; however, there is no area of study that evaluates the implications or the impact that these types of platforms have on users belonging to any knowledge area. Research Question: Do user interfaces influence the development of computational thinking skills? Methodology: To address this issue, a systematic review of the literature was conducted using the preferred reporting items for systematic reviews and meta-analyses (PRISMA) methodology for analyzing and evaluating scientific publications. Findings: The results show that despite the dearth of literature on the subject, the specific design of a user interface has a significant impact on the development of computational thinking. Bearing the above in mind, it is necessary to conduct research that delves more deeply into the effects caused by the technologies that are used to develop computational thinking, this being a line of research that is worthy of consideration. [ABSTRACT FROM AUTHOR]

In this article, we study the problem of wireless human sensing, which refers to human activity recognition (HAR). HAR based on wireless signals plays an important role in security, human-computer interaction, and healthcare in the 5G era. Most state-of-the-art human activity recognition applications rely on deep learning approaches, which require a large amount of training data to achieve good performance. However, wireless signal data is difficult to collect and label, and it also carries private information, making it challenging to construct large-scale datasets.The recent advances in federated learning provide a chance to aggregate a wide range of users to collaboratively train a HAR model using decentralized datasets under data-preserving constraints. However, since a wireless signal is easily interrupted by the environment, the data across all participants is non-IID, thus decreasing the performance of an aggregated model. Additionally, due to the resource-constrained nature of edge devices, training the HAR model on an end user usually takes too long, resulting in straggler problems in federated learning training. In this article, we proposed a cross-domain federated learning framework (CDFL) to address the lack of labeled wireless data. A transfer learning approach was proposed to simulate wireless data by converting from widely available image datasets, and solving the distribution mismatch problem by domain adaption. Additionally, a customized federated learning approach was proposed to reduce the computational overhead of local model training. Using a case study of ultrasonic signal-based gesture recognition, we demonstrate the effectiveness of the proposed framework. Our method achieves over 90 percent accuracy on a 5-category task without real data, and 88 percent accuracy on a 10-category task when the user collects only one piece of data. [ABSTRACT FROM AUTHOR]

Hand gesture recognition is essential to human-computer interaction as the most natural way of communicating. Furthermore, with the development of 3D hand pose estimation technology and the performance improvement of low-cost depth cameras, skeleton-based dynamic hand gesture recognition has received much attention. This paper proposes a novel multi-stream improved spatio-temporal graph convolutional network (MS-ISTGCN) for skeleton-based dynamic hand gesture recognition. We adopt an adaptive spatial graph convolution that can learn the relationship between distant hand joints and propose an extended temporal graph convolution with multiple dilation rates that can extract informative temporal features from short to long periods. Furthermore, we add a new attention layer consisting of effective spatio-temporal attention and channel attention between the spatial and temporal graph convolution layers to find and focus on key features. Finally, we propose a multi-stream structure that feeds multiple data modalities (i.e., joints, bones, and motions) as inputs to improve performance using the ensemble technique. Each of the three-stream networks is independently trained and fused to predict the final hand gesture. The performance of the proposed method is verified through extensive experiments with two widely used public dynamic hand gesture datasets: SHREC’17 Track and DHG-14/28. Our proposed method achieves the highest recognition accuracy in various gesture categories for both datasets compared with state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Novel view synthesis of remote sensing (RS) scenes is of great significance for scene visualization, human–computer interaction, and various downstream applications. Despite the recent advances in computer graphics and photogrammetry technology, generating novel views is still challenging particularly for RS images due to its high complexity, view sparsity, and limited view-perspective variations. In this article, we propose a novel RS view synthesis method by leveraging the recent advances in implicit neural representations. Considering the overhead and far depth imaging of RS images, we represent the 3-D space by combining implicit multiplane images (ImMPI) representation and deep neural networks. The 3-D scene is reconstructed under a self-supervised optimization paradigm through a differentiable multiplane renderer with multiview input constraints. Images from any novel views thus can be freely rendered on the basis of the reconstructed model. As a by-product, the depth maps corresponding to the given viewpoint can be generated along with the rendering output. We refer to our method as ImMPIs. To further improve the view synthesis under sparse-view inputs, we explore the learning-based initialization of RS 3-D scenes and proposed a neural-network-based prior extractor to accelerate the optimization process. In addition, we propose a new dataset for RS novel view synthesis with multiview real-world Google Earth images. Extensive experiments demonstrate the superiority of the ImMPI over previous state-of-the-art methods in terms of reconstruction accuracy, visual fidelity, and time efficiency. Ablation experiments also suggest the effectiveness of our methodology design. [ABSTRACT FROM AUTHOR]

The thrill of scientific discovery, the excitement of engineering development, and the fresh thinking of design explorations were invigorating as we participated in the birth of a new discipline: Information Visualization. This discipline, based on graphical user interfaces with pointing devices, became possible as software matured, hardware sped up, and screen resolution improved. Driven by the concepts of direct manipulation and dynamic queries, we made interactive interfaces that empowered users and opened up new possibilities for the next generation of designers. We worked with professionals who had real problems and tested real users to get their feedback. Some projects failed and some papers never got published, but many of the new ideas found their way into widely used commercial products. Our great satisfaction is that our students have spread the community spirit of the Human-Computer Interaction Laboratory as they continue to make further contributions. [ABSTRACT FROM AUTHOR]

In this paper, we aim to explore the fine-grained perception ability of deep models for the newly proposed scene sketch semantic segmentation task. Scene sketches are abstract drawings containing multiple related objects. It plays a vital role in daily communication and human-computer interaction. The study has only recently started due to a main obstacle of the absence of large-scale datasets. The currently available dataset SketchyScene is composed of clip art-style edge maps, which lacks abstractness and diversity. To drive further research, we contribute two new large-scale datasets based on real hand-drawn object sketches. A general automatic scene sketch synthesis process is developed to assist with new dataset composition. Furthermore, we propose to enhancing local detail perception in deep models to realize accurate stroke-oriented scene sketch segmentation. Due to the inherent differences between hand-drawn sketches and natural images, extreme low-level local features of strokes are incorporated to improve detail discrimination. Stroke masks are also integrated into model training to guide the learning attention. Extensive experiments are conducted on three large-scale scene sketch datasets. Our method achieves state-of-the-art performance under four evaluation metrics and yields meaningful interpretability via visual analytics. [ABSTRACT FROM AUTHOR]

Vision-based detection of fingertips is useful for freehand Human-Computer Interaction (HCI)—especially in virtual, augmented, and mixed reality—to have a seamless experience. The estimation of fingertips position in an RGB image involves overcoming various challenges like occlusion, appearance ambiguities, etc. The general approach relies on a two-stage pipeline involving hand location and detection of fingertips for a single hand. This paper presents an effective single-stage Convolutional Neural Network (CNN) for the detection of fingertips of both hands. We use a set of reference points, referred to as pose particles, and train a CNN model end-to-end to find the N-nearest particles in the proximity of each fingertip. Moreover, the same CNN model is used to compute the position vector’s components with reference to these N-nearest neighbors. Finally, a fingertip position is estimated by computing the centroid of all the points given by these position vectors. With the proposed approach, it is possible to estimate the fingertips position for single or double hands. Moreover, there is no requirement for prior hand localization. We demonstrated the feasibility and effectiveness of the proposed methodology by performing experiments on three different datasets. [ABSTRACT FROM AUTHOR]

Conversational systems have recently attracted significant attention. Both the research community and industry believe that it will exert huge impact on human-computer interaction, and specifically, the IR/RecSys community has begun to explore Conversational Recommendation. In real-life scenarios, such systems are often urgently needed in helping users accomplishing different tasks under various situations. However, existing works still face several shortcomings: (1) Most efforts are largely confined in single task setting. They fall short of hands in handling tasks across domains. (2) Aside from soliciting user preference from dialogue history, a conversational recommender naturally has access to the back-end data structure which should be fully leveraged to yield good recommendations. In this paper, we thus present a Topic-guided Conversational Recommender (TCR) which is specifically designed for the multi-domain setting. It augments the sequence-to-sequence (seq2seq) models with a neural latent topic component to better guide the response generation. To better leverage the dialogue history and the back-end data structure, we adopt a graph convolutional network (GCN) to model the relationships between different recommendation candidates while also capture the match between candidates and the dialogue history. We then seamlessly combine these two parts with the idea of pointer networks. We perform extensive evaluation on a large-scale task-oriented multi-domain dialogue dataset and the results show that our method achieves superior performance as compared to a wide range of baselines. [ABSTRACT FROM AUTHOR]

Visual question answering (VQA) for remote sensing scene has great potential in intelligent human–computer interaction system. Although VQA in computer vision has been widely researched, VQA for remote sensing data (RSVQA) is still in its infancy. There are two characteristics that need to be specially considered for the RSVQA task: 1) no object annotations are available in the RSVQA datasets, which makes it difficult for models to exploit informative region representation and 2) there are questions with clearly different difficulty levels for each image in the RSVQA task. Directly training a model with questions in a random order may confuse the model and limit the performance. To address these two problems, in this article, a multi-level visual feature learning method is proposed to jointly extract language-guided holistic and regional image features. Besides, a self-paced curriculum learning (SPCL)-based VQA model is developed to train networks with samples in an easy-to-hard way. To be more specific, a language-guided SPCL method with a soft weighting strategy is explored in this work. The proposed model is evaluated on three public datasets, and extensive experimental results show that the proposed RSVQA framework can achieve promising performance. Code will be available at https://gitlab.lrz.de/ai4eo/reasoning/VQA-easy2hard. [ABSTRACT FROM AUTHOR]

Facial expression recognition is important for human–computer interaction and other applications. Several facial expression datasets have been published in recent decades and have enabled improvements in algorithms for classifying emotions. However, recognition of realistic expressions in real-world conditions is still challenging because of uncontrolled conditions, such as lighting, brightness, pose, and occlusion. In this paper, we propose a light attention embedding network based on the spatial attention mechanism (LAENet-SA), which can focus on locations in an image that are relevant to emotion. LAENet-SA allows a small number of attention modules to be embedded and can be constructed from typical convolutional neural networks. The performance of LAENet-SA on facial expression recognition has been validated on three facial expression datasets, including a lab-controlled dataset and two in-the-wild datasets. Experimental results show that LAENet-SA improved the performance on each dataset, compared with state-of-the-art methods, and achieved better generalization when tested on facial images with occlusion. [ABSTRACT FROM AUTHOR]

Graph query autocompletion (gQAC) generates a small list of ranked query suggestions during the graph query formulation process in a visual environment. The current state-of-the-art of gQAC provides suggestions that are formed by adding subgraph increments to arbitrary places of an existing (partial) user query. However, according to the research results on human-computer interaction (HCI), humans can only interact with a small number of recent software artifacts in hand. Hence, many of such suggestions could be irrelevant. In this paper, we present the GFocus framework that exploits a novel notion of user focus of graph query formulation (or simply focus). Intuitively, the focus is the subgraph that a user is working on. We formulate locality principles inspired by the HCI research to automatically identify and maintain the focus. We propose novel monotone submodular ranking functions for generating popular and comprehensive query suggestions only at the focus. In particular, the query suggestions of GFocus have high result counts (when they are used as queries) and maximally cover the possible suggestions at the focus. We propose efficient algorithms and an index for ranking the suggestions. Our results show that GFocus saves 12-32 percent more mouse clicks and is 35× more efficient than the state-of-the-art competitor. [ABSTRACT FROM AUTHOR]

Facial expression plays an important role in human communication as a type of nonverbal language and has been widely used in various areas such as psychology, human-computer interaction and robotics. Nowadays, convolutional neural network is a promising approach for facial expression recognition. However, convolutional layers can be time-consuming and computationally expensive because a large number of parameters participate in the calculations and need to be updated during training. To improve the performance of deep neural network in facial expression recognition and accelerate training and calculation, we propose a novel framework which adopts efficient element-wise multiplication to replace traditional convolution. To disentangle reliable feature representation for more effective recognition and further enhance the recognition performance while maintaining the efficiency, we propose a representation scheme which can retain informative feature components while removing unreliable ones in Fourier domain based on the proposed multiplication framework. Extensive comparison and ablation studies are conducted on several benchmark datasets, which shows the efficiency and effectiveness of the proposed model. [ABSTRACT FROM AUTHOR]

We are witnessing a fast development of Artificial Intelligence (AI), but it becomes dramatically challenging to explain AI models in the past decade. “Explanation” has a flexible philosophical concept of “satisfying the subjective curiosity for causal information”, driving a wide spectrum of methods being invented and/or adapted from many aspects and communities, including machine learning, visual analytics, human-computer interaction and so on. Nevertheless, from the view-point of data and knowledge engineering (DKE), a best explaining practice that is cost-effective in terms of extra intelligence acquisition should exploit the causal information and explaining scenarios which is hidden richly in the data itself. In the past several years, there are plenty of works contributing in this line but there is a lack of a clear taxonomy and systematic review of the current effort. To this end, we propose this survey, reviewing and taxonomizing existing efforts from the view-point of DKE, summarizing their contribution, technical essence and comparative characteristics. Specifically, we categorize methods into data-driven methods where explanation comes from the task-related data, and knowledge-aware methods where extraneous knowledge is incorporated. Furthermore, in the light of practice, we provide survey of state-of-art evaluation metrics and deployed explanation applications in industrial practice. [ABSTRACT FROM AUTHOR]

Internet of Everything (IoE) applications such as haptics, human-computer interaction, and extended reality, using the sixth-generation (6G) of wireless systems have diverse requirements in terms of latency, reliability, data rate, and user-defined performance metrics. Therefore, enabling IoE applications over 6G requires a new framework that can be used to manage, operate, and optimize the 6G wireless system and its underlying IoE services. Such a new framework for 6G can be based on digital twins. Digital twins use a virtual representation of the 6G physical system along with the associated algorithms (e.g., machine learning, optimization), communication technologies (e.g., millimeter-wave and terahertz communication), computing systems (e.g., edge computing and cloud computing), as well as privacy and security-related technologists (e.g., blockchain). First, we present the key design requirements for enabling 6G through the use of a digital twin. Next, the architectural components and trends such as edge-based twins, cloud-based-twins, and edge-cloud-based twins are presented. Furthermore, we provide a comparative description of various twins. Finally, we outline and recommend guidelines for several future research directions. [ABSTRACT FROM AUTHOR]

Dynamic emotion is typically used to facilitate human–machine interactions. Conversational data from social media contain a considerable amount of useful information, and such data are the foundation for researching dynamic and artificial emotion. At present, most human–machine interaction systems focus on the complexity and accuracy of the dialog but neglect the emotional characteristics of the speaker. When generating a dialog considering the emotional personality of the interlocutor, controlling, and guiding the dialog to a specified direction are essential. This article presents a system for studying dynamic emotions in human-computer interaction from the perspective of emotional transfer and guidance. Based on the emotional state of the interlocutor and the distribution of emotional transfer, the process of emotional transfer is simulated and sampled, and the sequence of emotional guidance is generated. In this system, two algorithms are proposed. A generative Markov chain Monte Carlo (GEN-MCMC) algorithm is proposed to generate a variety of emotional transfer sequences that fit the talke’s personality dynamically based on the real-world dialog. Further, a guiding MCMC (GUI-MCMC) algorithm-based GEN-MCMC is proposed to generate the emotional guiding sequences. The generated emotional sequences by GEN-MCMC were evaluated in two aspects: 1) consistency and 2) diversity. The experimental results show that the GEN-MCMC algorithm performs better than the general sequence generation algorithm in terms of consistency and diversity in generating emotional states. The GUI-MCMC was able to generate a proper stimulus sequence when given the first and target emotions. An emotional stimulus sequence can simulate the emotional transfer of the interlocutor in the process of dialogue, and give the observer appropriate reference to guide and control the emotions of dialogue. The experimental results show that the proposed system can effectively model the dynamic emotion in emotional transfer and guidance, which can be further used to build chat robots, intelligent assistants, and human–machine interaction systems. The models can also be used for emotional induction and enhance the feel-good or feel-terrible factor in human–machine communication applications, such as medical treatment of mental diseases, interrogation, and psychological attack and defense. [ABSTRACT FROM AUTHOR]

As a key technology to help bridging the gap between deaf and hearing people, sign language recognition (SLR) has become one of the most active research topics in the human–computer interaction field. Although several SLR methodologies have been proposed, the development of a real-world SLR system is still a very challenging task. One of the main challenges is related to the large intersigner variability that exists in the manual signing process of sign languages. To address this problem, we propose a novel end-to-end deep neural network that explicitly models highly discriminative signer-independent latent representations from the input data. The key idea of our model is to learn a distribution over latent representations, conditionally independent of signer identity. Accordingly, the learned latent representations will preserve as much information as possible about the signs, and discard signer-specific traits that are irrelevant for recognition. By imposing such regularization in the representation space, the result is a truly signer-independent model which is robust to different and new test signers. The experimental results demonstrate the effectiveness of the proposed model in several SLR databases. [ABSTRACT FROM AUTHOR]

This article studies the problem of RFID-based gesture recognition, which is practically important in various human-computer interaction scenarios, for example, smart homes, intelligent logistics, and smart cities. However, the existing solutions normally suffer from two major limitations: the model-driven methods are sensitive to specific environmental factors, and usually do not adapt well to a complex scenario that is full of multipath; the data-driven methods normally need the collection of massive RFID training data, and deploying the model in the remote cloud leads to long response delay. To overcome the above limitations, this article proposes a cross-domain augmentation-based AI learning (CAL) framework in the context of cloud-edge computing. In the CAL framework, we can simulate massive RFID phase profiles by converting the computer vision data that contains the gesture movement information, instead of costing lots of manpower to actually collect RFID training data. The simulated RFID phase profiles are used to train an AI model in the high-performance cloud. Note that since many sources of this kind of computer vision data are available online, we actually do not even need any manpower to collect training data. To achieve time-efficient recognition, knowledge distillation is applied to get a light and accurate model, which is deployed at the edge side. Thus, recognition response delay can be significantly reduced because the edge server where the AI model is actually deployed is much closer to users than the cloud server. We use commercial off-the-shelf RFID, Kinect, a high-performance server, and a laptop to implement the CAL framework. Extensive experiments are conducted to evaluate the performance of CAL. The results reveal that gesture recognition accuracy of CAL can reach nearly 90 percent without collection of any RFID training data. [ABSTRACT FROM AUTHOR]

Saccadic eye movement as a voluntary activity has become the basis of a promising approach for the implementation of human–computer interaction. Seeking blind source separation (BSS) model that can exactly describe eye movement generation and improve the quality of saccadic electrooculography (EOG) signals, we performed a comparative study between an instantaneous and a convolutional mixing model for multichannel EOG signals. Experiments involving seven subjects were carried out in a laboratory environment. The independent component analysis (ICA) method was adopted for BSS. The statistical indicators of the residual statistical dependence and the total square cross correlation for the convolutional model were 0.04 (second order)/0.08 (fourth order) and 1.25, respectively, which were lower than those of the instantaneous model by 0.3 (second order)/0.17 (fourth order) and 0.32, respectively. In addition, the average classification accuracies of the convolutional model were 95.76% (within subject) and 94.08% (between subject) in the case of upward, downward, left, and right saccade tasks, which were higher than those of the instantaneous model by 1.69% and 5.53%, respectively. Experiments revealed that the quality of the saccadic signals separated using the convolutional model was higher than that achieved using the instantaneous model. The outcomes of this research can serve as a valuable reference for multichannel EOG analysis and application. [ABSTRACT FROM AUTHOR]

With a background in both computer science and art, Tomás Laurenzo's research spans across different practices and interests, including artificial intelligence (machine learning), human–computer interaction, physical computing, and extended reality. In reviewing his work to date, we found many themes that resonated with recent events highlighted in mass media and were curious to find out more about his process and perspective. He had much to say on the matter as can be read in our interview with Tom documented herein. [ABSTRACT FROM AUTHOR]

Facial expression recognition has become a newly-emerging topic in recent decades, which has important value in the field of human-computer interaction. In this paper, we present a deep learning based approach, named frequency neural network (FreNet), for facial expression recognition. Different from convolutional neural network in spatial domain, FreNet inherits the advantages of processing image in frequency domain, such as efficient computation and spatial redundancy elimination. First, we propose the learnable multiplication kernel and construct multiple multiplication layers to learn features in frequency domain. Second, a summarization layer is proposed following multiplication layers to further yield high-level features. Third, based on the property of discrete cosine transform (DCT), we utilize multiplication layers and summarization layer to construct the Basic-FreNet, which can yield high-level features on the widely used DCT feature. Finally, to further achieve better performance on Basic-FreNet, we propose the Block-FreNet in which the weight-shared multiplication kernel is designed for feature learning and the block sub-sampling is designed for dimension reduction. The experimental results show that the Block-FreNet not only achieves superior performance, but also greatly reduces the computational cost. To our best knowledge, the proposed approach is the first attempt to fill in the blank of frequency based deep learning model for facial expression recognition. [ABSTRACT FROM AUTHOR]

Evaluating whole-body motion is challenging because of the articulated nature of the skeleton structure. Each joint moves in an unpredictable way with uncountable possibilities of movements direction under the influence of one or many of its parent joints. This paper presents a method for human motion quantification via three-dimensional (3-D) body joints coordinates. We calculate a set of metrics that influence the joints movement considering the motion of its parent joints without requiring prior knowledge of the motion parameters. Only the raw joints coordinates data of a motion sequence are needed to automatically estimate the transformation matrix of the joints between frames. We also consider the angles between limbs as a fundamental factor to follow the joints directions. We classify the joints motion as global motion and local motion. The global motion represents the joint movement according to a fixed joint, and the local motion represents the joint movement according to its first parent joint. In order to evaluate the performance of the proposed method, we also propose a comparison algorithm between two skeletons motions based on the quantified metrics. We measured the comparative similarity between the 3-D joints coordinates on Microsoft Kinect V2 and UTD-MHAD dataset. User studies were conducted to evaluate the performance under different factors. Various results and comparisons have shown that our method effectively quantifies and evaluates the motion similarity. [ABSTRACT FROM AUTHOR]

Contribution: This work presents a teaching and learning process for human–computer interaction (HCI) courses, combining traditional lecture-based classroom (TLBC), active learning (AL) and project-based learning elements, such that students can increase their understanding over HCI. Background: In undergraduate science computing, the academic curriculum is composed of theoretical and practical courses. The theoretical courses address more abstract content. This process intends to increase the involving of students over the theoretical discipline of HCI. Research Questions: This work shows a different approach to improve student retention for a better understanding of the theoretical aspects over HCI. Methodology: This proposed process has three mainstages and, as newness, uses the $t$ -learning concept to simulate interactive educational videos. In the first stage, the teacher presents concepts of HCI using TLBC. The second stage is AL, where students make oral presentations or make of learning objects based on interactive videos (LOBIVs) to reinforce their self-comprehension over theoretical content. The third stage is the PBL activity of the production of new LOBIVs. Findings: The results show higher statistical scores using the proposed method (PM) in comparison with the TLBC used in HCI course. A total of 131 students concepts were compared in this work, that uses TLBC, with 113 students concepts that use the PM. The results indicate that there are advantages in the PM, because it helps to reduce the number of reproved students and increases the average students grade. [ABSTRACT FROM AUTHOR]

Research on intelligent vehicles has been popular in the past decade. To fill the gap between automatic approaches and man-machine control systems, it is indispensable to integrate visual human–computer interactions (VHCIs) into intelligent vehicles systems. In this article, we review existing studies on VHCI in intelligent vehicles from three aspects: 1) visual intelligence; 2) decision making; and 3) macro deployment. We discuss how VHCI evolves in intelligent vehicles and how it enhances the capability of intelligent vehicles. We present several simulated scenarios and cases for future intelligent transportation system. [ABSTRACT FROM AUTHOR]

Categorizing free-hand human sketches has profound implications in applications such as human computer interaction and image retrieval. The task is non-trivial due to the iconic nature of sketches, signified by large variances in both appearance and structure when compared with photographs. Despite recent advances made by deep learning methods, the requirement of a large training set is commonly imposed making them impractical for real-world applications where training sketches are cumbersome to obtain – sketches have to be hand-drawn one by one other than crawled freely on the Internet. In this work, we aim to delve further into the data scarcity problem of sketch-related research, by proposing a few-shot sketch classification framework. The model is based on a co-regularized embedding algorithm where common/shareable parts of learned human sketches are exploited, thereby can embed query sketch into a co-regularized sparse representation space for few-shot classification. A new dataset of 8,000 part-level annotated sketches of 100 categories is also proposed to facilitate future research. Experiment shows that our approach can achieve an 5-way one-shot classification accuracy of 85%, and 20-way one-shot at 51%. [ABSTRACT FROM AUTHOR]

Background: Visualizations are used to communicate data about important political, social, environmental, and health topics to a wide range of audiences; however, perceptions of graphs as objective conduits of factual data make them an easy means for spreading misinformation. Research questions: 1. Are people deceived by common deceptive tactics or exaggerated titles used in data visualizations about non-controversial topics? 2. Does a person's previous data visualization coursework mitigate the extent to which they are deceived by deceptive tactics used in data visualizations? 3. What parts of data visualizations (title, shape, data labels) do people use to answer questions about the information being presented in data visualizations? Literature review: Although scholarship from psychology, human-computer interaction, and computer science has examined how data visualizations are processed by readers, scholars have not adequately researched how susceptible people are to a range of deceptive tactics used in data visualizations, especially when paired with textual content. Methodology: Participants (n = 329) were randomly assigned to view one of four treatments for four different graph types (bar, line, pie, and bubble) and then asked to answer a question about each graph. Participants were asked to rank the ease with which they read each graph and comment on what they used to respond to the question about each graph. Results/Discussion: Results show that deceptive tactics caused participants to misinterpret information in the deceptive versus control visualizations across all graph types. Neither graph titles nor previous coursework impacted responses for any of the graphs. Qualitative responses illuminate people's perceptions of graph readability and what information they use to read different types of graphs. Conclusions: Recommendations are made to improve data visualization instruction, including critically examining software defaults and the ease with which people give agency over to software when preparing data visualizations. Avenues of future research are discussed. [ABSTRACT FROM AUTHOR]

Noncontact gesture recognition is gradually being applied to emerging applications, such as smart cars and smart phones. Negative latency gesture recognition (recognition before a gesture is finished) is desirable due to the instantaneous feedback. However, it is difficult for existing methods to achieve a high precision and negative latency gesture recognition. A fragment can provide too few features to directly identify all gestures well. By observing a large number of existing gesture sets and people’s daily operating habits, we found that some high frequency used gestures are similar. To the best of our knowledge, it is the first time to redivide the gestures into two subsets according to their movement physical states. We divided the gestures with different shapes or motion states into a parent-class subset, and further divided each pair of parent-class gestures to obtain a child-class subset. In order to achieve a better tradeoff between the high-precision and negative latency, an approach of motion pattern and behavior intention (MPBI) is proposed. Taking full advantage of the characteristics of each subset, MPBI includes two models. First, pattern model coarsely classify the parent-class gestures by a convolutional network, and then intention model further classifies child-class gestures according to their opposite motion direction. MPBI is evaluated on a 340-GHz terahertz radar. With the advantage of its accurate ranging, intention model can recognize child-class gestures directly without training. MPBI is evaluated on 12 gestures and achieves a recognition accuracy of 94.13%, which only needs a 0.033-s gesture fragment as an input sample. [ABSTRACT FROM AUTHOR]

This paper uses the case of ELEA 9000, the first Olivetti computer series, to demonstrate the close relationship between industrial design, semiotics, ergonomics and the history of computing. A focus on the Olivetti ELEA series invites scholars to reconsider the history of computer interface design well before the emergence of HCI as a widely recognized field of research. The console and racks of the mainframe computer were designed in Italy by architect Ettore Sottsass Jr. (1917–2007) along with Dutch designer Andries van Onck (1928–2018) at the end of the 1950s. Aiming to launch the ELEA computer on the international market, Olivetti developed the idea of a visual language for human–computer interaction that could be learned by any operator, regardless of their native language. Sottsass and Olivetti assigned the task of designing this sign system to Tomás Maldonado (1922–2018), director of the Hochschule für Gestaltung design school in Ulm, Germany. Together with Gui Bonsiepe (b. 1934), Maldonado designed a visual language that incorporated grammatical and syntactic reasoning. Later discarded, the sign system for ELEA prefigured the contemporary use of icons in computer interfaces. [ABSTRACT FROM AUTHOR]

This essay examines the state of the history of human–computer interaction (HCI) from multiple fields of study. In many ways, there is a lack of consensus about what we mean by not only HCI but also computer interfaces. I find that the different definitions and approaches found primarily in history and media studies offer many different avenues for scholars to continue exploring how humans and computers relate to each other. A theoretical background on the topic is provided and then historical case studies concerning both computer interfaces and the embodied use of computers are discussed to understand what conversations are held in common and what areas are still left to be explored. [ABSTRACT FROM AUTHOR]

This is the first paper to discuss the Geromat III teaching machine developed at the Berlin Institute for Cybernetics in relationship to the Geromat III prototypes developed at Ulm School of Design in the 1960s. The Geromat III was an electro-mechanical teaching machine that could train up to three students at three separate multimedia consoles. The system required all learners to answer correctly in order to progress through the instructional program; if not, the students were required to speak to each other in order to reach agreement. The paper analyzes archival material and first-hand accounts with the Geromat III, as well as the social context of federal funding for research into computing and education in West Germany of the 1960s, in order to understand how this teaching machine, an early form of computer-aided instruction (CAI), came to be. The Geromat III, though utilized on a small scale, found an audience of enthusiastic supporters on the other side of the country, with artist-designers at the Ulm School of Design who created prototypes for consoles to be used with the system. While hands-on knowledge about the system was unavailable to the console designers, an ideological affinity between the Ulm School of Design and the Berlin Institute for Cybernetics bridged any technical gaps in knowledge; the resulting design prototypes, one for mass-production and one for DIY (do-it-yourself) construction, point to speculative futures for the system. [ABSTRACT FROM AUTHOR]

Device-free wireless sensing (DFWS) is a promising technology which could sense target states without requiring them equipped with any device. In recent years, it has drawn considerable attention due to its potential application in the fields of fatigue driving detection, vital sign monitoring, human-computer interaction, etc. State-of-the-art work has achieved excellent sensing performance when there is one target only. However, when there are multiple targets need to be sensed simultaneously, the influenced signals from different targets will be mixed together, and thus traditional DFWS methods will fail. There still lacks an effective system solution to this problem. Inspired by the multiplexing mechanisms utilized in communication systems, in this paper, we explore and exploit the idea of separating and extracting the influenced signal from each target by leveraging three novel multiplexing mechanisms, i.e., angle division multiplexing sensing, range division multiplexing sensing, and source division multiplexing sensing, and thus realize multi-target DFWS accordingly. Meanwhile, we also give theory analysis on the sensing capability of the proposed multiplexing mechanism based multi-target sensing systems. Furthermore, taking multi-target vital sign monitoring as a case study, we develop a 77 GHz FMCW hardware based prototype system, and evaluate the proposed mechanisms extensively. Experimental results reveal the effectiveness of the proposed mechanisms. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have achieved great success in several face-related tasks, such as face detection, alignment and recognition. As a fundamental problem in computer vision, face tracking plays a crucial role in various applications, such as video surveillance, human emotion detection and human-computer interaction. However, few CNN-based approaches are proposed for face (bounding box) tracking. In this article, we propose a face tracking method based on Siamese CNNs, which takes advantages of powerful representations of hierarchical CNN features learned from massive face images. The proposed method captures discriminative face information at both local and global levels. At the local level, representations for attribute patches (i.e., eyes, nose and mouth) are learned to distinguish a face from another one, which are robust to pose changes and occlusions. At the global level, representations for each whole face are learned, which take into account the spatial relationships among local patches and facial characters, such as skin color and nevus. In addition, we build a new large-scale challenging face tracking dataset to evaluate face tracking methods and to facilitate the research forward in this field. Extensive experiments on the collected dataset demonstrate the effectiveness of our method in comparison to several state-of-the-art visual tracking methods. [ABSTRACT FROM AUTHOR]

Recent years have seen a resurgence of virtual reality (VR), sparked by the repurposing of low-cost COTS components. VR aims to generate stimuli that appear to come from a source other than the interface through which they are delivered. The synthetic stimuli replace real-world stimuli, and transport the user to another, perhaps imaginary, "place." To do this, we must overcome many challenges, often related to matching the synthetic stimuli to the expectations and behavior of the real world. One way in which the stimuli can fail is its latency–– the time between a user's action and the computer's response. We constructed a novel VR renderer, that optimized latency above all else. Our prototype allowed us to explore how latency affects human–computer interaction. We had to completely reconsider the interaction between time, space, and synchronization on displays and in the traditional graphics pipeline. Using a specialized architecture––dataflow computing––we combined consumer, industrial, and prototype components to create an integrated 1:1 room-scale VR system with a latency of under 3 ms. While this was prototype hardware, the considerations in achieving this performance inform the design of future VR pipelines, and our human factors studies have provided new and sometimes surprising contributions to the body of knowledge on latency in HCI. [ABSTRACT FROM AUTHOR]

The newly rediscovered frontier between data visualization and the digital humanities has proven to be an exciting field of experimentation for scholars from both disciplines. This fruitful collaboration is attracting researchers from other areas of science who may be willing to create visual analysis tools that promote humanities research in its many forms. However, as the collaboration grows in complexity, it may become intimidating for these scholars to get engaged in the discipline. To facilitate this task, we have built an introduction to visualization for the digital humanities that sits on a data-driven stance adopted by the authors. In order to construct a dataset representative of the discipline, we analyze citations from a core corpus on 300 publications in visualization for the humanities obtained from recent editions of the InfoVis Vis4DH workshop, the ADHO Digital Humanities Conference, and the specialized digital humanities journal Digital Humanities Quarterly. From here, we extract referenced works and analyze more than 1900 publications in search of citation patterns, prominent authors in the field, and other interesting insights. Finally, following the path set by other researchers in the visualization and Human–Computer Interaction (HCI) communities, we analyze paper keywords to identify significant themes and research opportunities in the field. [ABSTRACT FROM AUTHOR]

The tracking-by-detection tracking framework usually consists of two stages: drawing samples around the target object and classifying each sample as either the target object or background. Current popular trackers under this framework typically draw many samples from the raw image and feed them into the deep neural networks, resulting in high computational burden and low tracking speed. In this article, we propose an adversarial feature sampling learning (AFSL) method to address this problem. A convolutional neural network is designed, which takes only one cropped image around the target object as input, and samples are collected from the feature maps with spatial bilinear resampling. To enrich the appearance variations of positive samples in the feature space, which has limited spatial resolution, we fuse the high-level features and low-level features to better describe the target by using a generative adversarial network. Extensive experiments on benchmark data sets demonstrate that the proposed ASFL achieves leading tracking accuracy while significantly accelerating the speed of tracking-by-detection trackers. Note to Practitioners—Visual tracking can be applied in many intelligent automation systems, such as robotic intelligent navigation system, intelligent human–computer interaction system, and so on. In a robotic intelligent navigation system, visual tracking can generate target’s motion trajectory from image sequences. Visual tracking can also obtain body movement information automatically during the interactive process in the intelligent human–computer interaction system. Accuracy and speed are two key indicators for visual tracking, and intelligent automation systems usually need a tracker with more accuracy and faster speed. This article aims to develop a fast and accurate tracking method by adversarial feature sampling learning (AFSL). In the concrete implementation process, AFSL gets samples by sampling in the feature space rather than on raw images to reduce computation. Then, an adversarial learning mechanism is adopted to boost the sampling features and enrich the target appearances in the feature space to improve the tracking accuracy. The proposed tracker is proven to be effective to keep leading tracking accuracy while significantly accelerating the tracking speed. [ABSTRACT FROM AUTHOR]

In this paper, we classify digits written in mid-air using hand gestures. Impulse radio ultrawideband (IR-UWB) radar sensors are used for data acquisition, with three radar sensors placed in a triangular geometry. Conventional radar-based gesture recognition methods use whole raw data matrices or a group of features for gesture classification using convolutional neural networks (CNNs) or other machine learning algorithms. However, if the training and testing data differ in distance, orientation, hand shape, hand size, or even gesture speed or the radar setup environment, these methods become less accurate. To develop a more robust gesture recognition method, we propose not using raw data for the CNN classifier, but instead employing the hand’s mid-air trajectory for classification. The hand trajectory has a stereotypical shape for a given digit, regardless of the hand’s orientation or speed, making its classification easy and robust. Our proposed method consists of three stages: signal preprocessing, hand motion localization, and tracking and transforming the trajectory data into an image to classify it using a CNN. Our proposed method outperforms conventional approaches because it is robust to changes in orientation, distance, and hand shape and size. Moreover, this method does not require building a huge training database of digits drawn by different users in different orientations; rather, we can use training databases already available in the image processing field. Overall, the proposed mid-air handwritten digit recognition system provides a user-friendly and accurate mid-air handwriting modality that does not place restrictions on users. [ABSTRACT FROM AUTHOR]

With the prevalence of commercial WiFi devices and the development of Internet of Things (IoT), researchers have extended the usage of WiFi from communication to sensing. Recently, device-free human activity recognition has been applied to support WiFi based remote control and human-computer interaction. However, prior works usually recognize each individual activity by extracting the feature of corresponding WiFi signals, and hence distinguishing differences between human activities. Once the activity has multiple variations of different body parts (such as head, arm and leg), distinguishing such sub-activities is extremely difficult. In this paper, we propose a Multi-variations Activity Recognition (MAR) system to identify multiple-variations of body parts. Our work is based on an observation that the Channel State Information (CSI) is sensitive to the activities of body parts. Firstly, CANDECOMP/ PARAFAC (CP) decomposition and Dynamic Time Warping (DTW) are applied to recognize multi-variations activities. Secondly, we theoretically analyse the uniqueness of CP decomposition. Then, we design specific experiment to verify the reliability and stability of uniqueness. Finally, we apply MAR in gaits recognition of multiple volunteers to evaluate the accuracy performance. The experiment results demonstrate that MAR achieves average 95% accuracy in gaits recognition. [ABSTRACT FROM AUTHOR]

Gesture recognition is important for human–computer interaction and a variety of emerging research and commercial areas including virtual and augmented reality. Current approaches typically require sensors to be placed on the forearm, wrist, or directly across finger joints; however, they can be cumbersome or hinder human movement and sensation. In this paper, we introduce a novel approach to recognize hand gestures by estimating skin strain with multiple soft sensors optimally placed across the back of the hand. A pilot study was first conducted by covering the back of the hand with 40 small 2.5 mm reflective markers and using a high-precision camera system to measure skin strain patterns for individual finger movements. Optimal strain locations are then determined and used for sensor placement in a stretchable e-skin patch prototype. Experimental testing is performed to evaluate the stretchable e-skin patch performance in classifying individual finger gestures and American Sign Language 0–9 number gestures. Results showed classification accuracies of 95.3% and 94.4% for finger gestures and American Sign Language 0–9 gestures, respectively. These results demonstrate the feasibility of a stretchable e-skin patch on the back of the hand for hand gesture recognition and their potential to significantly enhance human–computer interaction. [ABSTRACT FROM AUTHOR]

Objective: Spectral power analysis plays a predominant role in electroencephalogram-based emotional recognition. It can reflect activity differences among multiple brain regions. In addition to activation difference, different emotions also involve different large-scale network during related information processing. In this paper, both information propagation patterns and activation difference in the brain were fused to improve the performance of emotional recognition. Methods: We constructed emotion-related brain networks with phase locking value and adopted a multiple feature fusion approach to combine the compensative activation and connection information for emotion recognition. Results: Recognition results on three public emotional databases demonstrated that the combined features are superior to either single feature based on power distribution or network character. Furthermore, the conducted feature fusion analysis revealed the common characters between activation and connection patterns involved in the positive, neutral, and negative emotions for information processing. Significance: The proposed feasible combination of both information propagation patterns and activation difference in the brain is meaningful for developing the effective human–computer interaction systems by adapting to human emotions in the real world applications. [ABSTRACT FROM AUTHOR]

Wireless human sensing plays a crucially important role in the human-computer interaction context, in which human activities and even emotions can be recognized and understood by computers. A batch of academic efforts have been proposed to use wireless techniques such as WiFi, RFID, Bluetooth, Radar, and Zigbee to address the human sensing problem. Each kind of sensing technology has its own characteristics and advantages and thus is suitable for specific application scenarios. For example, WiFi-based solutions can achieve non-intrusive human sensing, and RFID-based solutions can enable individual human sensing for multi-person scenarios. To let users better understand the existing wireless human sensing solutions and choose the most suitable one according to their demands, this article presents a comprehensive survey of the existing wireless human sensing approaches. Specifically, we discuss promising human sensing applications and partition them into three categories: vital sign monitoring, gesture recognition, and activity recognition. For each category of applications, we further conduct a taxonomy of the existing solutions and summarize their ideas, characteristics, pros, and cons from various perspectives, such as design approaches, system configuration, wireless technology, and used information. Finally, we discuss some technical challenges and problems that have not been noticed yet and point out the potential opportunities in the future study of wireless human sensing. [ABSTRACT FROM AUTHOR]

This paper presents a sensory fusion method for estimation of joint angles of serial kinematic chains with rotational degrees of freedom based on magnetoinertial measurements—Magnetoinertial tracking based on JAcobian PseudoInverse (MIJAPI). The concept takes into account the mechanism kinematic model, and the computation relies on the differential kinematics inversion (inverse kinematics solution based on the Jacobian inverse). A Moore–Penrose weighted left pseudoinverse of the mechanism Jacobian matrix is applied to solve a (typically) overdetermined system (redundant measurements resulting from constraints related to attachments of magnetoinertial sensors) in a least-squares approach. Calculation of a gain matrix for correcting the estimated angles is based on Kalman-adaptive algorithm. The quality of the proposed approach was compared to different solutions based on the Unscented Kalman filter. In terms of computational complexity, the MIJAPI concept outperforms the Kalman-based approaches. Better results were also noticed in conditions with significant measurement disturbances and sensor misalignments. The method is applicable in the fields of human motion tracking/analysis as well as robotics. [ABSTRACT FROM AUTHOR]

Magnetoresistive (MR) sensors have been identified as promising candidates for the development of high-performance magnetometers due to their high sensitivity, low cost, low power consumption, and small size. The rapid advance of MR sensor technology has opened up a variety of MR sensor applications. These applications are in different areas that require MR sensors with different properties. Future MR sensor development in each of these areas requires an overview and a strategic guide. An MR sensor roadmap (non-recording applications) was therefore developed and made public by the Technical Committee of the IEEE Magnetics Society with the aim to provide an research and development (R&D) guide for MR sensors intended to be used by industry, government, and academia. The roadmap was developed over a three-year period and coordinated by an international effort of 22 taskforce members from ten countries and 17 organizations, including universities, research institutes, and sensor companies. In this paper, the current status of MR sensors for non-recording applications was identified by analyzing the patent and publication statistics. As a result, timescales for MR sensor development were established and critical milestones for sensor parameters were extracted in order to gain insight into potential MR sensor applications (non-recording). Five application areas were identified, and five MR sensor roadmaps were established. These include biomedical applications, flexible electronics, position sensing and human–computer interactions, non-destructive evaluation and monitoring, and navigation and transportation. Each roadmap was analyzed using a logistic growth model, and new opportunities were predicted based on the extrapolated curve, forecast milestones, and professional judgment of the taskforce members. This paper provides a framework for MR sensor technology (non-recording applications) to be used for public and private R&D planning, in order to provide guidance into likely MR sensor applications, products, and services expected in the next 15 years and beyond. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel event-triggered tracking framework for fast and robust visual tracking in the presence of model drift and occlusion. The resulting tracker not only operates in real time, but also is resilient to tracking failures caused by factors such as fast motion and heavy occlusion. Specifically, the tracker consists of an event-triggered decision model as the core module that coordinates other functional modules, including a short-term tracker, occlusion and drift identification, target redetection, short-term tracker updating, and online discriminative learning for a detector. Each functional module is associated with a defined event that is triggered when the proposed conditions are met. The occlusion and drift identification module is intended to perform online evaluation of the short-term tracking. When a model drift event occurs, the target redetection module is activated by the event-triggered decision model to relocate the target and reinitialize the short-term tracker. The short-term tracker updating is carried out at each frame with a variable learning rate depending on the degree of occlusion. A sampling pool is constructed to store discriminative samples that are used to update the detector model. Extensive experiments on large benchmark datasets demonstrate that the proposed tracking algorithm can effectively detect model drift and restore tracking, and more importantly, it outperforms the state-of-the-art approaches in terms of accuracy, efficiency, and robustness. [ABSTRACT FROM AUTHOR]

The detection of groups of individuals is attracting the attention of many researchers in diverse fields, from automated surveillance to human-computer interaction, with a growing number of approaches published every year. Unexpectedly, the evaluation metrics for this problem are not consolidated, with some measures inherited from the people detection field, other from clustering, other designed specifically for a particular approach, thus lacking in generalization and making the comparisons between different approaches hard to be carried out. Moreover, most of the existent metrics are scarcely expressive, addressing groups as they are atomic entities, ignoring that they may have different cardinalities, and that group detection approaches may fail in capturing the exact number of individuals that compose it. This paper fills this gap presenting the GROup DEtection (GRODE) metrics, which formally define precision and recall on the groups, including the group cardinality as a variable. This gives the possibility to investigate aspects never considered so far, such as the tendency of a method of over- or under-segmenting, or of better dealing with specific group cardinalities. The GRODE metrics have been evaluated first on controlled scenarios, where the differences with alternative metrics are evident. Then, the metrics have been applied to eight approaches of group detection, on eight public datasets, providing a fresh-new panorama of the state-of-the-art, discovering interesting strengths and pitfalls of the recent approaches. [ABSTRACT FROM AUTHOR]

We propose an effective method to boost the accuracy of multi-person pose estimation in images. Initially, the three-layer fractal network was constructed to regress multi-person joints location heatmap that can help to enhance an image region with receptive field and capture more joints local-contextual feature information, thereby producing keypoints heatmap intermediate prediction to optimize human body joints regression results. Subsequently, the hierarchical bi-directional inference algorithm was proposed to calculate the degree of relatedness (call it Kinship) for adjacent joints, and it combines the Kinship between adjacent joints with the spatial constraints, which we refer to as joints kinship pattern matching mechanism, to determine the best matched joints pair. We iterate the above-mentioned joints matching process layer by layer until all joints are assigned to a corresponding individual. Comprehensive experiments demonstrate that the proposed approach outperforms the state-of-the-art schemes and achieves about 1% and 0.6% increase in mAP on MPII multi-person subset and MSCOCO 2016 keypoints challenge. [ABSTRACT FROM AUTHOR]

An action is something which is done by an agent. Most action recognition researchers merely focus on the actions to be recognized, and ignore the differences of agents. Philosophers and behaviorists discover that actions are common among many species, but are performed in different ways and with different levels of sophistication. In this paper, in order to bridge action recognition tasks between different agents, we introduce a new problem, cross-agent action recognition, i.e., recognizing action for one particular agent (target) while training from other agents (source). We model this problem under three different scenarios: single source and single target, multiple sources and single target, and multiple sources and multiple targets. To this end, corresponding methods based on transfer learning are proposed to address these problems. We further design three different strategies to model the situation when a partial labeled data is provided for the target. Experimental results show that the performances of the transfer method are generally better than those of the comparative method without transfer learning, especially when we have multiple sources. Particularly, the transfer method outperforms the others significantly when the source is a human adult. In addition, cross-agent method significantly improves the results when partially labeled data is provided for the target. These demonstrate that for action recognition, knowledge can be transferred across different agents. A straightforward application of this finding is to use human action (training data is abundant) data to enhance animal action recognition. [ABSTRACT FROM AUTHOR]

There has been an increasing interest in cable-driven robotic rehabilitation systems with adjustable stiffness to provide safe and natural interactions for physical therapy. In this paper, we investigate the effectiveness of various stiffness modulation schema and alternate attachment configurations within a scaled planar elastic articulated-cable leg-orthosis emulator for gait training. Elasticity, incorporated by series-elastic springs or adjustable stiffness modules connected to nonflexible cables, can add substantial robustness during forceful interactions with uncertain environments. Other benefits include force-sensor-free tension control, i.e., without using force sensors connected to cables and better overall tension distribution. However, elasticity also degrades the accuracy of positioning and makes the system more disposed to external disturbances. Hence, we examine the performance of actively modulating the effective stiffness to smooth out any perturbations to the scaled normative rehabilitative motion patterns and achieve natural gaits (by simulation and experiment). Stiffness modulation can now be achieved by varying system configuration, antagonistic cable tension, or joint stiffness, and we examine the benefits of each. Finally, we also comparatively evaluate the functional performance of alternate attachment configurations ankle–cable configuration, articulated-cable configuration by way of simulation and experiments. [ABSTRACT FROM AUTHOR]

Humans can skilfully use tools and interact with the environment by adapting their movement trajectory, contact force, and impedance. Motivated by the human versatility, we develop here a robot controller that concurrently adapts feedforward force, impedance, and reference trajectory when interacting with an unknown environment. In particular, the robot's reference trajectory is adapted to limit the interaction force and maintain it at a desired level, while feedforward force and impedance adaptation compensates for the interaction with the environment. An analysis of the interaction dynamics using Lyapunov theory yields the conditions for convergence of the closed-loop interaction mediated by this controller. Simulations exhibit adaptive properties similar to human motor adaptation. The implementation of this controller for typical interaction tasks including drilling, cutting, and haptic exploration shows that this controller can outperform conventional controllers in contact tooling. [ABSTRACT FROM AUTHOR]