Search

This research explores prospective determinants of trust in the recommendations of artificial agents regarding decisions to kill, using a novel visual challenge paradigm simulating threat-identification (enemy combatants vs. civilians) under uncertainty. In Experiment 1, we compared trust in the advice of a physically embodied versus screen-mediated anthropomorphic robot, observing no effects of embodiment; in Experiment 2, we manipulated the relative anthropomorphism of virtual robots, observing modestly greater trust in the most anthropomorphic agent relative to the least. Across studies, when any version of the agent randomly disagreed, participants reversed their threat-identifications and decisions to kill in the majority of cases, substantially degrading their initial performance. Participants' subjective confidence in their decisions tracked whether the agent (dis)agreed, while both decision-reversals and confidence were moderated by appraisals of the agent's intelligence. The overall findings indicate a strong propensity to overtrust unreliable AI in life-or-death decisions made under uncertainty. [ABSTRACT FROM AUTHOR]

Taking into account the profound impact of technology on modern education, especially during the covid19 pandemic, increasing academic interest has focused towards the design and application of such tools on different learning contexts. A specific area of Human–Computer Interaction, called affordance theory, focuses on the perception, design and use of different technologies by educators and learners in learning contexts. This paper explores the impact of affordances in the process of creative problem in the context of playful educational robotics, with an intension of informing the design of future educational experiences around the field. The study capitalizes upon previous affordance propositions and frameworks in order to establish an affordance-based framework in the scope of playful educational robotics contexts, through the adoption of a qualitative research methodology, which was considered more appropriate as an exploratory tool. As part of the qualitative analysis, the study is mapping different types of affordances, related to such technologies, as well as an iterative creative problem-solving process that stems from learners' interactions with robotic artifacts, like the CreaCube playful robotics activity, which is presented in this study. [ABSTRACT FROM AUTHOR]

The array of tactile information processing capabilities is an important index for modern intelligent devices advancing toward a humanoid form, and it greatly improves the recognition of different objects in human‐computer interactions. Herein, a deep‐learning‐assisted intelligent grasping recognition system based on a piezoresistive sensing glove, hardware conditioning, and acquisition circuits, and a multibranch deep‐capsule network is reported. Owing to the multiscale 3D structure of carbon nanotube (CNTs)/carbon fiber (CFs) embedded in polydimethylsiloxane (PDMS), the piezoresistive sensing glove is highly sensitive to the pressure exerted by external objects. The acquired signals are reflected on a hand‐like background map, and a combination of multiple subgraphs is used to build the dataset. A multibranch deep‐capsule network is constructed to encode spatial information while realizing object recognition with an accuracy of 99.4%. Therefore, the proposed intelligent grasping recognition system possesses good human‐robot interaction capabilities, providing a new approach for the development of intelligent robots in the field of perceptual recognition applications. [ABSTRACT FROM AUTHOR]

Abstract This research explores prospective determinants of trust in the recommendations of artificial agents regarding decisions to kill, using a novel visual challenge paradigm simulating threat-identification (enemy combatants vs. civilians) under uncertainty. In Experiment 1, we compared trust in the advice of a physically embodied versus screen-mediated anthropomorphic robot, observing no effects of embodiment; in Experiment 2, we manipulated the relative anthropomorphism of virtual robots, observing modestly greater trust in the most anthropomorphic agent relative to the least. Across studies, when any version of the agent randomly disagreed, participants reversed their threat-identifications and decisions to kill in the majority of cases, substantially degrading their initial performance. Participants’ subjective confidence in their decisions tracked whether the agent (dis)agreed, while both decision-reversals and confidence were moderated by appraisals of the agent’s intelligence. The overall findings indicate a strong propensity to overtrust unreliable AI in life-or-death decisions made under uncertainty.

The paper presents a study of library and information science students' perceptions and experiences related to the use of telepresence robots (TPRs) in libraries, with a specific focus on students assuming the role of library consultants. The study explores the innovative application of TPRs, such as Double, Ohmni and Temi, within library settings. The findings of the research reveal a positive perception of TPRs, emphasising their potential for fostering innovation, increasing engagement, providing flexibility and overcoming geographical constraints. The study also identifies distinct advantages of TPRs, including remote accessibility, enhanced communication, flexibility, a personalised user experience and cost savings. However, concurrent challenges, such as limitations in nonverbal communication, pointing accuracy, connectivity issues, space requirements, costs, safety, security concerns and varying levels of user acceptance emerge as crucial areas for refinement. While the results may not be generalisable due to the limited sample size, the study underscores the transformative potential of TPRs in libraries and the necessity of addressing these challenges to optimise their application. Further research, technological advancements and user feedback can help refine and optimise the use of TPRs in libraries to provide innovative and accessible assistance. [ABSTRACT FROM AUTHOR]

Robots increasingly interact with humans through touch, where people are touching or being touched by robots. Yet, little is known about how such interactions shape a user's experience. To inform future work in this area, we conduct a systematic review of 44 studies on physical human–robot interaction (pHRI). Our review examines the parameters of the touch (e.g., the role of touch, location), the experimental variations used by researchers, and the methods used to assess user experience. We identify five facets of user experience metrics from the questionnaire items and data recordings for pHRI studies. We highlight gaps and methodological issues in studying pHRI and compare user evaluation trends with the Human–Computer Interaction (HCI) literature. Based on the review, we propose a conceptual model of the pHRI experience. The model highlights the components of such touch experiences to guide the design and evaluation of physical interactions with robots and inform future user experience questionnaire development. [ABSTRACT FROM AUTHOR]

Hand tracking is relevant to such a variety of applications including human-robot interaction (HRI), human-computer interaction (HCI), virtual reality (VR), and augmented reality (AR). Accurate and robust hand tracking however is challenging due to the intricacies of dynamic motion within small space and the complex interactions with nearby objects, coupled with the hurdles in real-time hand mesh reconstruction. In this paper, we conduct a comprehensive examination and analysis of existing hand tracking technologies. Through the review of major works in the literature, we have discovered numerous studies employing a diverse array of sensors, leading us to propose their categorization into seven types: vision, soft wearable, encoder, magnetic, inertial measurement unit (IMU), electromyography (EMG), and the fusion of sensor modalities. Our findings indicate that no singular solution surpasses all others, attributing to the inherent limitations of using a single sensor modality. As a result, we assert that integrating multiple sensor modalities presents a viable path toward devising a superior hand tracking solution. Ultimately, this survey paper aims to bolster interdisciplinary research efforts across the spectrum of hand tracking technologies, thereby contributing to the advancement of the field. [ABSTRACT FROM AUTHOR]

Hand gesture recognition has great applications in human-computer interaction (HCI), human-robot interaction (HRI), and supporting the deaf and mute. To build a hand gesture recognition model using deep learning (DL) with high results then needs to be trained on many data and in many different conditions and contexts. In this paper, we publish the TQU-HG dataset of large RGB images with low resolution (640×480) pixels, low light conditions, and fast speed (16 fps). TQU-HG dataset includes 60,000 images collected from 20 people (10 male, 10 female) with 15 gestures of both left and right hands. A comparative study with two branches: i) based on Mediapipe TML and ii) Based on convolutional neural networks (CNNs) (you only look once (YOLO); YOLOv5, YOLOv6, YOLOv7, YOLOv8, YOLO-Nas, single shot multiBox detector (SSD) VGG16, residual network (ResNet)18, ResNext50, ResNet152, ResNext50, MobileNet V3 small, and MobileNet V3 large), the architecture and operation of CNNs models are also introduced in detail. We especially fine-tune the model and evaluate it on TQU-HG and HaGRID datasets. The quantitative results of the training and testing are presented (F1-score of YOLOv8, YOLO-Nas, MobileNet V3 small, ResNet50 is 98.99%, 98.98%, 99.27%, 99.36%, respectively on the TQU-HG dataset and is 99.21%, 99.37%, 99.36%, 86.4%, 98.3%, respectively on the HaGRID dataset). The computation time of YOLOv8 is 6.19 fps on the CPU and 18.28 fps on the GPU. [ABSTRACT FROM AUTHOR]

Objective: The effects of asset degradation on trust in human-swarm interaction were investigated through the lens of system-wide trust theory. Background: Researchers have begun investigating contextual features that shape human interactions with robotic swarms—systems comprising assets that coordinate behavior based on their nearest neighbors. Recent work has begun investigating how human trust toward swarms is affected by asset degradation through the lens of system-wide trust theory, but these studies have been marked by several limitations. Method: In an online study, the current work manipulated asset degradation and measured trust-relevant criteria in a within-subjects design and addressed the limitations of past work. Results: Controlling for swarm performance (i.e., target acquisition), asset degradation and trust (i.e., reliance intentions) in swarms were negatively related. In addition, as degradation increased, perceptions of swarm cohesion, obstacle avoidance, target acquisition, and terrain exploration efficiency decreased, the latter two of which (coupled with the reliance intentions criterion) support the tenets of system-wide trust theory as well as replicate and extend past work on the effects of asset degradation on trust in swarms. Conclusion: Human-swarm interaction is a context in which system-wide trust is relevant, and future work ought to investigate how to calibrate human trust toward swarm systems. Applications: Based on these findings, design professionals should prioritize ways to depict swarm performance and system health such that humans do not abandon trust in systems that are still functional yet not over-trust those systems which are indeed performing poorly. [ABSTRACT FROM AUTHOR]

The purpose of this study was to investigate decision-makers' views on changes that robotics will create in welfare services. The purpose was also to discover what the opportunities and challenges are in human–robot interaction during these changes and how to manage these changes. As a research method, an online survey was used. The survey was sent to Finnish decision-makers (N = 184). They were divided into three groups: Techno-positive (n = 66), Techno-neutral (n = 47), and Techno-critical (n = 71). According to the results, more than 80% of the respondents saw that robots can offer support in existing work tasks, and more than 70% saw that the robots can do existing tasks. The most often mentioned challenges were the reduction of interaction and the reduction of human touch. Further, there are various knowledge needs among the respondents. Most of the knowledge needs were not based on the technical use of the robots; rather, they were quite scattered. The results suggest that successful use and implementation of robots in welfare services require a comprehensive plan and change agents. This study suggests that techno-positive people could act as change agents, assisting in implementing the changes. In addition, to manage change in the welfare services it is essential to improve the quality of the information, solve the resistance to change, create organizational awareness, and understanding, and establish a psychological commitment to change the processes. [ABSTRACT FROM AUTHOR]

Mixed reality offers unique opportunities to situate complex tasks within spatial environments. One such task is the creation and manipulation of intricate, threedimensional paths, which remains a crucial challenge in many fields, including animation, architecture, and robotics. This paper presents an investigation into the possibilities of spatially situated path creation using new virtual and augmented reality technologies and examines how these technologies can be leveraged to afford more intuitive and natural path creation. We present a formative study (n = 20) evaluating an initial path planning interface situated in the context of augmented reality and human-robot interaction. Based on the findings of this study, we detail the development of two novel techniques for spatially situated path planning and manipulation that afford intuitive, expressive path creation at varying scales. We describe a comprehensive user study (n = 36) investigating the effectiveness, learnability, and efficiency of both techniques when paired with a range of canonical placement strategies. The results of this study confirm the usability of these interaction metaphors and provide further insight into how spatial interaction can be discreetly leveraged to enable interaction at scale. Overall, this work contributes to the development of 3DUIs that expand the possibilities for situating path-driven tasks in spatial environments. [ABSTRACT FROM AUTHOR]

Successful conversational interaction with a social robot requires not only an assessment of a user's contribution to an interaction, but also awareness of their emotional and attitudinal states as the interaction unfolds. To this end, our research aims to systematically trigger, but then interpret human behaviors to track different states of potential user confusion in interaction so that systems can be primed to adjust their policies in light of users entering confusion states. In this paper, we present a detailed human-robot interaction study to prompt, investigate, and eventually detect confusion states in users. The study itself employs a Wizard-of-Oz (WoZ) style design with a Pepper robot to prompt confusion states for task-oriented dialogues in a well-defined manner. The data collected from 81 participants includes audio and visual data, from both the robot's perspective and the environment, as well as participant survey data. From these data, we evaluated the correlations of induced confusion conditions with multimodal data, including eye gaze estimation, head pose estimation, facial emotion detection, silence duration time, and user speech analysis -- including emotion and pitch analysis. Analysis shows significant differences of participants' behaviors in states of confusion based on these signals, as well as a strong correlation between confusion conditions and participants own self-reported confusion scores. The paper establishes strong correlations between confusion levels and these observable features, and lays the ground or a more complete social and affect oriented strategy for task-oriented human-robot interaction. The contributions of this paper include the methodology applied, dataset, and our systematic analysis. [ABSTRACT FROM AUTHOR]

Virtual, Augmented, and Mixed Reality for Human-Robot Interaction (VAM-HRI) has been gaining considerable attention in HRI research in recent years. However, the HRI community lacks a set of shared terminology and framework for characterizing aspects of mixed reality interfaces, presenting serious problems for future research. Therefore, it is important to have a common set of terms and concepts that can be used to precisely describe and organize the diverse array of work being done within the field. In this article, we present a novel taxonomic framework for different types of VAM-HRI interfaces, composed of four main categories of virtual design elements (VDEs). We present and justify our taxonomy and explain how its elements have been developed over the past 30 years as well as the current directions VAM-HRI is headed in the coming decade. [ABSTRACT FROM AUTHOR]

Abstract User experience (UX) research in the Human-Technology Interaction (HTI) field makes use mainly of explicit (self-reported) measures to evaluate the UX of a technical product. Unfortunately, explicit measures present some limitations. For this reason, UX research could benefit from the inclusion of implicit measures (IMs) in the UX evaluation. In the social-psychological literature, the application of IMs evaluating implicit attitudes is a topic of active research; however, less knowledge is present in HTI. To this aim, we present a scoping review of the application of IMs evaluating implicit attitudes in the UX field. We used the Joanna Briggs Institute framework for scoping reviews. A database search was undertaken using IEEE Xplore, Web of Science, ACM digital library, and Science Direct databases. Three research questions have been identified. Of the 8943 identified records, only 12 articles matched the inclusion/exclusion criteria. Results suggest that a combined use of implicit and explicit measures provide a more comprehensive understanding of users’ evaluation of UX yet revealing participant’s divergences in attitudes toward technology especially in studies with robots or chatbots. More research is needed to disentangle this discrepancy and single out the specific factors that could influence this relationship. Taken together, IMs evaluating implicit attitudes should be strongly considered within a more integrated approach to evaluating UX. [ABSTRACT FROM AUTHOR]

Critical voices within and beyond the scientific community have pointed to a grave matter of concern regarding who is included in research and who is not. Subsequent investigations have revealed an extensive form of sampling bias across a broad range of disciplines that conduct human subjects research called "WEIRD": Western, Educated, Industrial, Rich, and Democratic. Recent work has indicated that this pattern exists within human–computer interaction (HCI) research, as well. How then does human–robot interaction (HRI) fare? And could there be other patterns of sampling bias at play, perhaps those especially relevant to this field of study? We conducted a systematic review of the premier ACM/IEEE International Conference on Human-Robot Interaction (2006–2022) to discover whether and how WEIRD HRI research is. Importantly, we expanded our purview to other factors of representation highlighted by critical work on inclusion and intersectionality as potentially underreported, overlooked, and even marginalized factors of human diversity. Findings from 827 studies across 749 papers confirm that participants in HRI research also tend to be drawn from WEIRD populations. Moreover, we find evidence of limited, obscured, and possible misrepresentation in participant sampling and reporting along key axes of diversity: sex and gender, race and ethnicity, age, sexuality and family configuration, disability, body type, ideology, and domain expertise. We discuss methodological and ethical implications for recruitment, analysis, and reporting, as well as the significance for HRI as a base of knowledge. [ABSTRACT FROM AUTHOR]

This study conducts a comparative analysis of user experiences of Augmented Reality (AR) and Virtual Reality (VR) headsets during an interactive semantic mapping task. This task entails the placement of virtual objects onto real-world counterparts. Our investigation focuses on discerning the distinctive features of each headset and their respective advantages within a semantic mapping context. The experiment employs a user interface enabling the creation, manipulation, and labeling of virtual 3D holograms. To ensure parity between the headsets, the VR headset mimics AR by relaying its camera feed to the user. A comprehensive user study, encompassing 12 participants tasked with mapping six tabletop objects, compares interface usability and performance between the headsets. The study participants' evaluations highlight that the VR headset offers enhanced user-friendliness and responsiveness compared to the AR headset. Nonetheless, the AR headset excels in augmenting environmental perception and interpretation, surpassing VR in this aspect. Consequently, the study underscores that current handheld motion controllers for interacting with virtual environments outperform existing hand gesture interfaces. Furthermore, it suggests potential improvements for VR devices, including an upgraded camera feed integration. Significantly, this experiment unveils the feasibility of leveraging VR headsets for AR applications without compromising user experience. However, it also points to the necessity of future research addressing prolonged usage scenarios for both types of headsets in various interactive tasks. [ABSTRACT FROM AUTHOR]

Facial expression recognition (FER) in the wild is an active and challenging field of research. A system for automatic FER finds use in a wide range of applications related to advanced human–computer interaction (HCI), human–robot interaction (HRI), human behavioral analysis, gaming and entertainment, etc. Since their inception, convolutional neural networks (CNNs) have attained state‐of‐the‐art accuracy in the facial analysis task. However, recognizing facial expressions in the wild with high confidence running on a low‐cost embedded device remains challenging. To this end, this study presents an efficient dual‐channel ensembled deep CNN (DCE‐DCNN) for FER in the wild. Initially, two DCNNs, namely the DCNNG$$ {\mathrm{DCNN}}_G $$ and DCNNS$$ {\mathrm{DCNN}}_S $$, are trained separately on the grayscale and Scharr‐convolved vertical gradient facial images, respectively. The proposed network later integrates the two pre‐trained DCNNs to obtain the dual‐channel integrated DCNN (DCI‐DCNN). Finally, all three neural networks, namely the DCNNG$$ {\mathrm{DCNN}}_G $$, DCNNS$$ {\mathrm{DCNN}}_S $$, and DCI‐DCNN, are jointly fine‐tuned to get a single dual‐channel‐multi‐output model. The multi‐output model produces three prediction scores for the given input facial image. The prediction scores are thus fused using the max‐voting ensemble scheme to obtain the DCE‐DCNN with the final classification label. On the FER2013, RAF‐DB, NCAER‐S, AffectNet, and CKPlus benchmark FER datasets, the proposed DCE‐DCNN consistently outperforms the two individual DCNNs and numerous state‐of‐the‐art CNNs. Moreover, the network achieves competitive recognition accuracy on all four FER in the wild datasets with reduced memory storage size and parameters. The proposed DCE‐DCNN model with high throughput on resource‐limited embedded devices is suitable for applications that seek real‐time classification of facial expressions in the wild with high confidence. [ABSTRACT FROM AUTHOR]

There is a growing demand for flexible pressure sensors in environmental monitoring and human–robot interaction robotics. A flexible and susceptible sensor can discriminate multidirectional pressure, thus effectively detecting signals of small environmental changes and providing solutions for personalized medicine. This paper proposes a multidimensional force detection sensor inspired by a wind chime structure with a three-dimensional force structure to detect and analyze normal and shear forces in real time. The force-sensing structure of the sensor consists of an upper and lower membrane on a polydimethylsiloxane substrate and four surrounding cylinders. A piezoelectric hemisphere is made of BTO/PVDF/PDMS composite material. The sensor columns in the wind chime structure surround the piezoelectric layer in the middle. When pressure is applied externally, the sensor columns are connected to the piezoelectric layer with a light touch. The piezoelectric hemisphere generates a voltage signal. Due to the particular structure of the sensor, it can accurately capture multidimensional forces and identify the direction of the external force by analyzing the position of the sensor and the output voltage amplitude. The development of such sensors shows excellent potential for self-powered wearable sensors, human–computer interaction, electronic skin, and soft robotics applications. [ABSTRACT FROM AUTHOR]

The social interaction restrictions imposed by COVID-19 required academic researchers to rethink the design of their research projects, especially those that involved human interaction. Considering this scenario, we report here the experience of conducting four studies in Human–Computer Interaction research projects that were adapted to be executed remotely: a focus group study, a communicability evaluation study, a speculative design study and a user observation study. Considering how the studies were planned, executed and analysed, we reflect on the adaptations needed, and lessons related to our experiences. From these lessons, we believe that, even after all the pandemic restrictions are lifted, we may continue conducting at least part of our research studies remotely to increase our geographical reach, reduce costs and facilitate data capture. [ABSTRACT FROM AUTHOR]

Recent work identified a concerning trend of disproportional gender representation in research participants in Human–Computer Interaction (HCI). Motivated by the fact that Human–Robot Interaction (HRI) shares many participant practices with HCI, we explored whether this trend is mirrored in our field. By producing a dataset covering participant gender representation in all 684 full papers published at the HRI conference from 2006–2021, we identify current trends in HRI research participation. We find an over-representation of men in research participants to date, as well as inconsistent and/or incomplete gender reporting, which typically engages in a binary treatment of gender at odds with published best practice guidelines. We further examine if and how participant gender has been considered in user studies to date, in-line with current discourse surrounding the importance and/or potential risks of gender based analyses. Finally, we complement this with a survey of HRI researchers to examine correlations between who is doing with the who is taking part, to further reflect on factors which seemingly influence gender bias in research participation across different sub-fields of HRI. Through our analysis, we identify areas for improvement, but also reason for optimism, and derive some practical suggestions for HRI researchers going forward. [ABSTRACT FROM AUTHOR]

Electronic sports show significant user churn caused by a toxic gaming atmosphere, and current GUI-based interventions are insufficient to address the issue. Based on the theoretical framework of Perceiving and Experiencing Fictional Characters, a new hybrid interaction interface and paradigm combined with tangibles is proposed to counter negative mood. To support the frustrated users of Massive Online Battle Arena (MOBA) games, we added AI teammates for better personal performance and social robots for the disclosure of negative mood. We hypothesized that AI teammates' invisibility and anonymity would mitigate negative emotions; an effect amplified by the presence of social robots. A comparative experiment was conducted with 111 participants. Social robots for emotion-oriented coping improved user mood but AI teammates for problem-oriented coping did so better, although their higher levels of experienced anonymity may not have been preferred. Unexpectedly, conversing with a robot after playing with an AI teammate brought the mood back to that experienced when talking to a robot alone, while increasing the distancing tendencies. With this in mind, AI and social robots can counter the negative atmosphere in MOBA games, positively contributing to game design and empathic human–computer interaction. [ABSTRACT FROM AUTHOR]

In order to streamline and summarize the status quo of human–computer interaction (HCI) design research in minimally invasive surgery robots, and to inspire and promote in-depth design research in related fields, this study utilizes literature research methods, inductive summarizing methods, and comparative analysis methods to analyze and organize the usage scenarios, users, interaction content and form, and relevant design methods of minimally invasive surgery robots, with the purpose of arriving at a review. Through a summary method, this study will obtain outcomes such as design requirements, interaction information classification, and the advantages and disadvantages of different interaction forms, and then make predictions of future trends in this field. Research findings show that the HCI design in the relevant field display a highly intelligent, human-centered, and multimodal development trend through the application of cutting-edge technology, taking full account of work efficiency and user needs. However, meanwhile, there are problems such as the absence of guidance by a systematic user knowledge framework and incomplete design evaluation factors, which need to be supplemented and improved by researchers in related fields in the future. [ABSTRACT FROM AUTHOR]

Children with autism spectrum disorder (ASD) have deficits in social interaction and expressing and understanding emotions. Based on this, robots for children with ASD have been proposed. However, few studies have been conducted about how to design a social robot for children with ASD. Non-experimental studies have been carried out to evaluate social robots; however, the general methodology that should be used to design a social robot is not clear. This study proposes a design path for a social robot for emotional communication for children with ASD following a user-centered design approach. This design path was applied to a case study and evaluated by a group of experts in psychology, human–robot interaction, and human–computer interaction from Chile and Colombia, as well as parents of children with ASD. Our results show that following the proposed design path for a social robot to communicate emotions for children with ASD is favorable. [ABSTRACT FROM AUTHOR]

Accurate recognition of disabled persons' behavioral intentions is the key to reconstructing hand function. Their intentions can be understood to some extent by electromyography (EMG), electroencephalogram (EEG), and arm movements, but they are not reliable enough to be generally accepted. In this paper, characteristics of foot contact force signals are investigated, and a method of expressing grasping intentions based on hallux (big toe) touch sense is proposed. First, force signals acquisition methods and devices are investigated and designed. By analyzing characteristics of signals in different areas of the foot, the hallux is selected. The peak number and other characteristic parameters are used to characterize signals, which can significantly express grasping intentions. Second, considering complex and fine tasks of the assistive hand, a posture control method is proposed. Based on this, many human-in-the-loop experiments are conducted using human–computer interaction methods. The results showed that people with hand disabilities could accurately express their grasping intentions through their toes, and could accurately grasp objects of different sizes, shapes, and hardness using their feet. The accuracy of the action completion for single-handed and double-handed disabled individuals was 99% and 98%, respectively. This proves that the method of using toe tactile sensation for assisting disabled individuals in hand control can help them complete daily fine motor activities. The method is easily acceptable in terms of reliability, unobtrusiveness, and aesthetics. [ABSTRACT FROM AUTHOR]

Emotions recognition is widely applied for many tasks in different fields, from human-computer and human-robot interaction to learning platforms. Also, it can be used as an intrinsic approach for face recognition tasks, in which an expression-independent face classifier is developed. Most approaches face the problem by designing deeper and deeper neural networks that consider an expression as a still image or, in some cases, a sequence of consecutive frames depicting the temporal component of the expression. However, these suffer the training phase's computational burden, which can take hours or days to be completed. In this work, a Web Shaped Model is proposed, which consists of a geometrical approach for extracting discriminant features from a face, depicting the characteristics of an expression. The model does not need to be trained since it is applied on a face and centred on the nose tip, resulting in image size and face size independence. Experiments on publicly available datasets show that this approach reaches comparable and even better results than those obtained applying DNN-based approaches. [ABSTRACT FROM AUTHOR]

Systems for automatic facial expression recognition (FER) have an enormous need in advanced human-computer interaction (HCI) and human-robot interaction (HRI) applications. Over the years, researchers developed many handcrafted feature descriptors for the FER task. These descriptors delivered good accuracy on publicly available FER benchmark datasets. However, these descriptors generate high dimensional features that increase the computational time of the classifiers. Also, a significant proportion of the features are irrelevant and do not provide additional information for facial expression analysis. Adversely, these redundant features degrade the classification accuracy of the FER algorithm. This study presents an alternate, simple, and efficient scheme for FER in static images using the Boosted Histogram of Oriented Gradient (BHOG) descriptor. The proposed BHOG descriptor employs the AdaBoost feature selection algorithm to select important facial features from the original high-dimensional Histogram of Oriented Gradient (HOG) features. The BHOG descriptor with a reduced feature dimension decreases the computational cost without diminishing the recognition accuracy. The proposed FER pipeline tuned on the optimal values of different hyperparameters achieves competitive recognition accuracy on five benchmark FER datasets, namely CK+, JAFFE, RaFD, TFE, and RAF-DB. Also, the cross-dataset experiments confirm the superior generalization performance of the proposed FER pipeline. Finally, the comparative analysis results with existing FER techniques revealed the effectiveness of the pipeline. The proposed FER scheme is computationally efficient and classifies facial expressions in real time. [ABSTRACT FROM AUTHOR]

This study conducts a comparative analysis of user experiences of Augmented Reality (AR) and Virtual Reality (VR) headsets during an interactive semantic mapping task. This task entails the placement of virtual objects onto real-world counterparts. Our investigation focuses on discerning the distinctive features of each headset and their respective advantages within a semantic mapping context. The experiment employs a user interface enabling the creation, manipulation, and labeling of virtual 3D holograms. To ensure parity between the headsets, the VR headset mimics AR by relaying its camera feed to the user. A comprehensive user study, encompassing 12 participants tasked with mapping six tabletop objects, compares interface usability and performance between the headsets. The study participants’ evaluations highlight that the VR headset offers enhanced user-friendliness and responsiveness compared to the AR headset. Nonetheless, the AR headset excels in augmenting environmental perception and interpretation, surpassing VR in this aspect. Consequently, the study underscores that current handheld motion controllers for interacting with virtual environments outperform existing hand gesture interfaces. Furthermore, it suggests potential improvements for VR devices, including an upgraded camera feed integration. Significantly, this experiment unveils the feasibility of leveraging VR headsets for AR applications without compromising user experience. However, it also points to the necessity of future research addressing prolonged usage scenarios for both types of headsets in various interactive tasks.

The proliferation and integration of social technologies has occurred quickly, and the specific technologies with which we engage are ever-changing. The dynamic nature of the development and use of social technologies is often acknowledged by researchers as a limitation. In this manuscript, however, we present a discussion on the implications of our modern technological context by focusing on processes of socialization and communication that are fundamentally different from their interpersonal corollary. These are presented and discussed with the goal of providing theoretical building blocks toward a more robust understanding of phenomena of human-computer interaction, human-robot interaction, human-machine communication, and interpersonal communication.

As robots are deployed to work in our environments, we must build appropriate expectations of their behavior so that we can trust them to perform their jobs autonomously as we attend to other tasks. Many types of explanations for robot behavior have been proposed, but they have not been fully analyzed for their impact on aligning expectations of robot paths for navigation. In this work, we evaluate several types of robot navigation explanations to understand their impact on the ability of humans to anticipate a robot's paths. We performed an experiment in which we gave participants an explanation of a robot path and then measured (i) their ability to predict that path, (ii) their allocation of attention on the robot navigating the path versus their own dot-tracking task, and (iii) their subjective ratings of the robot's predictability and trustworthiness. Our results show that explanations do significantly affect people's ability to predict robot paths and that explanations that are concise and do not require readers to perform mental transformations are most effective at reducing attention to the robot. [ABSTRACT FROM AUTHOR]

Robots can learn from humans by asking questions. In these questions, the robot demonstrates a few different behaviors and asks the human for their favorite. But how should robots choose which questions to ask? Today's robots optimize for informative questions that actively probe the human's preferences as efficiently as possible. But while informative questions make sense from the robot's perspective, human onlookers may find them arbitrary and misleading. For example, consider an assistive robot learning to put away the dishes. Based on your answers to previous questions this robot knows where it should stack each dish; however, the robot is unsure about right height to carry these dishes. A robot optimizing only for informative questions focuses purely on this height: it shows trajectories that carry the plates near or far from the table, regardless of whether or not they stack the dishes correctly. As a result, when we see this question, we mistakenly think that the robot is still confused about where to stack the dishes! In this article, we formalize active preference-based learning from the human's perspective. We hypothesize that—from the human's point-of-view —the robot's questions reveal what the robot has and has not learned. Our insight enables robots to use questions to make their learning process transparent to the human operator. We develop and test a model that robots can leverage to relate the questions they ask to the information these questions reveal. We then introduce a tradeoff between informative and revealing questions that considers both human and robot perspectives: a robot that optimizes for this tradeoff actively gathers information from the human while simultaneously keeping the human up to date with what it has learned. We evaluate our approach across simulations, online surveys, and in-person user studies. We find that robots, which consider the human's point of view learn just as quickly as state-of-the-art baselines while also communicating what they have learned to the human operator. Videos of our user studies and results are available here: https://youtu.be/tC6y_jHN7Vw. [ABSTRACT FROM AUTHOR]

Advances in exoskeletons and robot arms have given us increasing opportunities for providing physical support and meaningful feedback in training and rehabilitation settings. However, the chosen control strategies must support motor learning and provide mathematical task definitions that are actionable for the actuation. Typical robot control architectures rely on measuring error from a reference trajectory. In physical human-robot interaction, this leads to low engagement, invariant practice, and few errors, which are not conducive to motor learning. A reliance on reference trajectories means that the task definition is both over-specified—requiring specific timings not critical to task success—and lacking information about normal variability. In this article, we examine a way to define tasks and close the loop using an ergodic measure that quantifies how much information about a task is encoded in the human-robot motion. This measure can capture the natural variability that exists in typical human motion, enabling therapy based on scientific principles of motor learning. We implement an ergodic hybrid shared controller (HSC) on a robotic arm as well as an error-based controller—virtual fixtures—in a timed drawing task. In a study of 24 participants, we compare ergodic HSC with virtual fixtures and find that ergodic HSC leads to improved training outcomes. [ABSTRACT FROM AUTHOR]

Potential applications of robots in private and public human spaces have prompted the design of so-called "social robots" that can interact with humans in social settings and potentially cause humans to attach to the robots. The focus of this article is an analysis of possible benefits and challenges arising from such human-robot attachment as reported in the HRI literature, followed by guidelines for the use and the design of robots that might elicit attachment bonds. We start by analyzing the potential benefits for humans becoming attached to robots, which might include increased natural interaction, effectiveness and acceptance of the robot, social companionship, and well-being for the human. Turning to the potential risks associated with human-robot attachment, we discuss the possibly suboptimal use of the robot in the most benign cases, but also the potential formation of unidirectional emotional bonds, and the potential for deception and subconscious influence of the robot on the person in more severe cases. The upshot of the analysis then is a recommendation to reconceptualize relationships with social robots in an attempt to retain potential benefits of human-robot attachment, while mitigating (to the extent possible) its downsides. [ABSTRACT FROM AUTHOR]

To facilitate continuous development of novel HRI systems, it is beneficial to have tools that enable quick adjustments, flexibility, or re-invention of the human interfaces when system requirements change due to updates in the state-of-the-art, application domain, etc. Thus, modularity is a key design principle which promotes software reuse and scalability, and reduces development time and cost. Hence, a robot's autonomous capabilities should not depend on the command interface and should be decoupled via a common format that possesses the descriptive capabilities for outlining tasks and has a sensible syntax for HRI. In this paper, we propose the Unified Meaning Representation Format (UMRF), which provides the syntax and semantics for passing both simple and complex commands modelled as control flow graphs. UMRF is a standalone meaning representation container that supports embedding other meaning representation formalisms, such as predicate-argument semantics and graphical meaning representation formats, making it adoptable as a standard task description format for semi-autonomous systems in HRI domains. In this article, we define the UMRF syntax and semantics, summarize its unique aspects relative to related task description formats, and demonstrate its descriptiveness by navigating a robot via concurrent (e.g., gestures and speech) and interchangeable input systems (e.g., Google Assistant, Amazon Alexa). [ABSTRACT FROM AUTHOR]

In recent years, recognizing the visual focus of attention (VFoA) has attracted much attention among computer vision experts due to its various Human–Computer Interaction (HCI) or Human-Robot Interaction (HRI) applications. Although eye gaze is a potential cue to determine someone's focus of attention (FOA), it is challenging to determine FOA alone when the interacting partners are far away or the camera cannot capture high-resolution images from long-distance. Therefore, the head pose can be used as an approximation to recognize the focus of someone's attention. This paper proposes a vision-based framework to detect the FOA of humans using nine head poses consisting of four main modules: face detection and facial key-point selection (FDKPSM), head pose classification (HPCM), object localization and classification (OLCM), and focus of attention estimation (FoAEM). The FDKPSM uses the Multi-task Cascaded Neural Network (MTCNN) framework to detect head poses, and the HPCM classifies them into nine classes using the ResNet18. To estimate the FoA, the FoAEM uses a mapping Algorithm (EFoA) which integrates head poses on the focused object. Experimental results show that the proposed model outperformed other deep learning models by achieving the highest accuracy on three datasets: BIWI-M (96.97 %), Pointing'04-M (96.04 %) and HPoD 9 (98.99 %). The visual focus of the attention model gained an accuracy of 94.12 % in the multi-object scenario. [ABSTRACT FROM AUTHOR]

Socially assistive robots (SARs) have been proposed to assist post-stroke patients in performing their exercise during their rehabilitation process, with the trust in the robot identified as an important factor in human–robot interaction. In the current study, we aimed to identify and characterize factors that influence post-stroke patients' trust in a robot-operated and a computer-operated rehabilitation platform during and after a long-term experience with the platform. We conducted 29 interviews with 16 stroke patients who underwent a long-term rehabilitation process, assisted by either a SAR or a computer interface. The intervention lasted 5–7 weeks per patient, for a total of 229 sessions over 18 months. By using a qualitative research method—extended interviews "in the wild" with stroke patients, over a long-term rehabilitation process—our study reveals users' perspectives regarding factors affecting trust in the SAR or in the computer interface during their rehabilitation process. The results support the assertion that SARs have an added value in the rehabilitative care of stroke patients; It appears that personal characteristics, such as age and gender, have an effect on the users' acceptance of a non-human operator as a practice assistant. Our findings support the notion that SARs augment rehabilitative therapies beyond a standard computer; Importantly, patients appreciated different aspects of the non-human operator in the two groups: In the SAR group, users preferred its functional performance over its anthropomorphized social skills; In the Computer group, users highlighted its contribution to the training of their memory skills. [ABSTRACT FROM AUTHOR]

On March 27–29, 2023, the AAAI symposium on "HRI in Academia and Industry: Bridging the Gap" was held in a hybrid format, with both in‐person and remote participants, gathering Human‐Robot Interaction (HRI) researchers and practitioners from academia, industry, and national research laboratories to find common ground, understand the different constraints at play, and determine how to work together. The use of robots that operate in spaces in which humans are physically co‐present is growing at a dramatic rate. We are seeing more and more robots in our warehouses, on our streets, and even in our homes. All of these robots will interact with humans in some way, whether intentionally or unintentionally. To be successful, their interactions with humans will have to be carefully designed. For more than a decade, the field of HRI has been growing at the intersection of robotics, Artificial Intelligence (AI) , human‐computer interaction (HCI), psychology, and other fields; however, until quite recently, it has been a largely academic area, with university researchers proposing, implementing, and reporting on experiments at a limited scale. With the current increase of commercially‐available robots, HRI is starting to make its way into the robotics industry in a meaningful way. This symposium brought together HRI researchers and practitioners from academia, industry, and national research laboratories to find common ground, understand the different constraints at play, and determine how to effectively work together. [ABSTRACT FROM AUTHOR]

Ever since its discovery, the applications of Shape Memory Alloys (SMA) can be found across a range of application domains, from structural design to medical technology. This is based upon the unique and inherent characteristics such as thermal Shape Memory Effect (SME) and Superelasticity (or Pseudoelasticity). While thermal SME is used for shape morphing applications wherein temperature change can govern the shape and dimension of the SMA, Superelasticity allows the alloy to withstand a comparatively very high magnitude of loads without undergoing plastic deformation at higher temperatures. These unique properties in wearables have revolutionized the field, and from fabrics to exoskeletons, SMA has found its place in robotics and cobotics. This review article focuses on the most recent research work in the field of SMA-based smart wearables paired with robotic applications for human-robot interaction. The literature is categorized based on SMA property incorporated and on actuator or sensor-based concept. Further, use-cases or conceptual frameworks for SMA fiber in fabric for 'Smart Jacket' and SMA springs in the shoe soles for 'Smart Shoes' are proposed. The conceptual frameworks are built upon existing technologies; however, their utility in a smart factory concept is emphasized, and algorithms to achieve the same are proposed. The integration of the two concepts with the Industrial Internet of Things (IIoT) is discussed, specifically regarding minimizing hazards for the worker/user in Industry 5.0. The article aims to propel a discussion regarding the multi-faceted applications of SMAs in human-robot interaction and Industry 5.0. Furthermore, the challenges and the limitations of the smart alloy and the technological barriers restricting the growth of SMA applications in the field of smart wearables are observed and elaborated. [ABSTRACT FROM AUTHOR]

This special issue of the Transactions on Human-Robot Interaction highlights, documents, and explores the metrics, test methods, and artifacts used in human-robot interaction (HRI) research. This collection of articles brings to attention the commonalities between the application of measurement science for the assessment and assurance of human-centric robotics in a variety of application domains, including industry, education, and defense. This special issue draws specific attention to the use and impact of metrology toward the advancement of HRI technologies and algorithms, and it promotes the application of measurement science toward the benchmarking and replication of HRI research. Special attention is given to the use cases, data sets, test methodologies, measurement techniques, metrics, and statistical analyses used to evaluate system performance. [ABSTRACT FROM AUTHOR]

The rise in artificial intelligence capabilities in autonomy-enabled systems and robotics has pushed research to address the unique nature of human-autonomy team collaboration. The goal of these advanced technologies is to enable rapid decision-making, enhance situation awareness, promote shared understanding, and improve team dynamics. Simultaneously, use of these technologies is expected to reduce risk to those who collaborate with these systems. Yet, for appropriate human-autonomy teaming to take place, especially as we move beyond dyadic partnerships, proper calibration of team trust is needed to effectively coordinate interactions during high-risk operations. But to meet this end, critical measures of team trust for this new dynamic of human-autonomy teams are needed. This article seeks to expand on trust measurement principles and the foundation of human-autonomy teaming to propose a "toolkit" of novel methods that support the development, maintenance, and calibration of trust in human-autonomy teams operating within uncertain, risky, and dynamic environments. [ABSTRACT FROM AUTHOR]

Humans report perceptions of unease or eeriness as humanoid/android robots and digital avatars approach human-like physical resemblance, a phenomenon alluded by the Uncanny Valley theory. This study extends the discussions on interactions and acceptance of digital avatars with findings from three experiments. In the first, perceptive evaluation of actors in clips from computer-generated animation and a live-action version of the same movie was examined. In the second experiment, we considered short clips with highly realistic digital avatars to measure recognition ability, the extent of eeriness, and specific physical features identified as unreal. The fixation area and pupil size variation recorded using an eye tracker were analyzed to infer attention to the body, face, and emotional response, respectively. Building on these findings, the third experiment looked at acceptance in roles requiring human skill, empathy, and cognitive ability. The results show that based on perceptions from physical attributes, the eeriness scores diverge from the uncanny valley theory as human-likeness increases. The realistic CGI and mocap technology could have helped cross the valley. Visual attention inferred from gaze behavior was similar for live-action and CGI. At the same time, we observe pupil size changes reflecting emotions like eeriness when the avatars either talked or smiled. Proficiency and acceptance scores were lower for roles requiring complex social cognition processes, such as friends and judicial decision-making. Interestingly, real-life stereotypes of gender roles were transferred to digital avatars too. The findings suggest an ambiguity in accepting human-like avatars in social and professional interactions, emphasizing the need for a multi-dimensional approach when applying the uncanny valley theory. A detailed and contextual examination is imperative as technological advancements have placed humans closer to co-existing with digital or physical android/humanoid robots. [ABSTRACT FROM AUTHOR]

As development of robots with the ability to self-assess their proficiency for accomplishing tasks continues to grow, metrics are needed to evaluate the characteristics and performance of these robot systems and their interactions with humans. This proficiency-based human-robot interaction (HRI) use case can occur before, during, or after the performance of a task. This article presents a set of metrics for this use case, driven by a four-stage cyclical interaction flow: (1) robot self-assessment of proficiency (RSA), (2) robot communication of proficiency to the human (RCP), (3) human understanding of proficiency (HUP), and (4) robot perception of the human's intentions, values, and assessments (RPH). This effort leverages work from related fields including explainability, transparency, and introspection, by repurposing metrics under the context of proficiency self-assessment. Considerations for temporal level (a priori, in situ, and post hoc) on the metrics are reviewed, as are the connections between metrics within or across stages in the proficiency-based interaction flow. This article provides a common framework and language for metrics to enhance the development and measurement of HRI in the field of proficiency self-assessment. [ABSTRACT FROM AUTHOR]

Proximate human-robot teaming (pxHRT) is a complex subspace within human-robot interaction. Studies in this space involve a range of equipment and methods, including the ability to sense people and robots precisely. Research in this area draws from a wide variety of other fields, from human-human interaction to control theory, making the study design complex, particularly for those outside the field of HRI. In this paper, we introduce a framework that helps researchers consider tradeoffs across various task contexts, platforms, sensors, and analysis methods; metrics frequently used in the field; and common challenges researchers may face. We demonstrate the use of the framework via a case study which employs an autonomous mobile manipulator continuously engaging in shared workspace, handover, and co-manipulation tasks with people, and explores the effect of cognitive workload on pxHRT dynamics. We also demonstrate the utility of the framework in a case study with two groups of researchers new to pxHRT. With this framework, we hope to enable researchers, especially those outside HRI, to more thoroughly consider these complex components within their studies, more easily design experiments, and more fully explore research questions within the space of pxHRT. [ABSTRACT FROM AUTHOR]

As social robots become more prominent in our lives, their interaction with humans takes an increasing role, and new collaborative scenarios emerge. This development brings the need to realize robust test methods enabling the design and evaluation of Human-Robot Interaction (HRI) teaming tasks to prove functionality and promote adoption. In this article, we present a general-purpose and repeatable methodology for conducting studies in collaborative HRI in the range of robotic competitions. The methodology includes a step-by-step approach to design HRI teaming tasks tailored to be enacted in a robotic competition and to evaluate the performance of social robots to execute the designed tasks, exploring the relationship between robots' performance and user perceptions based on the feedback of the users participating to such tasks. We assess the feasibility of the methodology to design and evaluate an HRI teaming task in the context of "Smart CIties RObotics Challenges" (SciRoc) competition, which targets at investigating the impact of social of robots in smart cities. [ABSTRACT FROM AUTHOR]

In order to give assertive support, robots need to understand the cognitive and emotional characteristics of learners while in the learning process. Also, if the task involves handling capacity, robots must have similar skills. Three concepts were explored to control the cognitive and emotional robot's behavior: the psychological flow theory, the scaffolding pedagogic strategy, and the multiagents-software paradigm. Based on these concepts, the Human-Robot Scaffolding architecture was designed. It is divided into five blocks. First, the sensory block recognizes body gestures, speech, and task state. Second, the beliefs block estimates the skills and emotional state of learners. Third, the desires block validates the goals the robot can reach; the goals are grouped in skills development, emotional control, cognitive control, challenge control, life signals, and immediate support. Fourth, the intentions block, based on the goals competition strategy, selects the goal that the robot will perform. Finally, the action-planner block regulates the robot's movements according to the robot's emotions. The validation procedure was done with 53 learners ranging between 10 and 13 years old who study in public and private schools. Based on the research achievements, the robot fosters learning the Mean-Ends Analysis strategy and the solution of a problem. A video fragment that summarizes the research process is available in https://youtu.be/qbohCjBIwYc. [ABSTRACT FROM AUTHOR]

Metrics for human-robot teaming should extend to teams consisting of multiple human and robotic agents, and to teams working in complex, dynamic work domains. This work proposes that to comprehensively analyze and evaluate multi-human, multi-robot teams, traditional HRI metrics of performance, and efficiency must be expanded upon to incorporate metrics of teamwork. We develop five distinct metrics to capture both ecological and cognitive aspects of teamwork found to be important in human-automation interaction, inspired by research in the cognitive systems engineering (CSE) community. We demonstrate the application of these metrics in a spacecraft maintenance case study comparing multiple human-robot team architectures. The case study demonstrates that the teamwork metrics capture aspects of human-robot interaction (HRI) not apparent when using only traditional performance and efficiency metrics. The article concludes that the proposed teamwork metrics are complementary to existing metrics in HRI and should be included in the evaluation of human-robot teams. [ABSTRACT FROM AUTHOR]