Search

User interface design patterns that document Human-Agent Team (HAT) interface best practices were developed by analyzing team video games with different types of agents, ranging from virtual players in sports games to advisors contributing to strategy and planning in combat games. One factor that may contribute to the enormous success of some games is the cohesive team interactions between humans and agents. A design pattern collection was developed by identifying 25 design patterns under interface themes most relevant to HATs. The design patterns describe how multiple successful game interfaces handle challenging aspects of HAT interaction, addressing contexts including planning, spatial orientation, task management, communication, and more. Based on the complex and well-tested interface solutions in video games, these design patterns can provide HAT user interface developers practical guidance for interface development and a starting point for developing a HAT design pattern language. [ABSTRACT FROM AUTHOR]

Virtual environments (VEs) can be infinitely large, but movement of the virtual reality (VR) user is constrained by the surrounding real environment. Teleporting has become a popular locomotion interface to allow complete exploration of the VE. To teleport, the user selects the intended position (and sometimes orientation) before being instantly transported to that location. However, locomotion interfaces such as teleporting can cause disorientation. This experiment explored whether practice and feedback when using the teleporting interface can reduce disorientation. VR headset owners participated remotely. On each trial of a triangle completion task, the participant traveled along two path legs through a VE before attempting to point to the path origin. Travel was completed with one of two teleporting interfaces that differed in the availability of rotational self-motion cues. Participants in the feedback condition received feedback about their pointing accuracy. For both teleporting interfaces tested, feedback caused significant improvement in pointing performance, and practice alone caused only marginal improvement. These results suggest that disorientation in VR can be reduced through feedback-based training. [ABSTRACT FROM AUTHOR]

Industrial systems can be complex and not intuitive to perceive. Therefore, students in technology and engineering programs can benefit from developing mental models of industrial systems during their journey in college. However, more often than not, these students do not have access to industrial facilities; thus, developing mental models for systems is a challenge. This paper examines the merit of an Immersive Virtual Reality (IVR) framework application in creating proper mental models for industrial systems in technology and engineering students. Two IVR applications were developed. One IVR application afforded interaction with components of a prefabricated industrial cooling water system (CWVR). In the other application, students designed and built industrial systems with IVR (system designer VR [SDVR]). SDVR facilitated constructive‐generative engagement. A group of 33 students was divided into two; one group (the Design, experimental group) was tasked with building a system with SDVR and interacting with the cooling water system in CWVR. The other group was tasked with directly interacting with the CWVR without building a system with SDVR (the Interaction, comparison group). Students' mental models of the cooling water system in CWVR were evaluated following completing the interaction experience with CWVR. The results demonstrate that the causal model notion of the mental model of the cooling water system was significantly higher in the Design, experimental group. The results suggest that designing a rich IVR application that facilitates constructive‐generative engagements may carry merit in informing student mental models of complex technical concepts. Lay Description: Industrial systems can be complex and not intuitive to perceive.Students in technology‐related programs may struggle to develop mental models for systems.Immersive virtual reality applications for industrial systems were developed and tested.Results indicate that virtual reality might inform mental models of systems in students. [ABSTRACT FROM AUTHOR]

This study explored the impact of task workload on virtual reality (VR) cybersickness. Cybersickness is a negative side effect of using VR to which many users are susceptible. Previous research on the impact on task workload on cybersickness has yielded no consistent relationships, but given that task workload requires attentional resources, it is worth further investigation of how a demand on attention might increase or decrease cybersickness. In this study, mental workload of participants (N = 151) was modified in three different task groups within the same virtual environment (VE). The Cybersickness Corn Maze VR testbed contained cybersickness-inducing stimuli and tasks with varying workload. The 0-Back group used a controller to select an object as a visual attention task. The 2-Back group performed the 2-Back memory detection task, using a controller to collect objects that matched the object presented two objects ago. The No-Task group passively moved through the environment and was not given a controller. Workload, cybersickness, dropout rate, presence, and task accuracy were compared across groups. Workload was found to be statistically significantly different in each group: highest in the 2-Back group, medium in the 0-Back group, and lowest in the No-Task group, validating the task design. Cybersickness in the 2-Back group was significantly higher than in the No-Task (140% higher) and 0-Back (54% higher) groups measured by the change in simulator sickness questionnaire (SSQ) total severity (Post SSQ - Pre SSQ). The rate of participants’ dropout due to cybersickness was significantly higher in 2-back (33%) as compared with 0-Back (10%), but 0-Back and NoTask dropout rates were not significantly different. These results indicate that 1) task workload affects cybersickness and 2) its effect could be based on a threshold of workload. Presence increased with the addition of a task but plateaued between the 0-Back and 2-Back groups, suggesting that presence can be affected by task workload but only to a certain extent. Task accuracy was shown to negatively correlate with cybersickness within the task groups. A relationship between workload and cybersickness was found and warrants further research into these concepts. This work highlights the need for task workload and attention to be studied as components of the mechanisms underlying cybersickness. [ABSTRACT FROM AUTHOR]

The Reading the Mind in the Eyes Test (RMET) has received attention due to its correlation with collective intelligence. If the RMET is a marker of collective intelligence, training to improve RMET could result in better teamwork, whether for human-human or human-AI (artificial intelligence) in composition. While training on related skills has proven effective in the literature, RMET training has not been studied. This research evaluates the development of RMET training, testing the impact of two training conditions (Naturalistic Training and Repeated RMET Practice) compared to a control. There were no significant differences in RMET scores due to training, but speed of response was positively correlated to RMET score for high-scoring participants. Both management professionals and AI creators looking to cultivate team skill through the application of the RMET may need to reconsider their tool selection. [ABSTRACT FROM AUTHOR]

This research assessed how the performance and team skills of three-person teams working with an Intelligent Team Tutoring System (ITTS) on a virtual military surveillance task were affected by feedback privacy, participant role, task experience, prior team experience, and teammate familiarity. Previous work in Intelligent Tutoring Systems (ITSs) has focused on outcomes for task skill training for individual learners. As research extends into intelligent tutoring for teams, both task skills and team skills are necessary for good team performance. This work includes a brief review of previous research on ITTSs, feedback, teams, and teamwork, including the recounting of two categories of a framework of teamwork performance, Communication and Cognition, which are relevant to the present study. This research examines the effects of an intelligent agent, as well as features of the team, its members, and the task being undertaken, on team communication (measured by relevant key-presses) and team situation awareness (as measured by scores on a quiz). Thirty-seven teams of three participants, each at their own computer running a multiplayer surveillance simulation, were given just-in-time private (individually delivered) or public (team-delivered) performance feedback during four 5-min trials. In the fourth trial, two of the three participants switched roles. Feedback type, teamwork experience, and teammate familiarity had no statistically significant effect on communication or team situation awareness. However, higher levels of role experience and task experience showed significant and medium-sized effects on communication performance. Results, based on performance data and structured interview responses, also revealed areas of improvement in future feedback design and a potential benchmark for feedback frequency in an action-oriented serious game-based ITTS. Among the conclusions are six design objectives for future ITTSs, establishing a foundation for future research on designing effective ITTSs that train interpersonal skills to nascent teams. [ABSTRACT FROM AUTHOR]

Virtual reality systems typically allow users to physically walk and turn, but virtual environments (VEs) often exceed the available walking space. Teleporting has become a common user interface, whereby the user aims a laser pointer to indicate the desired location, and sometimes orientation, in the VE before being transported without self-motion cues. This study evaluated the influence of rotational self-motion cues on spatial updating performance when teleporting, and whether the importance of rotational cues varies across movement scale and environment scale. Participants performed a triangle completion task by teleporting along two outbound path legs before pointing to the unmarked path origin. Rotational self-motion reduced overall errors across all levels of movement scale and environment scale, though it also introduced a slight bias toward under-rotation. The importance of rotational self-motion was exaggerated when navigating large triangles and when the surrounding environment was large. Navigating a large triangle within a small VE brought participants closer to surrounding landmarks and boundaries, which led to greater reliance on piloting (landmark-based navigation) and therefore reduced—but did not eliminate—the impact of rotational self-motion cues. These results indicate that rotational self-motion cues are important when teleporting, and that navigation can be improved by enabling piloting. [ABSTRACT FROM AUTHOR]

Open data standards (e.g. LandXML, TransXML, CityGML) are a key to addressing the interoperability issue in exchanging civil information modeling (CIM) data throughout the project life-cycle. Since these schemas include rich sets of data types covering a wide range of assets and disciplines, model view definitions (MVDs) which define subsets of a schema are required to specify what types of data to be shared in accordance with a specific exchange scenario. The traditional procedure for generating and implementing MVDs is time-consuming and laborious as entities and attributes relevant to a particular data exchange context are manually identified by domain experts. This paper presents a method that can locate relevant information from a source XML data schema for a specific domain based on the user's keyword. The study employs a semantic resource of civil engineering terms to understand the semantics of a keyword-based query. The study also introduces a novel context-based search technique for retrieving related entities and their referenced objects. The developed method was tested on a gold standard of several LandXML subschemas. The experiment results show that the semantic MVD retrieval algorithm achieves a mean average precision of nearly 90%. The research is original, being a novel method for extracting partial civil information models given a keyword from the end user. The method is expected to become a fundamental tool assisting professionals in extracting data from complex digital datasets. [ABSTRACT FROM AUTHOR]

Systematic reviews are increasingly using data from preclinical animal experiments in evidence networks. Further, there are ever-increasing efforts to automate aspects of the systematic review process. When assessing systematic bias and unit-of-analysis errors in preclinical experiments, it is critical to understand the study design elements employed by investigators. Such information can also inform prioritization of automation efforts that allow the identification of the most common issues. The aim of this study was to identify the design elements used by investigators in preclinical research in order to inform unique aspects of assessment of bias and error in preclinical research. Using 100 preclinical experiments each related to brain trauma and toxicology, we assessed design elements described by the investigators. We evaluated Methods and Materials sections of reports for descriptions of the following design elements: 1) use of comparison group, 2) unit of allocation of the interventions to study units, 3) arrangement of factors, 4) method of factor allocation to study units, 5) concealment of the factors during allocation and outcome assessment, 6) independence of study units, and 7) nature of factors. Many investigators reported using design elements that suggested the potential for unit-of-analysis errors, i.e., descriptions of repeated measurements of the outcome (94/200) and descriptions of potential for pseudo-replication (99/200). Use of complex factor arrangements was common, with 112 experiments using some form of factorial design (complete, incomplete or split-plot-like). In the toxicology dataset, 20 of the 100 experiments appeared to use a split-plot-like design, although no investigators used this term. The common use of repeated measures and factorial designs means understanding bias and error in preclinical experimental design might require greater expertise than simple parallel designs. Similarly, use of complex factor arrangements creates novel challenges for accurate automation of data extraction and bias and error assessment in preclinical experiments. [ABSTRACT FROM AUTHOR]

With the movement in education towards collaborative learning, it is becoming more important that learners be able to work together in groups and teams. Intelligent tutoring systems (ITSs) have been used successfully to teach individuals, but so far only a few ITSs have been used for the purpose of training teams. This is due to the difficulty of creating such systems. An ITS for teams must be able to assess complex interactions between team members (team skills) as well as the way they interact with the system itself (task skills). Assessing team skills can be difficult because they contain social components such as communication and coordination that are not readily quantifiable. This article addresses these difficulties by developing a framework to guide the authoring process for team tutors. The framework is demonstrated using a case study about a particular team tutor that was developed using a military surveillance scenario for teams of two. The Generalized Intelligent Framework for Tutoring (GIFT) software provided the team tutoring infrastructure for this task. A new software architecture required to support the team tutor is described. This theoretical framework and the lessons learned from its implementation offer conceptual scaffolding for future authors of ITSs. [ABSTRACT FROM AUTHOR]

The goal of this research was the development of a practical architecture for the computer-based tutoring of teams. This article examines the relationship of team behaviors as antecedents to successful team performance and learning during adaptive instruction guided by Intelligent Tutoring Systems (ITSs). Adaptive instruction is a training or educational experience tailored by artificially-intelligent, computer-based tutors with the goal of optimizing learner outcomes (e.g., knowledge and skill acquisition, performance, enhanced retention, accelerated learning, or transfer of skills from instructional environments to work environments). The core contribution of this research was the identification of behavioral markers associated with the antecedents of team performance and learning thus enabling the development and refinement of teamwork models in ITS architectures. Teamwork focuses on the coordination, cooperation, and communication among individuals to achieve a shared goal. For ITSs to optimally tailor team instruction, tutors must have key insights about both the team and the learners on that team. To aid the modeling of teams, we examined the literature to evaluate the relationship of teamwork behaviors (e.g., communication, cooperation, coordination, cognition, leadership/coaching, and conflict) with team outcomes (learning, performance, satisfaction, and viability) as part of a large-scale meta-analysis of the ITS, team training, and team performance literature. While ITSs have been used infrequently to instruct teams, the goal of this meta-analysis make team tutoring more ubiquitous by: identifying significant relationships between team behaviors and effective performance and learning outcomes; developing instructional guidelines for team tutoring based on these relationships; and applying these team tutoring guidelines to the Generalized Intelligent Framework for Tutoring (GIFT), an open source architecture for authoring, delivering, managing, and evaluating adaptive instructional tools and methods. In doing this, we have designed a domain-independent framework for the adaptive instruction of teams. [ABSTRACT FROM AUTHOR]

The second meeting of the International Collaboration for Automation of Systematic Reviews (ICASR) was held 3-4 October 2016 in Philadelphia, Pennsylvania, USA. ICASR is an interdisciplinary group whose aim is to maximize the use of technology for conducting rapid, accurate, and efficient systematic reviews of scientific evidence. Having automated tools for systematic review should enable more transparent and timely review, maximizing the potential for identifying and translating research findings to practical application. The meeting brought together multiple stakeholder groups including users of summarized research, methodologists who explore production processes and systematic review quality, and technologists such as software developers, statisticians, and vendors. This diversity of participants was intended to ensure effective communication with numerous stakeholders about progress toward automation of systematic reviews and stimulate discussion about potential solutions to identified challenges. The meeting highlighted challenges, both simple and complex, and raised awareness among participants about ongoing efforts by various stakeholders. An outcome of this forum was to identify several short-term projects that participants felt would advance the automation of tasks in the systematic review workflow including (1) fostering better understanding about available tools, (2) developing validated datasets for testing new tools, (3) determining a standard method to facilitate interoperability of tools such as through an application programming interface or API, and (4) establishing criteria to evaluate the quality of tools' output. ICASR 2016 provided a beneficial forum to foster focused discussion about tool development and resources and reconfirm ICASR members' commitment toward systematic reviews' automation. [ABSTRACT FROM AUTHOR]

Students learn when they connect new information to existing understanding or when they modify existing understanding to accept new information. Most current teaching methods focus on trying to get students to solve problems in a manner identical to that of an expert. This study investigates the effectiveness of assessing student understanding related to context specific problem solving decisions, prescribing feedback based on the assessment, and improving student understanding to the point where they can make correct decisions. Students were given a refrigeration problem unlike their prior problems and were asked to draw the cycle on a T-v diagram using a tutor system. Every group tested (a total of 373 students) showed a significant improvement in their understanding (p<0.0001, Cohen's d>0.8) using a single 40 minute tutor activity. [ABSTRACT FROM AUTHOR]

Objective: Factors influencing long-term viability of an intermediated regional food supply network (food hub) were modeled using agent-based modeling techniques informed by interview data gathered from food hub participants.Background: Previous analyses of food hub dynamics focused primarily on financial drivers rather than social factors and have not used mathematical models.Method: Based on qualitative and quantitative data gathered from 22 customers and 11 vendors at a midwestern food hub, an agent-based model (ABM) was created with distinct consumer personas characterizing the range of consumer priorities. A comparison study determined if the ABM behaved differently than a model based on traditional economic assumptions. Further simulation studies assessed the effect of changes in parameters, such as producer reliability and the consumer profiles, on long-term food hub sustainability.Results: The persona-based ABM model produced different and more resilient results than the more traditional way of modeling consumers. Reduced producer reliability significantly reduced trade; in some instances, a modest reduction in reliability threatened the sustainability of the system. Finally, a modest increase in price-driven consumers at the outset of the simulation quickly resulted in those consumers becoming a majority of the overall customer base.Conclusion: Results suggest that social factors, such as desire to support the community, can be more important than financial factors.Application: An ABM of food hub dynamics, based on human factors data gathered from the field, can be a useful tool for policy decisions. Similar approaches can be used for modeling customer dynamics with other sustainable organizations. [ABSTRACT FROM AUTHOR]

In training assembly workers in a factory, there are often barriers such as cost and lost productivity due to shutdown. The use of virtual reality (VR) training has the potential to reduce these costs. This research compares virtual bimanual haptic training versus traditional physical training and the effectiveness for learning transfer. In a mixed experimental design, participants were assigned to either virtual or physical training and trained by assembling a wooden burr puzzle as many times as possible during a twenty minute time period. After training, participants were tested using the physical puzzle and were retested again after two weeks. All participants were trained using brightly colored puzzle pieces. To examine the effect of color, testing involved the assembly of colored physical parts and natural wood colored physical pieces. Spatial ability as measured using a mental rotation test, was shown to correlate with the number of assemblies they were able to complete in the training. While physical training outperformed virtual training, after two weeks the virtually trained participants actually improved their test assembly times. The results suggest that the color of the puzzle pieces helped the virtually trained participants in remembering the assembly process. [ABSTRACT FROM AUTHOR]

Intelligent Tutoring Systems (ITSs) that employ a model-tracing methodology have consistently shown their effectiveness. However, what evidently makes these tutors effective, the cognitive model embedded within them, has traditionally been difficult to create, requiring great expertise and time, both of which come at a cost. Furthermore, an interface has to be constructed that communicates with the cognitive model. Together these constitute a high bar that needs to be crossed in order to create such a tutor. We outline a system that lowers this bar on both accounts and that has been used to produce commercial-quality tutors. First, we discuss and evaluate a tool that allows authors who are not cognitive scientists or programmers to create a cognitive model. Second, we detail a way for this cognitive model to communicate with third-party interfaces. [ABSTRACT FROM AUTHOR]

Background: Although many aspects of systematic reviews use computational tools, systematic reviewers have been reluctant to adopt machine learning tools. Discussion: We discuss that the potential reason for the slow adoption of machine learning tools into systematic reviews is multifactorial. We focus on the current absence of trust in automation and set-up challenges as major barriers to adoption. It is important that reviews produced using automation tools are considered non-inferior or superior to current practice. However, this standard will likely not be sufficient to lead to widespread adoption. As with many technologies, it is important that reviewers see "others" in the review community using automation tools. Adoption will also be slow if the automation tools are not compatible with workflows and tasks currently used to produce reviews. Many automation tools being developed for systematic reviews mimic classification problems. Therefore, the evidence that these automation tools are non-inferior or superior can be presented using methods similar to diagnostic test evaluations, i.e., precision and recall compared to a human reviewer. However, the assessment of automation tools does present unique challenges for investigators and systematic reviewers, including the need to clarify which metrics are of interest to the systematic review community and the unique documentation challenges for reproducible software experiments. Conclusion: We discuss adoption barriers with the goal of providing tool developers with guidance as to how to design and report such evaluations and for end users to assess their validity. Further, we discuss approaches to formatting and announcing publicly available datasets suitable for assessment of automation technologies and tools. Making these resources available will increase trust that tools are non-inferior or superior to current practice. Finally, we identify that, even with evidence that automation tools are non-inferior or superior to current practice, substantial set-up challenges remain for main stream integration of automation into the systematic review process. [ABSTRACT FROM AUTHOR]

The third meeting of the International Collaboration for Automation of Systematic Reviews (ICASR) was held 17–18 October 2017 in London, England. ICASR is an interdisciplinary group whose goal is to maximize the use of technology for conducting rapid, accurate, and efficient systematic reviews of scientific evidence. The group seeks to facilitate the development and widespread acceptance of automated techniques for systematic reviews. The meeting's conclusion was that the most pressing needs at present are to develop approaches for validating currently available tools and to provide increased access to curated corpora that can be used for validation. To that end, ICASR's short-term goals in 2018–2019 are to propose and publish protocols for key tasks in systematic reviews and to develop an approach for sharing curated corpora for validating the automation of the key tasks. [ABSTRACT FROM AUTHOR]