Search

Research in multi-agent teaming has increased substantially over recent years. Underneath these attempts sits a suite of communication functions to enable effective teaming. The Artificial Intelligence (AI) systems supporting the teaming arrangement have primarily relied on knowledge-based systems, with rules triggered based on the mode of interaction. Enabling humans to join the team of AI agents effectively calls for both the humans and AI agents to share their understanding and representation of their shared worlds. Such shared understanding requires formal representations of concepts to support transparency during bi-directional communications between team members. Little research has been done in this space, especially when humans need to team with a swarm of agents. We present an ontology designed specifically for human-agent teaming to address this research gap. The ontology is general, but we then contextualise it into a particular swarm-shepherding scenario to illustrate its use in a particular context. The proposed Ontology for Generalised Multi-Agent Teaming Onto4MAT offers the underlying building blocks for effective communication and shared understanding between humans and multi-agent teams.

Robot swarms are becoming popular in domains that require spatial coordination. Effective human control over swarm members is pivotal for ensuring swarm behaviours align with the dynamic needs of the system. Several techniques have been proposed for scalable human–swarm interaction. However, these techniques were mostly developed in simple simulation environments without guidance on how to scale them up to the real world. This paper addresses this research gap by proposing a metaverse for scalable control of robot swarms and an adaptive framework for different levels of autonomy. In the metaverse, the physical/real world of a swarm symbiotically blends with a virtual world formed from digital twins representing each swarm member and logical control agents. The proposed metaverse drastically decreases swarm control complexity due to human reliance on only a few virtual agents, with each agent dynamically actuating on a sub-swarm. The utility of the metaverse is demonstrated by a case study where humans controlled a swarm of uncrewed ground vehicles (UGVs) using gestural communication, and via a single virtual uncrewed aerial vehicle (UAV). The results show that humans could successfully control the swarm under two different levels of autonomy, while task performance increases as autonomy increases.

Bi-directional communication between humans and swarm systems begs for efficient languages to communicate information between the humans and the Artificial Intelligence (AI)-enabled agents in a manner that is most appropriate for the context. We discuss the criteria for effective teaming and functional bi-directional communication between humans and AI, and the design choices required to create effective languages. We then present a human-AI-teaming communication language inspired by the Australian Aboriginal language of Jingulu, which we call JSwarm. We present the motivation and structure of the language. An example is used to demonstrate how the language operates for a shepherding swarm guidance task.

Swarm systems consist of large numbers of agents that collaborate autonomously. With an appropriate level of human control, swarm systems could be applied in a variety of contexts ranging from urban search and rescue situations to cyber defence. However, the successful deployment of the swarm in such applications is conditioned by the effective coupling between human and swarm. While adaptive autonomy promises to provide enhanced performance in human-machine interaction, distinct factors must be considered for its implementation within human-swarm interaction. This paper reviews the multidisciplinary literature on different aspects contributing to the facilitation of adaptive autonomy in human-swarm interaction. Specifically, five aspects that are necessary for an adaptive agent to operate properly are considered and discussed, including mission objectives, interaction, mission complexity, automation levels, and human states. We distill the corresponding indicators in each of the five aspects, and propose a framework, named MICAH (i.e., Mission-Interaction-Complexity-Automation-Human), which maps the primitive state indicators needed for adaptive human-swarm teaming.

Sheepdogs smartly herd a flock of sheep and guide them towards a goal. A single dog can herd a few hundred sheep in easy to navigate environments. Understanding the interaction space between the sheepdogs, sheep and the environment is important due to the possibility of transferring this knowledge to solve practical swarm robotics problems. This interaction space is a complex mesh of influencing factors. We scrutinize this interaction space to identify areas where the complexity of the herding problem changes from low (easy to solve) to high (harder to solve or becoming unsolvable) complexity. In particular, we study reactive models for shepherding, whereby agents respond directly to stimuli in the environments by fusing the set of force vectors influencing their behaviour. We present an enhanced shepherding model with higher success rate than its predecessor. We investigate four key factors that influence the complexity of the problem: the relative speed between the sheepdog and sheep, the spatial configuration of the sheep at the start of the task, the number of sheepdogs, and the density of obstacles in the environment. We discovered a phase transition in shepherding resulting from the interaction between the number of sheepdogs and obstacles. The phase transition occurs as the density of obstacles range from 0.2% for a single shepherding agent to 5% for 10 shepherding agents. During this phase transition, the problem changes from being an easy problem where the flock gets collected quickly, to a hard one where the overall herding task becomes utterly not achievable using reactive approaches.

Autonomous vehicles are becoming a reality in places with advanced infrastructure to support their operations. In crowded places, harsh environments, missions that require these vehicles to be aware of the context in which they are operating, and situations requiring continuous coordination with humans such as in disaster relief, advanced-vehicle systems (AVSs) need to be better contextually aware. The vast literature referring to “context-aware systems” is still sparse, focusing on very limited forms of contextual awareness. It requires a structured approach to bring it together to truly realize contextual awareness in AVSs. This paper uses a human-AVSs (HAVSs) lens to polarize the literature in a coherent form suitable for designing distributed HAVSs. We group the relevant literature into two categories: contextual-awareness related to the vehicle infrastructure itself that enables AVSs to operate, and contextual-awareness related to HAVSs. The former category focuses on the communication backbone for AVSs including ad-hoc networks, services, wireless communication, radio systems, and the cyber security and privacy challenges that arise in these contexts. The latter category covers recommender systems, which are used to coordinate the actions that sit at the interface of the human and AVSs, human-machine interaction issues, and the activity recognition systems as the enabling technology for recommender systems to operate autonomously. The structured analysis of the literature has identified a number of open research questions and opportunities for further research in this area.

Stories are useful tools with which we can exchange experience learnt in social contexts, ways to communicate futures in strategic planning, and unique building blocks that connect meanings in a movie or a virtual environment. Evolutionary computation (EC) techniques have the potential to overcome existing limitations in automated storytelling, whereby evolution can provide a process of innovation. However, one source of complexity lies in the transformation of a story in a natural language into a representation that EC can evolve easily. Another complexity arises from the fact that the ultimate judge for the quality of a story is a human being, and humans are diverse in their taste. This paper attempts to tackle the above complexities through an automatic story narration application. We present a methodology which can transform a story written in English into an event-level and hierarchical-level grammar using a network representation. This approach makes it possible to devise an encoding scheme that translates a story narration with flashback into a chromosome and vice versa. We then discuss different metrics for the evolutionary narration problem and use 42 human participants to evaluate the generated narrations. To incorporate diversified human opinions, we propose to build individual human-surrogate models from the human-evaluation experiment and further fuse them into an ensemble. The ensembles of human surrogate models serve as the objective functions of multi-objective EC to guide the generation of desirable stories from human perspectives. We demonstrate that this approach is successful in evolving better narrations as assessed by 31 human participants.

Despite the advances made in artificial intelligence, software agents, and robotics, there is little we see today that we can truly call a fully autonomous system. We conjecture that the main inhibitor for advancing autonomy is lack of trust. Trusted autonomy is the scientific and engineering field to establish the foundations and ground work for developing trusted autonomous systems (robotics and software agents) that can be used in our daily life, and can be integrated with humans seamlessly, naturally, and efficiently. In this paper, we review this literature to reveal opportunities for researchers and practitioners to work on topics that can create a leap forward in advancing the field of trusted autonomy. We focus this paper on the trust component as the uniting technology between humans and machines. Our inquiry into this topic revolves around three subtopics: (1) reviewing and positioning the trust modeling literature for the purpose of trusted autonomy; (2) reviewing a critical subset of sensor technologies that allow a machine to sense human states; and (3) distilling some critical questions for advancing the field of trusted autonomy. The inquiry is augmented with conceptual models that we propose along the way by recompiling and reshaping the literature into forms that enable trusted autonomous systems to become a reality. This paper offers a vision for a Trusted Cyborg Swarm, an extension of our previous Cognitive Cyber Symbiosis concept, whereby humans and machines meld together in a harmonious, seamless, and coordinated manner.

Computer-based swarm systems, aiming to replicate the flocking behavior of birds, were first introduced by Reynolds in 1987. In his initial work, Reynolds noted that while it was difficult to quantify the dynamics of the behavior from the model, observers of his model immediately recognized them as a representation of a natural flock. Considerable analysis has been conducted since then on quantifying the dynamics of flocking/swarming behavior. However, no systematic analysis has been conducted on human identification of swarming. In this paper, we assess subjects’ assessment of the behavior of a simplified version of Reynolds’ model. Factors that affect the identification of swarming are discussed and future applications of the resulting models are proposed. Differences in decision times for swarming-related questions asked during the study indicate that different brain mechanisms may be involved in different elements of the behavior assessment task. The relatively simple but finely tunable model used in this study provides a useful methodology for assessing individual human judgment of swarming behavior.