Search

Onboard far-field aircraft detection is needed for safe non-cooperative traffic mitigation in autonomous small unmanned aerial system (sUAS) operations. Machine vision systems, based on standard op...

Rocket payload and fairing size have placed strict mass and volume limitations on single launch in-space structures. These limitations are what motivates In-Space Assembly (ISA)and where Inertial Transfer has potential to improve or augment ISA capabilities by utilizing an efficient multi-agent autonomous system to transport untethered payloads. Adoption of an inertial transfer approach significantly reduces system complexity, energy and time needed for assembly compared to conventional ISA capabilities, increasing the likelihood of early adoption for assembly. The Inertial Transfer concept is presented including autonomy capabilities, potential risks, and three system configurations. These configurations are based on the autonomous system’s available sensor coverage that tracks the payload’s state. Participating agents have well-defined roles, expectations, and assumptions in their physical organization and coordination of actions depending the mode of operation. The sensor information available to participating agents is considered as it affects the uncertainty in the estimated state and theactions required to capture the payload.

As the number of viable applications for unmanned aerial vehicle (UAV) systems increases at an exponential rate, interfaces that reduce the reliance on highly skilled engineers and pilots must be developed. Recent work aims to make use of common human communication modalities such as speech and gesture. This paper explores a multimodal natural language interface that uses a combination of speech and gesture input modalities to build complex UAV flight paths by defining trajectory segment primitives. Gesture inputs are used to define the general shape of a segment while speech inputs provide additional geometric information needed to fully characterize a trajectory segment. A user study is conducted in order to evaluate the efficacy of the multimodal interface.

Space operations are on the cusp of a revolutionary new operational paradigm that leverages modular systems and recurring robotic visits to “Persistent Assets” enabling asset maintenance, repair, and enhancement. A “Persistent Asset” is defined here as any zero-g or planetary surface system that benefits from in-space assembly (ISA) or multiple visits for servicing, repairs, and upgrades. This term is an extension of the term “Persistent Platform” used by Ms. Pam Melroy at the Defense Advanced Research Projects Agency to describe the vision of the Agency for a geosynchronous Earth orbiting platform. In this paper, the term “Persistent Asset” is introduced to encompass not only zero-g systems; such as telecommunication platforms, Earth observing science platform, Department of Defense platforms, and scientific telescope systems, but also planetary surface systems that support missions such as human outposts, science stations, and in-situ resource utilization systems. In contrast to the current state of the art, where space systems are typically launched as a single unit and operated without any further physical intervention after launch; future systems will be maintained, enhanced and reconfigured in-situ as new technology becomes available or mission needs change. Visits to the persistent asset can be regularly scheduled or dictated by funding constraints enabling a pay-as-you-go approach, which is largely independent of time-constraints and able to exploit launches of opportunity. In this paper, historical in-space assembly activities which relied heavily on astronaut extra-vehicular activity will be reviewed as well as early robotic assembly activities. These approaches will be contrasted with emerging modular approaches supporting realization of a new Persistent Asset operational paradigm. The paper will define attributes of the Persistent Asset paradigm and illustrate advantages by applying the paradigm to two relevant applications: 1) a large space telescope backing structure and 2) the backbone structure for a solar electric transport vehicle. Finally, recently developed unique Persistent Asset elements (modules, and interface approaches) will be described.

Current rapidly-exploring random tree (RRT) algorithms rely on proximity query packages that often include collision checkers, tolerance verification, and distance computation algorithms for the generation of safe paths. In this paper, we broaden the information available to the path-planning algorithm by incorporating silhouette information of nearby obstacles in conflict. A silhouette-informed tree (SIT) is generated through the flight-safe region of a wire maze for a single unmanned aerial system (UAS). The silhouette is used to extract local geometric information of nearby obstacles and provide path alternatives around these obstacles. Thus, focusing the search for the generation of new tree branches near these obstacles, and decreasing the number of samples required to explore the narrow corridors within the wire maze. The SIT is then processed to extract a path that connects the initial location of the UAS with the goal, reduce the number of line segments in this path if possible, and smooth the resulting path using Pythagorean Hodograph Bezier curves. To ensure that the smoothed path remains in the flight-safe region of the configuration space, a tolerance verification algorithm for Bezier curves and convex polytopes in three dimensions is proposed. Lastly, temporal specifications are imposed on the smoothed path in the shape of an arbitrary speed profile.

As part of swarm search and service (SSS) missions, swarms are tasked with searching an area while simultaneously servicing jobs as they are encountered. Jobs must be immediately serviced and can be one of multiple types. Each type requires that vehicle(s) break off from the swarm and travel to the job site for a specified amount of time. The number of vehicles needed and the service time for each job type are known. Once a job has been successfully serviced, vehicles return to the swarm and are available for reallocation. When planning SSS missions, human operators are tasked with determining the required number of vehicles needed to handle the expected job demand. The complex relationship between job type parameters makes this choice challenging. This work presents a prediction model used to estimate the swarm size necessary to achieve a given performance. User studies were conducted to determine the usefulness and ease of use of such a prediction model as an aid during mission planning. Results show that using the planning tool leads to 7 times less missed area and a 50 percent cost reduction.

Autonomous systems governed by a variety of adaptive and nondeterministic algorithms are being planned for inclusion into safety-critical environments, such as unmanned aircraft and space systems in both civilian and military applications. However, until autonomous systems are proven and perceived to be capable and resilient in the face of unanticipated conditions, humans will be reluctant or unable to delegate authority, remaining in control aided by machine-based information and decision support. Proving capability, or trustworthiness, is a necessary component of certification. Perceived capability is a component of trust. Trustworthiness is an attribute of a cyber-physical system that requires context-driven metrics to prove and certify. Trust is an attribute of the agents participating in the system and is gained over time and multiple interactions through trustworthy behavior and transparency. Historically, artificial intelligence and machine learning systems provide answers without explanation - without a rationale or insight into the machine “thinking”. In order to function as trusted teammates, machines must be able to explain their decisions and actions. This transparency is a product of both content and communication. NASA’s Autonomy Teaming & TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR) project seeks to build a basis for certification of autonomous systems via establishing metrics for trustworthiness and trust in multi-agent team interactions, using AI (Artificial Intelligence) explainability and persistent modeling and simulation, in the context of mission planning and execution, with analyzable trajectories. Inspired by Massively Multiplayer Online Role Playing Games (MMORPG) and Serious Gaming, the proposed ATTRACTOR modeling and simulation environment is similar to online gaming environments in which player (aka agent) participants interact with each other, affect their environment, and expect the simulation to persist and change regardless of any individual agent’s active participation. This persistent simulation environment will accommodate individual agents, groups of self-organizing agents, and large-scale infrastructure behavior. The effects of the emerging adaptation and coevolution can be observed and measured to building a basis of measurable trustworthiness and trust, toward certification of safety-critical autonomous systems.

The age of Autonomous Unmanned Aircraft Systems (AUAS) is creating new challenges for the accreditation and certification requiring new standards, policies and procedures that sanction whether a UAS is safe to fly. Establishing a basis for certification of autonomous systems via research into trust and trustworthiness is the focus of Autonomy Teaming and TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR), a new NASA Convergent Aeronautics Solution (CAS) project. Simulation Environments to test and evaluate AUAS decision making may be a low-cost solution to help certify that various AUAS systems are trustworthy enough to be allowed to fly in current general and commercial aviation airspace. NASA is working to build a peer-to-peer persistent simulation (P3 Sim) environment. The P3 Sim will be a Massively Multiplayer Online (MMO) environment were AUAS avatars can interact with a complex dynamic environment and each other. The focus of the effort is to provide AUAS researchers a low-cost intuitive testing environment that will aid training for and assessment of decisions made by autonomous systems such as AUAS. This presentation focuses on the design approach and challenges faced in development of the P3 Sim Environment is support of investigating trustworthiness of autonomous systems.

Natural language interfaces are becoming more ubiquitous. By allowing for more natural communication, reducing the complexity of interacting with machines, and enabling non-expert users, these interfaces have found homes in numerous common products. However, these natural language interfaces still have great room for growth and development in order to better reflect human speech patterns. Intuitive speech communication is often accompanied by gestural information that is currently lacking from most speech interfaces. Exclusion of gestural data reduces a machine’s ability to interpret deictic information and understand some semantic intent. To allow for truly intuitive communication between humans and machines, a natural language interface must understand not only speech but also gestural data. This paper will outline the limitations and restrictions of some of the most popular and common speech-only natural language processing algorithms and systems in use today. Focus will be given to extra-linguistic communication aspects, including gestural information. Current research trends will then be presented that have been designed to compensate for these gaps by incorporating extra-linguistic information. The success of each of these trends will then be evaluated, as well as the hopefulness of continued investigative efforts. Additionally, a model multimodal interface will be presented that incorporates language and gesture data in order to demonstrate the effectiveness of such an interface. The gestural portion of this interface is included to compensate for some of the limitations of speech-only natural language interfaces. Combining these two types of natural language interfaces thereby works to reduce the limitations of natural language interfaces and increase their success. This presentation will discuss how the two interfaces work together and will specify how the speech interface limitations are addressed through the inclusion of a gestural system.

As the use of autonomous systems continues to proliferate, their user base has transitioned from one primarily comprised of pilots and engineers with knowledge of the low level systems and algorithms to non-expert UAV users like scientists. This shift has highlighted the need to develop more intuitive and easy-to-use interfaces such that the strengths of the autonomous system can still be utilized without requiring any prior knowledge about the complexities of running such a system. Gesture-based natural language interfaces have emerged as a promising new alternative input modality. While on their own gesture-based interfaces can build general descriptions of desired inputs (e.g., flight path shapes), it is difficult to define more specific information (e.g., lengths, radii, height) while simultaneously preserving the intuitiveness of the interface. In order to assuage this issue, multimodal interfaces that integrate both gesture and speech can be used. These interfaces are intended to model typical human-human communication patterns which supplement gestures with speech. However, challenges arise when integrating gestures into a multimodal HMI architecture such as user perception of their ability vs. actual performance, system feedback, synchronization between input modalities, and the bounds on gesture execution requirements. We discuss these challenges, their possible causes and provide suggestions for mitigating these issues in the design of future multimodal interfaces. Although this paper discusses these challenges in the context of unmanned aerial vehicle mission planning, similar issues and solutions can be extended to unmanned ground and underwater missions.

Time-critical coordination tools for unmanned systems can be employed to enforce the type of temporal constraints required in terminal control areas, ensure minimum distance requirements among vehicles are satisfied, and successfully perform coordinated missions. In comparison with previous literature, this paper presents an ampler spectrum of coordination and temporal specifications for unmanned systems, and proposes a general control law that can enforce this range of constraints. The constraint classification presented con- siders the nature of the desired arrival window and the permissible coordination errors to define six different types of time-coordination strategies. The resulting decentralized coordination control law allows the vehicles to negotiate their speeds along their paths in response to information exchanged over the communication network. This control law organizes the different members in the fleet hierarchically per their behavior and informational needs as reference agent, leaders, and followers. Examples and simulation results for all the coordination strategies presented demonstrate the applicability and efficacy of the coordination control law for multiple unmanned systems.

Traditionally, flight paths for unmanned aerial vehicles (UAVs) are generated offline by trained pilots and engineers using assumed environmental conditions, terrain and obstacles as constraints. As new applications for UAVs emerge, their user base shifts from one of operators with knowledge of low level systems to that of non-experts. These new operators require a more intuitive method for building desired UAV flight paths such that they can leverage the full capabilities of the vehicle without needing to understand its system complexities. We present a gesture-based natural language interface for defining trajectory segments using a library of twelve simple hand gestures. A user study is presented to analyze the effectiveness, ease-of-use and accuracy of the gesture-based interface as compared to a baseline mouse interface. We explore differences seen between subjects given their hand dominance, their prior UAV flight experience (or lack thereof), and whether they chose to sit or stand while using the gesture interface. Given limited training time, subjects were able to accurately define an average of 74.36% of trajectory segments. Overall the user study highlights the favorable potential for the use of the gesture-based interface as an alternative input modality, as well as, feedback for future interface improvements and training methods.

In recent years, natural language machine interfaces have become increasingly common. These interfaces allow for more intuitive communication with machines, reducing the complexity of interacting with these systems and enabling their use by non-expert users. Most of these natural language interfaces rely on speech, including such well-known devices as the iPhone's Siri application, Cortana, Amazon's Alexa and Echo devices, and others. Given their intuitive functionality, natural language interfaces have also been investigated as a method for controlling unmanned aerial vehicles (UAVs), allowing non-subject matter experts to use these tools in their scientific pursuits. This paper examines a speech-based natural language interface for defining UAV trajectories. To determine the efficacy of this interface, a user study is also presented that examines how users perform with this interface compared to a traditional mouse-based interface. The results of the user study are described in order to show how accurately users were able to define trajectories as well as user preference for using the speech-based system both before and after participating in the user study. Additional data are presented on whether users had previous experience with speech-based interfaces and how long they spent training with the interface before participating in the study. The user study demonstrates the potential of speech- based interfaces for UAV trajectory generation and suggests methods for future improvement and incorporation of natural language interfaces for UAV pilots.

The popularity of unmanned aerial vehicles (UAVs) is increasing as technological advancements boost their favorability for a broad range of applications. One application is science data collection. In fields like earth and atmospheric science, researchers are seeking to use UAVs to augment their current portfolio of platforms and increase their accessibility to geographic areas of interest. By increasing the number of data collection platforms, UAVs will significantly improve system robustness and allow for more sophisticated studies. Scientists would like the ability to deploy an available fleet of UAVs to traverse a desired flight path and collect sensor data without needing to understand the complex low-level controls required to describe and coordinate such a mission. A natural interaction interface for a Ground Control System (GCS) using gesture recognition is developed to allow non-expert users (e.g., scientists) to define a complex flight path for a UAV using intuitive hand gesture inputs from the constructed gesture library. The GCS calculates the combined trajectory on-line, verifies the trajectory with the user, and sends it to the UAV controller to be flown.

A greater need for sophisticated autonomous piloting systems has risen in direct correlation with the ubiquity of Unmanned Aerial Vehicle (UAV) technology. Whether surveying unknown or unexplored areas of the world, collecting scientific data from regions in which humans are typically incapable of entering, locating lost or wanted persons, or delivering emergency supplies, an unmanned vehicle moving in close proximity to people and other vehicles, should fly smoothly and predictably. The mathematical application of spline interpolation can play an important role in autopilots' on-board trajectory planning. Spline interpolation allows for the connection of Three-Dimensional Euclidean Space coordinates through a continuous set of smooth curves. This paper explores the motivation, application, and methodology used to compute the spline control points, which shape the curves in such a way that the autopilot trajectory is able to meet vehicle-dynamics limitations. The spline algorithms developed used to generate these curves supply autopilots with the information necessary to compute vehicle paths through a set of coordinate waypoints.

Incremental approaches to air transportation system development inherit current architectural constraints, which, in turn, place hard bounds on system capacity, efficiency of performance, and complexity. To enable airspace operations of the future, a clean-slate (ab initio) airspace design(s) must be considered. This ab initio National Airspace System (NAS) must be capable of accommodating increased traffic density, a broader diversity of aircraft, and on-demand mobility. System and subsystem designs should scale to accommodate the inevitable demand for airspace services that include large numbers of autonomous Unmanned Aerial Vehicles and a paradigm shift in general aviation (e.g., personal air vehicles) in addition to more traditional aerial vehicles such as commercial jetliners and weather balloons. The complex and adaptive nature of ab initio designs for the future NAS requires new approaches to validation, adding a significant physical experimentation component to analytical and simulation tools. In addition to software modeling and simulation, the ability to exercise system solutions in a flight environment will be an essential aspect of validation. The NASA Langley Research Center (LaRC) Autonomy Incubator seeks to develop a flight simulation infrastructure for ab initio modeling and simulation that assumes no specific NAS architecture and models vehicle-to-vehicle behavior to examine interactions and emergent behaviors among hundreds of intelligent aerial agents exhibiting collaborative, cooperative, coordinative, selfish, and malicious behaviors. The air transportation system of the future will be a complex adaptive system (CAS) characterized by complex and sometimes unpredictable (or unpredicted) behaviors that result from temporal and spatial interactions among large numbers of participants. A CAS not only evolves with a changing environment and adapts to it, it is closely coupled to all systems that constitute the environment. Thus, the ecosystem that contains the system and other systems evolves with the CAS as well. The effects of the emerging adaptation and co-evolution are difficult to capture with only combined mathematical and computational experimentation. Therefore, an ab initio flight simulation environment must accommodate individual vehicles, groups of self-organizing vehicles, and large-scale infrastructure behavior. Inspired by Massively Multiplayer Online Role Playing Games (MMORPG) and Serious Gaming, the proposed ab initio simulation environment is similar to online gaming environments in which player participants interact with each other, affect their environment, and expect the simulation to persist and change regardless of any individual player's active participation.

Software tools are being developed by the Autonomy Incubator at NASA's Langley Research Center that will provide an integrated and scalable capability to support research and non-research flight operations across several flight domains, including urban and mixed indoor-outdoor operations. These tools incorporate a full range of data products to support mission planning, approval, flight operations, and post-flight review. The system can support a number of different operational scenarios that can incorporate live and archived data streams for UAS operators, airspace regulators, and other important stakeholders. Example use cases are described that illustrate how the tools will benefit a variety of users in nominal and off-nominal operational scenarios. An overview is presented for the current state of the toolset, including a summary of current demonstrations that have been completed. Details of the final, fully operational capability are also presented, including the interfaces that will be supported to ensure compliance with existing and future airspace operations environments.

With the anticipated increase of small unmanned aircraft systems (sUAS) entering into the National Airspace System, it is highly likely that vehicle operators will be teaming with fleets of small autonomous vehicles. The small vehicles may consist of sUAS, which are 55 pounds or less that typically will y at altitudes 400 feet and below, and small ground vehicles typically operating in buildings or defined small campuses. Typically, the vehicle operators are not concerned with manual control of the vehicle; instead they are concerned with the overall mission. In order for this vision of high-level mission operators working with fleets of vehicles to come to fruition, many human factors related challenges must be investigated and solved. First, the interface between the human operator and the autonomous agent must be at a level that the operator needs and the agents can understand. This paper details the natural language human factors e orts that NASA Langley's Autonomy Incubator is focusing on. In particular these e orts focus on allowing the operator to interact with the system using speech and gestures rather than a mouse and keyboard. With this ability of the system to understand both speech and gestures, operators not familiar with the vehicle dynamics will be able to easily plan, initiate, and change missions using a language familiar to them rather than having to learn and converse in the vehicle's language. This will foster better teaming between the operator and the autonomous agent which will help lower workload, increase situation awareness, and improve performance of the system as a whole.

The Earth System Science Pathfinder (ESSP) Program Office (PO) is responsible for programmatic management of National Aeronautics and Space Administration's (NASA) Science Mission Directorate's (SMD) Earth Venture (EV) missions. EV is composed of both orbital and suborbital Earth science missions. The first of the Earth Venture missions is EV-1, which are Principal Investigator-led, temporally-sustained, suborbital (airborne) science investigations costcapped at $30M each over five years. Traditional orbital procedures, processes and standards used to manage previous ESSP missions, while effective, are disproportionally comprehensive for suborbital missions. Conversely, existing airborne practices are primarily intended for smaller, temporally shorter investigations, and traditionally managed directly by a program scientist as opposed to a program office such as ESSP. In 2010, ESSP crafted a management approach for the successful implementation of the EV-1 missions within the constructs of current governance models. NASA Research and Technology Program and Project Management Requirements form the foundation of the approach for EV-1. Additionally, requirements from other existing NASA Procedural Requirements (NPRs), systems engineering guidance and management handbooks were adapted to manage programmatic, technical, schedule, cost elements and risk. As the EV-1 missions are nearly at the end of their successful execution and project lifecycle and the submission deadline of the next mission proposals near, the ESSP PO is taking the lessons learned and updated the programmatic management approach for all future Earth Venture Suborbital (EVS) missions for an even more flexible and streamlined management approach.

On 27 May 2010, NASA announced the first Earth Venture (EV-1) selections in response to a recommendation made by the National Research Council for low-cost investigations fostering innovation in Earth Science. The five EV-1 investigations span the Earth science focus areas of atmosphere, weather, climate, water and energy and, carbon and represent Earth science researchers from NASA as well as other government agencies, academia and industry from around the world. The five EV-1 missions are (1) Airborne Microwave Observatory of Subcanopy and Subsurface (AirMOSS); (2) Airborne Tropical TRopopause Experiment (ATTREX); (3) Carbon in Arctic Reservoirs Vulnerability Experiment (CARVE); (4) Deriving Information on Surface Conditions from Column and VERtically Resolved Observations Relevant to Air Quality (DISCOVER-AQ), and (5) Hurricane and Severe Storm Sentinel (HS3). The Earth Venture missions are managed out of the Earth System Science Pathfinder (ESSP) Program Office [3]

The Earth System Science Pathfinder (ESSP) Program Office (PO) is responsible for programmatic management of National Aeronautics and Space Administration (NASA) Science Mission Directorate's (SMD) Earth Venture (EV) missions. EV is composed of both orbital and suborbital Earth science missions. The first of the Earth Venture missions is EV-1, which are Principal Investigator-led, temporally-sustained, suborbital (airborne) science investigations cost-capped at $30M each over five years. Traditional orbital procedures, processes and standards used to manage previous ESSP missions, while effective, are disproportionally comprehensive for suborbital missions. Conversely, existing airborne practices are primarily intended for smaller, temporally shorter investigations, and traditionally managed directly by a program scientist as opposed to a program office such as ESSP. ESSP has crafted a management approach for the successful implementation of the EV-1 missions within the constructs of current governance models. NASA Research and Technology Program and Project Management Requirements form the foundation of the approach for EV-1. Additionally, requirements from other existing NASA Procedural Requirements (NPRs), systems engineering guidance and management handbooks were adapted to manage programmatic, technical, schedule, cost elements and risk. The program management approach presented here for EV-1 will set the precedent for future suborbital EV missions.

We introduce a general method for evaluating and comparing the expected performance of sensing systems for interactive computer graphics. Example applications include head tracking systems for virtual environments, motion capture systems for movies, and even multi-camera 3D vision systems for image-based visual hulls.Our approach is to estimate the asymptotic position and/or orientation uncertainty at many points throughout the desired working volume, and to visualize the results graphically. This global performance estimation can provide both a quantitative assessment of the expected performance, and intuition about the type and arrangement of sources and sensors, in the context of the desired working volume and expected scene dynamics.