Search

Unmanned airspace access is a key topic to enable unmanned flights in the U.S. National Airspace System. Above the 400-foot AGL limit, unmanned operations need special care and systems to fly safely and reliably. For autonomous flights to be safe, regular parts of daily, economical operations, designs for both the greater airspace and specific avionics must be built on robust systems design principles. For unmanned systems with a predefined flight plan, it is possible to autonomously and safely fly above 400 feet AGL in manned airspace by interfacing with ADS-B out and under the control of a licensed pilot. Utah State University's AggieAir group has been flying autonomously in this manner for real remote sensing missions, and in this paper, design methodologies for aircraft/avionics systems, ground support equipment, as well as flight logistics and airspace data interfaces are described. Lastly, flight data under FAA Certificates of Authorization (COAs) above 400 feet AGL demonstrate AggieAir's safe operations in manned airspace.

NASA's Unmanned Aircraft Systems Traffic Management research aims to develop policies, procedures, requirements, and other artifacts to inform the implementation of a future system that enables small drones to access the low altitude airspace. In this endeavor, NASA conducted a geographically diverse flight test in conjunction with the FAA's six unmanned aircraft systems Test Sites. A control center at NASA Ames Research Center autonomously managed the airspace for all participants in eight states as they flew operations (both real and simulated). The system allowed for common situational awareness across all stakeholders, kept traffic procedurally separated, offered messages to inform the participants of activity relevant to their operations. Over the 3-hour test, 102 flight operations connected to the central research platform with 17 different vehicle types and 8 distinct software client implementations while seamlessly interacting with simulated traffic.

Trajectory predictors are core components of many air traffic applications. This includes cockpit, dispatch, flight planning, strategic fleet planning, air traffic control, traffic flow management, and aviation research applications. In all these different approaches, aircraft performance models are often required such as, for instance, drag polar aerodynamic data and maximum thrust propulsion data. For many applications these performance data may be required for a large number of different airframes and propulsion models. The emergence of new and non-traditional vehicle types, such as unmanned aerospace vehicles, further adds to and complicates this effort. Furthermore, the trajectory predictor software itself often is highly complicated. It has its own development and maintenance costs, and inevitably raises issues of portability, standardization and modeling consistency. Finally, two common drawbacks in typical trajectory predictors are that (i) they are deterministic and do not model the many uncertainties that impact flights in the national airspace system, and (ii) they often do not account for air traffic control constraints which flights are subjected to. Here I describe a Probabilistic Trajectory Constraint Modeler (PTM) that addresses these many issues, and stochastically models flight constraints and flight times. These outputs can be used by existing trajectory models or other decision support tools.

According to a 2013 AUVSI report, delays in integrating Unmanned Aerial Systems (UAS) into the National Airspace System (NAS) could cost more than $10 billions a year for the United States alone. Worldwide regulatory bodies are under pressure by the UAS industry to accelerate the regulation process, but safety remains their main objective. One condition for the safe introduction of UAS in the NAS is for them to be equipped with a collision avoidance system. Though the existing Airborne Collision Avoidance System II (ACAS II) could have been an option, the transformations of air traffic management engaged through NextGen (US) and SESAR (Europe) led to the definition of a new ACAS based on new logics, namely ACAS X. Its definition contains in particular two variations : ACAS Xa, for large aircraft, and ACAS Xu, for unmanned aircraft. As noted in a 2014 RTCA annual report, divide in technological knowledge between those experienced in ACAS II and those involved in the development of ACAS X is a concern. To help preventing this divide we believe it is essential to keep the community updated with the latest evolutions of the ACAS X standards. As work on Minimum Operational Performance Standards (MOPS) for ACAS Xu just started, it is of interest to know which parts of the MOPS are already decided, which remain flexible for the industries to make the difference and which are open research problems. Being a member of the ACAS X family, ACAS Xu lays on the same foundations as the well defined ACAS Xa standard. This work proposes an introduction to the ACAS Xa/Xu common basis, as it is unlikely to change, including the general architecture and Collision Avoidance (CA) logics. It is followed by a presentation of concepts specific to ACAS Xu such as the tailored threat logic, horizontal CA logic, CA coordination and automatic responses. For the flexible part, we believe it mainly concerns the surveillance sources. Instead of a precise standard, the regulation is likely to ask for requirements on the sensors capabilities. A state of the art of recent works allows proposing minimum sensor performances and focusing on an essential set of sensors. This work is concluded by presenting future challenges that need to be addressed to build a safe ACAS Xu baseline and to extend it to smaller and lower altitude UAS.

This paper presents a Research and Development (R&D) effort to design and develop a flight simulation system with Adaptive Surveillance Broadcast (ADS-B), and Traffic Collision Avoidance System (TCAS), for integration of Unmanned Aerial Systems (UAS) into National Airspace System (NAS). The paper will delineate the design and development of the Next Generation Applied Research (NEAR) Lab's FMS (NEAR-FMS) including the UAS aircraft models, and the surveillance simulation equipment. The NEAR-FMS will be used in NextGen projects such as Four-Dimensional Trajectory 4DT operations, as well as Alliance for System Safety of UAS through Research Excellence (ASSURE) projects funded by Federal Aviation Association (FAA). Middle-fidelity Flight Dynamic Models (FDM) of two UASs, a Predator class and a Global Hawk have been designed and tested on JSBSim aerodynamic simulator. Autopilots have been designed and implemented for both UAS FDMs. The paper will present the results of tests on the UAS simulated flights and performance data. The UAS models will be used to test the criticality of the conventional surveillance equipment of the UASs as well as identifying the requirements of these systems for such aircraft. Finally, the assessment of the data generated by the simulated flights will drive the requirements and constraints on the conventional surveillance equipment on the UAS. This project is a work in progress and the detailed results will be published in future publications.

Small unmanned air vehicles (UAVs) have unique advantages and limitations which will affect their safe inclusion into the national airspace system. In particular, challenges associated with emergency handling in beyond line of sight operations will be especially critical to address. This paper proposes initial designs for an autonomous decision system for UAVs to select emergency landing sites in a vehicle fault scenario. The overall design consists of two main components: pre-planning and realtime optimization. In the pre-planning component, the system uses offline information such as geographical and population data to generate landing loss maps over the operating environment, which can be used to constrain planning of flight routes to satisfy a bound on the expected landing loss under worst-case fault. In the real-time component, onboard sensor data is used to update a probabilistic risk assessment for potential landing areas allowing for refinement of the expected loss calculation and landing site selection at the time of a fault. The mathematical models and computational algorithms constituting these system components are based upon methodologies in optimal control and statistical inference. Simulation results are provided to demonstrate the application of the proposed algorithms in an example of quadrotor emergency landing over a section of UC Berkeley campus.

NASA's Unmanned Aerial System (UAS) Traffic Management (UTM) project aims at enabling near-term, safe operations of small UAS vehicles in uncontrolled airspace, i.e., Class G airspace. A far-term goal of UTM research and development is to accommodate the expected rise in small UAS traffic density throughout the National Airspace System (NAS) at low altitudes for beyond visual line-of-sight operations. This paper describes a new capability referred to as ICAROUS (Integrated Configurable Algorithms for Reliable Operations of Unmanned Systems), which is being developed under the UTM project. ICAROUS is a software architecture comprised of highly assured algorithms for building safety-centric, autonomous, unmanned aircraft applications. Central to the development of the ICAROUS algorithms is the use of well-established formal methods to guarantee higher levels of safety assurance by monitoring and bounding the behavior of autonomous systems. The core autonomy-enabling capabilities in ICAROUS include constraint conformance monitoring and contingency control functions. ICAROUS also provides a highly configurable user interface that enables the modular integration of mission-specific software components.

Accident statistics cite the flight crew as a causal factor in over 60% of accidents involving transport category airplanes. Yet, a well-trained and well-qualified pilot is acknowledged as the critical center point of aircraft systems safety and an integral safety component of the entire commercial aviation system. No data currently exists that quantifies the contribution of the flight crew in this role. Neither does data exist for how often the flight crew handles non-normal procedures or system failures on a daily basis in the National Airspace System. A pilot-in-the-loop high fidelity motion simulation study was conducted by the NASA Langley Research Center in partnership with the Federal Aviation Administration (FAA) to evaluate the pilot's contribution to flight safety during normal flight and in response to aircraft system failures. Eighteen crews flew various normal and non-normal procedures over a two-day period and their actions were recorded in response to failures. To quantify the human's contribution, crew complement was used as the experiment independent variable in a between-subjects design. Pilot actions and performance when one of the flight crew was impaired were also recorded for comparison against the nominal two-crew operations. This paper details a portion of the results of this study.

There is a checkbox in the Time-Based Flow Management (TBFM) scheduling window which, when checked by a Traffic Management Coordinator (TMC), makes room for a departure to fit into a crowded airborne stream. The checkbox ON algorithm accomplishes this by delaying the Scheduled Times of Arrivals (STAs) of the airborne flights upstream of the TBFM freeze horizons and compressing these flights to their minimum required spacing, thereby creating a full departure slot. Hence, having the checkbox ON can reduce the frequent ground delays of aircraft departing near high volume airports but can increase delays for airborne arrivals. A Human-in-the-Loop (HITL) simulation compared arrival and departure delays to Newark Airport (EWR) with the checkbox ON vs. OFF as the default position. Three other conditions in this HITL involved various National Airspace System (NAS)-wide approaches for timely delivery of aircraft to the TBFM region. These conditions were: Baseline, using current Mile-in-Trail (MIT) spacing restrictions; Integrated Demand Management (IDM), where all aircraft were given departure times (Expect Departure Clearance Times, or EDCTs), ultimately based on the EWR Airport Arrival Rate; and IDM plus Required Time of Arrival (RTA), a flight deck tool which allowed some aircraft to meet a controlled time of arrival to the TBFM area more precisely. Results showed that the checkbox tool was powerful: with the checkbox ON, departure delays decreased and airborne delays increased, as predicted. However, assuming that the cost ratio of a minute of airborne delay to a minute of departure delay is in the range of 1.2 to 3, as commonly indicated by the literature, checkbox ON and checkbox OFF conditions showed approximately equal total delay costs, i.e., the cost of delays in the air balanced the cost of the delay on the ground. The three scheduling conditions also had approximately equal total delay costs, although a simulation artifact may have reduced the delays in the Baseline condition. In the debrief following the simulation, the TMCs concluded that the checkbox should be used flexibly depending on the current delay situation, and suggested modifications to the checkbox tool which would help them use it in this way, along with enhanced training. The relatively similar total cost of both checkbox default options in this simulation indicates that this might be a fruitful approach, and replace the necessity to have the checkbox rigidly set to either ON or OFF.

This paper describes the development of a real-time composite interactive display suite or “dashboard” capability for monitoring and alerting significant National Airspace System (NAS) events. The dashboard is designed to address the operational needs of air traffic managers in gathering data from multiple sources, thus gaining situational awareness of the NAS status. It monitors several system performance areas in real-time and identifies alerting events, pushing to the user relevant information and notifications regarding evolving problems. (“Pushing” information on a display means autonomous presentation — no query or other interrogation by the user is required.) While high-level system alerts are provided, drill-down information is also available for the user to determine whether further actions are warranted. The dashboard also allows the user to customize the alerts for a specific context by setting thresholds. Field evaluation results suggest that the dashboard capability shortens traffic managers' data acquisition time and, since the dashboard capability is a shared web application, it improves common situational awareness on NAS operations. Ongoing research focuses on calibrating alert thresholds for site-specific needs and expanding the selection of performance metrics enabled by System Wide Information Management feeds.

As air traffic grows, improving the resilience of an increasingly congested airspace is becoming a primary concern to ensure its sustainability and avoid the propagation of localized disruptions. The fire set at Chicago Air Traffic Control facility in September 2014 is a striking and worrisome case where one ill-intentioned individual halted traffic in a facility which caused ripple effects across the National Airspace. This paper analyzes the particular role that the Chicago center plays in the National airspace. Then it provides a detailed analysis of the local and global impact of the fire, through the day of the event and for the next two weeks while the facility remained closed for repairs. Finally, it draws the lessons learnt from this event as well as other recent infrastructure failures.

This chapter describes current and future unmanned aircraft system (UAS) operations in the context of the regulatory regime that governs their use. Because the uses of UAS are inextricably linked to the ways in which they receive airworthiness and operational approval, it is challenging to understand how UAS are and will be used without an understanding of that regime. The use of a certificate of authorization or waiver process for public entities is described, followed by a review of the airworthiness approval process for civil operators using “Section 333 exemptions.” A description of the long-standing rules that have been in place for model aircraft hobbyists is also presented. Next, current UAS operations under these regulatory regimes are reviewed. Public and civil uses of UAS are described through examination of the publicly available approvals the FAA has provided, while hobbyist operations are described from a review of tens of thousands of UAS flights uploaded to an online drone community. The chapter identifies three regulatory regimes that have been proposed or are being developed, and then describes the types of operations that could be enabled by one or more of the regimes. These new operations are based on data collection needs for experts in non-UAS fields rather than missions proposed by those who operate UAS directly. Keywords: certificate of authorization or waiver; national airspace system; operational profile; policy; regulatory framework; UAS traffic management; unmanned aircraft system

The Next Generation Air Transportation System (NextGen) refers to the federal programs (predominately airspace, air traffic, or avionics related) that are designed to modernize the National Airspace System (NAS). ACRP’s NextGen initiative aims to inform airport operators about some of these programs and how the enabling practices, data, and technologies resulting from them will affect airports and change how they operate. This volume, the first report in this series, provides comprehensive information to practitioners concerning all aspects of Performance-Based Navigation (PBN) and how implementation affects overall airport operations. This Resource Guide encompasses background information, description of effects on short- and long-term airport development, impacts on safety and performance measures, and other critical factors affecting future airport operations. In addition to providing guidance to users on available resources for additional assistance, this volume also includes lessons learned and best practices based on findings from case studies that examined the airport operator’s role in PBN implementation.

The National Airspace System (NAS) Enterprise Architecture (EA) describes Next Generation Air Transportation System (NextGen) goals, operational changes, planned infrastructure changes, and guidance materials that are referenced by FAA programs throughout the acquisition process. To strengthen the presence of human factors in NAS infrastructure plans and improvements, the FAA Human Factors Research and Engineering Division executed a redesign of the Human System Integration (HSI) Roadmap. This paper will present the methods utilized to redesign the HSI Roadmap, provide an overview of sample human factors integration opportunities, and key lessons learned.

Small unmanned aircraft systems (sUAS) represent a significant instrument for improving task efficiency and effectiveness across numerous industries and operational environments. However, concern has grown regarding potentially irresponsible operation and public apprehension to potential privacy loss. These concerns, combined with unique sUAS human factors challenges, may lead to unwanted and dangerous results, including reduction of safety, property damage, and loss of life. Such challenges include lack of command, control, and communication (C3) standardization; detection, tracking, and managing operations; and human perceptual and cognitive issues. Issues and concerns could be significant barriers to permitting routine and sustainable operations in the National Airspace System (NAS), but by closely examining these factors may be possible to devise strategies to better support future application. This exploratory study seeks to provide a review of relevant exigent literature as well as condense findings into sets of recommendations and guidelines for human factors in sUAS adoption and use.

The air transportation system is a network of many interacting, capacity-constrained elements. When the demand for airport and airspace resources exceed the available capacities of these resources, delays occur. The state of the air transportation system at any time can be represented as a weighted directed graph in which the nodes correspond to airports, and the weight on each arc is the delay experienced by departures on that origin-destination pair. Over the course of any day, the state of the system progresses through a time-series, where the state at any time-step is the weighted directed graph described above. This paper presents algorithms for the clustering of air traffic delay network data from the US National Airspace System, in order to identify characteristic delay states (i.e., weighted directed graphs) as well as characteristic types-of-days (i.e., sequences of such weighted directed graphs) that are experienced by the air transportation system. The similarity of delay states during clustering are evaluated on the basis of not only the in- and out-degrees of the nodes (the total inbound and outbound delays), but also network-theoretic properties such as the eigenvector centralities, and the hub and authority scores of different nodes. Finally, the paper looks at community detection, that is, the grouping of nodes (airports) based on their similarities within a system delay state. The type of day is found to have an impact on the observed community structures., United States. National Aeronautics and Space Administration (FA8721-05-C-0002), National Science Foundation (U.S.) (1239054)

There is an overwhelming desire to create and enhance communication mechanisms between entities that operate within the National Airspace System. Furthermore, airlines are always extremely interested in increasing the efficiency of their flights. An innovative system prototype was developed and tested that improves collaborative decision making without modifying existing infrastructure or operational procedures within the current Air Traffic Management System. This system enables collaboration between flight crew and airline dispatchers to share and assess optimized flight routes through an Internet connection. Using a sophisticated medium-fidelity flight simulation environment, a rapid-prototyping development, and a unified modeling language, the software was designed to ensure reliability and scalability for future growth and applications. Ensuring safety and security were primary design goals, therefore the software does not interact or interfere with major flight control or safety systems. The system prototype demonstrated an unprecedented use of in-flight Internet to facilitate effective communication with Airline Operations Centers, which may contribute to increased flight efficiency for airlines.

NASA is currently conducting concept exploration studies toward the definition of a far-term, gate-to-gate concept for Trajectory-Based Operations. This paper presents a basic architectural framework for the far-term concept and discusses some observations about implementation of trajectory-based operations in the National Airspace System. Within the concept, operators and service providers collaboratively negotiate aircraft trajectories, providing agile, optimized, aircraft-specific routing to meet service provider gate-to-gate flow-management constraints and increasing capacity by smoothly and effectively combining flight-deck-based and ground-based metering, merging, and spacing in a mixed-equipage environment. The far-term TBO concept is intended to influence the direction of mid-term TBO research and to inform the definition of stable requirements and standards for TBO communications infrastructure and user equipage.

The goal of the Airspace operations and Safety Program (AOSP) at NASA is to develop new concepts and technologies for air traffic management that can be integrated within the National Airspace (NAS ...

There is a current drive towards enabling the deployment of increasingly autonomous systems in the National Airspace System (NAS). However, shifting the traditional roles and responsibilities between humans and automation for safety critical tasks must be managed carefully, otherwise the current emergent safety properties of the NAS may be disrupted. In this paper, a verification activity to assess the emergent safety properties of a clearly defined, safety critical, operational scenario that possesses tasks that can be fluidly allocated between human and automated agents is conducted. Task allocation role sets were proposed for a human-automation team performing a contingency maneuver in a reduced crew context. A safety critical contingency procedure (engine out on takeoff) was modeled in the Soar cognitive architecture, then translated into the Hybrid Input Output formalism. Verification activities were then performed to determine whether or not the safety properties held over the increasingly autonomous system. The verification activities lead to the development of several key insights regarding the implicit assumptions on agent capability. It subsequently illustrated the usefulness of task annotations associated with specialized requirements (e.g., communication, timing etc.), and demonstrated the feasibility of this approach.

The purpose of Air Transportation is to move people and cargo safely, efficiently and swiftly to their destinations. The companies and individuals who use aircraft for this purpose, the airspace users, desire to operate their aircraft according to a dynamically optimized business trajectory for their specific mission and operational business model. In current operations, the dynamic optimization of business trajectories is limited by constraints built into operations in the National Airspace System (NAS) for reasons of safety and operational needs of the air navigation service providers. NASA has been developing and testing means to overcome many of these constraints and permit operations to be conducted closer to the airspace user's changing business trajectory as conditions unfold before and during the flight. A roadmap of logical steps progressing toward increased user autonomy is proposed, beginning with NASA's Traffic Aware Strategic Aircrew Requests (TASAR) concept that enables flight crews to make informed, deconflicted flight-optimization requests to air traffic control. These steps include the use of data communications for route change requests and approvals, integration with time-based arrival flow management processes under development by the Federal Aviation Administration (FAA), increased user authority for defining and modifying downstream, strategic portions of the trajectory, and ultimately application of self-separation. This progression takes advantage of existing FAA NextGen programs and RTCA standards development, and it is designed to minimize the number of hardware upgrades required of airspace users to take advantage of these advanced capabilities to achieve dynamically optimized business trajectories in NAS operations. The roadmap is designed to provide operational benefits to first adopters so that investment decisions do not depend upon a large segment of the user community becoming equipped before benefits can be realized. The issues of equipment certification and operational approval of new procedures are addressed in a way that minimizes their impact on the transition by deferring a change in the assignment of separation responsibility until a large body of operational data is available to support the safety case for this change in the last roadmap step.This paper will relate the roadmap steps to ongoing activities to clarify the economics-based transition to these technologies for operational use.

Measuring and monitoring resources in the National Airspace System (NAS) is a key activity of Federal Aviation Administration (FAA) operational managers. Airports and associated terminal areas are complex systems and require a number of metrics to properly characterize performance. In this document, 13 performance measures are described, in three different categories: (1) flow efficiency, for both taxiing and airborne flights, (2) runway utilization, for both arrivals and departures, and (3) rate of flights not cancelled or diverted, for both arrivals and departures. These 13 metrics capture performance in the different stages of flight from push-back to gate arrival. Sample calculations and use cases are presented, using Chicago O’Hare International Airport (ORD) as a subject airport. The multiple measures may be combined for an omnibus metric, for quick assessment of performance at that airport, in a next-day review context.

In order to enable arrival management concepts and solutions in a Next Generation Air Transportation System (NextGen) environment, ground-based sequencing and scheduling functions were developed to support metering operations in the National Airspace System. These sequencing and scheduling tools are designed to assist air traffic controllers in developing an overall arrival strategy, from enroute down to the terminal area boundary. NASA developed a ground system concept and protoype capability called Terminal Sequencing and Spacing (TSAS) to extend metering operations into the terminal area to the runway. To demonstrate the use of these scheduling and spacing tools in an operational-like environment, the FAA, NASA, and MITRE conducted an Operational Integration Assessment (OIA) of a prototype TSAS system at the FAA's William J. Hughes Technical Center (WJHTC). This paper presents an analysis of the arrival management strategies utilized and delivery accuracy achieved during the OIA. The analysis demonstrates how en route preconditioning, in various forms, and schedule disruptions impact delivery accuracy. As the simulation spanned both enroute and terminal airspace, the use of Ground Interval Management - Spacing (GIM-S) enroute speed advisories was investigated. Delivery accuracy was measured as the difference between the Scheduled Time of Arrival (STA) and the Actual Time of Arrival (ATA). The delivery accuracy was computed across all runs conducted during the OIA, which included deviations from nominal operations which are known to commonly occur in real operations, such as schedule changes and missed approaches. Overall, 83% of all flights were delivered into the terminal airspace within +/- 30 seconds of their STA and 94% of flights were delivered within +/- 60 seconds. The meter fix delivery accuracy standard deviation was found to be between 36 and 55 seconds across all arrival procedures. The data also showed when schedule disruptions were excluded, the percentage of aircraft delivered within +/- 30 seconds was between 85 and 90% across the various arrival procedures at the meter fix. This paper illustrates the ability to meet new delivery accuracy requirements in an operational-like environment using operational systems and NATCA controller participants, while also including common events that might cause disruptions to the schedule and overall system.

Incremental approaches to air transportation system development inherit current architectural constraints, which, in turn, place hard bounds on system capacity, efficiency of performance, and complexity. To enable airspace operations of the future, a clean-slate (ab initio) airspace design(s) must be considered. This ab initio National Airspace System (NAS) must be capable of accommodating increased traffic density, a broader diversity of aircraft, and on-demand mobility. System and subsystem designs should scale to accommodate the inevitable demand for airspace services that include large numbers of autonomous Unmanned Aerial Vehicles and a paradigm shift in general aviation (e.g., personal air vehicles) in addition to more traditional aerial vehicles such as commercial jetliners and weather balloons. The complex and adaptive nature of ab initio designs for the future NAS requires new approaches to validation, adding a significant physical experimentation component to analytical and simulation tools. In addition to software modeling and simulation, the ability to exercise system solutions in a flight environment will be an essential aspect of validation. The NASA Langley Research Center (LaRC) Autonomy Incubator seeks to develop a flight simulation infrastructure for ab initio modeling and simulation that assumes no specific NAS architecture and models vehicle-to-vehicle behavior to examine interactions and emergent behaviors among hundreds of intelligent aerial agents exhibiting collaborative, cooperative, coordinative, selfish, and malicious behaviors. The air transportation system of the future will be a complex adaptive system (CAS) characterized by complex and sometimes unpredictable (or unpredicted) behaviors that result from temporal and spatial interactions among large numbers of participants. A CAS not only evolves with a changing environment and adapts to it, it is closely coupled to all systems that constitute the environment. Thus, the ecosystem that contains the system and other systems evolves with the CAS as well. The effects of the emerging adaptation and co-evolution are difficult to capture with only combined mathematical and computational experimentation. Therefore, an ab initio flight simulation environment must accommodate individual vehicles, groups of self-organizing vehicles, and large-scale infrastructure behavior. Inspired by Massively Multiplayer Online Role Playing Games (MMORPG) and Serious Gaming, the proposed ab initio simulation environment is similar to online gaming environments in which player participants interact with each other, affect their environment, and expect the simulation to persist and change regardless of any individual player's active participation.

Air transportation networks are being disrupted with increasing frequency by failures in their cyber- (computing, communication, control) systems. Whether these cyber- failures arise due to deliberate attacks or incidental errors, they can have far-reaching impact on the performance of the air traffic control and management systems. For instance, a computer failure in the Washington DC Air Route Traffic Control Center (ZDC) on August 15, 2015, caused nearly complete closure of the Centers airspace for several hours. This closure had a propagative impact across the United States National Airspace System, causing changed congestion patterns and requiring placement of a suite of traffic management initiatives to address the capacity reduction and congestion. A snapshot of traffic on that day clearly shows the closure of the ZDC airspace and the resulting congestion at its boundary, which required augmented traffic management at multiple locations. Cyber- events also have important ramifications for private stakeholders, particularly the airlines. During the last few months, computer-system issues have caused several airlines fleets to be grounded for significant periods of time: these include United Airlines (twice), LOT Polish Airlines, and American Airlines. Delays and regional stoppages due to cyber- events are even more common, and may have myriad causes (e.g., failure of the Department of Homeland Security systems needed for security check of passengers, see [3]). The growing frequency of cyber- disruptions in the air transportation system reflects a much broader trend in the modern society: cyber- failures and threats are becoming increasingly pervasive, varied, and impactful. In consequence, an intense effort is underway to develop secure and resilient cyber- systems that can protect against, detect, and remove threats, see e.g. and its many citations. The outcomes of this wide effort on cyber- security are applicable to the air transportation infrastructure, and indeed security solutions are being implemented in the current system. While these security solutions are important, they only provide a piecemeal solution. Particular computers or communication channels are protected from particular attacks, without a holistic view of the air transportation infrastructure. On the other hand, the above-listed incidents highlight that a holistic approach is needed, for several reasons. First, the air transportation infrastructure is a large scale cyber-physical system with multiple stakeholders and diverse legacy assets. It is impractical to protect every cyber- asset from known and unknown disruptions, and instead a strategic view of security is needed. Second, disruptions to the cyber- system can incur complex propagative impacts across the air transportation network, including its physical and human assets. Also, these implications of cyber- events are exacerbated or modulated by other disruptions and operational specifics, e.g. severe weather, operator fatigue or error, etc. These characteristics motivate a holistic and strategic perspective on protecting the air transportation infrastructure from cyber- events. The analysis of cyber- threats to the air traffic system is also inextricably tied to the integration of new autonomy into the airspace. The replacement of human operators with cyber functions leaves the network open to new cyber threats, which must be modeled and managed. Paradoxically, the mitigation of cyber events in the airspace will also likely require additional autonomy, given the fast time scale and myriad pathways of cyber-attacks which must be managed. The assessment of new vulnerabilities upon integration of new autonomy is also a key motivation for a holistic perspective on cyber threats.

When weather or congestion impacts the National Airspace System, multiple different Traffic Management Initiatives can be implemented, sometimes with unintended consequences. One particular perceived inequity that is commonly identified is in the interaction between Ground Delay Programs (GDPs) and time based scheduling of internal departures by the Traffic Management Advisor (TMA) (now operationally superseded by the FAA's the Time-Based Flow Management system). Internal departures under TMA scheduling can take large GDP delays, followed by large TMA scheduling delays, because they cannot easily fit into the arrival flow at the runway. In this paper we examine the causes of these double delays through an analysis of arrival operations at Newark Liberty International Airport (EWR) from June to August 2010. TMA scheduling delays are found to be generally higher than TMA airborne metering delays, regardless of prior GDP delays. Depending on how the double delay is defined, between 42 and 62 of all internal departures in GDP and TMA scheduling experienced double delays in this period. A deep dive into the data reveals that contributors to double delays include upstream flights departing before their Expect Departure Clearance Times (EDCTs); differences in the rates used for setting EDCTs and TMA Scheduled Times of Arrival; differences in the arrival demand expected based on EDCTs and the arrival demand entering TMA; and shorter en route times between takeoff and entry into TMA than assumed in the calculation of flight EDCTs, all of which undermine the sequencing and spacing underlying flight EDCTs. Double delays are also found to coincide with periods in which the virtual runway arrival queue being served by a TMA is large, there are periods of high demand relative to capacity, and there are high airborne metering delays. Data mining techniques are used to confirm that each of these factors contribute to the occurrence of double delay andor high internal departure scheduling delay across three months of data from June to August 2010. Predictors of the occurrence of double delay and high TMA scheduling delay are built using logistic regression, providing prediction accuracies of 69 and 73, respectively.

The airspace is becoming more and more complicated, and will continue to do so in the future with the integration of Unmanned Aerial Vehicles (UAVs), autonomy, spacecraft, other forms of aviation technology into the airspace. The new technology and complexity increases the importance and difficulty of safety assurance. Additionally, testing new technologies on complex aviation systems & systems of systems can be very difficult, expensive, and sometimes unsafe in real life scenarios. Prognostic methodology provides an estimate of the health and risks of a component, vehicle, or airspace and knowledge of how that will change over time. That measure is especially useful in safety determination, mission planning, and maintenance scheduling. The developed testbed will be used to validate prediction algorithms for the real-time safety monitoring of the National Airspace System (NAS) and the prediction of unsafe events. The framework injects flight related anomalies related to ground systems, routing, airport congestion, etc. to test and verify algorithms for NAS safety. In our research work, we develop a live, distributed, hardware-in-the-loop testbed for aviation and airspace prognostics along with exploring further research possibilities to verify and validate future algorithms for NAS safety. The testbed integrates virtual aircraft using the X-Plane simulator and X-PlaneConnect toolbox, UAVs using onboard sensors and cellular communications, and hardware in the loop components. In addition, the testbed includes an additional research framework to support and simplify future research activities. It enables safe, accurate, and inexpensive experimentation and research into airspace and vehicle prognosis that would not have been possible otherwise. This paper describes the design, development, and testing of this system. Software reliability, safety and latency are some of the critical design considerations in development of the testbed. Integration of HITL elements in the development phases and veri cation/ validation are key elements to this report.

This paper explores the Single Runway Scheduling (SRS) problem with arrivals, departures, and crossing aircraft on the airport surface. Constraints for wake vortex separations, departure area navigation separations and departure time window restrictions are explicitly considered. The main objective of this research is to develop exact and heuristic based algorithms that can be used in real-time decision support tools for Air Traffic Control Tower (ATCT) controllers. The paper provides a multi-objective dynamic programming (DP) based algorithm that finds the exact solution to the SRS problem, but may prove unusable for application in real-time environment due to large computation times for moderate sized problems. We next propose a second algorithm that uses heuristics to restrict the search space for the DP based algorithm. A third algorithm based on a combination of insertion and local search (ILS) heuristics is then presented. Simulation conducted for the east side of Dallas/Fort Worth International Airport allows comparison of the three proposed algorithms and indicates that the ILS algorithm performs favorably in its ability to find efficient solutions and its computation times.

Large scale distributed visualizations are routinely used by organizations in both the commercial and government sectors for training and simulation. These simulations are of particular importance in the research associated with the United States Federal Aviation Administration's current work on updating the National Airspace System to the Next Generation Air Transportation System. This new system will include the integration of unmanned aerial systems into civil airspace for routine flights. In this paper we propose a system for integrating unmanned aerial system visual observers into the distributed simulation capability of the Federal Aviation Administration using large-scale virtual reality systems. We demonstrate the replication of terrain surrounding flight tests in virtual environments and the generation of the observer's views from both ground-based and airborne scenarios.

A key component to solving many of the engineering challenges for integrating UAS into the National Airspace System (NAS) is being able to state within discrete airspace, the numbers of forecasted UAS by airframe type and the type of mission or operational use being performed.

Ongoing measurement of system performance is an essential input to management of the National Airspace System (NAS) by the Federal Aviation Administration (FAA). MITRE has been working with the Air Traffic Management System Command Center (ATCSCC) since 2000 to provide new performance metrics and data visualizations to better measure and understand system performance, and inform changes to behavior to reduce delays in the future.

This paper advocates for a specific design approach, based on simple principals, yet addresses challenges faced by the system engineers when designing complex data and information infrastructure. The document provides guidance for breaking out various work elements in the overall network architecture design, so that communication systems are conceived and effectively realized regardless of their location, size and local specifics. Although targeted at the Global Airspace System (GAS) and National Airspace System (NAS), this framework can be applied to any network-centric architecture.

Commercial space operations introduce challenges in both the integration of spacecraft national airspace system and support the coordination of orbital spacecraft. There exists a need for a low-cost tracking system capable of supporting these vehicles that does not require a substantial development of additional infrastructure. Automatic Dependent Surveillance - Broadcast (ADS-B) technology is a tracking system used for aviation that is supported by a nationwide network of ground-based transceivers tied to FAA systems and commercially subscribable data feeds. This paper provides a progress report on the development and technical maturation of a 978 MHz ADS-B prototype developed by Embry-Riddle Aeronautical University under the support of the FAA Office of Commercial Space Transportation with the goal of supporting these operations. This paper discusses the prototype's design and history, the integration of the payload onboard a re-entry vehicle, proposed future upgrades, and planned flight tests.

Air Traffic Management helps to ensure all National Airspace System (NAS) operators have the opportunity to safely accomplish their operational goals while maintaining NAS efficiency. Advancements in technology and a robust global demand for access to space are leading to new space flight participants and increasing numbers of space launch and reentry operations, adding complexity to NAS management coordination and decision-making processes. Diversity of launch and reentry profiles and locations, coupled with increased activity [1, 2], will impact airspace management planning by requiring increased coordination among the Federal Aviation Administration (FAA) and other federal agencies, civilian facilities, and commercial operators. Current processes will not scale as space activity in the NAS increases. The MITRE Corporation is working with the FAA to improve launch and reentry operations efficiency and repeatability. The approach includes analyzing current processes and identifying how more standardized communication mechanisms, integrated information flows, and consistent analysis capabilities can be applied. This paper describes a concept for sharing information and integrating it as part of a standardized, repeatable process that can help the FAA meet future space activity needs in the NAS. A key feature is to integrate information related to NAS effects assessment into the process to facilitate rapid analysis and feedback.

In the last few years, radio technologies for unmanned aircraft vehicle (UAV) have advanced very rapidly. The increasing need to fly unmanned aircraft systems (UAS) in the national airspace system (NAS) to perform missions of vital importance to national security, defense, and science has pushed ahead the design and implementation of new radio platforms. However, a lot still has to be done to improve those radios in terms of performance and capabilities. In addition, an important aspect to account for is hardware cost and the feasibility to implement these radios using commercial off-the-shelf (COTS) components. UAV radios come with numerous technical challenges and their development involves contributions at different levels of the design. Cognitive algorithms need to be developed in order to perform agile communications using appropriate frequency allocation while maintaining safe and efficient operations in the NAS and, digital reconfigurable architectures have to be designed in order to ensure a prompt response to environmental changes. Command and control (C2) communications have to be preserved during “standard” operations while crew operations have to be minimized. It is clear that UAV radios have to be software-defined systems, where size, weight and power consumption (SWaP) are critical parameters. This paper provides preliminary results of the efforts performed to design a fully digital radio architecture as part of a NASA Phase I STTR. In this paper, we will explain the basic idea and technical principles behind our dynamic/adaptive frequency hopping radio for UAVs. We will present our Simulink model of the dynamic FH radio transmitter design for UAV communications and show simulation results and FPGA system analysis.

System Wide Information Management (SWIM) is a key transformational information exchange program that is an important element of the Federal Aviation Administration's (FAA's) Next Generation Air Transportation System (NextGen). SWIM provides a mechanism for National Airspace System (NAS) and non-NAS consumers to have access to FAA-generated aeronautical, flight, and weather information.

The Federal Aviation Administration's (FAA) Next Generation Air Transportation (NextGen) program is a long-term modernization and transformation of the current National Airspace System (NAS) into a more effective and coordinated decision-making system. NextGen provides a more reliable, secure, and dependable aviation capability for both users and operators ensuring more capacity, throughput, and safety. System Wide Information Management (SWIM) is a NextGen Service Oriented Architecture (SOA) technology that communicates aviation data among the various stakeholders incorporating trajectory based operations and optimum profile descent. SWIM data is delivered to the aircrew via Aircraft Access to SWIM (AAtS) framework. This research paper investigates the information security threats and performance caveats in the main operating components of AAtS by identifying the information security breaches associated with the NAS Enterprise Message Service (NEMS), particularly its data exchange path with Electronic Flight Bag (EFB) through the Data Management Service (DMS). This research addresses the risks associated with the information flow passing through the NAS Enterprise Security Gateway (NESG) internal Data Management Zone (DMZ) and external DMZ. This research delineates the overall strategy for identifying the methodology in testing the data flow from the aircraft EFB via the NEMS network to the external service providers as well as identifying test procedures to emulate the threat vectors on this network. After conducting the necessary tests, the key technical research areas are comprehensively evaluated and recommendations to support the security of the AAtS will be documented.

The Federal Aviation Administration (FAA) and the MITRE Corporation (MITRE), among other entities, are working together to integrate space launch and re-entry operations into the National Airspace System (NAS). As part of the effort, MITRE is building a flexible, fast-time Monte Carlo modeling and simulation capability that offers operational measures of safety. Once validated, the FAA can use it to assess different separation concepts and associated standards, and support the FAA's Safety Management System process. The developed capability will help the FAA determine what separation concepts and associated standards ensure safety for each space launch and re-entry operation and will provide insight into the required surveillance performance, air traffic control response times, and traffic limits to enable new separation concepts. MITRE built the capability using our models for aircraft flight trajectories, aircraft-to-aircraft conflict prediction and detection, and surveillance performance. To tailor these models for use in examining space launch and re-entry operations, MITRE also developed a space vehicle trajectory model and debris model (based on Stanford University's work—Range Safety Assessment Tool) for use in the simulation of the scenarios. This paper focuses on describing the modeling capabilities, algorithms, and some preliminary simulation results.

This research describes a new method to ensure the safety of sense and avoid (SAA) functions for unmanned aircraft systems (UAS) in the presence of multiple intruders. In this work, integrity and continuity are used as quantifiable safety performance metrics. Whereas previous research addressed a single intruder, the approach described in this paper can account for multiple intruders. Integrity and continuity are addressed though determination of the probability of data mis-associations for multiple intruders. A mis-association occurs when the system incorrectly associates one intruder's measurement with another intruder's trajectory. Incorrect intruder associations are hazardously misleading information impacting integrity. This impact on integrity has a second order effect on continuity. A sensitivity analysis is performed based on two two-intruder encounters. The resulting impact of mis-associations between multiple intruders on integrity and continuity is quantified for a nominal composite SAA sensor. The sensor characteristic trade space is explored. The concepts and methods in this research are intended to be used as a tool for certification authorities to set potential requirements for integration into the national airspace system.

Since 2013 Unmanned Aerial Systems, commonly called drones, have become a disruptive technology in the consumer market and the National Airspace System due to their decreasing costs and increased consumer interest. Sales of drones have experienced growth in excess of 500 percent from 2014 to 2016. According to the Federal Aviation Administration the number of incidents involving drones has also grown. The Federal Aviation Administration currently provides few regulations for drones, instead offering advisories to drone operators. In order to determine a course of action for safe integration, the Federal Aviation Administration instituted a pilot reporting database of incidents involving drones. In over 97% of the incidents reported, the drone operators were not following altitude and location advisories. This paper examines the effectiveness of geofencing systems for drones and compares its effectiveness to the current proposed regulations and traditional collision avoidance systems. A Monte Carlo simulation using growth and incident rates of drones based on the incident data is used to determine the utility of each alternative. The alternatives are then analyzed through a utility function and evaluated on a cost/utility basis. The result shows that limiting flight operations to the regulated thresholds provides a cost effective method for adding safety to the operation of drones within the National Airspace System.

Congressional attention to Unmanned Aircraft Systems (UAS)1 has caused the Federal Aviation Administration (FAA) to move the National Airspace System (NAS) Integration project forward, but using guidelines, practices and procedures that are yet to be fully integrated with the FAA Aviation Management System. The real drive for change in the NAS will to come from both UAS operators and the government jointly seeing an accurate forecast of UAS usage demand data. This solid forecast information would truly get the attention of planners. This requires not an aggregate demand, but rather a picture of how the demand is spread across small to large UAS, how it is spread across a wide range of missions, how it is expected over time and where, in terms of geospatial locations, will the demand appear. In 2012 the Volpe Center performed a study of the overall future demand for UAS. This was done by aggregate classes of aircraft types. However, the realistic expected demand will appear in clusters of aircraft activities grouped by similar missions on a smaller geographical footprint and then growing from those small cells. In general, there is not a demand forecast that is tightly coupled to the real purpose of the mission requirements (e.g. in terms of real locations and physical structures such as wind mills to inspect, farms to survey, pipelines to patrol, etc.). Being able to present a solid basis for the demand is crucial to getting the attention of investment, government and other fiscal planners. To this end, Mosaic ATM under NASA guidance is developing a crowd sourced, demand forecast engine that can draw forecast details from commercial and government users and vendors. These forecasts will be vetted by a governance panel and then provide for a sharable accurate set of projection data. Our paper describes the project and the technical approach we are using to design and create access for users to the forecast system.

The safe integration of unmanned aircraft into the national airspace is a topic receiving considerable attention. The US Congress has mandated that the Federal Aviation Administration (FAA) engage in rulemaking that will allow for civil operation of unmanned aircraft in the national airspace system (NAS), with a release of those rules in 2016*. Preparations are occurring simultaneously for the implementation of the FAA's NextGEN System on 1 January 2020, changing Air Traffic Control (ATC) in the US from one that relies upon radar to a GPS-based system. The introduction of Unmanned Aerial Systems (UAS) into the NAS is expected to be integrated with NextGEN. The availability of small, light ADS-B Out equipment (Autonomous Dependent System-Broadcast) is anticipated to facilitate that, and is discussed. Further, control of UASs will likely adhere to the procedural rules present in today's ATC System and in NextGEN, rules that require a pilot in the loop, making fully autonomous operation forbidden. UASs are therefore expected to employ additional systems and equipment to enhance their sense & avoid capability. Collision avoidance in manned aircraft was improved in 2005 when certain aircraft were required to use the Traffic Collision Avoidance System, TCAS II. The sense & avoid performance of UASs within the NextGEN-controlled airspace should similarly be enhanced by a new system called the Airborne Collision Avoidance System, ACAS. This TCAS replacement will have several variants for various aircraft types including UASs, and it will be capable of accepting data from various types of onboard sensors, making “optimal” use of that data when available. Thus collision avoidance on future aircraft should benefit from enhanced sensing devices such as LIDAR, Mode-S transponder interrogation, IR sensors, and possibly from that provided by the “Due Regard” radar system currently in test on the Predator UAS.

In response to the predicted increase in frequency and complexity of commercial launch and reentry operations, the FAA must continue its effort to develop integrated capabilities to meet the Administration’s objectives and realize the anticipated operational benefits of the integration of these operations into the National Airspace System (NAS). The Space Data Integrator (SDI) platform is such a capability that automates the FAA’s currently manual, time-consuming and resource-intensive procedures to support commercial launch and reentry operations. The objectives of the SDI are to allow for extending the operations’ planning to include filing a flight plan, to provide situational awareness to the Joint Space Operations Group (JSpOG) at the FAA Air Traffic Control System Command Center (ATCSCC) on the launch and reentry vehicle position and mission parameters during the full operations, and to support the detection and response to abnormal events. The SDI is the result of a non-traditional approach for FAA that is flexible, with high-implementation readiness, directly tailored to the end-user needs, and both low-cost/low impact on existing NAS automation systems. The SDI can be used as a shadow system while continuing to mature, a training tool for the JSpOG personnel, and a familiarization tool for commercial space vehicle operators, spaceport representatives and air traffic control personnel.

This paper presents a time-based path planning optimizer for separation assurance for unmanned aircraft systems (UAS). Given Automatic Dependent Surveillance-Broadcast (ADS-B) as a sensor, intruder information such as position, velocity, and identification information is available at ranges on the order of 50 nautical miles. Such long-range intruder detection facilitates path planning for separation assurance, but also poses computational challenges. The time-based path optimizer presented in this paper provides a path-planning method that takes advantage of long-range ADS-B information and addresses the associated challenges. It is capable of long-range path planning and, due to the convex formulation, is computationally efficient enough to run successively for increased robustness. The ultimate result of this research is a convex, time-based path planner that is suitable for a detectand-avoid solution on small UAS in the National Airspace System.

This research aims at developing methods to meet safety targets for sense and avoid (SAA) sensors of Unmanned Aircraft Systems (UAS) through integrity and continuity risk evaluation. This paper builds upon previous work that assumed a constant velocity aircraft encounter model. We refine this model by accounting for intruder aircraft accelerations due to pilot actions, such as changes in thrust. These methods can determine the acceptability of a sensor or set of sensors. A sensitivity analysis evaluates the impact on integrity and continuity of sensor noise, range and sample interval. The methods described in this research can set potential SAA sensor requirements for UAS integration into the National Airspace System.

The Federal Aviation Administration (FAA) is undertaking a National Airspace System (NAS) modernization effort known as NextGen. NextGen includes a significant number of new system acquisitions that must be integrated with legacy NAS infrastructure which is continuously undergoing modifications in parallel. Each change to the NAS requires a safety analysis to identify and control safety risks. The FAA commissioned a software tool to assist with evaluating these safety analyses so that oversight activities may be planned accordingly. To accomplish this mission, a model of system and safety data that identifies interdependencies between NAS systems and safety hazards was developed. This model, along with a taxonomy of the data captured, can be used to identify relationships and trends in system and safety data that might otherwise be overlooked in safety analyses prepared separately for each NAS change. This paper discusses the approach used to build the integrated system safety model, the data captured to-date, and preliminary findings and trends from analysis of the model. This model can be used by safety practitioners to evaluate system changes and their contributions to existing hazards in the NAS and to focus oversight activities on systems and changes with the greatest predicted impacts on NAS safety. (Note: The views, opinions, and findings contained in this paper are those of the author and should not be construed as an official FAA position, policy, or decision.)

This paper describes the results of a set of experiments and analyses conducted to evaluate the capability of small unmanned aircraft systems (sUAS) to spot nascent fires in the Great Dismal Swamp (GDS) National Wildlife Refuge. This work is the result of a partnership between the National Aeronautics and Space Administration and the US Fish and Wildlife service specifically to investigate sUAS usage for fire-spotting. The objectives of the current effort were to: 1) Determine suitability and utility of low-cost Small Unmanned Aircraft Systems (sUAS) to detect nascent fires at GDS; 2) Identify and assess the necessary National Airspace System (NAS) integration issues; and 3) Provide information to GDS and the community on system requirements and concepts-of-operation (CONOPS) for conducting fire detection/support mission in the National Airspace and (4) Identify potential applications of intelligent autonomy that would enable or benefit this high-value mission. In addition, data on the ability of various low-cost sensors to detect smoke plumes and fire hot spots was generated during the experiments as well as identifying a path towards a future practical mission utility by using sUAS in beyond visual-line-of-sight operation in the National Airspace System (NAS).

In this chapter, we focus more closely on processes and policies related to passenger and baggage screening. Methods and techniques for screening passengers and baggage are fundamental to sustaining effective aviation security. Within the layered aviation security system, screening passengers and baggage are the most important processes that security practitioners use to mitigate potential threats to the airport and aircraft environments. You will learn about these fundamental concepts and special issues in screening that are vital to securing the national airspace system.

Date of Conference: 4-8 January 2016 Conference Name: AIAA SciTech Forum, 2016 The focus of this paper is to present a game theoretical modeling and simulation frame- work for the integration of Unmanned Aircraft Systems (UAS) into the National Airspace system (NAS). The problem of predicting the outcome of complex scenarios, where UAS and manned air vehicles co-exist, is the research problem of this work. The fundamental gap in the literature in terms of developing models for UAS integration into NAS is that the models of interaction between manned and unmanned vehicles are insufficient. These models are insufficient because a) they assume that human behavior is known a priori and b) they disregard human reaction and decision making process. The contribution of this paper is proposing a realistic modeling and simulation framework that will fill this gap in the literature. The foundations of the proposed modeling method is formed by game theory, which analyzes strategic decision making between intelligent agents, bounded rationality concept, which is based on the fact that humans cannot always make perfect decisions, and reinforcement learning, which is shown to be effective in human behavior in psychology literature. These concepts are used to develop a simulator which can be used to obtain the outcomes of scenarios consisting of UAS, manned vehicles, automation and their interactions. An analysis of the UAS integration is done with a specifically designed scenario for this paper. In the scenario, a UAS equipped with sense and avoid algorithm, moves along a predefined trajectory in a crowded airspace. Then the effect of various system parameters on the safety and performance of the overall system is investigated.