Search

Driving in dense traffic with human and autonomous drivers is a challenging task that requires high-level planning and reasoning. Human drivers can achieve this task comfortably, and there has been many efforts to model human driver strategies. These strategies can be used as inspirations for developing autonomous driving algorithms or to create high-fidelity simulators. Reinforcement learning is a common tool to model driver policies, but conventional training of these models can be computationally expensive and time-consuming. To address this issue, in this paper, we propose ``skill-based" hierarchical driving strategies, where motion primitives, i.e. skills, are designed and used as high-level actions. This reduces the training time for applications that require multiple models with varying behavior. Simulation results in a merging scenario demonstrate that the proposed approach yields driver models that achieve higher performance with less training compared to baseline reinforcement learning methods., Comment: This work has been submitted to the IEEE for possible publication

In this study, we propose a novel adaptive control architecture, which provides dramatically better transient response performance compared to conventional adaptive control methods. What makes this architecture unique is the synergistic employment of a traditional, Adaptive Neural Network (ANN) controller and a Long Short-Term Memory (LSTM) network. LSTM structures, unlike the standard feed-forward neural networks, can take advantage of the dependencies in an input sequence, which can contain critical information that can help predict uncertainty. Through a novel training method we introduced, the LSTM network learns to compensate for the deficiencies of the ANN controller during sudden changes in plant dynamics. This substantially improves the transient response of the system and allows the controller to quickly react to unexpected events. Through careful simulation studies, we demonstrate that this architecture can improve the estimation accuracy on a diverse set of uncertainties for an indefinite time span. We also provide an analysis of the contributions of the ANN controller and LSTM network to the control input, identifying their individual roles in compensating low and high-frequency error dynamics. This analysis provides insight into why and how the LSTM augmentation improves the system's transient response. The stability of the overall system is also shown via a rigorous Lyapunov analysis., Comment: 11 pages, 12 figures

Practical adaptive control implementations where human pilots coexist in the loop are still uncommon, despite their success in handling uncertain dynamical systems. This is owing to their special nonlinear characteristics which lead to unfavorable interactions between pilots and adaptive controllers. To pave the way for the implementation of adaptive controllers in piloted applications, we propose an adaptive human pilot model that takes into account the time delay in the pilot's response while operating on an adaptive control system. The model can be utilized in the evaluation of adaptive controllers through the simulation environment and guide in their design., Comment: 15 pages, 3 figures

This paper proposes a method for modeling human driver interactions that relies on multi-output gaussian processes. The proposed method is developed as a refinement of the game theoretical hierarchical reasoning approach called "level-k reasoning" which conventionally assigns discrete levels of behaviors to agents. Although it is shown to be an effective modeling tool, the level-k reasoning approach may pose undesired constraints for predicting human decision making due to a limited number (usually 2 or 3) of driver policies it extracts. The proposed approach is put forward to fill this gap in the literature by introducing a continuous domain framework that enables an infinite policy space. By using the approach presented in this paper, more accurate driver models can be obtained, which can then be employed for creating high fidelity simulation platforms for the validation of autonomous vehicle control algorithms. The proposed method is validated on a real traffic dataset and compared with the conventional level-k approach to demonstrate its contributions and implications.

The main purpose of a control allocator is to distribute a total control effort among redundant actuators. This paper proposes a discrete adaptive control allocator for over-actuated sampled-data systems in the presence of actuator uncertainty. The proposed method does not require uncertainty estimation or persistency of excitation. Furthermore, the presented algorithm employs a closed loop reference model, which provides fast convergence without introducing excessive oscillations. To generate the total control signal, an LQR controller with reference tracking is used to guarantee the outer loop asymptotic stability. The discretized version of the Aerodata Model in Research Environment (ADMIRE) is used as an over-actuated system, to demonstrate the efficacy of the proposed method.

The focus of this paper is to propose a driver model that incorporates human reasoning levels as actions during interactions with other drivers. Different from earlier work using game theoretical human reasoning levels, we propose a dynamic approach, where the actions are the levels themselves, instead of conventional driving actions such as accelerating or braking. This results in a dynamic behavior, where the agent adapts to its environment by exploiting different behavior models as available moves to choose from, depending on the requirements of the traffic situation. The bounded rationality assumption is preserved since the selectable strategies are designed by adhering to the fact that humans are cognitively limited in their understanding and decision making. Using a highway merging scenario, it is demonstrated that the proposed dynamic approach produces more realistic outcomes compared to the conventional method that employs fixed human reasoning levels., Comment: 17 pages, 10 figures

This paper introduces an analytical framework for the derivation of hybrid equations of motion of a flexible quadrotor. This approach helps obtain rigid and elastic equations of motion simultaneously, in a decoupled form, which facilitates the controller design. A delay-dependent stability condition is obtained for the overall system dynamics, including the operator with reaction time delay, the adaptive controller and the flexible quadrotor dynamics. Two different adaptive controllers are implemented to address the uncertainties. It is demonstrated via simulations that the flexible arm tip oscillations are mitigated when a closed loop reference model adaptive controller is used, compared to a conventional model reference adaptive controller., Comment: 29 pages, 9 Figures, 2 Tables

This paper proposes a projection algorithm which can be employed to bound actuator signals, in terms of both magnitude and rate, for uncertain systems with redundant actuators. The investigated closed loop control system is assumed to contain an adaptive control allocator to distribute the total control input among actuators. Although conventional control allocation methods can handle actuator rate and magnitude constraints, they cannot consider actuator uncertainty. On the other hand, adaptive allocators manage uncertainty and actuator magnitude limits. The proposed projection algorithm enables adaptive control allocators to handle both magnitude and rate saturation constraints. A mathematically rigorous analysis is provided to show that with the help of the proposed projection algorithm, the performance of the adaptive control allocator can be guaranteed, in terms of error bounds. Simulation results are presented, where the Aero-Data Model In Research Environment (ADMIRE) is used as an over-actuated system, to demonstrate the effectiveness of the proposed method.

The focus of this paper is an integrated, fault-tolerant vehicle supervisory control algorithm for the overall stability of ground vehicles. Vehicle control systems contain many sensors and actuators that can communicate with each other over communication networks. The proposed supervisory control scheme is composed of a high-level controller that creates a virtual control input vector and a low-level control allocator that distributes the virtual control effort among redundant actuators. Virtual control input incorporates the required traction force, yaw, pitch, and roll moment corrections, and the lateral force correction to ensure stability while following a maneuvering reference initiated by the driver. Based on the virtual control input vector, the allocation module determines front steering angle correction, rear steering angle, traction forces at each tire, and active suspension forces. The proposed control framework distinguishes itself from earlier results in the literature by its ability to adapt to failures and uncertainties by updating its parameters online, without the need for fault identification. The control structure is validated in the simulation environment using a fourteen degree of freedom nonlinear vehicle model. Our results demonstrate that the proposed approach ensures that the vehicle follows references created by the driver despite the loss of actuator effectiveness up to 30% higher longitudinal maneuver velocity and approximately 35% lower roll and pitch angles during steering with representative driving scenarios., Comment: 15 pages, 11 Figures, 1 table

This paper proposes an adaptive human pilot model that is able to mimic the crossover model in the presence of uncertainties. The proposed structure is based on the model reference adaptive control, and the adaptive laws are obtained using the Lyapunov-Krasovskii stability criteria. The model can be employed for human-in-the-loop stability and performance analyses incorporating different types of controllers and plant types. For validation purposes, an experimental setup is employed to collect data and a statistical analysis is conducted to measure the predictive power of the pilot model.

In this paper, a synergistic combination of deep reinforcement learning and hierarchical game theory is proposed as a modeling framework for behavioral predictions of drivers in highway driving scenarios. The need for a modeling framework that can address multiple human-human and human-automation interactions, where all the agents can be modeled as decision makers simultaneously, is the main motivation behind this work. Such a modeling framework may be utilized for the validation and verification of autonomous vehicles: It is estimated that for an autonomous vehicle to reach the same safety level of cars with drivers, millions of miles of driving tests are required. The modeling framework presented in this paper may be used in a high-fidelity traffic simulator consisting of multiple human decision makers to reduce the time and effort spent for testing by allowing safe and quick assessment of self-driving algorithms. To demonstrate the fidelity of the proposed modeling framework, game theoretical driver models are compared with real human driver behavior patterns extracted from traffic data., Comment: 22 pages, 19 figures

For a foreseeable future, autonomous vehicles (AVs) will operate in traffic together with human-driven vehicles. Their planning and control systems need extensive testing, including early-stage testing in simulations where the interactions among autonomous/human-driven vehicles are represented. Motivated by the need for such simulation tools, we propose a game-theoretic approach to modeling vehicle interactions, in particular, for urban traffic environments with unsignalized intersections. We develop traffic models with heterogeneous (in terms of their driving styles) and interactive vehicles based on our proposed approach, and use them for virtual testing, evaluation, and calibration of AV control systems. For illustration, we consider two AV control approaches, analyze their characteristics and performance based on the simulation results with our developed traffic models, and optimize the parameters of one of them., Comment: IEEE Intelligent Transportation Systems Transactions

Predicting the outcomes of cyber-physical systems with multiple human interactions is a challenging problem. This article reviews a game theoretical approach to address this issue, where reinforcement learning is employed to predict the time-extended interaction dynamics. We explain that the most attractive feature of the method is proposing a computationally feasible approach to simultaneously model multiple humans as decision makers, instead of determining the decision dynamics of the intelligent agent of interest and forcing the others to obey certain kinematic and dynamic constraints imposed by the environment. We present two recent exploitations of the method to model 1) unmanned aircraft integration into the National Airspace System and 2) highway traffic. We conclude the article by providing ongoing and future work about employing, improving and validating the method. We also provide related open problems and research opportunities.

This paper proposes an adaptive control allocation approach for uncertain over-actuated systems with actuator saturation. The proposed method does not require uncertainty estimation or a persistent excitation assumption. Using the element-wise non-symmetric projection algorithm, the adaptive parameters are restricted to satisfy certain optimality conditions leading to overall closed loop system stability. Furthermore, a sliding mode controller with a time-varying sliding surface, working in tandem with the adaptive control allocation, is proposed to guarantee the outer loop stability and reference tracking in the presence of control allocation errors and disturbances. Simulation results are provided, where the Aerodata Model in Research Environment is used as an over-actuated system with actuator saturation, to demonstrate the effectiveness of the proposed method., Comment: 19 pages, 12 figures

Although increased automation has made it easier to control aircraft, ensuring a safe interaction between the pilots and the autopilots is still a challenging problem, especially in the presence of severe anomalies. Current approach consists of autopilot solutions that disengage themselves when they become ineffective. This may cause reengagement of the pilot at the worst possible time, which can result in undesired consequences. In this paper, a series of research studies that propose pilot-autopilot interaction schemes based on the Capacity for Maneuver (CfM) concept, are covered. CfM refers to the remaining capacity of the actuators that can be used for bringing the aircraft to safety. It is demonstrated that CfM-based pilot-autopilot interaction schemes, or Shared Control Architectures (SCA), can be promising alternatives to the existing schemes. Two SCAs are tested in the experiments. In SCA1, the pilot takes over the control from the autopilot using the monitored CfM information. In SCA2, the pilot takes on the role of a supervisor helping the autopilot whenever a need arises, based on the CfM information. Whenever the aircraft experiences a severe anomaly, the pilot assesses the situation based on his/her CfM monitoring and intervenes by providing two control system parameter estimates to the autopilot. This increases the effectiveness of the autopilot. Using human-in-the-loop simulations, it is shown that CfM based interactions provides smaller tracking errors and larger overall CfM. The subjects including a commercial airline pilot and several university students are trained using a systematic procedure. Creation of new cyber physical & human systems is inevitable along with deeper engagement with the social science community so as to get better insight into human decision making. The results reported here should be viewed as a first of several steps in this research direction., Comment: 26 pages, 13 figures, 12 tables

In this paper, we propose a decision making algorithm for autonomous vehicle control at a roundabout intersection. The algorithm is based on a game-theoretic model representing the interactions between the ego vehicle and an opponent vehicle, and adapts to an online estimated driver type of the opponent vehicle. Simulation results are reported., Comment: 2018 IEEE Conference on Decision and Control (CDC)

Predicting the outcomes of integrating Unmanned Aerial Systems (UAS) into the National Airspace System (NAS) is a complex problem which is required to be addressed by simulation studies before allowing the routine access of UAS into the NAS. This paper focuses on providing a 3-dimensional (3D) simulation framework using a game theoretical methodology to evaluate integration concepts using scenarios where manned and unmanned air vehicles co-exist. In the proposed method, human pilot interactive decision making process is incorporated into airspace models which can fill the gap in the literature where the pilot behavior is generally assumed to be known a priori. The proposed human pilot behavior is modeled using dynamic level-k reasoning concept and approximate reinforcement learning. The level-k reasoning concept is a notion in game theory and is based on the assumption that humans have various levels of decision making. In the conventional "static" approach, each agent makes assumptions about his or her opponents and chooses his or her actions accordingly. On the other hand, in the dynamic level-k reasoning, agents can update their beliefs about their opponents and revise their level-k rule. In this study, Neural Fitted Q Iteration, which is an approximate reinforcement learning method, is used to model time-extended decisions of pilots with 3D maneuvers. An analysis of UAS integration is conducted using an example 3D scenario in the presence of manned aircraft and fully autonomous UAS equipped with sense and avoid algorithms., Comment: Added 'Acknowledgement' Section

The precise control of gas generator pressure is important to obtain variable thrust in ducted rockets. A delay resistant closed loop reference model adaptive control is proposed in this paper to address this problem. The proposed controller combines delay compensation and adaptation with improved transient response. The controller is successfully implemented using an industrial grade cold air test setup, which is a milestone towards obtaining a fully developed throttleable rocket gas generator controller. Simulation and experimental comparisons with alternative adaptive approaches and a fixed controller demonstrate improved performance and effective handling of time delays and uncertainties. A step by step design methodology, covering robustfying schemes, selection of adaptation rates and initial controller parameters, is also provided to facilitate implementations.

Autonomous driving has been the subject of increased interest in recent years both in industry and in academia. Serious efforts are being pursued to address legal, technical and logistical problems and make autonomous cars a viable option for everyday transportation. One significant challenge is the time and effort required for the verification and validation of the decision and control algorithms employed in these vehicles to ensure a safe and comfortable driving experience. Hundreds of thousands of miles of driving tests are required to achieve a well calibrated control system that is capable of operating an autonomous vehicle in an uncertain traffic environment where multiple interactions between vehicles and drivers simultaneously occur. Traffic simulators where these interactions can be modeled and represented with reasonable fidelity can help decrease the time and effort necessary for the development of the autonomous driving control algorithms by providing a venue where acceptable initial control calibrations can be achieved quickly and safely before actual road tests. In this paper, we present a game theoretic traffic model that can be used to 1) test and compare various autonomous vehicle decision and control systems and 2) calibrate the parameters of an existing control system. We demonstrate two example case studies, where, in the first case, we test and quantitatively compare two autonomous vehicle control systems in terms of their safety and performance, and, in the second case, we optimize the parameters of an autonomous vehicle control system, utilizing the proposed traffic model and simulation environment., Comment: 13 pages, 16 figures

Recent years have seen increased interest in the design and deployment of smart grid devices and control algorithms. Each of these smart communicating devices represents a potential access point for an intruder spurring research into intruder prevention and detection. However, no security measures are complete, and intruding attackers will compromise smart grid devices leading to the attacker and the system operator interacting via the grid and its control systems. The outcome of these machine-mediated human-human interactions will depend on the design of the physical and control systems mediating the interactions. If these outcomes can be predicted via simulation, they can be used as a tool for designing attack-resilient grids and control systems. However, accurate predictions require good models of not just the physical and control systems, but also of the human decision making. In this manuscript, we present an approach to develop such tools, i.e. models of the decisions of the cyber-physical intruder who is attacking the systems and the system operator who is defending it, and demonstrate its usefulness for design., Comment: 8 pages, 7 figures, IEEE Transactions on Smart Grids pending