12 results on '"Evangelos A. Theodorou"'
Search Results
2. Spatiotemporal Costmap Inference for MPC Via Deep Inverse Reinforcement Learning
- Author
-
Keuntaek Lee, David Isele, Evangelos A. Theodorou, and Sangjae Bae
- Subjects
FOS: Computer and information sciences ,Human-Computer Interaction ,Computer Science - Robotics ,Artificial Intelligence (cs.AI) ,Control and Optimization ,Computer Science - Artificial Intelligence ,Artificial Intelligence ,Control and Systems Engineering ,Mechanical Engineering ,Biomedical Engineering ,Computer Vision and Pattern Recognition ,Robotics (cs.RO) ,Computer Science Applications - Abstract
It can be difficult to autonomously produce driver behavior so that it appears natural to other traffic participants. Through Inverse Reinforcement Learning (IRL), we can automate this process by learning the underlying reward function from human demonstrations. We propose a new IRL algorithm that learns a goal-conditioned spatiotemporal reward function. The resulting costmap is used by Model Predictive Controllers (MPCs) to perform a task without any hand-designing or hand-tuning of the cost function. We evaluate our proposed Goal-conditioned SpatioTemporal Zeroing Maximum Entropy Deep IRL (GSTZ)-MEDIRL framework together with MPC in the CARLA simulator for autonomous driving, lane keeping, and lane changing tasks in a challenging dense traffic highway scenario. Our proposed methods show higher success rates compared to other baseline methods including behavior cloning, state-of-the-art RL policies, and MPC with a learning-based behavior prediction model., Comment: IEEE Robotics and Automation Letters (RA-L)
- Published
- 2022
3. Safety Embedded Differential Dynamic Programming Using Discrete Barrier States
- Author
-
Hassan Almubarak, Kyle Stachowicz, Nader Sadegh, and Evangelos A. Theodorou
- Subjects
FOS: Computer and information sciences ,Control and Optimization ,Mechanical Engineering ,Biomedical Engineering ,Systems and Control (eess.SY) ,Electrical Engineering and Systems Science - Systems and Control ,Computer Science Applications ,Human-Computer Interaction ,Computer Science - Robotics ,Artificial Intelligence ,Control and Systems Engineering ,FOS: Electrical engineering, electronic engineering, information engineering ,Computer Vision and Pattern Recognition ,Robotics (cs.RO) - Abstract
Certified safe control is a growing challenge in robotics, especially when performance and safety objectives must be concurrently achieved. In this work, we extend the barrier state (BaS) concept, recently proposed for safe stabilization of continuous time systems, to safety embedded trajectory optimization for discrete time systems using discrete barrier states (DBaS). The constructed DBaS is embedded into the discrete model of the safety-critical system integrating safety objectives into the system's dynamics and performance objectives. Thereby, the control policy is directly supplied by safety-critical information through the barrier state. This allows us to employ the DBaS with differential dynamic programming (DDP) to plan and execute safe optimal trajectories. The proposed algorithm is leveraged on various safety-critical control and planning problems including a differential wheeled robot safe navigation in randomized and complex environments and on a quadrotor to safely perform reaching and tracking tasks. The DBaS-based DDP (DBaS-DDP) is shown to consistently outperform penalty methods commonly used to approximate constrained DDP problems as well as CBF-based safety filters., Comment: Added extensive quantitative comparisons and analysis in the implementation examples, and revised discussions and illustrations
- Published
- 2022
4. Safety Embedded Control of Nonlinear Systems via Barrier States
- Author
-
Hassan Almubarak, Evangelos A. Theodorou, and Nader Sadegh
- Subjects
Equilibrium point ,Control and Optimization ,Computer science ,Hamilton–Jacobi–Bellman equation ,Systems and Control (eess.SY) ,Optimal control ,Electrical Engineering and Systems Science - Systems and Control ,Controllability ,Nonlinear system ,Control and Systems Engineering ,Control theory ,Control system ,Stability theory ,Full state feedback ,FOS: Electrical engineering, electronic engineering, information engineering - Abstract
In many safety-critical control systems, possibly opposing safety restrictions and control performance objectives arise. To confront such a conflict, this letter proposes a novel methodology that embeds safety into stability of control systems. The development enforces safety by means of barrier functions used in optimization through the construction of barrier states (BaS) which are embedded in the control system's model. As a result, as long as the equilibrium point of interest of the closed loop system is asymptotically stable, the generated trajectories are guaranteed to be safe. Consequently, a conflict between control objectives and safety constraints is substantially avoided. To show the efficacy of the proposed technique, we employ barrier states with the simple pole placement method to design safe linear controls. Nonlinear optimal control is subsequently employed to fulfill safety, stability and performance objectives by solving the associated Hamilton-Jacobi-Bellman (HJB) which minimizes a cost functional that can involve the BaS. Following this further, we exploit optimal control with barrier states on an unstable, constrained second dimensional pendulum on a cart model that is desired to avoid low velocities regions where the system may exhibit some controllability loss and on two mobile robots to safely arrive to opposite targets with an obstacle on the way., Updates: Corrected typos and added clarifying equations and discussions
- Published
- 2022
5. Discrete-Time Differential Dynamic Programming on Lie Groups: Derivation, Convergence Analysis, and Numerical Results
- Author
-
George I. Boutselis and Evangelos A. Theodorou
- Subjects
Dynamic programming ,Quadratic equation ,Discrete time and continuous time ,Control and Systems Engineering ,Computer science ,Convergence (routing) ,Applied mathematics ,Lie group ,Differential dynamic programming ,Electrical and Electronic Engineering ,Element (category theory) ,Optimal control ,Computer Science Applications - Abstract
We develop a discrete-time optimal control framework for systems evolving on Lie groups. Our article generalizes the original differential dynamic programming method, by employing a coordinate-free, Lie-theoretic approach for its derivation. A key element lies, specifically, in the use of quadratic expansion schemes for cost functions and dynamics defined on Lie groups. The obtained algorithm iteratively optimizes local approximations of the control problem, until reaching a (sub)optimal solution. On the theoretical side, we also study the conditions under which convergence is attained. Details about the behavior and implementation of our method are provided through a simulated example on $TSO(3)$ .
- Published
- 2021
6. Schrödinger Approach to Optimal Control of Large-Size Populations
- Author
-
David D. Fan, Kaivalya Bakshi, and Evangelos A. Theodorou
- Subjects
Stochastic control ,0209 industrial biotechnology ,Partial differential equation ,Stochastic process ,02 engineering and technology ,Function (mathematics) ,Optimal control ,Action (physics) ,Computer Science Applications ,Nonlinear system ,Mathematics - Analysis of PDEs ,020901 industrial engineering & automation ,Control and Systems Engineering ,Applied mathematics ,Mathematics - Dynamical Systems ,Electrical and Electronic Engineering ,Langevin dynamics ,Mathematics - Optimization and Control ,Nonlinear Sciences - Cellular Automata and Lattice Gases ,Mathematical Physics - Abstract
Large-size populations consisting of a continuum of identical and non-cooperative agents with stochastic dynamics are useful in modeling various biological and engineered systems. This paper addresses the stochastic control problem of designing optimal state-feedback controllers which guarantee the closed-loop stability of the stationary density of such agents with nonlinear Langevin dynamics, under the action of their individual steady state controls. We represent the corresponding coupled forward-backward PDEs as decoupled Schr\"odinger equations, by applying two variable transforms. Spectral properties of the linear Schr\"odinger operator which underlie the stability analysis are used to obtain explicit control design constraints. Our interpretation of the Schr\"odinger potential as the cost function of a closely related optimal control problem motivates a quadrature based algorithm to compute the finite-time optimal control., Comment: 21 pages, 2 figures
- Published
- 2021
7. Aggressive Perception-Aware Navigation Using Deep Optical Flow Dynamics and PixelMPC
- Author
-
Jason Gibson, Keuntaek Lee, and Evangelos A. Theodorou
- Subjects
FOS: Computer and information sciences ,Computer Science - Machine Learning ,0209 industrial biotechnology ,Control and Optimization ,Computer science ,Computer Vision and Pattern Recognition (cs.CV) ,Computer Science - Computer Vision and Pattern Recognition ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,Biomedical Engineering ,Optical flow ,Systems and Control (eess.SY) ,02 engineering and technology ,010501 environmental sciences ,Visual servoing ,Electrical Engineering and Systems Science - Systems and Control ,01 natural sciences ,Machine Learning (cs.LG) ,Computer Science::Robotics ,Computer Science - Robotics ,020901 industrial engineering & automation ,Artificial Intelligence ,FOS: Electrical engineering, electronic engineering, information engineering ,Simulation ,0105 earth and related environmental sciences ,Mechanical Engineering ,Visibility (geometry) ,Pipeline (software) ,Computer Science Applications ,Human-Computer Interaction ,Model predictive control ,Control and Systems Engineering ,Trajectory ,Eye tracking ,Robot ,Computer Vision and Pattern Recognition ,Robotics (cs.RO) - Abstract
Recently, vision-based control has gained traction by leveraging the power of machine learning. In this work, we couple a model predictive control (MPC) framework to a visual pipeline. We introduce deep optical flow (DOF) dynamics, which is a combination of optical flow and robot dynamics. Using the DOF dynamics, MPC explicitly incorporates the predicted movement of relevant pixels into the planned trajectory of a robot. Our implementation of DOF is memory-efficient, data-efficient, and computationally cheap so that it can be computed in real-time for use in an MPC framework. The suggested Pixel Model Predictive Control (PixelMPC) algorithm controls the robot to accomplish a high-speed racing task while maintaining visibility of the important features (gates). This improves the reliability of vision-based estimators for localization and can eventually lead to safe autonomous flight. The proposed algorithm is tested in a photorealistic simulation with a high-speed drone racing task.
- Published
- 2020
8. On Mean Field Games for Agents With Langevin Dynamics
- Author
-
Piyush Grover, Kaivalya Bakshi, and Evangelos A. Theodorou
- Subjects
education.field_of_study ,Control and Optimization ,Partial differential equation ,Computer Networks and Communications ,Computer science ,Population ,Detailed balance ,Fixed point ,Optimal control ,Nonlinear system ,Mean field theory ,Control and Systems Engineering ,Signal Processing ,Applied mathematics ,Langevin dynamics ,education - Abstract
Mean field games (MFG) have emerged as a viable tool in the analysis of large-scale self-organizing networked systems. In particular, MFGs provide a game-theoretic optimal control interpretation of the emergent behavior of noncooperative agents. The purpose of this paper is to study MFG models in which individual agents obey multidimensional nonlinear Langevin dynamics, and analyze the closed-loop stability of fixed points of the corresponding coupled forward-backward partial differential equation (PDE) systems. In such MFG models, the detailed balance property of the reversible diffusions underlies the perturbation dynamics of the forward–backward system. We use our approach to analyze closed-loop stability of two specific models. Explicit control design constraints, which guarantee stability, are obtained for a population distribution model and a mean consensus model. We also show that static state feedback using the steady-state controller can be employed to locally stabilize an MFG equilibrium.
- Published
- 2019
9. Vision-Based High-Speed Driving With a Deep Dynamic Observer
- Author
-
Brian Goldfain, James M. Rehg, Grady Williams, Evangelos A. Theodorou, and Paul Drews
- Subjects
0209 industrial biotechnology ,Control and Optimization ,Observer (quantum physics) ,Computer science ,Biomedical Engineering ,02 engineering and technology ,Convolutional neural network ,Vehicle dynamics ,020901 industrial engineering & automation ,Artificial Intelligence ,Inertial measurement unit ,0202 electrical engineering, electronic engineering, information engineering ,Computer vision ,Artificial neural network ,business.industry ,Mechanical Engineering ,Deep learning ,Computer Science Applications ,Human-Computer Interaction ,Model predictive control ,Control and Systems Engineering ,020201 artificial intelligence & image processing ,Computer Vision and Pattern Recognition ,Artificial intelligence ,business ,Particle filter - Abstract
In this letter, we present a framework for combining deep learning-based road detection, particle filters, and model predictive control (MPC) to drive aggressively using only a monocular camera, IMU, and wheel speed sensors. This framework uses deep convolutional neural networks combined with LSTMs to learn a local cost map representation of the track in front of the vehicle. A particle filter uses this dynamic observation model to localize in a schematic map, and MPC is used to drive aggressively using this particle filter based state estimate. We show extensive real world testing results and demonstrate reliable operation of the vehicle at the friction limits on a complex dirt track. We reach speeds above 27 m/h (12 m/s) on a dirt track with a 105 ft (32 m) long straight using our 1:5 scale test vehicle.
- Published
- 2019
10. Efficient Reinforcement Learning via Probabilistic Trajectory Optimization
- Author
-
George I. Boutselis, Yunpeng Pan, and Evangelos A. Theodorou
- Subjects
Mathematical optimization ,Computer Networks and Communications ,Computer science ,Gaussian ,Probabilistic logic ,02 engineering and technology ,Trajectory optimization ,010501 environmental sciences ,Optimal control ,01 natural sciences ,Stationary point ,Computer Science Applications ,Dynamic programming ,symbols.namesake ,Artificial Intelligence ,0202 electrical engineering, electronic engineering, information engineering ,symbols ,Reinforcement learning ,020201 artificial intelligence & image processing ,Differential dynamic programming ,Gaussian process ,Software ,0105 earth and related environmental sciences - Abstract
We present a trajectory optimization approach to reinforcement learning in continuous state and action spaces, called probabilistic differential dynamic programming (PDDP). Our method represents systems dynamics using Gaussian processes (GPs), and performs local dynamic programming iteratively around a nominal trajectory in Gaussian belief spaces. Different from model-based policy search methods, PDDP does not require a policy parameterization and learns a time-varying control policy via successive forward-backward sweeps. A convergence analysis of the iterative scheme is given, showing that our algorithm converges to a stationary point globally under certain conditions. We show that prior model knowledge can be incorporated into the proposed framework to speed up learning, and a generalized optimization criterion based on the predicted cost distribution can be employed to enable risk-sensitive learning. We demonstrate the effectiveness and efficiency of the proposed algorithm using nontrivial tasks. Compared with a state-of-the-art GP-based policy search method, PDDP offers a superior combination of learning speed, data efficiency, and applicability.
- Published
- 2018
11. Reinforcement Learning and Synergistic Control of the ACT Hand
- Author
-
Evangelos A. Theodorou, Yoky Matsuoka, Emo Todorov, Mark Malhotra, and Eric Rombokas
- Subjects
Engineering ,business.industry ,Control engineering ,Index finger ,Space (commercial competition) ,Computer Science Applications ,Task (project management) ,medicine.anatomical_structure ,Control and Systems Engineering ,medicine ,Reinforcement learning ,Robot ,Motion planning ,Electrical and Electronic Engineering ,Control (linguistics) ,business ,Curse of dimensionality - Abstract
Tendon-driven systems are ubiquitous in biology and provide considerable advantages for robotic manipulators, but control of these systems is challenging because of the increase in dimensionality and intrinsic nonlinearities. Researchers in biological movement control have suggested that the brain may employ “muscle synergies” to make planning, control, and learning more tractable by expressing the tendon space in a lower dimensional virtual synergistic space. We employ synergies that respect the differing constraints of actuation and sensation, and apply path integral reinforcement learning in the virtual synergistic space as well as the full tendon space. Path integral reinforcement learning has been used successfully on torque-driven systems to learn episodic tasks without using explicit models, which is particularly important for difficult-to-model dynamics like tendon networks and contact transitions. We show that optimizing a small number of trajectories in virtual synergy space can produce comparable performance to optimizing the trajectories of the tendons individually. The six tendons of the index finger and eight tendons of the thumb, each actuating four degrees of joint freedom, are used to slide a switch and turn a knob. The learned control strategies provide a method for discovery of novel task strategies and system phenomena without explicitly modeling the physics of the robot and environment.
- Published
- 2013
12. Computational Models for Neuromuscular Function
- Author
-
Heiko Hoffmann, Evangelos A. Theodorou, Francisco J. Valero-Cuevas, Jason J. Kutch, and M. U. Kurse
- Subjects
Computational model ,business.industry ,Computer science ,Biomedical Engineering ,Experimental data ,Statistical model ,Machine learning ,computer.software_genre ,Article ,Neuromuscular stimulation ,Leverage (statistics) ,Experimental work ,Statistical analysis ,Artificial intelligence ,Neuromuscular control ,business ,computer - Abstract
Computational models of the neuromuscular system hold the potential to allow us to reach a deeper understanding of neuromuscular function and clinical rehabilitation by complementing experimentation. By serving as a means to distill and explore specific hypotheses, computational models emerge from prior experimental data and motivate future experimental work. Here we review computational tools used to understand neuromuscular function including musculoskeletal modeling, machine learning, control theory, and statistical model analysis. We conclude that these tools, when used in combination, have the potential to further our understanding of neuromuscular function by serving as a rigorous means to test scientific hypotheses in ways that complement and leverage experimental data.
- Published
- 2009
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.