Search

Due to its state-of-the-art estimation performance complemented by rigorous and non-conservative uncertainty bounds, Gaussian process regression is a popular tool for enhancing dynamical system models and coping with their inaccuracies. This has enabled a plethora of successful implementations of Gaussian process-based model predictive control in a variety of applications over the last years. However, despite its evident practical effectiveness, there are still many open questions when attempting to analyze the associated optimal control problem theoretically and to exploit the full potential of Gaussian process regression in view of safe learning-based control. The contribution of this review is twofold. The first is to survey the available literature on the topic, highlighting the major theoretical challenges such as (i) addressing scalability issues of Gaussian process regression; (ii) taking into account the necessary approximations to obtain a tractable MPC formulation; (iii) including online model updates to refine the dynamics description, exploiting data collected during operation. The second is to provide an extensive discussion of future research directions, collecting results on uncertainty quantification that are related to (but yet unexploited in) optimal control, among others. Ultimately, this paper provides a toolkit to study and advance Gaussian process-based model predictive control.

We consider the problem of optimizing the economic performance of nonlinear constrained systems subject to uncertain time-varying parameters and bounded disturbances. In particular, we propose an adaptive economic model predictive control (MPC) framework that: (i) directly minimizes transient economic costs, (ii) addresses parametric uncertainty through online model adaptation, (iii) determines optimal setpoints online, and (iv) ensures robustness by using a tube-based approach. The proposed design ensures recursive feasibility, robust constraint satisfaction, and a transient performance bound. In case the disturbances have a finite energy and the parameter variations have a finite path length, the asymptotic average performance is (approximately) not worse than the performance obtained when operating at the best reachable steady-state. We highlight performance benefits in a numerical example involving a chemical reactor with unknown time-invariant and time-varying parameters.

Incorporating learning-based models, such as Gaussian processes (GPs), into model predictive control (MPC) strategies can significantly improve control performance and online adaptation capabilities for real-world applications. Still, despite recent advances in numerical optimization and real-time GP inference, its widespread application is limited by the lack of an efficient and modular open-source implementation. This work aims at filling this gap by providing an efficient implementation of zero-order Gaussian process-based MPC in acados, as well as L4acados, a general framework for incorporating non-CasADi (learning-based) residual models in acados. By providing the required sensitivities via a user-defined Python module, L4acados enables the implementation of MPC controllers with learning-based residual models in acados, while supporting custom Jacobian approximations, as well as parallelization of sensitivity computations when preparing the quadratic subproblems. The computational efficiency of L4acados is benchmarked against available software using a neural network-based control example. Last, it is used demonstrate the performance of the zero-order GP-MPC method applied to two hardware examples: autonomous miniature racing, as well as motion control of a full-scale autonomous vehicle for an ISO lane change maneuver.

This paper presents a robust moving horizon estimation (MHE) approach with provable estimation error bounds for solving the simultaneous localization and mapping (SLAM) problem. We derive sufficient conditions to guarantee robust stability in ego-state estimates and bounded errors in landmark position estimates, even under limited landmark visibility which directly affects overall system detectability. This is achieved by decoupling the MHE updates for the ego-state and landmark positions, enabling individual landmark updates only when the required detectability conditions are met. The decoupled MHE structure also allows for parallelization of landmark updates, improving computational efficiency. We discuss the key assumptions, including ego-state detectability and Lipschitz continuity of the landmark measurement model, with respect to typical SLAM sensor configurations, and introduce a streamlined method for the range measurement model. Simulation results validate the considered method, highlighting its efficacy and robustness to noise., Comment: 8 pages, 3 figures

We propose integrating an explicit approximation of a predictive control barrier function (PCBF) in a safety filter framework. The approximated PCBF is implicitly defined through an optimal control problem and allows guaranteeing invariance of an implicitly defined safe set as well as stability of this safe set within a larger domain of attraction. By extending existing theoretical analysis of the PCBF, we establish inherent robustness of the original algorithm and translate the guarantees to input-to-state stability of the proposed algorithm with respect to possible approximation errors, recovering the same guarantees in the absence of approximation errors. The proposed algorithm allows certifying inputs with respect to state constraint satisfaction through a single function evaluation and filtering unsafe inputs through a control barrier function based safety filter, which is independent of the time horizon of the original predictive optimisation problem, resulting in significant online computational benefits. We demonstrate the stability properties of the proposed algorithm on a linear system example as well as its use a fast safety filter for miniature race cars in simulation.

Rank collapse, a phenomenon where embedding vectors in sequence models rapidly converge to a uniform token or equilibrium state, has recently gained attention in the deep learning literature. This phenomenon leads to reduced expressivity and potential training instabilities due to vanishing gradients. Empirical evidence suggests that architectural components like skip connections, LayerNorm, and MultiLayer Perceptrons (MLPs) play critical roles in mitigating rank collapse. While this issue is well-documented for transformers, alternative sequence models, such as State Space Models (SSMs), which have recently gained prominence, have not been thoroughly examined for similar vulnerabilities. This paper extends the theory of rank collapse from transformers to SSMs using a unifying framework that captures both architectures. We study how a parametrized version of the classic skip connection component, which we call \emph{lambda-skip connections}, provides guarantees for rank collapse prevention. Through analytical results, we present a sufficient condition to guarantee prevention of rank collapse across all the aforementioned architectures. We also study the necessity of this condition via ablation studies and analytical examples. To our knowledge, this is the first study that provides a general guarantee to prevent rank collapse, and that investigates rank collapse in the context of SSMs, offering valuable understanding for both theoreticians and practitioners. Finally, we validate our findings with experiments demonstrating the crucial role of architectural components such as skip connections and gating mechanisms in preventing rank collapse.

We propose a novel data-driven stochastic model predictive control framework for uncertain linear systems with noisy output measurements. Our approach leverages multi-step predictors to efficiently propagate uncertainty, ensuring chance constraint satisfaction with minimal conservatism. In particular, we present a strategy to identify multi-step predictors and quantify the associated uncertainty using a surrogate (data-driven) state space model. Then, we utilize the derived distribution to formulate a constraint tightening that ensures chance constraint satisfaction. A numerical example highlights the reduced conservatism of handling parametric uncertainty in the proposed method compared to state-of-the-art solutions.

Learning uncertain dynamics models using Gaussian process~(GP) regression has been demonstrated to enable high-performance and safety-aware control strategies for challenging real-world applications. Yet, for computational tractability, most approaches for Gaussian process-based model predictive control (GP-MPC) are based on approximations of the reachable set that are either overly conservative or impede the controller's safety guarantees. To address these challenges, we propose a robust GP-MPC formulation that guarantees constraint satisfaction with high probability. For its tractable implementation, we propose a sampling-based GP-MPC approach that iteratively generates consistent dynamics samples from the GP within a sequential quadratic programming framework. We highlight the improved reachable set approximation compared to existing methods, as well as real-time feasible computation times, using two numerical examples., Comment: to be published in 63rd IEEE Conference on Decision and Control (CDC 2024)

One of the most challenging tasks in control theory is arguably the design of a regulator for nonlinear systems when the dynamics are unknown. To tackle it, a popular strategy relies on finding a direct map between system responses and the controller, and the key ingredient is a predictor for system outputs trained on past trajectories. Focusing on continuous-time, input-affine systems, we show that the so-called signature transform provides rigorous and practically effective features to represent and predict system trajectories. Building upon such a tool, we propose a novel signature-based control strategy that is promising in view of data-driven predictive control., Comment: Submitted to IEEE Control Systems Letters

MICADO is a first light instrument for the Extremely Large Telescope (ELT), set to start operating later this decade. It will provide diffraction limited imaging, astrometry, high contrast imaging, and long slit spectroscopy at near-infrared wavelengths. During the initial phase operations, adaptive optics (AO) correction will be provided by its own natural guide star wavefront sensor. In its final configuration, that AO system will be retained and complemented by the laser guide star multi-conjugate adaptive optics module MORFEO (formerly known as MAORY). Among many other things, MICADO will study exoplanets, distant galaxies and stars, and investigate black holes, such as Sagittarius A* at the centre of the Milky Way. After their final design phase, most components of MICADO have moved on to the manufacturing and assembly phase. Here we summarize the final design of the instrument and provide an overview about its current manufacturing status and the timeline. Some lessons learned from the final design review process will be presented in order to help future instrumentation projects to cope with the challenges arising from the substantial differences between projects for 8-10m class telescopes (e.g. ESO-VLT) and the next generation Extremely Large Telescopes (e.g. ESO-ELT). Finally, the expected performance will be discussed in the context of the current landscape of astronomical observatories and instruments. For instance, MICADO will have similar sensitivity as the James Webb Space Telescope (JWST), but with six times the spatial resolution., Comment: Proceedings of the SPIE, Volume 13096, id. 1309611 11 pp. (2024)

Direct volume rendering using ray-casting is widely used in practice. By using GPUs and applying acceleration techniques as empty space skipping, high frame rates are possible on modern hardware. This enables performance-critical use-cases such as virtual reality volume rendering. The currently fastest known technique uses volumetric distance maps to skip empty sections of the volume during ray-casting but requires the distance map to be updated per transfer function change. In this paper, we demonstrate a technique for subdividing the volume intensity range into partitions and deriving what we call partitioned distance maps. These can be used to accelerate the distance map computation for a newly changed transfer function by a factor up to 30. This allows the currently fastest known empty space skipping approach to be used while maintaining high frame rates even when the transfer function is changed frequently., Comment: 5 pages, 5 figures, 1 table, presented at IEEE VIS 2024, for associated mp4 file, see https://osf.io/n5k6z ; source code available at https://github.com/CalamityMichL/DVR_ESS_PDM

We introduce data to predictive control, D2PC, a framework to facilitate the design of robust and predictive controllers from data. The proposed framework is designed for discrete-time stochastic linear systems with output measurements and provides a principled design of a predictive controller based on data. The framework starts with a parameter identification method based on the Expectation-Maximization algorithm, which incorporates pre-defined structural constraints. Additionally, we provide an asymptotically correct method to quantify uncertainty in parameter estimates. Next, we develop a strategy to synthesize robust dynamic output-feedback controllers tailored to the derived uncertainty characterization. Finally, we introduce a predictive control scheme that guarantees recursive feasibility and satisfaction of chance constraints. This framework marks a significant advancement in integrating data into robust and predictive control schemes. We demonstrate the efficacy of D2PC through a numerical example involving a $10$-dimensional spring-mass-damper system., Comment: Code link: https://github.com/haldunbalim/D2PC

We present a stochastic model predictive control framework for nonlinear systems subject to unbounded process noise with closed-loop guarantees. First, we provide a conceptual shrinking-horizon framework that utilizes general probabilistic reachable sets and minimizes the expected cost. Then, we provide a tractable receding-horizon formulation that uses a nominal state to minimize a deterministic quadratic cost and satisfy tightened constraints. Our theoretical analysis demonstrates recursive feasibility, satisfaction of chance constraints, and bounds on the expected cost for the resulting closed-loop system. We provide a constructive design for probabilistic reachable sets of nonlinear continuously differentiable systems using stochastic contraction metrics. Numerical simulations highlight the computational efficiency and theoretical guarantees of the proposed method. Overall, this paper provides a framework for computationally tractable stochastic predictive control with closed-loop guarantees for nonlinear systems with unbounded noise., Comment: Code: https://gitlab.ethz.ch/ics/SMPC-CCM Update: added numerical comparisons to sampling-based approaches; included nonlinear constraints; streamlined design

Tube-based model predictive control (MPC) is the principal robust control technique for constrained linear systems affected by additive disturbances. While tube-based methods that compute the tubes online have been successfully applied to systems with additive disturbances, their application to systems affected by additional model uncertainties is challenging. This paper introduces a new tube-based MPC method - named filter-based system level tube-MPC (SLTMPC) - which overapproximates both uncertainties with an online optimized disturbance set, while simultaneously computing the tube controller online. Extending prior work, we generalize the method to polytopic disturbance sets and for the first time provide rigorous closed-loop guarantees for the receding horizon controller. These guarantees are obtained by virtue of a new terminal controller design and an online optimized terminal set. To reduce the computational complexity of the proposed method, we additionally introduce an asynchronous computation scheme that separates the optimization of the tube controller and the nominal trajectory. Finally, we provide a comprehensive numerical evaluation of the proposed methods to demonstrate their effectiveness.

This paper provides a comprehensive tutorial on a family of Model Predictive Control (MPC) formulations, known as MPC for tracking, which are characterized by including an artificial reference as part of the decision variables in the optimization problem. These formulations have several benefits with respect to the classical MPC formulations, including guaranteed recursive feasibility under online reference changes, as well as asymptotic stability and an increased domain of attraction. This tutorial paper introduces the concept of using an artificial reference in MPC, presenting the benefits and theoretical guarantees obtained by its use. We then provide a survey of the main advances and extensions of the original linear MPC for tracking, including its non-linear extension. Additionally, we discuss its application to learning-based MPC, and discuss optimization aspects related to its implementation., Comment: (15 pages, 1 figure)

Softmax attention is the principle backbone of foundation models for various artificial intelligence applications, yet its quadratic complexity in sequence length can limit its inference throughput in long-context settings. To address this challenge, alternative architectures such as linear attention, State Space Models (SSMs), and Recurrent Neural Networks (RNNs) have been considered as more efficient alternatives. While connections between these approaches exist, such models are commonly developed in isolation and there is a lack of theoretical understanding of the shared principles underpinning these architectures and their subtle differences, greatly influencing performance and scalability. In this paper, we introduce the Dynamical Systems Framework (DSF), which allows a principled investigation of all these architectures in a common representation. Our framework facilitates rigorous comparisons, providing new insights on the distinctive characteristics of each model class. For instance, we compare linear attention and selective SSMs, detailing their differences and conditions under which both are equivalent. We also provide principled comparisons between softmax attention and other model classes, discussing the theoretical conditions under which softmax attention can be approximated. Additionally, we substantiate these new insights with empirical validations and mathematical arguments. This shows the DSF's potential to guide the systematic development of future more efficient and scalable foundation models., Comment: NeurIPS 2024

Despite its amazing quantitative successes and contributions to revolutionary technologies, physics currently faces many unsolved mysteries ranging from the meaning of quantum mechanics to the nature of the dark energy that will determine the future of the Universe. It is clearly prohibitive for the general reader, and even the best informed physicists, to follow the vast number of technical papers published in the thousands of specialized journals. For this reason, we have asked the leading experts across many of the most important areas of physics to summarise their global assessment of some of the most important issues. In lieu of an extremely long abstract summarising the contents, we invite the reader to look at the section headings and their authors, and then to indulge in a feast of stimulating topics spanning the current frontiers of fundamental physics from The Future of Physics by William D Phillips and What characterises topological effects in physics? by Gerard t Hooft through the contributions of the widest imaginable range of world leaders in their respective areas. This paper is presented as a preface to exciting developments by senior and young scientists in the years that lie ahead, and a complement to the less authoritative popular accounts by journalists., Comment: 54 pages, 13 figures

This paper presents an open-source miniature car-like robot with low-cost sensing and a pipeline for optimization-based system identification, state estimation, and control. The overall robotics platform comes at a cost of less than \$\,700 and thus significantly simplifies the verification of advanced algorithms in a realistic setting. We present a modified bicycle model with Pacejka tire forces to model the dynamics of the considered all-wheel drive vehicle and to prevent singularities of the model at low velocities. Furthermore, we provide an optimization-based system identification approach and a moving horizon estimation (MHE) scheme. In extensive hardware experiments, we show that the presented system identification approach results in a model with high prediction accuracy, while the MHE results in accurate state estimates. Finally, the overall closed-loop system is shown to perform well even in the presence of sensor failure for limited time intervals. All hardware, firmware, and control and estimation software is released under a BSD 2-clause license to promote widespread adoption and collaboration within the community., Comment: This version contains an additional appendix giving an overview of the software, hardware and firmware

We propose a novel gradient-based online optimization framework for solving stochastic programming problems that frequently arise in the context of cyber-physical and robotic systems. Our problem formulation accommodates constraints that model the evolution of a cyber-physical system, which has, in general, a continuous state and action space, is nonlinear, and where the state is only partially observed. We also incorporate an approximate model of the dynamics as prior knowledge into the learning process and show that even rough estimates of the dynamics can significantly improve the convergence of our algorithms. Our online optimization framework encompasses both gradient descent and quasi-Newton methods, and we provide a unified convergence analysis of our algorithms in a non-convex setting. We also characterize the impact of modeling errors in the system dynamics on the convergence rate of the algorithms. Finally, we evaluate our algorithms in simulations of a flexible beam, a four-legged walking robot, and in real-world experiments with a ping-pong playing robot., Comment: 46 pages, 16 figures

In Model Predictive Control (MPC), discrepancies between the actual system and the predictive model can lead to substantial tracking errors and significantly degrade performance and reliability. While such discrepancies can be alleviated with more complex models, this often complicates controller design and implementation. By leveraging the fact that many trajectories of interest are periodic, we show that perfect tracking is possible when incorporating a simple observer that estimates and compensates for periodic disturbances. We present the design of the observer and the accompanying tracking MPC scheme, proving that their combination achieves zero tracking error asymptotically, regardless of the complexity of the unmodelled dynamics. We validate the effectiveness of our method, demonstrating asymptotically perfect tracking on a high-dimensional soft robot with nearly 10,000 states and a fivefold reduction in tracking errors compared to a baseline MPC on small-scale autonomous race car experiments., Comment: 8 pages, 3 figures; 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024)

We propose a field-free switching mechanism that utilizes two spatially orthogonal spin-orbit torque (SOT) currents. Initially applied simultaneously, one of the currents is subsequently switched off. The superposition of these two currents results in an in-plane magnetization, which is not orthogonal to the remaining SOT current after the second one is deactivated. This symmetry-breaking procedure leads to reproducible and rapid switching, with field pulse durations as short as 0.25 nanoseconds.

We present a model predictive control (MPC) formulation to directly optimize economic criteria for linear constrained systems subject to disturbances and uncertain model parameters. The proposed formulation combines a certainty equivalent economic MPC with a simple least-squares parameter adaptation. For the resulting adaptive economic MPC scheme, we derive strong asymptotic and transient performance guarantees. We provide a numerical example involving building temperature control and demonstrate performance benefits of online parameter adaptation., Comment: Final version, IEEE Conference on Decision and Control (CDC), 2024

Quadrotor flight is an extremely challenging problem due to the limited control authority encountered at the limit of handling. Model Predictive Contouring Control (MPCC) has emerged as a promising model-based approach for time optimization problems such as drone racing. However, the standard MPCC formulation used in quadrotor racing introduces the notion of the gates directly in the cost function, creating a multi objective optimization that continuously trades off between maximizing progress and tracking the path accurately. This paper introduces three key components that enhance the state-of-the-art MPCC approach for drone racing. First and foremost, we provide safety guarantees in the form of a track constraint and terminal set. The track constraint is designed as a spatial constraint which prevents gate collisions while allowing for time optimization only in the cost function. Second, we augment the existing first principles dynamics with a residual term that captures complex aerodynamic effects and thrust forces learned directly from real-world data. Third, we use Trust Region Bayesian Optimization (TuRBO), a state-of-the-art global Bayesian Optimization algorithm, to tune the hyperparameters of the MPCC controller given a sparse reward based on lap time minimization. The proposed approach achieves similar lap times to the best-performing RL policy and outperforms the best model-based controller while satisfying constraints. In both simulation and real world, our approach consistently prevents gate crashes with 100% success rate, while pushing the quadrotor to its physical limits reaching speeds of more than 80km/h., Comment: 12 pages, 6 figures

In recent years, there has been a growing interest in integrating linear state-space models (SSM) in deep neural network architectures of foundation models. This is exemplified by the recent success of Mamba, showing better performance than the state-of-the-art Transformer architectures in language tasks. Foundation models, like e.g. GPT-4, aim to encode sequential data into a latent space in order to learn a compressed representation of the data. The same goal has been pursued by control theorists using SSMs to efficiently model dynamical systems. Therefore, SSMs can be naturally connected to deep sequence modeling, offering the opportunity to create synergies between the corresponding research areas. This paper is intended as a gentle introduction to SSM-based architectures for control theorists and summarizes the latest research developments. It provides a systematic review of the most successful SSM proposals and highlights their main features from a control theoretic perspective. Additionally, we present a comparative analysis of these models, evaluating their performance on a standardized benchmark designed for assessing a model's efficiency at learning long sequences.

Safely exploring environments with a-priori unknown constraints is a fundamental challenge that restricts the autonomy of robots. While safety is paramount, guarantees on sufficient exploration are also crucial for ensuring autonomous task completion. To address these challenges, we propose a novel safe guaranteed exploration framework using optimal control, which achieves first-of-its-kind results: guaranteed exploration for non-linear systems with finite time sample complexity bounds, while being provably safe with arbitrarily high probability. The framework is general and applicable to many real-world scenarios with complex non-linear dynamics and unknown domains. Based on this framework we propose an efficient algorithm, SageMPC, SAfe Guaranteed Exploration using Model Predictive Control. SageMPC improves efficiency by incorporating three techniques: i) exploiting a Lipschitz bound, ii) goal-directed exploration, and iii) receding horizon style re-planning, all while maintaining the desired sample complexity, safety and exploration guarantees of the framework. Lastly, we demonstrate safe efficient exploration in challenging unknown environments using SageMPC with a car model.

Appropriate time discretization is crucial for real-time applications of numerical optimal control, such as nonlinear model predictive control. However, if the discretization error strongly depends on the applied control input, meeting accuracy and sampling time requirements simultaneously can be challenging using classical discretization methods. In particular, neither fixed-grid nor adaptive-grid discretizations may be suitable, when they suffer from large integration error or exceed the prescribed sampling time, respectively. In this work, we take a first step at closing this gap by utilizing probabilistic numerical integrators to approximate the solution of the initial value problem, as well as the computational uncertainty associated with it, inside the optimal control problem (OCP). By taking the viewpoint of probabilistic numerics and propagating the numerical uncertainty in the cost, the OCP is reformulated such that the optimal input reduces the computational uncertainty insofar as it is beneficial for the control objective. The proposed approach is illustrated using a numerical example, and potential benefits and limitations are discussed., Comment: to be published in the 6th Annual Learning for Dynamics & Control Conference (L4DC 2024)

System level synthesis enables improved robust MPC formulations by allowing for joint optimization of the nominal trajectory and controller. This paper introduces a tailored algorithm for solving the corresponding disturbance feedback optimization problem for linear time-varying systems. The proposed algorithm iterates between optimizing the controller and the nominal trajectory while converging q-linearly to an optimal solution. We show that the controller optimization can be solved through Riccati recursions leading to a horizon-length, state, and input scalability of $\mathcal{O}(N^2 ( n_x^3 +n_u^3))$ for each iterate. On a numerical example, the proposed algorithm exhibits computational speedups by a factor of up to $10^3$ compared to general-purpose commercial solvers., Comment: Young Author Award (finalist): IFAC Conference on Nonlinear Model Predictive Control (NMPC) 2024

Predictive safety filters provide a way of projecting potentially unsafe inputs, proposed, e.g. by a human or learning-based controller, onto the set of inputs that guarantee recursive state and input constraint satisfaction by leveraging model predictive control techniques. In this paper, we extend this framework such that in addition, robust asymptotic stability of the closed-loop system can be guaranteed by enforcing a decrease of an implicit Lyapunov function which is constructed using a predicted system trajectory. Differently from previous results, we show robust asymptotic stability with respect to a predefined disturbance set on an extended state consisting of the system state and a warmstart input sequence. The proposed strategy is applied to an automotive lane keeping example in simulation., Comment: Accepted at NMPC'24

In recent years, the increasing need for high-performance controllers in applications like autonomous driving has motivated the development of optimization routines tailored to specific control problems. In this paper, we propose an efficient inexact model predictive control (MPC) strategy for autonomous miniature racing with inherent robustness properties. We rely on a feasible sequential quadratic programming (SQP) algorithm capable of generating feasible intermediate iterates such that the solver can be stopped after any number of iterations, without jeopardizing recursive feasibility. In this way, we provide a strategy that computes suboptimal and yet feasible solutions with a computational footprint that is much lower than state-of-the-art methods based on the computation of locally optimal solutions. Under suitable assumptions on the terminal set and on the controllability properties of the system, we can state that, for any sufficiently small disturbance affecting the system's dynamics, recursive feasibility can be guaranteed. We validate the effectiveness of the proposed strategy in simulation and by deploying it onto a physical experiment with autonomous miniature race cars. Both the simulation and experimental results demonstrate that, using the feasible SQP method, a feasible solution can be obtained with moderate additional computational effort compared to strategies that resort to early termination without providing a feasible solution. At the same time, the proposed method is significantly faster than the state-of-the-art solver Ipopt.

In this paper, we study state estimation for general nonlinear systems with unknown parameters and persistent process and measurement noise. In particular, we are interested in stability properties of the state estimate in the absence of persistency of excitation (PE). With a simple academic example, we show that existing moving horizon estimation (MHE) approaches as well as classical adaptive observers can result in diverging state estimates in the absence of PE, even if the noise is small. We propose a novel MHE formulation involving a regularization based on a constant prior estimate of the unknown system parameters. Only assuming the existence of a stable estimator, we prove that the proposed MHE results in practically robustly stable state estimates even in the absence of PE. We discuss the relation of the proposed MHE formulation to state-of-the-art results from MHE, adaptive estimation, and functional estimation. The properties of the proposed MHE approach are illustrated with a numerical example of a car with unknown tire friction parameters., Comment: 13 pages, 5 figures

We consider the design of functional estimators, i.e., approaches to compute an estimate of a nonlinear function of the state of a general nonlinear dynamical system subject to process noise based on noisy output measurements. To this end, we introduce a novel functional detectability notion in the form of incremental input/output-to-output stability ($\delta$-IOOS). We show that $\delta$-IOOS is a necessary condition for the existence of a functional estimator satisfying an input-to-output type stability property. Additionally, we prove that a system is functional detectable if and only if it admits a corresponding $\delta$-IOOS Lyapunov function. Furthermore, $\delta$-IOOS is shown to be a sufficient condition for the design of a stable functional estimator by introducing the design of a full information estimation (FIE) approach for functional estimation. Together, we present a unified framework to study functional estimation with a detectability condition, which is necessary and sufficient for the existence of a stable functional estimator, and a corresponding functional estimator design. The practical need for and applicability of the proposed functional estimator design is illustrated with a numerical example of a power system., Comment: Version accepted for publication in Automatica

Safety guarantees are vital in many control applications, such as robotics. Model predictive control (MPC) provides a constructive framework for controlling safety-critical systems, but is limited by its computational complexity. We address this problem by presenting a novel algorithm that automatically computes an explicit approximation to nonlinear MPC schemes while retaining closed-loop guarantees. Specifically, the problem can be reduced to a function approximation problem, which we then tackle by proposing ALKIA-X, the Adaptive and Localized Kernel Interpolation Algorithm with eXtrapolated reproducing kernel Hilbert space norm. ALKIA-X is a non-iterative algorithm that ensures numerically well-conditioned computations, a fast-to-evaluate approximating function, and the guaranteed satisfaction of any desired bound on the approximation error. Hence, ALKIA-X automatically computes an explicit function that approximates the MPC, yielding a controller suitable for safety-critical systems and high sampling rates. We apply ALKIA-X to approximate two nonlinear MPC schemes, demonstrating reduced computational demand and applicability to realistic problems., Comment: Submitted to IEEE Transactions on Automatic Control. Compared to the previously uploaded version, this version contains an additional numerical example

Inverse optimal control (IOC) is about estimating an unknown objective of interest given its optimal control sequence. However, truly optimal demonstrations are often difficult to obtain, e.g., due to human errors or inaccurate measurements. This paper presents an IOC framework for objective estimation from multiple sub-optimal demonstrations in constrained environments. It builds upon the Karush-Kuhn-Tucker optimality conditions, and addresses the Errors-In-Variables problem that emerges from the use of sub-optimal data. The approach presented is applied to various systems in simulation, and consistency guarantees are provided for linear systems with zero mean additive noise, polytopic constraints, and objectives with quadratic features.

Robust and stochastic optimal control problem (OCP) formulations allow a systematic treatment of uncertainty, but are typically associated with a high computational cost. The recently proposed zero-order robust optimization (zoRO) algorithm mitigates the computational cost of uncertainty-aware MPC by propagating the uncertainties outside of the MPC problem. This paper details the combination of zoRO with the real-time iteration (RTI) scheme and presents an efficient open-source implementation in acados, utilizing BLASFEO for the linear algebra operations. In addition to the scaling advantages posed by the zoRO algorithm, the efficient implementation drastically reduces the computational overhead, and, combined with an RTI scheme, enables the use of tube-based MPC for a wider range of applications. The flexibility, usability and effectiveness of the proposed implementation is demonstrated on two examples. On the practical example of a differential drive robot, the proposed implementation results in a tenfold reduction of computation time with respect to the previously available zoRO implementation., Comment: 7 pages, 4 figures, submitted to ECC 2024

This paper presents a novel model order reduction technique tailored for power systems with a large share of inverter-based energy resources. Such systems exhibit an increased level of dynamic stiffness compared to traditional power systems, posing challenges for time-domain simulations and control design. Our approach involves rotation of the coordinate system of a linearized system using a transformation matrix derived from the real Jordan canonical form, leading to mode decoupling. The fast modes are then truncated in the rotated coordinate system to obtain a lower-order model with reduced stiffness. Applying the same transformation to the original nonlinear system results in an approximate separation of slow and fast states, which can be truncated to reduce the stiffness. The resulting reduced-order model demonstrates an accurate time-domain performance, the slow eigenvalues of the linearized system are correctly preserved, and a reduction in the model stiffness is achieved, allowing for accurate integration with increased step size. Our methodology is assessed in detail for a 3-bus system with generation units involving grid-forming/following converters and synchronous machines, where it allows for a computational speed-up of up to 100x compared to the original system. Several standard larger test systems are also considered., Comment: 8 pages, 5 figures, presented at PSCC 2024

We present an adaptive procedure for aligning quantum non-locality experiments without any knowledge of the two-qudit state shared by the participating parties. The quantum state produced by the source, its unitary evolution as well as the actual measurement bases remain unknown to both parties at all times. The entanglement of the quantum state helps establish desired correlations between individual measurement bases of the two distant parties. We implement the procedure in a fiber-based quantum key distribution (QKD) setup with polarization-entangled photons, where we do not rely on any additional alignment tools such as lasers or polarizers. In a QKD scenario the procedure can be done without any additional measurements as those that are performed regardless., Comment: 8 pages, 5 figures

We present a robust model predictive control (MPC) framework for linear systems facing bounded parametric uncertainty and bounded disturbances. Our approach deviates from standard MPC formulations by integrating multi-step predictors, which provide reduced error bounds. These bounds, derived from multi-step predictors, are utilized in a homothetic tube formulation to mitigate conservatism. Lastly, a multi-rate formulation is adopted to handle the incompatibilities of multi-step predictors. We provide a theoretical analysis, guaranteeing robust recursive feasibility, constraint satisfaction, and (practical) stability of the desired setpoint. We use a simulation example to compare it to existing literature and demonstrate advantages in terms of conservatism and computational complexity., Comment: Extended version of accepted paper in IEEE Control Systems Letters, 2023. Contains additional details regarding the numerical example and LMI derivation

Real-world systems are often characterized by high-dimensional nonlinear dynamics, making them challenging to control in real time. While reduced-order models (ROMs) are frequently employed in model-based control schemes, dimensionality reduction introduces model uncertainty which can potentially compromise the stability and safety of the original high-dimensional system. In this work, we propose a novel reduced-order model predictive control (ROMPC) scheme to solve constrained optimal control problems for nonlinear, high-dimensional systems. To address the challenges of using ROMs in predictive control schemes, we derive an error bounding system that dynamically accounts for model reduction error. Using these bounds, we design a robust MPC scheme that ensures robust constraint satisfaction, recursive feasibility, and asymptotic stability. We demonstrate the effectiveness of our proposed method in simulations on a high-dimensional soft robot with nearly 10,000 states., Comment: 9 pages, 3 figures, To be presented at Conference for Decision and Control 2023

In reinforcement learning (RL), rewards of states are typically considered additive, and following the Markov assumption, they are $\textit{independent}$ of states visited previously. In many important applications, such as coverage control, experiment design and informative path planning, rewards naturally have diminishing returns, i.e., their value decreases in light of similar states visited previously. To tackle this, we propose $\textit{submodular RL}$ (SubRL), a paradigm which seeks to optimize more general, non-additive (and history-dependent) rewards modelled via submodular set functions which capture diminishing returns. Unfortunately, in general, even in tabular settings, we show that the resulting optimization problem is hard to approximate. On the other hand, motivated by the success of greedy algorithms in classical submodular optimization, we propose SubPO, a simple policy gradient-based algorithm for SubRL that handles non-additive rewards by greedily maximizing marginal gains. Indeed, under some assumptions on the underlying Markov Decision Process (MDP), SubPO recovers optimal constant factor approximations of submodular bandits. Moreover, we derive a natural policy gradient approach for locally optimizing SubRL instances even in large state- and action- spaces. We showcase the versatility of our approach by applying SubPO to several applications, such as biodiversity monitoring, Bayesian experiment design, informative path planning, and coverage maximization. Our results demonstrate sample efficiency, as well as scalability to high-dimensional state-action spaces., Comment: Spotlight paper at ICLR 2024

Physics-informed machine learning (PIML) is a set of methods and tools that systematically integrate machine learning (ML) algorithms with physical constraints and abstract mathematical models developed in scientific and engineering domains. As opposed to purely data-driven methods, PIML models can be trained from additional information obtained by enforcing physical laws such as energy and mass conservation. More broadly, PIML models can include abstract properties and conditions such as stability, convexity, or invariance. The basic premise of PIML is that the integration of ML and physics can yield more effective, physically consistent, and data-efficient models. This paper aims to provide a tutorial-like overview of the recent advances in PIML for dynamical system modeling and control. Specifically, the paper covers an overview of the theory, fundamental concepts and methods, tools, and applications on topics of: 1) physics-informed learning for system identification; 2) physics-informed learning for control; 3) analysis and verification of PIML models; and 4) physics-informed digital twins. The paper is concluded with a perspective on open challenges and future research opportunities., Comment: 16 pages, 4 figures, to be published in 2023 American Control Conference (ACC)

The choice of objective is critical for the performance of an optimal controller. When control requirements vary during operation, e.g. due to changes in the environment with which the system is interacting, these variations should be reflected in the cost function. In this paper we consider the problem of identifying a time dependent cost function from given trajectories. We propose a strategy for explicitly representing time dependency in the cost function, i.e. decomposing it into the product of an unknown time dependent parameter vector and a known state and input dependent vector, modelling the former via a linear combination of trigonometric basis functions. These are incorporated within an inverse optimal control framework that uses the Karush-Kuhn-Tucker (KKT) conditions for ensuring optimality, and allows for formulating an optimization problem with respect to a finite set of basis function hyperparameters. Results are shown for two systems in simulation and evaluated against state-of-the-art approaches.

(abridged) We present simulations of halo formation and evolution in scalar field dark matter (SFDM) cosmologies in the Thomas-Fermi regime, aka ``SFDM-TF", where a strong repulsive 2-particle self-interaction (SI) is included, being a valuable alternative to CDM, with the potential to resolve its ``cusp-core" problem. In general, SFDM behaves like a quantum fluid. Previous literature has presented two fluid approximations for SFDM-TF, as well as simulations of halo formation. These results confirmed earlier expectations and are generally in mutual agreement, but discrepancies were also reported. Therefore, we perform dedicated 3D cosmological simulations for the SFDM-TF model, applying both fluid approximations, as well as for CDM. Our results are very well in accordance with previous works and extend upon them, in that we can explain the reported discrepancies as a result of different simulation setups. We find some interesting details: The evolution of both SFDM-TF and CDM halos follows a 2-stage process. In the early stage, the density profile in the center becomes close to a $(n=1.5)$-polytropic core, dominated by an "effective" velocity-dispersion pressure $P_{\sigma}$ which is common to both dark matter models. Consecutively, for CDM halos, the core transitions into a central cusp. In SFDM-TF halos, the additional pressure $P_\text{SI}$ due to SI determines the second stage of the evolution, where the central region follows closely a $(n=1)$-polytropic core, embedded in a nearly isothermal envelope, i.e. the outskirts are similar to CDM. We also encounter a new effect, namely a late-time expansion of both polytropic core plus envelope, because the size of the almost isothermal halo envelope is affected by the expansion of the background universe. So, an initial primordial core of $\sim 100$ pc can evolve into a larger core of $\gtrsim 1$ kpc, even without feedback from baryons., Comment: final version; 26 pages, 16 figures

Model predictive control (MPC) achieves stability and constraint satisfaction for general nonlinear systems, but requires computationally expensive online optimization. This paper studies approximations of such MPC controllers via neural networks (NNs) to achieve fast online evaluation. We propose safety augmentation that yields deterministic guarantees for convergence and constraint satisfaction despite approximation inaccuracies. We approximate the entire input sequence of the MPC with NNs, which allows us to verify online if it is a feasible solution to the MPC problem. We replace the NN solution by a safe candidate based on standard MPC techniques whenever it is infeasible or has worse cost. Our method requires a single evaluation of the NN and forward integration of the input sequence online, which is fast to compute on resource-constrained systems. The proposed control framework is illustrated using two numerical non-linear MPC benchmarks of different complexity, demonstrating computational speedups that are orders of magnitude higher than online optimization. In the examples, we achieve deterministic safety through the safety-augmented NNs, where a naive NN implementation fails.

In control system networks, reconfiguration of the controller when agents are leaving or joining the network is still an open challenge, in particular when operation constraints that depend on each agent's behavior must be met. Drawing our motivation from mobile robot swarms, in this paper, we address this problem by optimizing individual agent performance while guaranteeing persistent constraint satisfaction in presence of bounded communication range and time-varying network topology. The approach we propose is a model predictive control (MPC) formulation, building on multi-trajectory MPC (mt-MPC) concepts. To enable plug and play operations when the system is in closed-loop without the need of a request, the proposed MPC scheme predicts two different state trajectories in the same finite horizon optimal control problem. One trajectory drives the system to the desired target, assuming that the network topology will not change in the prediction horizon, while the second one ensures constraint satisfaction assuming a worst-case scenario in terms of new agents joining the network in the planning horizon. Recursive feasibility and stability of the closed-loop system during plug and play operations are shown. The approach effectiveness is illustrated with a numerical simulation.

This paper addresses the problem of optimally controlling nonlinear systems with norm-bounded disturbances and parametric uncertainties while robustly satisfying constraints. The proposed approach jointly optimizes a nominal nonlinear trajectory and an error feedback, requiring minimal offline design effort and offering low conservatism. This is achieved by decomposing the affine-in-the-parameter uncertain nonlinear system into a nominal $\textit{nonlinear}$ system and an uncertain linear time-varying system. Using this decomposition, we can apply established tools from system level synthesis to $\textit{convexly}$ over-bound all uncertainties in the nonlinear optimization problem. Moreover, it enables tight joint optimization of the linearization error bounds, parametric uncertainties bounds, nonlinear trajectory, and error feedback. With this novel controller parameterization, we can formulate a convex constraint to ensure robust performance guarantees for the nonlinear system. The presented method is relevant for numerous applications related to trajectory optimization, e.g., in robotics and aerospace engineering. We demonstrate the performance of the approach and its low conservatism through the simulation example of a post-capture satellite stabilization., Comment: Accepted for CDC (Singapore, 13-15 December 2023). Code: https://gitlab.ethz.ch/ics/nonlinear-parametric-SLS