Search

Robust Reinforcement Learning (RRL) is a promising Reinforcement Learning (RL) paradigm aimed at training robust to uncertainty or disturbances models, making them more efficient for real-world applications. Following this paradigm, uncertainty or disturbances are interpreted as actions of a second adversarial agent, and thus, the problem is reduced to seeking the agents' policies robust to any opponent's actions. This paper is the first to propose considering the RRL problems within the positional differential game theory, which helps us to obtain theoretically justified intuition to develop a centralized Q-learning approach. Namely, we prove that under Isaacs's condition (sufficiently general for real-world dynamical systems), the same Q-function can be utilized as an approximate solution of both minimax and maximin Bellman equations. Based on these results, we present the Isaacs Deep Q-Network algorithms and demonstrate their superiority compared to other baseline RRL and Multi-Agent RL algorithms in various environments.

Real-time environmental monitoring using a multi-agent system (MAS) has long been a focal point of cooperative control. It is still a challenging task to provide cost-effective services for potential emergencies in surface environments. This paper explores the transformation of a general surface into a two-dimensional (2D) disk through the construction of a conformal mapping. Multiple agents are strategically deployed within the mapped convex disk, followed by mapping back to the original surface environment. This approach circumvents the complexities associated with handling the difficulties and intricacies of path planning. Technical analysis encompasses the design of distributed control laws and the method to eliminate distortions introduced by the mapping. Moreover, the developed coverage algorithm is applied to a scenario of monitoring surface deformation. Finally, the effectiveness of the proposed algorithm is validated through numerical simulations.

This paper presents optimization models for electric Mobility-as-a-Service systems, whereby electric vehicles not only provide on-demand mobility, but also perform charging and Vehicle-to-Grid (V2G) operations to enhance the fleet operator profitability. Specifically, we formulate the optimal fleet operation problem as a mixed-integer linear program, with the objective combining of operational costs and revenues generated from servicing requests and grid electricity sales. Our cost function explicitly captures battery price and degradation, reflecting their impact on the fleet total cost of ownership due to additional charging and discharging activities. Simulation results for Eindhoven, The Netherlands, show that integrating V2G activities does not compromise the number of travel requests being served. Moreover, we emphasize the significance of accounting for battery degradation, as the costs associated with it can potentially outweigh the revenues stemming from V2G operations.

This paper proposes a novel real-time affordable solution to the trajectory tracking control problem for self-driving cars subject to longitudinal and steering angular velocity constraints. To this end, we develop a dual-mode Model Predictive Control (MPC) solution starting from an input-output feedback linearized description of the vehicle kinematics. First, we derive the state-dependent input constraints acting on the linearized model and characterize their worst-case time-invariant inner approximation. Then, a dual-mode MPC is derived to be real-time affordable and ensuring, by design, constraints fulfillment, recursive feasibility, and uniformly ultimate boundedness of the tracking error in an ad-hoc built robust control invariant region. The approach's effectiveness and performance are experimentally validated via laboratory experiments on a Quanser Qcar. The obtained results show that the proposed solution is computationally affordable and with tracking capabilities that outperform two alternative control schemes., Comment: Preprint of a manuscript currently under review for TCTS

In this paper, we consider the design of data-driven predictive controllers for nonlinear systems from input-output data via linear-in-control input Koopman lifted models. Instead of identifying and simulating a Koopman model to predict future outputs, we design a subspace predictive controller in the Koopman space. This allows us to learn the observables minimizing the multi-step output prediction error of the Koopman subspace predictor, preventing the propagation of prediction errors. To avoid losing feasibility of our predictive control scheme due to prediction errors, we compute a terminal cost and terminal set in the Koopman space and we obtain recursive feasibility guarantees through an interpolated initial state. As a third contribution, we introduce a novel regularization cost yielding input-to-state stability guarantees with respect to the prediction error for the resulting closed-loop system. The performance of the developed Koopman data-driven predictive control methodology is illustrated on a nonlinear benchmark example from the literature.

We study how high charging rate demands from electric vehicles (EVs) in a power distribution grid may collectively cause its dynamic instability, and, accordingly, how a price incentivization strategy can be used to steer customers to settle for lesser charging rate demands so that these instabilities can be avoided. We pose the problem as a joint optimization and optimal control formulation. The optimization determines the optimal charging setpoints for EVs to minimize the $\mathcal{H}_2$-norm of the transfer function of the grid model, while the optimal control simultaneously develops a linear quadratic regulator (LQR) based state-feedback control signal for the battery-currents of those EVs to jointly minimize the risk of grid instability. A subsequent algorithm is developed to determine how much customers may be willing to sacrifice their intended charging rate demands in return for financial incentives. Results are derived for both unidirectional and bidirectional charging, and validated using numerical simulations of multiple EV charging stations in the IEEE 33-bus power distribution model.

We study the problem of stabilizing infinite-dimensional systems with input and output quantization. The closed-loop system we consider is subject to packet loss in the sensor-to-controller channels, whose duration is assumed to be averagely bounded. Given a bound on the initial state, we propose design methods for dynamic quantizers with zoom parameters. We show that the closed-loop state staring in a given region exponentially converges to zero if the bounds of quantization errors and packet-loss duration satisfy suitable conditions. Since the norms of the operators representing the system dynamics are used in the proposed quantization schemes, we also present methods for approximately computing the operator norms., Comment: 26 pages, 8 figures

We present a fast trajectory optimization algorithm for the soft capture of uncooperative tumbling space objects. Our algorithm generates safe, dynamically feasible, and minimum-fuel trajectories for a six-degree-of-freedom servicing spacecraft to achieve soft capture (near-zero relative velocity at contact) between predefined locations on the servicer spacecraft and target body. We solve a convex problem by enforcing a convex relaxation of the field-of-view constraint, followed by a sequential convex program correcting the trajectory for collision avoidance. The optimization problems can be solved with a standard second-order cone programming solver, making the algorithm both fast and practical for implementation in flight software. We demonstrate the performance and robustness of our algorithm in simulation over a range of object tumble rates up to 10{\deg}/s., Comment: Accepted to ISpaRo24

This paper presents a convex optimization framework for eco-driving and vehicle energy management problems. We will first show that several types of eco-driving and vehicle energy management problems can be modelled using the same notions of energy storage buffers and energy storage converters that are connected to a power network. It will be shown that these problems can be formulated as optimization problems with linear cost functions and linear dynamics, and nonlinear constraints representing the power converters. We will show that under some mild conditions, the (non-convex) optimization problem has the same (globally) optimal solution as a convex relaxation. This means that the problems can be solved efficiently and that the solution is guaranteed to be globally optimal. Finally, a numerical example of the eco-driving problem is used to illustrate this claim.

Buildings account for 40 % of global energy consumption. A considerable portion of building energy consumption stems from heating, ventilation, and air conditioning (HVAC), and thus implementing smart, energy-efficient HVAC systems has the potential to significantly impact the course of climate change. In recent years, model-free reinforcement learning algorithms have been increasingly assessed for this purpose due to their ability to learn and adapt purely from experience. They have been shown to outperform classical controllers in terms of energy cost and consumption, as well as thermal comfort. However, their weakness lies in their relatively poor data efficiency, requiring long periods of training to reach acceptable policies, making them inapplicable to real-world controllers directly. Hence, common research goals are to improve the learning speed, as well as to improve their ability to generalize, in order to facilitate transfer learning to unseen building environments. In this paper, we take a federated learning approach to training the reinforcement learning controller of an HVAC system. A global control policy is learned by aggregating local policies trained on multiple data centers located in different climate zones. The goal of the policy is to simultaneously minimize energy consumption and maximize thermal comfort. The federated optimization strategy indirectly increases both the rate at which experience data is collected and the variation in the data. We demonstrate through experimental evaluation that these effects lead to a faster learning speed, as well as greater generalization capabilities in the federated policy compared to any individually trained policy.

This paper develops an adaptive traffic control policy inspired by Maximum Pressure (MP) while imposing coordination across intersections. The proposed Coordinated Maximum Pressure-plus-Penalty (CMPP) control policy features a local objective for each intersection that consists of the total pressure within the neighborhood and a penalty accounting for the queue capacities and continuous green time for certain movements. The corresponding control task is reformulated as a distributed optimization problem and solved via two customized algorithms: one based on the alternating direction method of multipliers (ADMM) and the other follows a greedy heuristic augmented with a majority vote. CMPP not only provides a theoretical guarantee of queuing network stability but also outperforms several benchmark controllers in simulations on a large-scale real traffic network with lower average travel and waiting time per vehicle, as well as less network congestion. Furthermore, CPMM with the greedy algorithm enjoys comparable computational efficiency as fully decentralized controllers without significantly compromising the control performance, which highlights its great potential for real-world deployment.

In this paper, we propose a two-stage robust planning model for an Integrated Energy System (IES) that serves an industrial park. The term 'Park-level IES' is used to refers to IES of a smaller scale but have high demands for various forms of energy. The proposed planning model considers uncertainties like load demand fluctuations and equipment contingencies, and provides a reliable scheme of equipment selection and sizing for IES investors. Inspired by the unit commitment problem, we formulate an equipment contingency uncertainty set to accurately describe the potential equipment contingencies which happen and can be repaired within a day. Then, a novel and modified nested column-and-constraint generation algorithm is applied to solve this two-stage robust planning model with integer recourse efficiently. In the case study, the role of energy storage system for IES reliability enhancement is analyzed in detail. Computational results demonstrate the advantage of the proposed models over the deterministic planning model in terms of improving reliability.

In this paper we compare two numerical methods to integrate Riemannian cubic polynomials on the Stiefel manifold $\textbf{St}_{n,k}$. The first one is the adjusted de Casteljau algorithm, and the second one is a symplectic integrator constructed through discretization maps. In particular, we choose the cases of $n=3$ together with $k=1$ and $k=2$. The first case is diffeomorphic to the sphere and the quasi-geodesics appearing in the adjusted de Casteljau algorithm are actually geodesics. The second case is an example where we have a pure quasi-geodesic different from a geodesic. We provide a numerical comparison of both methods and discuss the obtained results to highlight the benefits of each method., Comment: 12 pages, 3 figures

With the advent of self-driving cars, experts envision autonomous mobility-on-demand services in the near future to cope with overloaded transportation systems in cities worldwide. Efficient operations are imperative to unlock such a system's maximum improvement potential. Existing approaches either consider a narrow planning horizon or ignore essential characteristics of the underlying problem. In this paper, we develop an algorithmic framework that allows the study of very large-scale pickup and delivery routing problems with more than 20 thousand requests, which arise in the context of integrated request pooling and vehicle-to-request dispatching. We conduct a computational study and present comparative results showing the characteristics of the developed approaches. Furthermore, we apply our algorithm to related benchmark instances from the literature to show the efficacy. Finally, we solve very large-scale instances and derive insights on upper-bound improvements regarding fleet sizing and customer delay acceptance from a practical perspective.

Exploring novel strategies for the regulation of axon growth, we introduce a periodic event-triggered control (PETC) to enhance the practical implementation of the associated PDE backstepping control law. Neurological injuries may impair neuronal function, but therapies like Chondroitinase ABC (ChABC) have shown promise in improving axon elongation by influencing the extracellular matrix. This matrix, composed of extracellular macromolecules and minerals, regulates tubulin protein concentration, potentially aiding in neuronal recovery. The concentration and spatial distribution of tubulin influence axon elongation dynamics. Recent research explores feedback control strategies for this model, leading to the development of an event-triggering control (CETC) approach. In this approach, the control law updates when the monitored triggering condition is met, reducing actuation resource consumption. Through the meticulous redesign of the triggering mechanism, we introduce a periodic event-triggering control (PETC), updating control inputs at specific intervals, but evaluating the event-trigger only periodically, an ideal tool for standard time-sliced actuators like ChABC. PETC is a step forward to the design of practically feasible feedback laws for the neuron growth process. The PETC strategy establishes an upper bound on event triggers between periodic examinations, ensuring convergence and preventing Zeno behavior. Through Lyapunov analysis, we demonstrate the local exponential convergence of the system with the periodic event-triggering mechanism in the $L^2$-norm sense. Numerical examples are presented to confirm the theoretical findings., Comment: Submitted to 2024 Conference on Decision and Control

This paper presents an integrated model for bidding energy storage in day-ahead and real-time markets to maximize profits. We show that in integrated two-stage bidding, the real-time bids are independent of day-ahead settlements, while the day-ahead bids should be based on predicted real-time prices. We utilize a transformer-based model for real-time price prediction, which captures complex dynamical patterns of real-time prices, and use the result for day-ahead bidding design. For real-time bidding, we utilize a long short-term memory-dynamic programming hybrid real-time bidding model. We train and test our model with historical data from New York State, and our results showed that the integrated system achieved promising results of almost a 20\% increase in profit compared to only bidding in real-time markets, and at the same time reducing the risk in terms of the number of days with negative profits.

We demonstrate that valid inequalities, or lifted nonlinear cuts (LNC), can be projected to tighten the Second Order Cone (SOC), Convex DistFlow (CDF), and Network Flow (NF) relaxations of the AC Optimal Power Flow (AC-OPF) problem. We conduct experiments on 36 cases from the PGLib-OPF library for two objective functions, (1) power generation maximization and (2) generation cost minimization. Significant optimality gap improvements are shown for the maximization problem, where the LNC strengthen the SOC and CDF relaxations in 100% of the test cases, with average and maximum differences in the optimality gaps of 23.1% and 93.5% respectively. The NF relaxation is strengthened in 79.2% of test cases, with average and maximum differences in the optimality gaps of 3.45% and 21.2% respectively. We also study the trade-off between relaxation quality and solve time, demonstrating that the strengthened CDF relaxation outperforms the strengthened SOC formulation in terms of runtime and number of iterations needed, while the strengthened NF formulation is the most scalable with the lowest relaxation quality provided by these LNC.

Long duration energy storage (LDES) may become a critical technology for the decarbonization of the power sector, as current commercially available Li-ion battery storage technologies cannot cost-effectively shift energy to address multi-day or seasonal variability in demand and renewable energy availability. LDES is difficult to model in existing energy system planning models (such as electricity system capacity expansion models), as it is much more dependent on an accurate representation of chronology than other resources. Techniques exist for modeling LDES in these planning models; however, it is not known how spatial and temporal resolution affect the performance of these techniques, creating a research gap. In this study we examine what spatial and temporal resolution is necessarily to accurately capture the full value of LDES, in the context of a continent-scale capacity expansion model. We use the results to draw conclusions and present best practices for modelers seeking to accurately model LDES in a macro-energy systems planning context. Our key findings are: 1) modeling LDES with linked representative periods is crucial to capturing its full value, 2) LDES value is highly sensitive to the cost and availability of other resources, and 3) temporal resolution is more important than spatial resolution for capturing the full value of LDES, although how much temporal resolution is needed will depend on the specific model context., Comment: Working paper

This paper presents a novel approach for distributed model predictive control (MPC) for piecewise affine (PWA) systems. Existing approaches rely on solving mixed-integer optimization problems, requiring significant computation power or time. We propose a distributed MPC scheme that requires solving only convex optimization problems. The key contribution is a novel method, based on the alternating direction method of multipliers, for solving the non-convex optimal control problem that arises due to the PWA dynamics. We present a distributed MPC scheme, leveraging this method, that explicitly accounts for the coupling between subsystems by reaching agreement on the values of coupled states. Stability and recursive feasibility are shown under additional assumptions on the underlying system. Two numerical examples are provided, in which the proposed controller is shown to significantly improve the CPU time and closed-loop performance over existing state-of-the-art approaches., Comment: 14 pages, 8 figures, submitted to IEEE Transactions on Automatic Control, code available at https://github.com/SamuelMallick/stable-dmpc-pwa/tree/paper_2024 and https://github.com/SamuelMallick/hybrid-vehicle-platoon/tree/paper-2024

In this paper, we consider the problem of minimum-time optimal control for a dynamical system with initial state uncertainties and propose a sequential convex programming (SCP) solution framework. We seek to minimize the expected terminal (mission) time, which is an essential capability for planetary exploration missions where ground rovers have to carry out scientific tasks efficiently within the mission timelines in uncertain environments. Our main contribution is to convert the underlying stochastic optimal control problem into a deterministic, numerically tractable, optimal control problem. To this end, the proposed solution framework combines two strategies from previous methods: i) a partial model predictive control with consensus horizon approach and ii) a sum-of-norm cost, a temporally strictly increasing weighted-norm, promoting minimum-time trajectories. Our contribution is to adopt these formulations into an SCP solution framework and obtain a numerically tractable stochastic control algorithm. We then demonstrate the resulting control method in multiple applications: i) a closed-loop linear system as a representative result (a spacecraft double integrator model), ii) an open-loop linear system (the same model), and then iii) a nonlinear system (Dubin's car)., Comment: submitted to CDC 2024

This paper introduces a novel distributed optimization technique for networked systems, which removes the dependency on specific parameter choices, notably the learning rate. Traditional parameter selection strategies in distributed optimization often lead to conservative performance, characterized by slow convergence or even divergence if parameters are not properly chosen. In this work, we propose a systems theory tool based on the port-Hamiltonian formalism to design algorithms for consensus optimization programs. Moreover, we propose the Mixed Implicit Discretization (MID), which transforms the continuous-time port-Hamiltonian system into a discrete time one, maintaining the same convergence properties regardless of the step size parameter. The consensus optimization algorithm enhances the convergence speed without worrying about the relationship between parameters and stability. Numerical experiments demonstrate the method's superior performance in convergence speed, outperforming other methods, especially in scenarios where conventional methods fail due to step size parameter limitations.

This article introduces a new transcription, change point localization, and mesh refinement scheme for direct optimization-based solutions and for uniform approximation of optimal control trajectories associated with a class of nonlinear constrained optimal control problems (OCPs). The base transcription algorithm for which we establish the refinement algorithm is a $\textit{direct multiple shooting technique}$ -- $\mathsf{QuITO}$ $\textsf{v.2}$ (Quasi-Interpolation based Trajectory Optimization). The mesh refinement technique consists of two steps -- localization of certain irregular regions in an optimal control trajectory via wavelets, followed by a targeted $h$-refinement approach around such regions of irregularity. Theoretical approximation guarantees on uniform grids are presented for optimal controls with certain regularity properties along with guarantees of localization of change points by wavelet transform. Numerical illustrations are provided for control profiles involving discontinuities to show the effectiveness of the localization and refinement strategy. We also announce and make freely available a new software package developed based on $\mathsf{QuITO}$ $\textsf{v.2}$ along with its functionalities to make the article complete., Comment: Submitted; 35 pages, 15 figures

The 2019-20 Australia bushfire incurred numerous economic losses and significantly affected the operations of power systems. A power station or transmission line can be significantly affected due to bushfires, leading to an increase in operational costs. We study a fundamental but challenging problem of planning the optimal power flow (OPF) for power systems subject to bushfires. Considering the stochastic nature of bushfire spread, we develop a model to capture such dynamics based on Moore's neighborhood model. Under a periodic inspection scheme that reveals the in-situ bushfire status, we propose an online optimization modeling framework that sequentially plans the power flows in the electricity network. Our framework assumes that the spread of bushfires is non-stationary over time, and the spread and containment probabilities are unknown. To meet these challenges, we develop a contextual online learning algorithm that treats the in-situ geographical information of the bushfire as a 'spatial context'. The online learning algorithm learns the unknown probabilities sequentially based on the observed data and then makes the OPF decision accordingly. The sequential OPF decisions aim to minimize the regret function, which is defined as the cumulative loss against the clairvoyant strategy that knows the true model parameters. We provide a theoretical guarantee of our algorithm by deriving a bound on the regret function, which outperforms the regret bound achieved by other benchmark algorithms. Our model assumptions are verified by the real bushfire data from NSW, Australia, and we apply our model to two power systems to illustrate its applicability.

This paper studies a risk-sensitive decision-making problem under uncertainty. It considers a decision-making process that unfolds over a fixed number of stages, in which a decision-maker chooses among multiple alternatives, some of which are deterministic and others are stochastic. The decision-maker's cumulative value is updated at each stage, reflecting the outcomes of the chosen alternatives. After formulating this as a stochastic control problem, we delineate the necessary optimality conditions for it. Two illustrative examples from optimal betting and inventory management are provided to support our theory., Comment: submitted for possible publication

This paper presents a distributed model predictive control (DMPC) algorithm for a heterogeneous platoon using arbitrary communication topologies, as long as each vehicle is able to communicate with a preceding vehicle in the platoon. The proposed DMPC algorithm is able to accommodate any spacing policy that is affine in a vehicle's velocity, which includes constant distance or constant time headway spacing policies. By analyzing the total cost for the entire platoon, a sufficient condition is derived to guarantee platoon asymptotic stability. Simulation experiments with a platoon of 50 vehicles and hardware experiments with a platoon of four 1/10th scale vehicles validate the algorithm and compare performance under different spacing policies and communication topologies.

Contraction theory is a mathematical framework for studying the convergence, robustness, and modularity properties of dynamical systems and algorithms. In this opinion paper, we provide five main opinions on the virtues of contraction theory. These opinions are (i) contraction theory is a unifying framework emerging from classical and modern works, (ii) contractivity is computationally-friendly, robust, and modular stability, (iii) numerous dynamical systems are contracting, (iv) contraction theory is relevant to modern applications, and (v) contraction theory can be vastly extended in numerous directions. We survey recent theoretical and applied research in each of these five directions.

Wildfires ignited by power systems infrastructure are among the most destructive wildfires; hence some utility companies in wildfire-prone regions have pursued a proactive policy of emergency power shutoffs. These shutoffs, while mitigating the risk of disastrous ignition events, result in power outages that could negatively impacts vulnerable communities. In this paper, we consider how to equitably allocate funds to underground and effectively de-risk power lines in transmission networks. We explore the impact of the 2021 White House resource allocation policy called the Justice40 initiative, which states that 40% of the benefits of federally-funded climate-related investments should go to socially vulnerable communities. The definition of what constitutes a vulnerable community varies by organization, and we consider two major recently proposed vulnerability indices: the Justice40 index created under the 2021 White House and the Social Vulnerability Index (SVI) developed by the Center for Disease Control and Prevention (CDC). We show that allocating budget according to these two indices fails to reduce power outages for indigenous communities and those subject to high wildfire ignition risk using a high-fidelity synthetic power grid dataset that matches the key features of the Texas transmission system. We discuss how aggregation of communities and "one size fits all" vulnerability indices might be the reasons for the misalignment between the goals of vulnerability indices and their realized impact in this particular case study. We provide a method of achieving an equitable investment plan by adding group-level protections on percentage of load that is shed across each population group of interest.

We develop a switched predictor-feedback law, which achieves global asymptotic stabilization of linear systems with input delay and with the plant and actuator states available only in (almost) quantized form. The control design relies on a quantized version of the nominal predictor-feedback law for linear systems, in which quantized measurements of the plant and actuator states enter the predictor state formula. A switching strategy is constructed to dynamically adjust the tunable parameter of the quantizer (in a piecewise constant manner), in order to initially increase the range and subsequently decrease the error of the quantizers. The key element in the proof of global asymptotic stability in the supremum norm of the actuator state is derivation of solutions' estimates combining a backstepping transformation with small-gain and input-to-state stability arguments, for addressing the error due to quantization. We extend this result to the input quantization case and illustrate our theory with a numerical example., Comment: 12 pages, 7 figures, submitted to Systems & Control Letters

We provide the first known algorithm that provably achieves $\varepsilon$-optimality within $\widetilde{\mathcal{O}}(1/\varepsilon)$ function evaluations for the discounted discrete-time LQR problem with unknown parameters, without relying on two-point gradient estimates. These estimates are known to be unrealistic in many settings, as they depend on using the exact same initialization, which is to be selected randomly, for two different policies. Our results substantially improve upon the existing literature outside the realm of two-point gradient estimates, which either leads to $\widetilde{\mathcal{O}}(1/\varepsilon^2)$ rates or heavily relies on stability assumptions.

Aquifer thermal energy storages (ATES) are used to temporally store thermal energy in groundwater saturated aquifers. Typically, two storages are combined, one for heat and one for cold, to support heating and cooling of buildings. This way, the use of classical fossil fuel-based heating, ventilation, and air conditioning can be significantly reduced. Exploiting the benefits of ATES beyond ``seasonal'' heating in winter and cooling in summer as well as meeting legislative restrictions requires sophisticated control. We propose a tailored model predictive control (MPC) scheme for the sustainable operation of ATES systems, which mainly builds on a novel model and objective function. The new approach leads to a mixed-integer quadratic program. Its performance is evaluated on real data from an ATES system in Belgium.

The power flow equations are central to many problems in power system planning, analysis, and control. However, their inherent non-linearity and non-convexity present substantial challenges during problem-solving processes, especially for optimization problems. Accordingly, linear approximations are commonly employed to streamline computations, although this can often entail compromises in accuracy and feasibility. This paper proposes an approach termed Conservative Bias Linear Approximations (CBLA) for addressing these limitations. By minimizing approximation errors across a specified operating range while incorporating conservativeness (over- or under-estimating quantities of interest), CBLA strikes a balance between accuracy and tractability by maintaining linear constraints. By allowing users to design loss functions tailored to the specific approximated function, the bias approximation approach significantly enhances approximation accuracy. We illustrate the effectiveness of our proposed approach through several test cases.

Product co-creation based on company-sponsored online community has come to be a paradigm of developing new products collaboratively with customers. In such a product co-creation campaign, the sponsoring company needs to interact intensively with active community members about the design scheme of the product. We call the collection of the rates of the company's response to active community members at all time in the co-creation campaign as a company response policy (CRP). This paper addresses the problem of finding a cost-effective CRP (the CRP problem). First, we introduce a novel community state evolutionary model and, thereby, establish an optimal control model for the CRP problem (the CRP model). Second, based on the optimality system for the CRP model, we present an iterative algorithm for solving the CRP model (the CRP algorithm). Thirdly, through extensive numerical experiments, we conclude that the CRP algorithm converges and the resulting CRP exhibits excellent cost benefit. Consequently, we recommend the resulting CRP to companies that embrace product co-creation. Next, we discuss how to implement the resulting CRP. Finally, we investigate the effect of some factors on the cost benefit of the resulting CRP. To our knowledge, this work is the first attempt to study value co-creation through optimal control theoretic approach.

Performance modeling is a key issue in queuing theory and operation research. It is well-known that the length of a queue that awaits service or the time spent by a job in a queue depends not only on the service rate, but also crucially on the fluctuations in service time. The larger the fluctuations, the longer the delay becomes and hence, this is a major hindrance for the queue to operate efficiently. Various strategies have been adapted to prevent this drawback. In this perspective, we investigate the effects of one such novel strategy namely resetting or restart, an emerging concept in statistical physics and stochastic complex process, that was recently introduced to mitigate fluctuations-induced delays in queues. In particular, we show that a service resetting mechanism accompanied with an overhead time can remarkably shorten the average queue lengths and waiting times. We examine various resetting strategies and further shed light on the intricate role of the overhead times to the queuing performance. Our analysis opens up future avenues in operation research where resetting-based strategies can be universally promising., Comment: Invited Perspective article to Journal of Physics: Complexity

Most existing necessary conditions for optimal control based on adjoining methods require both state information and costate information, yet the lack of costates for a given feasible trajectory in practice impedes the determination of optimality. This paper establishes a novel theoretical framework for time-optimal control of controllable linear systems, proposing the augmented switching law that represents the input control and the feasibility in a compact form. Given a feasible trajectory, the disturbed trajectory under the constraints of augmented switching law is guaranteed to be feasible, resulting in a novel state-centric necessary condition without dependence on costate information. A first order necessary condition is proposed that the Jacobian matrix of the augmented switching law is not full row rank, which also results in an approach to optimizing a given feasible trajectory further. The proposed necessary condition is applied to the chain-of-integrators systems with full box constraints, contributing to some conclusions challenging to reason by traditional costate-based necessary conditions.

Constraint satisfaction is a critical component in a wide range of engineering applications, including but not limited to safe multi-agent control and economic dispatch in power systems. This study explores violation-free distributed optimization techniques for problems characterized by separable objective functions and coupling constraints. First, we incorporate auxiliary decision variables together with a network-dependent linear mapping to each coupling constraint. For the reformulated problem, we show that the projection of its feasible set onto the space of primal variables is identical to that of the original problem, which is the key to achieving all-time constraint satisfaction. Upon treating the reformulated problem as a min-min optimization problem with respect to auxiliary and primal variables, we demonstrate that the gradients in the outer minimization problem have a locally computable closed-form. Then, two violation-free distributed optimization algorithms are developed and their convergence under reasonable assumptions is analyzed. Finally, the proposed algorithm is applied to implement a control barrier function based controller in a distributed manner, and the results verify its effectiveness., Comment: 13 pages, 6 figures

In this work, we propose a continuous-time distributed optimization algorithm with guaranteed zero coupling constraint violation and apply it to safe distributed control in the presence of multiple control barrier functions (CBF). The optimization problem is defined over a network that collectively minimizes a separable cost function with coupled linear constraints. An equivalent optimization problem with auxiliary decision variables and a decoupling structure is proposed. A sensitivity analysis demonstrates that the subgradient information can be computed using local information. This then leads to a subgradient algorithm for updating the auxiliary variables. A case with sparse coupling constraints is further considered, and it is shown to have better memory and communication efficiency. For the specific case of a CBF-induced time-varying quadratic program (QP), an update law is proposed that achieves finite-time convergence. Numerical results involving a static resource allocation problem and a safe coordination problem for a multi-agent system demonstrate the efficiency and effectiveness of our proposed algorithms.

The focus of this paper is on linear system identification in the setting where it is known that the underlying partially-observed linear dynamical system lies within a finite collection of known candidate models. We first consider the problem of identification from a given trajectory, which in this setting reduces to identifying the index of the true model with high probability. We characterize the finite-time sample complexity of this problem by leveraging recent advances in the non-asymptotic analysis of linear least-square methods in the literature. In comparison to the earlier results that assume no prior knowledge of the system, our approach takes advantage of the smaller hypothesis class and leads to the design of a learner with a dimension-free sample complexity bound. Next, we consider the switching control of linear systems, where there is a candidate controller for each of the candidate models and data is collected through interaction of the system with a collection of potentially destabilizing controllers. We develop a dimension-dependent criterion that can detect those destabilizing controllers in finite time. By leveraging these results, we propose a data-driven switching strategy that identifies the unknown parameters of the underlying system. We then provide a non-asymptotic analysis of its performance and discuss its implications on the classical method of estimator-based supervisory control.

Learning-based neural network (NN) control policies have shown impressive empirical performance in a wide range of tasks in robotics and control. However, formal (Lyapunov) stability guarantees over the region-of-attraction (ROA) for NN controllers with nonlinear dynamical systems are challenging to obtain, and most existing approaches rely on expensive solvers such as sums-of-squares (SOS), mixed-integer programming (MIP), or satisfiability modulo theories (SMT). In this paper, we demonstrate a new framework for learning NN controllers together with Lyapunov certificates using fast empirical falsification and strategic regularizations. We propose a novel formulation that defines a larger verifiable region-of-attraction (ROA) than shown in the literature, and refines the conventional restrictive constraints on Lyapunov derivatives to focus only on certifiable ROAs. The Lyapunov condition is rigorously verified post-hoc using branch-and-bound with scalable linear bound propagation-based NN verification techniques. The approach is efficient and flexible, and the full training and verification procedure is accelerated on GPUs without relying on expensive solvers for SOS, MIP, nor SMT. The flexibility and efficiency of our framework allow us to demonstrate Lyapunov-stable output feedback control with synthesized NN-based controllers and NN-based observers with formal stability guarantees, for the first time in literature. Source code at https://github.com/Verified-Intelligence/Lyapunov_Stable_NN_Controllers.

The collision avoidance constraints are prominent as non-convex, non-differentiable, and challenging when defined in optimization-based motion planning problems. To overcome these issues, this paper presents a novel non-conservative collision avoidance technique using the notion of convex optimization to establish the distance between robotic spacecraft and space structures for autonomous on-orbit assembly operations. The proposed technique defines each ellipsoidal- and polyhedral-shaped object as the union of convex compact sets, each represented non-conservatively by a real-valued convex function. Then, the functions are introduced as a set of constraints to a convex optimization problem to produce a new set of differentiable constraints resulting from the optimality conditions. These new constraints are later fed into an optimal control problem to enforce collision avoidance where the motion planning for the autonomous on-orbit assembly takes place. Numerical experiments for two assembly scenarios in tight environments are presented to demonstrate the capability and effectiveness of the proposed technique. The results show that this framework leads to optimal non-conservative trajectories for robotic spacecraft in tight environments. Although developed for autonomous on-orbit assembly, this technique could be used for any generic motion planning problem where collision avoidance is crucial., Comment: Technical Note version of this paper is under review in the Journal of Guidance, Control, and Dynamics (JGCD), revision submitted in April 2024

Scenario reduction algorithms can be an effective means to provide a tractable description of the uncertainty in optimal control problems. However, they might significantly compromise the performance of the controlled system. In this paper, we propose a method to compensate for the effect of scenario reduction on stochastic optimal control problems for chance-constrained linear systems with additive uncertainty. We consider a setting in which the uncertainty has a discrete distribution, where the number of possible realizations is large. We then propose a reduction algorithm with a problem-dependent loss function, and we define sufficient conditions on the stochastic optimal control problem to ensure out-of-sample guarantees (i.e., against the original distribution of the uncertainty) for the controlled system in terms of performance and chance constraint satisfaction. Finally, we demonstrate the effectiveness of the approach on a numerical example., Comment: 7 pages, 1 figure, submitted to the European Control Conference

The purpose of the present work is to expand substantially the type of control and estimation problems that can be addressed following the paradigm of Schr\"odinger bridges, by incorporating stopping (freezing) of a given stochastic flow. Specifically, in the context of estimation, we seek the most likely evolution realizing prescribed spatio-temporal marginals of stopped particles. In the context of control, we seek the control action directing the stochastic flow toward spatio-temporal probabilistic constraints. To this end, we derive a new Schr\"odinger system of coupled, in space and time, partial differential equations to construct the solution of the proposed problem. Further, we show that a Fortet-Sinkhorn type of algorithm is, once again, available to attain the associated bridge. A key feature of the framework is that the obtained bridge retains the Markovian structure in the prior process, and thereby, the corresponding control action takes the form of state feedback., Comment: 6 pages, 2 figures

This paper proposes a novel approach for locally stable convergence to Nash equilibrium in duopoly noncooperative games based on a distributed event-triggered control scheme. The proposed approach employs extremum seeking, with sinusoidal perturbation signals applied to estimate the Gradient (first derivative) of unknown quadratic payoff functions. This is the first instance of noncooperative games being tackled in a model-free fashion integrated with the event-triggered methodology. Each player evaluates independently the deviation between the corresponding current state variable and its last broadcasted value to update the player action, while they preserve control performance under limited bandwidth of the actuation paths and still guarantee stability for the closed-loop dynamics. In particular, the stability analysis is carried out using time-scaling technique, Lyapunov's direct method and averaging theory for discontinuous systems. We quantify the size of the ultimate small residual sets around the Nash equilibrium and illustrate the theoretical results numerically on an example.

Two subsets of a given set are path-disconnected if they lie in different connected components of the larger set. Verification of path-disconnectedness is essential in proving the infeasibility of motion planning and trajectory optimization algorithms. We formulate path-disconnectedness as the infeasibility of a single-integrator control task to move between an initial set and a target set in a sufficiently long time horizon. This control-infeasibility task is certified through the generation of a time-dependent barrier function that separates the initial and final sets. The existence of a time-dependent barrier function is a necessary and sufficient condition for path-disconnectedness under compactness conditions. Numerically, the search for a polynomial barrier function is formulated using the moment-sum-of-squares hierarchy of semidefinite programs. The barrier function proves path-disconnectedness at a sufficiently large polynomial degree. The computational complexity of these semidefinite programs can be reduced by elimination of the control variables. Disconnectedness proofs are synthesized for example systems., Comment: 17 pages, 2 tables, 3 figures

This paper develops a method to upper-bound extreme-values of time-windowed risks for stochastic processes. Examples of such risks include the maximum average or 90% quantile of the current along a transmission line in any 5-minute window. This work casts the time-windowed risk analysis problem as an infinite-dimensional linear program in occupation measures. In particular, we employ the coherent risk measures of the mean and the expected shortfall (conditional value at risk) to define the maximal time-windowed risk along trajectories. The infinite-dimensional linear program must then be truncated into finite-dimensional optimization problems, such as by using the moment-sum of squares hierarchy of semidefinite programs. The infinite-dimensional linear program will have the same optimal value as the original nonconvex risk estimation task under compactness and regularity assumptions, and the sequence of semidefinite programs will converge to the true value under additional properties of algebraic characterization. The scheme is demonstrated for risk analysis of example stochastic processes., Comment: 26 pages, 11 figures

The rapid expansion of renewable energy sources introduces significant volatility and unpredictability to the energy supply chain, challenging the stability and reliability of the power grid. This work presents a method that enhances existing static optimization models by converting them into dynamic models suitable for the real-time optimization of flexible energy resources. By adapting static models for real-time application, the proposed two-stage optimization strategy allows for flexible adjustments of operational plans, facilitating the seamless integration of renewable energy sources. This approach not only ensures grid reliability but also improves economic efficiency by optimizing resource utilization. The effectiveness of this method is demonstrated through a case study involving a system of electrolyzers, showcasing significant advantages over traditional static optimization methods in aligning energy consumption with renewable energy generation.

In this study, we investigate the robust feedback stability problem for multiple-input-multiple-output linear time-invariant systems involving sectored-disk uncertainty, namely, dynamic uncertainty subject to simultaneous gain and phase constraints. This problem is thereby called a sectored-disk problem. Employing a frequency-wise analysis approach, we derive a fundamental static matrix problem that serves as a key component in addressing the feedback stability. The study of this matrix problem heavily relies on the Davis-Wielandt (DW) shells of matrices, providing a profound insight into matrices subjected to simultaneous gain and phase constraints. This understanding is pivotal for establishing a less conservative sufficient condition for the matrix sectored-disk problem, from which we formulate several robust feedback stability conditions against sectored-disk uncertainty. Finally, several conditions based on linear matrix inequalities are developed for efficient computation and verification of feedback robust stability against sectored-disk uncertainty.

We propose a combinatorial method for computing explicit solutions to multi-parametric quadratic programs, which can be used to compute explicit control laws for linear model predictive control. In contrast to classical methods, which are based on geometrical adjacency, the proposed method is based on combinatorial adjacency. After introducing the notion of combinatorial adjacency, we show that the explicit solution forms a connected graph in terms of it. We then leverage this connectedness to propose an algorithm that computes the explicit solution. The purely combinatorial nature of the algorithm leads to computational advantages since it enables demanding geometrical operations (such as computing facets of polytopes) to be avoided. Compared with classical combinatorial methods, the proposed method requires fewer combinations to be considered by exploiting combinatorial connectedness. We show that an implementation of the proposed method can yield a speedup of about two orders of magnitude compared with state-of-the-art software packages such as MPT and POP., Comment: Submitted to CDC24

Model Predictive Control (MPC) is a method to control nonlinear systems with guaranteed stability and constraint satisfaction but suffers from high computation times. Approximate MPC (AMPC) with neural networks (NNs) has emerged to address this limitation, enabling deployment on resource-constrained embedded systems. However, when tuning AMPCs for real-world systems, large datasets need to be regenerated and the NN needs to be retrained at every tuning step. This work introduces a novel, parameter-adaptive AMPC architecture capable of online tuning without recomputing large datasets and retraining. By incorporating local sensitivities of nonlinear programs, the proposed method not only mimics optimal MPC inputs but also adjusts to changes in physical parameters of the model using linear predictions while still guaranteeing stability. We showcase the effectiveness of parameter-adaptive AMPC by controlling the swing-ups of two different real cartpole systems with a severely resource-constrained microcontroller (MCU). We use the same NN across both system instances that have different parameters. This work not only represents the first experimental demonstration of AMPC for fast-moving systems on low-cost MCUs to the best of our knowledge, but also showcases generalization across system instances and variations through our parameter-adaptation method. Taken together, these contributions represent a marked step toward the practical application of AMPC in real-world systems., Comment: Accepted to L4DC 2024

Discrete-time robust optimal control problems generally take a min-max structure over continuous variable spaces, which can be difficult to solve in practice. In this paper, we extend the class of such problems that can be solved through a previously proposed local reduction method to consider those with existence constraints on the uncountable variables. We also consider the possibility of non-unique trajectories that satisfy equality and inequality constraints. Crucially, we show that the problems of interest can be cast into a standard semi-infinite program and demonstrate how to generate optimal uncertainty scenario sets in order to obtain numerical solutions. We also include examples on model predictive control for obstacle avoidance with logical conditions, control with input saturation affected by uncertainty, and optimal parameter estimation to highlight the need for the proposed extension. Our method solves each of the examples considered, producing violation-free and locally optimal solutions., Comment: 6 pages, 1 figure. To be published in the proceedings of the 8th IFAC Conference on Nonlinear Model Predictive Control (NMPC 2024)

This study addresses a distributed state feedback controller design problem for continuous-time linear time-invariant systems by means of linear matrix inequalities (LMI). As the exact convexification is still open, the block-diagonal relaxation of Lyapunov functions has been prevalent despite its conservatism. In this work, we target a class of non-block-diagonal Lyapunov functions that has the same sparsity as distributed controllers. By leveraging a block-diagonal factorization of sparse matrices and Finsler's lemma, we first present a (nonlinear) matrix inequality for stabilizing distributed controllers with such Lyapunov functions, which boils down to a necessary and sufficient condition for such controllers if the sparsity pattern is chordal. As a relaxation of the inequality, we derive an LMI that completely covers the conventional relaxation and then provide analogous results for $H_\infty$ control. Lastly, numerical examples underscore the efficacy of our results., Comment: 8 pages