Search

We propose an early termination technique for mixed integer conic programming for use within branch-and-bound based solvers. Our approach generalizes previous early termination results for ADMM-based solvers to a broader class of primal-dual algorithms, including both operator splitting methods and interior point methods. The complexity for checking early termination is $O(n)$ for each termination check assuming a bounded problem domain. We show that this domain restriction can be relaxed for problems whose data satisfies a simple rank condition, in which case each check requires an $O(n^2)$ solve using a linear system that must be factored only once at the root node. We further show how this approach can be used in hybrid model predictive control as long as system inputs are bounded. Numerical results show that our method leads to a moderate reduction in the total iterations required for branch-and-bound conic solvers with interior-point based subsolvers.

While much effort has been devoted to deriving and analyzing effective convex formulations of signal processing problems, the gradients of convex functions also have critical applications ranging from gradient-based optimization to optimal transport. Recent works have explored data-driven methods for learning convex objective functions, but learning their monotone gradients is seldom studied. In this work, we propose C-MGN and M-MGN, two monotone gradient neural network architectures for directly learning the gradients of convex functions. We show that, compared to state of the art methods, our networks are easier to train, learn monotone gradient fields more accurately, and use significantly fewer parameters. We further demonstrate their ability to learn optimal transport mappings to augment driving image data., ICASSP 2023 Camera-Ready Version

Coupled matrix and tensor factorizations (CMTF) have emerged as an effective data fusion tool to jointly analyze data sets in the form of matrices and higher-order tensors. The PARAFAC2 model has shown to be a promising alternative to the CANDECOMP/PARAFAC (CP) tensor model due to its flexibility and capability to handle irregular/ragged tensors. While fusion models based on a PARAFAC2 model coupled with matrix/tensor decompositions have been recently studied, they are limited in terms of possible regularizations and/or types of coupling between data sets. In this paper, we propose an algorithmic framework for fitting PARAFAC2-based CMTF models with the possibility of imposing various constraints on all modes and linear couplings, using Alternating Optimization (AO) and the Alternating Direction Method of Multipliers (ADMM). Through numerical experiments, we demonstrate that the proposed algorithmic approach accurately recovers the underlying patterns using various constraints and linear couplings.

Plug & Play methods combine proximal algorithms with denoiser priors to solve inverse problems. These methods rely on the computability of the proximal operator of the data fidelity term. In this paper, we propose a Plug & Play framework based on linearized ADMM that allows us to bypass the computation of intractable proximal operators. We demonstrate the convergence of the algorithm and provide results on restoration tasks such as super-resolution and deblurring with non-uniform blur.

In this paper, we study mixed traffic systems that move along a single-lane ring-road or open-road. The traffic flow firms a platoon, which includes a number of heterogeneous human-driven vehicles (HDVs) together with only one connected and automated vehicle (CAV) that receives information from a subset of neighbors. The dynamics of HDVs are assumed to follow a general continuous-time nonlinear car-following model, which is a function of their velocity, spacing, and the relative velocity. The acceleration of the single CAV is also directly controlled by a dynamical output-feedback controller. The ultimate goal of this work is to present a robust control strategy that can smoothen the traffic flow in the presence of undesired disturbances (e.g. abrupt deceleration) and parametric uncertainties. A prerequisite for synthesizing a dynamical output controller is the stabilizability and detectability of the underlying system. Accordingly, a theoretical analysis is presented first to prove the stabilizability and detectability of the mixed traffic flow system. Then, two H-infinity control strategies, with and without considering uncertainties in the system dynamics, are designed. The efficiency of the two control methods is subsequently illustrated through numerical simulations, and various experimental results are presented to demonstrate the effectiveness of the proposed controller to mitigate disturbance amplification and achieve platoon stability. ISSN:1524-9050 ISSN:1558-0016

This paper proposes a general formulation for temporal parallelisation of dynamic programming for optimal control problems. We derive the elements and associative operators to be able to use parallel scans to solve these problems with logarithmic time complexity rather than linear time complexity. We apply this methodology to problems with finite state and control spaces, linear quadratic tracking control problems, and to a class of nonlinear control problems. The computational benefits of the parallel methods are demonstrated via numerical simulations run on a graphics processing unit., Comment: To appear in IEEE Transactions on Automatic Control

We consider the problem of synchronization of coupled oscillators in a Kuramoto-type model with lossy couplings. Kuramoto models have been used to gain insight on the stability of power networks which are usually nonlinear and involve large scale interconnections. Such models commonly assume lossless couplings and Lyapunov functions have predominantly been employed to prove stability. However, coupling conductances can impact synchronization. We therefore consider a more advanced Kuramoto model that includes coupling conductances, and is characterized by nonhomogeneous coupling weights and noncomplete coupling graphs. Lyapunov analysis once such coupling conductances and aforementioned properties are included becomes nontrivial and more conventional energy-like Lyapunov functions are not applicable or are conservative. Small-signal analysis has been performed for such models, but due to the fact that we have convergence to a manifold, stability analysis via a linearization is on its own inconclusive for the nonlinear model. In this paper, we provide a formal derivation using centre manifold theory that if a particular condition on the equilibrium point associated with the coupling conductances and susceptances holds, then the synchronization manifold for the nonlinear system considered is asymptotically stable. Our analysis is demonstrated with simulations., 6 pages, 1 figure

International audience; The paper makes the first steps towards a behavioral theory of LPV state-space representations with an affine dependency on scheduling, by characterizing minimality of such state-space representations. It is shown that minimality is equivalent to observability, and that minimal realizations of the same behavior are isomorphic. Finally, we establish a formal relationship between minimality of LPV state-space representations with an affine dependence on scheduling and minimality of LPV state-space representations with a dynamic and meromorphic dependence on scheduling.

Despite the evolution of Convolutional Neural Networks (CNNs), their performance is surprisingly dependent on the choice of hyperparameters. However, it remains challenging to efficiently explore large hyperparameter search space due to the long training times of modern CNNs. Multi-fidelity optimization enables the exploration of more hyperparameter configurations given budget by early termination of unpromising configurations. However, it often results in selecting a sub-optimal configuration as training with the high-performing configuration typically converges slowly in an early phase. In this paper, we propose Multi-fidelity Optimization with a Recurring Learning rate (MORL) which incorporates CNNs' optimization process into multi-fidelity optimization. MORL alleviates the problem of slow-starter and achieves a more precise low-fidelity approximation. Our comprehensive experiments on general image classification, transfer learning, and semi-supervised learning demonstrate the effectiveness of MORL over other multi-fidelity optimization methods such as Successive Halving Algorithm (SHA) and Hyperband. Furthermore, it achieves significant performance improvements over hand-tuned hyperparameter configuration within a practical budget.

The New York City (NYC) youth shelter system provides housing, counseling, and other support services to runaway and homeless youth and young adults (RHY). These resources reduce RHYs vulnerability to human trafficking, yet most shelters are unable to meet demand. This paper presents a Discrete Event Simulation (DES) model of a crisis-emergency and drop-in center for LGBTQ+ youth in NYC, which aims to analyze the current operations and test potential capacity expansion interventions. The model uses data from publicly available resources and interviews with service providers and key stakeholders. The simulated shelter has 66 crisis-emergency beds, offers five different support services, and serves on average 1,399 LGBTQ+ RHY per year. The capacity expansion interventions examined in this paper are adding crisis-emergency beds and psychiatric therapists. This application of DES serves as a tool to communicate with policymakers, funders, and service providers potentially having a strong humanitarian impact.

We consider the popular tree-based search strategy within the framework of reinforcement learning, the Monte Carlo Tree Search (MCTS), in the context of finite-horizon Markov decision process. We propose a dynamic sampling tree policy that efficiently allocates limited computational budget to maximize the probability of correct selection of the best action at the root node of the tree. Experimental results on Tic-Tac-Toe and Gomoku show that the proposed tree policy is more efficient than other competing methods.

We propose a simulation-optimization-based methodology to improve the way that organ transplant offers are made to potential recipients. Our policy can be applied to all types of organs, is implemented starting at the local level, is flexible with respect to simultaneous offers of an organ to multiple patients, and takes into account the quality of the organs under consideration. We describe in detail our simulation-optimization procedure and how it uses underlying real-world transplant data to inform the decision-making process. We present results using our liver and kidney models, where we show that, under our policy recommendations, more organs are utilized and the required times to allocate the organs are reduced -- sometimes dramatically., Comment: 12 pages, accepted in the Winter Simulation Conference (2022)

In this paper, we propose a Bi-layer Predictionbased Reduction Branch (BP-RB) framework to speed up the process of finding a high-quality feasible solution for Mixed Integer Programming (MIP) problems. A graph convolutional network (GCN) is employed to predict binary variables' values. After that, a subset of binary variables is fixed to the predicted value by a greedy method conditioned on the predicted probabilities. By exploring the logical consequences, a learning-based problem reduction method is proposed, significantly reducing the variable and constraint sizes. With the reductive sub-MIP problem, the second layer GCN framework is employed to update the prediction for the remaining binary variables' values and to determine the selection of variables which are then used for branching to generate the Branch and Bound (B&B) tree. Numerical examples show that our BP-RB framework speeds up the primal heuristic and finds the feasible solution with high quality., This paper has been accepted for the publication of ICARCV 2022

Social influence is largely recognized as a key factor in opinion formation processes. Recently, the role of external forces in inducing opinion displacement and polarization in social networks has attracted significant attention. This is in particular motivated by the necessity to understand and possibly prevent interference phenomena during political campaigns and elections. In this paper, we formulate and solve a targeted intervention problem for opinion displacement minimization on a social network. Specifically, we consider a min-max problem whereby a social planner (the defender) aims at selecting the optimal network intervention within her given budget constraint in order to minimize the opinion displacement in the system that an adversary (the attacker) is instead trying to maximize. Our results show that the optimal intervention of the defender has two regimes. For large enough budget, the optimal intervention of the social planner acts on all nodes proportionally to a new notion of network centrality. For lower budget values, such optimal intervention has a more delicate structure and is rather concentrated on a few target individuals.

We present a multi-agent algorithm for multi-objective optimization problems, which extends the class of consensus-based optimization methods and relies on a scalarization strategy. The optimization is achieved by a set of interacting agents exploring the search space and attempting to solve all scalar sub-problems simultaneously. We show that those dynamics are described by a mean-field model, which is suitable for a theoretical analysis of the algorithm convergence. Numerical results show the validity of the proposed method.

In this paper, we show that a hybrid integrator-gain system (HIGS) is a nonlinear negative imaginary (NNI) system. We prove that the positive feedback interconnection of a linear negative imaginary (NI) system and a HIGS is asymptotically stable. We apply the HIGS to a MEMS nanopositioner, as an example of a linear NI system, in a single-input single-output framework. We analyze the stability and the performance of the closed-loop interconnection in both time and frequency domains through simulations and demonstrate the applicability of HIGS as an NNI controller to a linear NI system., This paper was presented at the 61st IEEE Conference on Decision and Control (CDC), 2022. A short version was published in the proceedings of the conference

We consider the security-by-design of a signal-free intersection for connected and autonomous vehicles in the face of strategic jamming attacks. We use a fluid model to characterize macroscopic traffic flow through the intersection, where the saturation rate is derived from a vehicle coordination algorithm. We model jamming attacks as sudden increase in communication latency induced on vehicle-to-infrastructure connectivity; such latency triggers the safety mode for vehicle coordination and thus reduces the intersection saturation rate. A strategic attacker selects the attacking rate, while a system operator selects key design parameters, either the saturation rate or the recovery rate. Both players' actions induce technological costs and jointly determine the mean travel delay. By analyzing the equilibrium of the security game, we study the preferable level of investment in the intersection's nominal discharging capability or recovery capability., Comment: Accepted by 2022 IEEE Conference on Decision and Control(CDC)

When Model Predictive Control (MPC) is used in real-time to control linear systems, quadratic programs (QPs) need to be solved within a limited time frame. Recently, several parametric methods have been proposed that certify the number of computations active-set QP solvers require to solve these QPs. These certification methods, hence, ascertain that the optimization problem can be solved within the limited time frame. A shortcoming in these methods is, however, that they do not account for numerical errors that might occur internally in the solvers, which ultimately might lead to optimistic complexity bounds if, for example, the solvers are implemented in single precision. In this paper we propose a general framework that can be incorporated in any of these certification methods to account for such numerical errors.

Oscillations often take place in populations of decision makers that are either a coordinator, who takes action only if enough others do so, or an anticoordinator, who takes action only if few others do so. Populations consisting of exclusively one of these types are known to reach an equilibrium, where every individual is satisfied with her decision. Yet it remains unknown whether oscillations take place in a population consisting of both types, and if they do, what features they share. We study a well-mixed population of individuals, which are either a coordinator or anticoordinator, each associated with a possibly unique threshold and initialized with the strategy A or B. At each time, an agent becomes active to update her strategy based on her threshold: an active coordinator (resp. anticoordinator) updates her strategy to A (resp. B) if the portion of other agents who have chosen A exceeds (falls short of) her threshold, and updates to B (resp. A) otherwise. We define the state of the population dynamics as the distribution over the thresholds of those who have chosen A. We show that the population can admit several minimally positively invariant sets, where the solution trajectory oscillates. We explicitly characterize a class of positively invariant sets, prove their invariance, and provide a necessary and sufficient condition for their stability. Our results highlight the possibility of non-trivial, complex oscillations in the absence of noise and population structure and shed light on the reported oscillations in nature and human societies.

Solutions to optimal control problems can be discontinuous, even if all the functionals defining the problem are smooth. This can cause difficulties when numerically computing solutions to these problems. While conventional numerical methods assume state and input trajectories are continuous and differentiable or smooth, our method is able to capture discontinuities in the solution by introducing time-mesh nodes as decision variables. This allows one to obtain a higher accuracy solution for the same number of mesh nodes compared to a fixed time-mesh approach. Furthermore, we propose to first solve a sequence of suitably-defined least-squares problems to ensure that the error in the dynamic equation is below a given tolerance. The cost functional is then minimized subject to an inequality constraint on the dynamic equation residual. We demonstrate our implementation on an optimal control problem that has a chattering solution. Solving such a problem is difficult, since the solution involves infinitely many switches of decreasing duration. This simulation shows how the flexible mesh is able to capture discontinuities present in the solution and achieve superlinear convergence as the number of mesh intervals is increased., 6 pages

Asynchronous optimization algorithms are at the core of modern machine learning and resource allocation systems. However, most convergence results consider bounded information delays and several important algorithms lack guarantees when they operate under total asynchrony. In this paper, we derive explicit convergence rates for the proximal incremental aggregated gradient (PIAG) and the asynchronous block-coordinate descent (Async-BCD) methods under a specific model of total asynchrony, and show that the derived rates are order-optimal. The convergence bounds provide an insightful understanding of how the growth rate of the delays deteriorates the convergence times of the algorithms. Our theoretical findings are demonstrated by a numerical example., Comment: 6 pages

This paper is concerned with convergence analysis for the mirror descent (MD) method, a well-known algorithm in convex optimization. An analysis framework via integral quadratic constraints (IQCs) is constructed to analyze the convergence rate of the MD method with strongly convex objective functions in both continuous-time and discrete-time. We formulate the problem of finding convergence rates of the MD algorithms into feasibility problems of linear matrix inequalities (LMIs) in both schemes. In particular, in continuous-time, we show that the Bregman divergence function, which is commonly used as a Lyapunov function for this algorithm, is a special case of the class of Lyapunov functions associated with the Popov criterion, when the latter is applied to an appropriate reformulation of the problem. Thus, applying the Popov criterion and its combination with other IQCs, can lead to convergence rate bounds with reduced conservatism. We also illustrate via examples that the convergence rate bounds derived can be tight., Comment: 6 pages, 3 figures

Distributed nonconvex optimization problems underlie many applications in learning and autonomy, and such problems commonly face asynchrony in agents' computations and communications. When delays in these operations are bounded, they are called partially asynchronous. In this paper, we present an uncoordinated stepsize selection rule for partially asynchronous block coordinate descent that only requires local information to implement, and it leads to faster convergence for a class of nonconvex problems than existing stepsize rules, which require global information in some form. The problems we consider satisfy the error bound condition, and the stepsize rule we present only requires each agent to know (i) a certain type of Lipschitz constant of its block of the gradient of the objective and (ii) the communication delays experienced between it and its neighbors. This formulation requires less information to be available to each agent than existing approaches, typically allows for agents to use much larger stepsizes, and alleviates the impact of stragglers while still guaranteeing convergence to a stationary point. Simulation results provide comparisons and validate the faster convergence attained by the stepsize rule we develop., 12 pages, 1 figure

Distributionally robust optimization (DRO) is a powerful tool for decision making under uncertainty. It is particularly appealing because of its ability to leverage existing data. However, many practical problems call for decision-making with some auxiliary information, and DRO in the context of conditional distribution is not straightforward. We propose a conditional kernel distributionally robust optimization (CKDRO) method that enables robust decision making under conditional distributions through kernel DRO and the conditional mean operator in the reproducing kernel Hilbert space (RKHS). In particular, we consider problems where there is a correlation between the unknown variable y and an auxiliary observable variable x. Given past data of the two variables and a queried auxiliary variable, CKDRO represents the conditional distribution P(y|x) as the conditional mean operator in the RKHS space and quantifies the ambiguity set in the RKHS as well, which depends on the size of the dataset as well as the query point. To justify the use of RKHS, we demonstrate that the ambiguity set defined in RKHS can be viewed as a ball under a metric that is similar to the Wasserstein metric. The DRO is then dualized and solved via a finite dimensional convex program. The proposed CKDRO approach is applied to a generation scheduling problem and shows that the result of CKDRO is superior to common benchmarks in terms of quality and robustness.

Very high dimensional nonlinear systems arise in many engineering problems due to semi-discretization of the governing partial differential equations, e.g. through finite element methods. The complexity of these systems present computational challenges for direct application to automatic control. While model reduction has seen ubiquitous applications in control, the use of nonlinear model reduction methods in this setting remains difficult. The problem lies in preserving the structure of the nonlinear dynamics in the reduced order model for high-fidelity control. In this work, we leverage recent advances in Spectral Submanifold (SSM) theory to enable model reduction under well-defined assumptions for the purpose of efficiently synthesizing feedback controllers., Comment: 8 pages, 6 figures, conference on decision and control 2022

Bilinear dynamical systems are ubiquitous in many different domains and they can also be used to approximate more general control-affine systems. This motivates the problem of learning bilinear systems from a single trajectory of the system's states and inputs. Under a mild marginal mean-square stability assumption, we identify how much data is needed to estimate the unknown bilinear system up to a desired accuracy with high probability. Our sample complexity and statistical error rates are optimal in terms of the trajectory length, the dimensionality of the system and the input size. Our proof technique relies on an application of martingale small-ball condition. This enables us to correctly capture the properties of the problem, specifically our error rates do not deteriorate with increasing instability. Finally, we show that numerical experiments are well-aligned with our theoretical results.

In this paper, we investigate a worst-case-scenario control problem with a partially observed state. We consider a non-stochastic formulation, where noises and disturbances in our dynamics are uncertain variables which take values in finite sets. In such problems, the optimal control strategy can be derived using a dynamic program (DP) with respect to the memory. The computational complexity of this DP can be improved using a conditional range of the state instead of the memory. We present a more general definition of an information state which is sufficient to construct a DP without loss of optimality, and show that the conditional range is an example of an information state. Next, we extend this notion to define an approximate information state and an approximate DP. We also bound the maximum loss of optimality when using an approximate DP to derive the control strategy. Finally, we illustrate our results in a numerical example.

Traditional reinforcement learning (RL) aims to maximize the expected total reward, while the risk of uncertain outcomes needs to be controlled to ensure reliable performance in a risk-averse setting. In this paper, we consider the problem of maximizing dynamic risk of a sequence of rewards in infinite-horizon Markov Decision Processes (MDPs). We adapt the Expected Conditional Risk Measures (ECRMs) to the infinite-horizon risk-averse MDP and prove its time consistency. Using a convex combination of expectation and conditional value-at-risk (CVaR) as a special one-step conditional risk measure, we reformulate the risk-averse MDP as a risk-neutral counterpart with augmented action space and manipulation on the immediate rewards. We further prove that the related Bellman operator is a contraction mapping, which guarantees the convergence of any value-based RL algorithms. Accordingly, we develop a risk-averse deep Q-learning framework, and our numerical studies based on two simple MDPs show that the risk-averse setting can reduce the variance and enhance robustness of the results.

We study the problem of data-driven attack detection for unknown LTI systems using only input-output behavioral data. In contrast with model-based detectors that use errors from an output predictor to detect attacks, we study behavior-based data-driven detectors. We construct a behavior-based chi-squared detector that uses a sequence of inputs and outputs and their covariance. The covariance of the behaviors is estimated using data by two methods. The first (direct) method employs the sample covariance as an estimate of the covariance of behaviors. The second (indirect) method uses a lower dimensional generative model identified from data to estimate the covariance of behaviors. We prove the consistency of the two methods of estimation and provide finite sample error bounds. Finally, we numerically compare the performance and establish a tradeoff between the methods at different regimes of the size of the data set and the length of the detection horizon. Our numerical study indicates that neither method is invariable superior, and reveals the existence of two regimes for the performance of the two methods, wherein the direct method is superior in cases with large data sets relative to the length of the detection horizon, while the indirect method is superior in cases with small data sets., 7 pages

Multireference alignment (MRA) problem is to estimate an underlying signal from a large number of noisy circularly-shifted observations. The existing methods are always proposed under the hypothesis of a single Gaussian noise. However, the hypothesis of a single-type noise is inefficient for solving practical problems like single particle cryo-EM. In this paper, We focus on the MRA problem under the assumption of Gaussian mixture noise. We derive an adaptive variational model by combining maximum a posteriori (MAP) estimation and soft-max method. There are two adaptive weights which are for detecting cyclical shifts and types of noise. Furthermore, we provide a statistical interpretation of our model by using expectation-maximization(EM) algorithm. The existence of a minimizer is mathematically proved. The numerical results show that the proposed model has a more impressive performance than the existing methods when one Gaussian noise is large and the other is small., This article is a preprint and has not been certified by peer review

In this work, we propose an algorithm for solving exact sparse linear regression problems over a network in a distributed manner. Particularly, we consider the problem where data is stored among different computers or agents that seek to collaboratively find a common regressor with a specified sparsity k, i.e., the L0-norm is less than or equal to k. Contrary to existing literature that uses L1 regularization to approximate sparseness, we solve the problem with exact sparsity k. The main novelty in our proposal lies in showing a problem formulation with zero duality gap for which we adopt a dual approach to solve the problem in a decentralized way. This sets a foundational approach for the study of distributed optimization with explicit sparsity constraints. We show theoretically and empirically that, under appropriate assumptions, where each agent solves smaller and local integer programming problems, all agents will eventually reach a consensus on the same sparse optimal regressor.

Robust design of autonomous systems under uncertainty is an important yet challenging problem. This work proposes a robust controller that consists of a state estimator and a tube based predictive control law. The class of linear systems under ellipsoidal uncertainty is considered. In contrast to existing approaches based on polytopic sets, the constraint tightening is directly computed from the ellipsoidal sets of disturbances without over-approximation, thus leading to less conservative bounds. Conditions to guarantee robust constraint satisfaction and robust stability are presented. Further, by avoiding the usage of Minkowski sum in set computation, the proposed approach can also scale up to high-dimensional systems. The results are illustrated by examples., Comment: Accepted at CDC 2022

Conventional state estimation routines of electrical grids are mainly reliant on dynamic models of fossil fuel-based resources. These models commonly contain differential equations describing synchronous generator models and algebraic equations modeling power flow/balance equations. Fuel-free power systems that are driven by inertia-less renewable energy resources will hence require new models and upgraded estimation routines. To that end, in this paper we propose a robust estimator for an interconnected model of power networks comprised of a comprehensive ninth order synchronous generator model, advanced power electronics-based models for photovoltaic (PV) power plants, constant power loads, constant impedance loads, and motor loads. The presented state estimator design is based on Lyapunov stability criteria for nonlinear differential algebraic equation (DAE) models and is posed as a convex semi-definite optimization problem. Thorough simulations studies have been carried out on IEEE-39 bus test system to showcase the robustness of the proposed estimator against unknown uncertainty from load demand and solar irradiance., To Appear in the 61st IEEE Conference on Decision and Control (CDC), Canc\'un, Mexico, December 2022

We study the problem of synthesizing polyhedral Lyapunov functions for hybrid linear systems. Such functions are defined as convex piecewise linear functions, with a finite number of pieces. We first prove that deciding whether there exists an $m$-piece polyhedral Lyapunov function for a given hybrid linear system is NP-hard. We then present a counterexample-guided algorithm for solving this problem. The algorithm alternates between choosing a candidate polyhedral function based on a finite set of counterexamples and verifying whether the candidate satisfies the Lyapunov conditions. If the verification fails, we find a new counterexample that is added to our set. We prove that if the algorithm terminates, it discovers a valid Lyapunov function or concludes that no such Lyapunov function exists. However, our initial algorithm can be non-terminating. We modify our algorithm to provide a terminating version based on the so-called cutting-plane argument from nonsmooth optimization. We demonstrate our algorithm on numerical examples.

In this paper, we introduce a novel first-order dual gradient algorithm for solving network utility maximization problems that arise in resource allocation schemes over networks with safety-critical constraints. Inspired by applications where customers' demand can only be affected through posted prices and real-time two-way communication with customers is not available, we require an algorithm to generate \textit{safe prices}. This means that at no iteration should the realized demand in response to the posted prices violate the safety constraints of the network. Thus, in contrast to existing first-order methods, our algorithm, called the safe dual gradient method (SDGM), is guaranteed to produce feasible primal iterates at all iterations. We ensure primal feasibility by 1) adding a diminishing safety margin to the constraints, and 2) using a sign-based dual update method with different step sizes for plus and minus directions. In addition, we prove that the primal iterates produced by the SDGM achieve a sublinear static regret of ${\cal O}(\sqrt{T})$., 7 pages, 1 figure, published in the proceedings of the 61st IEEE Conference on Decision and Control

First order policy optimization has been widely used in reinforcement learning. It guarantees to find the optimal policy for the state-feedback linear quadratic regulator (LQR). However, the performance of policy optimization remains unclear for the linear quadratic Gaussian (LQG) control where the LQG cost has spurious suboptimal stationary points. In this paper, we introduce a novel perturbed policy gradient (PGD) method to escape a large class of bad stationary points (including high-order saddles). In particular, based on the specific structure of LQG, we introduce a novel reparameterization procedure which converts the iterate from a high-order saddle to a strict saddle, from which standard random perturbations in PGD can escape efficiently. We further characterize the high-order saddles that can be escaped by our algorithm., Comment: 15 pages; Y. Zheng and Y. Sun contributed to this work equally

We propose a Model Predictive Control (MPC) for collision avoidance between an autonomous agent and dynamic obstacles with uncertain predictions. The collision avoidance constraints are imposed by enforcing positive distance between convex sets representing the agent and the obstacles, and tractably reformulating them using Lagrange duality. This approach allows for smooth collision avoidance constraints even for polytopes, which otherwise require mixed-integer or non-smooth constraints. We consider three widely used descriptions of the uncertain obstacle position: 1) Arbitrary distribution with polytopic support, 2) Gaussian distributions and 3) Arbitrary distribution with first two moments known. For each case we obtain deterministic reformulations of the collision avoidance constraints. The proposed MPC formulation optimizes over feedback policies to reduce conservatism in satisfying the collision avoidance constraints. The proposed approach is validated using simulations of traffic intersections in CARLA., Comment: Accepted to CDC'22

In tabular multi-agent reinforcement learning with average-cost criterion, a team of agents sequentially interacts with the environment and observes local incentives. We focus on the case that the global reward is a sum of local rewards, the joint policy factorizes into agents' marginals, and full state observability. To date, few global optimality guarantees exist even for this simple setting, as most results yield convergence to stationarity for parameterized policies in large/possibly continuous spaces. To solidify the foundations of MARL, we build upon linear programming (LP) reformulations, for which stochastic primal-dual methods yields a model-free approach to achieve \emph{optimal sample complexity} in the centralized case. We develop multi-agent extensions, whereby agents solve their local saddle point problems and then perform local weighted averaging. We establish that the sample complexity to obtain near-globally optimal solutions matches tight dependencies on the cardinality of the state and action spaces, and exhibits classical scalings with respect to the network in accordance with multi-agent optimization. Experiments corroborate these results in practice.

This paper contrasts recursive state space models and direct multi-step predictors for linear predictive control. We provide a tutorial exposition for both model structures to solve the following problems: 1. stochastic optimal control; 2. system identification; 3. stochastic optimal control based on the estimated model. Throughout the paper, we provide detailed discussions of the benefits and limitations of these two model parametrizations for predictive control and highlight the relation to existing works. Additionally, we derive a novel (partially tight) constraint tightening for stochastic predictive control with parametric uncertainty in the multi-step predictor., Comment: Final version, IEEE Conference on Decision and Control (CDC), 2022

Cluster synchronization underlies various functions in the brain. Abnormal patterns of cluster synchronization are often associated with neurological disorders. Deep brain stimulation (DBS) is a neurosurgical technique used to treat several brain diseases, which has been observed to regulate neuronal synchrony patterns. Despite its widespread use, the mechanisms of DBS remain largely unknown. In this paper, we hypothesize that DBS plays a role similar to vibrational control since they both highly rely on high-frequency excitation to function. Under the framework of Kuramoto-oscillator networks, we study how vibrations introduced to network connections can stabilize cluster synchronization. We derive some sufficient conditions and also provide an effective approach to design vibrational control. Also, a numerical example is presented to demonstrate our theoretical findings., 7 pages, 4 figures

The paper deals with a well-known extremum seeking scheme by proving uniformity properties with respect to the amplitudes of the dither signal and of the cost function. Those properties are then used to show that the scheme guarantees the global minimiser to be semi-global practically stable despite the presence of local saddle points. To achieve these results, we analyse the average system associated with the extremum seeking scheme via arguments based on the Fourier series.

We propose a methodology for performing risk-averse quadratic regulation of partially observed Linear Time-Invariant (LTI) systems disturbed by process and output noise. To compensate against the induced variability due to both types of noises, state regulation is subject to two risk constraints. The latter renders the resulting controller cautious of stochastic disturbances, by restricting the statistical variability, namely, a simplified version of the cumulative expected predictive variance of both the state and the output. Our proposed formulation results in an optimal risk-averse policy that preserves favorable characteristics of the classical Linear Quadratic (LQ) control. In particular, the optimal policy has an affine structure with respect to the minimum mean square error (mmse) estimates. The linear component of the policy regulates the state more strictly in riskier directions, where the process and output noise covariance, cross-covariance, and the corresponding penalties are simultaneously large. This is achieved by "inflating" the state penalty in a systematic way. The additional affine terms force the state against pure and cross third-order statistics of the process and output disturbances. Another favorable characteristic of our optimal policy is that it can be pre-computed off-line, thus, avoiding limitations of prior work. Stability analysis shows that the derived controller is always internally stable regardless of parameter tuning. The functionality of the proposed risk-averse policy is illustrated through a working example via extensive numerical simulations.

In this paper, we extend a framework that we developed earlier for coordination of connected and automated vehicles (CAVs) at a signal-free intersection by integrating a safety layer using control barrier functions. First, in our motion planning module, each CAV computes the optimal control trajectory using simple vehicle dynamics. The trajectory does not make any of the state, control, and safety constraints active. A vehicle-level tracking controller employs a combined feedforward-feedback control law to track the resulting optimal trajectory from the motion planning module. Then, a barrier-certificate module, acting as a middle layer between the vehicle-level tracking controller and physical vehicle, receives the control law from the vehicle-level tracking controller and using realistic vehicle dynamics ensures that none of the state, control, and safety constraints becomes active. The latter is achieved through a quadratic program, which can be solved efficiently in real time. We demonstrate the effectiveness of our extended framework through a numerical simulation., 8 pages, 5 figures, 1 table

In this paper we study robust output tracking and disturbance rejection of linear partial differential equation (PDE) models. We focus on demonstrating how the abstract internal model based controller design methods developed for "regular linear systems" can be utilised in controller design for concrete PDE systems. We show that when implemented for PDE systems, the abstract control design methods lead in a natural way to controllers with "PDE parts". Moreover, we formulate the controller construction in a way which utilises minimal knowledge of the abstract system representation and is instead solely based on natural properties of the original PDE. We also discuss computation and approximation of the controller parameters, and illustrate the results with an example on control design for a boundary controlled diffusion equation., 13 pages, 2 figures. Proceedings of the 61st Conference on Decision and Control, 2022

A major challenge in multi-agent systems is that the system complexity grows dramatically with the number of agents as well as the size of their action spaces, which is typical in real world scenarios such as autonomous vehicles, robotic teams, network routing, etc. It is hence in imminent need to design decentralized or independent algorithms where the update of each agent is only based on their local observations without the need of introducing complex communication/coordination mechanisms. In this work, we study the finite-time convergence of independent entropy-regularized natural policy gradient (NPG) methods for potential games, where the difference in an agent's utility function due to unilateral deviation matches exactly that of a common potential function. The proposed entropy-regularized NPG method enables each agent to deploy symmetric, decentralized, and multiplicative updates according to its own payoff. We show that the proposed method converges to the quantal response equilibrium (QRE) -- the equilibrium to the entropy-regularized game -- at a sublinear rate, which is independent of the size of the action space and grows at most sublinearly with the number of agents. Appealingly, the convergence rate further becomes independent with the number of agents for the important special case of identical-interest games, leading to the first method that converges at a dimension-free rate. Our approach can be used as a smoothing technique to find an approximate Nash equilibrium (NE) of the unregularized problem without assuming that stationary policies are isolated.

Distributed energy resources (DERs) such as solar panels have small supply capacities and cannot be directly integrated into wholesale markets. So, the presence of an intermediary is critical. The intermediary could be a profit-seeking entity (called the aggregator) that buys DER supply from prosumers, and then sells them in the wholesale electricity market. Thus, DER integration has an influence on wholesale market prices, demand, and supply. The purpose of this article is to shed light onto the impact of efficient DER aggregation on the market power of conventional generators. Firstly, under efficient DER aggregation, we quantify the social welfare gap between two cases: when conventional generators are truthful, and when they are strategic. We also do the same when DERs are not present. Secondly, we show that the gap due to market power of generators in the presence of DERs is smaller than the one when there is no DER participation. Finally, we provide explicit expressions of the gaps and conduct numerical experiments to gain deeper insights. The main message of this article is that market power of conventional generators can be mitigated by adopting an efficient DER aggregation model.

We consider a discounted cost constrained Markov decision process (CMDP) policy optimization problem, in which an agent seeks to maximize a discounted cumulative reward subject to a number of constraints on discounted cumulative utilities. To solve this constrained optimization program, we study an online actor-critic variant of a classic primal-dual method where the gradients of both the primal and dual functions are estimated using samples from a single trajectory generated by the underlying time-varying Markov processes. This online primal-dual natural actor-critic algorithm maintains and iteratively updates three variables: a dual variable (or Lagrangian multiplier), a primal variable (or actor), and a critic variable used to estimate the gradients of both primal and dual variables. These variables are updated simultaneously but on different time scales (using different step sizes) and they are all intertwined with each other. Our main contribution is to derive a finite-time analysis for the convergence of this algorithm to the global optimum of a CMDP problem. Specifically, we show that with a proper choice of step sizes the optimality gap and constraint violation converge to zero in expectation at a rate $\mathcal{O}(1/K^{1/6})$, where K is the number of iterations. To our knowledge, this paper is the first to study the finite-time complexity of an online primal-dual actor-critic method for solving a CMDP problem. We also validate the effectiveness of this algorithm through numerical simulations.

We develop a novel formulation of the Performance Estimation Problem (PEP) for decentralized optimization whose size is independent of the number of agents in the network. The PEP approach allows computing automatically the worst-case performance and worst-case instance of first-order optimization methods by solving an SDP. Unlike previous work, the size of our new PEP formulation is independent of the network size. For this purpose, we take a global view of the decentralized problem and we also decouple the consensus subspace and its orthogonal complement. We apply our methodology to different decentralized methods such as DGD, DIGing and EXTRA and obtain numerically tight performance guarantees that are valid for any network size., 8 pages, 3 figures, accepted for Conference on Decision and Control 2022. arXiv admin note: text overlap with arXiv:2203.05963

In this work, we revisit the Linear Quadratic Gaussian (LQG) optimal control problem from a behavioral perspective. Motivated by the suitability of behavioral models for data-driven control, we begin with a reformulation of the LQG problem in the space of input-output behaviors and obtain a complete characterization of the optimal solutions. In particular, we show that the optimal LQG controller can be expressed as a static behavioral-feedback gain, thereby eliminating the need for dynamic state estimation characteristic of state space methods. The static form of the optimal LQG gain also makes it amenable to its computation by gradient descent, which we investigate via numerical experiments. Furthermore, we highlight the advantage of this approach in the data-driven control setting of learning the optimal LQG controller from expert demonstrations., To appear in CDC 2022

This work investigates a problem of simultaneous global cost minimization and Nash equilibrium seeking, which commonly exists in N-cluster non-cooperative games. Specifically, the players in the same cluster collaborate to minimize a global cost function, being a summation of their individual cost functions, and jointly play a non-cooperative game with other clusters as players. For the problem settings, we suppose that the explicit analytical expressions of the players' local cost functions are unknown, but the function values can be measured. We propose a gradient-free Nash equilibrium seeking algorithm by a synthesis of Gaussian smoothing techniques and gradient tracking. Furthermore, instead of using the uniform coordinated step-size, we allow the players across different clusters to choose different constant step-sizes. When the largest step-size is sufficiently small, we prove a linear convergence of the players' actions to a neighborhood of the unique Nash equilibrium under a strongly monotone game mapping condition, with the error gap being propotional to the largest step-size and the smoothing parameter. The performance of the proposed algorithm is validated by numerical simulations. National Research Foundation (NRF) Submitted/Accepted version This work was supported in part by the Republic of Singapore's National Research Foundation under its Campus for Research Excellence and Technological Enterprise (CREATE) Programme through a grant to the Berkeley Education Alliance for Research in Singapore (BEARS) for the Singapore-Berkeley Building Efficiency and Sustainability in the Tropics (SinBerBEST) Program.