Search

In this paper, a novel control strategy namely disturbance observer-based control is first applied to stabilization and disturbance rejection for an anti-stable stochastic heat equation with Neumann boundary actuation and unknown boundary external disturbance generated by an exogenous system. A disturbance observer-based boundary control is designed based on the backstepping approach and estimation/cancellation strategy, where the unknown disturbance is estimated in real time by a disturbance observer and rejected in the closed-loop, while the in-domain multiplicative noise whose intensity is within a known finite interval is attenuated. It is shown that the resulting closed-loop system is exponentially stable in the sense of both mean square and almost surely. A numerical example is demonstrated to validate the effectiveness of the proposed control approach.

We illustrate the potential of PDE-based traffic flow control in cardiovascular flow analysis, monitoring, and control, presenting a PDE-based control-oriented formulation, for 1-D blood flow dynamics in the presence of stenosis. This is achieved adopting an approach for modeling and analysis that relies on the potential correspondence of 1-D blood flow dynamics in the presence of stenosis, with 1-D traffic flow dynamics in the presence of bottleneck. We reveal such correspondence in relation to the respective (for the two flow types), speed dynamics and a (consistent with them) fundamental diagram-based reduction; bottleneck dynamic effects description and resulting boundary conditions; and free-flow/congested regimes characterization., Comment: Submitted to IEEE Transactions on Automatic Control on October 22, 2021

Linear models with additive unknown-but-bounded input disturbances are extensively used to model uncertainty in robust control systems design. Typically, the disturbance set is either assumed to be known a priori or estimated from data through set-membership identification. However, the problem of computing a suitable input disturbance set in case the set of possible output values is assigned a priori has received relatively little attention. This problem arises in many contexts, such as in supervisory control, actuator design, decentralized control, and others. In this paper, we propose a method to compute input disturbance sets (and the corresponding set of states) such that the resulting set of outputs matches as closely as possible a given set of outputs, while additionally satisfying strict (inner or outer) inclusion constraints. We formulate the problem as an optimization problem by relying on the concept of robust invariance. The effectiveness of the approach is demonstrated in numerical examples that illustrate how to solve safe reference set and input-constraint set computation problems.

In this paper, we address the efficient implementation of moving horizon state estimation of constrained discrete-time linear systems. We propose a novel iteration scheme which employs a proximity-based formulation of the underlying optimization algorithm and reduces computational effort by performing only a limited number of optimization iterations each time a new measurement is received. We outline conditions under which global exponential stability of the underlying estimation errors is ensured. Performance guarantees of the iteration scheme in terms of regret upper bounds are also established. A combined result shows that both exponential stability and a sublinear regret which can be rendered smaller by increasing the number of optimization iterations can be guaranteed. The stability and regret results of the proposed estimator are showcased through numerical simulations.

The rate-regulation tradeoff, defined between two objective functions, one penalizing the packet rate and one the regulation cost, can express the fundamental performance bound of networked control systems. However, the characterization of the set of globally optimal solutions in this tradeoff for multi-dimensional Gauss-Markov processes has been an open problem. In the present article, we characterize a policy profile that belongs to this set without imposing any restrictions on the information structure or the policy structure. We prove that such a policy profile consists of a symmetric threshold triggering policy based on the value of information and a certainty-equivalent control policy based on a non-Gaussian linear estimator. These policies are deterministic and can be designed separately. Besides, we provide a global optimality analysis for the value of information $\text{VoI}_k$, a semantic metric that emerges from the rate-regulation tradeoff as the difference between the benefit and the cost of a data packet. We prove that it is globally optimal that a data packet containing sensory information at time $k$ be transmitted to the controller only if $\text{VoI}_k$ becomes nonnegative. These results have important implications in the areas of communication and control., Comment: arXiv admin note: text overlap with arXiv:1812.07534

A dual adaptive model predictive control (MPC) algorithm is presented for linear, time-invariant systems subject to bounded disturbances and parametric uncertainty in the state-space matrices. Online set-membership identification is performed to reduce the uncertainty and thus control affects both the informativity of identification and the system’s performance. The main contribution of the paper is to include this dual effect in the MPC optimization problem using a predicted worst-case cost in the objective function. This allows the controller to perform active exploration, that is, the control input reduces the uncertainty in the regions of the parameter space that have most influence on the performance. Additionally, the MPC algorithm ensures robust constraint satisfaction of state and input constraints. Advantages of the proposed algorithm are shown by comparing it to a passive adaptive MPC algorithm from the literature., IEEE Transactions on Automatic Control, 68 (5), ISSN:0018-9286, ISSN:1558-2523

In the past few years, Online Convex Optimization (OCO) has received notable attention in the control literature thanks to its flexible real-time nature and powerful performance guarantees. In this paper, we propose new step-size rules and OCO algorithms that simultaneously exploit gradient predictions, function predictions and dynamics, features particularly pertinent to control applications. The proposed algorithms enjoy static and dynamic regret bounds in terms of the dynamics of the reference action sequence, gradient prediction error, and function prediction error, which are generalizations of known regularity measures from the literature. We present results for both convex and strongly convex costs. We validate the performance of the proposed algorithms in a trajectory tracking case study, as well as portfolio optimization using real-world datasets.

This paper considers the distributed online convex optimization problem with time-varying constraints over a network of agents. This is a sequential decision making problem with two sequences of arbitrarily varying convex loss and constraint functions. At each round, each agent selects a decision from the decision set, and then only a portion of the loss function and a coordinate block of the constraint function at this round are privately revealed to this agent. The goal of the network is to minimize the network-wide loss accumulated over time. Two distributed online algorithms with full-information and bandit feedback are proposed. Both dynamic and static network regret bounds are analyzed for the proposed algorithms, and network cumulative constraint violation is used to measure constraint violation, which excludes the situation that strictly feasible constraints can compensate the effects of violated constraints. In particular, we show that the proposed algorithms achieve $\mathcal{O}(T^{\max\{\kappa,1-\kappa\}})$ static network regret and $\mathcal{O}(T^{1-\kappa/2})$ network cumulative constraint violation, where $T$ is the time horizon and $\kappa\in(0,1)$ is a user-defined trade-off parameter. Moreover, if the loss functions are strongly convex, then the static network regret bound can be reduced to $\mathcal{O}(T^{\kappa})$. Finally, numerical simulations are provided to illustrate the effectiveness of the theoretical results.

Mirror descent (MD) is a powerful first-order optimization technique that subsumes several optimization algorithms including gradient descent (GD). In this work, we develop a semi-definite programming (SDP) framework to analyze the convergence rate of MD in centralized and distributed settings under both strongly convex and non-strongly convex assumptions. We view MD with a dynamical system lens and leverage quadratic constraints (QCs) to provide explicit convergence rates based on Lyapunov stability. For centralized MD under strongly convex assumption, we develop a SDP that certifies exponential convergence rates. We prove that the SDP always has a feasible solution that recovers the optimal GD rate as a special case. We complement our analysis by providing the $O(1/k)$ convergence rate for convex problems. Next, we analyze the convergence of distributed MD and characterize the rate using SDP. To the best of our knowledge, the numerical rate of distributed MD has not been previously reported in the literature. We further prove an $O(1/k)$ convergence rate for distributed MD in the convex setting. Our numerical experiments on strongly convex problems indicate that our framework certifies superior convergence rates compared to the existing rates for distributed GD.

Determining the reachable set for a given nonlinear control system is crucial for system control and planning. However, computing such a set is impossible if the system's dynamics are not fully known. This paper is motivated by a scenario where a system suffers an adverse event mid-operation, resulting in a substantial change to the system's dynamics, rendering them largely unknown. Our objective is to conservatively approximate the system's reachable set solely from its local dynamics at a single point and the bounds on the rate of change of its dynamics. We translate this knowledge about the system dynamics into an ordinary differential inclusion. We then derive a conservative approximation of the velocities available to the system at every system state. An inclusion using this approximation can be interpreted as a control system; the trajectories of the derived control system are guaranteed to be trajectories of the unknown system. To illustrate the practical implementation and consequences of our work, we apply our algorithm to a simplified model of an unmanned aerial vehicle.

We develop a probabilistic control algorithm, $\texttt{GTLProCo}$, for swarms of agents with heterogeneous dynamics and objectives, subject to high-level task specifications. The resulting algorithm not only achieves decentralized control of the swarm but also significantly improves scalability over state-of-the-art existing algorithms. Specifically, we study a setting in which the agents move along the nodes of a graph, and the high-level task specifications for the swarm are expressed in a recently-proposed language called graph temporal logic (GTL). By constraining the distribution of the swarm over the nodes of the graph, GTL can specify a wide range of properties, including safety, progress, and response. $\texttt{GTLProCo}$, agnostic to the number of agents comprising the swarm, controls the density distribution of the swarm in a decentralized and probabilistic manner. To this end, it synthesizes a time-varying Markov chain modeling the time evolution of the density distribution under the GTL constraints. We first identify a subset of GTL, namely reach-avoid specifications, for which we can reduce the synthesis of such a Markov chain to either linear or semi-definite programs. Then, in the general case, we formulate the synthesis of the Markov chain as a mixed-integer nonlinear program (MINLP). We exploit the structure of the problem to provide an efficient sequential mixed-integer linear programming scheme with trust regions to solve the MINLP. We empirically demonstrate that our sequential scheme is at least three orders of magnitude faster than off-the-shelf MINLP solvers and illustrate the effectiveness of $\texttt{GTLProCo}$ in several swarm scenarios., Comment: Initial submission to TAC

This work contributes to the field of optimal control of bilinear systems. It concerns a continuous time, finite dimensional, bilinear state equation with a quadratic performance index to be minimized. The state equation is non-autonomous and comprises a deterministic, a-priori known excitation. The control trajectory is constrained to an admissible set without a specific structure. The performance index is a functional, quadratic in the state variables and control signals. Krotov's method is used for solving this problem by means of an improving sequence. To this end, the required sequence of an improving functions is formulated. Finally, the solution is encapsulated in an algorithm form and a numerical example of structural control problem is provided., \copyright 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

In this work, we study decentralized convex constrained optimization problems in networks. We focus on the dual averaging-based algorithmic framework that is well-documented to be superior in handling constraints and complex communication environments simultaneously. Two new decentralized dual averaging (DDA) algorithms are proposed. In the first one, a second-order dynamic average consensus protocol is tailored for DDA-type algorithms, which equips each agent with a provably more accurate estimate of the global dual variable than conventional schemes. We rigorously prove that the proposed algorithm attains $\mathcal{O}(1/t)$ convergence for general convex and smooth problems, for which existing DDA methods were only known to converge at $\mathcal{O}(1/\sqrt{t})$ prior to our work. In the second one, we use the extrapolation technique to accelerate the convergence of DDA. Compared to existing accelerated algorithms, where typically two different variables are exchanged among agents at each time, the proposed algorithm only seeks consensus on local gradients. Then, the extrapolation is performed based on two sequences of primal variables which are determined by the accumulations of gradients at two consecutive time instants, respectively. The algorithm is proved to converge at $\mathcal{O}(1)\left(\frac{1}{t^2}+\frac{1}{t(1-\beta)^2}\right)$, where $\beta$ denotes the second largest singular value of the mixing matrix. We remark that the condition for the algorithmic parameter to guarantee convergence does not rely on the spectrum of the mixing matrix, making itself easy to satisfy in practice. Finally, numerical results are presented to demonstrate the efficiency of the proposed methods., Comment: 15 pages, 4 figures

Distributed linear algebraic equation over networks, where nodes hold a part of problem data and cooperatively solve the equation via node-to-node communications, is a basic distributed computation task receiving an increasing research attention. Communications over a network have a stochastic nature, with both temporal and spatial dependence due to link failures, packet dropouts or node recreation, etc. In this paper, we study the convergence and convergence rate of distributed linear equation protocols over a $\ast$-mixing random network, where the temporal and spatial dependencies between the node-to-node communications are allowed. When the network linear equation admits exact solutions, we prove the mean-squared exponential convergence rate of the distributed projection consensus algorithm, while the lower and upper bound estimations of the convergence rate are also given for independent and identically distributed (i.i.d.) random graphs. Motivated by the randomized Kaczmarz algorithm, we also propose a distributed randomized projection consensus algorithm, where each node randomly selects one row of local linear equations for projection per iteration, and establish an exponential convergence rate. When the network linear equation admits no exact solution, we prove that a distributed gradient-descent-like algorithm with diminishing step-sizes can drive all nodes' states to a least-squares solution at a sublinear rate. These results collectively illustrate that distributed computations may overcome communication correlations if the prototype algorithms enjoy certain contractive properties or are designed with suitable parameters., 17 pages

This paper studies the adaptive optimal stationary control of continuous-time linear stochastic systems with both additive and multiplicative noises, using reinforcement learning techniques. Based on policy iteration, a novel off-policy reinforcement learning algorithm, named optimistic least-squares-based policy iteration, is proposed which is able to find iteratively near-optimal policies of the adaptive optimal stationary control problem directly from input/state data without explicitly identifying any system matrices, starting from an initial admissible control policy. The solutions given by the proposed optimistic least-squares-based policy iteration are proved to converge to a small neighborhood of the optimal solution with probability one, under mild conditions. The application of the proposed algorithm to a triple inverted pendulum example validates its feasibility and effectiveness., Comment: 10 pages, 3 figures

Motivated by various distributed control applications, we consider a linear system with Gaussian noise observed by multiple sensors which transmit measurements over a dynamic lossy network. We characterize the stationary optimal sensor scheduling policy for the finite horizon, discounted, and long-term average cost problems and show that the value iteration algorithm converges to a solution of the average cost problem. We further show that the suboptimal policies provided by the rolling horizon truncation of the value iteration also guarantee stability and provide near-optimal average cost. Lastly, we provide qualitative characterizations of the multidimensional set of measurement loss rates for which the system is stabilizable for a static network, significantly extending earlier results on intermittent observations., 25 pages

Robustness is a basic property of any control system. In the context of linear output regulation, it was proved that embedding an internal model of the exogenous signals is necessary and sufficient to achieve tracking of the desired reference signals in spite of external disturbances and parametric uncertainties. This result is commonly known as "internal model principle". A complete extension of such linear result to general nonlinear systems is still an open problem, which is exacerbated by the large number of alternative definitions of uncertainty and desired control goals that are possible in a nonlinear setting. In this paper, we develop a general framework in which all these different notions can be formally characterized in a unifying way. Classical results are reinterpreted in the proposed setting, and new results and insights are presented with a focus on robust rejection/tracking of arbitrary harmonic content. Moreover, we show by counter-example that, in the relevant case of continuous unstructured uncertainties, there are problems for which no smooth finite-dimensional robust regulator exists ensuring exact regulation., updated version 04.2021

Semidefinite and sum-of-squares (SOS) optimization are fundamental computational tools in many areas, including linear and nonlinear systems theory. However, the scale of problems that can be addressed reliably and efficiently is still limited. In this paper, we introduce a new notion of block factor-width-two matrices and build a new hierarchy of inner and outer approximations of the cone of positive semidefinite (PSD) matrices. This notion is a block extension of the standard factor-width-two matrices, and allows for an improved inner-approximation of the PSD cone. In the context of SOS optimization, this leads to a block extension of the scaled diagonally dominant sum-of-squares (SDSOS) polynomials. By varying a matrix partition, the notion of block factor-width-two matrices can balance a trade-off between the computation scalability and solution quality for solving semidefinite and SOS optimization problems. Numerical experiments on a range of large-scale instances confirm our theoretical findings., Accepted for publication as a regular paper at IEEE TAC. Code is available through https://github.com/zhengy09/SDPfw

The persistence of excitation (PE) condition is sufficient to ensure parameter convergence in adaptive estimation problems. Recent results on adaptive estimation in reproducing kernel Hilbert spaces (RKHS) introduce PE conditions for RKHS. This paper presents sufficient conditions for PE for the particular class of uniformly embedded reproducing kernel Hilbert spaces (RKHS) defined over smooth Riemannian manifolds. This paper also studies the implications of the sufficient condition in the case when the RKHS is finite or infinite-dimensional. When the RKHS is finite-dimensional, the sufficient condition implies parameter convergence as in the conventional analysis. On the other hand, when the RKHS is infinite-dimensional, the same condition implies that the function estimate error is ultimately bounded by a constant that depends on the approximation error in the infinite-dimensional RKHS. We illustrate the effectiveness of the sufficient condition in a practical example., Comment: 11 pages, 2 figures

This paper presents a family of algorithms for decentralized convex composite problems. We consider the setting of a network of agents that cooperatively minimize a global objective function composed of a sum of local functions plus a regularizer. Through the use of intermediate consensus variables, we remove the need for inner communication loops between agents when computing curvature-guided updates. A general scheme is presented which unifies the analysis for a plethora of computing choices, including gradient descent, Newton updates, and BFGS updates. Our analysis establishes sublinear convergence rates under convex objective functions with Lipschitz continuous gradients, as well as linear convergence rates when the local functions are further assumed to be strongly convex. Moreover, we explicitly characterize the acceleration due to curvature information. Last but not the least, we present an asynchronous implementation for the proposed algorithms, which removes the need for a central clock, with linear convergence rates established in expectation under strongly convex objectives. We ascertain the effectiveness of the proposed methods with numerical experiments on benchmark datasets.

This paper builds Wasserstein ambiguity sets for the unknown probability distribution of dynamic random variables leveraging noisy partial-state observations. The constructed ambiguity sets contain the true distribution of the data with quantifiable probability and can be exploited to formulate robust stochastic optimization problems with out-of-sample guarantees. We assume the random variable evolves in discrete time under uncertain initial conditions and dynamics, and that noisy partial measurements are available. All random elements have unknown probability distributions and we make inferences about the distribution of the state vector using several output samples from multiple realizations of the process. To this end, we leverage an observer to estimate the state of each independent realization and exploit the outcome to construct the ambiguity sets. We illustrate our results in an economic dispatch problem involving distributed energy resources over which the scheduler has no direct control.

In this paper, we introduce a fast row-stochastic decentralized algorithm, referred to as FRSD, to solve consensus optimization problems over directed communication graphs. The proposed algorithm only utilizes row-stochastic weights, leading to certain practical advantages in broadcast communication settings over those requiring column-stochastic weights. Under the assumption that each node-specific function is smooth and strongly convex, we show that the FRSD iterate sequence converges with a linear rate to the optimal consensus solution. In contrast to the existing methods for directed networks, FRSD enjoys linear convergence without employing a gradient tracking (GT) technique explicitly, rather it implements GT implicitly with the use of a novel momentum term, which leads to a significant reduction in communication and storage overhead for each node when FRSD is implemented for solving high-dimensional problems over small-to-medium scale networks. In the numerical tests, we compare FRSD with other state-of-the-art methods, which use row-stochastic and/or column-stochastic weights.

This work considered an online distributed optimization problem, with a group of agents whose local objective functions vary with time. Moreover, the value of the objective function is revealed to the corresponding agent after the decision is executed per time-step. Thus, each agent can only update the decision variable based on the revealed value and information collected from the neighbors, without the knowledge on the explicit expression of the objective function. To solve this problem, an online gradient-free distributed projected gradient descent (DPGD) algorithm is presented, where each agent locally approximates the gradient based on two point values. With some standard assumptions on the communication graph and the objective functions, we provide the bound for the dynamic regret as a function of the minimizer path length, step-size and smoothing parameter. Under appropriate selections of the step-size and smoothing parameter, we prove that the dynamic regret is sublinear with respect to the time duration $T$ if the minimizer path length also grows sublinearly. Finally, the effectiveness of the proposed algorithm is illustrated through numerical simulations.

A control system is called self-powered if the only energy it requires for operation is that which it absorbs from the plant. For a linear feedback law to be feasible for a self-powered control system, its feedback signal must be colocated with the control inputs, and its input-output mapping must satisfy an associated passivity constraint. The imposition of such a feedback law can be viewed equivalently as the imposition of a linear passive shunt admittance at the actuation ports of the plant. In this paper we consider the use of actively-controlled electronics to impose a self-powered linear feedback law. To be feasible, it is insufficient that the imposed admittance be passive, because parasitic losses must additionally be overcome. We derive sufficient feasibility conditions which explicitly account for these losses. In the finite-dimensional, time-invariant case, the feasibility condition distills to a more conservative version of the Positive Real Lemma, which is parametrized by various loss parameters. Three examples are given, in which this condition is used to determine the least-efficient loss parameters necessary to realize a desired feedback law.

In this paper, we study the landscape of an online nonconvex optimization problem, for which the input data vary over time and the solution is a trajectory rather than a single point. To understand the complexity of finding a global solution of this problem, we introduce the notion of \textit{spurious (i.e., non-global) local trajectory} as a generalization to the notion of spurious local solution in nonconvex (time-invariant) optimization. We develop an ordinary differential equation (ODE) associated with a time-varying nonlinear dynamical system which, at limit, characterizes the spurious local solutions of the time-varying optimization problem. We prove that the absence of spurious local trajectory is closely related to the transient behavior of the developed system. In particular, we show that if the problem is time-varying, the data variation may force all of the ODE trajectories initialized at arbitrary local minima at the initial time to gradually converge to the global solution trajectory. We study the Jacobian of the dynamical system along a local minimum trajectory and show how its eigenvalues are manipulated by the natural data variation in the problem, which may consequently trigger escaping poor local minima over time.

This paper studies periodic event-triggered networked control for nonlinear systems, where the plants and controllers are connected by multiple independent communication channels. Several network-induced imperfections are considered simultaneously, including time-varying inter-sampling times, sensor node scheduling, and especially, large time-varying transmission delays, where the transmitted signal may arrive at the destination node after the next transmission occurs. A new hybrid system approach is provided to model the closed-loop system that contains all communication related behavior. Then, by constructing new storage functions on the system state and updating errors, the relationship between the maximum allowable sampling period (MASP) and maximum allowable delay number in sampling (MADNS) is analyzed, where the latter denotes how many inter-sampling periods can be included in one transmission delay. Moreover, to efficiently reduce unnecessary transmissions, a new dynamic event-triggered control scheme is proposed, where the event-triggering conditions are detected only at aperiodic and asynchronous sampling instants. From emulation-based method, where the controllers are initially designed by ignoring all the network-induced imperfections, sufficient conditions on the dynamic event-triggered control are given to ensure closed-loop input-to-state stability with respect to external disturbances. Moreover, according to different capacities of the communication channels, the corresponding implementation strategies of the designed dynamic event-triggered control schemes are discussed. Finally, two nonlinear examples are simulated to illustrate the feasibility and efficiency of the theoretical results., Comment: 18 pages, 6 figures, 3 tables

We propose a new distributed optimization algorithm for solving a class of constrained optimization problems in which (a) the objective function is separable (i.e., the sum of local objective functions of agents), (b) the optimization variables of distributed agents, which are subject to nontrivial local constraints, are coupled by global constraints, and (c) only noisy observations are available to estimate (the gradients of) local objective functions. In many practical scenarios, agents may not be willing to share their optimization variables with others. For this reason, we propose a distributed algorithm that does not require the agents to share their optimization variables with each other; instead, each agent maintains a local estimate of the global constraint functions and share the estimate only with its neighbors. These local estimates of constraint functions are updated using a consensus-type algorithm, while the local optimization variables of each agent are updated using a first-order method based on noisy estimates of gradient. We prove that, when the agents adopt the proposed algorithm, their optimization variables converge with probability 1 to an optimal point of an approximated problem based on the penalty method., 8 pages

We consider the problem of controlling the group behavior of a large number of dynamic systems that are constantly interacting with each other. These systems are assumed to have identical dynamics (e.g., birds flock, robot swarm) and their group behavior can be modeled by a distribution. Thus, this problem can be viewed as an optimal control problem over the space of distributions. We propose a novel algorithm to compute a feedback control strategy so that, when adopted by the agents, the distribution of them would be transformed from an initial one to a target one over a finite time window. Our method is built on optimal transport theory but differs significantly from existing work in this area in that our method models the interactions among agents explicitly. From an algorithmic point of view, our algorithm is based on a generalized version of the proximal gradient descent algorithm and has a convergence guarantee with a sublinear rate. We further extend our framework to account for the scenarios where the agents are from multiple species. In the linear quadratic setting, the solution is characterized by coupled Riccati equations which can be solved in closed-form. Finally, several numerical examples are presented to illustrate our framework., Comment: 14 pages, 10 figures

We study privacy-utility trade-offs where users share privacy-correlated useful information with a service provider to obtain some utility. The service provider is adversarial in the sense that it can infer the users' private information based on the shared useful information. To minimize the privacy leakage while maintaining a desired level of utility, the users carefully perturb the useful information via a probabilistic privacy mapping before sharing it. We focus on the setting in which the adversary attempting an inference attack on the users' privacy has potentially biased information about the statistical correlation between the private and useful variables. This information asymmetry between the users and the limited adversary leads to better privacy guarantees than the case of the omniscient adversary under the same utility requirement. We first identify assumptions on the adversary's information so that the inference costs are well-defined and finite. Then, we characterize the impact of the information asymmetry and show that it increases the inference costs for the adversary. We further formulate the design of the privacy mapping against a limited adversary using a difference of convex functions program and solve it via the concave-convex procedure. When the adversary's information is not precisely available, we adopt a Bayesian view and represent the adversary's information by a probability distribution. In this case, the expected cost for the adversary does not admit a closed-form expression, and we establish and maximize a lower bound of the expected cost. We provide a numerical example regarding a census data set to illustrate the theoretical results.

We consider the static output feedback control for Linear Quadratic Regulator problems with structured constraints under the assumption that system parameters are unknown. To solve the problem in the model free setting, we propose the policy gradient algorithm based on the gradient projection method and show its global convergence to $\varepsilon$-stationary points. In addition, we introduce a variance reduction technique and show both theoretically and numerically that it significantly reduces the variance in the gradient estimation. We also show in the numerical experiments that the model free approach efficiently solves the problem.

This work focuses on the fulfillment of the Persistent Excitation (PE) condition for signals which result from transformations by means of polynomials. This is essential e.g. for the convergence of Adaptive Dynamic Programming algorithms due to commonly used polynomial function approximators. As theoretical statements are scarce regarding the nonlinear transformation of PE signals, we propose conditions on the system state such that its transformation by polynomials is PE. To validate our theoretical statements, we develop an exemplary excitation procedure based on our conditions using a feedforward control approach and demonstrate the effectiveness of our method in a nonzero-sum differential game. In this setting, our approach outperforms commonly used probing noise in terms of convergence time and the degree of PE, shown by a numerical example.

In this technical note we analyse the performance improvement and optimality properties of the Learning Model Predictive Control (LMPC) strategy for linear deterministic systems. The LMPC framework is a policy iteration scheme where closed-loop trajectories are used to update the control policy for the next execution of the control task. We show that, when a Linear Independence Constraint Qualification (LICQ) condition holds, the LMPC scheme guarantees strict iterative performance improvement and optimality, meaning that the closed-loop cost evaluated over the entire task converges asymptotically to the optimal cost of the infinite-horizon control problem. Compared to previous works this sufficient LICQ condition can be easily checked, it holds for a larger class of systems and it can be used to adaptively select the prediction horizon of the controller, as demonstrated by a numerical example., Comment: technical note

Information compression is essential to reduce communication cost in distributed optimization over peer-to-peer networks. This paper proposes a communication-efficient linearly convergent distributed (COLD) algorithm to solve strongly convex optimization problems. By compressing innovation vectors, which are the differences between decision vectors and their estimates, COLD is able to achieve linear convergence for a class of $\delta$-contracted compressors. We explicitly quantify how the compression affects the convergence rate and show that COLD matches the same rate of its uncompressed version. To accommodate a wider class of compressors that includes the binary quantizer, we further design a novel dynamical scaling mechanism and obtain the linearly convergent Dyna-COLD. Importantly, our results strictly improve existing results for the quantized consensus problem. Numerical experiments demonstrate the advantages of both algorithms under different compressors., Comment: 14 pages

We consider a multi-agent optimal resource sharing problem that is represented by a linear program. The amount of resource to be shared is fixed, and agents belong to a population that is characterized probabilistically so as to allow heterogeneity among the agents. In this paper, we provide a characterization of the probability that the arrival of a new agent affects the resource share of other agents, which means that accommodating the new agent request at the detriment of the other agents allocation provides some payoff. This probability represents a sensitivity index for the optimal solution of a linear programming resource sharing problem when a new agent shows up, and it is of fundamental importance for a correct and profitable operation of the multi-agent system. Our developments build on the equivalence between the resource sharing problem and certain dual reformulations which can be interpreted as scenario programs with the number of scenarios corresponding to the number of agents in the primal problem. The recent "wait-and-judge" scenario approach is then used to obtain the sought sensitivity index. Our theoretical findings are demonstrated through a numerical example on optimal cargo aircraft loading., Comment: 14 pages, 3 figures

In this work, we propose a tube-based MPC scheme for state- and input-constrained linear systems subject to dynamic uncertainties characterized by dynamic integral quadratic constraints (IQCs). In particular, we extend the framework of $\rho$-hard IQCs for exponential stability analysis to external inputs. This result yields that the error between the true uncertain system and the nominal prediction model is bounded by an exponentially stable scalar system. In the proposed tube-based MPC scheme, the state of this error bounding system is predicted along with the nominal model and used as a scaling parameter for the tube size. We prove that this method achieves robust constraint satisfaction and input-to-state stability despite dynamic uncertainties and additive bounded disturbances. A numerical example demonstrates the reduced conservatism of this IQC approach compared to state-of-the-art robust MPC approaches for dynamic uncertainties.

In this work, we characterize a class of nonlinear monotone dynamical systems that have a certain translation invariance property which goes by the name of plus-homogeneity; usually called "topical" systems. Such systems need not be asymptotically stable, since they are merely nonexpansive but not contractive. Thus, we introduce a stricter version of monotonicity, termed "type-K" in honor of Kamke, and we prove the asymptotic stability of the equilibrium points, as well as the convergence of all trajectories to such equilibria for type-K monotone and plus-homogeneous systems: we call them "K-topical". Since topical maps are the natural nonlinear counterpart of linear maps defined by row-stochastic matrices, which are a cornerstone in the convergence analysis of linear multi-agent systems (MASs), we exploit our results for solving the consensus problem over nonlinear K-topical MASs. We first provide necessary and sufficient conditions on the local interaction rules of the agents ensuring the K-topicality of a MAS. Then, we prove that the agents achieve consensus asymptotically if the graph describing their interactions contains a globally reachable node. Finally, several examples for continuous-time and discrete-time systems are discussed to corroborate the enforceability of our results in different applications.

This paper presents converse theorems for safety in terms of barrier functions for unconstrained continuous-time systems modeled as differential inclusions. Via a counterexample, we show the lack of existence of autonomous and continuous barrier functions certifying safety for a nonlinear system that is not only safe but also has a smooth right-hand side. Guided by converse Lyapunov theorems for (non-asymptotic) stability,time-varying barrier functions and appropriate infinitesimal conditions are shown to be both necessary as well as sufficient under mild regularity conditions on the right-hand side of the system. More precisely, we propose a general construction of a time-varying barrier function in terms of a marginal function involving the finite-horizon reachable set. Using techniques from set-valued and nonsmooth analysis, we show that such a function guarantees safety when the system is safe. Furthermore, we show that the proposed barrier function construction inherits the regularity properties of the proposed reachable set. In addition, when the system is safe and smooth, we build upon the constructed barrier function to show the existence of a smooth barrier function guaranteeing safety. Comparisons and relationships to results in the literature are also presented.

We provide a stability and performance analysis for nonlinear model predictive control (NMPC) schemes subject to input constraints. Given an exponential stabilizability and detectability condition w.r.t. the employed state cost, we provide a sufficiently long prediction horizon to ensure asymptotic stability and a desired performance bound w.r.t. the infinite-horizon optimal controller. Compared to existing results, the provided analysis is applicable to positive semi-definite (detectable) cost functions, provides tight bounds using a linear programming analysis, and allows for a seamless integration of general positive-definite terminal cost functions in the analysis. The practical applicability of the derived theoretical results are demonstrated with numerical examples., Comment: This is the accepted version of the paper in IEEE Transaction on Automatic Control, 2023. This version contains additionally the proof of Theorem 7 in the appendix

In this paper we propose novel optimization-based methods for verifying reach-avoid (or, eventuality) properties of continuous-time systems modelled by ordinary differential equations. Given a system, an initial set, a safe set and a target set of states, we say that the reach-avoid property holds if for all initial conditions in the initial set, any trajectory of the system starting at them will eventually, i.e.\ in unbounded yet finite time, enter the target set while remaining inside the safe set until that first target hit. Based on a discount value function, two sets of quantified constraints are derived for verifying the reach-avoid property via the computation of exponential/asymptotic guidance-barrier functions (they form a barrier escorting the system to the target set safely at an exponential or asymptotic rate). It is interesting to find that one set of constraints whose solution is termed exponential guidance-barrier functions is just a simplified version of the existing one derived from the moment based method, while the other one whose solution is termed asymptotic guidance-barrier functions is completely new. Furthermore, built upon this new set of constraints, we derive a set of more expressive constraints, which includes the aforementioned two sets of constraints as special instances, providing more chances for verifying the reach-avoid properties successfully. When the involved datum are polynomials, i.e., the initial set, safe set and target set are semi-algebraic, and the system has polynomial dynamics, the problem of solving these sets of constraints can be framed as a semi-definite optimization problem using sum-of-squares decomposition techniques and thus can be efficiently solved in polynomial time via interior point methods. Finally, several examples demonstrate the theoretical developments and performance of proposed methods.

Extremum seeking systems are powerful methods able to steer the input of a (dynamical) cost function towards an optimizer, without any prior knowledge of the cost function. To achieve their objective, they typically combine time-periodic signals with the on-line measurement of the cost. However, in some practical applications, the cost can only be measured during some regular time-intervals, and not continuously, contravening the classical extremum seeking framework. In this paper, we first analyze how existing Lie-bracket based extremum seeking systems behave when being fed with intermittent measurements, instead of continuous ones. We then propose two modifications of those schemes to improve both the convergence time and the steady-state accuracy in presence of intermittent measurements. The performances of the different schemes are compared on a case study.

We study an optimal targeting problem for super-modular games with binary actions and finitely many players. The considered problem consists in the selection of a subset of players of minimum size such that, when the actions of these players are forced to a controlled value while the others are left to repeatedly play a best response action, the system will converge to the greatest Nash equilibrium of the game. Our main contributions consist in showing that the problem is NP-complete and in proposing an efficient iterative algorithm with provable convergence properties for its solution. We discuss in detail the special case of network coordination games and its relation with the notion of cohesiveness. Finally, we show with simulations the strength of our approach with respect to naive heuristics based on classical network centrality measures.

This paper considers a distributed reinforcement learning problem for decentralized linear quadratic control with partial state observations and local costs. We propose a Zero-Order Distributed Policy Optimization algorithm (ZODPO) that learns linear local controllers in a distributed fashion, leveraging the ideas of policy gradient, zero-order optimization and consensus algorithms. In ZODPO, each agent estimates the global cost by consensus, and then conducts local policy gradient in parallel based on zero-order gradient estimation. ZODPO only requires limited communication and storage even in large-scale systems. Further, we investigate the nonasymptotic performance of ZODPO and show that the sample complexity to approach a stationary point is polynomial with the error tolerance's inverse and the problem dimensions, demonstrating the scalability of ZODPO. We also show that the controllers generated throughout ZODPO are stabilizing controllers with high probability. Lastly, we numerically test ZODPO on multi-zone HVAC systems.

We propose decentralized primal-dual methods for cooperative multi-agent consensus optimization problems over both static and time-varying communication networks, where only local communications are allowed. The objective is to minimize the sum of agent-specific convex functions over conic constraint sets defined by agent-specific nonlinear functions; hence, the optimal consensus decision should lie in the intersection of these private sets. Under the strong convexity assumption, we provide convergence rates for sub-optimality, infeasibility, and consensus violation in terms of the number of communications required; examine the effect of underlying network topology on the convergence rates., A preliminary result of this paper was presented in arXiv:1706.07907 by Hamedani and Aybat. In this paper, we generalize our results to the setting where agent-specific constraints are defined by nonlinear functions rather than linear ones which greatly improves the modeling capability. This generalization requires a more complicated analysis which is studied in this separate arXiv submission

We study the convergence properties of an overlapping Schwarz decomposition algorithm for solving nonlinear optimal control problems (OCPs). The algorithm decomposes the time domain into a set of overlapping subdomains, and solves all subproblems defined over subdomains in parallel. The convergence is attained by updating primal-dual information at the boundaries of overlapping subdomains. We show that the algorithm exhibits local linear convergence, and that the convergence rate improves exponentially with the overlap size. We also establish global convergence results for a general quadratic programming, which enables the application of the Schwarz scheme inside second-order optimization algorithms (e.g., sequential quadratic programming). The theoretical foundation of our convergence analysis is a sensitivity result of nonlinear OCPs, which we call "exponential decay of sensitivity" (EDS). Intuitively, EDS states that the impact of perturbations at domain boundaries (i.e. initial and terminal time) on the solution decays exponentially as one moves into the domain. Here, we expand a previous analysis available in the literature by showing that EDS holds for both primal and dual solutions of nonlinear OCPs, under uniform second-order sufficient condition, controllability condition, and boundedness condition. We conduct experiments with a quadrotor motion planning problem and a PDE control problem to validate our theory; and show that the approach is significantly more efficient than ADMM and as efficient as the centralized solver Ipopt., 16 pages

The distributed optimization problem is set up in a collection of nodes interconnected via a communication network. The goal is to find the minimizer of a global objective function formed by the addition of partial functions locally known at each node. A number of methods are available for addressing this problem, having different advantages. The goal of this work is to achieve the maximum possible convergence rate. As the first step towards this end, we propose a new method which we show converges faster than other available options. As with most distributed optimization methods, convergence rate depends on a step size parameter. As the second step towards our goal we complement the proposed method with a fully distributed method for estimating the optimal step size that maximizes convergence speed. We provide theoretical guarantees for the convergence of the resulting method in a neighborhood of the solution. Also, for the case in which the global objective function has a single local minimum, we provide a different step size selection criterion together with theoretical guarantees for convergence. We present numerical experiments showing that, when using the same step size, our method converges significantly faster than its rivals. Experiments also show that the distributed step size estimation method achieves an asymptotic convergence rate very close to the theoretical maximum.

Finite-dimensional observer-based controller design for PDEs is a challenging problem. Recently, such controllers were introduced for the 1D heat equation, under the assumption that one of the observation or control operators is bounded. This paper suggests a constructive method for such controllers for 1D parabolic PDEs with both (observation and control) operators being unbounded. We consider the Kuramoto-Sivashinsky equation (KSE) under either boundary or in-domain point measurement and boundary actuation. We employ a modal decomposition approach via dynamic extension, using eigenfunctions of a Sturm-Liouville operator. The controller dimension is defined by the number of unstable modes, whereas the observer dimension $N$ may be larger than this number. We suggest a direct Lyapunov approach to the full-order closed-loop system, which results in an LMI whose elements and dimension depend on $N$. The value of $N$ and the decay rate are obtained from the LMI. We extend our approach to internal stabilization with guaranteed $L^2$-gain and input-to-state stabilization. We prove two crucial properties of the derived LMIs. First, We prove that the LMIs are always feasible provided $N$ and the $L^2$ or ISS gains are large enough, thereby obtaining guarantees for our approach. Moreover, for the case of stabilization, we show that feasibility of the LMI for some $N$ implies its feasibility for $N+1$ (i.e., enlarging $N$ in the LMI cannot deteriorate the resulting decay rate of the closed-loop system). Numerical examples demonstrate the efficiency of the method.

This paper proposes a method for enlarging the region of attraction of Linear Model Predictive Controllers (MPC) when tracking piecewise-constant references in the presence of pointwise-in-time constraints. It consists of an add-on unit, the Feasibility Governor (FG), that manipulates the reference command so as to ensure that the optimal control problem that underlies the MPC feedback law remains feasible. Offline polyhedral projection algorithms based on multi-objective linear programming are employed to compute the set of feasible states and reference commands. Online, the action of the FG is computed by solving a convex quadratic program. The closed-loop system is shown to satisfy constraints, be asymptotically stable, exhibit zero-offset tracking, and display finite-time convergence of the reference.

Thermostatically controlled loads (TCLs) can provide ancillary services to the power network by aiding existing frequency control mechanisms. TCLs are, however, characterized by an intrinsic limit cycle behavior which raises the risk that these could synchronize when coupled with the frequency dynamics of the power grid, i.e. simultaneously switch, inducing persistent and possibly catastrophic power oscillations. To address this problem, schemes with a randomized response time in their control policy have been proposed in the literature. However, such schemes introduce delays in the response of TCLs to frequency feedback that may limit their ability to provide fast support at urgencies. In this paper, we present a deterministic control mechanism for TCLs such that those switch when prescribed frequency thresholds are exceeded in order to provide ancillary services to the power network. For the considered scheme, we provide analytic conditions which ensure that synchronization is avoided. In particular, we show that as the number of loads tends to infinity, there exist arbitrarily long time intervals where the frequency deviations are arbitrarily small. Our analytical results are verified with simulations on the Northeast Power Coordinating Council (NPCC) 140-bus system, which demonstrate that the proposed scheme offers improved frequency response compared to conventional implementations., Comment: 10 pages, 5 figures, 1 table

We present a novel data-driven model predictive control (MPC) approach to control unknown nonlinear systems using only measured input-output data with closed-loop stability guarantees. Our scheme relies on the data-driven system parametrization provided by the Fundamental Lemma of Willems et al. We use new input-output measurements online to update the data, exploiting local linear approximations of the underlying system. We prove that our MPC scheme, which only requires solving strictly convex quadratic programs online, ensures that the closed loop (practically) converges to the (unknown) optimal reachable equilibrium that tracks a desired output reference while satisfying polytopic input constraints. As intermediate results of independent interest, we extend the Fundamental Lemma to affine systems and we derive novel robustness bounds w.r.t. noisy data for the open-loop optimal control problem, which are directly transferable to other data-driven MPC schemes in the literature. The applicability of our approach is illustrated with a numerical application to a continuous stirred tank reactor.

We optimize a general model of bioprocesses, which is nonconvex due to the microbial growth in the biochemical reactors. We formulate a convex relaxation and give conditions guaranteeing its exactness in both the transient and steady state cases. When the growth kinetics are modeled by the Monod function under constant biomass or the Contois function, the relaxation is a second-order cone program, which can be solved efficiently at large scales. We implement the model on a numerical example based on a wastewater treatment system.