Search

The ability to direct a Probabilistic Boolean Network (PBN) to a desired state is important to applications such as targeted therapeutics in cancer biology. Reinforcement Learning (RL) has been proposed as a framework that solves a discrete-time optimal control problem cast as a Markov Decision Process. We focus on an integrative framework powered by a model-free deep RL method that can address different flavours of the control problem (e.g., withorwithout control inputs; attractor stateora subset of the state space as the target domain). The method is agnostic to the distribution of probabilities for the next state, hence it does not use the probability transition matrix. The time complexity is onlylinearon the time steps, or interactions between the agent (deep RL) and the environment (PBN), during training. Indeed, we explore thescalabilityof the deep RL approach to (set) stabilization of large-scale PBNs and demonstrate successful control on large networks, including a metastatic melanoma PBN with200 nodes.

We study the French-DeGroot opinion dynamics in a social network with two polarizing parties. We consider a network in which the leaders of one party are given, and we pose the problem of selecting the leader set of the opposing party so as to shift the average opinion to a desired value. When each party has only one leader, we express the average opinion in terms of the transition matrix and the stationary distribution of random walks in the network. The analysis shows balance of influence between the two leader nodes. We show that the problem of selecting at most $k$ absolute leaders to shift the average opinion is $\mathbf{NP}$-hard. Then, we reduce the problem to a problem of submodular maximization with a submodular knapsack constraint and an additional cardinality constraint and propose a greedy algorithm with upper bound search to approximate the optimum solution. We also conduct experiments in random networks and real-world networks to show the effectiveness of the algorithm., Comment: 14 pages, 4 figures

Designing the optimal linear quadratic regulator (LQR) for a large-scale multi-agent system (MAS) is time-consuming since it involves solving a large-size matrix Riccati equation. The situation is further exasperated when the design needs to be done in a model-free way using schemes such as reinforcement learning (RL). To reduce this computational complexity, we decompose the large-scale LQR design problem into multiple smaller-size LQR design problems. We consider the objective function to be specified over an undirected graph, and cast the decomposition as a graph clustering problem. The graph is decomposed into two parts, one consisting of independent clusters of connected components, and the other containing edges that connect different clusters. Accordingly, the resulting controller has a hierarchical structure, consisting of two components. The first component optimizes the performance of each independent cluster by solving the smaller-size LQR design problem in a model-free way using an RL algorithm. The second component accounts for the objective coupling different clusters, which is achieved by solving a least squares problem in one shot. Although suboptimal, the hierarchical controller adheres to a particular structure as specified by inter-agent couplings in the objective function and by the decomposition strategy. Mathematical formulations are established to find a decomposition that minimizes the number of required communication links or reduces the optimality gap. Numerical simulations are provided to highlight the pros and cons of the proposed designs., Comment: This paper proposes a hierarchical learning and control framework for model-free LQR of heterogeneous linear multi-agent systems

In this paper, we consider a network of processors aiming at cooperatively solving mixed-integer convex programs subject to uncertainty. Each node only knows a common cost function and its local uncertain constraint set. We propose a randomized, distributed algorithm working under asynchronous, unreliable and directed communication. The algorithm is based on a local computation and communication paradigm. At each communication round, nodes perform two updates: (i) a verification in which they check---in a randomized fashion---the robust feasibility of a candidate optimal point, and (ii) an optimization step in which they exchange their candidate basis (the minimal set of constraints defining a solution) with neighbors and locally solve an optimization problem. As main result, we show that processors can stop the algorithm after a finite number of communication rounds (either because verification has been successful for a sufficient number of rounds or because a given threshold has been reached), so that candidate optimal solutions are consensual. The common solution is proven to be---with high confidence---feasible and hence optimal for the entire set of uncertainty except a subset having an arbitrary small probability measure. We show the effectiveness of the proposed distributed algorithm using two examples: a random, uncertain mixed-integer linear program and a distributed localization in wireless sensor networks. The distributed algorithm is implemented on a multi-core platform in which the nodes communicate asynchronously., Comment: Submitted for publication. arXiv admin note: text overlap with arXiv:1706.00488

We consider a distribution grid used to charge electric vehicles such that voltage drops stay bounded. We model this as a class of resource-sharing networks, known as bandwidth-sharing networks in the communication network literature. We focus on resource-sharing networks that are driven by a class of greedy control rules that can be implemented in a decentralized fashion. For a large number of such control rules, we can characterize the performance of the system by a fluid approximation. This leads to a set of dynamic equations that take into account the stochastic behavior of EVs. We show that the invariant point of these equations is unique and can be computed by solving a specific ACOPF problem, which admits an exact convex relaxation. We illustrate our findings with a case study using the SCE 47-bus network and several special cases that allow for explicit computations., 13 pages, 8 figures

Large-scale integration of distributed energy resources into residential distribution feeders necessitates careful control of their operation through power flow analysis. While the knowledge of the distribution system model is crucial for this type of analysis, it is often unavailable or outdated. The recent introduction of synchrophasor technology in low-voltage distribution grids has created an unprecedented opportunity to learn this model from high-precision, time-synchronized measurements of voltage and current phasors at various locations. This paper focuses on joint estimation of model parameters (admittance values) and operational structure of a poly-phase distribution network from the available telemetry data via the lasso, a method for regression shrinkage and selection. We propose tractable convex programs capable of tackling the low rank structure of the distribution system and develop an online algorithm for early detection and localization of critical events that induce a change in the admittance matrix. The efficacy of these techniques is corroborated through power flow studies on four three-phase radial distribution systems serving real household demands.

We study a class of structured optimal control problems in which the main diagonal of the dynamic matrix is a linear function of the design variable. While such problems are in general challenging and nonconvex, for positive systems we prove convexity of the $H_2$ and $H_\infty$ optimal control formulations which allow for arbitrary convex constraints and regularization of the control input. Moreover, we establish differentiability of the $H_\infty$ norm when the graph associated with the dynamical generator is weakly connected and develop a customized algorithm for computing the optimal solution even in the absence of differentiability. We apply our results to the problems of leader selection in directed consensus networks and combination drug therapy for HIV treatment. In the context of leader selection, we address the combinatorial challenge by deriving upper and lower bounds on optimal performance. For combination drug therapy, we develop a customized subgradient method for efficient treatment of diseases whose mutation patterns are not connected., 11 pages, 7 figures

In this paper, we study a strategic model of marketing and product consumption in social networks. We consider two firms in a market competing to maximize the consumption of their products. Firms have a limited budget which can be either invested on the quality of the product or spent on initial seeding in the network in order to better facilitate spread of the product. After the decision of firms, agents choose their consumptions following a myopic best response dynamics which results in a local, linear update for their consumption decision. We characterize the unique Nash equilibrium of the game between firms and study the effect of the budgets as well as the network structure on the optimal allocation. We show that at the equilibrium, firms invest more budget on quality when their budgets are close to each other. However, as the gap between budgets widens, competition in qualities becomes less effective and firms spend more of their budget on seeding. We also show that given equal budget of firms, if seeding budget is nonzero for a balanced graph, it will also be nonzero for any other graph, and if seeding budget is zero for a star graph it will be zero for any other graph as well. As a practical extension, we then consider a case where products have some preset qualities that can be only improved marginally. At some point in time, firms learn about the network structure and decide to utilize a limited budget to mount their market share by either improving the quality or new seeding some agents to incline consumers towards their products. We show that the optimal budget allocation in this case simplifies to a threshold strategy. Interestingly, we derive similar results to that of the original problem, in which preset qualities simulate the role that budgets had in the original setup., arXiv admin note: substantial text overlap with arXiv:1404.1405

We consider the problem of estimating the state of a noisy linear dynamical system when an unknown subset of sensors is arbitrarily corrupted by an adversary. We propose a secure state estimation algorithm, and derive (optimal) bounds on the achievable state estimation error given an upper bound on the number of attacked sensors. The proposed state estimator involves Kalman filters operating over subsets of sensors to search for a sensor subset which is reliable for state estimation. To further improve the subset search time, we propose Satisfiability Modulo Theory based techniques to exploit the combinatorial nature of searching over sensor subsets. Finally, as a result of independent interest, we give a coding theoretic view of attack detection and state estimation against sensor attacks in a noiseless dynamical system., Submitted to IEEE TCNS special issue on secure control of cyber physical systems. A preliminary version of this work appeared in the proceedings of ISIT 2015. arXiv admin note: text overlap with arXiv:1504.05566

We study the problem of motion feasibility for multiagent control systems on Lie groups with collision avoidance constraints. We first consider the problem for kinematic left invariant control systems and next, for dynamical control systems given by a left-trivialized Lagrangian function. Solutions of the kinematic problem give rise to linear combinations of the control inputs in a linear subspace annihilating the collision avoidance constraints. In the dynamical problem, motion feasibility conditions are obtained by using techniques from variational calculus on manifolds, given by a set of equations in a vector space, and Lagrange multipliers annihilating the constraint force that prevents deviation of solutions from a constraint submanifold., L. J. Colombo was partially supported by ACCESS Linnaeus Center (KTH), Ministerio de Econom{\i}a, Industria y Competitividad grant MTM2016-76702-P, i-Link project and La Caixa Foundation (LCF/BQ/PI19/11690016). D. V. Dimarogonas is supported by the Swedish Research Council (VR), Knut och Alice Wallenberg foundation (KAW), the H2020 Project Co4Robots and the H2020 ERC Starting Grant BUCOPHSYS