Search

We study task selection to enhance sample efficiency in model-agnostic meta-reinforcement learning (MAML-RL). Traditional meta-RL typically assumes that all available tasks are equally important, which can lead to task redundancy when they share significant similarities. To address this, we propose a coreset-based task selection approach that selects a weighted subset of tasks based on how diverse they are in gradient space, prioritizing the most informative and diverse tasks. Such task selection reduces the number of samples needed to find an $\epsilon$-close stationary solution by a factor of O(1/$\epsilon$). Consequently, it guarantees a faster adaptation to unseen tasks while focusing training on the most relevant tasks. As a case study, we incorporate task selection to MAML-LQR (Toso et al., 2024b), and prove a sample complexity reduction proportional to O(log(1/$\epsilon$)) when the task specific cost also satisfy gradient dominance. Our theoretical guarantees underscore task selection as a key component for scalable and sample-efficient meta-RL. We numerically validate this trend across multiple RL benchmark problems, illustrating the benefits of task selection beyond the LQR baseline.

We investigate the profitability and risk of energy storage arbitrage in electricity markets under price uncertainty, exploring both robust and chance-constrained optimization approaches. We analyze various uncertainty representations, including polyhedral, ellipsoidal uncertainty sets and probabilistic approximations, to model price fluctuations and construct efficient frontiers that highlight the tradeoff between risk and profit. Using historical electricity price data, we quantify the impact of uncertainty on arbitrage strategies and compare their performance under distinct market conditions. The results reveal that arbitrage strategies under uncertainties can effectively secure expected profits, and robust strategies perform better in risk management across varying levels of conservativeness, especially under highly volatile market conditions. This work provides insights into storage arbitrage strategy selection for market participants with differing risk preferences, emphasizing the adaptability of efficient frontiers to the electricity market.

Defective coloring (also known as relaxed or improper coloring) is a generalization of proper coloring defined as follows: for $d \in \mathbb{N}$, a coloring of a graph is $d$-defective if every vertex is colored the same as at most $d$ of its neighbors. We investigate defective coloring of planar graphs in the context of correspondence coloring, a generalization of list coloring introduced by Dvo\v{r}\'ak and Postle. First we show there exists a planar graph that is not $3$-defective $3$-correspondable, strengthening a recent result of Cho, Choi, Kim, Park, Shan, and Zhu. Then we construct a planar graph that is $1$-defective $3$-correspondable but not $4$-correspondable, thereby extending a recent result of Ma, Xu, and Zhu from list coloring to correspondence coloring. Finally we show all outerplanar graphs are $3$-defective $2$-correspondence colorable, with 3 defects being best possible., Comment: 23 pages

This paper presents two novel algorithms for approximately projecting symmetric matrices onto the Positive Semidefinite (PSD) cone using Randomized Numerical Linear Algebra (RNLA). Classical PSD projection methods rely on full-rank deterministic eigen-decomposition, which can be computationally prohibitive for large-scale problems. Our approach leverages RNLA to construct low-rank matrix approximations before projection, significantly reducing the required numerical resources. The first algorithm utilizes random sampling to generate a low-rank approximation, followed by a standard eigen-decomposition on this smaller matrix. The second algorithm enhances this process by introducing a scaling approach that aligns the leading-order singular values with the positive eigenvalues, ensuring that the low-rank approximation captures the essential information about the positive eigenvalues for PSD projection. Both methods offer a trade-off between accuracy and computational speed, supported by probabilistic error bounds. To further demonstrate the practical benefits of our approach, we integrate the randomized projection methods into a first-order Semi-Definite Programming (SDP) solver. Numerical experiments, including those on SDPs derived from Sum-of-Squares (SOS) programming problems, validate the effectiveness of our method, especially for problems that are infeasible with traditional deterministic methods.

Representation learning is a powerful tool that enables learning over large multitudes of agents or domains by enforcing that all agents operate on a shared set of learned features. However, many robotics or controls applications that would benefit from collaboration operate in settings with changing environments and goals, whereas most guarantees for representation learning are stated for static settings. Toward rigorously establishing the benefit of representation learning in dynamic settings, we analyze the regret of multi-task representation learning for linear-quadratic control. This setting introduces unique challenges. Firstly, we must account for and balance the $\textit{misspecification}$ introduced by an approximate representation. Secondly, we cannot rely on the parameter update schemes of single-task online LQR, for which least-squares often suffices, and must devise a novel scheme to ensure sufficient improvement. We demonstrate that for settings where exploration is "benign", the regret of any agent after $T$ timesteps scales as $\tilde O(\sqrt{T/H})$, where $H$ is the number of agents. In settings with "difficult" exploration, the regret scales as $\tilde O(\sqrt{d_u d_\theta} \sqrt{T} + T^{3/4}/H^{1/5})$, where $d_x$ is the state-space dimension, $d_u$ is the input dimension, and $d_\theta$ is the task-specific parameter count. In both cases, by comparing to the minimax single-task regret $O(\sqrt{d_x d_u^2}\sqrt{T})$, we see a benefit of a large number of agents. Notably, in the difficult exploration case, by sharing a representation across tasks, the effective task-specific parameter count can often be small $d_\theta < d_x d_u$. Lastly, we provide numerical validation of the trends we predict.

We explore a Federated Reinforcement Learning (FRL) problem where $N$ agents collaboratively learn a common policy without sharing their trajectory data. To date, existing FRL work has primarily focused on agents operating in the same or ``similar" environments. In contrast, our problem setup allows for arbitrarily large levels of environment heterogeneity. To obtain the optimal policy which maximizes the average performance across all potentially completely different environments, we propose two algorithms: FedSVRPG-M and FedHAPG-M. In contrast to existing results, we demonstrate that both FedSVRPG-M and FedHAPG-M, both of which leverage momentum mechanisms, can exactly converge to a stationary point of the average performance function, regardless of the magnitude of environment heterogeneity. Furthermore, by incorporating the benefits of variance-reduction techniques or Hessian approximation, both algorithms achieve state-of-the-art convergence results, characterized by a sample complexity of $\mathcal{O}\left(\epsilon^{-\frac{3}{2}}/N\right)$. Notably, our algorithms enjoy linear convergence speedups with respect to the number of agents, highlighting the benefit of collaboration among agents in finding a common policy.

Meta-learning methods typically learn tasks under the assumption that all tasks are equally important. However, this assumption is often not valid. In real-world applications, tasks can vary both in their importance during different training stages and in whether they contain noisy labeled data or not, making a uniform approach suboptimal. To address these issues, we propose the Data-Efficient and Robust Task Selection (DERTS) algorithm, which can be incorporated into both gradient and metric-based meta-learning algorithms. DERTS selects weighted subsets of tasks from task pools by minimizing the approximation error of the full gradient of task pools in the meta-training stage. The selected tasks are efficient for rapid training and robust towards noisy label scenarios. Unlike existing algorithms, DERTS does not require any architecture modification for training and can handle noisy label data in both the support and query sets. Analysis of DERTS shows that the algorithm follows similar training dynamics as learning on the full task pools. Experiments show that DERTS outperforms existing sampling strategies for meta-learning on both gradient-based and metric-based meta-learning algorithms in limited data budget and noisy task settings., Comment: Accepted by CVPR 2024 Wrokshop

Electricity markets are experiencing a rapid increase in energy storage unit participation. Unlike conventional generation resources, quantifying the competitive operation and identifying if a storage unit is exercising market power is challenging, particularly in the context of multi-interval bidding strategies. We present a framework to differentiate strategic capacity withholding behaviors attributed to market power from inherent competitive bidding in storage unit strategies. Our framework evaluates the profitability of strategic storage unit participation, analyzing bidding behaviors as both price takers and price makers using a self-scheduling model, and investigates how they leverage market inefficiencies. Specifically, we propose a price sensitivity model derived from the linear supply function equilibrium model to examine the price-anticipating bidding strategy, effectively capturing the influence of market power. We introduce a sufficient ex-post analysis for market operators to identify potential exploitative behaviors by monitoring instances of withholding within the bidding profiles, ensuring market resilience and competitiveness. We discuss and verify applicability of the proposed framework to realistic settings. Our analysis substantiates commonly observed economic bidding behaviors of storage units. Furthermore, it demonstrates that significant price volatility offers considerable profit opportunities not only for participants possessing market power but also for typical strategic profit seekers.

We address the problem of designing an LQR controller in a distributed setting, where M similar but not identical systems share their locally computed policy gradient (PG) estimates with a server that aggregates the estimates and computes a controller that, on average, performs well on all systems. Learning in a distributed setting has the potential to offer statistical benefits - multiple datasets can be leveraged simultaneously to produce more accurate policy gradient estimates. However, the interplay of heterogeneous trajectory data and varying levels of local computational power introduce bias to the aggregated PG descent direction, and prevents us from fully exploiting the parallelism in the distributed computation. The latter stems from synchronous aggregation, where straggler systems negatively impact the runtime. To address this, we propose an asynchronous policy gradient algorithm for LQR control design. By carefully controlling the "staleness" in the asynchronous aggregation, we show that the designed controller converges to each system's $\epsilon$-near optimal controller up to a heterogeneity bias. Furthermore, we prove that our asynchronous approach obtains exact local convergence at a sub-linear rate., Comment: Leonardo F. Toso and Han Wang contributed equally to this work

A graph $G$ is $k$-locally sparse if for each vertex $v \in V(G)$, the subgraph induced by its neighborhood contains at most $k$ edges. Alon, Krivelevich, and Sudakov showed that for $f > 0$ if a graph $G$ of maximum degree $\Delta$ is $\Delta^2/f$-locally-sparse, then $\chi(G) = O\left(\Delta/\log f\right)$. We introduce a more general notion of local sparsity by defining graphs $G$ to be $(k, F)$-locally-sparse for some graph $F$ if for each vertex $v \in V(G)$ the subgraph induced by the neighborhood of $v$ contains at most $k$ copies of $F$. Employing the R\"{o}dl nibble method, we prove the following generalization of the above result: for every bipartite graph $F$, if $G$ is $(k, F)$-locally-sparse, then $\chi(G) = O\left( \Delta /\log\left(\Delta k^{-1/|V(F)|}\right)\right)$. This improves upon results of Davies, Kang, Pirot, and Sereni who consider the case when $F$ is a path. Our results also recover the best known bound on $\chi(G)$ when $G$ is $K_{1, t, t}$-free for $t \geq 4$, and hold for list and correspondence coloring in the more general so-called ''color-degree'' setting., Comment: 30 pages, 1 figure

Federated reinforcement learning (FRL) has emerged as a promising paradigm for reducing the sample complexity of reinforcement learning tasks by exploiting information from different agents. However, when each agent interacts with a potentially different environment, little to nothing is known theoretically about the non-asymptotic performance of FRL algorithms. The lack of such results can be attributed to various technical challenges and their intricate interplay: Markovian sampling, linear function approximation, multiple local updates to save communication, heterogeneity in the reward functions and transition kernels of the agents' MDPs, and continuous state-action spaces. Moreover, in the on-policy setting, the behavior policies vary with time, further complicating the analysis. In response, we introduce FedSARSA, a novel federated on-policy reinforcement learning scheme, equipped with linear function approximation, to address these challenges and provide a comprehensive finite-time error analysis. Notably, we establish that FedSARSA converges to a policy that is near-optimal for all agents, with the extent of near-optimality proportional to the level of heterogeneity. Furthermore, we prove that FedSARSA leverages agent collaboration to enable linear speedups as the number of agents increases, which holds for both fixed and adaptive step-size configurations., Comment: Published as a conference paper at ICLR 2024

We introduce a new tool useful for greedy coloring, which we call the forb-flex method, and apply it to odd coloring and proper conflict-free coloring of planar graphs. The odd chromatic number, denoted $\chi_{\mathsf{o}}(G)$, is the smallest number of colors needed to properly color $G$ such that every non-isolated vertex of $G$ has a color appearing an odd number of times in its neighborhood. The proper conflict-free chromatic number, denoted $\chi_{\mathsf{PCF}}(G)$, is the smallest number of colors needed to properly color $G$ such that every non-isolated vertex of $G$ has a color appearing uniquely in its neighborhood. Our new tool works by carefully counting the structures in the neighborhood of a vertex and determining if a neighbor of a vertex can be recolored at the end of a greedy coloring process to avoid conflicts. Combining this with the discharging method allows us to prove $\chi_{\mathsf{PCF}}(G) \leq 4$ for planar graphs of girth at least 11, and $\chi_{\mathsf{o}}(G) \leq 4$ for planar graphs of girth at least 10. These results improve upon the recent works of Cho, Choi, Kwon, and Park., Comment: 32 pages, 11 figures

We investigate the problem of learning linear quadratic regulators (LQR) in a multi-task, heterogeneous, and model-free setting. We characterize the stability and personalization guarantees of a policy gradient-based (PG) model-agnostic meta-learning (MAML) (Finn et al., 2017) approach for the LQR problem under different task-heterogeneity settings. We show that our MAML-LQR algorithm produces a stabilizing controller close to each task-specific optimal controller up to a task-heterogeneity bias in both model-based and model-free learning scenarios. Moreover, in the model-based setting, we show that such a controller is achieved with a linear convergence rate, which improves upon sub-linear rates from existing work. Our theoretical guarantees demonstrate that the learned controller can efficiently adapt to unseen LQR tasks.

We characterize Borel line graphs in terms of 10 forbidden induced subgraphs, namely the 9 finite graphs from the classical result of Beineke together with a 10th infinite graph associated to the equivalence relation $\mathbb{E}_0$ on the Cantor space. As a corollary, we prove a partial converse to the Feldman--Moore theorem, which allows us to characterize all locally countable Borel line graphs in terms of their Borel chromatic numbers., Comment: 18 pages

Federated reinforcement learning (FedRL) enables agents to collaboratively train a global policy without sharing their individual data. However, high communication overhead remains a critical bottleneck, particularly for natural policy gradient (NPG) methods, which are second-order. To address this issue, we propose the FedNPG-ADMM framework, which leverages the alternating direction method of multipliers (ADMM) to approximate global NPG directions efficiently. We theoretically demonstrate that using ADMM-based gradient updates reduces communication complexity from ${O}({d^{2}})$ to ${O}({d})$ at each iteration, where $d$ is the number of model parameters. Furthermore, we show that achieving an $\epsilon$-error stationary convergence requires ${O}(\frac{1}{(1-\gamma)^{2}{\epsilon}})$ iterations for discount factor $\gamma$, demonstrating that FedNPG-ADMM maintains the same convergence rate as the standard FedNPG. Through evaluation of the proposed algorithms in MuJoCo environments, we demonstrate that FedNPG-ADMM maintains the reward performance of standard FedNPG, and that its convergence rate improves when the number of federated agents increases., Comment: Accepted at the 37th Conference on Neural Information Processing Systems (NeurIPS 2023)

We investigate the problem of learning an $\epsilon$-approximate solution for the discrete-time Linear Quadratic Regulator (LQR) problem via a Stochastic Variance-Reduced Policy Gradient (SVRPG) approach. Whilst policy gradient methods have proven to converge linearly to the optimal solution of the model-free LQR problem, the substantial requirement for two-point cost queries in gradient estimations may be intractable, particularly in applications where obtaining cost function evaluations at two distinct control input configurations is exceptionally costly. To this end, we propose an oracle-efficient approach. Our method combines both one-point and two-point estimations in a dual-loop variance-reduced algorithm. It achieves an approximate optimal solution with only $O\left(\log\left(1/\epsilon\right)^{\beta}\right)$ two-point cost information for $\beta \in (0,1)$.

We study a model-free federated linear quadratic regulator (LQR) problem where M agents with unknown, distinct yet similar dynamics collaboratively learn an optimal policy to minimize an average quadratic cost while keeping their data private. To exploit the similarity of the agents' dynamics, we propose to use federated learning (FL) to allow the agents to periodically communicate with a central server to train policies by leveraging a larger dataset from all the agents. With this setup, we seek to understand the following questions: (i) Is the learned common policy stabilizing for all agents? (ii) How close is the learned common policy to each agent's own optimal policy? (iii) Can each agent learn its own optimal policy faster by leveraging data from all agents? To answer these questions, we propose a federated and model-free algorithm named FedLQR. Our analysis overcomes numerous technical challenges, such as heterogeneity in the agents' dynamics, multiple local updates, and stability concerns. We show that FedLQR produces a common policy that, at each iteration, is stabilizing for all agents. We provide bounds on the distance between the common policy and each agent's local optimal policy. Furthermore, we prove that when learning each agent's optimal policy, FedLQR achieves a sample complexity reduction proportional to the number of agents M in a low-heterogeneity regime, compared to the single-agent setting.

A powerful concept behind much of the recent progress in machine learning is the extraction of common features across data from heterogeneous sources or tasks. Intuitively, using all of one's data to learn a common representation function benefits both computational effort and statistical generalization by leaving a smaller number of parameters to fine-tune on a given task. Toward theoretically grounding these merits, we propose a general setting of recovering linear operators $M$ from noisy vector measurements $y = Mx + w$, where the covariates $x$ may be both non-i.i.d. and non-isotropic. We demonstrate that existing isotropy-agnostic representation learning approaches incur biases on the representation update, which causes the scaling of the noise terms to lose favorable dependence on the number of source tasks. This in turn can cause the sample complexity of representation learning to be bottlenecked by the single-task data size. We introduce an adaptation, $\texttt{De-bias & Feature-Whiten}$ ($\texttt{DFW}$), of the popular alternating minimization-descent scheme proposed independently in Collins et al., (2021) and Nayer and Vaswani (2022), and establish linear convergence to the optimal representation with noise level scaling down with the $\textit{total}$ source data size. This leads to generalization bounds on the same order as an oracle empirical risk minimizer. We verify the vital importance of $\texttt{DFW}$ on various numerical simulations. In particular, we show that vanilla alternating-minimization descent fails catastrophically even for iid, but mildly non-isotropic data. Our analysis unifies and generalizes prior work, and provides a flexible framework for a wider range of applications, such as in controls and dynamical systems., Comment: Appeared at ICLR 2024 (spotlight presentation)

Graph signal processing (GSP) has emerged as a powerful tool for practical network applications, including power system monitoring. Recent research has focused on developing GSP-based methods for state estimation, attack detection, and topology identification using the representation of the power system voltages as smooth graph signals. Within this framework, efficient methods have been developed for detecting false data injection (FDI) attacks, which until now were perceived as non-smooth with respect to the graph Laplacian matrix. Consequently, these methods may not be effective against smooth FDI attacks. In this paper, we propose a graph FDI (GFDI) attack that minimizes the Laplacian-based graph total variation (TV) under practical constraints. We present the GFDI attack as the solution for a non-convex constrained optimization problem. The solution to the GFDI attack problem is obtained through approximating it using $\ell_1$ relaxation. A series of quadratic programming problems that are classified as convex optimization problems are solved to obtain the final solution. We then propose a protection scheme that identifies the minimal set of measurements necessary to constrain the GFDI output to a high graph TV, thereby enabling its detection by existing GSP-based detectors. Our numerical simulations on the IEEE-57 and IEEE-118 bus test cases reveal the potential threat posed by well-designed GSP-based FDI attacks. Moreover, we demonstrate that integrating the proposed protection design with GSP-based detection can lead to significant hardware cost savings compared to previous designs of protection methods against FDI attacks., Comment: This work has been submitted to the IEEE for possible publication

We present results from the first 22 GHz space very-long-baseline interferometric (VLBI) imaging observations of M87 by RadioAstron. As a part of the Nearby AGN Key Science Program, the source was observed in Feb 2014 at 22 GHz with 21 ground stations, reaching projected $(u,v)$-spacings up to $\sim11\,$G$\lambda$. The imaging experiment was complemented by snapshot RadioAstron data of M87 obtained during 2013--2016 from the AGN Survey Key Science Program. Their longest baselines extend up to $\sim25\,$G$\lambda$. For all these measurements, fringes are detected only up to $\sim$2.8 Earth Diameter or $\sim$3 G$\lambda$ baseline lengths, resulting in a new image with angular resolution of $\sim150\,\mu$as or $\sim20$ Schwarzschild radii spatial resolution. The new image not only shows edge-brightened jet and counterjet structures down to submilliarcsecond scales but also clearly resolves the VLBI core region. While the overall size of the core is comparable to those reported in the literature, the ground-space fringe detection and slightly super-resolved RadioAstron image suggest the presence of substructures in the nucleus, whose minimum brightness temperature exceeds $T_{\rm B, min}\sim10^{12}\,$K. It is challenging to explain the origin of this record-high $T_{\rm B, min}$ value for M87 by pure Doppler boosting effect with a simple conical jet geometry and known jet speed. Therefore, this can be evidence for more extreme Doppler boosting due to a blazar-like small jet viewing angle or highly efficient particle acceleration processes occurring already at the base of the outflow., Comment: 27 pages, 13 figures, accepted for publication in ApJ

We address the problem of learning linear system models from observing multiple trajectories from different system dynamics. This framework encompasses a collaborative scenario where several systems seeking to estimate their dynamics are partitioned into clusters according to their system similarity. Thus, the systems within the same cluster can benefit from the observations made by the others. Considering this framework, we present an algorithm where each system alternately estimates its cluster identity and performs an estimation of its dynamics. This is then aggregated to update the model of each cluster. We show that under mild assumptions, our algorithm correctly estimates the cluster identities and achieves an approximate sample complexity that scales inversely with the number of systems in the cluster, thus facilitating a more efficient and personalized system identification process.

We initiate the study of federated reinforcement learning under environmental heterogeneity by considering a policy evaluation problem. Our setup involves $N$ agents interacting with environments that share the same state and action space but differ in their reward functions and state transition kernels. Assuming agents can communicate via a central server, we ask: Does exchanging information expedite the process of evaluating a common policy? To answer this question, we provide the first comprehensive finite-time analysis of a federated temporal difference (TD) learning algorithm with linear function approximation, while accounting for Markovian sampling, heterogeneity in the agents' environments, and multiple local updates to save communication. Our analysis crucially relies on several novel ingredients: (i) deriving perturbation bounds on TD fixed points as a function of the heterogeneity in the agents' underlying Markov decision processes (MDPs); (ii) introducing a virtual MDP to closely approximate the dynamics of the federated TD algorithm; and (iii) using the virtual MDP to make explicit connections to federated optimization. Putting these pieces together, we rigorously prove that in a low-heterogeneity regime, exchanging model estimates leads to linear convergence speedups in the number of agents.

We study the problem of learning a linear system model from the observations of $M$ clients. The catch: Each client is observing data from a different dynamical system. This work addresses the question of how multiple clients collaboratively learn dynamical models in the presence of heterogeneity. We pose this problem as a federated learning problem and characterize the tension between achievable performance and system heterogeneity. Furthermore, our federated sample complexity result provides a constant factor improvement over the single agent setting. Finally, we describe a meta federated learning algorithm, FedSysID, that leverages existing federated algorithms at the client level.

Market power exercise in the electricity markets distorts market prices and diminishes social welfare. Many markets have implemented market power mitigation processes to eliminate the impact of such behavior. The design of mitigation mechanisms has a direct influence on investors' profitability and thus mid-/long-term resource adequacy. In order to evaluate the effectiveness of the existing market power mitigation mechanisms, this paper proposes a mitigation-aware strategic bidding model and studies the bidding strategies of the market participants under current practice. The proposed bidding model has a bilevel structure with strategic participant's profit maximization problem in the upper level and the dispatch problem for market operators in the lower level. In particular, the consideration of potential offer mitigation is incorporated as upper-level constraints based on the conduct and impact tests. This bilevel problem is reduced to a single-level mixed-integer linear program using the KKT optimality conditions, duality theory, and linearization. Numerical results illustrate how a strategic player can exercise market power to achieve a higher profit even under the current market power mitigation process and we analyze the social impact that the market power exercise results.

From 2018 to 2020, the Scientific Balloon Roadmap Program Analysis Group (Balloon Roadmap PAG) served as an community-based, interdisciplinary forum for soliciting and coordinating community analysis and input in support of the NASA Scientific Balloon Program. The Balloon Roadmap PAG was tasked with articulating and prioritizing the key science drivers and needed capabilities of the Balloon Program for the next decade. Additionally, the Balloon Roadmap PAG was asked to evaluate the potential for achieving science goals and maturing technologies of the Science Mission Directorate, evaluate the Balloon Program goals towards community outreach, and asses commercial balloon launch opportunities. The culmination of this work has been a written report submitted to the NASA Astrophysics Division Director., Comment: 95 pages, 69 figures, prepared by the NASA Balloon Program Analysis Group for the NASA Astrophysics Division Director and the 2020 Astrophysics Decadal Survey

The urgent need for the decarbonization of power girds has accelerated the integration of renewable energy. Concurrently the increasing distributed energy resources (DER) and advanced metering infrastructures (AMI) have transformed the power grids into a more sophisticated cyber-physical system with numerous communication devices. While these transitions provide economic and environmental value, they also impose increased risk of cyber attacks and operational challenges. This paper investigates the vulnerability of the power grids with high renewable penetration against an intraday false data injection (FDI) attack on DER dispatch signals and proposes a kernel support vector regression (SVR) based detection model as a countermeasure. The intraday FDI attack scenario and the detection model are demonstrated in a numerical experiment using the HCE 187-bus test system.

Distributionally robust optimization (DRO) is a powerful tool for decision making under uncertainty. It is particularly appealing because of its ability to leverage existing data. However, many practical problems call for decision-making with some auxiliary information, and DRO in the context of conditional distribution is not straightforward. We propose a conditional kernel distributionally robust optimization (CKDRO) method that enables robust decision making under conditional distributions through kernel DRO and the conditional mean operator in the reproducing kernel Hilbert space (RKHS). In particular, we consider problems where there is a correlation between the unknown variable y and an auxiliary observable variable x. Given past data of the two variables and a queried auxiliary variable, CKDRO represents the conditional distribution P(y|x) as the conditional mean operator in the RKHS space and quantifies the ambiguity set in the RKHS as well, which depends on the size of the dataset as well as the query point. To justify the use of RKHS, we demonstrate that the ambiguity set defined in RKHS can be viewed as a ball under a metric that is similar to the Wasserstein metric. The DRO is then dualized and solved via a finite dimensional convex program. The proposed CKDRO approach is applied to a generation scheduling problem and shows that the result of CKDRO is superior to common benchmarks in terms of quality and robustness.

Federated learning is a framework for distributed optimization that places emphasis on communication efficiency. In particular, it follows a client-server broadcast model and is particularly appealing because of its ability to accommodate heterogeneity in client compute and storage resources, non-i.i.d. data assumptions, and data privacy. Our contribution is to offer a new federated learning algorithm, FedADMM, for solving non-convex composite optimization problems with non-smooth regularizers. We prove converges of FedADMM for the case when not all clients are able to participate in a given communication round under a very general sampling model.

Alon, Krivelevich, and Sudakov conjectured in 1999 that for every finite graph $F$, there exists a quantity $c(F)$ such that $\chi(G) \leqslant (c(F) + o(1)) \Delta / \log\Delta$ whenever $G$ is an $F$-free graph of maximum degree $\Delta$. The largest class of connected graphs $F$ for which this conjecture has been verified so far, by Alon, Krivelevich, and Sudakov themselves, comprises the almost bipartite graphs (i.e., subgraphs of the complete tripartite graph $K_{1,t,t}$ for some $t \in \mathbb{N}$). However, the optimal value for $c(F)$ remains unknown even for such graphs. Bollob\'{a}s showed, using random regular graphs, that $c(F) \geqslant 1/2$ when $F$ contains a cycle. On the other hand, Davies, Kang, Pirot, and Sereni recently established an upper bound of $c(K_{1,t,t}) \leqslant t$. We improve this to a constant, showing $c(F) \leqslant 4$ for every almost bipartite graph $F$. This surprisingly makes the bound independent of $F$ in all the known cases of the conjecture, leading us to make a new conjecture that $c(F)$ is always bounded above by some constant $c$ independent of $F$. We also establish a more general version of our bound in the setting of DP-coloring (also known as correspondence coloring) and consider some algorithmic consequences of our results., Comment: 35 pp

Engineered cyberphysical systems are growing increasingly large and complex. These systems require scalable controllers that robustly satisfy state and input constraints in the presence of additive noise -- such controllers should also be accompanied by theoretical guarantees on feasibility and stability. In our companion paper, we introduced Distributed and Localized Model Predictive Control (DLMPC) for large-scale linear systems; DLMPC is a scalable closed-loop MPC scheme in which subsystems need only exchange local information in order to synthesize and implement local controllers. In this paper, we provide recursive feasibility and asymptotic stability guarantees for DLMPC. We leverage the System Level Synthesis framework to express the maximal positive robust invariant set for the closed-loop system and its corresponding Lyapunov function, both in terms of the closed-loop system responses. We use the invariant set as the terminal set for DLMPC, and show that this guarantees feasibility with minimal conservatism. We use the Lyapunov function as the terminal cost, and show that this guarantees stability. We provide fully distributed and localized algorithms to compute the terminal set offline, and also provide necessary additions to the online DLMPC algorithm to accommodate coupled terminal constraint and cost. In all algorithms, only local information exchanges are necessary, and computational complexity is independent of the global system size -- we demonstrate this analytically and experimentally. This is the first distributed MPC approach that provides minimally conservative yet fully distributed guarantees for recursive feasibility and asymptotic stability, for both nominal and robust settings.

This work considers the problem of learning the Markov parameters of a linear system from observed data. Recent non-asymptotic system identification results have characterized the sample complexity of this problem in the single and multi-rollout setting. In both instances, the number of samples required in order to obtain acceptable estimates can produce optimization problems with an intractably large number of decision variables for a second-order algorithm. We show that a randomized and distributed Newton algorithm based on Hessian-sketching can produce $\epsilon$-optimal solutions and converges geometrically. Moreover, the algorithm is trivially parallelizable. Our results hold for a variety of sketching matrices and we illustrate the theory with numerical examples.

The increasing presence of large-scale distributed systems highlights the need for scalable control strategies where only local communication is required. Moreover, in safety-critical systems it is imperative that such control strategies handle constraints in the presence of disturbances. In response to this need, we present the Distributed and Localized Model Predictive Control (DLMPC) algorithm for large-scale linear systems. DLMPC is a distributed closed-loop model predictive control (MPC) scheme wherein only local state and model information needs to be exchanged between subsystems for the computation and implementation of control actions. We use the System Level Synthesis (SLS) framework to reformulate the centralized MPC problem, and show that this allows us to naturally impose localized communication constraints between sub-controllers. The structure of the resulting problem can be exploited to develop an Alternating Direction Method of Multipliers (ADMM) based algorithm that allows for distributed and localized computation of closed-loop control policies. We demonstrate that computational complexity of the subproblems solved by each subsystem in DLMPC is independent of the size of the global system. To the best of our knowledge, DLMPC is the first MPC algorithm that allows for the scalable distributed computation as well as implementation of distributed closed-loop control policies, and seemingly deals with additive disturbances. In our companion paper, we show that this approach enjoys recursive feasibility and asymptotic stability., Comment: arXiv admin note: text overlap with arXiv:2103.14171, arXiv:1909.10074

The VLBI Global Observing System, which is the next generation of geodetic VLBI and is called VGOS, observes simultaneously at four frequency bands in the range 3.0--10.7 GHz (expected to be extended to 14 GHz). Because source structure changes with frequency, we aim to study the source position estimates from the observations of this new VLBI system. Based on an ideal point source model, simulations are made to determine the relation between the source positions as determined by VGOS observations and the locations of the radio emission at the four bands. We obtained the source positions as determined by VGOS observations as a function of the source positions at the four frequency bands for both group and phase delays. The results reveal that if the location of the radio emission at one band is offset with respect to that at the other bands, the position estimate can be shifted to the opposite direction and even by more than three times that offset. The VGOS source positions will be very variable with time and very imprecise in the sense of relating to the locations of the radio emission at the four bands, if the effects of source structure are not modeled. The image alignment over frequency is essential in order to model the effects of source structure in VGOS observations, which is the only way to mitigate these strong frequency-dependent impacts on VGOS source positions., Comment: 10 pages, 4 figures, submitted to A&A

Learning a dynamical system from input/output data is a fundamental task in the control design pipeline. In the partially observed setting there are two components to identification: parameter estimation to learn the Markov parameters, and system realization to obtain a state space model. In both sub-problems it is implicitly assumed that standard numerical algorithms such as the singular value decomposition (SVD) can be easily and reliably computed. When trying to fit a high-dimensional model to data, for example in the cyber-physical system setting, even computing an SVD is intractable. In this work we show that an approximate matrix factorization obtained using randomized methods can replace the standard SVD in the realization algorithm while maintaining the non-asymptotic (in data-set size) performance and robustness guarantees of classical methods. Numerical examples illustrate that for large system models, this is the only method capable of producing a model.

A conjecture of Alon, Krivelevich, and Sudakov states that, for any graph $F$, there is a constant $c_F > 0$ such that if $G$ is an $F$-free graph of maximum degree $\Delta$, then $\chi(G) \leq c_F \Delta / \log\Delta$. Alon, Krivelevich, and Sudakov verified this conjecture for a class of graphs $F$ that includes all bipartite graphs. Moreover, it follows from recent work by Davies, Kang, Pirot, and Sereni that if $G$ is $K_{t,t}$-free, then $\chi(G) \leq (t + o(1)) \Delta / \log\Delta$ as $\Delta \to \infty$. We improve this bound to $(1+o(1)) \Delta/\log \Delta$, making the constant factor independent of $t$. We further extend our result to the DP-coloring setting (also known as correspondence coloring), introduced by Dvo\v{r}\'ak and Postle., Comment: 22 pp

We present a robust Distributed and Localized Model Predictive Control (rDLMPC) framework for large-scale structured linear systems. The proposed algorithm uses the System Level Synthesis to provide a distributed closed-loop model predictive control scheme that is robust to exogenous disturbances. The resulting controllers require only local information exchange for both synthesis and implementation. We exploit the fact that for polytopic disturbance constraints, SLS- based distributed control problems have been shown to have structure amenable for distributed optimization techniques. We show that similar to the disturbance-free DLMPC algorithm, the computational complexity of rDLMPC is independent of the size of the global system. To the best of our knowledge, robust DLMPC is the first MPC algorithm that allows for the scalable distributed computation of distributed closed-loop control policies in the presence of additive disturbances., Comment: Submission to 2021 Control and Decision Conference

The next-generation, broadband geodetic very long baseline interferometry system, named VGOS, is developing its global network, and VGOS networks with a small size of 3--7 stations have already made broadband observations from 2017 to 2019. We made quality assessments for two kinds of observables in the 21 VGOS sessions currently available: group delay and differential total electron content ($\delta$TEC). Our study reveals that the random measurement noise of VGOS group delays is at the level of less than 2 ps (1 ps = 10$^{-12}$ s), while the contributions from systematic error sources, mainly source structure related, are at the level of 20 ps. Due to the significant improvement in measurement noise, source structure effects with relatively small magnitudes that are not overwhelming in the S/X VLBI system, for instance 10 ps, are clearly visible in VGOS observations. Another critical error source in VGOS observations is discrete delay jumps, for instance, a systematic offset of about 310 ps or integer multiples of that. The predominant causative factor is found to be related to source structure. The measurement noise level of $\delta$TEC observables is about 0.07 TECU, but the systematic effects are five times larger than that. A strong correlation between group delay and $\delta$TEC observables is discovered with a trend of 40 ps/TECU for observations with large structure effects; there is a second trend in the range 60 ps/TECU to 70 ps/TECU when the measurement noise is dominant., Comment: 25 pages, 11 figures, and accepted for publication in Journal of Geodesy

We report the relationship between the $Gaia$--VLBI position differences and the magnitudes of source structure effects in VLBI observations. Because the $Gaia$--VLBI position differences are statistically significant for a considerable number of common sources, we attempt to discuss and explain these position differences based on VLBI observations and available source images at cm-wavelengths. Based on the derived closure amplitude root-mean-square (CARMS), which quantifies the magnitudes of source structure effects in the VLBI observations used for building the third realization of the International Celestial Reference Frame, the arc lengths and normalized arc lengths of the position differences are examined in detail. The radio jet directions and the directions of the $Gaia$--VLBI position differences are investigated for a small sample of sources. Both the arc lengths and normalized arc lengths of the $Gaia$ and VLBI positions are found to increase with the CARMS values. The majority of the sources with statistically significant position differences are associated with the sources having extended structure. Radio source structure is the one of the major factors of these position differences, and it can be the dominate factor for a number of sources. The vectors of the $Gaia$ and VLBI position differences are parallel to the radio-jet directions, which is confirmed with stronger evidence., Comment: 13 pages, 7 figures, and 5 tables. Accepted to be published in Astronomy & Astrophysics

We consider the problem of how to deploy a controller to a (networked) cyber-physical system (CPS). Controlling a CPS is an involved task, and synthesizing a controller to respect sensing, actuation, and communication constraints is only part of the challenge. In addition to controller synthesis, one should also consider how the controller will be incorporated within the CPS. Put another way, the cyber layer and its interaction with the physical layer need to be taken into account. In this work, we aim to bridge the gap between theoretical controller synthesis and practical CPS deployment. We adopt the system level synthesis (SLS) framework to synthesize a controller, either state-feedback or output-feedback, and provide deployment architectures for the standard SLS controllers. Furthermore, we derive new controller realizations for open-loop stable systems and introduce different state-feedback and output-feedback architectures for deployment, ranging from fully centralized to fully distributed. Finally, we compare the trade-offs among them in terms of robustness, memory, computation, and communication overhead., Comment: arXiv admin note: substantial text overlap with arXiv:1911.01510

Distributed linear control design is crucial for large-scale cyber-physical systems. It is generally desirable to both impose information exchange (communication) constraints on the distributed controller, and to limit the propagation of disturbances to a local region without cascading to the global network (localization). Recently proposed System Level Synthesis (SLS) theory provides a framework where such communication and localization requirements can be tractably incorporated in controller design and implementation. In this work, we derive a solution to the localized and distributed H2 state feedback control problem without resorting to Finite Impulse Response (FIR) approximation. Our proposed synthesis algorithm allows a column-wise decomposition of the resulting convex program, and is therefore scalable to arbitrary large-scale networks. We demonstrate superior cost performance and computation time of the proposed procedure over previous methods via numerical simulation., Comment: 8 pages, To appear in ACC 2021

An explicit Model Predictive Control algorithm for large-scale structured linear systems is presented. We base our results on Distributed and Localized Model Predictive Control (DLMPC), a closed-loop model predictive control scheme based on the System Level Synthesis (SLS) framework wherein only local state and model information needs to be exchanged between subsystems for the computation and implementation of control actions. We provide an explicit solution for each of the subproblems resulting from the distributed MPC scheme. We show that given the separability of the problem, the explicit solution is only divided into three regions per state and input instantiation, making the point location problem very efficient. Moreover, given the locality constraints, the subproblems are of much smaller dimension than the full problem, which significantly reduces the computational overhead of explicit solutions. We conclude with numerical simulations to demonstrate the computational advantages of our method, in which we show a large improvement in runtime per MPC iteration as compared with the results of computing the optimization with a solver online.

In this paper we consider the problem of analyzing the effect a change in the load vector can have on the optimal power generation in a DC power flow model. The methodology is based upon the recently introduced concept of the $\mathcal{OPF}$ operator. It is shown that for general network topologies computing the worst-case sensitivities is computationally intractable. However, we show that certain problems involving the $\mathcal{OPF}$ operator can be equivalently converted to a graphical discrete optimization problem. Using the discrete formulation, we provide a decomposition algorithm that reduces the computational cost of computing the worst-case sensitivity. A 27-bus numerical example is used to illustrate our results.

We present RadioAstron Space VLBI imaging observations of the BL Lac object S5 0716+71 made on January 3-4 2015 at a frequency of 22 GHz (wavelength $\lambda=1.3$ cm). The observations were made in the framework of the AGN Polarization Key Science Program. The source was detected on projected space-ground baselines up to 70 833 km (5.6 Earth diameters) for both, parallel hand and cross-hand interferometric visibilities. We have used these detections to obtain a full-polarimetric image of the blazar at an unprecedented angular resolution of 24 $\mu$as, the highest for this source to date. This enabled us to estimate the size of the radio core to be $<12\times5~\mu$as and to reveal a complex structure and a significant curvature of the blazar jet in the inner 100 $\mu$as, which is an indication that the jet viewing angle lies inside the opening angle of the jet conical outflow. Fairly highly (15%) linearly polarized emission is detected in a jet region of 19 $\mu$as in size, located 58 $\mu$as downstream from the core. The highest brightness temperature in the source frame is estimated to be $>2.2\times10^{13}$ K for the blazar core. This implies that the inverse Compton limit must be violated in the rest frame of the source, even for the largest Doppler factor $\delta\thicksim25$ reported for 0716+714., Comment: 16 pages, 6 figures, 3 tables; accepted to ApJ

This paper presents the results from one of the first observations of ionospheric scintillation taken using the Low-Frequency Array (LOFAR). The observation was of the strong natural radio source Cas A, taken overnight on 18-19 August 2013, and exhibited moderately strong scattering effects in dynamic spectra of intensity received across an observing bandwidth of 10-80MHz. Delay-Doppler spectra (the 2-D FFT of the dynamic spectrum) from the first hour of observation showed two discrete parabolic arcs, one with a steep curvature and the other shallow, which can be used to provide estimates of the distance to, and velocity of, the scattering plasma. A cross-correlation analysis of data received by the dense array of stations in the LOFAR "core" reveals two different velocities in the scintillation pattern: a primary velocity of ~30m/s with a north-west to south-east direction, associated with the steep parabolic arc and a scattering altitude in the F-region or higher, and a secondary velocity of ~110m/s with a north-east to south-west direction, associated with the shallow arc and a scattering altitude in the D-region. Geomagnetic activity was low in the mid-latitudes at the time, but a weak sub-storm at high latitudes reached its peak at the start of the observation. An analysis of Global Navigation Satellite Systems (GNSS) and ionosonde data from the time reveals a larger-scale travelling ionospheric disturbance (TID), possibly the result of the high-latitude activity, travelling in the north-west to south-east direction, and, simultaneously, a smaller--scale TID travelling in a north-east to south-west direction, which could be associated with atmospheric gravity wave activity. The LOFAR observation shows scattering from both TIDs, at different altitudes and propagating in different directions. To the best of our knowledge this is the first time that such a phenomenon has been reported., Comment: 24 pages, 16 figures. Accepted for open-access publication in the Journal of Space Weather and Space Climate. For associated movie file, see https://www.swsc-journal.org/10.1051/swsc/2020010/olm

This paper aims at the safety-critical control synthesis of network systems such that the satisfaction of the safety constraints can be guaranteed. To handle the large state dimension of such systems, an assume-guarantee contract is used to break the large synthesis problem into smaller subproblems. Parameterized signal temporal logic (pSTL) is used to formally describe the behaviors of the subsystems, which we use as the template for the contract. We show that robust control invariant sets (RCIs) for the subsystems can be composed to form a robust control invariant set for the whole network system under a valid assume-guarantee contract. An epigraph algorithm is proposed to solve for a contract that is valid, ---an approach that has linear complexity for sparse networks, which leads to a robust control invariant set for the whole network system. Implemented with control barrier function (CBF), the state of each subsystem is guaranteed to stay within the safe set. Furthermore, we propose a contingency tube Model Predictive Control approach based on the RCI, which is capable of handling severe contingencies, including topology changes of the network. A power grid example is used to demonstrate the proposed method. The simulation result includes both set point control and contingency recovery, and the safety constraint is always satisfied., Comment: arXiv admin note: text overlap with arXiv:1810.10636

We consider the problem of how to deploy a controller to a (networked) cyber-physical system (CPS). Controlling a CPS is an involved task, and synthesizing a controller to respect sensing, actuation, and communication constraints is only part of the challenge. In addition to controller synthesis, one should also consider how the controller will work in the CPS. Put another way, the cyber layer and its interaction with the physical layer need to be taken into account. In this work, we aim to bridge the gap between theoretical controller synthesis and practical CPS deployment. We adopt the system level synthesis (SLS) framework to synthesize a state-feedback controller and provide a deployment architecture for the standard SLS controller. Furthermore, we derive a new controller realization for open-loop stable systems and introduce four different architectures for deployment, ranging from fully centralized to fully distributed. Finally, we compare the trade-offs among them in terms of robustness, memory, computation, and communication overhead., Comment: in Proc. IEEE ACC, 2020

Optimal power flow problems (OPFs) are mathematical programs used to determine how to distribute power over networks subject to network operation constraints and the physics of power flows. In this work, we take the view of treating an OPF problem as an operator which maps user demand to generated power, and allow the network parameters (such as generator and power flow limits) to take values in some admissible set. The contributions of this paper are to formalize this operator theoretic approach, define and characterize restricted parameter sets under which the mapping has a singleton output, independent binding constraints, and is differentiable. In contrast to related results in the optimization literature, we do not rely on introducing auxiliary slack variables. Indeed, our approach provides results that have a clear interpretation with respect to the power network under study. We further provide a closed-form expression for the Jacobian matrix of the OPF operator and describe how various derivatives can be computed using a recently proposed scheme based on homogenous self-dual embedding. Our framework of treating a mathematical program as an operator allows us to pose sensitivity and robustness questions from a completely different mathematical perspective and provide new insights into well studied problems., Comment: 16 pages, 7 figures

Crop production needs to increase in a sustainable manner to meet the growing global demand for food. To identify crop varieties with high yield potential, plant scientists and breeders evaluate the performance of hundreds of lines in multiple locations over several years. To facilitate the process of selecting advanced varieties, an automated framework was developed in this study. A hyperspectral camera was mounted on an unmanned aerial vehicle to collect aerial imagery with high spatial and spectral resolution. Aerial images were captured in two consecutive growing seasons from three experimental yield fields composed of hundreds experimental plots (1x2.4 meter), each contained a single wheat line. The grain of more than thousand wheat plots was harvested by a combine, weighed, and recorded as the ground truth data. To leverage the high spatial resolution and investigate the yield variation within the plots, images of plots were divided into sub-plots by integrating image processing techniques and spectral mixture analysis with the expert domain knowledge. Afterwards, the sub-plot dataset was divided into train, validation, and test sets using stratified sampling. Subsequent to extracting features from each sub-plot, deep neural networks were trained for yield estimation. The coefficient of determination for predicting the yield of the test dataset at sub-plot scale was 0.79 with root mean square error of 5.90 grams. In addition to providing insights into yield variation at sub-plot scale, the proposed framework can facilitate the process of high-throughput yield phenotyping as a valuable decision support tool. It offers the possibility of (i) remote visual inspection of the plots, (ii) studying the effect of crop density on yield, and (iii) optimizing plot size to investigate more lines in a dedicated field each year.

This article surveys the System Level Synthesis framework, which presents a novel perspective on constrained robust and optimal controller synthesis for linear systems. We show how SLS shifts the controller synthesis task from the design of a controller to the design of the entire closed loop system, and highlight the benefits of this approach in terms of scalability and transparency. We emphasize two particular applications of SLS, namely large-scale distributed optimal control and robust control. In the case of distributed control, we show how SLS allows for localized controllers to be computed, extending robust and optimal control methods to large-scale systems under practical and realistic assumptions. In the case of robust control, we show how SLS allows for novel design methodologies that, for the first time, quantify the degradation in performance of a robust controller due to model uncertainty -- such transparency is key in allowing robust control methods to interact, in a principled way, with modern techniques from machine learning and statistical inference. Throughout, we emphasize practical and efficient computational solutions, and demonstrate our methods on easy to understand case studies., Comment: To appear in Annual Reviews in Control