Search

We present a reinforcement learning (RL) framework in which the learned policy comes with a machine-checkable certificate of provable adversarial robustness. Our approach, called CAROL, learns a model of the environment. In each learning iteration, it uses the current version of this model and an external abstract interpreter to construct a differentiable signal for provable robustness. This signal is used to guide learning, and the abstract interpretation used to construct it directly leads to the robustness certificate returned at convergence. We give a theoretical analysis that bounds the worst-case accumulative reward of CAROL. We also experimentally evaluate CAROL on four MuJoCo environments with continuous state and action spaces. On these tasks, CAROL learns policies that, when contrasted with policies from the state-of-the-art robust RL algorithms, exhibit: (i) markedly enhanced certified performance lower bounds; and (ii) comparable performance under empirical adversarial attacks.

In reinforcement learning for safety-critical settings, it is often desirable for the agent to obey safety constraints at all points in time, including during training. We present a novel neurosymbolic approach called SPICE to solve this safe exploration problem. SPICE uses an online shielding layer based on symbolic weakest preconditions to achieve a more precise safety analysis than existing tools without unduly impacting the training process. We evaluate the approach on a suite of continuous control benchmarks and show that it can achieve comparable performance to existing safe learning techniques while incurring fewer safety violations. Additionally, we present theoretical results showing that SPICE converges to the optimal safe policy under reasonable assumptions.

We present Revel, a partially neural reinforcement learning (RL) framework for provably safe exploration in continuous state and action spaces. A key challenge for provably safe deep RL is that repeatedly verifying neural networks within a learning loop is computationally infeasible. We address this challenge using two policy classes: a general, neurosymbolic class with approximate gradients and a more restricted class of symbolic policies that allows efficient verification. Our learning algorithm is a mirror descent over policies: in each iteration, it safely lifts a symbolic policy into the neurosymbolic space, performs safe gradient updates to the resulting policy, and projects the updated policy into the safe symbolic subset, all without requiring explicit verification of neural networks. Our empirical results show that Revel enforces safe exploration in many scenarios in which Constrained Policy Optimization does not, and that it can discover policies that outperform those learned through prior approaches to verified exploration.

In recent years, the notion of local robustness (or robustness for short) has emerged as a desirable property of deep neural networks. Intuitively, robustness means that small perturbations to an input do not cause the network to perform misclassifications. In this paper, we present a novel algorithm for verifying robustness properties of neural networks. Our method synergistically combines gradient-based optimization methods for counterexample search with abstraction-based proof search to obtain a sound and ({\delta}-)complete decision procedure. Our method also employs a data-driven approach to learn a verification policy that guides abstract interpretation during proof search. We have implemented the proposed approach in a tool called Charon and experimentally evaluated it on hundreds of benchmarks. Our experiments show that the proposed approach significantly outperforms three state-of-the-art tools, namely AI^2 , Reluplex, and Reluval.

Many example-guided program synthesis techniques use abstractions to prune the search space. While abstraction-based synthesis has proven to be very powerful, a domain expert needs to provide a suitable abstract domain, together with the abstract transformers of each DSL construct. However, coming up with useful abstractions can be non-trivial, as it requires both domain expertise and knowledge about the synthesizer. In this paper, we propose a new technique for learning abstractions that are useful for instantiating a general synthesis framework in a new domain. Given a DSL and a small set of training problems, our method uses tree interpolation to infer reusable predicate templates that speed up synthesis in a given domain. Our method also learns suitable abstract transformers by solving a certain kind of second-order constraint solving problem in a data-driven way. We have implemented the proposed method in a tool called ATLAS and evaluate it in the context of the BLAZE meta-synthesizer. Our evaluation shows that (a) ATLAS can learn useful abstract domains and transformers from few training problems, and (b) the abstractions learned by ATLAS allow BLAZE to achieve significantly better results compared to manually-crafted abstractions.

We show that any matrix-polynomial combination of free noncommutative random variables each having an algebraic law has again an algebraic law. Our result answers a question raised by a recent paper of Shlyakhtenko and Skoufranis. The result belongs to a family of results with origins outside free probability theory, including a result of Aomoto asserting algebraicity of the Green function of random walk of quite general type on a free group., Comment: 41 pages, LaTeX, no figures. In v2, we added references, corrected typos, reorganized some material, and added explanations. Main results remain the same. In this version, v3, we added references and explanation, and simplified the second half of the proof of the main result

It is well-known that the number of planar maps with prescribed vertex degree distribution and suitable labeling can be represented as the leading coefficient of the $\frac{1}{N}$-expansion of a joint cumulant of traces of powers of an $N$-by-$N$ GUE matrix. Here we undertake the calculation of this leading coefficient in a different way. Firstly, we tridiagonalize the GUE matrix in the manner of Trotter and Dumitriu-Edelman and then alter it by conjugation to make the subdiagonal identically equal to $1$. Secondly, we apply the cluster expansion technique (specifically, the Brydges-Kennedy-Abdesselam-Rivasseau formula) from rigorous statistical mechanics. Thirdly, by sorting through the terms of the expansion thus generated we arrive at an alternate interpretation for the leading coefficient related to factorizations of the long cycle $(12\cdots n)\in S_n$. Finally, we reconcile the group-theoretical objects emerging from our calculation with the labeled mobiles of Bouttier-Di Francesco-Guitter., Comment: 42 pages, LaTeX, 17 figures. The present paper completely supercedes arXiv1203.3185 in terms of methods but addresses a different problem

Our main result is a local limit law for the empirical spectral distribution of the anticommutator of independent Wigner matrices, modeled on the local semicircle law. Our approach is to adapt some techniques from one of the recent papers of Erd\"os-Yau-Yin. We also use an algebraic description of the law of the anticommutator of free semicircular variables due to Nica-Speicher, a self-adjointness-preserving variant of the linearization trick due to Haagerup-Schultz-Thorbj\o rnsen, and the Schwinger-Dyson equation. A byproduct of our work is a relatively simple deterministic version of the local semicircle law., Comment: 33 pages, LaTeX, 2 figures. In v2 (this version) we make minor revisions, add references and correct typos

A fundamental result of free probability theory due to Voiculescu and subsequently refined by many authors states that conjugation by independent Haar-distributed random unitary matrices delivers asymptotic freeness. In this paper we exhibit many other systems of random unitary matrices that, when used for conjugation, lead to freeness. We do so by first proving a general result asserting "asymptotic liberation" under quite mild conditions, and then we explain how to specialize these general results in a striking way by exploiting Hadamard matrices. In particular, we recover and generalize results of the second-named author and of Tulino-Caire-Shamai-Verd\'{u}., Comment: 26 pages, no figures, LaTeX. In v2, Prop. 2.6 and its proof were corrected, and the proof of Cor. 3.4 was correspondingly modified. In v3, typos were corrected, a corollary was added, extra details and explanations were added, and the proof of Cor. 3.4 (now Cor. 3.7) was rewritten. In v4 (this version) light editing and new reference added

Our main result is an explicit operator-theoretic formula for the number of colored planar maps with a fixed set of stars each of which has a fixed set of half-edges with fixed coloration. The formula bounds the number of such colored planar maps well enough to prove convergence near the origin of generating functions arising naturally in the matrix model context. Such convergence is known but the proof of convergence proceeding by way of our main result is relatively simple. Besides Voiculescu's generalization of Wigner's semicircle law, our main technical tool is an integration identity representing the joint cumulant of several functions of a Gaussian random vector. The latter identity in the case of cumulants of order 2 reduces to one well-known as a means to prove the Poincare inequality. We derive the identity by combining the heat equation with the so-called BKAR formula from constructive quantum field theory and rigorous statistical mechanics., Comment: 19 pages, no figures, LaTeX. This new shorter version uses the BKAR formula to derive the main technical result; the previously given proof has been suppressed. New references to the mathematical physics literature have been added. A co-author has been added. But the main result and its proof, apart from some changes in notation, remain the same

For polynomials in independent Wigner matrices, we prove convergence of the largest singular value to the operator norm of the corresponding polynomial in free semicircular variables, under fourth moment hypotheses. We actually prove a more general result of the form "no eigenvalues outside the support of the limiting eigenvalue distribution." We build on ideas of Haagerup-Schultz-Thorbj{\o}rnsen on the one hand and Bai-Silverstein on the other. We refine the linearization trick so as to preserve self-adjointness and we develop a secondary trick bearing on the calculation of correction terms. Instead of Poincar\'{e}-type inequalities, we use a variety of matrix identities and $L^p$ estimates. The Schwinger-Dyson equation controls much of the analysis., Comment: Published in at http://dx.doi.org/10.1214/11-AOP739 the Annals of Probability (http://www.imstat.org/aop/) by the Institute of Mathematical Statistics (http://www.imstat.org)

We provide a period interpretation for multizeta values (in the function field context) in terms of explicit iterated extensions of tensor powers of Carlitz motives (mixed Carlitz-Tate t-motives). We give examples of combinatorially involved relations that these multizeta values satisfy., Comment: 12 pages, LaTeX

We consider the spectral properties of a class of regularized estimators of (large) empirical covariance matrices corresponding to stationary (but not necessarily Gaussian) sequences, obtained by banding. We prove a law of large numbers (similar to that proved in the Gaussian case by Bickel and Levina), which implies that the spectrum of a banded empirical covariance matrix is an efficient estimator. Our main result is a central limit theorem in the same regime, which to our knowledge is new, even in the Gaussian setup., Comment: Published in at http://dx.doi.org/10.1214/07-AOS503 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)