Search

We present a method for using previously-trained 'teacher' agents to kickstart the training of a new 'student' agent. To this end, we leverage ideas from policy distillation and population based training. Our method places no constraints on the architecture of the teacher or student agents, and it regulates itself to allow the students to surpass their teachers in performance. We show that, on a challenging and computationally-intensive multi-task benchmark (DMLab-30), kickstarted training improves the data efficiency of new agents, making it significantly easier to iterate on their design. We also show that the same kickstarting pipeline can allow a single student agent to leverage multiple 'expert' teachers which specialize on individual tasks. In this setting kickstarting yields surprisingly large gains, with the kickstarted agent matching the performance of an agent trained from scratch in almost 10x fewer steps, and surpassing its final performance by 42 percent. Kickstarting is conceptually simple and can easily be incorporated into reinforcement learning experiments.

We compute the next-to-next-to-leading order (NNLO) QCD corrections to the decay $t(\uparrow) \to X_b +W^+$ of a polarized top quark. The spin-momentum correlation in this quasi two-body decay is described by the polar angle distribution $\mathrm{d}\Gamma/\mathrm{d}\cos\theta_P=\frac{\Gamma}{2}(1+P_t\, \alpha_P\, \cos\theta_P)$ where $P_t$ is the polarization of the top quark and $\alpha_P$ denotes the asymmetry parameter of the decay. For the latter we find $\alpha^{\mathrm{NNLO}}_P=0.3792\pm 0.0037$., Comment: 17 pages, 3 figures; v2: added Fig. 2 and 3 and corresponding discussion

We classify all integrable complex structures on 6-dimensional Lie algebras of the form $\mathfrak{g}\times\mathfrak{g}$.

Experimental studies of neutron decay, $n\to pe\bar\nu$, exhibit two anomalies. The first is a 8.6(2.1)s, roughly $4\sigma$ difference between the average beam measured neutron lifetime, $\tau_n^\text{beam}=888.0(2.0)$s, and the more precise average trapped ultra cold neutron determination, $\tau_n^\text{trap}=879.4(6)$s. The second is a $5\sigma$ difference between the pre2002 average axial coupling, $g_A$, as measured in neutron decay asymmetries $g_A^\text{pre2002}=1.2637(21)$, and the more recent, post2002, average $g_A^\text{post2002}=1.2755(11)$, where, following the UCNA collaboration division, experiments are classified by the date of their most recent result. In this study, we correlate those $\tau_n$ and $g_A$ values using a (slightly) updated relation $\tau_n(1+3g_A^2)=5172.0(1.1)$s. Consistency with that relation and better precision suggest $\tau_n^\text{favored}=879.4(6)$s and $g_A^\text{favored}=1.2755(11)$ as preferred values for those parameters. Comparisons of $g_A^\text{favored}$ with recent lattice QCD and muonic hydrogen capture results are made. A general constraint on exotic neutron decay branching ratios, $<0.27\%$, is discussed and applied to a recently proposed solution to the neutron lifetime puzzle.

Given a geodesic line $\gamma$ the hyperbolic space $\mathbb H^n$ we formulate a necessary and sufficient condition for a function along this geodesic which measure the mean curvature of totally umbilical leaves of a foliation orthogonal to $\gamma$. Then we extend the result to $\gamma$ being a hypercycle i.e. a geodesic on a hypersurface equidistant from the totally geodesic one., Comment: 14 pages, 4 figures

Neural networks dominate the modern machine learning landscape, but their training and success still suffer from sensitivity to empirical choices of hyperparameters such as model architecture, loss function, and optimisation algorithm. In this work we present \emph{Population Based Training (PBT)}, a simple asynchronous optimisation algorithm which effectively utilises a fixed computational budget to jointly optimise a population of models and their hyperparameters to maximise performance. Importantly, PBT discovers a schedule of hyperparameter settings rather than following the generally sub-optimal strategy of trying to find a single fixed set to use for the whole course of training. With just a small modification to a typical distributed hyperparameter training framework, our method allows robust and reliable training of models. We demonstrate the effectiveness of PBT on deep reinforcement learning problems, showing faster wall-clock convergence and higher final performance of agents by optimising over a suite of hyperparameters. In addition, we show the same method can be applied to supervised learning for machine translation, where PBT is used to maximise the BLEU score directly, and also to training of Generative Adversarial Networks to maximise the Inception score of generated images. In all cases PBT results in the automatic discovery of hyperparameter schedules and model selection which results in stable training and better final performance.

The current status of electroweak radiative corrections to the muon anomalous magnetic moment is discussed. Asymptotic expansions for some important electroweak two loop top quark triangle diagrams are illustrated and extended to higher order. Results are compared with the more general integral representation solution for generic fermion triangle loops coupled to pseudoscalar and scalar bosons of arbitrary mass. Excellent agreement is found for a broader than expected range of mass parameters.

We compute corrections to the gyromagnetic factor of an electron bound in a hydrogen-like ion at order alpha^2(Z alpha)^5. This result removes a major uncertainty in predictions for silicon and carbon ions, used to determine the atomic mass of the electron.

Machine learning (ML) plays an ever-increasing role in advanced automotive functionality for driver assistance and autonomous operation; however, its adequacy from the perspective of safety certification remains controversial. In this paper, we analyze the impacts that the use of ML as an implementation approach has on ISO 26262 safety lifecycle and ask what could be done to address them. We then provide a set of recommendations on how to adapt the standard to accommodate ML., Comment: 6 pages, 3 figures

We consider the interactions in a mesonic system, referred here to as `tetron', consisting of two heavy quarks and two lighter antiquarks (which may still be heavy in the scale of QCD), i.e. generally $Q_a Q_b \bar q_c \bar q_d$, and study the existence of bound states below the threshold for decay into heavy meson pairs. At a small ratio of the lighter to heavier quark masses an expansion parameter arises for treatment of the binding in such systems. We find that in the limit where all the quarks and antiquarks are so heavy that a Coulomb-like approximation can be applied to the gluon exchange between all of them, such bound states arise when this parameter is below a certain critical value. We find the parametric dependence of the critical mass ratio on the number of colors $N_c$, and confirm this dependence by numerical calculations. In particular there are no stable tetrons when all constituents have the same mass. We discuss an application of a similar expansion in the large $N_c$ limit to realistic systems where the antiquarks are light and their interactions are nonperturbative. In this case our findings are in agreement with the recent claims from a phenomenological analysis that a stable $b b \bar u \bar d$ tetron is likely to exist, unlike those where one or both bottom quarks are replaced by the charmed quark., Comment: 10 pages, 2 figures

We calculate the decay rate and spectrum for the decay of parapositronium into three photons. This process violates charge conjugation symmetry. It is forbidden within quantum electrodynamics but proceeds through the weak interaction. The branching ratio we find is fifty orders of magnitude smaller than the previous estimate., Comment: Accpted to PRD

Most deep reinforcement learning algorithms are data inefficient in complex and rich environments, limiting their applicability to many scenarios. One direction for improving data efficiency is multitask learning with shared neural network parameters, where efficiency may be improved through transfer across related tasks. In practice, however, this is not usually observed, because gradients from different tasks can interfere negatively, making learning unstable and sometimes even less data efficient. Another issue is the different reward schemes between tasks, which can easily lead to one task dominating the learning of a shared model. We propose a new approach for joint training of multiple tasks, which we refer to as Distral (Distill & transfer learning). Instead of sharing parameters between the different workers, we propose to share a "distilled" policy that captures common behaviour across tasks. Each worker is trained to solve its own task while constrained to stay close to the shared policy, while the shared policy is trained by distillation to be the centroid of all task policies. Both aspects of the learning process are derived by optimizing a joint objective function. We show that our approach supports efficient transfer on complex 3D environments, outperforming several related methods. Moreover, the proposed learning process is more robust and more stable---attributes that are critical in deep reinforcement learning.

Over the years complexity theorists have proposed many structural parameters to explain the surprising efficiency of conflict-driven clause-learning (CDCL) SAT solvers on a wide variety of large industrial Boolean instances. While some of these parameters have been studied empirically, until now there has not been a unified comparative study of their explanatory power on a comprehensive benchmark. We correct this state of affairs by conducting a large-scale empirical evaluation of CDCL SAT solver performance on nearly 7000 industrial and crafted formulas against several structural parameters such as backdoors, treewidth, backbones, and community structure. Our study led us to several results. First, we show that while such parameters only weakly correlate with CDCL solving time, certain combinations of them yield much better regression models. Second, we show how some parameters can be used as a "lens" to better understand the efficiency of different solving heuristics. Finally, we propose a new complexity-theoretic parameter, which we call learning-sensitive with restarts (LSR) backdoors, that extends the notion of learning-sensitive (LS) backdoors to incorporate restarts and discuss algorithms to compute them. We mathematically prove that for certain class of instances minimal LSR-backdoors are exponentially smaller than minimal-LS backdoors.

The paper consider regulous functions on the real affine space $\mathbb{R}^N$. We shall study some algebraic properties of the ring of those functions. It is presented a proof of the regulous version of Nullstellensatz based on the substitution property and the Artin-Lang property for the considered function ring. We prove that every maximal ideal in the ring of regulous functions on $\mathbb{R}^N$ when $N\geq 2$ is not finitely generated. Finally, we extend the latter result to an arbitrary, smooth, real affine algebraic variety of dimension $d\geq 2$., Comment: Some improvements made

We are increasingly surrounded by artificially intelligent technology that takes decisions and executes actions on our behalf. This creates a pressing need for general means to communicate with, instruct and guide artificial agents, with human language the most compelling means for such communication. To achieve this in a scalable fashion, agents must be able to relate language to the world and to actions; that is, their understanding of language must be grounded and embodied. However, learning grounded language is a notoriously challenging problem in artificial intelligence research. Here we present an agent that learns to interpret language in a simulated 3D environment where it is rewarded for the successful execution of written instructions. Trained via a combination of reinforcement and unsupervised learning, and beginning with minimal prior knowledge, the agent learns to relate linguistic symbols to emergent perceptual representations of its physical surroundings and to pertinent sequences of actions. The agent's comprehension of language extends beyond its prior experience, enabling it to apply familiar language to unfamiliar situations and to interpret entirely novel instructions. Moreover, the speed with which this agent learns new words increases as its semantic knowledge grows. This facility for generalising and bootstrapping semantic knowledge indicates the potential of the present approach for reconciling ambiguous natural language with the complexity of the physical world., Comment: 16 pages, 8 figures

We study the problem of cooperative multi-agent reinforcement learning with a single joint reward signal. This class of learning problems is difficult because of the often large combined action and observation spaces. In the fully centralized and decentralized approaches, we find the problem of spurious rewards and a phenomenon we call the "lazy agent" problem, which arises due to partial observability. We address these problems by training individual agents with a novel value decomposition network architecture, which learns to decompose the team value function into agent-wise value functions. We perform an experimental evaluation across a range of partially-observable multi-agent domains and show that learning such value-decompositions leads to superior results, in particular when combined with weight sharing, role information and information channels.

At the heart of deep learning we aim to use neural networks as function approximators - training them to produce outputs from inputs in emulation of a ground truth function or data creation process. In many cases we only have access to input-output pairs from the ground truth, however it is becoming more common to have access to derivatives of the target output with respect to the input - for example when the ground truth function is itself a neural network such as in network compression or distillation. Generally these target derivatives are not computed, or are ignored. This paper introduces Sobolev Training for neural networks, which is a method for incorporating these target derivatives in addition the to target values while training. By optimising neural networks to not only approximate the function's outputs but also the function's derivatives we encode additional information about the target function within the parameters of the neural network. Thereby we can improve the quality of our predictors, as well as the data-efficiency and generalization capabilities of our learned function approximation. We provide theoretical justifications for such an approach as well as examples of empirical evidence on three distinct domains: regression on classical optimisation datasets, distilling policies of an agent playing Atari, and on large-scale applications of synthetic gradients. In all three domains the use of Sobolev Training, employing target derivatives in addition to target values, results in models with higher accuracy and stronger generalisation.

We compute the decay rate of dimuonium into a neutral pion and a photon. We find that approximately one in 10^5 ortho-dimuonia decays into this channel. We also determine the contribution of the virtual photon-pion loop to the hyperfine splitting in dimuonium and reproduce its leading effect in the anomalous magnetic moment of the muon., Comment: corrected and extended list of references

We present examples of foliations with infinite dimensional basic symplectic and com- plex cohomologies, along with a general sufficient condition for such phenomena. This puts re- strictions on possible generalizations of several finiteness results from Riemannian foliations to any broader class. The examples are also noteworthy for the unusual behaviour of their basic de Rham cohomology.

We present a construction (and classification) of certain invariant 2-forms on the real symplectic group. They are used to define a symplectic form on the quotient by a maximal torus and to "lift" a symplectic structure from a symplectic manifold to the bundle of frames. This is a by-product of a failed attempt to prove certain finiteness theorems for basic symplectic cohomologies. In the last part of the paper we include a valid proof.

When training neural networks, the use of Synthetic Gradients (SG) allows layers or modules to be trained without update locking - without waiting for a true error gradient to be backpropagated - resulting in Decoupled Neural Interfaces (DNIs). This unlocked ability of being able to update parts of a neural network asynchronously and with only local information was demonstrated to work empirically in Jaderberg et al (2016). However, there has been very little demonstration of what changes DNIs and SGs impose from a functional, representational, and learning dynamics point of view. In this paper, we study DNIs through the use of synthetic gradients on feed-forward networks to better understand their behaviour and elucidate their effect on optimisation. We show that the incorporation of SGs does not affect the representational strength of the learning system for a neural network, and prove the convergence of the learning system for linear and deep linear models. On practical problems we investigate the mechanism by which synthetic gradient estimators approximate the true loss, and, surprisingly, how that leads to drastically different layer-wise representations. Finally, we also expose the relationship of using synthetic gradients to other error approximation techniques and find a unifying language for discussion and comparison.

Deep neural networks are currently among the most commonly used classifiers. Despite easily achieving very good performance, one of the best selling points of these models is their modular design - one can conveniently adapt their architecture to specific needs, change connectivity patterns, attach specialised layers, experiment with a large amount of activation functions, normalisation schemes and many others. While one can find impressively wide spread of various configurations of almost every aspect of the deep nets, one element is, in authors' opinion, underrepresented - while solving classification problems, vast majority of papers and applications simply use log loss. In this paper we try to investigate how particular choices of loss functions affect deep models and their learning dynamics, as well as resulting classifiers robustness to various effects. We perform experiments on classical datasets, as well as provide some additional, theoretical insights into the problem. In particular we show that L1 and L2 losses are, quite surprisingly, justified classification objectives for deep nets, by providing probabilistic interpretation in terms of expected misclassification. We also introduce two losses which are not typically used as deep nets objectives and show that they are viable alternatives to the existing ones., Comment: Presented at Theoretical Foundations of Machine Learning 2017 (TFML 2017)

Maybe the single most important goal of representation learning is making subsequent learning faster. Surprisingly, this fact is not well reflected in the way embeddings are evaluated. In addition, recent practice in word embeddings points towards importance of learning specialized representations. We argue that focus of word representation evaluation should reflect those trends and shift towards evaluating what useful information is easily accessible. Specifically, we propose that evaluation should focus on data efficiency and simple supervised tasks, where the amount of available data is varied and scores of a supervised model are reported for each subset (as commonly done in transfer learning). In order to illustrate significance of such analysis, a comprehensive evaluation of selected word embeddings is presented. Proposed approach yields a more complete picture and brings new insight into performance characteristics, for instance information about word similarity or analogy tends to be non--linearly encoded in the embedding space, which questions the cosine-based, unsupervised, evaluation methods. All results and analysis scripts are available online.