Search

Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to "real world" events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of "Lifelong Learning" systems that are capable of 1) Continuous Learning, 2) Transfer and Adaptation, and 3) Scalability. Unfortunately, efforts to improve these capabilities are typically treated as distinct areas of research that are assessed independently, without regard to the impact of each separate capability on other aspects of the system. We instead propose a holistic approach, using a suite of metrics and an evaluation framework to assess Lifelong Learning in a principled way that is agnostic to specific domains or system techniques. Through five case studies, we show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems. We highlight how the proposed suite of metrics quantifies performance trade-offs present during Lifelong Learning system development - both the widely discussed Stability-Plasticity dilemma and the newly proposed relationship between Sample Efficient and Robust Learning. Further, we make recommendations for the formulation and use of metrics to guide the continuing development of Lifelong Learning systems and assess their progress in the future., Comment: To appear in Neural Networks

We describe a process for cross-calibrating the effective areas of X-ray telescopes that observe common targets. The targets are not assumed to be "standard candles" in the classic sense, in that we assume that the source fluxes have well-defined, but {\it a priori} unknown values. Using a technique developed by Chen et al. (2019, arXiv:1711.09429) that involves a statistical method called {\em shrinkage estimation}, we determine effective area correction factors for each instrument that brings estimated fluxes into the best agreement, consistent with prior knowledge of their effective areas. We expand the technique to allow unique priors on systematic uncertainties in effective areas for each X-ray astronomy instrument and to allow correlations between effective areas in different energy bands. We demonstrate the method with several data sets from various X-ray telescopes., Comment: 37 pages, 9 figures, and 10 tables plus an appendix with 7 data tables. Accepted for publication in the Astronomical Journal

Identifying covariate shift is crucial for making machine learning systems robust in the real world and for detecting training data biases that are not reflected in test data. However, detecting covariate shift is challenging, especially when the data consists of high-dimensional images, and when multiple types of localized covariate shift affect different subspaces of the data. Although automated techniques can be used to detect the existence of covariate shift, our goal is to help human users characterize the extent of covariate shift in large image datasets with interfaces that seamlessly integrate information obtained from the detection algorithms. In this paper, we design and evaluate a new visual interface that facilitates the comparison of the local distributions of training and test data. We conduct a quantitative user study on multi-attribute facial data to compare two different learned low-dimensional latent representations (pretrained ImageNet CNN vs. density ratio) and two user analytic workflows (nearest-neighbor vs. cluster-to-cluster). Our results indicate that the latent representation of our density ratio model, combined with a nearest-neighbor comparison, is the most effective at helping humans identify covariate shift., Comment: IEEE VIS 2021

Recently, particle-based variational inference (ParVI) methods have gained interest because they can avoid arbitrary parametric assumptions that are common in variational inference. However, many ParVI approaches do not allow arbitrary sampling from the posterior, and the few that do allow such sampling suffer from suboptimality. This work proposes a new method for learning to approximately sample from the posterior distribution. We construct a neural sampler that is trained with the functional gradient of the KL-divergence between the empirical sampling distribution and the target distribution, assuming the gradient resides within a reproducing kernel Hilbert space. Our generative ParVI (GPVI) approach maintains the asymptotic performance of ParVI methods while offering the flexibility of a generative sampler. Through carefully constructed experiments, we show that GPVI outperforms previous generative ParVI methods such as amortized SVGD, and is competitive with ParVI as well as gold-standard approaches like Hamiltonian Monte Carlo for fitting both exactly known and intractable target distributions., Comment: 22 pages, 9 figures, 10 tables, 1 algorithm

Reward function specification can be difficult. Rewarding the agent for making a widget may be easy, but penalizing the multitude of possible negative side effects is hard. In toy environments, Attainable Utility Preservation (AUP) avoided side effects by penalizing shifts in the ability to achieve randomly generated goals. We scale this approach to large, randomly generated environments based on Conway's Game of Life. By preserving optimal value for a single randomly generated reward function, AUP incurs modest overhead while leading the agent to complete the specified task and avoid many side effects. Videos and code are available at https://avoiding-side-effects.github.io/., Comment: Accepted as spotlight paper at NeurIPS 2020. 10 pages main paper; 19 pages with appendices

Efficient exploration remains a challenging problem in reinforcement learning, especially for those tasks where rewards from environments are sparse. A commonly used approach for exploring such environments is to introduce some "intrinsic" reward. In this work, we focus on model uncertainty estimation as an intrinsic reward for efficient exploration. In particular, we introduce an implicit generative modeling approach to estimate a Bayesian uncertainty of the agent's belief of the environment dynamics. Each random draw from our generative model is a neural network that instantiates the dynamic function, hence multiple draws would approximate the posterior, and the variance in the future prediction based on this posterior is used as an intrinsic reward for exploration. We design a training algorithm for our generative model based on the amortized Stein Variational Gradient Descent. In experiments, we compare our implementation with state-of-the-art intrinsic reward-based exploration approaches, including two recent approaches based on an ensemble of dynamic models. In challenging exploration tasks, our implicit generative model consistently outperforms competing approaches regarding data efficiency in exploration., Comment: 14 pages, 9 figures, Accepted to ICML 2020

The Askaryan Radio Array (ARA) is an ultra-high energy (UHE) neutrino telescope at the South Pole consisting of an array of radio antennas aimed at detecting the Askaryan radiation produced by neutrino interactions in the ice. Currently, the experiment has five stations in operation that have been deployed in stages since 2012. This contribution focuses on the development of a search for a diffuse flux of neutrinos in two ARA stations (A2 and A3) from 2013-2016. A background of $\sim 0.01-0.02$ events is expected in one station in each of two search channels in horizontal- and vertical-polarizations. The expected new constraints on the flux of ultra-high energy neutrinos based on four years of analysis with two stations improve on the previous limits set by ARA by a factor of about two. The projected sensitivity of ARA's five-station dataset is beginning to be competitive with other neutrino telescopes at high energies near $10^{10.5}\,$GeV., Comment: 7 pages, 5 figures, prepared for proceedings for the International Cosmic Ray Conference, Madison, WI, 2019

Standard neural networks are often overconfident when presented with data outside the training distribution. We introduce HyperGAN, a new generative model for learning a distribution of neural network parameters. HyperGAN does not require restrictive assumptions on priors, and networks sampled from it can be used to quickly create very large and diverse ensembles. HyperGAN employs a novel mixer to project prior samples to a latent space with correlated dimensions, and samples from the latent space are then used to generate weights for each layer of a deep neural network. We show that HyperGAN can learn to generate parameters which label the MNIST and CIFAR-10 datasets with competitive performance to fully supervised learning, while learning a rich distribution of effective parameters. We also show that HyperGAN can also provide better uncertainty estimates than standard ensembles by evaluating on out of distribution data as well as adversarial examples., Comment: 11 pages, 10 figures, 6 tables, Accepted to ICML 2019

The Askaryan Radio Array (ARA) reports an observation of radio emission coincident with the "Valentine's Day" solar flare on Feb. 15$^{\rm{th}}$, 2011 in the prototype "Testbed" station. We find $\sim2000$ events that passed our neutrino search criteria during the 70 minute period of the flare, all of which reconstruct to the location of the sun. A signal analysis of the events reveals them to be consistent with that of bright thermal noise correlated across antennas. This is the first natural source of radio emission reported by ARA that is tightly reconstructable on an event-by-event basis. The observation is also the first for ARA to point radio from individual events to an extraterrestrial source on the sky. We comment on how the solar flares, coupled with improved systematic uncertainties in reconstruction algorithms, could aid in a mapping of any above-ice radio emission, such as that from cosmic-ray air showers, to astronomical locations on the sky., Comment: 18 pages, 22 figures

Recent analysis of deep neural networks has revealed their vulnerability to carefully structured adversarial examples. Many effective algorithms exist to craft these adversarial examples, but performant defenses seem to be far away. In this work, we explore the use of edge-aware bilateral filtering as a projection back to the space of natural images. We show that bilateral filtering is an effective defense in multiple attack settings, where the strength of the adversary gradually increases. In the case of an adversary who has no knowledge of the defense, bilateral filtering can remove more than 90% of adversarial examples from a variety of different attacks. To evaluate against an adversary with complete knowledge of our defense, we adapt the bilateral filter as a trainable layer in a neural network and show that adding this layer makes ImageNet images significantly more robust to attacks. When trained under a framework of adversarial training, we show that the resulting model is hard to fool with even the best attack methods., Comment: 9 pages, 14 figures

The balloon-borne HiCal radio-frequency (RF) transmitter, in concert with the ANITA radio-frequency receiver array, is designed to measure the Antarctic surface reflectivity in the RF wavelength regime. The amplitude of surface-reflected transmissions from HiCal, registered as triggered events by ANITA, can be compared with the direct transmissions preceding them by O(10) microseconds, to infer the surface power reflection coefficient $\cal{R}$. The first HiCal mission (HiCal-1, Jan. 2015) yielded a sample of 100 such pairs, resulting in estimates of $\cal{R}$ at highly-glancing angles (i.e., zenith angles approaching $90^\circ$), with measured reflectivity for those events which exceeded extant calculations. The HiCal-2 experiment, flying from Dec., 2016-Jan., 2017, provided an improvement by nearly two orders of magnitude in our event statistics, allowing a considerably more precise mapping of the reflectivity over a wider range of incidence angles. We find general agreement between the HiCal-2 reflectivity results and those obtained with the earlier HiCal-1 mission, as well as estimates from Solar reflections in the radio-frequency regime. In parallel, our calculations of expected reflectivity have matured; herein, we use a plane-wave expansion to estimate the reflectivity R from both a flat, smooth surface (and, in so doing, recover the Fresnel reflectivity equations) and also a curved surface. Multiplying our flat-smooth reflectivity by improved Earth curvature and surface roughness corrections now provides significantly better agreement between theory and the HiCal 2a/2b measurements., Comment: submitted to Astropart. Phys

Using data collected by the Askaryan Radio Array (ARA) experiment at the South Pole, we have used long-baseline propagation of radio-frequency signals to extract information on the radio-frequency index-of-refraction in South Polar ice. Owing to the increasing ice density over the upper 150--200 meters, rays are observed along two, nearly parallel paths, one of which is direct and a second which refracts through an inflection point, with differences in both arrival time and arrival angle that can be used to constrain the neutrino properties. We also observe indications, for the first time, of radio-frequency ice birefringence for signals propagating along predominantly horizontal trajectories, corresponding to an asymmetry of order 0.1% between the ordinary and extra-ordinary paths, numerically compatible with previous measurements of birefringent asymmetries for vertically-propagating radio-frequency signals at South Pole. Taken together, these effects offer the possibility of redundantly measuring the range from receiver to a neutrino interaction in Antarctic ice, if receiver antennas are deployed at shallow (25 m

The NASA supported High-Altitude Calibration (HiCal)-2 instrument flew as a companion balloon to the ANITA-4 experiment in December 2016. Based on a HV discharge pulser producing radio-frequency (RF) calibration pulses, HiCal-2 comprised two payloads, which flew for a combined 18 days, covering 1.5 revolutions of the Antarctic continent. ANITA-4 captured over 10,000 pulses from HiCal, both direct and reflected from the surface, at distances varying from 100-800 km, providing a large dataset for surface reflectivity measurements. Herein we present details on the design, construction and performance of HiCal-2., Comment: Published in NIM-A, final version

Interest in Radio-based detection schemes for ultra-high energy cosmic rays (UHECR) has surged in recent years, owing to the potentially very low cost/detection ratio. The method of radio-frequency (RF) scatter has been proposed as potentially the most economical detection technology. Though the first dedicated experiment to employ this method, the Telescope Array RADAR experiment (TARA), reported no signal, efforts to develop more robust and sensitive trigger techniques continue. This paper details the development of a time-domain firmware trigger that exploits characteristics of the expected scattered signal from an UHECR extensive-air shower (EAS). The improved sensitivity of this trigger is discussed, as well as implementation in two separate field deployments from 2016-2017.

The primary science goal of the NASA-sponsored ANITA project is measurement of ultra-high energy neutrinos and cosmic rays, observed via radio-frequency signals resulting from a neutrino- or cosmic ray- interaction with terrestrial matter (atmospheric or ice molecules, e.g.). Accurate inference of the energies of these cosmic rays requires understanding the transmission/reflection of radio wave signals across the ice-air boundary. Satellite-based measurements of Antarctic surface reflectivity, using a co-located transmitter and receiver, have been performed more-or-less continuously for the last few decades. Satellite-based reflectivity surveys, at frequencies ranging from 2--45 GHz and at near-normal incidence, yield generally consistent reflectivity maps across Antarctica. Using the Sun as an RF source, and the ANITA-3 balloon borne radio-frequency antenna array as the RF receiver, we have also measured the surface reflectivity over the interval 200-1000 MHz, at elevation angles of 12-30 degrees, finding agreement with the Fresnel equations within systematic errors. To probe low incidence angles, inaccessible to the Antarctic Solar technique and not probed by previous satellite surveys, a novel experimental approach ("HiCal-1") was devised. Unlike previous measurements, HiCal-ANITA constitute a bi-static transmitter-receiver pair separated by hundreds of kilometers. Data taken with HiCal, between 200--600 MHz shows a significant departure from the Fresnel equations, constant with frequency over that band, with the deficit increasing with obliquity of incidence, which we attribute to the combined effects of possible surface roughness, surface grain effects, radar clutter and/or shadowing of the reflection zone due to Earth curvature effects., Comment: updated to match publication version

In models with multiple nondegenerate Higgs bosons, the decay chain $H/A \to A/H Z\to t\bar{t} Z$ may have a partial width comparable to the $t\bar{t}$ decay mode. We recast the ATLAS standard model $t\bar{t} Z$ measurement to put limits on the rate for this process. Limits are also set on the two Higgs doublet model at low $\tan \beta$ that are sensitive to a heavy Higgs mass as high as $\sim 750$ GeV. We then discuss the $750$ GeV diphoton excess in terms of a pseudoscalar that also has the decays $A\to HZ$ and $A\to H^\pm W^\mp$. These decays strongly constrain the partial widths for $A\to\gamma\gamma$ and $A\to gg$ when combined with the $t\bar t$ resonance search limits. In a benchmark model the mass of $H$ should be close to 650 GeV., Comment: 12 pages, 3 figures, matches published version

The ARIANNA collaboration completed the installation of the hexagonal radio array (HRA) in December 2014, serving as a pilot program for a planned high energy neutrino telescope located about 110 km south of McMurdo Station on the Ross Ice Shelf near the coast of Antarctica. The goal of ARIANNA is to measure both diffuse and point fluxes of astrophysical neutrinos at energies in excess of 1016 eV. Upgraded hardware has been installed during the 2014 deployment season and stations show a livetime of better than 90% between commissioning and austral sunset. Though designed to observe radio pulses from neutrino interactions originating within the ice below each detector, one station was modified to study the low-frequency environment and signals from above. We provide evidence that the HRA observed both continuous emission from the Galaxy and a transient solar burst. Preliminary work on modeling the (weak) Galactic signal confirm the absolute sensitivity of the HRA detector system., Comment: Proceedings from the 34th ICRC2015, http://icrc2015.nl/, 8 pages, 6 figures

Installation of the ARIANNA Hexagonal Radio Array (HRA) on the Ross Ice Shelf of Antarctica has been completed. This detector serves as a pilot program to the ARIANNA neutrino telescope, which aims to measure the diffuse flux of very high energy neutrinos by observing the radio pulse generated by neutrino-induced charged particle showers in the ice. All HRA stations ran reliably and took data during the entire 2014-2015 austral summer season. A new radio signal direction reconstruction procedure is described, and is observed to have a resolution better than a degree. The reconstruction is used in a preliminary search for potential neutrino candidate events in the data from one of the newly installed detector stations. Three cuts are used to separate radio backgrounds from neutrino signals. The cuts are found to filter out all data recorded by the station during the season while preserving 85.4% of simulated neutrino events that trigger the station. This efficiency is similar to that found in analyses of previous HRA data taking seasons., Comment: Proceedings from the 34th ICRC2015, http://icrc2015.nl/ . 8 pages, 6 figures

Ultra-high energy neutrinos are interesting messenger particles since, if detected, they can transmit exclusive information about ultra-high energy processes in the Universe. These particles, with energies above $10^{16}\mathrm{eV}$, interact very rarely. Therefore, detectors that instrument several gigatons of matter are needed to discover them. The ARA detector is currently being constructed at South Pole. It is designed to use the Askaryan effect, the emission of radio waves from neutrino-induced cascades in the South Pole ice, to detect neutrino interactions at very high energies. With antennas distributed among 37 widely-separated stations in the ice, such interactions can be observed in a volume of several hundred cubic kilometers. Currently 3 deep ARA stations are deployed in the ice of which two have been taking data since the beginning of the year 2013. In this publication, the ARA detector "as-built" and calibrations are described. Furthermore, the data reduction methods used to distinguish the rare radio signals from overwhelming backgrounds of thermal and anthropogenic origin are presented. Using data from only two stations over a short exposure time of 10 months, a neutrino flux limit of $3 \cdot 10^{-6} \mathrm{GeV} / (\mathrm{cm^2 \ s \ sr})$ is calculated for a particle energy of 10^{18}eV, which offers promise for the full ARA detector., Comment: 21 pages, 34 figures, 1 table, includes supplementary material

We report on a search for ultra-high-energy (UHE) neutrinos from gamma-ray bursts (GRBs) in the data set collected by the Testbed station of the Askaryan Radio Array (ARA) in 2011 and 2012. From 57 selected GRBs, we observed no events that survive our cuts, which is consistent with 0.12 expected background events. Using NeuCosmA as a numerical GRB reference emission model, we estimate upper limits on the prompt UHE GRB neutrino fluence and quasi-diffuse flux from $10^{7}$ to $10^{10}$ GeV. This is the first limit on the prompt UHE GRB neutrino quasi-diffuse flux above $10^{7}$ GeV., Comment: 14 pages, 8 figures, Published in Astroparticle Physics Journal

Employing a 40-kW radio-frequency transmitter just west of Delta, UT, and operating at 54.1 MHz, the TARA (Telescope Array RAdar) experiment seeks radar detection of extensive air showers (EAS) initiated by ultra-high energy cosmic rays (UHECR). For UHECR with energies in excess of $10^{19}$ eV, the Doppler-shifted "chirps" resulting from EAS shower core radar reflections should be observable above background (dominantly galactic) at distances of tens of km from the TARA transmitter. In order to stereoscopically reconstruct cosmic ray chirps, two remote, autonomous self-powered receiver stations have been deployed. Each remote station (RS) combines both low power consumption as well as low cost. Triggering logic, the powering and communication systems, and some specific details of hardware components are discussed., Comment: version accepted for publication in Nucl. Instr. Meth. A

For a heavy Higgs boson above the $t\bar t$ threshold we consider three decay modes that could compete with decay to $t\bar t$, even in the alignment limit of a 2-Higgs doublet model. The decays $H\to AZ$, $ Z'Z'$ and $\tau'\tau'$ show that heavy Higgs decays can be an effective probe of new physics both within and beyond this model., Comment: 6 pages, 1 figure