Search

Federated learning (FL) is an approach to training machine learning models that takes advantage of multiple distributed datasets while maintaining data privacy and reducing communication costs associated with sharing local datasets. Aggregation strategies have been developed to pool or fuse the weights and biases of distributed deterministic models; however, modern deterministic deep learning (DL) models are often poorly calibrated and lack the ability to communicate a measure of epistemic uncertainty in prediction, which is desirable for remote sensing platforms and safety-critical applications. Conversely, Bayesian DL models are often well calibrated and capable of quantifying and communicating a measure of epistemic uncertainty along with a competitive prediction accuracy. Unfortunately, because the weights and biases in Bayesian DL models are defined by a probability distribution, simple application of the aggregation methods associated with FL schemes for deterministic models is either impossible or results in sub-optimal performance. In this work, we use independent and identically distributed (IID) and non-IID partitions of the CIFAR-10 dataset and a fully variational ResNet-20 architecture to analyze six different aggregation strategies for Bayesian DL models. Additionally, we analyze the traditional federated averaging approach applied to an approximate Bayesian Monte Carlo dropout model as a lightweight alternative to more complex variational inference methods in FL. We show that aggregation strategy is a key hyperparameter in the design of a Bayesian FL system with downstream effects on accuracy, calibration, uncertainty quantification, training stability, and client compute requirements., Comment: 22 pages, 9 figures

Transfer learning (TL) is an increasingly popular approach to training deep learning (DL) models that leverages the knowledge gained by training a foundation model on diverse, large-scale datasets for use on downstream tasks where less domain- or task-specific data is available. The literature is rich with TL techniques and applications; however, the bulk of the research makes use of deterministic DL models which are often uncalibrated and lack the ability to communicate a measure of epistemic (model) uncertainty in prediction. Unlike their deterministic counterparts, Bayesian DL (BDL) models are often well-calibrated, provide access to epistemic uncertainty for a prediction, and are capable of achieving competitive predictive performance. In this study, we propose variational inference pre-trained audio neural networks (VI-PANNs). VI-PANNs are a variational inference variant of the popular ResNet-54 architecture which are pre-trained on AudioSet, a large-scale audio event detection dataset. We evaluate the quality of the resulting uncertainty when transferring knowledge from VI-PANNs to other downstream acoustic classification tasks using the ESC-50, UrbanSound8K, and DCASE2013 datasets. We demonstrate, for the first time, that it is possible to transfer calibrated uncertainty information along with knowledge from upstream tasks to enhance a model's capability to perform downstream tasks., Comment: Published in IEEE Access

Host galaxy identification is a crucial step for modern supernova (SN) surveys such as the Dark Energy Survey (DES) and the Large Synoptic Survey Telescope (LSST), which will discover SNe by the thousands. Spectroscopic resources are limited, so in the absence of real-time SN spectra these surveys must rely on host galaxy spectra to obtain accurate redshifts for the Hubble diagram and to improve photometric classification of SNe. In addition, SN luminosities are known to correlate with host-galaxy properties. Therefore, reliable identification of host galaxies is essential for cosmology and SN science. We simulate SN events and their locations within their host galaxies to develop and test methods for matching SNe to their hosts. We use both real and simulated galaxy catalog data from the Advanced Camera for Surveys General Catalog and MICECATv2.0, respectively. We also incorporate "hostless" SNe residing in undetected faint hosts into our analysis, with an assumed hostless rate of 5%. Our fully automated algorithm is run on catalog data and matches SNe to their hosts with 91% accuracy. We find that including a machine learning component, run after the initial matching algorithm, improves the accuracy (purity) of the matching to 97% with a 2% cost in efficiency (true positive rate). Although the exact results are dependent on the details of the survey and the galaxy catalogs used, the method of identifying host galaxies we outline here can be applied to any transient survey., Comment: 22 pages, 18 figures; Accepted by AJ, revised to incorporate referee comments

Using the largest single-survey sample of Type Ia supernovae (SNe Ia) to date, we study the relationship between properties of SNe Ia and those of their host galaxies, focusing primarily on correlations with Hubble residuals (HR). Our sample consists of 345 photometrically-classified or spectroscopically-confirmed SNeIa discovered as part of the SDSS-II Supernova Survey (SDSS-SNS). This analysis utilizes host-galaxy spectroscopy obtained during the SDSS-I/II spectroscopic survey and from an ancillary program on the SDSS-III Baryon Oscillation Spectroscopic Survey (BOSS) that obtained spectra for nearly all host galaxies of SDSS-II SN candidates. In addition, we use photometric host-galaxy properties from the SDSS-SNS data release (Sako et al. 2014) such as host stellar mass and star-formation rate. We confirm the well-known relation between HR and host-galaxy mass and find a 3.6{\sigma} significance of a non-zero linear slope. We also recover correlations between HR and host-galaxy gas-phase metallicity and specific star-formation rate as they are reported in the literature. With our large dataset, we examine correlations between HR and multiple host-galaxy properties simultaneously and find no evidence of a significant correlation. We also independently analyze our spectroscopically-confirmed and photometrically-classified SNe Ia and comment on the significance of similar combined datasets for future surveys., Comment: Accepted for publication in ApJ