Search

In recent years, Large Language Models (LLMs) have demonstrated remarkable capabilities across a wide array of text-centric tasks. However, their `large' scale introduces significant computational and storage challenges, particularly in managing the key-value states of the transformer, which limits their wider applicability. Therefore, we propose to adaptively release resources from caches and rebuild the necessary key-value states. Particularly, we accomplish this by a lightweight controller module to approximate an ideal top-$K$ sparse attention. This module retains the tokens with the highest top-$K$ attention weights and simultaneously rebuilds the discarded but necessary tokens, which may become essential for future decoding. Comprehensive experiments in natural language generation and modeling reveal that our method is not only competitive with full attention in terms of performance but also achieves a significant throughput improvement of up to 221.8%. The code for replication is available on the https://github.com/WHUIR/ADORE., Comment: Accepted at ACL 2024(Findings)

SN 1987A was an unusual hydrogen-rich core-collapse supernova originating from a blue supergiant star. Similar blue supergiant explosions remain a small family of events, and are broadly characterized by their long rises to peak. The Zwicky Transient Facility (ZTF) Census of the Local Universe (CLU) experiment aims to construct a spectroscopically complete sample of transients occurring in galaxies from the CLU galaxy catalog. We identify 13 long-rising (>40 days) Type II supernovae from the volume-limited CLU experiment during a 3.5 year period from June 2018 to December 2021, approximately doubling the previously known number of these events. We present photometric and spectroscopic data of these 13 events, finding peak r-band absolute magnitudes ranging from -15.6 to -17.5 mag and the tentative detection of Ba II lines in 9 events. Using our CLU sample of events, we derive a long-rising Type II supernova rate of $1.37^{+0.26}_{-0.30}\times10^{-6}$ Mpc$^{-3}$ yr$^{-1}$, $\approx$1.4% of the total core-collapse supernova rate. This is the first volumetric rate of these events estimated from a large, systematic, volume-limited experiment., Comment: 33 pages, 18 figures, 5 tables. Updated to ApJ accepted version

Careful placement of a computational application within a target device cluster is critical for achieving low application completion time. The problem is challenging due to its NP-hardness and combinatorial nature. In recent years, learning-based approaches have been proposed to learn a placement policy that can be applied to unseen applications, motivated by the problem of placing a neural network across cloud servers. These approaches, however, generally assume the device cluster is fixed, which is not the case in mobile or edge computing settings, where heterogeneous devices move in and out of range for a particular application. We propose a new learning approach called GiPH, which learns policies that generalize to dynamic device clusters via 1) a novel graph representation gpNet that efficiently encodes the information needed for choosing a good placement, and 2) a scalable graph neural network (GNN) that learns a summary of the gpNet information. GiPH turns the placement problem into that of finding a sequence of placement improvements, learning a policy for selecting this sequence that scales to problems of arbitrary size. We evaluate GiPH with a wide range of task graphs and device clusters and show that our learned policy rapidly find good placements for new problem instances. GiPH finds placements with up to 30.5% lower completion times, searching up to 3X faster than other search-based placement policies., Comment: to be published in Proceedings of Machine Learning and Systems 5 (MLSys 2023)

One of the open questions following the discovery of GW170817 is whether neutron star mergers are the only astrophysical sites capable of producing $r$-process elements. Simulations have shown that 0.01-0.1M$_\odot$ of $r$-process material could be generated in the outflows originating from the accretion disk surrounding the rapidly rotating black hole that forms as a remnant to both neutron star mergers and collapsing massive stars associated with long-duration gamma-ray bursts (collapsars). The hallmark signature of $r$-process nucleosynthesis in the binary neutron star merger GW170817 was its long-lasting near-infrared emission, thus motivating a systematic photometric study of the light curves of broadlined stripped-envelope (Ic-BL) supernovae (SNe) associated with collapsars. We present the first systematic study of 25 SNe Ic-BL -- including 18 observed with the Zwicky Transient Facility and 7 from the literature -- in the optical/near-infrared bands to determine what quantity of $r$-process material, if any, is synthesized in these explosions. Using semi-analytic models designed to account for $r$-process production in SNe Ic-BL, we perform light curve fitting to derive constraints on the $r$-process mass for these SNe. We also perform independent light curve fits to models without $r$-process. We find that the $r$-process-free models are a better fit to the light curves of the objects in our sample. Thus we find no compelling evidence of $r$-process enrichment in any of our objects. Further high-cadence infrared photometric studies and nebular spectroscopic analysis would be sensitive to smaller quantities of $r$-process ejecta mass or indicate whether all collapsars are completely devoid of $r$-process nucleosynthesis., Comment: 37 pages, 8 figures, 4 tables

Luminous red novae (LRNe) are transients characterized by low luminosities and expansion velocities, and are associated with mergers or common envelope ejections in stellar binaries. Intermediate-luminosity red transients (ILRTs) are an observationally similar class with unknown origins, but generally believed to either be electron capture supernovae (ECSN) in super-AGB stars, or outbursts in dusty luminous blue variables (LBVs). In this paper, we present a systematic sample of 8 LRNe and 8 ILRTs detected as part of the Census of the Local Universe (CLU) experiment on the Zwicky Transient Facility (ZTF). The CLU experiment spectroscopically classifies ZTF transients associated with nearby ($<150$ Mpc) galaxies, achieving 80% completeness for m$_{r}<20$\,mag. Using the ZTF-CLU sample, we derive the first systematic LRNe volumetric-rate of 7.8$^{+6.5}_{-3.7}\times10^{-5}$ Mpc$^{-3}$ yr$^{-1}$ in the luminosity range $-16\leq$M$_{\rm{r}}$$\leq -11$ mag. We find that in this luminosity range, the LRN rate scales as dN/dL $\propto L^{-2.5\pm0.3}$ - significantly steeper than the previously derived scaling of $L^{-1.4\pm0.3}$ for lower luminosity LRNe (M$_{V}\geq-10$). The steeper power law for LRNe at high luminosities is consistent with the massive merger rates predicted by binary population synthesis models. We find that the rates of the brightest LRNe (M$_{r}\leq-13$ mag) are consistent with a significant fraction of them being progenitors of double compact objects (DCOs) that merge within a Hubble time. For ILRTs, we derive a volumetric rate of $2.6^{+1.8}_{-1.4}\times10^{-6}$ Mpc$^{-3}$yr$^{-1}$ for M$_{\rm{r}}\leq-13.5$, that scales as dN/dL $\propto L^{-2.5\pm0.5}$. This rate is $\approx1-5\%$ of the local core-collapse supernova rate, and is consistent with theoretical ECSN rate estimates., Comment: 32 pages, 16 figures. Submitted to ApJ, comments welcome

This paper studies faithful explanations for Graph Neural Networks (GNNs). First, we provide a new and general method for formally characterizing the faithfulness of explanations for GNNs. It applies to existing explanation methods, including feature attributions and subgraph explanations. Second, our analytical and empirical results demonstrate that feature attribution methods cannot capture the nonlinear effect of edge features, while existing subgraph explanation methods are not faithful. Third, we introduce \emph{k-hop Explanation with a Convolutional Core} (KEC), a new explanation method that provably maximizes faithfulness to the original GNN by leveraging information about the graph structure in its adjacency matrix and its \emph{k-th} power. Lastly, our empirical results over both synthetic and real-world datasets for classification and anomaly detection tasks with GNNs demonstrate the effectiveness of our approach.

The Zwicky Transient Facility (ZTF), a public-private enterprise, is a new time domain survey employing a dedicated camera on the Palomar 48-inch Schmidt telescope with a 47 deg$^2$ field of view and 8 second readout time. It is well positioned in the development of time domain astronomy, offering operations at 10% of the scale and style of the Large Synoptic Survey Telescope (LSST) with a single 1-m class survey telescope. The public surveys will cover the observable northern sky every three nights in g and r filters and the visible Galactic plane every night in g and r. Alerts generated by these surveys are sent in real time to brokers. A consortium of universities which provided funding ("partnership") are undertaking several boutique surveys. The combination of these surveys producing one million alerts per night allows for exploration of transient and variable astrophysical phenomena brighter than r $\sim$ 20.5 on timescales of minutes to years. We describe the primary science objectives driving ZTF including the physics of supernovae and relativistic explosions, multi-messenger astrophysics, supernova cosmology, active galactic nuclei and tidal disruption events, stellar variability, and Solar System objects., Comment: 26 pages, 7 figures, Published in PASP Focus Issue on the Zwicky Transient Facility