Search

In order to satisfy their ever increasing capacity and compute requirements, machine learning models are distributed across multiple nodes using numerous parallelism strategies. As a result, collective communications are often on the critical path, and hiding their latency by overlapping kernel-granular communication and computation is difficult due to the absence of independent computation. In this work, we propose fusing computation with dependent collective communication by leveraging GPUs' massive parallelism and GPU-initiated communication. We have developed self-contained GPU kernels where workgroups (WGs) immediately communicate their results to remote GPUs when they complete their computation. Meanwhile, other WGs within the same kernel perform overlapping computation, maintaining high ALU utilization. We demonstrate our approach by creating three prototype fused operators (embedding + All-to-All, GEMV + AllReduce, and GEMM + All-to-All) to address the pervasive communication overheads observed in DLRM, Transformers and MoE model architectures. In order to demonstrate that our approach can be integrated into ML frameworks for wide adoption in production environments, we expose our fused operators as new PyTorch operators as well as extend the Triton framework to enable them. Our evaluations show that our approach can effectively overlap communication with computations, subsequently reducing their combined execution time than the current collective library-based approaches. Our scale-up GEMV + AllReduce and GEMM + All-to-All implementations achieve up to 22% and 20% lower execution time, while our fused embedding + All-to-All reduces execution time by 20% and 31% for intra-node and inter-node configurations. Large scale-out simulations indicate that our approach reduces DLRM execution time by 21% for 128 node system.

In this era of spatially resolved observations of planet forming disks with ALMA and large ground-based telescopes such as the VLT, Keck and Subaru, we still lack statistically relevant information on the quantity and composition of the material that is building the planets, such as the total disk gas mass, the ice content of dust, and the state of water in planetesimals. SPICA is an infrared space mission concept developed jointly by JAXA and ESA to address these questions. The key unique capabilities of SPICA that enable this research are (1) the wide spectral coverage 10-220 micron, (2) the high line detection sensitivity of (1-2) 10-19 W m-2 with R~2000-5000 in the far-IR (SAFARI) and 10-20 W m-2 with R~29000 in the mid-IR (SMI, spectrally resolving line profiles), (3) the high far-IR continuum sensitivity of 0.45 mJy (SAFARI), and (4) the observing efficiency for point source surveys. This paper details how mid- to far-IR infrared spectra will be unique in measuring the gas masses and water/ice content of disks and how these quantities evolve during the planet forming period. These observations will clarify the crucial transition when disks exhaust their primordial gas and further planet formation requires secondary gas produced from planetesimals. The high spectral resolution mid-IR is also unique for determining the location of the snowline dividing the rocky and icy mass reservoirs within the disk and how the divide evolves during the build-up of planetary systems. Infrared spectroscopy (mid- to far-IR) of key solid state bands is crucial for assessing whether extensive radial mixing, which is part of our Solar System history, is a general process occurring in most planetary systems and whether extrasolar planetesimals are similar to our Solar System comets/asteroids. ... (abbreviated), Comment: accepted for publication in PASA

In recent years, machine intelligence (MI) applications have emerged as a major driver for the computing industry. Optimizing these workloads is important but complicated. As memory demands grow and data movement overheads increasingly limit performance, determining the best GPU caching policy to use for a diverse range of MI workloads represents one important challenge. To study this, we evaluate 17 MI applications and characterize their behaviors using a range of GPU caching strategies. In our evaluations, we find that the choice of caching policy in GPU caches involves multiple performance trade-offs and interactions, and there is no one-size-fits-all GPU caching policy for MI workloads. Based on detailed simulation results, we motivate and evaluate a set of cache optimizations that consistently match the performance of the best static GPU caching policies., Comment: Extended version of short paper published in the 2019 IEEE International Symposium on Workload Characterization

With the recent discovery of a dozen dusty star-forming galaxies and around 30 quasars at z>5 that are hyper-luminous in the infrared ($\mu$$L_{\rm IR}>10^{13}$ L$_{\odot}$, where $\mu$ is a lensing magnification factor), the possibility has opened up for SPICA, the proposed ESA M5 mid-/far-infrared mission, to extend its spectroscopic studies toward the epoch of reionization and beyond. In this paper, we examine the feasibility and scientific potential of such observations with SPICA's far-infrared spectrometer SAFARI, which will probe a spectral range (35-230 $\mu$m) that will be unexplored by ALMA and JWST. Our simulations show that SAFARI is capable of delivering good-quality spectra for hyper-luminous infrared galaxies (HyLIRGs) at z=5-10, allowing us to sample spectral features in the rest-frame mid-infrared and to investigate a host of key scientific issues, such as the relative importance of star formation versus AGN, the hardness of the radiation field, the level of chemical enrichment, and the properties of the molecular gas. From a broader perspective, SAFARI offers the potential to open up a new frontier in the study of the early Universe, providing access to uniquely powerful spectral features for probing first-generation objects, such as the key cooling lines of low-metallicity or metal-free forming galaxies (fine-structure and H2 lines) and emission features of solid compounds freshly synthesized by Population III supernovae. Ultimately, SAFARI's ability to explore the high-redshift Universe will be determined by the availability of sufficiently bright targets (whether intrinsically luminous or gravitationally lensed). With its launch expected around 2030, SPICA is ideally positioned to take full advantage of upcoming wide-field surveys such as LSST, SKA, Euclid, and WFIRST, which are likely to provide extraordinary targets for SAFARI., Comment: 24 pages, 8 figures, accepted for publication in PASA (SPICA special issue)

We investigate the far infrared spectrum of NGC 1266, a S0 galaxy that contains a massive reservoir of highly excited molecular gas. Using the SPIRE-FTS, we detect the $^{12}$CO ladder up to J=(13-12), [C I] and [N II] lines, and also strong water lines more characteristic of UltraLuminous IR Galaxies (ULIRGs). The 12CO line emission is modeled with a combination of a low-velocity C-shock and a PDR. Shocks are required to produce the H2O and most of the high-J 12CO emission. Despite having an infrared luminosity thirty times less than a typical ULIRG, the spectral characteristics and physical conditions of the ISM of NGC 1266 closely resemble those of ULIRGs, which often harbor strong shocks and large-scale outflows., Comment: Accepted for publication in ApJL 11/15/2013

We present new Herschel-SPIRE imaging spectroscopy (194-671 microns) of the bright starburst galaxy M82. Covering the CO ladder from J=4-3 to J=13-12, spectra were obtained at multiple positions for a fully sampled ~ 3 x 3 arcminute map, including a longer exposure at the central position. We present measurements of 12CO, 13CO, [CI], [NII], HCN, and HCO+ in emission, along with OH+, H2O+ and HF in absorption and H2O in both emission and absorption, with discussion. We use a radiative transfer code and Bayesian likelihood analysis to model the temperature, density, column density, and filling factor of multiple components of molecular gas traced by 12CO and 13CO, adding further evidence to the high-J lines tracing a much warmer (~ 500 K), less massive component than the low-J lines. The addition of 13CO (and [CI]) is new and indicates that [CI] may be tracing different gas than 12CO. No temperature/density gradients can be inferred from the map, indicating that the single-pointing spectrum is descriptive of the bulk properties of the galaxy. At such a high temperature, cooling is dominated by molecular hydrogen. Photon-dominated region (PDR) models require higher densities than those indicated by our Bayesian likelihood analysis in order to explain the high-J CO line ratios, though cosmic-ray enhanced PDR models can do a better job reproducing the emission at lower densities. Shocks and turbulent heating are likely required to explain the bright high-J emission., Comment: Accepted to The Astrophysical Journal. 17 pages + 20 pages of appendix figures

We present high-resolution maps of stars, dust, and molecular gas in a strongly lensed submillimeter galaxy (SMG) at z = 3.259. HATLAS12--00 is selected from the Herschel-Astrophysical Terahertz Large Area Survey (H-ATLAS) as a strong lens candidate mainly based on its unusually high 500um flux density (~300 mJy). It is the only high-redshift Planck detection in the 130 deg^2 H-ATLAS Phase 1 area. Keck Adaptive Optics images reveal a quadruply imaged galaxy in the K-band while the Submillimeter Array and the Extended Very Large Array show doubly imaged 880um and CO(1-0) sources, indicating differentiated distributions of the various components in the galaxy. In the source plane, the stars reside in three major kpc-scale clumps extended over ~1.6 kpc, the dust in a compact (~1 kpc) region ~3 kpc north of the stars, and the cold molecular gas in an extended (~7 kpc) disk ~5 kpc northeast of the stars. The emission from the stars, dust, and gas are magnified by ~17, 8, and 7 times, respectively, by four lensing galaxies at z ~ 1. Intrinsically, the lensed galaxy is a warm (T_dust ~ 40-65 K), hyper-luminous (L_IR ~ 1.7e13 Lsun; SFR ~ 2000 Msun/yr), gas-rich (M_gas/M_baryon ~ 70%), young (M_stellar/SFR ~ 20 Myr), and short-lived (M_gas/SFR ~ 40 Myr) starburst, without a significant active galactic nucleus. With physical properties similar to unlensed z > 2 SMGs, HATLAS12--00 offers a detailed view of a typical SMG through a powerful cosmic microscope., Comment: ApJ accepted version. Minor revisions. 12 pages, 4 figures, emulateapj style

The Sunyaev-Zel'dovich (SZ) effect spectrum crosses through a null where dT_CMB = 0 near nu_0 = 217 GHz. In a cluster of galaxies, nu_0 can be shifted from the canonical thermal SZ effect value by corrections to the SZ effect scattering due to the properties of the inter-cluster medium. We have measured the SZ effect in the hot galaxy cluster RX J1347.5-1145 with Z-Spec, an R ~ 300 grating spectrometer sensitive between 185 and 305 GHz. These data comprise a high spectral resolution measurement around the null of the SZ effect and clearly exhibit the transition from negative to positive dT_CMB over the Z-Spec band. The SZ null position is measured to be nu_0 = 225.8 \pm 2.5 (stat.) \pm 1.2 (sys.) GHz, which differs from the canonical null frequency by 3.0 sigma and is evidence for modifications to the canonical thermal SZ effect shape. Assuming the measured shift in nu_0 is due only to relativistic corrections to the SZ spectrum, we place the limit T_e = 17.1 \pm 5.3 keV from the zero-point measurement alone. By simulating the response of the instrument to the sky, we are able to generate likelihood functions in {y_0, T_e, v_pec} space. For v_pec = 0 km/s, we measure the best fitting SZ model to be y_0 = 4.6 (+0.6, -0.9) x 10^-4, T_e,0 = 15.2 (+12,-7.4) keV. When v_pec is allowed to vary, a most probable value of v_pec = +450 \pm 810 km/s is found., Comment: 15 pages, 12 figures, matches version published in April 2012 ApJ

Galactic cirrus emission at far-infrared wavelengths affects many extragalactic observations. Separating this emission from that associated with extragalactic objects is both important and difficult. In this paper we discuss a particular case, the M81 group, and the identification of diffuse structures prominent in the infrared, but also detected at optical wavelengths. The origin of these structures has previously been controversial, ranging from them being the result of a past interaction between M81 and M82 or due to more local Galactic emission. We show that over of order a few arcminute scales the far-infrared (Herschel 250 &\mu&m) emission correlates spatially very well with a particular narrow velocity (2-3 km/s) component of the Galactic HI. We find no evidence that any of the far-infrared emission associated with these features actually originates in the M81 group. Thus we infer that the associated diffuse optical emission must be due to galactic light back scattered off dust in our galaxy. Ultra-violet observations pick out young stellar associations around M81, but no detectable far-infrared emission. We consider in detail one of the Galactic cirrus features, finding that the far-infrared HI relation breaks down below arc minute scales and that at smaller scales there can be quite large dust temperature variations., Comment: 6 pages, 4 figures, accepted for publication in MNRAS

By combining Herschel-SPIRE data with archival Spitzer, HI, and CO maps, we investigate the spatial distribution of gas and dust in the two famous grand-design spirals M99 and M100 in the Virgo cluster. Thanks to the unique resolution and sensitivity of the Herschel-SPIRE photometer, we are for the first time able to measure the distribution and extent of cool, submillimetre (submm)-emitting dust inside and beyond the optical radius. We compare this with the radial variation in both the gas mass and the metallicity. Although we adopt a model-independent, phenomenological approach, our analysis provides important insights. We find the dust extending to at least the optical radius of the galaxy and showing breaks in its radial profiles at similar positions as the stellar distribution. The colour indices f350/f500 and f250/f350 decrease radially consistent with the temperature decreasing with radius. We also find evidence of an increasing gas to dust ratio with radius in the outer regions of both galaxies., Comment: LaTeX, 5 pages, 5 figures, Astronomy and Astrophysics, Herschel Special Issue, in press as a Letter

We present Herschel-SPIRE observations of the perturbed galaxy NGC4438 in the Virgo cluster. These images reveal the presence of extra-planar dust up to ~4-5 kpc away from the galaxy's disk. The dust closely follows the distribution of the stripped atomic and molecular hydrogen, supporting the idea that gas and dust are perturbed in a similar fashion by the cluster environment. Interestingly, the extra-planar dust lacks a warm temperature component when compared to the material still present in the disk, explaining why it was missed by previous far-infrared investigations. Our study provides evidence for dust stripping in clusters of galaxies and illustrates the potential of Herschel data for our understanding of environmental effects on galaxy evolution., Comment: 5 pages, 2 figures. Accepted for publication on the A&A Herschel Special Issue

We present Herschel SPIRE and PACS photometeric observations of the low metallicity (Z ~ 0.35 solar) nearby dwarf galaxy, NGC 1705, in six wavelength bands as part of the Dwarf Galaxy Survey guaranteed time Herschel Key Program. We confirm the presence of two dominant circumnuclear IR-bright regions surrounding the central super star cluster that had been previously noted at mid-IR wavelengths and in the sub-mm by LABOCA. On constructing a global spectral energy distribution using the SPIRE and PACS photometry, in conjunction with archival IR measurements, we note the presence of an excess at sub-mm wavelengths. This excess suggests the presence of a significant cold dust component within NGC 1705 and was modeled as an additional cold component in the SED. Although alternative explanations for the sub-mm excess beyond 350 microns, such as changes to the dust emissivity cannot be ruled out, the most likely explanation for the observed submillimetre excess is that of an additional cold dust component., Comment: 5 pages, 2 figures, accepted for publication in A&A (Herschel special issue)

The standard method of mapping the interstellar medium in a galaxy, by observing the molecular gas in the CO 1-0 line and the atomic gas in the 21-cm line, is largely limited with current telescopes to galaxies in the nearby universe. In this letter, we use SPIRE observations of the galaxies M99 and M100 to explore the alternative approach of mapping the interstellar medium using the continuum emission from the dust. We have compared the methods by measuring the relationship between the star-formation rate and the surface density of gas in the galaxies. We find the two methods give relationships with a similar dispersion, confirming that observing the continuum emission from the dust is a promising method of mapping the interstellar medium in galaxies., Comment: Astronomy and Astrophysics in press

We present the first Herschel PACS and SPIRE images of the low-metallicity galaxy NGC6822 observed from 70 to 500 mu and clearly resolve the HII regions with PACS and SPIRE. We find that the ratio 250/500 is dependent on the 24 mu surface brightness in NGC6822, which would locally link the heating processes of the coldest phases of dust in the ISM to the star formation activity. We model the SEDs of some regions HII regions and less active regions across the galaxy and find that the SEDs of HII regions show warmer ranges of dust temperatures. We derive very high dust masses when graphite is used in our model to describe carbon dust. Using amorphous carbon, instead, requires less dust mass to account for submm emission due to its lower emissivity properties. This indicates that SED models including Herschel constraints may require different dust properties than commonly used., Comment: Accepted for publication in the A&A Herschel first results Special Issue

With appropriate spatial resolution, images of spiral galaxies in thermal infrared (~10 micron and beyond) often reveal a bright central component, distinct from the stellar bulge, superimposed on a disk with prominent spiral arms. ISO and Spitzer studies have shown that much of the scatter in the mid-infrared colors of spiral galaxies is related to changes in the relative importance of these two components, rather than to other modifications, such as the morphological type or star formation rate, that affect the properties of the galaxy as a whole. With the Herschel imaging capability from 70 to 500 micron, we revisit this two-component approach at longer wavelengths, to see if it still provides a working description of the brightness distribution of galaxies, and to determine its implications on the interpretation of global far-infrared properties of galaxies., Comment: 5 pages, 2 figures, accepted for the A&A Herschel Special Issue

We use Herschel Space Observatory data to place observational constraints on the peak and Rayleigh-Jeans slope of dust emission observed at 70-500 microns in the nearby spiral galaxy M81. We find that the ratios of wave bands between 160 and 500 microns are primarily dependent on radius but that the ratio of 70 to 160 micron emission shows no clear dependence on surface brightness or radius. These results along with analyses of the spectral energy distributions imply that the 160-500 micron emission traces 15-30 K dust heated by evolved stars in the bulge and disc whereas the 70 micron emission includes dust heated by the active galactic nucleus and young stars in star forming regions., Comment: Accepted for A & A Herschel special issue