Search

Run to run variability in parallel programs caused by floating-point non-associativity has been known to significantly affect reproducibility in iterative algorithms, due to accumulating errors. Non-reproducibility can critically affect the efficiency and effectiveness of correctness testing for stochastic programs. Recently, the sensitivity of deep learning training and inference pipelines to floating-point non-associativity has been found to sometimes be extreme. It can prevent certification for commercial applications, accurate assessment of robustness and sensitivity, and bug detection. New approaches in scientific computing applications have coupled deep learning models with high-performance computing, leading to an aggravation of debugging and testing challenges. Here we perform an investigation of the statistical properties of floating-point non-associativity within modern parallel programming models, and analyze performance and productivity impacts of replacing atomic operations with deterministic alternatives on GPUs. We examine the recently-added deterministic options in PyTorch within the context of GPU deployment for deep learning, uncovering and quantifying the impacts of input parameters triggering run to run variability and reporting on the reliability and completeness of the documentation. Finally, we evaluate the strategy of exploiting automatic determinism that could be provided by deterministic hardware, using the Groq accelerator for inference portions of the deep learning pipeline. We demonstrate the benefits that a hardware-based strategy can provide within reproducibility and correctness efforts.

Scientific workflows have become integral tools in broad scientific computing use cases. Science discovery is increasingly dependent on workflows to orchestrate large and complex scientific experiments that range from execution of a cloud-based data preprocessing pipeline to multi-facility instrument-to-edge-to-HPC computational workflows. Given the changing landscape of scientific computing and the evolving needs of emerging scientific applications, it is paramount that the development of novel scientific workflows and system functionalities seek to increase the efficiency, resilience, and pervasiveness of existing systems and applications. Specifically, the proliferation of machine learning/artificial intelligence (ML/AI) workflows, need for processing large scale datasets produced by instruments at the edge, intensification of near real-time data processing, support for long-term experiment campaigns, and emergence of quantum computing as an adjunct to HPC, have significantly changed the functional and operational requirements of workflow systems. Workflow systems now need to, for example, support data streams from the edge-to-cloud-to-HPC enable the management of many small-sized files, allow data reduction while ensuring high accuracy, orchestrate distributed services (workflows, instruments, data movement, provenance, publication, etc.) across computing and user facilities, among others. Further, to accelerate science, it is also necessary that these systems implement specifications/standards and APIs for seamless (horizontal and vertical) integration between systems and applications, as well as enabling the publication of workflows and their associated products according to the FAIR principles. This document reports on discussions and findings from the 2022 international edition of the Workflows Community Summit that took place on November 29 and 30, 2022.

This paper assesses and reports the experience of ten teams working to port,validate, and benchmark several High Performance Computing applications on a novel GPU-accelerated Arm testbed system. The testbed consists of eight NVIDIA Arm HPC Developer Kit systems built by GIGABYTE, each one equipped with a server-class Arm CPU from Ampere Computing and A100 data center GPU from NVIDIA Corp. The systems are connected together using Infiniband high-bandwidth low-latency interconnect. The selected applications and mini-apps are written using several programming languages and use multiple accelerator-based programming models for GPUs such as CUDA, OpenACC, and OpenMP offloading. Working on application porting requires a robust and easy-to-access programming environment, including a variety of compilers and optimized scientific libraries. The goal of this work is to evaluate platform readiness and assess the effort required from developers to deploy well-established scientific workloads on current and future generation Arm-based GPU-accelerated HPC systems. The reported case studies demonstrate that the current level of maturity and diversity of software and tools is already adequate for large-scale production deployments.

The extraction of nuclear charge radii from spectroscopy experiments in muonic atoms is currently limited by the large uncertainties associated with the theoretical evaluation of the nuclear polarizability effects. To facilitate calculations, these polarizability corrections are conventionally expressed as an expansion in a dimensionless parameter $\eta$, which has been argued in previous literature to hold an approximate value of 0.33 in light-nuclear systems. In this work, we check this claim by doing a Bayesian analysis of the nuclear-polarizability corrections to the Lamb shift in $\mu^2$H and $\mu^3$H atoms and in $\mu^3$He$^+$ and $\mu^4$He$^+$ ions at various orders in the $\eta$-expansion. Our analysis supports the claim that $\eta \ll 1$ in these systems and finds truncation uncertainties that are similar to the previous estimate, the only exception being the truncation uncertainties in the $\mu^3$He$^+$ ion, which are found to be larger., Comment: Published in JPG

In previous works we discussed the 21 cm signature of a single cosmic string wake. However the 21 cm brightness temperature is influenced by a network of cosmic string wakes, and not one single wake. In this work we consider the signal from a network of wakes laid down during the matter era. We also improve on the previous calculation of a single wake signature. Finally we calculate the enhancement of the global 21 cm brightness temperature due to a network of wakes and discuss its affects of the signal measured in the Wouthuysen-Field absorption trough. We estimated that for string tensions between $10^{-8}$ to $10^{-7}$ there would be between a 10% to a factor 2 enhancement in the signal., Comment: 6 pages, 4 figure. v2 to v3: minor typo corrected in references. v1 to v2: explanation in section 5 expanded, figures enlarged, NANOGrav data description changed in section 2, 1 references added in section 2. Matches the version accepted for publication by MNRAS

This paper presents a methodology for using LLVM-based tools to tune the DCA++ (dynamical clusterapproximation) application that targets the new ARM A64FX processor. The goal is to describethe changes required for the new architecture and generate efficient single instruction/multiple data(SIMD) instructions that target the new Scalable Vector Extension instruction set. During manualtuning, the authors used the LLVM tools to improve code parallelization by using OpenMP SIMD,refactored the code and applied transformation that enabled SIMD optimizations, and ensured thatthe correct libraries were used to achieve optimal performance. By applying these code changes, codespeed was increased by 1.98X and 78 GFlops were achieved on the A64FX processor. The authorsaim to automatize parts of the efforts in the OpenMP Advisor tool, which is built on top of existingand newly introduced LLVM tooling.

Scientific applications that run on leadership computing facilities often face the challenge of being unable to fit leading science cases onto accelerator devices due to memory constraints (memory-bound applications). In this work, the authors studied one such US Department of Energy mission-critical condensed matter physics application, Dynamical Cluster Approximation (DCA++), and this paper discusses how device memory-bound challenges were successfully reduced by proposing an effective "all-to-all" communication method -- a ring communication algorithm. This implementation takes advantage of acceleration on GPUs and remote direct memory access (RDMA) for fast data exchange between GPUs. Additionally, the ring algorithm was optimized with sub-ring communicators and multi-threaded support to further reduce communication overhead and expose more concurrency, respectively. The computation and communication were also analyzed by using the Autonomic Performance Environment for Exascale (APEX) profiling tool, and this paper further discusses the performance trade-off for the ring algorithm implementation. The memory analysis on the ring algorithm shows that the allocation size for the authors' most memory-intensive data structure per GPU is now reduced to 1/p of the original size, where p is the number of GPUs in the ring communicator. The communication analysis suggests that the distributed Quantum Monte Carlo execution time grows linearly as sub-ring size increases, and the cost of messages passing through the network interface connector could be a limiting factor.

Task graphs have been studied for decades as a foundation for scheduling irregular parallel applications and incorporated in programming models such as OpenMP. While many high-performance parallel libraries are based on task graphs, they also have additional scheduling requirements, such as synchronization from inner levels of data parallelism and internal blocking communications. In this paper, we extend task-graph scheduling to support efficient synchronization and communication within tasks. Our scheduler avoids deadlock and oversubscription of worker threads, and refines victim selection to increase the overlap of sibling tasks. To the best of our knowledge, our approach is the first to combine gang-scheduling and work-stealing in a single runtime. Our approach has been evaluated on the SLATE highperformance linear algebra library. Relative to the LLVM OMP runtime, our runtime demonstrates performance improvements of up to 13.82%, 15.2%, and 36.94% for LU, QR, and Cholesky, respectively, evaluated across different configurations.

This paper describes how we successfully used the HPX programming model to port the DCA++ application on multiple architectures that include POWER9, x86, ARM v8, and NVIDIA GPUs. We describe the lessons we can learn from this experience as well as the benefits of enabling the HPX in the application to improve the CPU threading part of the code, which led to an overall 21% improvement across architectures. We also describe how we used HPX-APEX to raise the level of abstraction to understand performance issues and to identify tasking optimization opportunities in the code, and how these relate to CPU/GPU utilization counters, device memory allocation over time, and CPU kernel-level context switches on a given architecture.

Protein-ligand docking is an in silico tool used to screen potential drug compounds for their ability to bind to a given protein receptor within a drug-discovery campaign. Experimental drug screening is expensive and time consuming, and it is desirable to carry out large scale docking calculations in a high-throughput manner to narrow the experimental search space. Few of the existing computational docking tools were designed with high performance computing in mind. Therefore, optimizations to maximize use of high-performance computational resources available at leadership-class computing facilities enables these facilities to be leveraged for drug discovery. Here we present the porting, optimization, and validation of the AutoDock-GPU program for the Summit supercomputer, and its application to initial compound screening efforts to target proteins of the SARS-CoV-2 virus responsible for the current COVID-19 pandemic.

We use a convolutional neural network (CNN) to study cosmic string detection in cosmic microwave background (CMB) flat sky maps with Nambu-Goto strings. On noiseless maps we can measure string tensions down to order $10^{-9}$, however when noise is included we are unable to measure string tensions below $10^{-7}$. Motivated by this impasse, we derive an information theoretic bound on the detection of the cosmic string tension $G\mu$ from CMB maps. In particular we bound the information entropy of the posterior distribution of $G\mu$ in terms of the resolution, noise level and total survey area of the CMB map. We evaluate these bounds for the ACT, SPT-3G, Simons Observatory, Cosmic Origins Explorer, and CMB-S4 experiments. These bounds cannot be saturated by any method., Comment: v3: corrected error in table 1 entry for the CORE experiment. This matches the erratum submitted for the version published in MNRAS. 7 pages, 1 figure, 1 table

Recent calculations of nuclear structure corrections to the Lamb shift in light muonic atoms are based on an expansion in a parameter eta, where only terms up to second order are retained. The parameter eta can be shown to be proportional to the square root of the muon/proton mass ratio, so that it is small and the expansion is expected to converge. However, practical implementations show that the eta convergence may be slower than expected. In this work we probe the uncertainties due to this expansion using a different formalism, which is based on a multipole expansion of the longitudinal and transverse response functions and was first introduced by Leidemann and Rosenfelder. We refer to this alternative expansion as the eta-less formalism. We generalize this formalism to account for the cancellation of elastic terms such as the third Zemach moment (or Friar moment) and embed it in a computationally efficient framework. We implement and test this approach in the case of muonic deuterium. The comparison of results in the point nucleon limit for both methods achieve sub-percent agreement. When nucleon form factors are introduced we find a 4% and 2% difference in the third Zemach moment and nuclear polarizability, respectively, compared to the eta-less expansion, indicating that the nucleon form factor approximations should be improved. However, we find that the sum of these terms removes this dependence and the uncertainty due to the eta-expansion and the related second-order approximation in the nucleon form factors amounts only to 0.2% and thus is fully justified in muonic deuterium. This computationally efficient framework paves the way to further studies in light muonic systems with more than two nucleons, where controlling and reducing uncertainties in nuclear structure corrections is key to the experimental efforts of the CREMA collaboration., Comment: 13 pages, 4 figures, 6 tables

The charge radius of the deuteron (D), was recently determined to three times the precision compared with previous measurements using the measured Lamb shift in muonic deuterium (muD). However, the muD value is 5.6 $\sigma$ smaller than the world averaged CODATA-2014 value [1]. To shed light on this discrepancy we analyze the uncertainties of the nuclear structure calculations of the Lamb shift in muD and conclude that nuclear theory uncertainty is not likely to be the source of the discrepancy., Comment: 4 pages, 1 figure, accepted proceedings of the "XXII International Conference on Few-Body Problems in Physics", 9-13 July 2018

In previous work, we constructed a convolutional neural network used to estimate the location of cosmic strings in simulated cosmic microwave background temperature anisotropy maps. We derived a connection between the estimates of cosmic string locations by this neural network and the posterior probability distribution of the cosmic string tension $G\mu$. Here, we significantly improve the calculation of the posterior distribution of the string tension $G\mu$. We also improve our previous plain convolutional neural network by using residual networks. We apply our new neural network and posterior calculation method to maps from the same simulation used in our previous work and quantify the improvement., Comment: 10 pages, 7 figures. v2->v3: minor typos corrected and formatting adjusted to more closely match published version. v1->v2: We have expanded the Introduction and the References, and changed the formatting to MNRAS style

We present the first evidence of excitation of the $5p_{3/2} \rightarrow 6p_{1/2}$ electric dipole-forbidden transition in atomic rubidium. The experiments were carried out in a rubidium vapor cell using Doppler-free optical-optical double-resonance spectroscopy with counter-propagating beams. This a similar scheme to the excitation of the other fine transition $J=3/2$ in the $6p_{J}$ manifold previously reported. A $5s_{1/2} \rightarrow 5p_{3/2}$ electric dipole preparation step using a diode laser locked to the maximum F cyclic transition of the $D_2$ line is used to prepare the atoms in the first excited state. This is followed by the $5p_{3/2} \rightarrow 6p_{1/2}$ dipole-forbidden excitation, in a two-photon ladder ($\Xi$) excitation scheme. Production of atoms in the $6p_{1/2}$ excited state is verified by detection of the $421$ nm fluorescence that results from direct decay into the $5s_{1/2}$ ground state. The polarization dependence of the relative intensities of the lines of the decay fluorescence is also investigated. Experimental data for different polarization configurations of the light beams are compared with the results of calculations that consider a strong atom-field coupling in the preparation step, followed by a weak electric quadrupole excitation and the blue fluorescence decay emission. Good agreement between experiment and this three-step model is found in the case of linear-linear polarizations in both isotopes., Comment: 11 Pages, 5 figures

The measurement of the Lamb shift in muonic hydrogen and the subsequent emergence of the proton-radius puzzle have motivated an experimental campaign devoted to measuring the Lamb shift in other light muonic atoms, such as muonic deuterium and helium. For these systems it has been shown that two-photon exchange nuclear structure corrections are the largest source of uncertainty and consequently the bottleneck for exploiting the experimental precision to extract the nuclear charge radius. Utilizing techniques and methods developed to study electromagnetic reactions in light nuclei, recent calculations of nuclear structure corrections to the muonic Lamb shift have reached unprecedented precision, reducing the uncertainty with respect to previous estimates by a factor of 5 in certain cases. These results will be useful for shedding light on the nature of the proton-radius puzzle and other open questions pertaining to it. Here, we review and update calculations for muonic deuterium and tritium atoms, and for muonic helium-3 and helium-4 ions. We present a thorough derivation of the formalism and discuss the results in relation to other approaches where available. We also describe how to assess theoretical uncertainties, for which the language of chiral effective field theory furnishes a systematic approach that could be further exploited in the future., Comment: 62 pages, 15 figures, 8 tables

The discovery of the proton-radius puzzle and the subsequent deuteron-radius puzzle is fueling an on-going debate on possible explanations for the difference in the observed radii obtained from muonic atoms and from electron-nucleus systems. Atomic nuclei have a complex internal structure that must be taken into account when analyzing experimental spectroscopic results. Ab initio nuclear structure theory provided the so far most precise estimates of important corrections to the Lamb shift in muonic atoms and is well poised to also investigate nuclear structure corrections to the hyperfine splitting in muonic atoms. Independently on whether the puzzle is due to beyond-the-standard-model physics or not, nuclear structure corrections are a necessary theoretical input to any experimental extraction of electric and magnetic radii from precise muonic atom measurements. Here, we review the status of the calculations performed by the TRIUMF-Hebrew University group, focusing on the deuteron, and discuss preliminary results on magnetic sum rules calculated with two-body currents at next-to-leading order. Two-body currents will be an important ingredient in future calculations of nuclear structure corrections to the hyperfine splitting in muonic atoms., Comment: 10 pages, accepted proceedings of the "55th International Winter Meeting on Nuclear Physics", 23-27 January 2017, to appear on PoS

With the advent of more and deeper sky surveys, the discovery of interstellar small objects entering into the Solar System has been finally possible. In October 19, 2017, using observations of the Pan-STARRS survey, a fast moving object, now officially named 1I/2017 U1 ('Oumuamua), was discovered in a heliocentric unbound trajectory suggesting an interstellar origin. Assessing the provenance of interstellar small objects is key for understanding their distribution, spatial density and the processes responsible for their ejection from planetary systems. However, their peculiar trajectories place a limit on the number of observations available to determine a precise orbit. As a result, when its position is propagated $\sim 10^5-10^6$ years backward in time, small errors in orbital elements become large uncertainties in position in the interstellar space. In this paper we present a general method for assigning probabilities to nearby stars of being the parent system of an observed interstellar object. We describe the method in detail and apply it for assessing the origin of 'Oumuamua. A preliminary list of potential progenitors and their corresponding probabilities is provided. In the future, when further information about the object and/or the nearby stars be refined, the probabilities computed with our method can be updated. We provide all the data and codes we developed for this purpose in the form of an open source {\tt C/C++/Python package}, {\bf\tt iWander} which is publicly available at http://github.com/seap-udea/iWander., Comment: 16 pages, 8 figures. Accepted for publication in the Astronomical Journal. iWander package available at http://github.com/seap-udea/iWander

To shed light on the deuteron radius puzzle we analyze the theoretical uncertainties of the nuclear structure corrections to the Lamb shift in muonic deuterium. We find that the discrepancy between the calculated two-photon exchange correction and the corresponding experimentally inferred value by Pohl et al. [1] remain. The present result is consistent with our previous estimate, although the discrepancy is reduced from 2.6 $\sigma$ to 2 $\sigma$. The error analysis includes statistic as well as systematic uncertainties stemming from the use of nucleon-nucleon interactions derived from chiral effective field theory at various orders. We therefore conclude that nuclear theory uncertainty is more likely not the source of the discrepancy., Comment: updated according to referee's comments

We present in detail the convolutional neural network used in our previous work to detect cosmic strings in cosmic microwave background (CMB) temperature anisotropy maps. By training this neural network on numerically generated CMB temperature maps, with and without cosmic strings, the network can produce prediction maps that locate the position of the cosmic strings and provide a probabilistic estimate of the value of the string tension $G\mu$. Supplying noiseless simulations of CMB maps with arcmin resolution to the network resulted in the accurate determination both of string locations and string tension for sky maps having strings with string tension as low as $G\mu=5\times10^{-9}$. The code is publicly available online. Though we trained the network with a long straight string toy model, we show the network performs well with realistic Nambu-Goto simulations., Comment: 8 pages, 2 figures. v2 to 3: Introduction shortened and formatting adjustments to more closely match published version. v1 to v2: Cleaned up confusing notation and writing. Added more details and examples to clarify our presentation. Announced the public availability of the code online. Changed the formatting of the paper

There exists various proposals to detect cosmic strings from Cosmic Microwave Background (CMB) or 21 cm temperature maps. Current proposals do not aim to find the location of strings on sky maps, all of these approaches can be thought of as a statistic on a sky map. We propose a Bayesian interpretation of cosmic string detection and within that framework, we derive a connection between estimates of cosmic string locations and cosmic string tension $G\mu$. We use this Bayesian framework to develop a machine learning framework for detecting strings from sky maps and outline how to implement this framework with neural networks. The neural network we trained was able to detect and locate cosmic strings on noiseless CMB temperature map down to a string tension of $G\mu=5 \times10^{-9}$ and when analyzing a CMB temperature map that does not contain strings, the neural network gives a 0.95 probability that $G\mu\leq2.3\times10^{-9}$., Comment: 21pp, 24 figs. agrees w/ published version. v3->v4: added arXiv no. to ref 36. v2->v3: Corrected typos. Expanded explanations of Eq 2.4, \delta_{string}, Eq 3.3, 3.4, KL divergence. Added Fig1b. Relabled Tables1,2 to Tables1a,1b. Fig7,8 changed to log-log scale. v1->v2: Corrected scale in fig 1. Added size and resolution to Fig 1-6, a description of our prior on G\mu, references