Search

Since the advent of parallel algorithms in the C++17 Standard Template Library (STL), the STL has become a viable framework for creating performance-portable applications. Given multiple existing implementations of the parallel algorithms, a systematic, quantitative performance comparison is essential for choosing the appropriate implementation for a particular hardware configuration. In this work, we introduce a specialized set of micro-benchmarks to assess the scalability of the parallel algorithms in the STL. By selecting different backends, our micro-benchmarks can be used on multi-core systems and GPUs. Using the suite, in a case study on AMD and Intel CPUs and NVIDIA GPUs, we were able to identify substantial performance disparities among different implementations, including GCC+TBB, GCC+HPX, Intel's compiler with TBB, or NVIDIA's compiler with OpenMP and CUDA., Comment: 15 pages, 24 figures, 4 tables

The controlled creation of quantum emitters in diamond represents a major research effort in the fabrication of single-photon devices. Here, we present the scalable production of silicon-vacancy (SiV) color centers in single-crystal diamond by ion implantation. The lateral position of the SiV is spatially controlled by a 1-$\mu$m pinhole placed in front of the sample, which can be moved nanometer precise using a piezo stage. The initial implantation position is controlled by monitoring the ion beam position with a camera. Hereby, silicon ions are implanted at the desired spots in an area comparable to the diffraction limit. We discuss the role of ions scattered by the pinhole and the activation yield of the SiV color centers for the creation of single quantum emitters., Comment: 11 pages, 4 figures

We investigate the optical properties of polycrystalline diamond membranes containing silicon-vacancy (SiV) color centers in combination with other nano-analytical techniques. We analyze the correlation between the Raman signal, the SiV emission, and the background luminescence in the crystalline grains and in the grain boundaries, identifying conditions for the addressability of single SiV centers. Moreover, we perform a scanning transmission electron microscopy (STEM) analysis, which associates the microscopic structure of the membranes and the evolution of the diamond crystal along the growth direction with the photoluminescence properties, as well as a time-of-flight secondary ion mass spectrometry (ToF-SIMS) to address the distribution of silicon in implanted and un-implanted membranes. The results of the STEM and ToF-SIMS studies are consistent with the outcome of the optical measurements and provide useful insight into the preparation of polycrystalline samples for quantum nano-optics., Comment: 21 pages, 8 figures

Good process-to-compute-node mappings can be decisive for well performing HPC applications. A special, important class of process-to-node mapping problems is the problem of mapping processes that communicate in a sparse stencil pattern to Cartesian grids. By thoroughly exploiting the inherently present structure in this type of problem, we devise three novel distributed algorithms that are able to handle arbitrary stencil communication patterns effectively. We analyze the expected performance of our algorithms based on an abstract model of inter- and intra-node communication. An extensive experimental evaluation on several HPC machines shows that our algorithms are up to two orders of magnitude faster in running time than a (sequential) high-quality general graph mapping tool, while obtaining similar results in communication performance. Furthermore, our algorithms also achieve significantly better mapping quality compared to previous state-of-the-art Cartesian grid mapping algorithms. This results in up to a threefold performance improvement of an MPI_Neighbor_alltoall exchange operation. Our new algorithms can be used to implement the MPI_Cart_create functionality., Comment: 18 pages, 9 Figures

We introduce the Scheduling.jl Julia package, which is intended for collaboratively conducting scheduling research and for sharing implementations of algorithms. It provides the fundamental building blocks for implementing scheduling algorithms following the three-field notation of Graham et al., i.e., it has functionality to describe machine environments, job characteristics, and optimality criteria. Our goal is to foster algorithm and code sharing in the scheduling community. Scheduling.jl can also be used to support teaching scheduling theory in classes. We will show the main functionalities of Scheduling.jl and give an example on how to use it by comparing different algorithms for the problem of P||Cmax ., Comment: 5 pages, 1 figure, 2 listings

Magnetic fields associated with currents flowing in tissue can be measured non-invasively by means of zero-field-encoded ultra-low-field magnetic resonance imaging (ULF MRI) enabling current density imaging (CDI) and possibly conductivity mapping of human head tissues. Since currents applied to a human are limited by safety regulations and only a small fraction of the current passes through the relatively high-resistive skull, a sufficient signal-to-noise ratio (SNR) may be difficult to obtain when using this method. In this work, we study the relationship between the image SNR and the SNR of the field reconstructions from zero-field-encoded data. We evaluate these results for two existing ULF MRI scanners, one ultra-sensitive single-channel system and one whole-head multi-channel system, by simulating sequences necessary for current-density reconstruction. We also derive realistic current-density and magnetic-field estimates from finite-element-method simulations based on a three-compartment head model. We found that existing ULF-MRI systems reach sufficient SNR to detect intra-cranial current distributions with statistical uncertainty below 10%. However, they also reveal that image artifacts influence the reconstruction quality. Further, our simulations indicate that current-density reconstruction in the scalp requires a resolution less than 5 mm and demonstrate that the necessary sensitivity coverage can be accomplished by multi-channel devices., Comment: 18 pages, 8 figures. This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 686865

Magnetoencephalography (MEG) and Electroencephalography (EEG) source estimates have thus far mostly been derived sample by sample, i.e., independent of each other in time. However, neuronal assemblies are heavily interconnected, constraining the temporal evolution of neural activity in space as detected by MEG and EEG. The observed neural currents are thus highly context dependent. Here, a new method is presented which integrates predictive deep learning networks with the Minimum-Norm Estimates (MNE) approach. Specifically, we employ Long Short-Term Memory (LSTM) networks, a type of recurrent neural network, for predicting brain activity. Because we use past activity (context) in the estimation, we call our method Contextual MNE (CMNE). We demonstrate that these contextual algorithms can be used for predicting activity based on previous brain states and when used in conjunction with MNE, they lead to more accurate source estimation. To evaluate the performance of CMNE, it was tested on simulated and experimental data from human auditory evoked response experiments., Comment: 14 pages, 9 figures

Physical head phantoms allow assessing source reconstruction procedures in electroencephalography and electrical stimulation profiles during transcranial electric stimulation. Volume conduction in the head is strongly influenced by the skull representing the main conductivity barrier. Realistic modeling of its characteristics is thus important for phantom development. In the present study, we proposed plastic clay as a material for modeling the skull in phantoms. We analyzed five clay types varying in granularity and fractions of fireclay, each with firing temperatures from 550 {\deg}C to 950 {\deg}C. We investigated the conductivity of standardized clay samples when immersed in a 0.9% sodium chloride solution with time-resolved four-point impedance measurements. To test the reusability of the clay model, these measurements were repeated after cleaning the samples by rinsing in deionized water for 5 h. We found time-dependent impedance changes for approximately 5 min after immersion in the solution. Thereafter, the conductivities stabilized between 0.0716 S/m and 0.0224 S/m depending on clay type and firing temperatures. The reproducibility of the measurement results proved the effectiveness of the rinsing procedure. Clay provides formability, is permeable for ions, can be adjusted in conductivity value and is thus suitable for the skull modeling in phantoms., Comment: 20 pages, 5 figures, 1 table

MPI collective operations provide a standardized interface for performing data movements within a group of processes. The efficiency of collective communication operations depends on the actual algorithm, its implementation, and the specific communication problem (type of communication, message size, number of processes). Many MPI libraries provide numerous algorithms for specific collective operations. The strategy for selecting an efficient algorithm is often times predefined (hard-coded) in MPI libraries, but some of them, such as Open MPI, allow users to change the algorithm manually. Finding the best algorithm for each case is a hard problem, and several approaches to tune these algorithmic parameters have been proposed. We use an orthogonal approach to the parameter-tuning of MPI collectives, that is, instead of testing individual algorithmic choices provided by an MPI library, we compare the latency of a specific MPI collective operation to the latency of semantically equivalent functions, which we call the mock-up implementations. The structure of the mock-up implementations is defined by self-consistent performance guidelines. The advantage of this approach is that tuning using mock-up implementations is always possible, whether or not an MPI library allows users to select a specific algorithm at run-time. We implement this concept in a library called PGMPITuneLib, which is layered between the user code and the actual MPI implementation. This library selects the best-performing algorithmic pattern of an MPI collective by intercepting MPI calls and redirecting them to our mock-up implementations. Experimental results show that PGMPITuneLib can significantly reduce the latency of MPI collectives, and also equally important, that it can help identifying the tuning potential of MPI libraries., Comment: 16 pages, 14 figures

We examine natural expectations on communication performance using MPI derived datatypes in comparison to the baseline, "raw" performance of communicating simple, non-contiguous data layouts. We show that common MPI libraries sometimes violate these datatype performance expectations, and discuss reasons why this happens, but also show cases where MPI libraries perform well. Our findings are in many ways surprising and disappointing. First, the performance of derived datatypes is sometimes worse than the semantically equivalent packing and unpacking using the corresponding MPI functionality. Second, the communication performance equivalence stated in the MPI standard between a single contiguous datatype and the repetition of its constituent datatype does not hold universally. Third, the heuristics that are typically employed by MPI libraries at type-commit time are insufficient to enforce natural performance guidelines, and better type normalization heuristics may have a significant performance impact. We show cases where all the MPI type constructors are necessary to achieve the expected performance for certain data layouts. We describe our benchmarking approach to verify the datatype performance guidelines, and present extensive verification results for different MPI libraries., Comment: 46 pages, 107 figures

Isomorphic (sparse) collective communication is a form of collective communication in which all involved processes communicate in small, identically structured neighborhoods of other processes. Isomorphic neighborhoods are defined via an embedding of the processes in a regularly structured topology, e.g., $d$-dimensional torus, which may correspond to the physical communication network of the underlying system. Isomorphic collective communication is useful for implementing stencil and other regular, sparse distributed computations, where the assumption that all processes behave (almost) symmetrically is justified. In this paper, we show how efficient message-combining communication schedules for isomorphic, sparse collective communication can easily and efficiently be computed by purely local computations. We give schemes for \emph{isomorphic \alltoall} and \emph{\allgather} communication that reduce the number of communication rounds and thereby the communication latency from $s$ to at most $Nd$, for neighborhoods consisting of $s$ processes with the (small) factor $N$ depending on the structure of the neighborhood and the capabilities of the communication system. Using these schedules, we give \emph{zero-copy implementations} of the isomorphic collectives using MPI and its derived datatypes to eliminate explicit, process-local copy operations. By benchmarking the collective communication algorithms against straightforward implementations and against the corresponding MPI neighborhood collectives, we document significant latency improvements of our implementations for block sizes of up to a few kilobytes. We discuss further optimizations for computing even better schedules, some of which have been implemented and benchmarked.

The Message Passing Interface (MPI) is the most commonly used application programming interface for process communication on current large-scale parallel systems. Due to the scale and complexity of modern parallel architectures, it is becoming increasingly difficult to optimize MPI libraries, as many factors can influence the communication performance. To assist MPI developers and users, we propose an automatic way to check whether MPI libraries respect self-consistent performance guidelines for collective communication operations. We introduce the PGMPI framework to detect violations of performance guidelines through benchmarking. Our experimental results show that PGMPI can pinpoint undesired and often unexpected performance degradations of collective MPI operations. We demonstrate how to overcome performance issues of several libraries by adapting the algorithmic implementations of their respective collective MPI calls., Comment: 14 pages, 4 figures, 13 tables

We summarize the results of a survey on reproducibility in parallel computing, which was conducted during the Euro-Par conference in August 2015. The survey form was handed out to all participants of the conference and the workshops. The questionnaire, which specifically targeted the parallel computing community, contained questions in four different categories: general questions on reproducibility, the current state of reproducibility, the reproducibility of the participants' own papers, and questions about the participants' familiarity with tools, software, or open-source software licenses used for reproducible research., Comment: 15 pages, 24 figures

The Message Passing Interface (MPI) is the prevalent programming model used on today's supercomputers. Therefore, MPI library developers are looking for the best possible performance (shortest run-time) of individual MPI functions across many different supercomputer architectures. Several MPI benchmark suites have been developed to assess the performance of MPI implementations. Unfortunately, the outcome of these benchmarks is often neither reproducible nor statistically sound. To overcome these issues, we show which experimental factors have an impact on the run-time of blocking collective MPI operations and how to control them. We address the problem of process and clock synchronization in MPI benchmarks. Finally, we present a new experimental method that allows us to obtain reproducible and statistically sound MPI measurements., Comment: 38 pages, 44 figures

Computer science is also an experimental science. This is particularly the case for parallel computing, which is in a total state of flux, and where experiments are necessary to substantiate, complement, and challenge theoretical modeling and analysis. Here, experimental work is as important as are advances in theory, that are indeed often driven by the experimental findings. In parallel computing, scientific contributions presented in research articles are therefore often based on experimental data, with a substantial part devoted to presenting and discussing the experimental findings. As in all of experimental science, experiments must be presented in a way that makes reproduction by other researchers possible, in principle. Despite appearance to the contrary, we contend that reproducibility plays a small role, and is typically not achieved. As can be found, articles often do not have a sufficiently detailed description of their experiments, and do not make available the software used to obtain the claimed results. As a consequence, parallel computational results are most often impossible to reproduce, often questionable, and therefore of little or no scientific value. We believe that the description of how to reproduce findings should play an important part in every serious, experiment-based parallel computing research article. We aim to initiate a discussion of the reproducibility issue in parallel computing, and elaborate on the importance of reproducible research for (1) better and sounder technical/scientific papers, (2) a sounder and more efficient review process and (3) more effective collective work. This paper expresses our current view on the subject and should be read as a position statement for discussion and future work. We do not consider the related (but no less important) issue of the quality of the experimental design.