Search

Massively multiplexed spectrographs will soon gather large statistical samples of stellar spectra. The accurate estimation of uncertainties on derived parameters, such as line-of-sight velocity $v_\mathrm{los}$, especially for spectra with low signal-to-noise ratios, is paramount. We generated an ensemble of simulated optical spectra of stars as if they were observed with low- and medium-resolution fiber-fed instruments on an 8-meter class telescope, similar to the Subaru Prime Focus Spectrograph, and determined $v_\mathrm{los}$ by fitting stellar templates to the simulations. We compared the empirical errors of the derived parameters -- calculated from an ensemble of simulations -- to the asymptotic error determined from the Fisher matrix, as well as from Monte Carlo sampling of the posterior probability. We confirm that the uncertainty of $v_\mathrm{los}$ scales with the inverse square root of $S/N$, but also show how this scaling breaks down at low $S/N$ and analyze the error and bias caused by template mismatch. We outline a computationally optimized algorithm to fit multi-exposure data and provide the mathematical model of stellar spectrum fitting that maximizes the so called significance, which allows for calculating the error from the Fisher matrix analytically. We also introduce the effective line count, and provide a scaling relation to estimate the error of $v_\mathrm{los}$ measurement based on the stellar type. Our analysis covers a range of stellar types with parameters that are typical of the Galactic outer disk and halo, together with analogs of stars in M31 and in satellite dwarf spheroidal galaxies around the Milky Way., Comment: 31 pages, 10 figures (accepted to Apj)

We present a scheme based on artificial neural networks (ANN) to estimate the line-of-sight velocities of individual galaxies from an observed redshift-space galaxy distribution. We find an estimate of the peculiar velocity at a galaxy based on galaxy counts and barycenters in shells around it. By training the network with environmental characteristics, such as the total mass and mass center within each shell surrounding every galaxy in redshift space, our ANN model can accurately predict the line-of-sight velocity of each individual galaxy. When this velocity is used to eliminate the RSD effect, the two-point correlation function (TPCF) in real space can be recovered with an accuracy better than 1% at $s$ > 8 $h^{-1}\mathrm{Mpc}$, and 4% on all scales compared to ground truth. The real-space power spectrum can be recovered within 3% on $k$< 0.5 $\mathrm{Mpc}^{-1}h$, and less than 5% for all $k$ modes. The quadrupole moment of the TPCF or power spectrum is almost zero down to $s$ = 10 $h^{-1}\mathrm{Mpc}$ or all $k$ modes, indicating an effective correction of the spatial anisotropy caused by the RSD effect. We demonstrate that on large scales, without additional training with new data, our network is adaptable to different galaxy formation models, different cosmological models, and mock galaxy samples at high redshifts and high biases, achieving less than 10% error for scales greater than 15 $h^{-1}\mathrm{Mpc}$. As it is sensitive to large-scale densities, it does not manage to remove Fingers of God in large clusters, but works remarkably well at recovering real-space galaxy positions elsewhere. Our scheme provides a novel way to predict the peculiar velocity of individual galaxies, to eliminate the RSD effect directly in future large galaxy surveys, and to reconstruct the 3-D cosmic velocity field accurately., Comment: 15 pages,15 figures, accepted for publication in MNRAS

A call for coordinated action from government, academia, and industry.

We discuss the history and lessons learned from a series of deployments of environmental sensors measuring soil parameters and CO2 fluxes over the last fifteen years, in an outdoor environment. We present the hardware and software architecture of our current Gen-3 system, and then discuss how we are simplifying the user facing part of the software, to make it easier and friendlier for the environmental scientist to be in full control of the system. Finally, we describe the current effort to build a large-scale Gen-4 sensing platform consisting of hundreds of nodes to track the environmental parameters for urban green spaces in Baltimore, Maryland., Comment: 12 pages, 16 figures, Sensornets 2021 Conference

Indra is a suite of large-volume cosmological $N$-body simulations with the goal of providing excellent statistics of the large-scale features of the distribution of dark matter. Each of the 384 simulations is computed with the same cosmological parameters and different initial phases, with 1024$^3$ dark matter particles in a box of length 1 Gpc/h, 64 snapshots of particle data and halo catalogs, and 505 time steps of the Fourier modes of the density field, amounting to almost a petabyte of data. All of the Indra data are immediately available for analysis via the SciServer science platform, which provides interactive and batch computing modes, personal data storage, and other hosted data sets such as the Millennium simulations and many astronomical surveys. We present the Indra simulations, describe the data products and how to access them, and measure ensemble averages, variances, and covariances of the matter power spectrum, the matter correlation function, and the halo mass function to demonstrate the types of computations that Indra enables. We hope that Indra will be both a resource for large-scale structure research and a demonstration of how to make very large datasets public and computationally accessible., Comment: matches published version. Indra is available on SciServer at http://sciserver.org

We present SciServer, a science platform built and supported by the Institute for Data Intensive Engineering and Science at the Johns Hopkins University. SciServer builds upon and extends the SkyServer system of server-side tools that introduced the astronomical community to SQL (Structured Query Language) and has been serving the Sloan Digital Sky Survey catalog data to the public. SciServer uses a Docker/VM based architecture to provide interactive and batch mode server-side analysis with scripting languages like Python and R in various environments including Jupyter (notebooks), RStudio and command-line in addition to traditional SQL-based data analysis. Users have access to private file storage as well as personal SQL database space. A flexible resource access control system allows users to share their resources with collaborators, a feature that has also been very useful in classroom environments. All these services, wrapped in a layer of REST APIs, constitute a scalable collaborative data-driven science platform that is attractive to science disciplines beyond astronomy., Comment: Accepted for publication in the Astronomy and Computing journal

A Kavli foundation sponsored workshop on the theme \emph{Petabytes to Science} was held 12$^{th}$ to 14$^{th}$ of February 2019 in Las Vegas. The aim of the this workshop was to discuss important trends and technologies which may support astronomy. We also tackled how to better shape the workforce for the new trends and how we should approach education and public outreach. This document was coauthored during the workshop and edited in the weeks after. It comprises the discussions and highlights many recommendations which came out of the workshop. We shall distill parts of this document and formulate potential white papers for the decadal survey., Comment: 70 pages 2 figures - this contains a few fixes

The standard explanation for galaxy spin starts with the tidal-torque theory (TTT), in which an ellipsoidal dark-matter protohalo, which comes to host the galaxy, is torqued up by the tidal gravitational field around it. We discuss a complementary picture, using the relatively familiar velocity field, instead of the tidal field, whose intuitive connection to the surrounding, possibly faraway matter arrangement is more obscure. In this 'spin from primordial inner motions' (SPIM) concept, implicit in TTT derivations but not previously emphasized, the angular momentum from the gravity-sourced velocity field inside a protohalo largely cancels out, but has some excess from the aspherical outskirts. At first, the net spin scales according to linear theory, a sort of comoving conservation of familiar angular momentum. Then, at collapse, it is conserved in physical coordinates. Small haloes are then typically subject to secondary exchanges of angular momentum. The TTT is useful for analytic estimates. But a literal interpretation of the TTT is inaccurate in detail, without some implicit concepts about smoothing of the velocity and tidal fields. This could lead to misconceptions, for those first learning about how galaxies come to spin. Protohaloes are not perfectly ellipsoidal and do not uniformly torque up, as in a naive interpretation of the TTT; their inner velocity fields retain substantial dispersion. Furthermore, quantitatively, given initial conditions and protohalo boundaries, SPIM is more direct and accurate than the TTT to predict halo spins. We also discuss how SPIM applies to rotating filaments, and the relation between halo mass and spin, in which the total spin of a halo can be thought of as a sum of random contributions., Comment: Accepted to OJA (Open Journal of Astrophysics). 3 movies at https://www.youtube.com/playlist?list=PLJxZSP5_6rBmo7Rz8JgnE1eI2bxRGXGCT

We present the multi-GPU realization of the StePS (Stereographically Projected Cosmological Simulations) algorithm with MPI-OpenMP-CUDA hybrid parallelization and nearly ideal scale-out to multiple compute nodes. Our new zoom-in cosmological direct N-body simulation method simulates the infinite universe with unprecedented dynamic range for a given amount of memory and, in contrast to traditional periodic simulations, its fundamental geometry and topology match observations. By using a spherical geometry instead of periodic boundary conditions, and gradually decreasing the mass resolution with radius, our code is capable of running simulations with a few gigaparsecs in diameter and with a mass resolution of $\sim 10^{9}M_{\odot}$ in the center in four days on three compute nodes with four GTX 1080Ti GPUs in each. The code can also be used to run extremely fast simulations with reasonable resolution for fitting cosmological parameters. These simulations are useful for prediction needs of large surveys. The StePS code is publicly available for the research community.

Cosmological $N$-body simulations play a vital role in studying models for the evolution of the Universe. To compare to observations and make a scientific inference, statistic analysis on large simulation datasets, e.g., finding halos, obtaining multi-point correlation functions, is crucial. However, traditional in-memory methods for these tasks do not scale to the datasets that are forbiddingly large in modern simulations. Our prior paper proposes memory-efficient streaming algorithms that can find the largest halos in a simulation with up to $10^9$ particles on a small server or desktop. However, this approach fails when directly scaling to larger datasets. This paper presents a robust streaming tool that leverages state-of-the-art techniques on GPU boosting, sampling, and parallel I/O, to significantly improve performance and scalability. Our rigorous analysis of the sketch parameters improves the previous results from finding the centers of the $10^3$ largest halos to $\sim 10^4-10^5$, and reveals the trade-offs between memory, running time and number of halos. Our experiments show that our tool can scale to datasets with up to $\sim 10^{12}$ particles while using less than an hour of running time on a single GPU Nvidia GTX 1080., Comment: preprint

Motivated by interest in the geometry of high intensity events of turbulent flows, we examine spatial correlation functions of sets where turbulent events are particularly intense. These sets are defined using indicator functions on excursion and iso-value sets. Their geometric scaling properties are analyzed by examining possible power-law decay of their radial correlation function. We apply the analysis to enstrophy, dissipation, and velocity gradient invariants $Q$ and $R$ and their joint spatial distibutions, using data from a direct numerical simulation of isotropic turbulence at ${\rm Re}_\lambda \approx 430$. While no fractal scaling is found in the inertial range using box-counting in the finite Reynolds number flow considered here, power-law scaling in the inertial range is found in the radial correlation functions. Thus a geometric characterization in terms of these sets' correlation dimension is possible. Strong dependence on the enstrophy and dissipation threshold is found, consistent with multifractal behavior. Nevertheless the lack of scaling of the box-counting analysis precludes direct quantitative comparisons with earlier work based on the multifractal formalism. Surprising trends, such as a lower correlation dimension for strong dissipation events compared to strong enstrophy events, are observed and interpreted in terms of spatial coherence of vortices in the flow. We show that sets defined by joint conditions on strain and enstrophy, and on $Q$ and $R$, also display power law scaling of correlation functions, providing further characterization of the complex spatial structure of these intersection sets., Comment: 20 pages, 18 figures, preprint

Observational astronomy in the time-domain era faces several new challenges. One of them is the efficient use of observations obtained at multiple epochs. The work presented here addresses faint object detection with multi-epoch data, and describes an incremental strategy for separating real objects from artifacts in ongoing surveys, in situations where the single-epoch data are summaries of the full image data, such as single-epoch catalogs of flux and direction estimates for candidate sources. The basic idea is to produce low-threshold single-epoch catalogs, and use a probabilistic approach to accumulate catalog information across epochs; this is in contrast to more conventional strategies based on co-added or stacked image data across all epochs. We adopt a Bayesian approach, addressing object detection by calculating the marginal likelihoods for hypotheses asserting there is no object, or one object, in a small image patch containing at most one cataloged source at each epoch. The object-present hypothesis interprets the sources in a patch at different epochs as arising from a genuine object; the no-object (noise) hypothesis interprets candidate sources as spurious, arising from noise peaks. We study the detection probability for constant-flux objects in a simplified Gaussian noise setting, comparing results based on single exposures and stacked exposures to results based on a series of single-epoch catalog summaries. Computing the detection probability based on catalog data amounts to generalized cross-matching: it is the product of a factor accounting for matching of the estimated fluxes of candidate sources, and a factor accounting for matching of their estimated directions. We find that probabilistic fusion of multi-epoch catalog information can detect sources with only modest sacrifice in sensitivity and selectivity compared to stacking., Comment: 11 pages, 11 figures

We present a flexible template-based photometric redshift estimation framework, implemented in C#, that can be seamlessly integrated into a SQL database (or DB) server and executed on-demand in SQL. The DB integration eliminates the need to move large photometric datasets outside a database for redshift estimation, and utilizes the computational capabilities of DB hardware. The code is able to perform both maximum likelihood and Bayesian estimation, and can handle inputs of variable photometric filter sets and corresponding broad-band magnitudes. It is possible to take into account the full covariance matrix between filters, and filter zero points can be empirically calibrated using measurements with given redshifts. The list of spectral templates and the prior can be specified flexibly, and the expensive synthetic magnitude computations are done via lazy evaluation, coupled with a caching of results. Parallel execution is fully supported. For large upcoming photometric surveys such as the LSST, the ability to perform in-place photo-z calculation would be a significant advantage. Also, the efficient handling of variable filter sets is a necessity for heterogeneous databases, for example the Hubble Source Catalog, and for cross-match services such as SkyQuery. We illustrate the performance of our code on two reference photo-z estimation testing datasets, and provide an analysis of execution time and scalability with respect to different configurations. The code is available for download at https://github.com/beckrob/Photo-z-SQL., Comment: 14 pages, 5 figures. Minor revision accepted by Astronomy & Computing on 2017 March 11

In view of future high-precision large-scale structure surveys, it is important to quantify the percent and subpercent level effects in cosmological $N$-body simulations from which theoretical predictions are drawn. One such effect involves deciding whether to zero all modes above the one-dimensional Nyquist frequency, the so-called "corner" modes, in the initial conditions. We investigate this effect by comparing power spectra, density distribution functions, halo mass functions, and halo profiles in simulations with and without these modes. For a simulation with a mass resolution of $m_p \sim 10^{11}\,h^{-1}\,M_{\odot}$, we find that at $z>6$, the difference in the matter power spectrum is large at wavenumbers above $\sim 80$\% of $k_{\rm{Ny}}$, reducing to below 2\% at all scales by $z\sim 3$. Including corner modes results in a better match between low- and high-resolution simulations at wavenumbers around the Nyquist frequency of the low-resolution simulation, but the effect of the corner modes is smaller than the effect of particle discreteness. The differences in mass functions are 3\% for the smallest halos at $z=6$ for the $m_p \sim 10^{11}\,h^{-1}\,M_{\odot}$ simulation, but we find no significant difference in the stacked profiles of well-resolved halos at $z \leq 6$. Thus removing power at $|\mathbf{k}|>k_{\rm{Ny}}$ in the initial conditions of cosmological simulations has a small effect on small scales and high redshifts, typically below a few percent., Comment: 10 pages, 15 figures, matches version accepted to ApJ

We present the methodology and data behind the photometric redshift database of the Sloan Digital Sky Survey Data Release 12 (SDSS DR12). We adopt a hybrid technique, empirically estimating the redshift via local regression on a spectroscopic training set, then fitting a spectrum template to obtain K-corrections and absolute magnitudes. The SDSS spectroscopic catalog was augmented with data from other, publicly available spectroscopic surveys to mitigate target selection effects. The training set is comprised of $1,976,978$ galaxies, and extends up to redshift $z\approx 0.8$, with a useful coverage of up to $z\approx 0.6$. We provide photometric redshifts and realistic error estimates for the $208,474,076$ galaxies of the SDSS primary photometric catalog. We achieve an average bias of $\overline{\Delta z_{\mathrm{norm}}} = 5.84 \times 10^{-5}$, a standard deviation of $\sigma \left(\Delta z_{\mathrm{norm}}\right)=0.0205$, and a $3\sigma$ outlier rate of $P_o=4.11\%$ when cross-validating on our training set. The published redshift error estimates and photometric error classes enable the selection of galaxies with high quality photometric redshifts. We also provide a supplementary error map that allows additional, sophisticated filtering of the data., Comment: 12 pages, 7 figures. Original submitted to MNRAS on 2016 March 03, revision submitted on 2016 April 21

We study the parity-odd part (that we shall call Doppler term) of the linear galaxy two-point correlation function that arises from wide-angle, velocity, Doppler lensing and cosmic acceleration effects. As it is important at low redshift and at large angular separations, the Doppler term is usually neglected in the current generation of galaxy surveys. For future wide-angle galaxy surveys such as Euclid, SPHEREx and SKA, however, we show that the Doppler term must be included. The effect of these terms is dominated by the magnification due to relativistic aberration effects and the slope of the galaxy redshift distribution and it generally mimics the effect of the local type primordial non-Gaussianity with the effective nonlinearity parameter $f_{\rm NL}^{\rm eff}$ of a few, we show that this would affect forecasts on measurements of $f_{\rm NL}$ at low-redshift. Our results show that a survey at low redshift with large number density over a wide area of the sky could detect the Doppler term with a signal-to-noise ratio of $\sim 1-20$, depending on survey specifications., Comment: 21 pages, 10 figures

Many eigensolvers such as ARPACK and Anasazi have been developed to compute eigenvalues of a large sparse matrix. These eigensolvers are limited by the capacity of RAM. They run in memory of a single machine for smaller eigenvalue problems and require the distributed memory for larger problems. In contrast, we develop an SSD-based eigensolver framework called FlashEigen, which extends Anasazi eigensolvers to SSDs, to compute eigenvalues of a graph with hundreds of millions or even billions of vertices in a single machine. FlashEigen performs sparse matrix multiplication in a semi-external memory fashion, i.e., we keep the sparse matrix on SSDs and the dense matrix in memory. We store the entire vector subspace on SSDs and reduce I/O to improve performance through caching the most recent dense matrix. Our result shows that FlashEigen is able to achieve 40%-60% performance of its in-memory implementation and has performance comparable to the Anasazi eigensolvers on a machine with 48 CPU cores. Furthermore, it is capable of scaling to a graph with 3.4 billion vertices and 129 billion edges. It takes about four hours to compute eight eigenvalues of the billion-node graph using 120 GB memory.

We simulate the tidal disruption of a collisionless N-body globular star cluster in a total of 300 different orbits selected to have galactocentric radii between 10 and 30 kpc in four dark matter halos: (a) a spherical halo with no subhalos, (b) a spherical halo with subhalos, (c) a realistic halo with no subhalos, and (d) a realistic halo with subhalos. This allows us to isolate and study how the halo's (lack of) dynamical symmetry and substructures affect the dispersal of tidal debris. The realistic halos are constructed from the snapshot of the Via Lactea II simulation at redshift zero. We find that the overall halo's lack of dynamical symmetry disperses tidal debris to make the streams fluffier, consistent with previous studies of tidal debris of dwarf galaxies in larger orbits than ours in this study. On the other hand, subhalos in realistic potentials can locally enhance the densities along streams, making streams denser than their counterparts in smooth potentials. We show that many long and thin streams can survive in a realistic and lumpy halo for a Hubble time. This suggests that upcoming stellar surveys will likely uncover more thin streams which may contain density gaps that have been shown to be promising probes for dark matter substructures., Comment: 11 pages, 9 figures, ApJ accepted. Minor clarification in abstract

We analyse the correlations between continuum properties and emission line equivalent widths of star-forming and active galaxies from the Sloan Digital Sky Survey. Since upcoming large sky surveys will make broad-band observations only, including strong emission lines into theoretical modelling of spectra will be essential to estimate physical properties of photometric galaxies. We show that emission line equivalent widths can be fairly well reconstructed from the stellar continuum using local multiple linear regression in the continuum principal component analysis (PCA) space. Line reconstruction is good for star-forming galaxies and reasonable for galaxies with active nuclei. We propose a practical method to combine stellar population synthesis models with empirical modelling of emission lines. The technique will help generate more accurate model spectra and mock catalogues of galaxies to fit observations of the new surveys. More accurate modelling of emission lines is also expected to improve template-based photometric redshift estimation methods. We also show that, by combining PCA coefficients from the pure continuum and the emission lines, automatic distinction between hosts of weak active galactic nuclei (AGNs) and quiescent star-forming galaxies can be made. The classification method is based on a training set consisting of high-confidence starburst galaxies and AGNs, and allows for the similar separation of active and star-forming galaxies as the empirical curve found by Kauffmann et al. We demonstrate the use of three important machine learning algorithms in the paper: k-nearest neighbour finding, k-means clustering and support vector machines., Comment: 14 pages, 14 figures. Accepted by MNRAS on 2015 December 22. The paper's website with data and code is at http://www.vo.elte.hu/papers/2015/emissionlines/

Accurate modeling of nonlinearities in the galaxy bispectrum, the Fourier transform of the galaxy three-point correlation function, is essential to fully exploit it as a cosmological probe. In this paper, we present numerical and theoretical challenges in modeling the nonlinear bispectrum. First, we test the robustness of the matter bispectrum measured from N-body simulations using different initial conditions generators. We run a suite of N-body simulations using the Zel'dovich approximation and second-order Lagrangian perturbation theory (2LPT) at different starting redshifts, and find that transients from initial decaying modes systematically reduce the nonlinearities in the matter bispectrum. To achieve 1% accuracy in the matter bispectrum for $z\le3$ on scales $k<1$ $h$/Mpc, 2LPT initial conditions generator with initial redshift of $z\gtrsim100$ is required. We then compare various analytical formulas and empirical fitting functions for modeling the nonlinear matter bispectrum, and discuss the regimes for which each is valid. We find that the next-to-leading order (one-loop) correction from standard perturbation theory matches with N-body results on quasi-linear scales for $z\ge1$. We find that the fitting formula in Gil-Mar\'{\i}n et al. (2012) accurately predicts the matter bispectrum for $z\le1$ on a wide range of scales, but at higher redshifts, the fitting formula given in Scoccimarro & Couchman (2001) gives the best agreement with measurements from N-body simulations., Comment: 14 pages, 11 figures, submitted to MNRAS. V2: fixed an error in model predictions

Solid state disks (SSDs) have advanced to outperform traditional hard drives significantly in both random reads and writes. However, heavy random writes trigger fre- quent garbage collection and decrease the performance of SSDs. In an SSD array, garbage collection of individ- ual SSDs is not synchronized, leading to underutilization of some of the SSDs. We propose a software solution to tackle the unsyn- chronized garbage collection in an SSD array installed in a host bus adaptor (HBA), where individual SSDs are exposed to an operating system. We maintain a long I/O queue for each SSD and flush dirty pages intelligently to fill the long I/O queues so that we hide the performance imbalance among SSDs even when there are few parallel application writes. We further define a policy of select- ing dirty pages to flush and a policy of taking out stale flush requests to reduce the amount of data written to SSDs. We evaluate our solution in a real system. Experi- ments show that our solution fully utilizes all SSDs in an array under random write-heavy workloads. It improves I/O throughput by up to 62% under random workloads of mixed reads and writes when SSDs are under active garbage collection. It causes little extra data writeback and increases the cache hit rate.

We simulate deep images from the Hubble Space Telescope (HST) using semi-empirical models of galaxy formation with only a few basic assumptions and parameters. We project our simulations all the way to the observational domain, adding cosmological and instrumental effects to the images, and analyze them in the same way as real HST images ("forward modeling"). This is a powerful tool for testing and comparing galaxy evolution models, since it allows us to make unbiased comparisons between the predicted and observed distributions of galaxy properties, while automatically taking into account all relevant selection effects. Our semi-empirical models populate each dark matter halo with a galaxy of determined stellar mass and scale radius. We compute the luminosity and spectrum of each simulated galaxy from its evolving stellar mass using stellar population synthesis models. We calculate the intrinsic scatter in the stellar mass-halo mass relation that naturally results from enforcing a monotonically increasing stellar mass along the merger history of each halo. The simulated galaxy images are drawn from cutouts of real galaxies from the Sloan Digital Sky Survey, with sizes and fluxes rescaled to match those of the model galaxies. The distributions of galaxy luminosities, sizes, and surface brightnesses depend on the adjustable parameters in the models, and they agree well with observations for reasonable values of those parameters. Measured galaxy magnitudes and sizes have significant magnitude-dependent biases, with both being underestimated near the magnitude detection limit. The fraction of galaxies detected and fraction of light detected also depend sensitively on the details of the model., Comment: 23 pages, 23 figures; published in the Astrophysical Journal; v2: minor changes to match final published version

We simulate tidal streams in the presence and absence of substructures inside the zero redshift snapshot of the Via Lactea II (VL-2) simulation. A halo finder is used to remove and isolate the subhalos found inside the high resolution dark matter halo of VL-2, and the potentials for both the main halo and all the subhalos are constructed individually using the self-consistent field (SCF) method. This allows us to make direct comparison of tidal streams between a smooth halo and a lumpy halo without assuming idealized profiles or triaxial fits. We simulate the kinematics of a star cluster starting with the same orbital position but two different velocities. Although these two orbits are only moderately eccentric and have similar apo- and pericentric distances, we find that the two streams have very different morphologies. We conclude that our model of the potential of VL-2 can provide insights about tidal streams that have not been explored by previous studies using idealized or axisymmetric models., Comment: 11 pages, 15 figures. ApJ accepted. Updated to match published version

Baryon acoustic oscillations (BAO) are a powerful probe of the expansion history of the universe, which can tell us about the nature of dark energy. In order to accurately characterize the dark energy equation of state using BAO, we must understand the effects of both nonlinearities and redshift space distortions on the location and shape of the acoustic peak. In a previous paper we introduced a novel approach to 2nd order perturbation theory in configuration space using the Zel'dovich approximation, and presented a simple result for the first nonlinear term of the correlation function. In this paper, we extend this approach to redshift space. We show how perform the computation, and present the analytic result for the first nonlinear term in the correlation function. Finally, we validate our result through comparison to numerical simulations., Comment: 8 pages; Accepted by Astrophysical Journal

We present a case study about the spatial indexing and regional classification of billions of geographic coordinates from geo-tagged social network data using Hierarchical Triangular Mesh (HTM) implemented for Microsoft SQL Server. Due to the lack of certain features of the HTM library, we use it in conjunction with the GIS functions of SQL Server to significantly increase the efficiency of pre-filtering of spatial filter and join queries. For example, we implemented a new algorithm to compute the HTM tessellation of complex geographic regions and precomputed the intersections of HTM triangles and geographic regions for faster false-positive filtering. With full control over the index structure, HTM-based pre-filtering of simple containment searches outperforms SQL Server spatial indices by a factor of ten and HTM-based spatial joins run about a hundred times faster., Comment: appears in Proceedings of the 26th International Conference on Scientific and Statistical Database Management (2014)

We present photometric redshifts and associated probability distributions for all detected sources in the Extended Chandra Deep Field South (ECDFS). The work makes use of the most up-to-date data from the Cosmic Assembly Near-IR Deep Legacy Survey (CANDELS) and the Taiwan ECDFS Near-Infrared Survey (TENIS) in addition to other data. We also revisit multi-wavelength counterparts for published X-ray sources from the 4Ms-CDFS and 250ks-ECDFS surveys, finding reliable counterparts for 1207 out of 1259 sources ($\sim 96\%$). Data used for photometric redshifts include intermediate-band photometry deblended using the TFIT method, which is used for the first time in this work. Photometric redshifts for X-ray source counterparts are based on a new library of AGN/galaxy hybrid templates appropriate for the faint X-ray population in the CDFS. Photometric redshift accuracy for normal galaxies is 0.010 and for X-ray sources is 0.014, and outlier fractions are $4\%$ and $5.4\%$ respectively. The results within the CANDELS coverage area are even better as demonstrated both by spectroscopic comparison and by galaxy-pair statistics. Intermediate-band photometry, even if shallow, is valuable when combined with deep broad-band photometry. For best accuracy, templates must include emission lines., Comment: The paper has been accepted by ApJ. The materials we provide are available under [Surveys] > [CDFS] through the portal http://www.mpe.mpg.de/XraySurveys

Future galaxy surveys will measure baryon acoustic oscillations (BAOs) with high significance, and a complete understanding of the anisotropies of BAOs in redshift space will be important to exploit the cosmological information in BAOs. Here we describe the anisotropies that arise in the redshift-space galaxy two-point correlation function (2PCF) and elucidate the origin of features that arise in the dependence of the BAOs on the angle between the orientation of the galaxy pair and the line of sight. We do so with a derivation of the configuration-space 2PCF using streaming model. We find that, contrary to common belief, the locations of BAO peaks in the redshift-space 2PCF are anisotropic even in the linear theory. Anisotropies in BAO depend strongly on the method of extracting the peak, showing maximum 3 % angular variation. We also find that extracting the BAO peak of $r^2\xi(r,\mu)$ significantly reduces the anisotropy to sub-percent level angular variation. When subtracting the tilt due to the broadband behavior of the 2PCF, the BAO bump is enhanced along the line of sight because of local infall velocities toward the BAO bump. Precise measurement of the angular dependence of the redshift-space 2PCF will allow new geometrical tests of dark energy beyond the BAO., Comment: 11 pages, 7 figures

Graph analysis performs many random reads and writes, thus, these workloads are typically performed in memory. Traditionally, analyzing large graphs requires a cluster of machines so the aggregate memory exceeds the graph size. We demonstrate that a multicore server can process graphs with billions of vertices and hundreds of billions of edges, utilizing commodity SSDs with minimal performance loss. We do so by implementing a graph-processing engine on top of a user-space SSD file system designed for high IOPS and extreme parallelism. Our semi-external memory graph engine called FlashGraph stores vertex state in memory and edge lists on SSDs. It hides latency by overlapping computation with I/O. To save I/O bandwidth, FlashGraph only accesses edge lists requested by applications from SSDs; to increase I/O throughput and reduce CPU overhead for I/O, it conservatively merges I/O requests. These designs maximize performance for applications with different I/O characteristics. FlashGraph exposes a general and flexible vertex-centric programming interface that can express a wide variety of graph algorithms and their optimizations. We demonstrate that FlashGraph in semi-external memory performs many algorithms with performance up to 80% of its in-memory implementation and significantly outperforms PowerGraph, a popular distributed in-memory graph engine., Comment: published in FAST'15

Not only source catalogs are extracted from astronomy observations. Their sky coverage is always carefully recorded and used in statistical analyses, such as correlation and luminosity function studies. Here we present a novel method for catalog matching, which inherently builds on the coverage information for better performance and completeness. A modified version of the Zones Algorithm is introduced for matching partially overlapping observations, where irrelevant parts of the data are excluded up front for efficiency. Our design enables searches to focus on specific areas on the sky to further speed up the process. Another important advantage of the new method over traditional techniques is its ability to quickly detect dropouts, i.e., the missing components that are in the observed regions of the celestial sphere but did not reach the detection limit in some observations. These often provide invaluable insight into the spectral energy distribution of the matched sources but rarely available in traditional associations., Comment: 14 pages, 6 figures

Observations of diffuse Galactic gamma ray emission (DGE) by the Fermi Large Area Telescope (LAT) allow a detailed study of cosmic rays and the interstellar medium. However, diffuse emission models of the inner Galaxy underpredict the Fermi-LAT data at energies above a few GeV and hint at possible non-astrophysical sources including dark matter (DM) annihilations or decays. We present a study of the possible emission components from DM using the high-resolution Via Lactea II N-body simulation of a Milky Way-sized DM halo. We generate full-sky maps of DM annihilation and decay signals that include modeling of the adiabatic contraction of the host density profile, Sommerfeld enhanced DM annihilations, $p$-wave annihilations, and decaying DM. We compare our results with the DGE models produced by the Fermi-LAT team over different sky regions, including the Galactic center, high Galactic latitudes, and the Galactic anti-center. This work provides possible templates to fit the observational data that includes the contribution of the subhalo population to DM gamma-ray emission, with the significance depending on the annihilation/decay channels and the Galactic regions being considered., Comment: Published by PRD

Structures like galaxies and filaments of galaxies in the Universe come about from the origami-like folding of an initially flat three-dimensional manifold in 6D phase space. The ORIGAMI method identifies these structures in a cosmological simulation, delineating the structures according to their outer folds. Structure identification is a crucial step in comparing cosmological simulations to observed maps of the Universe. The ORIGAMI definition is objective, dynamical and geometric: filament, wall and void particles are classified according to the number of orthogonal axes along which dark-matter streams have crossed. Here, we briefly review these ideas, and speculate on how ORIGAMI might be useful to find cosmic voids., Comment: Talk summary to appear in the Proceedings of the 13th Marcel Grossmann Meeting (MG13), Stockholm, July 2012. 3 pages, 3 figures

We introduce the AGORA project, a comprehensive numerical study of well-resolved galaxies within the LCDM cosmology. Cosmological hydrodynamic simulations with force resolutions of ~100 proper pc or better will be run with a variety of code platforms to follow the hierarchical growth, star formation history, morphological transformation, and the cycle of baryons in and out of 8 galaxies with halo masses M_vir ~= 1e10, 1e11, 1e12, and 1e13 Msun at z=0 and two different ("violent" and "quiescent") assembly histories. The numerical techniques and implementations used in this project include the smoothed particle hydrodynamics codes GADGET and GASOLINE, and the adaptive mesh refinement codes ART, ENZO, and RAMSES. The codes will share common initial conditions and common astrophysics packages including UV background, metal-dependent radiative cooling, metal and energy yields of supernovae, and stellar initial mass function. These are described in detail in the present paper. Subgrid star formation and feedback prescriptions will be tuned to provide a realistic interstellar and circumgalactic medium using a non-cosmological disk galaxy simulation. Cosmological runs will be systematically compared with each other using a common analysis toolkit, and validated against observations to verify that the solutions are robust - i.e., that the astrophysical assumptions are responsible for any success, rather than artifacts of particular implementations. The goals of the AGORA project are, broadly speaking, to raise the realism and predictive power of galaxy simulations and the understanding of the feedback processes that regulate galaxy "metabolism." The proof-of-concept dark matter-only test of the formation of a galactic halo with a z=0 mass of M_vir ~= 1.7e11 Msun by 9 different versions of the participating codes is also presented to validate the infrastructure of the project., Comment: 21 pages, 6 figures, Accepted for publication in the Astrophysical Journal Supplement, Image resolution greatly reduced, High-resolution version of this article and more information about the AGORA Project including the science goals of the 13 Working Groups are available at http://www.AGORAsimulations.org/ and at http://sites.google.com/site/santacruzcomparisonproject/details/

Many fields of science rely on relational database management systems to analyze, publish and share data. Since RDBMS are originally designed for, and their development directions are primarily driven by, business use cases they often lack features very important for scientific applications. Horizontal scalability is probably the most important missing feature which makes it challenging to adapt traditional relational database systems to the ever growing data sizes. Due to the limited support of array data types and metadata management, successful application of RDBMS in science usually requires the development of custom extensions. While some of these extensions are specific to the field of science, the majority of them could easily be generalized and reused in other disciplines. With the Graywulf project we intend to target several goals. We are building a generic platform that offers reusable components for efficient storage, transformation, statistical analysis and presentation of scientific data stored in Microsoft SQL Server. Graywulf also addresses the distributed computational issues arising from current RDBMS technologies. The current version supports load balancing of simple queries and parallel execution of partitioned queries over a set of mirrored databases. Uniform user access to the data is provided through a web based query interface and a data surface for software clients. Queries are formulated in a slightly modified syntax of SQL that offers a transparent view of the distributed data. The software library consists of several components that can be reused to develop complex scientific data warehouses: a system registry, administration tools to manage entire database server clusters, a sophisticated workflow execution framework, and a SQL parser library., Comment: SSDBM 2013 proceedings

We describe a scalable database cluster for the spatial analysis and annotation of high-throughput brain imaging data, initially for 3-d electron microscopy image stacks, but for time-series and multi-channel data as well. The system was designed primarily for workloads that build connectomes---neural connectivity maps of the brain---using the parallel execution of computer vision algorithms on high-performance compute clusters. These services and open-science data sets are publicly available at http://openconnecto.me. The system design inherits much from NoSQL scale-out and data-intensive computing architectures. We distribute data to cluster nodes by partitioning a spatial index. We direct I/O to different systems---reads to parallel disk arrays and writes to solid-state storage---to avoid I/O interference and maximize throughput. All programming interfaces are RESTful Web services, which are simple and stateless, improving scalability and usability. We include a performance evaluation of the production system, highlighting the effectiveness of spatial data organization., Comment: 11 pages, 13 figures

Citizen science, in which volunteers work with professional scientists to conduct research, is expanding due to large online datasets. To plan projects, it is important to understand volunteers' motivations for participating. This paper analyzes results from an online survey of nearly 11,000 volunteers in Galaxy Zoo, an astronomy citizen science project. Results show that volunteers' primary motivation is a desire to contribute to scientific research. We encourage other citizen science projects to study the motivations of their volunteers, to see whether and how these results may be generalized to inform the field of citizen science., Comment: 41 pages, including 6 figures and one appendix. In press at Astronomy Education Review

Large-scale bulk flows in the Universe distort the initial density field, broadening the baryon-acoustic-oscillation (BAO) feature that was imprinted when baryons were strongly coupled to photons. Additionally, there is a small shift inward in the peak of the conventional overdensity correlation function, a mass-weighted statistic. This shift occurs when high density peaks move toward each other. We explore whether this shift can be removed by applying to the density field a transform (such as a logarithm) that gives fairer statistical weight to fluctuations in underdense regions. Using configuration-space perturbation theory in the Zel'dovich approximation, we find that the log-density correlation function shows a much smaller inward shift in the position of the BAO peak at low redshift than is seen in the overdensity correlation function. We also show that if the initial, Lagrangian density of matter parcels could be estimated at their Eulerian positions, giving a displaced-initial-density field, its peak shift would be even smaller. In fact, a transformed field that accentuates underdensities, such as the reciprocal of the density, pushes the peak the other way, outward. In our model, these shifts in the peak position can be attributed to shift terms, involving the derivative of the linear correlation function, that entirely vanish in this displaced-initial-density field., Comment: 5 pages, 3 figures, Accepted by ApjL

We propose to describe the variety of galaxies from SDSS by using only one affine parameter. To this aim, we build the Principal Curve (P-curve) passing through the spine of the data point cloud, considering the eigenspace derived from Principal Component Analysis of morphological, physical and photometric galaxy properties. Thus, galaxies can be labeled, ranked and classified by a single arc length value of the curve, measured at the unique closest projection of the data points on the P-curve. We find that the P-curve has a "W" letter shape with 3 turning points, defining 4 branches that represent distinct galaxy populations. This behavior is controlled mainly by 2 properties, namely u-r and SFR. We further present the variations of several galaxy properties as a function of arc length. Luminosity functions variate from steep Schechter fits at low arc length, to double power law and ending in Log-normal fits at high arc length. Galaxy clustering shows increasing autocorrelation power at large scales as arc length increases. PCA analysis allowed to find peculiar galaxy populations located apart from the main cloud of data points, such as small red galaxies dominated by a disk, of relatively high stellar mass-to-light ratio and surface mass density. The P-curve allows not only dimensionality reduction, but also provides supporting evidence for relevant physical models and scenarios in extragalactic astronomy: 1) Evidence for the hierarchical merging scenario in the formation of a selected group of red massive galaxies. These galaxies present a log-normal r-band luminosity function, which might arise from multiplicative processes involved in this scenario. 2) Connection between the onset of AGN activity and star formation quenching, which appears in green galaxies when transitioning from blue to red populations. (Full abstract in downloadable version), Comment: Full abstract in downloadable version

Multi-wavelength astronomical studies require cross-identification of detections of the same celestial objects in multiple catalogs based on spherical coordinates and other properties. Because of the large data volumes and spherical geometry, the symmetric N-way association of astronomical detections is a computationally intensive problem, even when sophisticated indexing schemes are used to exclude obviously false candidates. Legacy astronomical catalogs already contain detections of more than a hundred million objects while the ongoing and future surveys will produce catalogs of billions of objects with multiple detections of each at different times. The varying statistical error of position measurements, moving and extended objects, and other physical properties make it necessary to perform the cross-identification using a mathematically correct, proper Bayesian probabilistic algorithm, capable of including various priors. One time, pair-wise cross-identification of these large catalogs is not sufficient for many astronomical scenarios. Consequently, a novel system is necessary that can cross-identify multiple catalogs on-demand, efficiently and reliably. In this paper, we present our solution based on a cluster of commodity servers and ordinary relational databases. The cross-identification problems are formulated in a language based on SQL, but extended with special clauses. These special queries are partitioned spatially by coordinate ranges and compiled into a complex workflow of ordinary SQL queries. Workflows are then executed in a parallel framework using a cluster of servers hosting identical mirrors of the same data sets.

Extreme value statistics (EVS) is applied to the distribution of galaxy luminosities in the Sloan Digital Sky Survey (SDSS). We analyze the DR8 Main Galaxy Sample (MGS), as well as the Luminous Red Galaxies (LRG). Maximal luminosities are sampled from batches consisting of elongated pencil beams in the radial direction of sight. For the MGS, results suggest a small and positive tail index $\xi$, effectively ruling out the possibility of having a finite maximum cutoff luminosity, and implying that the luminosity distribution function may decay as a power law at the high luminosity end. Assuming, however, $\xi=0$, a non-parametric comparison of the maximal luminosities with the Fisher-Tippett-Gumbel distribution (limit distribution for variables distributed by the Schechter fit) indicates a good agreement provided uncertainties arising both from the finite batch size and from the batch size distribution are accounted for. For a volume limited sample of LRGs, results show that they can be described as being the extremes of a luminosity distribution with an exponentially decaying tail, provided the uncertainties related to batch-size distribution are taken care of., Comment: Accepted for publication in ApJ

Contrary to the common view voids have very complex internal structure and dynamics. Here we show how the hierarchy of structures in the density field inside voids is reflected by a similar hierarchy of structures in the velocity field. Voids defined by dense filaments and clusters can de described as simple expanding domains with coherent flows everywhere except at their boundaries. At scales smaller that the void radius the velocity field breaks into expanding sub-domains corresponding to sub- voids. These sub-domains break into even smaller sub-sub domains at smaller scales resulting in a nesting hierarchy of locally expanding domains. The ratio between the magnitude of the velocity field responsible for the expansion of the void and the velocity field defining the sub voids is approximately one order of magnitude. The small-scale components of the velocity field play a minor role in the shaping of the voids but they define the local dynamics directly affecting the processes of galaxy formation and evolution. The super-Hubble expansion inside voids makes them cosmic magnifiers by stretching their internal primordial density fluctuations allowing us to probe the small scales in the primordial density field. Voids also act like time machines by "freezing" the development of the medium-scale density fluctuations responsible for the formation of the tenuous web of structures seen connecting proto galaxies in computer simulations. As a result of this freezing haloes in voids can remain "connected" to this tenuous web until the present time. This may have an important effect in the formation and evolution of galaxies in voids by providing an efficient gas accretion mechanism via coherent low-velocity streams that can keep a steady inflow of matter for extended periods of time., Comment: High-res version are related media here: http://skysrv.pha.jhu.edu/~miguel/Papers/Hierarchy_voids/index.html

Baryon acoustic oscillations are an excellent technique to constrain the properties of dark energy in the Universe. In order to accurately characterize the dark energy equation of state, we must understand the effects of both the nonlinearities and redshift space distortions on the location and shape of the acoustic peak. In this paper, we consider these effects using the Zel'dovich approximation and a novel approach to 2nd order perturbation theory. The second order term of the Zel'dovich power spectrum is built from convolutions of the linear power spectrum with polynomial kernels in Fourier space, suggesting that the corresponding term of the the Zel'dovich correlation function can be written as a sum of quadratic products of a broader class of correlation functions, expressed through simple spherical Bessel transforms of the linear power spectrum. We show how to systematically perform such a computation. We explicitly prove that our result is the Fourier transform of the Zel'dovich power spectrum, and compare our expressions to numerical simulations. Finally, we highlight the advantages of writing the nonlinear expansion in configuration space, as this calculation is easily extended to redshift space, and the higher order terms are mathematically simpler than their Fourier counterparts., Comment: Accepted to ApJ. 7 pages, 2 figures

We present the ORIGAMI method of identifying structures, particularly halos, in cosmological N-body simulations. Structure formation can be thought of as the folding of an initially flat three-dimensional manifold in six-dimensional phase space. ORIGAMI finds the outer folds that delineate these structures. Halo particles are identified as those that have undergone shell-crossing along 3 orthogonal axes, providing a dynamical definition of halo regions that is independent of density. ORIGAMI also identifies other morphological structures: particles that have undergone shell-crossing along 2, 1, or 0 orthogonal axes correspond to filaments, walls, and voids respectively. We compare this method to a standard Friends-of-Friends halo-finding algorithm and find that ORIGAMI halos are somewhat larger, more diffuse, and less spherical, though the global properties of ORIGAMI halos are in good agreement with other modern halo-finding algorithms., Comment: 11 pages, 14 figures; matches version accepted to ApJ

We investigate the use of a logarithmic density variable in estimating the Lagrangian displacement field, motivated by the success of a logarithmic transformation in restoring information to the matter power spectrum. The logarithmic relation is an extension of the linear relation, motivated by the continuity equation, in which the density field is assumed to be proportional to the divergence of the displacement field; we compare the linear and logarithmic relations by measuring both of these fields directly in a cosmological N-body simulation. The relative success of the logarithmic and linear relations depends on the scale at which the density field is smoothed. Thus we explore several ways of measuring the density field, including Cloud-In-Cell smoothing, adaptive smoothing, and the (scale-independent) Delaunay tessellation, and we use both a Fourier space and a geometrical tessellation approach to measuring the divergence. We find that the relation between the divergence of the displacement field and the density is significantly tighter with a logarithmic density variable, especially at low redshifts and for very small (~2 Mpc/h) smoothing scales. We find that the grid-based methods are more reliable than the tessellation-based method of calculating both the density and the divergence fields, though in both cases the logarithmic relation works better in the appropriate regime, which corresponds to nonlinear scales for the grid-based methods and low densities for the tessellation-based method., Comment: 6 pages, 3 figures, accepted to ApJ

In this work we present an atlas of composite spectra of galaxies based on the data of the Sloan Digital Sky Survey Data Release 7 (SDSS DR7). Galaxies are classified by colour, nuclear activity and star-formation activity to calculate average spectra of high signal-to-noise ratio and resolution (S/N = 132 - 4760 at {Dlambda = 1 A), using an algorithm that is robust against outliers. Besides composite spectra, we also compute the first five principal components of the distributions in each galaxy class to characterize the nature of variations of individual spectra around the averages. The continua of the composite spectra are fitted with BC03 stellar population synthesis models to extend the wavelength coverage beyond the coverage of the SDSS spectrographs. Common derived parameters of the composites are also calculated: integrated colours in the most popular filter systems, line strength measurements, and continuum absorption indices (including Lick indices). These derived parameters are compared with the distributions of parameters of individual galaxies and it is shown on many examples that the composites of the atlas cover much of the parameter space spanned by SDSS galaxies. By co-adding thousands of spectra, a total integration time of several months can be reached, which results in extremely low noise composites. The variations in redshift not only allow for extending the spectral coverage bluewards to the original wavelength limit of the SDSS spectrographs, but also make higher spectral resolution achievable. The composite spectrum atlas is available online at http://www.vo.elte.hu/compositeatlas.

We present a statistical study of the correlations and dimensionality of emission lines carried out on a sample of over 40,000 Sloan Digital Sky Survey (SDSS) galaxies. Using principal component analysis, we found that the equivalent widths of the 11 strongest lines can be well represented using three parameters. We also explore correlations of the emission pattern with the eigenspace representation of the continuum spectrum. The observed relations are used to provide an empirical prescription for expectation values and variances of emission-line strengths as a function of spectral shape. We show that this estimation of emission lines has a sufficient accuracy to make it suitable for photometric applications. The method has already proved useful in SDSS photometric redshift estimation., Comment: 38 pages, 20 figures

Star formation is one of the most important processes in galaxy formation. The luminosity of Halpha recombination and [OII] forbidden emissions remain to be most used in measuring formation rate of massive stars in galaxies. Here we report the inclination dependency of continuum-subtracted and aperture-corrected nebular luminosities, including Halpha, Hbeta, Hgamma, [OII], [NII], of disk-dominated galaxies in the local universe. Their luminosities decrease by a factor of three from face-on to edge-on (axis ratio limit = 0.17) orientations. This dependence is deduced to be caused by extinction due to diffuse dust within the disks with an amplitude of 1.2 mag. The line-luminosity--inclination relation provides a novel way to remove extinction in emission lines and present star formation rate of disk galaxies out to redshift of 1.6., Comment: 2 tables, 3 figures. Submitted to ApJ Letter