Search

State-of-the-art deep learning systems rely on iterative distributed training to tackle the increasing complexity of models and input data. The iteration time in these communication-heavy systems depends on the computation time, communication time and the extent of overlap of computation and communication. In this work, we identify a shortcoming in systems with graph representation for computation, such as TensorFlow and PyTorch, that result in high variance in iteration time --- random order of received parameters across workers. We develop a system, TicTac, to improve the iteration time by fixing this issue in distributed deep learning with Parameter Servers while guaranteeing near-optimal overlap of communication and computation. TicTac identifies and enforces an order of network transfers which improves the iteration time using prioritization. Our system is implemented over TensorFlow and requires no changes to the model or developer inputs. TicTac improves the throughput by up to $37.7\%$ in inference and $19.2\%$ in training, while also reducing straggler effect by up to $2.3\times$. Our code is publicly available.

In this paper we introduce Yoke graphs, a family of flip graphs that generalizes several previously studied families of graphs: colored triangle free triangulations, arc permutations and caterpillars. Our main result is the computation of the diameter of an arbitrary Yoke graph.

Supervisory signals can help topic models discover low-dimensional data representations that are more interpretable for clinical tasks. We propose a framework for training supervised latent Dirichlet allocation that balances two goals: faithful generative explanations of high-dimensional data and accurate prediction of associated class labels. Existing approaches fail to balance these goals by not properly handling a fundamental asymmetry: the intended task is always predicting labels from data, not data from labels. Our new prediction-constrained objective trains models that predict labels from heldout data well while also producing good generative likelihoods and interpretable topic-word parameters. In a case study on predicting depression medications from electronic health records, we demonstrate improved recommendations compared to previous supervised topic models and high- dimensional logistic regression from words alone., Comment: Accepted poster at NIPS 2017 Workshop on Machine Learning for Health (https://ml4health.github.io/2017/)

In preparation for the upcoming all-sky data releases of the Gaia mission, we compiled a catalog of known hot subdwarf stars and candidates drawn from the literature and yet unpublished databases. The catalog contains 5613 unique sources and provides multi-band photometry from the ultraviolet to the far infrared, ground based proper motions, classifications based on spectroscopy and colors, published atmospheric parameters, radial velocities and light curve variability information. Using several different techniques, we removed outliers and misclassified objects. By matching this catalog with astrometric and photometric data from the Gaia mission, we will develop selection criteria to construct a homogeneous, magnitude-limited all-sky catalog of hot subdwarf stars based on Gaia data. As first application of the catalog data, we present the quantitative spectral analysis of 280 sdB and sdOB stars from the Sloan Digital Sky Survey Data Release 7. Combining our derived parameters with state-of-the-art proper motions, we performed a full kinematic analysis of our sample. This allowed us to separate the first significantly large sample of 78 sdBs and sdOBs belonging to the Galactic halo. Comparing the properties of hot subdwarfs from the disk and the halo with hot subdwarf samples from the globular clusters $\omega$ Cen and NGC 2808, we found the fraction of intermediate He-sdOBs in the field halo population to be significantly smaller than in the globular clusters., Comment: 5 pages, to appear in the Proceedings of the 8th Meeting on Hot Subdwarf Stars and Related Objects, 9. to 15. July 2017, Cracow, Poland. Eds. A. Baran, A. E. Lynas-Gray, Open Astronomy, in press

Predicting the future in real-world settings, particularly from raw sensory observations such as images, is exceptionally challenging. Real-world events can be stochastic and unpredictable, and the high dimensionality and complexity of natural images requires the predictive model to build an intricate understanding of the natural world. Many existing methods tackle this problem by making simplifying assumptions about the environment. One common assumption is that the outcome is deterministic and there is only one plausible future. This can lead to low-quality predictions in real-world settings with stochastic dynamics. In this paper, we develop a stochastic variational video prediction (SV2P) method that predicts a different possible future for each sample of its latent variables. To the best of our knowledge, our model is the first to provide effective stochastic multi-frame prediction for real-world video. We demonstrate the capability of the proposed method in predicting detailed future frames of videos on multiple real-world datasets, both action-free and action-conditioned. We find that our proposed method produces substantially improved video predictions when compared to the same model without stochasticity, and to other stochastic video prediction methods. Our SV2P implementation will be open sourced upon publication.

We present a mechanism that puts users in the center of control and empowers them to dictate the access to their collections of data. Revisiting the fundamental mechanisms in security for providing protection, our solution uses capabilities, access lists, and access rights following well-understood formal notions for reasoning about access. This contribution presents a practical, correct, auditable, transparent, distributed, and decentralized mechanism that is well-matched to the current emerging environments including Internet of Things, smart city, precision medicine, and autonomous cars. It is based on well-tested principles and practices used in a distributed authorization, cryptocurrencies, and scalable computing.

In an effort to overcome the data deluge in computational biology and bioinformatics and to facilitate bioinformatics research in the era of big data, we identify some of the most influential algorithms that have been widely used in the bioinformatics community. These top data mining and machine learning algorithms cover classification, clustering, regression, graphical model-based learning, and dimensionality reduction. The goal of this study is to guide the focus of scalable computing experts in the endeavor of applying new storage and scalable computation designs to bioinformatics algorithms that merit their attention most, following the engineering maxim of "optimize the common case".

We consider the bifurcations of standing wave solutions to the nonlinear Schr\"odinger equation (NLS) posed on a quantum graph consisting of two loops connected by a single edge, the so-called dumbbell, recently studied by Marzuola and Pelinovsky. The authors of that study found the ground state undergoes two bifurcations, first a symmetry-breaking, and the second which they call a symmetry-preserving bifurcation. We clarify the type of the symmetry-preserving bifurcation, showing it to be transcritical. We then reduce the question, and show that the phenomena described in that paper can be reproduced in a simple discrete self-trapping equation on a combinatorial graph of bowtie shape. This allows for complete analysis both by geometric methods and by parameterizing the full solution space. We then expand the question, and describe the bifurcations of all the standing waves of this system, which can be classified into three families, and of which there exists a countably infinite set.

In language production, humans are confronted with considerable word selection demands. Often, we must select a word from among similar, acceptable, and competing alternative words in order to construct a sentence that conveys an intended meaning. In recent years, the left inferior frontal gyrus (LIFG) has been identified as critical to this ability. Despite a recent emphasis on network approaches to understanding language, how the LIFG interacts with the brain's complex networks to facilitate controlled language performance remains unknown. Here, we take a novel approach to understand word selection as a network control process in the brain. Using an anatomical brain network derived from high-resolution diffusion spectrum imaging (DSI), we computed network controllability underlying the site of transcranial magnetic stimulation in the LIFG between administrations of two word selection tasks. We find that a statistic that quantifies the LIFG's theoretically predicted control of difficult-to-reach states explains vulnerability to TMS in language tasks that vary in response (cognitive control) demands: open-response (word generation) vs. closed-response (number naming) tasks. Moreover, we find that a statistic that quantifies the LIFG's theoretically predicted control of communication across modules in the human connectome explains TMS-induced changes in open-response language task performance only. These findings establish a link between network controllability, cognitive function, and TMS effects.

While there exist many isolation mechanisms that are available to cloud service providers, including virtual machines, containers, etc., the problem of side-channel increases in importance as a remaining security vulnerability, particularly in the presence of shared caches and multicore processors. In this paper we present a hardware-software mechanism that improves the isolation of cloud processes in the presence of shared caches on multicore chips. Combining the Intel CAT architecture that enables cache partitioning on the fly with novel scheduling techniques and state cleansing mechanisms, we enable cache-side-channel free computing for Linux-based containers and virtual machines, in particular, those managed by KVM. We do a preliminary evaluation of our system using a CPU bound workload. Our system allows Simultaneous Multithreading (SMT) to remain enabled and does not require application level changes.

Supervisory signals have the potential to make low-dimensional data representations, like those learned by mixture and topic models, more interpretable and useful. We propose a framework for training latent variable models that explicitly balances two goals: recovery of faithful generative explanations of high-dimensional data, and accurate prediction of associated semantic labels. Existing approaches fail to achieve these goals due to an incomplete treatment of a fundamental asymmetry: the intended application is always predicting labels from data, not data from labels. Our prediction-constrained objective for training generative models coherently integrates loss-based supervisory signals while enabling effective semi-supervised learning from partially labeled data. We derive learning algorithms for semi-supervised mixture and topic models using stochastic gradient descent with automatic differentiation. We demonstrate improved prediction quality compared to several previous supervised topic models, achieving predictions competitive with high-dimensional logistic regression on text sentiment analysis and electronic health records tasks while simultaneously learning interpretable topics.

The predicted orbital-period distribution of the subdwarf-B (sdB) population is bi-modal with a peak at short (< 10 days) and long (> 500 days) periods. Observationally, many short-period sdB systems are known, but only few wide sdB binaries have been studied in detail. Based on a long-term monitoring program the wide sdB sample has been increased, finding an unexpected correlation between the eccentricity and period. In this article we present the orbital solution and spectral analysis of four new systems, BD-7.5977, EC11031-1348, TYC2084-448-1 and TYC3871-835-1, and update the orbital solution of PG1104+243. Using the whole sample of wide sdBs, we aim at finding possible correlations between orbital and spectral properties, with as goal improving theoretical models of Roche-lobe overflow. High-resolution spectra were obtained to determine the radial velocities of both the sdB and MS components. Surface gravities and temperatures of both component were derived from photometric spectral-energy distributions. Spectral parameters of the cool companion were verified using the GSSP code. Furthermore the amount of accreted mass was estimated. Orbital parameters matching the earlier observed period-eccentricity relation were found for three systems, while TYC 2084-448-1 is found to have a lower eccentricity than expected from the period-eccentricity trend indicated by the other systems. Based on new observations, the orbit of PG 1104+243 has a small but significant eccentricity of 0.04 $\pm$ 0.02, matching other systems with similar periods. Furthermore, a correlation between accreted mass and orbital period was found, as well as a possible relation between the initial mass-ratio and the final period-eccentricity. The wide sdB-binary sample shows interesting possible correlations between orbital and spectral properties. However, a larger sample is necessary to statistically validate them., Comment: 18 pages, 9 figures, Accepted for publication in A&A

Convolutional autoregressive models have recently demonstrated state-of-the-art performance on a number of generation tasks. While fast, parallel training methods have been crucial for their success, generation is typically implemented in a na\"{i}ve fashion where redundant computations are unnecessarily repeated. This results in slow generation, making such models infeasible for production environments. In this work, we describe a method to speed up generation in convolutional autoregressive models. The key idea is to cache hidden states to avoid redundant computation. We apply our fast generation method to the Wavenet and PixelCNN++ models and achieve up to $21\times$ and $183\times$ speedups respectively., Comment: Accepted at ICLR 2017 Workshop

We present an analysis of three new pulsating subdwarf B stars discovered in the SuperWASP archive. Two of the stars, J1938+5609 and J0902-0720, are p-mode pulsators; J1938+5609 shows a pulsation at 231.62 d$^{-1}$ ($P=373$ s; 2681 $\mu$Hz) with an amplitude of 4 mmag, whereas J0902-0720 pulsates at frequencies 636.74 d$^{-1}$ ($P=136$ s; 7370 $\mu$Hz) and 615.34 d$^{-1}$ ($P=140$ s; 7122 $\mu$Hz), with amplitudes 7.27 and 1.53 mmag, respectively. The third star, J2344-3427, is a hybrid pulsator with a p-mode frequency at 223.16 d$^{-1}$ ($P=387$ s; 2583 $\mu$Hz) and a corresponding amplitude of 1.5 mmag, and g modes in the frequency range $8.68-28.56$ d$^{-1}$ ($P=3025-9954$ s; $100-331$ $\mu$Hz) and amplitudes between 0.76 and 1.17 mmag. Spectroscopic results place J1938+5609 and J2344-3427 among the long-period or hybrid pulsators, suggesting there may be further modes in these stars below our detection limits, with J0902-0720 placed firmly amongst the p-mode pulsators., Comment: 13 pages, 8 figures, 4 tables, accepted for publication in MNRAS

The third-order Jeffery-Hamel ODE governing the flow of an incompressible fluid in a two-dimensional wedge is briefly derived, and a C^1 finite element formulation of the equation is developed. This formulation has several advantages, including a natural framework for enforcing the boundary conditions, a numerically efficient solution procedure, and suitability for implementation within well-established, open, scientific computing tools. The finite element formulation is shown to be non-coercive, and therefore not ideal for proving existence, uniqueness, or a priori error estimates, but the numerical solutions computed with quartic Hermite elements are nevertheless found to converge to reference solutions at nearly optimal rates (O(h^4) in both L^2 and H^1 norms). Further work is required to better understand the cause of the suboptimal convergence rates, and a linear model problem which exhibits analogous characteristics is also discussed as a possible starting point for future theoretical analyses., Comment: 17 pages, 7 figures

The nonlinear Schr\"odinger/Gross-Pitaevskii (NLS/GP) equation is considered in the presence of three equally-spaced potentials. The problem is reduced to a finite-dimensional Hamiltonian system by a Galerkin truncation. Families of oscillatory orbits are sought in the neighborhoods of the system's nine branches of standing wave solutions. Normal forms are computed in the neighborhood of these branches' various Hamiltonian Hopf and saddle-node bifurcations, showing how the oscillatory orbits change as a parameter is increased. Numerical experiments show agreement between normal form theory and numerical solutions to the reduced system and NLS/GP near the Hamiltonian Hopf bifurcations and some subtle disagreements near the saddle-node bifurcations due to exponentially small terms in the asymptotics.

During the past decade, machine learning has become extremely popular and can be found in many aspects of our every day life. Nowayadays with explosion of data while rapid growth of computation capacity, Distributed Deep Neural Networks (DDNNs) which can improve their performance linearly with more computation resources, have become hot and trending. However, there has not been an in depth study of the performance of these systems, and how well they scale. In this paper we analyze CNTK, one of the most commonly used DDNNs, by first building a performance model and then evaluating the system two settings: a small cluster with all nodes in a single rack connected to a top of rack switch, and in large scale using Blue Waters with arbitary placement of nodes. Our main focus was the scalability of the system with respect to adding more nodes. Based on our results, this system has an excessive initialization overhead because of poor I/O utilization which dominates the whole execution time. Because of this, the system does not scale beyond a few nodes (4 in Blue Waters). Additionally, due to a single server-multiple worker design the server becomes a bottleneck after 16 nodes limiting the scalability of the CNTK.