Search

Gradient descent and backpropagation have enabled neural networks to achieve remarkable results in many real-world applications. Despite ongoing success, training a neural network with gradient descent can be a slow and strenuous affair. We present a simple yet faster training algorithm called Zeroth-Order Relaxed Backpropagation (ZORB). Instead of calculating gradients, ZORB uses the pseudoinverse of targets to backpropagate information. ZORB is designed to reduce the time required to train deep neural networks without penalizing performance. To illustrate the speed up, we trained a feed-forward neural network with 11 layers on MNIST and observed that ZORB converged 300 times faster than Adam while achieving a comparable error rate, without any hyperparameter tuning. We also broaden the scope of ZORB to convolutional neural networks, and apply it to subsamples of the CIFAR-10 dataset. Experiments on standard classification and regression benchmarks demonstrate ZORB's advantage over traditional backpropagation with Gradient Descent., Comment: To appear in "Beyond Backpropagation - Novel Ideas for Training Neural Architectures" Workshop at NeurIPS 2020

Hypothesis: Aging in colloidal suspensions manifests as a reduction in kinetic freedom of the colloids. In aqueous suspensions of charged colloids, the role of inter-particle electrostatics interactions on the aging dynamics is well debated. Despite water being the dispersion medium, the influence of water structure on the physicochemical properties of aging colloids has never been considered before. Laponite, a model hectorite clay, could be used to evaluate the relative contributions of medium structure and electrostatics in determining the physicochemical properties of aging colloidal suspensions. Experiments: The structure of the dispersion medium is modified either by incorporating uncharged/charged kosmotropic (structure-inducing) or chaotropic (structure-disrupting) molecules or by changing suspension temperature. A new protocol, wherein the medium is heated before adding clay particles, is also introduced to evaluate the effects of hydrogen bond disruptions on suspension aging. Dynamic light scattering, rheological measurements and particle-scale imaging are employed to evaluate the physicochemical properties of the suspensions. Findings: A strong influence of medium structure is evident when inter-particle electrostatic interactions are weak. Enhancement and disruption of hydrogen bonds in the medium are, respectively, strongly correlated with acceleration and delay of suspension aging dynamics. The physicochemical properties of charged clay colloidal suspensions are therefore controlled by altering hydrogen bonding in the dispersion medium., Comment: 25 pages, 6 figures

We present the properties of the inverse Evershed flow (IEF) based on the center-to-limb variation of the plasma speed and loop geometry of chromospheric superpenumbral fibrils in eleven sunspots that were located at a wide range of heliocentric angles from 12 to 79 deg. The observations were acquired at the Dunn Solar Telescope in the spectral lines of Halpha at 656nm, CaII IR at 854 nm and HeI at 1083 nm. All sunspots display opposite line-of-sight (LOS) velocities on the limb and center side with a distinct shock signature near the outer penumbral edge. We developed a simplified flexible sunspot model assuming axisymmetry and prescribing the radial flow speed profile at a known loop geometry to replicate the observed two-dimensional IEF patterns under different viewing angles. The simulated flow maps match the observations for chromospheric loops with 10-20 Mm length starting at 0.8-1.1 sunspot radii, an apex height of 2-3Mm and a true constant flow speed of 2-9km/s. We find on average a good agreement of the simulated velocities and the observations on elliptical annuli around the sunspot. Individual IEF channels show a significant range of variation in their properties and reach maximal LOS speeds of up to 12km/s. Upwards or downwards directed flows do not show a change of sign in the LOS velocities for heliocentric angles above 30 deg. Our results are consistent with the IEF being caused by a siphon flow mechanism driving a flow at a constant sonic speed along elevated loops with a flattened top in the chromosphere., Comment: 14 pages, 12 figures; accepted for publication by ApJ

Memristive devices have drawn considerable research attention due to their potential applications in non-volatile memory and neuromorphic computing. The combination of resistive switching devices with light-responsive materials is considered a novel way to integrate optical information with electrical circuitry. On the other hand, 2D materials have attracted substantial consideration thank to their unique crystal structure, as reflected in their chemical and physical properties. Although not the major focus, van der Waals solids were proven to be potential candidates in memristive devices. In this scheme, the majority of the resistive switching devices were implemented on planar flakes, obtained by mechanical exfoliation. Here we utilize a facile and robust methodology to grow large-scale vertically aligned MoS$_2$ (VA-MoS$_2$) films on standard silicon substrates. Memristive devices with the structure silver/VA-MoS$_2$/Si are shown to have low set-ON voltages (<0.5V), large-retention times ($>2\times10^4$ s) and high thermal stability (up to 350 $^\circ$C). The proposed memristive device also exhibits long term potentiation / depression (LTP/LTD) and photo-active memory states. The large-scale fabrication, together with the low operating voltages, high thermal stability, light-responsive behaviour and long-term potentiation/depression, makes this approach very appealing for real-life non-volatile memory applications., Comment: Advanced Functional Materials - In Press 18 pages, 7 figures

Metal organic chemical vapor deposition (MOCVD) is one of the main methodologies used for thin film fabrication in the semiconductor industry today and is considered one of the most promising routes to achieve large-scale and high-quality 2D transition metal dichalcogenides (TMDCs). However, if not taken special measures, MOCVD suffers from some serious drawbacks, such as small domain size and carbon contamination, resulting in poor optical and crystal quality, which may inhibit its implementation for the large-scale fabrication of atomic-thin semiconductors. Here we present a Growth-Etch MOCVD (GE-MOCVD) methodology, in which a small amount of water vapor is introduced during the growth, while the precursors are delivered in pulses. The evolution of the growth as a function of the amount of water vapor, the number and type of cycles and the gas composition is described. We show a significant domain size increase is achieved relative to our conventional process. The improved crystal quality of WS2 (and WSe2) domains was demonstrated by means of Raman spectroscopy, photoluminescence (PL) spectroscopy and HRTEM studies. Moreover, time-resolved PL studies show very long exciton lifetimes, comparable to those observed in mechanically exfoliated flakes. Thus, the GE-MOCVD approach presented here may facilitate their integration into a wide range of applications., Comment: 29 pages, 6 figures

Many applications involve data with qualitative and quantitative responses. When there is an association between the two responses, a joint model will provide improved results than modeling them separately. In this paper, we propose a Bayesian method to jointly model such data. The joint model links the qualitative and quantitative responses and can assess their dependency strength via a latent variable. The posterior distributions of parameters are obtained through an efficient MCMC sampling algorithm. The simulation shows that the proposed method can improve the prediction capacity for both responses. We apply the proposed joint model to the birth records data acquired by the Virginia Department of Health and study the mutual dependence between preterm birth of infants and their birth weights., Comment: 27 pages, 3 figures. 3 tables

Program execution speed critically depends on increasing cache hits, as cache hits are orders of magnitude faster than misses. To increase cache hits, we focus on the problem of cache replacement: choosing which cache line to evict upon inserting a new line. This is challenging because it requires planning far ahead and currently there is no known practical solution. As a result, current replacement policies typically resort to heuristics designed for specific common access patterns, which fail on more diverse and complex access patterns. In contrast, we propose an imitation learning approach to automatically learn cache access patterns by leveraging Belady's, an oracle policy that computes the optimal eviction decision given the future cache accesses. While directly applying Belady's is infeasible since the future is unknown, we train a policy conditioned only on past accesses that accurately approximates Belady's even on diverse and complex access patterns, and call this approach Parrot. When evaluated on 13 of the most memory-intensive SPEC applications, Parrot increases cache miss rates by 20% over the current state of the art. In addition, on a large-scale web search benchmark, Parrot increases cache hit rates by 61% over a conventional LRU policy. We release a Gym environment to facilitate research in this area, as data is plentiful, and further advancements can have significant real-world impact., Comment: International Conference on Machine Learning (ICML), 2020

A significant effort has been made to train neural networks that replicate algorithmic reasoning, but they often fail to learn the abstract concepts underlying these algorithms. This is evidenced by their inability to generalize to data distributions that are outside of their restricted training sets, namely larger inputs and unseen data. We study these generalization issues at the level of numerical subroutines that comprise common algorithms like sorting, shortest paths, and minimum spanning trees. First, we observe that transformer-based sequence-to-sequence models can learn subroutines like sorting a list of numbers, but their performance rapidly degrades as the length of lists grows beyond those found in the training set. We demonstrate that this is due to attention weights that lose fidelity with longer sequences, particularly when the input numbers are numerically similar. To address the issue, we propose a learned conditional masking mechanism, which enables the model to strongly generalize far outside of its training range with near-perfect accuracy on a variety of algorithms. Second, to generalize to unseen data, we show that encoding numbers with a binary representation leads to embeddings with rich structure once trained on downstream tasks like addition or multiplication. This allows the embedding to handle missing data by faithfully interpolating numbers not seen during training., Comment: Accepted at 34th Conference on Neural Information Processing Systems (NeurIPS 2020)

Let $X$ be a smooth threefold with a simple normal crossings divisor $D$. We construct the Donaldson-Thomas theory of the pair $(X|D)$ enumerating ideal sheaves on $X$ relative to $D$. These moduli spaces are compactified by studying subschemes in expansions of the target geometry, and the moduli space carries a virtual fundamental class leading to numerical invariants with expected properties. We formulate punctual evaluation, rationality and wall-crossing conjectures, in parallel with the standard theory. Our formalism specializes to the Li-Wu theory of relative ideal sheaves when the divisor is smooth, and is parallel to recent work on logarithmic Gromov-Witten theory with expansions., Comment: v4: 71 pages. Expository changes. Final version to appear in Forum of Mathematics, Pi

We report investigations of the structural, magnetic, electrical transport and thermal properties of five compositions of the metallic perovskite GdRh$_3$C$_{1-x}$B$_x$ ($0.00 \le x \le 1.00$). Our results show that all the five compositions undergo magnetic ordering at low temperatures, but the nature of the ordered state is significantly different in the carbon- and the boron-rich compositions, where the former shows signatures of an amplitude-modulated magnetic structure and the latter exhibits evidences of an equal-moment incommensurate antiferromagnetic ordering. We also observe a remarkable field-dependent evolution of conduction carrier polarization in the compositionally disordered compounds. The outcomes indicate that this system is energetically situated in proximity to a magnetic instability where small variations in the control parameter(s), such as lattice constant and/or electron density, lead to considerably different ground states., Comment: 10 pages and 9 figures

A comparative study of the non-equilibrium magnetic phenomena (magnetic blocking, memory, exchange bias and aging effect) has been presented for ferrimagnetic Co$_{1.75}$Fe$_{1.25}$O$_4$ (CFO) and its composite with non-magnetic BaTiO$_3$ (BTO). Synchrotron X-Ray diffraction patterns have confirmed coexistence of CFO and BTO structures in composite, but magnetic spin dynamics have been remarkably modified. The blocking phenomenon of ferrimagnetic domains below the room temperature has been studied by different modes of (zero field cooled and field cooled) magnetic measurements in collaboration with magnetic fields ON and OFF modes and time dependent magnetization. The applications of unconventional protocols during time dependent magnetization measurement at different stages of the temperature and field dependence of the magnetization curves have been useful to reveal the non-equilibrium dynamics of magnetic spin order. The applying of off-field relaxation experiments has made possible to tune the magnetic state and coercivity of the systems. The role of interfacial coupling between magnetic and non-magnetic particles has been understood on different magnetic phenomena (meta-stable magnetic state, exchange bias and memory effect) by comparing the experimental results of Co$_{1.75}$Fe$_{1.25}$O$_4$ spinel oxide and its composite with BaTiO$_3$ particles.

Proteins and nucleic acids can spontaneously self-assemble into membraneless droplet-like compartments, both in vitro and in vivo. A key component of these droplets are multi-valent proteins that possess several adhesive domains with specific interaction partners (whose number determines total valency of the protein) separated by disordered regions. Here, using multi-scale simulations we show that such proteins self-organize into micro-phase separated droplets of various sizes as opposed to the Flory-like macro-phase separated equilibrium state of homopolymers or equilibrium physical gels. We show that the micro-phase separated state is a dynamic outcome of the interplay between two competing processes: a diffusion-limited encounter between proteins, and the dynamics within small clusters that results in exhaustion of available valencies whereby all specifically interacting domains find their interacting partners within smaller clusters, leading to arrested phase separation. We first model these multi-valent chains as bead-spring polymers with multiple adhesive domains separated by semi-flexible linkers and use Langevin Dynamics (LD) to assess how key timescales depend on the molecular properties of associating polymers. Using the time-scales from LD simulations, we develop a coarse-grained kinetic model to study this phenomenon at longer times. Consistent with LD simulations, the macro-phase separated state was only observed at high concentrations and large interaction valencies. Further, in the regime where cluster sizes approach macro-phase separation, the condensed phase becomes dynamically solid-like, suggesting that it might no longer be biologically functional. Therefore, the micro-phase separated state could be a hallmark of functional droplets formed by proteins with the sticker-spacer architecture.