Search

In this paper, we present two computational methods for performing simulations of pollution propagation described by advection-diffusion equations. The first method employs graph grammars to describe the generation process of the computational mesh used in simulations with the meshless solver of the three-dimensional finite element method. The graph transformation rules express the three-dimensional Rivara longest-edge refinement algorithm. This solver is used for an exemplary application: performing three-dimensional simulations of pollution generation by the coal-burning power plant and its propagation in the city of Longyearbyen, the capital of Spitsbergen. The second computational code is based on the Physics Informed Neural Networks method. It is used to calculate the dissipation of the pollution along the valley in which the city of Longyearbyen is located. We discuss the instantiation and execution of the PINN method using Google Colab implementation. We discuss the benefits and limitations of the PINN implementation., Comment: 34 pages, 21 figures, 1 table

Strain relaxation of thick InGaN layers was studied in order to develop technology of InGaN templates for deposition of InGaN Quantum Wells (QWs) and InGaN layers of high-In-content. In this paper, we show that InGaN layers grown on misoriented (0001)-GaN substrates relax by preferential activation of certain glide planes for misfit dislocation formation. Substrate misorientation changes resolved shear stresses, affecting the distribution of misfit dislocations within each dislocation set. We demonstrate that this mechanism leads to an anisotropic strain as well as a tilt of the InGaN layer with respect to the GaN substrate. It appears that these phenomena are more pronounced in structures grown on substrates misoriented toward <11-20> direction than corresponding structures with <-1100> misorientation. These features would influence the properties of the overgrown InGaN QWs and should be taken into consideration during designing structures grown on relaxed InGaN templates. We reveal that the lattice of partially relaxed InGaN has a triclinic deformation, thus requiring advanced XRD analysis. The presentation of just a single asymmetric reciprocal space map commonly practiced in the literature can lead to misleading information regarding the relaxation state of partially relaxed wurtzite structures., Comment: 29 pages, 6 figures, 3 tables

We present a general framework for modeling power magnetic materials characteristics using deep neural networks. Magnetic materials represented by multidimensional characteristics (that mimic measurements) are used to train the neural autoencoder model in an unsupervised manner. The encoder is trying to predict the material parameters of a theoretical model, which is then used in a decoder part. The decoder, using the predicted parameters, reconstructs the input characteristics. The neural model is trained to capture a synthetically generated set of characteristics that can cover a broad range of material behaviors, leading to a model that can generalize on the underlying physics rather than just optimize the model parameters for a single measurement. After setting up the model, we prove its usefulness in the complex problem of modeling magnetic materials in the frequency and current (out-of-linear range) domains simultaneously, for which we use measured characteristics obtained for frequency up to $10$ MHz and H-field up to saturation., Comment: 9 pages, 6 figures

Users in consumption domains, like music, are often able to more efficiently provide preferences over a set of items (e.g. a playlist or radio) than over single items (e.g. songs). Unfortunately, this is an underexplored area of research, with most existing recommendation systems limited to understanding preferences over single items. Curating an item set exponentiates the search space that recommender systems must consider (all subsets of items!): this motivates conversational approaches-where users explicitly state or refine their preferences and systems elicit preferences in natural language-as an efficient way to understand user needs. We call this task conversational item set curation and present a novel data collection methodology that efficiently collects realistic preferences about item sets in a conversational setting by observing both item-level and set-level feedback. We apply this methodology to music recommendation to build the Conversational Playlist Curation Dataset (CPCD), where we show that it leads raters to express preferences that would not be otherwise expressed. Finally, we propose a wide range of conversational retrieval models as baselines for this task and evaluate them on the dataset., Comment: Appearing in Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Recommender systems are ubiquitous yet often difficult for users to control, and adjust if recommendation quality is poor. This has motivated conversational recommender systems (CRSs), with control provided through natural language feedback. However, as with most application domains, building robust CRSs requires training data that reflects system usage$\unicode{x2014}$here conversations with user utterances paired with items that cover a wide range of preferences. This has proved challenging to collect scalably using conventional methods. We address the question of whether it can be generated synthetically, building on recent advances in natural language. We evaluate in the setting of item set recommendation, noting the increasing attention to this task motivated by use cases like music, news, and recipe recommendation. We present TalkTheWalk, which synthesizes realistic high-quality conversational data by leveraging domain expertise encoded in widely available curated item collections, generating a sequence of hypothetical yet plausible item sets, then using a language model to produce corresponding user utterances. We generate over one million diverse playlist curation conversations in the music domain, and show these contain consistent utterances with relevant item sets nearly matching the quality of an existing but small human-collected dataset for this task. We demonstrate the utility of the generated synthetic dataset on a conversational item retrieval task and show that it improves over both unsupervised baselines and systems trained on a real dataset.

Three-dimensional imaging is one of the crucial tools of modern sciences, that allows non-invasive inspection of physical objects. We propose and demonstrate a method to reconstruct a three-dimensional structure of coherence stored in an atomic ensemble. Our method relies on time-and-space resolved heterodyne measurement that allows the reconstruction of a complex three-dimensional profile of the coherence with a single measurement of the light emitted from the ensemble. Our tomographic technique provides a robust diagnostic tool for various atom-based quantum information protocols and could be applied to three-dimensional magnetometry, electrometry and imaging of electromagnetic fields., Comment: 7 pages, 7 figures - early version, comments welcome!

Entity retrieval--retrieving information about entity mentions in a query--is a key step in open-domain tasks, such as question answering or fact checking. However, state-of-the-art entity retrievers struggle to retrieve rare entities for ambiguous mentions due to biases towards popular entities. Incorporating knowledge graph types during training could help overcome popularity biases, but there are several challenges: (1) existing type-based retrieval methods require mention boundaries as input, but open-domain tasks run on unstructured text, (2) type-based methods should not compromise overall performance, and (3) type-based methods should be robust to noisy and missing types. In this work, we introduce TABi, a method to jointly train bi-encoders on knowledge graph types and unstructured text for entity retrieval for open-domain tasks. TABi leverages a type-enforced contrastive loss to encourage entities and queries of similar types to be close in the embedding space. TABi improves retrieval of rare entities on the Ambiguous Entity Retrieval (AmbER) sets, while maintaining strong overall retrieval performance on open-domain tasks in the KILT benchmark compared to state-of-the-art retrievers. TABi is also robust to incomplete type systems, improving rare entity retrieval over baselines with only 5% type coverage of the training dataset. We make our code publicly available at https://github.com/HazyResearch/tabi., Comment: Accepted to Findings of ACL 2022

This doctoral thesis focuses on the issue of phase matching in a quantum memory operating on a cold cloud of $^{87}$Rb atoms, where non-resonant Raman scattering is used as an interface between light and atoms. Experimental results and considerations for controlling the memory readout process by using the Zeeman effect, ac-Stark effect, controlling the laser beam geometry, or using an optical ring cavity are presented., Comment: in Polish language

Named entity disambiguation (NED), which involves mapping textual mentions to structured entities, is particularly challenging in the medical domain due to the presence of rare entities. Existing approaches are limited by the presence of coarse-grained structural resources in biomedical knowledge bases as well as the use of training datasets that provide low coverage over uncommon resources. In this work, we address these issues by proposing a cross-domain data integration method that transfers structural knowledge from a general text knowledge base to the medical domain. We utilize our integration scheme to augment structural resources and generate a large biomedical NED dataset for pretraining. Our pretrained model with injected structural knowledge achieves state-of-the-art performance on two benchmark medical NED datasets: MedMentions and BC5CDR. Furthermore, we improve disambiguation of rare entities by up to 57 accuracy points., Comment: Accepted to Findings of EMNLP 2021

The industrial machine learning pipeline requires iterating on model features, training and deploying models, and monitoring deployed models at scale. Feature stores were developed to manage and standardize the engineer's workflow in this end-to-end pipeline, focusing on traditional tabular feature data. In recent years, however, model development has shifted towards using self-supervised pretrained embeddings as model features. Managing these embeddings and the downstream systems that use them introduces new challenges with respect to managing embedding training data, measuring embedding quality, and monitoring downstream models that use embeddings. These challenges are largely unaddressed in standard feature stores. Our goal in this tutorial is to introduce the feature store system and discuss the challenges and current solutions to managing these new embedding-centric pipelines.

Existing super-resolution methods of optical imaging hold a solid place as an application in natural sciences, but many new developments allow for beating the diffraction limit in a more subtle way. One of the recently explored strategies to fully exploit information already present in the field is to perform a quantum-inspired tailored measurements. Here we exploit the full spectral information of the optical field in order to beat the Rayleigh limit in spectroscopy. We employ an optical quantum memory with spin-wave storage and an embedded processing capability to implement a time-inversion interferometer for input light, projecting the optical field in the symmetric-antisymmetric mode basis. Our tailored measurement achieves a resolution of 15 kHz and requires 20 times less photons than a corresponding Rayleigh-limited conventional method. We demonstrate the advantage of our technique over both conventional spectroscopy and heterodyne measurements, showing potential for application in distinguishing ultra-narrowband emitters, optical communication channels, or signals transduced from lower-frequency domains., Comment: 17 pages, 9 figures, 1 table

In this work, we study the emergence of polarization doping in AlxGa1-xN layers with graded composition from a theoretical viewpoint. We demonstrate that the charge conservation law applies for fixed and mobile charges separately, leading to nonlocal compensation phenomena involving bulk fixed and mobile charge and polarization sheet charge at the heterointerfaces. The magnitude of the effect allows obtaining technically viable mobile charge density for optoelectronic devices without impurity doping (donors or acceptors). Therefore, it provides an additional tool for the device designer, with the potential to attain high conductivities: high carrier concentrations can be obtained even in materials with high dopant ionization energies, and the mobility is not limited by scattering at ionized impurities., Comment: 16 pages, 4 figures

High-rate generation of hybrid photon-matter entanglement remains a fundamental building block of quantum network architectures enabling protocols such as quantum secure communication or quantum distributed computing. While a tremendous effort has been made to overcome technological constraints limiting the efficiency and coherence times of current systems, an important complementary approach is to employ parallel and multiplexed architectures. Here we follow this approach experimentally demonstrating the generation of bipartite polarization-entangled photonic states across more than 500 modes, with a programmable delay for the second photon enabled by qubit storage in a wavevector multiplexed cold-atomic quantum memory. We demonstrate Clauser, Horne, Shimony, Holt inequality violation by over 3 standard deviations, lasting for at least 45 {\mu}s storage time for half of the modes. The ability to shape hybrid entanglement between the polarization and wavevector degrees of freedom provides not only multiplexing capabilities but also brings prospects for novel protocols.

Modern neural network architectures use structured linear transformations, such as low-rank matrices, sparse matrices, permutations, and the Fourier transform, to improve inference speed and reduce memory usage compared to general linear maps. However, choosing which of the myriad structured transformations to use (and its associated parameterization) is a laborious task that requires trading off speed, space, and accuracy. We consider a different approach: we introduce a family of matrices called kaleidoscope matrices (K-matrices) that provably capture any structured matrix with near-optimal space (parameter) and time (arithmetic operation) complexity. We empirically validate that K-matrices can be automatically learned within end-to-end pipelines to replace hand-crafted procedures, in order to improve model quality. For example, replacing channel shuffles in ShuffleNet improves classification accuracy on ImageNet by up to 5%. K-matrices can also simplify hand-engineered pipelines -- we replace filter bank feature computation in speech data preprocessing with a learnable kaleidoscope layer, resulting in only 0.4% loss in accuracy on the TIMIT speech recognition task. In addition, K-matrices can capture latent structure in models: for a challenging permuted image classification task, a K-matrix based representation of permutations is able to learn the right latent structure and improves accuracy of a downstream convolutional model by over 9%. We provide a practically efficient implementation of our approach, and use K-matrices in a Transformer network to attain 36% faster end-to-end inference speed on a language translation task., Comment: International Conference on Learning Representations (ICLR) 2020 spotlight

A challenge for named entity disambiguation (NED), the task of mapping textual mentions to entities in a knowledge base, is how to disambiguate entities that appear rarely in the training data, termed tail entities. Humans use subtle reasoning patterns based on knowledge of entity facts, relations, and types to disambiguate unfamiliar entities. Inspired by these patterns, we introduce Bootleg, a self-supervised NED system that is explicitly grounded in reasoning patterns for disambiguation. We define core reasoning patterns for disambiguation, create a learning procedure to encourage the self-supervised model to learn the patterns, and show how to use weak supervision to enhance the signals in the training data. Encoding the reasoning patterns in a simple Transformer architecture, Bootleg meets or exceeds state-of-the-art on three NED benchmarks. We further show that the learned representations from Bootleg successfully transfer to other non-disambiguation tasks that require entity-based knowledge: we set a new state-of-the-art in the popular TACRED relation extraction task by 1.0 F1 points and demonstrate up to 8% performance lift in highly optimized production search and assistant tasks at a major technology company

Tungsten carbide (WC) contacts have been investigated as a novel gold-free Schottky metallization for AlGaN/GaN heterostructures. The evolution of the electrical and structural/compositional properties of the WC/AlGaN contact has been monitored as a function of the annealing temperature in the range from 400 to 800{\deg}C. The Schottky barrier height ($\Phi$B) at WC/AlGaN interface, extracted from the forward current-voltage characteristics of the diode, decreased from 0.8 eV in the as-deposited and 400{\deg}C annealed sample, to 0.56 eV after annealing at 800 {\deg}C. This large reduction of $\Phi$B was accompanied by a corresponding increase of the reverse bias leakage current. Transmission electron microscopy coupled to electron energy loss spectroscopy analyses revealed the presence of oxygen (O) uniformly distributed in the WC layer, both in the as-deposited and 400{\deg}C annealed sample. Conversely, oxygen accumulation in a 2-3 nm thin W-O-C layer at the interface with AlGaN was observed after the annealing at 800 {\deg}C, as well as the formation of W2C grains within the film (confirmed by X-ray diffraction analyses). The formation of this interfacial W-O-C layer is plausibly the main origin of the decreased $\Phi$B and the increased leakage current in the 800{\deg}C annealed Schottky diode, whereas the decreased O content inside the WC film can explain the reduced resistivity of the metal layer. The results provide an assessment of the processing conditions for the application of WC as Schottky contact for AlGaN/GaN heterostructures.

Many industrial machine learning (ML) systems require frequent retraining to keep up-to-date with constantly changing data. This retraining exacerbates a large challenge facing ML systems today: model training is unstable, i.e., small changes in training data can cause significant changes in the model's predictions. In this paper, we work on developing a deeper understanding of this instability, with a focus on how a core building block of modern natural language processing (NLP) pipelines---pre-trained word embeddings---affects the instability of downstream NLP models. We first empirically reveal a tradeoff between stability and memory: increasing the embedding memory 2x can reduce the disagreement in predictions due to small changes in training data by 5% to 37% (relative). To theoretically explain this tradeoff, we introduce a new measure of embedding instability---the eigenspace instability measure---which we prove bounds the disagreement in downstream predictions introduced by the change in word embeddings. Practically, we show that the eigenspace instability measure can be a cost-effective way to choose embedding parameters to minimize instability without training downstream models, outperforming other embedding distance measures and performing competitively with a nearest neighbor-based measure. Finally, we demonstrate that the observed stability-memory tradeoffs extend to other types of embeddings as well, including knowledge graph and contextual word embeddings., Comment: In Proceedings of the 3rd MLSys Conference, 2020

We examined the solar gravitational lens (SGL) as the means to produce direct high-resolution, multipixel images of exoplanets. The properties of the SGL are remarkable: it offers maximum light amplification of ~1e11 and angular resolution of ~1e-10 arcsec. A probe with a 1-m telescope in the SGL focal region can image an exoplanet at 30 pc with 10-kilometer resolution on its surface, sufficient to observe seasonal changes, oceans, continents, surface topography. We reached and exceeded all objectives set for our study: We developed a new wave-optical approach to study the imaging of exoplanets while treating them as extended, resolved, faint sources at large but finite distances. We properly accounted for the solar corona brightness. We developed deconvolution algorithms and demonstrated the feasibility of high-quality image reconstruction under realistic conditions. We have proven that multipixel imaging and spectroscopy of exoplanets with the SGL are feasible. We have developed a new mission concept that delivers an array of optical telescopes to the SGL focal region relying on three innovations: i) a new way to enable direct exoplanet imaging, ii) use of smallsats solar sails fast transit through the solar system and beyond, iii) an open architecture to take advantage of swarm technology. This approach enables entirely new missions, providing a great leap in capabilities for NASA and the greater aerospace community. Our results are encouraging as they lead to a realistic design for a mission that will be able to make direct resolved images of exoplanets in our stellar neighborhood. It could allow exploration of exoplanets relying on the SGL capabilities decades, if not centuries, earlier than possible with other extant technologies. The architecture and mission concepts for a mission to the strong interference region of the SGL are promising and should be explored further., Comment: Final Report for the NASA's Innovative Advanced Concepts (NIAC) Phase II proposal. 91 pages, 50 figures, 5 tables

Plenty of quantum information protocols are enabled by manipulation and detection of photonic spectro-temporal degrees of freedom via light-matter interfaces. While present implementations are well suited for high-bandwidth photon sources such as quantum dots, they lack the high resolution required for intrinsically narrow-band light-atom interactions. Here, we demonstrate far-field temporal imaging based on ac-Stark spatial spin-wave phase manipulation in a multimode gradient echo memory. We achieve spectral resolution of 20 kHz with MHz-level bandwidth and ultra-low noise equivalent to 0.023 photons, enabling operation in the single-quantum regime., Comment: 10 pages, 9 figures

Atomic-ensemble spin waves carrying single-photon Fock states exhibit nonclassical many-body correlations in-between atoms. The same correlations are inherently associated with single-photon superradiance, forming the basis of a plethora of quantum light-matter interfaces. We devise a scheme allowing the preparation of spatially-structured superradiant states in the atomic two-photon cascade using spin-wave light storage. We thus show that long-lived atomic ground-state spin waves can be converted to photon pairs opening the way towards nonlinear optics of spin waves via multi-wave mixing processes., Comment: 9 pages, 6 figures

Memory is increasingly often the bottleneck when training neural network models. Despite this, techniques to lower the overall memory requirements of training have been less widely studied compared to the extensive literature on reducing the memory requirements of inference. In this paper we study a fundamental question: How much memory is actually needed to train a neural network? To answer this question, we profile the overall memory usage of training on two representative deep learning benchmarks -- the WideResNet model for image classification and the DynamicConv Transformer model for machine translation -- and comprehensively evaluate four standard techniques for reducing the training memory requirements: (1) imposing sparsity on the model, (2) using low precision, (3) microbatching, and (4) gradient checkpointing. We explore how each of these techniques in isolation affects both the peak memory usage of training and the quality of the end model, and explore the memory, accuracy, and computation tradeoffs incurred when combining these techniques. Using appropriate combinations of these techniques, we show that it is possible to the reduce the memory required to train a WideResNet-28-2 on CIFAR-10 by up to 60.7x with a 0.4% loss in accuracy, and reduce the memory required to train a DynamicConv model on IWSLT'14 German to English translation by up to 8.7x with a BLEU score drop of 0.15., Comment: Version notes: Copyedits and citation fixes

Utilization of the spatial degree of freedom vastly enhances informational capacity of light at the cost of stringent requirements on the processing devices. Multi-mode quantum memories constitute a viable candidate for quantum and classical information processing; however, full utilization of the assets of high-dimensionality requires a flexible processing technique. We employ a spatially varying ac-Stark effect to perform arbitrary 1D phase modulation of a coherent spin-wave state stored in a wavevector-multiplexed quantum memory. A far-field and an interferometric near-field characterizations of the introduced phase profiles are presented. Additionally, coherence between temporally separated partial readouts of a single coherent spin-wave state is demonstrated, offering possible applications in adaptive measurements via conditional spin-wave modulation., Comment: 9 pages, 6 figures

A device being a pinnacle of development of an optical quantum memory should combine the capabilities of storage, inter-communication and processing of stored information. In particular, the ability to capture a train of optical pulses, interfere them in an arbitrary way and finally perform on-demand release would in a loose sense realize an optical analogue of a Turing Machine. Here we demonstrate the operation of an optical quantum memory being able to store optical pulses in the form of collective spin-wave excitations in a multi-dimensional wavevector space. During storage, we perform complex beamsplitter operations and demonstrate a variety of protocol implemented as the processing stage, including interfering a pair of spin-wave modes with 95\% visibility. By engineering the phase-matching at the readout stage we realize the on-demand retrieval. The highly multimode structure of the presented quantum memory lends itself both to enhancing classical optical telecommunication as well as parallel processing of optical qubits at the single-photon level., Comment: 11 pages, 7 figures

We bring the set of linear quantum operations, important for many fundamental studies in photonic systems, to the material domain of collective excitations known as spin waves. Using the ac Stark effect we realize quantum operations on single excitations and demonstrate a spin-wave analogue of Hong-Ou-Mandel effect, realized via a beamsplitter implemented in the spin wave domain. Our scheme equips atomic-ensemble-based quantum repeaters with quantum information processing capability and can be readily brought to other physical systems, such as doped crystals or room-temperature atomic ensembles., Comment: 6 pages + 12 pages supplemental, 5 figures + 13 supplementary figures

The parent compound of high-Tc superconducting cuprates is a unique Mott state consisting of layers of spin-1/2 ions arranged on a square lattice and with a record high antiferromagnetic coupling within the layers. Compounds with similar characteristics have long been searched for. Nickelates and iridates had been proposed as cuprate analogs but so far have not reached a satisfactory similarity. Here we use a combination of experimental and theoretical tools to show that the commercial compound AgF2 is an excellent cuprate analog with remarkably similar electronic parameters to La2CuO4 but larger buckling of planes. Two-magnon Raman scattering reveals a superexchange constant which reaches 70% of that of a typical cuprate. We argue that structures that reduce or eliminate the buckling of the AgF2 planes could have an antiferromagnetic coupling that matches or surpasses the cuprates., Comment: 15 pages, 3 figures plus Supplementary Information in ancillary folder

Low-precision computation is often used to lower the time and energy cost of machine learning, and recently hardware accelerators have been developed to support it. Still, it has been used primarily for inference - not training. Previous low-precision training algorithms suffered from a fundamental tradeoff: as the number of bits of precision is lowered, quantization noise is added to the model, which limits statistical accuracy. To address this issue, we describe a simple low-precision stochastic gradient descent variant called HALP. HALP converges at the same theoretical rate as full-precision algorithms despite the noise introduced by using low precision throughout execution. The key idea is to use SVRG to reduce gradient variance, and to combine this with a novel technique called bit centering to reduce quantization error. We show that on the CPU, HALP can run up to $4 \times$ faster than full-precision SVRG and can match its convergence trajectory. We implemented HALP in TensorQuant, and show that it exceeds the validation performance of plain low-precision SGD on two deep learning tasks.

Effective and unrestricted engineering of atom-photon interactions requires precise spatially-resolved control of light beams. The significant potential of such manipulations lies in a set of disciplines ranging from solid state to atomic physics. Here we use a Zeeman-like ac-Stark shift of a shaped laser beam to perform rotations of spins with spatial resolution in a large ensemble of cold rubidium atoms. We show that inhomogeneities of light intensity are the main source of dephasing and thus decoherence, yet with proper beam shaping this deleterious effect is strongly mitigated allowing rotations of 15 rad within one spin-precession lifetime. Finally, as a particular example of a complex manipulation enabled by our scheme, we demonstrate a range of collapse-and-revival behaviours of a free-induction decay signal by imprinting comb-like patterns on the atomic ensemble., Comment: 4 pages, 4 figures

The advent of structured, high-dimensional entangled states brings new possibilities for quantum imaging, information processing and quantum key distribution. We experimentally generate and characterize a spatially entangled state stored in a quantum memory system, using entropic EPR-steering inequality, yielding genuine violation of $1.06\pm0.15$ bits and certifying entanglement of formation of at least $0.70\pm0.15$ ebits. The supremacy of the entropic witness is demonstrated for a wide class of experimentally available states, giving prospects for EPR-steering applications in noisy systems or with lossy quantum channels., Comment: 7 pages, 6 figures (with Supplemental Material)