Search

Safety has been recognized as the central obstacle to preventing the use of reinforcement learning (RL) for real-world applications. Different methods have been developed to deal with safety concerns in RL. However, learning reliable RL-based solutions usually require a large number of interactions with the environment. Likewise, how to improve the learning efficiency, specifically, how to utilize transfer learning for safe reinforcement learning, has not been well studied. In this work, we propose an adaptive aggregation framework for safety-critical control. Our method comprises two key techniques: 1) we learn to transfer the safety knowledge by aggregating the multiple source tasks and a target task through the attention network; 2) we separate the goal of improving task performance and reducing constraint violations by utilizing a safeguard. Experiment results demonstrate that our algorithm can achieve fewer safety violations while showing better data efficiency compared with several baselines.

Chern numbers are gaining traction as they characterize topological phases in various physical systems. However, the resilience of the system topology to external perturbations makes it challenging to experimentally investigate transitions between different phases. In this study, we demonstrate the transitions of Chern number from 0 to 3, synthesized in an electronic-nuclear spin system associated with the nitrogen-vacancy (NV) centre in diamond. The Chern number is characterized by the number of degeneracies enclosed in a control Hamiltonian parameter sphere. The topological transitions between different phases are depicted by varying the radius and offset of the sphere. We show that the measured topological phase diagram is not only consistent with the numerical calculations but can also be mapped onto an interacting three-qubit system. The NV system may also allow access to even higher Chern numbers, which can be applied to exploring exotic topology or topological quantum information.

The building sector has been recognized as one of the primary sectors for worldwide energy consumption. Improving the energy efficiency of the building sector can help reduce the operation cost and reduce the greenhouse gas emission. The energy management system (EMS) can monitor and control the operations of built-in appliances in buildings, so an efficient EMS is of crucial importance to improve the building operation efficiency and maintain safe operations. With the growing penetration of renewable energy and electrical appliances, increasing attention has been paid to the development of intelligent building EMS. Recently, reinforcement learning (RL) has been applied for building EMS and has shown promising potential. However, most of the current RL-based EMS solutions would need a large amount of data to learn a reliable control policy, which limits the applicability of these solutions in the real world. In this work, we propose MetaEMS, which can help achieve better energy management performance with the benefits of RL and meta-learning. Experiment results showcase that our proposed MetaEMS can adapt faster to environment changes and perform better in most situations compared with other baselines., Comment: arXiv admin note: text overlap with arXiv:1909.10165 by other authors

Vision-Language Navigation requires the agent to follow natural language instructions to reach a specific target. The large discrepancy between seen and unseen environments makes it challenging for the agent to generalize well. Previous studies propose data augmentation methods to mitigate the data bias explicitly or implicitly and provide improvements in generalization. However, they try to memorize augmented trajectories and ignore the distribution shifts under unseen environments at test time. In this paper, we propose an Unseen Discrepancy Anticipating Vision and Language Navigation (DAVIS) that learns to generalize to unseen environments via encouraging test-time visual consistency. Specifically, we devise: 1) a semi-supervised framework DAVIS that leverages visual consistency signals across similar semantic observations. 2) a two-stage learning procedure that encourages adaptation to test-time distribution. The framework enhances the basic mixture of imitation and reinforcement learning with Momentum Contrast to encourage stable decision-making on similar observations under a joint training stage and a test-time adaptation stage. Extensive experiments show that DAVIS achieves model-agnostic improvement over previous state-of-the-art VLN baselines on R2R and RxR benchmarks. Our source code and data are in supplemental materials.

Building energy management is one of the core problems in modern power grids to reduce energy consumption while ensuring occupants' comfort. However, the building energy management system (BEMS) is now facing more challenges and uncertainties with the increasing penetration of renewables and complicated interactions between humans and buildings. Classical model predictive control (MPC) has shown its capacity to reduce building energy consumption, but it suffers from labor-intensive modelling and complex on-line control optimization. Recently, with the growing accessibility to the building control and automation data, data-driven solutions have attracted more research interest. This paper presents a compact review of the recent advances in data-driven MPC and reinforcement learning based control methods for BEMS. The main challenges in these approaches and insights on the selection of a control method are discussed.

The electron-hole plasma in charge-neutral graphene is predicted to realize a quantum critical system whose transport features a universal hydrodynamic description, even at room temperature. This quantum critical "Dirac fluid" is expected to have a shear viscosity close to a minimum bound, with an inter-particle scattering rate saturating at the Planckian time $\hbar/(k_B T)$. While electrical transport measurements at finite carrier density are consistent with hydrodynamic electron flow in graphene, a "smoking gun" of viscous behavior remains elusive. In this work, we directly image viscous Dirac fluid flow in graphene at room temperature via measurement of the associated stray magnetic field. Nanoscale magnetic imaging is performed using quantum spin magnetometers realized with nitrogen vacancy (NV) centers in diamond. Scanning single-spin and wide-field magnetometry reveals a parabolic Poiseuille profile for electron flow in a graphene channel near the charge neutrality point, establishing the viscous transport of the Dirac fluid. This measurement is in contrast to the conventional uniform flow profile imaged in an Ohmic conductor. Via combined imaging-transport measurements, we obtain viscosity and scattering rates, and observe that these quantities are comparable to the universal values expected at quantum criticality. This finding establishes a nearly-ideal electron fluid in neutral graphene at room temperature. Our results pave the way to study hydrodynamic transport in quantum critical fluids relevant to strongly-correlated electrons in high-$T_c$ superconductors. This work also highlights the capability of quantum spin magnetometers to probe correlated-electronic phenomena at the nanoscale., Comment: Author list and title have been updated in published version

Magnetometry is a powerful technique for the non-invasive study of biological and physical systems. A key challenge lies in the simultaneous optimization of magnetic field sensitivity and maximum field range. In interferometry-based magnetometry, a quantum two-level system acquires a dynamic phase in response to an applied magnetic field. However, due to the 2{\pi} periodicity of the phase, increasing the coherent interrogation time to improve sensitivity results in reduced field range. Here we introduce a route towards both large magnetic field range and high sensitivity via measurements of the geometric phase acquired by a quantum two-level system. We experimentally demonstrate geometric-phase magnetometry using the optically addressable electronic spin associated with the nitrogen vacancy (NV) color center in diamond. Our approach enables unwrapping of the 2{\pi} phase ambiguity, decoupling of magnetic field range from sensitivity, and enhancement of the field range by about 400 times. We also find additional improvement in sensitivity in the nonadiabatic regime, and study how geometric-phase decoherence depends on adiabaticity. Our results show that the geometric phase can be a versatile tool for quantum sensing applications., Comment: 14 pages, 4 figures

The spin chemical potential characterizes the tendency of spins to diffuse. Probing the spin chemical potential could provide insight into materials such as magnetic insulators and spin liquids and aid optimization of spintronic devices. Here, we introduce single-spin magnetometry as a generic platform for non-perturbative, nanoscale characterization of spin chemical potentials. We use this platform to investigate magnons in a magnetic insulator, surprisingly finding that the magnon chemical potential can be efficiently controlled by driving the system's ferromagnetic resonance. We introduce a symmetry-based two-fluid theory describing the underlying magnon processes, realize the first experimental determination of the local thermomagnonic torque, and illustrate the detection sensitivity using electrically controlled spin injection. Our results open the way for nanoscale control and imaging of spin transport in mesoscopic spin systems., Comment: 18 pages, 4 figures

Nanodiamonds are of interest as nontoxic substrates for targeted drug delivery and as highly biostable fluorescent markers for cellular tracking. Beyond optical techniques however, options for noninvasive imaging of nanodiamonds in vivo are severely limited. Here, we demonstrate that the Overhauser effect, a proton-electron double resonance technique developed to detect free radicals in vivo, can enable high contrast magnetic resonance imaging (MRI) of nanodiamonds in water at room temperature and ultra-low magnetic field. The technique transfers spin polarization from paramagnetic impurities at nanodiamond surfaces to $^1$H spins in the surrounding water solution, creating MRI contrast on-demand. We further examine the conditions required for maximum enhancement as well as the ultimate sensitivity of the technique. The ability to perform continuous hyperpolarization via the Overhauser mechanism, in combination with excellent in vivo stability, raises the possibility of performing noninvasive tracking of nanodiamonds over indefinitely long periods of time.

The nitrogen-vacancy (NV) centre in diamond is a leading platform for nanoscale sensing and imaging, as well as quantum information processing in the solid state. To date, individual control of two NV electronic spins at the nanoscale has been demonstrated. However, a key challenge is to scale up such control to arrays of NV spins. Here we apply nanoscale magnetic resonance frequency encoding to realize site-selective addressing and coherent control of a four-site array of NV spins. Sites in the array are separated by 100 nm, with each site containing multiple NVs separated by ~15 nm. Microcoils fabricated on the diamond chip provide electrically tuneable magnetic-field gradients ~0.1 G/nm. Tailored application of gradient fields and resonant microwaves allow site-selective NV spin manipulation and sensing applications, including Rabi oscillations, imaging, and nuclear magnetic resonance (NMR) spectroscopy with nanoscale resolution. Microcoil-based magnetic resonance of solid-state spins provides a practical platform for quantum-assisted sensing, quantum information processing, and the study of nanoscale spin networks.

Correlated electron microscopy and cathodoluminescence (CL) imaging using functionalized nanoparticles is a promising nanoscale probe of biological structure and function. Nanodiamonds (NDs) that contain CL-emitting color centers are particularly well suited for such applications. The intensity of CL emission from NDs is determined by a combination of factors, including: particle size; density of color centers; efficiency of energy deposition by electrons passing through the particle; and conversion efficiency from deposited energy to CL emission. We report experiments and numerical simulations that investigate the relative importance of each of these factors in determining CL emission intensity from NDs containing nitrogen-vacancy (NV) color centers. In particular, we find that CL can be detected from NV-doped NDs with dimensions as small as ~ 40 nm, although CL emission decreases significantly for smaller NDs.