Search

Quantum computing holds immense potential for addressing a myriad of intricate challenges, which is significantly amplified when scaled to thousands of qubits. However, a major challenge lies in developing an efficient and scalable quantum control system. To address this, we propose a novel Hierarchical MicroArchitecture (HiMA) designed to facilitate qubit scaling and exploit quantum process-level parallelism. This microarchitecture is based on three core elements: (i) discrete qubit-level drive and readout, (ii) a process-based hierarchical trigger mechanism, and (iii) multiprocessing with a staggered triggering technique to enable efficient quantum process-level parallelism. We implement HiMA as a control system for a 72-qubit tunable superconducting quantum processing unit, serving a public quantum cloud computing platform, which is capable of expanding to 6144 qubits through three-layer cascading. In our benchmarking tests, HiMA achieves up to a 4.89x speedup under a 5-process parallel configuration. Consequently, to the best of our knowledge, we have achieved the highest CLOPS (Circuit Layer Operations Per Second), reaching up to 43,680, across all publicly available platforms.

Quantum computational fluid dynamics (QCFD) offers a promising alternative to classical computational fluid dynamics (CFD) by leveraging quantum algorithms for higher efficiency. This paper introduces a comprehensive QCFD method, including an iterative method "Iterative-QLS" that suppresses error in quantum linear solver, and a subspace method to scale the solution to a larger size. We implement our method on a superconducting quantum computer, demonstrating successful simulations of steady Poiseuille flow and unsteady acoustic wave propagation. The Poiseuille flow simulation achieved a relative error of less than $0.2\%$, and the unsteady acoustic wave simulation solved a 5043-dimensional matrix. We emphasize the utilization of the quantum-classical hybrid approach in applications of near-term quantum computers. By adapting to quantum hardware constraints and offering scalable solutions for large-scale CFD problems, our method paves the way for practical applications of near-term quantum computers in computational science., Comment: 31 pages, 10 figures

Quantum machine learning has demonstrated significant potential in solving practical problems, particularly in statistics-focused areas such as data science and finance. However, challenges remain in preparing and learning statistical models on a quantum processor due to issues with trainability and interpretability. In this letter, we utilize the maximum entropy principle to design a statistics-informed parameterized quantum circuit (SI-PQC) for efficiently preparing and training of quantum computational statistical models, including arbitrary distributions and their weighted mixtures. The SI-PQC features a static structure with trainable parameters, enabling in-depth optimized circuit compilation, exponential reductions in resource and time consumption, and improved trainability and interpretability for learning quantum states and classical model parameters simultaneously. As an efficient subroutine for preparing and learning in various quantum algorithms, the SI-PQC addresses the input bottleneck and facilitates the injection of prior knowledge., Comment: 19 pages, 5 figures

The rapid development of quantum computers has enabled demonstrations of quantum advantages on various tasks. However, real quantum systems are always dissipative due to their inevitable interaction with the environment, and the resulting non-unitary dynamics make quantum simulation challenging with only unitary quantum gates. In this work, we present an innovative and scalable method to simulate open quantum systems using quantum computers. We define an adjoint density matrix as a counterpart of the true density matrix, which reduces to a mixed-unitary quantum channel and thus can be effectively sampled using quantum computers. This method has several benefits, including no need for auxiliary qubits and noteworthy scalability. Moreover, accurate long-time simulation can also be achieved as the adjoint density matrix and the true dissipated one converge to the same state. Finally, we present deployments of this theory in the dissipative quantum $XY$ model for the evolution of correlation and entropy with short-time dynamics and the disordered Heisenberg model for many-body localization with long-time dynamics. This work promotes the study of real-world many-body dynamics with quantum computers, highlighting the potential to demonstrate practical quantum advantages., Comment: 13 pages, 6 figures

Shape compactness is a key geometrical property to describe interesting regions in many image segmentation tasks. In this paper, we propose two novel algorithms to solve the introduced image segmentation problem that incorporates a shape-compactness prior. Existing algorithms for such a problem often suffer from computational inefficiency, difficulty in reaching a local minimum, and the need to fine-tune the hyperparameters. To address these issues, we propose a novel optimization model along with its equivalent primal-dual model and introduce a new optimization algorithm based on primal-dual threshold dynamics (PD-TD). Additionally, we relax the solution constraint and propose another novel primal-dual soft threshold-dynamics algorithm (PD-STD) to achieve superior performance. Based on the variational explanation of the sigmoid layer, the proposed PD-STD algorithm can be integrated into Deep Neural Networks (DNNs) to enforce compact regions as image segmentation results. Compared to existing deep learning methods, extensive experiments demonstrated that the proposed algorithms outperformed state-of-the-art algorithms in numerical efficiency and effectiveness, especially while applying to the popular networks of DeepLabV3 and IrisParseNet with higher IoU, dice, and compactness metrics on noisy Iris datasets. In particular, the proposed algorithms significantly improve IoU by 20% training on a highly noisy image dataset., Comment: 28 pages

The field of quantum deep learning presents significant opportunities for advancing computational capabilities, yet it faces a major obstacle in the form of the "information loss problem" due to the inherent limitations of the necessary quantum tomography in scaling quantum deep neural networks. This paper introduces an end-to-end Quantum Vision Transformer (QViT), which incorporates an innovative quantum residual connection technique, to overcome these challenges and therefore optimize quantum computing processes in deep learning. Our thorough complexity analysis of the QViT reveals a theoretically exponential and empirically polynomial speedup, showcasing the model's efficiency and potential in quantum computing applications. We conducted extensive numerical tests on modern, large-scale transformers and datasets, establishing the QViT as a pioneering advancement in applying quantum deep neural networks in practical scenarios. Our work provides a comprehensive quantum deep learning paradigm, which not only demonstrates the versatility of current quantum linear algebra algorithms but also promises to enhance future research and development in quantum deep learning., Comment: 24pages, 10 figures

Cardiovascular diseases (CVDs) are the leading cause of death worldwide, with blood pressure serving as a crucial indicator. Arterial blood pressure (ABP) waveforms provide continuous pressure measurements throughout the cardiac cycle and offer valuable diagnostic insights. Consequently, there is a significant demand for non-invasive and cuff-less methods to measure ABP waveforms continuously. Accurate prediction of ABP waveforms can also improve the estimation of mean blood pressure, an essential cardiovascular health characteristic. This study proposes a novel framework based on the physics-informed DeepONet approach to predict ABP waveforms. Unlike previous methods, our approach requires the predicted ABP waveforms to satisfy the Navier-Stokes equation with a time-periodic condition and a Windkessel boundary condition. Notably, our framework is the first to predict ABP waveforms continuously, both with location and time, within the part of the artery that is being simulated. Furthermore, our method only requires ground truth data at the outlet boundary and can handle periodic conditions with varying periods. Incorporating the Windkessel boundary condition in our solution allows for generating natural physical reflection waves, which closely resemble measurements observed in real-world cases. Moreover, accurately estimating the hyper-parameters in the Navier-Stokes equation for our simulations poses a significant challenge. To overcome this obstacle, we introduce the concept of meta-learning, enabling the neural networks to learn these parameters during the training process.

In this study, our goal is to integrate classical mathematical models with deep neural networks by introducing two novel deep neural network models for image segmentation known as Double-well Nets. Drawing inspirations from the Potts model, our models leverage neural networks to represent a region force functional. We extend the well-know MBO (Merriman-Bence-Osher) scheme to solve the Potts model. The widely recognized Potts model is approximated using a double-well potential and then solved by an operator-splitting method, which turns out to be an extension of the well-known MBO scheme. Subsequently, we replace the region force functional in the Potts model with a UNet-type network, which is data-driven and is designed to capture multiscale features of images, and also introduce control variables to enhance effectiveness. The resulting algorithm is a neural network activated by a function that minimizes the double-well potential. What sets our proposed Double-well Nets apart from many existing deep learning methods for image segmentation is their strong mathematical foundation. They are derived from the network approximation theory and employ the MBO scheme to approximately solve the Potts model. By incorporating mathematical principles, Double-well Nets bridge the MBO scheme and neural networks, and offer an alternative perspective for designing networks with mathematical backgrounds. Through comprehensive experiments, we demonstrate the performance of Double-well Nets, showcasing their superior accuracy and robustness compared to state-of-the-art neural networks. Overall, our work represents a valuable contribution to the field of image segmentation by combining the strengths of classical variational models and deep neural networks. The Double-well Nets introduce an innovative approach that leverages mathematical foundations to enhance segmentation performance.

Novelty adaptation is the ability of an intelligent agent to adjust its behavior in response to changes in its environment. This is an important characteristic of intelligent agents, as it allows them to continue to function effectively in novel or unexpected situations, but still stands as a critical challenge for deep reinforcement learning (DRL). To tackle this challenge, we propose a simple yet effective novel method, NAPPING (Novelty Adaptation Principles Learning), that allows trained DRL agents to respond to different classes of novelties in open worlds rapidly. With NAPPING, DRL agents can learn to adjust the trained policy only when necessary. They can quickly generalize to similar novel situations without affecting the part of the trained policy that still works. To demonstrate the efficiency and efficacy of NAPPING, we evaluate our method on four action domains that are different in reward structures and the type of task. The domains are CartPole and MountainCar (classic control), CrossRoad (path-finding), and AngryBirds (physical reasoning). We compare NAPPING with standard online and fine-tuning DRL methods in CartPole, MountainCar and CrossRoad, and state-of-the-art methods in the more complicated AngryBirds domain. Our evaluation results demonstrate that with our proposed method, DRL agents can rapidly and effectively adjust to a wide range of novel situations across all tested domains.

Investigating blood flow in the cardiovascular system is crucial for assessing cardiovascular health. Computational approaches offer some non-invasive alternatives to measure blood flow dynamics. Numerical simulations based on traditional methods such as finite-element and other numerical discretizations have been extensively studied and have yielded excellent results. However, adapting these methods to real-life simulations remains a complex task. In this paper, we propose a method that offers flexibility and can efficiently handle real-life simulations. We suggest utilizing the physics-informed neural network (PINN) to solve the Navier-Stokes equation in a deformable domain, specifically addressing the simulation of blood flow in elastic vessels. Our approach models blood flow using an incompressible, viscous Navier-Stokes equation in an Arbitrary Lagrangian-Eulerian form. The mechanical model for the vessel wall structure is formulated by an equation of Newton's second law of momentum and linear elasticity to the force exerted by the fluid flow. Our method is a mesh-free approach that eliminates the need for discretization and meshing of the computational domain. This makes it highly efficient in solving simulations involving complex geometries. Additionally, with the availability of well-developed open-source machine learning framework packages and parallel modules, our method can easily be accelerated through GPU computing and parallel computing. To evaluate our approach, we conducted experiments on regular cylinder vessels as well as vessels with plaque on their walls. We compared our results to a solution calculated by Finite Element Methods using a dense grid and small time steps, which we considered as the ground truth solution. We report the relative error and the time consumed to solve the problem, highlighting the advantages of our method.

Deep reinforcement learning suffers from catastrophic forgetting and sample inefficiency making it less applicable to the ever-changing real world. However, the ability to use previously learned knowledge is essential for AI agents to quickly adapt to novelties. Often, certain spatial information observed by the agent in the previous interactions can be leveraged to infer task-specific rules. Inferred rules can then help the agent to avoid potentially dangerous situations in the previously unseen states and guide the learning process increasing agent's novelty adaptation speed. In this work, we propose a general framework that is applicable to deep reinforcement learning agents. Our framework provides the agent with an autonomous way to discover the task-specific rules in the novel environments and self-supervise it's learning. We provide a rule-driven deep Q-learning agent (RDQ) as one possible implementation of that framework. We show that RDQ successfully extracts task-specific rules as it interacts with the world and uses them to drastically increase its learning efficiency. In our experiments, we show that the RDQ agent is significantly more resilient to the novelties than the baseline agents, and is able to detect and adapt to novel situations faster.

Independent component analysis (ICA) is a fundamental data processing technique to decompose the captured signals into as independent as possible components. Computing the contrast function, which serves as a measure of independence of signals, is vital in the separation process using ICA. This paper presents a quantum ICA algorithm which focuses on computing a specified contrast function on a quantum computer. Using the quantum acceleration in matrix operations, we efficiently deal with Gram matrices and estimate the contrast function with the complexity of $O(\epsilon_1^{-2}\mbox{poly}\log(N/\epsilon_1))$. This estimation subprogram, combined with the classical optimization framework, enables our quantum ICA algorithm, which exponentially reduces the complexity dependence on the data scale compared with classical algorithms. The outperformance is further supported by numerical experiments, while a source separation of a transcriptomic dataset is shown as an example of application.

Ordered electronic states have been extensively explored in cuprates and iron-based unconventional superconductors, but seldom observed in the epitaxial FeSe/SrTiO3(001) monolayer (FeSe/STO) with an enhanced superconducting transition temperature (Tc). Here, by using scanning tunneling microscopy/ spectroscopy (STM/STS), we reveal a checkerboard charge order in the epitaxial FeSe/STO monolayer, with a period of four times the inter-Fe-atom distance along two perpendicular directions of the Fe lattice. This ordered state is uniquely present in the superconducting FeSe/STO monolayer, even at liquid nitrogen temperature, but absent in the non-superconducting FeSe monolayer or bilayer. Quasiparticle interference (QPI) measurements further confirm it as a static order without an energy-dependent dispersion and gapped out within the superconductivity gap. The intensity of the charge order shows an enhancement near the superconducting transition temperature, thus implying a correlation with the high-Tc superconductivity in the FeSe/STO monolayer. This study provides a new basis for exploring the ordered electronic states and their interplay with high-Tc superconductivity in the FeSe monolayer., Comment: 22 pages, 4 figures

The Euler Elastica (EE) model with surface curvature can generate artifact-free results compared with the traditional total variation regularization model in image processing. However, strong nonlinearity and singularity due to the curvature term in the EE model pose a great challenge for one to design fast and stable algorithms for the EE model. In this paper, we propose a new, fast, hybrid alternating minimization (HALM) algorithm for the EE model based on a bilinear decomposition of the gradient of the underlying image and prove the global convergence of the minimizing sequence generated by the algorithm under mild conditions. The HALM algorithm comprises three sub-minimization problems and each is either solved in the closed form or approximated by fast solvers making the new algorithm highly accurate and efficient. We also discuss the extension of the HALM strategy to deal with general curvature-based variational models, especially with a Lipschitz smooth functional of the curvature. A host of numerical experiments are conducted to show that the new algorithm produces good results with much-improved efficiency compared to other state-of-the-art algorithms for the EE model. As one of the benchmarks, we show that the average running time of the HALM algorithm is at most one-quarter of that of the fast operator-splitting-based Deng-Glowinski-Tai algorithm.

Designing deep learning algorithms for gland segmentation is crucial for automatic cancer diagnosis and prognosis, yet the expensive annotation cost hinders the development and application of this technology. In this paper, we make a first attempt to explore a deep learning method for unsupervised gland segmentation, where no manual annotations are required. Existing unsupervised semantic segmentation methods encounter a huge challenge on gland images: They either over-segment a gland into many fractions or under-segment the gland regions by confusing many of them with the background. To overcome this challenge, our key insight is to introduce an empirical cue about gland morphology as extra knowledge to guide the segmentation process. To this end, we propose a novel Morphology-inspired method via Selective Semantic Grouping. We first leverage the empirical cue to selectively mine out proposals for gland sub-regions with variant appearances. Then, a Morphology-aware Semantic Grouping module is employed to summarize the overall information about the gland by explicitly grouping the semantics of its sub-region proposals. In this way, the final segmentation network could learn comprehensive knowledge about glands and produce well-delineated, complete predictions. We conduct experiments on GlaS dataset and CRAG dataset. Our method exceeds the second-best counterpart over 10.56% at mIOU., Comment: MICCAI 2023 Accepted

Deep neural network is a powerful tool for many tasks. Understanding why it is so successful and providing a mathematical explanation is an important problem and has been one popular research direction in past years. In the literature of mathematical analysis of deep neural networks, a lot of works is dedicated to establishing representation theories. How to make connections between deep neural networks and mathematical algorithms is still under development. In this paper, we give an algorithmic explanation for deep neural networks, especially in their connections with operator splitting. We show that with certain splitting strategies, operator-splitting methods have the same structure as networks. Utilizing this connection and the Potts model for image segmentation, two networks inspired by operator-splitting methods are proposed. The two networks are essentially two operator-splitting algorithms solving the Potts model. Numerical experiments are presented to demonstrate the effectiveness of the proposed networks.

For problems in image processing and many other fields, a large class of effective neural networks has encoder-decoder-based architectures. Although these networks have made impressive performances, mathematical explanations of their architectures are still underdeveloped. In this paper, we study the encoder-decoder-based network architecture from the algorithmic perspective and provide a mathematical explanation. We use the two-phase Potts model for image segmentation as an example for our explanations. We associate the segmentation problem with a control problem in the continuous setting. Then, multigrid method and operator splitting scheme, the PottsMGNet, are used to discretize the continuous control model. We show that the resulting discrete PottsMGNet is equivalent to an encoder-decoder-based network. With minor modifications, it is shown that a number of the popular encoder-decoder-based neural networks are just instances of the proposed PottsMGNet. By incorporating the Soft-Threshold-Dynamics into the PottsMGNet as a regularizer, the PottsMGNet has shown to be robust with the network parameters such as network width and depth and achieved remarkable performance on datasets with very large noise. In nearly all our experiments, the new network always performs better or as good on accuracy and dice score than existing networks for image segmentation.

The foundation models based on pre-training technology have significantly advanced artificial intelligence from theoretical to practical applications. These models have facilitated the feasibility of computer-aided diagnosis for widespread use. Medical contrastive vision-language pre-training, which does not require human annotations, is an effective approach for guiding representation learning using description information in diagnostic reports. However, the effectiveness of pre-training is limited by the large-scale semantic overlap and shifting problems in medical field. To address these issues, we propose the Knowledge-Boosting Contrastive Vision-Language Pre-training framework (KoBo), which integrates clinical knowledge into the learning of vision-language semantic consistency. The framework uses an unbiased, open-set sample-wise knowledge representation to measure negative sample noise and supplement the correspondence between vision-language mutual information and clinical knowledge. Extensive experiments validate the effect of our framework on eight tasks including classification, segmentation, retrieval, and semantic relatedness, achieving comparable or better performance with the zero-shot or few-shot settings. Our code is open on https://github.com/ChenXiaoFei-CS/KoBo., Comment: accepted by MICCAI 2023

Applying low-depth quantum neural networks (QNNs), variational quantum algorithms (VQAs) are both promising and challenging in the noisy intermediate-scale quantum (NISQ) era: Despite its remarkable progress, criticisms on the efficiency and feasibility issues never stopped. However, whether VQAs can demonstrate quantum advantages is still undetermined till now, which will be investigated in this paper. First, we will prove that there exists a dependency between the parameter number and the gradient-evaluation cost when training QNNs. Noticing there is no such direct dependency when training classical neural networks with the backpropagation algorithm, we argue that such a dependency limits the scalability of VQAs. Second, we estimate the time for running VQAs in ideal cases, i.e., without considering realistic limitations like noise and reachability. We will show that the ideal time cost easily reaches the order of a 1-year wall time. Third, by comparing with the time cost using classical simulation of quantum circuits, we will show that VQAs can only outperform the classical simulation case when the time cost reaches the scaling of $10^0$-$10^2$ years. Finally, based on the above results, we argue that it would be difficult for VQAs to outperform classical cases in view of time scaling, and therefore, demonstrate quantum advantages, with the current workflow. Since VQAs as well as quantum computing are developing rapidly, this work does not aim to deny the potential of VQAs. The analysis in this paper provides directions for optimizing VQAs, and in the long run, seeking more natural hybrid quantum-classical algorithms would be meaningful., Comment: 18 pages, 7 figures

Quantum computing offers potential solutions for finding ground states in condensed-matter physics and chemistry. However, achieving effective ground state preparation is also computationally hard for arbitrary Hamiltonians. It is necessary to propose certain assumptions to make this problem efficiently solvable, including preparing a trial state of a non-trivial overlap with the genuine ground state. Here, we propose a classical-assisted quantum ground state preparation method for quantum many-body systems, combining Tensor Network States (TNS) and Monte Carlo (MC) sampling as a heuristic method to prepare a trial state with a non-trivial overlap with the genuine ground state. We extract a sparse trial state by sampling from TNS, which can be efficiently prepared by a quantum algorithm on early fault-tolerant quantum computers. Our method demonstrates a polynomial improvement in scaling of overlap between the trial state and genuine ground state compared to random trial states, as evidenced by numerical tests on the spin-$1/2$ $J_1$-$J_2$ Heisenberg model. Furthermore, our method is a novel approach to hybridize a classical numerical method and a quantum algorithm and brings inspiration to ground state preparation in other fields.

The increasing focus on long-term time series prediction across various fields has been significantly strengthened by advancements in quantum computation. In this paper, we introduce a data-driven method designed for long-term time series prediction with quantum dynamical embedding (QDE). This approach enables a trainable embedding of the data space into an extended state space, allowing for the recursive retrieval of time series information. Based on its independence of time series length, this method achieves depth-efficient quantum circuits that are crucial for near-term quantum computers. Numerical simulations demonstrate the model's improved performance in prediction accuracy and resource efficiency over existing methods, as well as its effective denoising capabilities. We implement this model on the Origin ''Wukong'' superconducting quantum processor with a learnable error-cancellation layer (LECL) for error mitigation, further validating the practical applicability of our approach on near-term quantum devices. Furthermore, the theoretical analysis of the QDE's dynamical properties and its universality enhances its potential for time series prediction. This study establishes a significant step towards the processing of long-term time series on near-term quantum computers, integrating data-driven learning with discrete dynamical embedding for enhanced forecasting capabilities., Comment: 20 pages, 12 figures

Program implementation and simulation are essential for research in the field of quantum algorithms. However, complex and large-scale quantum algorithms can pose challenges for existing quantum programming languages and simulators. Here, we present a scalable program implementation of the quantum walk on a sparse matrix and the quantum linear solver based on the quantum walk. Our implementation is based on a practical scenario in which the sparse matrix is stored in the compressed-sparse-column format in quantum random access memory. All necessary modules are implemented unitarily and are ensured to be decomposed at the quantum gate level, including implementing a quantum binary search and a modification of the original algorithm. The program is validated using a highly efficient quantum circuit simulator which is based on the register level and sparse state representation. With only a single core, we simulate the quantum walk on a 16384-dimensional matrix with 582 qubits in 1.1 minutes per step, as well as a quantum linear solver up to 1024 dimensions and 212245 steps in 70 hours. Our work narrows the gap between the simulation of a quantum algorithm and its classical counterparts, where the asymptotic complexity of our quantum linear solver simulation approximates a classical linear solver. These program implementation and simulation techniques have the potential to expand the boundary of numerical research for large-scale quantum algorithms, with implications for the development of error-correction-era quantum computing solutions., Comment: 17 pages, 11 figures

Quantum Random Access Memory (QRAM) is a critical component for loading classical data into quantum computers. While constructing a practical QRAM presents several challenges, including the impracticality of an infinitely large QRAM size and a fully error-correction implementation, it is essential to consider a practical case where the QRAM has a limited size. In this work, we focus on the access of larger data sizes without keeping on increasing the size of the QRAM. Firstly, we address the challenge of word length, as real-world datasets typically have larger word lengths than the single-bit data that most previous studies have focused on. We propose a novel protocol for loading data with larger word lengths $k$ without increasing the number of QRAM levels $n$. By exploiting the parallelism in the data query process, our protocol achieves a time complexity of $O(n+k)$ and improves error scaling performance compared to existing approaches. Secondly, we provide a data-loading method for general-sized data access tasks when the number of data items exceeds $2^n$, which outperforms the existing hybrid QRAM+QROM architecture. Our method contributes to the development of time and error-optimized data access protocols for QRAM devices, reducing the qubit count and error requirements for QRAM implementation, and making it easier to construct practical QRAM devices with a limited number of physical qubits., Comment: 19 pages, 7 figures. Changes in v2: Some minor revisions, and extend the Supplementary Information

Due to the emergence of AI systems that interact with the physical environment, there is an increased interest in incorporating physical reasoning capabilities into those AI systems. But is it enough to only have physical reasoning capabilities to operate in a real physical environment? In the real world, we constantly face novel situations we have not encountered before. As humans, we are competent at successfully adapting to those situations. Similarly, an agent needs to have the ability to function under the impact of novelties in order to properly operate in an open-world physical environment. To facilitate the development of such AI systems, we propose a new testbed, NovPhy, that requires an agent to reason about physical scenarios in the presence of novelties and take actions accordingly. The testbed consists of tasks that require agents to detect and adapt to novelties in physical scenarios. To create tasks in the testbed, we develop eight novelties representing a diverse novelty space and apply them to five commonly encountered scenarios in a physical environment. According to our testbed design, we evaluate two capabilities of an agent: the performance on a novelty when it is applied to different physical scenarios and the performance on a physical scenario when different novelties are applied to it. We conduct a thorough evaluation with human players, learning agents, and heuristic agents. Our evaluation shows that humans' performance is far beyond the agents' performance. Some agents, even with good normal task performance, perform significantly worse when there is a novelty, and the agents that can adapt to novelties typically adapt slower than humans. We promote the development of intelligent agents capable of performing at the human level or above when operating in open-world physical environments. Testbed website: https://github.com/phy-q/novphy, Comment: Testbed website: https://github.com/phy-q/novphy

During training, reinforcement learning systems interact with the world without considering the safety of their actions. When deployed into the real world, such systems can be dangerous and cause harm to their surroundings. Often, dangerous situations can be mitigated by defining a set of rules that the system should not violate under any conditions. For example, in robot navigation, one safety rule would be to avoid colliding with surrounding objects and people. In this work, we define safety rules in terms of the relationships between the agent and objects and use them to prevent reinforcement learning systems from performing potentially harmful actions. We propose a new safe epsilon-greedy algorithm that uses safety rules to override agents' actions if they are considered to be unsafe. In our experiments, we show that a safe epsilon-greedy policy significantly increases the safety of the agent during training, improves the learning efficiency resulting in much faster convergence, and achieves better performance than the base model.