Search

We present a comparison of different quantum state preparation algorithms and their overall efficiency for the Schwinger model with a theta term. While adiabatic state preparation (ASP) is proved to be effective, in practice it leads to large CNOT gate counts to prepare the ground state. The quantum approximate optimization algorithm (QAOA) provides excellent results while keeping the CNOT counts small by design, at the cost of an expensive classical minimization process. We introduce a ``blocked'' modification of the Schwinger Hamiltonian to be used in the QAOA that further decreases the length of the algorithms as the size of the problem is increased. The rodeo algorithm (RA) provides a powerful tool to efficiently prepare any eigenstate of the Hamiltonian, as long as its overlap with the initial guess is large enough. We obtain the best results when combining the blocked QAOA ansatz and the RA, as this provides an excellent initial state with a relatively short algorithm without the need to perform any classical steps for large problem sizes., Comment: 11 pages, 9 figures, 4 tables

Motivated by attempts to quantum simulate lattice models with continuous Abelian symmetries using discrete approximations, we study an extended-O(2) model in two dimensions that differs from the ordinary O(2) model by the addition of an explicit symmetry breaking term $-h_q\cos(q\varphi)$. Its coupling $h_q$ allows to smoothly interpolate between the O(2) model ($h_q=0$) and a $q$-state clock model ($h_q\rightarrow\infty$). In the latter case, a $q$-state clock model can also be defined for noninteger values of $q$. Thus, such a limit can also be considered as an analytic continuation of an ordinary $q$-state clock model to noninteger $q$. In previous work, we established the phase diagram for noninteger $q$ in the infinite coupling limit ($h_q\rightarrow\infty$). We showed that there is a second-order phase transition at low temperature and a crossover at high temperature. In this work, we seek to establish the phase diagram at finite values of the coupling using Monte Carlo and tensor methods. We show that for noninteger $q$, the second-order phase transition at low temperature and crossover at high temperature persist to finite coupling. For integer $q=2,3,4$, we know there is a second-order phase transition at infinite coupling (i.e. the well-known clock models). At finite coupling, we find that the critical exponents for $q=3,4$ vary with the coupling, and for $q=4$ the transition may turn into a BKT transition at small coupling. We comment on the similarities and differences of the phase diagrams with those of quantum simulators of the Abelian-Higgs model based on ladder-shaped arrays of Rydberg atoms., Comment: 22 pages, 14 figures, 6 tables

Open-source Large Language Models enable projects such as NASA SciX (i.e., NASA ADS) to think out of the box and try alternative approaches for information retrieval and data augmentation, while respecting data copyright and users' privacy. However, when large language models are directly prompted with questions without any context, they are prone to hallucination. At NASA SciX we have developed an experiment where we created semantic vectors for our large collection of abstracts and full-text content, and we designed a prompt system to ask questions using contextual chunks from our system. Based on a non-systematic human evaluation, the experiment shows a lower degree of hallucination and better responses when using Retrieval Augmented Generation. Further exploration is required to design new features and data augmentation processes at NASA SciX that leverages this technology while respecting the high level of trust and quality that the project holds., Comment: To appear in the proceedings of the 33th annual international Astronomical Data Analysis Software & Systems (ADASS XXXIII)

The zero forcing number was introduced as a combinatorial bound on the maximum nullity taken over the set of real symmetric matrices that respect the pattern of an underlying graph. The $Z_q$-forcing game is an analog to the standard zero forcing game which incorporates inertia restrictions on the set of matrices associated with a graph. This work proves an upper bound on the $Z_q$-forcing number for trees. Furthermore, we consider the $Z_q$-forcing number for caterpillar cycles on $n$ vertices. We focus on developing game theoretic proofs of upper and lower bounds.

Artificial intelligence (AI) governance is the body of standards and practices used to ensure that AI systems are deployed responsibly. Current AI governance approaches consist mainly of manual review and documentation processes. While such reviews are necessary for many systems, they are not sufficient to systematically address all potential harms, as they do not operationalize governance requirements for system engineering, behavior, and outcomes in a way that facilitates rigorous and reproducible evaluation. Modern AI systems are data-centric: they act on data, produce data, and are built through data engineering. The assurance of governance requirements must also be carried out in terms of data. This work explores the systematization of governance requirements via datasets and algorithmic evaluations. When applied throughout the product lifecycle, data-centric governance decreases time to deployment, increases solution quality, decreases deployment risks, and places the system in a continuous state of assured compliance with governance requirements., Comment: 26 pages, 13 figures

Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to "real world" events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of "Lifelong Learning" systems that are capable of 1) Continuous Learning, 2) Transfer and Adaptation, and 3) Scalability. Unfortunately, efforts to improve these capabilities are typically treated as distinct areas of research that are assessed independently, without regard to the impact of each separate capability on other aspects of the system. We instead propose a holistic approach, using a suite of metrics and an evaluation framework to assess Lifelong Learning in a principled way that is agnostic to specific domains or system techniques. Through five case studies, we show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems. We highlight how the proposed suite of metrics quantifies performance trade-offs present during Lifelong Learning system development - both the widely discussed Stability-Plasticity dilemma and the newly proposed relationship between Sample Efficient and Robust Learning. Further, we make recommendations for the formulation and use of metrics to guide the continuing development of Lifelong Learning systems and assess their progress in the future., Comment: To appear in Neural Networks

Motivated by attempts to quantum simulate lattice models with continuous Abelian symmetries using discrete approximations, we consider an extended-O(2) model that differs from the ordinary O(2) model by an explicit symmetry breaking term. Its coupling allows to smoothly interpolate between the O(2) model (zero coupling) and a $q$-state clock model (infinite coupling). In the latter case, a $q$-state clock model can also be defined for non-integer values of $q$. Thus, such a limit can also be considered as an analytic continuation of an ordinary $q$-state clock model to non-integer $q$. The phase diagram of the extended-O(2) model in the infinite coupling limit was established in our previous work, where it was shown that for non-integer $q$, there is a second-order phase transition at low temperature and a crossover at high temperature. In this work, we investigate the model at finite values of the coupling using Monte Carlo and tensor methods. The results may be relevant for configurable Rydberg-atom arrays., Comment: 7 pages, 4 Figures; Talk given at the 39th International Symposium on Lattice Field theory (LATTICE2022) in Bonn, Germany; August 8-13 2022

As Artificial and Robotic Systems are increasingly deployed and relied upon for real-world applications, it is important that they exhibit the ability to continually learn and adapt in dynamically-changing environments, becoming Lifelong Learning Machines. Continual/lifelong learning (LL) involves minimizing catastrophic forgetting of old tasks while maximizing a model's capability to learn new tasks. This paper addresses the challenging lifelong reinforcement learning (L2RL) setting. Pushing the state-of-the-art forward in L2RL and making L2RL useful for practical applications requires more than developing individual L2RL algorithms; it requires making progress at the systems-level, especially research into the non-trivial problem of how to integrate multiple L2RL algorithms into a common framework. In this paper, we introduce the Lifelong Reinforcement Learning Components Framework (L2RLCF), which standardizes L2RL systems and assimilates different continual learning components (each addressing different aspects of the lifelong learning problem) into a unified system. As an instantiation of L2RLCF, we develop a standard API allowing easy integration of novel lifelong learning components. We describe a case study that demonstrates how multiple independently-developed LL components can be integrated into a single realized system. We also introduce an evaluation environment in order to measure the effect of combining various system components. Our evaluation environment employs different LL scenarios (sequences of tasks) consisting of Starcraft-2 minigames and allows for the fair, comprehensive, and quantitative comparison of different combinations of components within a challenging common evaluation environment., Comment: The Second International Conference on AIML Systems, October 12--15, 2022, Bangalore, India

The NASA Astrophysics Data System (ADS) is an essential tool for researchers that allows them to explore the astronomy and astrophysics scientific literature, but it has yet to exploit recent advances in natural language processing. At ADASS 2021, we introduced astroBERT, a machine learning language model tailored to the text used in astronomy papers in ADS. In this work we: - announce the first public release of the astroBERT language model; - show how astroBERT improves over existing public language models on astrophysics specific tasks; - and detail how ADS plans to harness the unique structure of scientific papers, the citation graph and citation context, to further improve astroBERT.

In recent years, advances in deep learning have resulted in a plethora of successes in the use of reinforcement learning (RL) to solve complex sequential decision tasks with high-dimensional inputs. However, existing systems lack the necessary mechanisms to provide humans with a holistic view of their competence, presenting an impediment to their adoption, particularly in critical applications where the decisions an agent makes can have significant consequences. Yet, existing RL-based systems are essentially competency-unaware in that they lack the necessary interpretation mechanisms to allow human operators to have an insightful, holistic view of their competency. In this paper, we extend a recently-proposed framework for explainable RL that is based on analyses of "interestingness." Our new framework provides various measures of RL agent competence stemming from interestingness analysis and is applicable to a wide range of RL algorithms. We also propose novel mechanisms for assessing RL agents' competencies that: 1) identify agent behavior patterns and competency-controlling conditions by clustering agent behavior traces solely using interestingness data; and 2) identify the task elements mostly responsible for an agent's behavior, as measured through interestingness, by performing global and local analyses using SHAP values. Overall, our tools provide insights about RL agent competence, both their capabilities and limitations, enabling users to make more informed decisions about interventions, additional training, and other interactions in collaborative human-machine settings., Comment: Appears in Proceedings of AAAI FSS-22 Symposium "Lessons Learned for Autonomous Assessment of Machine Abilities (LLAAMA)"

What is the value of a graduate degree? It is a calculation: time plus tuition minus the chance of getting the right job after graduation. When the costs far outweigh the return, a new question might occur to graduates: Can I have my money back? Most prospective graduate students run the math before applying to a program, but it is next to impossible to solve this equation until they have accepted that first job. And yet, it is only when new graduates enter the job market that they discover what it takes to survive in their chosen field. That is when they feel those metrics on a more visceral level. This article discusses how iSchools are producing graduates who possess the teaching skills to tackle the growing problem of the spread of misinformation. The value of iSchool degrees has a long history of being questioned by both recent graduates and academic librarians who wonder if the curriculum has kept up with the changing demands of the profession. If iSchools are not graduating professionals who possess the teaching skills necessary for rising to the challenges of today's information landscape, librarians are left to calculate the value of their degree and ask themselves: Am I actually prepared for my job?

Recent years have seen significant advances in explainable AI as the need to understand deep learning models has gained importance with the increased emphasis on trust and ethics in AI. Comprehensible models for sequential decision tasks are a particular challenge as they require understanding not only individual predictions but a series of predictions that interact with environmental dynamics. We present a framework for learning comprehensible models of sequential decision tasks in which agent strategies are characterized using temporal logic formulas. Given a set of agent traces, we first cluster the traces using a novel embedding method that captures frequent action patterns. We then search for logical formulas that explain the agent strategies in the different clusters. We evaluate our framework on combat scenarios in StarCraft II (SC2), using traces from a handcrafted expert policy and a trained reinforcement learning agent. We implemented a feature extractor for SC2 environments that extracts traces as sequences of high-level features describing both the state of the environment and the agent's local behavior from agent replays. We further designed a visualization tool depicting the movement of units in the environment that helps understand how different task conditions lead to distinct agent behavior patterns in each trace cluster. Experimental results show that our framework is capable of separating agent traces into distinct groups of behaviors for which our approach to strategy inference produces consistent, meaningful, and easily understood strategy descriptions.

One approach to meet the challenges of deep lifelong reinforcement learning (LRL) is careful management of the agent's learning experiences, to learn (without forgetting) and build internal meta-models (of the tasks, environments, agents, and world). Generative replay (GR) is a biologically inspired replay mechanism that augments learning experiences with self-labelled examples drawn from an internal generative model that is updated over time. We present a version of GR for LRL that satisfies two desiderata: (a) Introspective density modelling of the latent representations of policies learned using deep RL, and (b) Model-free end-to-end learning. In this paper, we study three deep learning architectures for model-free GR, starting from a na\"ive GR and adding ingredients to achieve (a) and (b). We evaluate our proposed algorithms on three different scenarios comprising tasks from the Starcraft-2 and Minigrid domains. We report several key findings showing the impact of the design choices on quantitative metrics that include transfer learning, generalization to unseen tasks, fast adaptation after task change, performance wrt task expert, and catastrophic forgetting. We observe that our GR prevents drift in the features-to-action mapping from the latent vector space of a deep RL agent. We also show improvements in established lifelong learning metrics. We find that a small random replay buffer significantly increases the stability of training. Overall, we find that "hidden replay" (a well-known architecture for class-incremental classification) is the most promising approach that pushes the state-of-the-art in GR for LRL and observe that the architecture of the sleep model might be more important for improving performance than the types of replay used. Our experiments required only 6% of training samples to achieve 80-90% of expert performance in most Starcraft-2 scenarios., Comment: Accepted to the First Conference on Lifelong Learning Agents (CoLLAs 2022)

We present a novel generative method for producing unseen and plausible counterfactual examples for reinforcement learning (RL) agents based upon outcome variables that characterize agent behavior. Our approach uses a variational autoencoder to train a latent space that jointly encodes information about the observations and outcome variables pertaining to an agent's behavior. Counterfactuals are generated using traversals in this latent space, via gradient-driven updates as well as latent interpolations against cases drawn from a pool of examples. These include updates to raise the likelihood of generated examples, which improves the plausibility of generated counterfactuals. From experiments in three RL environments, we show that these methods produce counterfactuals that are more plausible and proximal to their queries compared to purely outcome-driven or case-based baselines. Finally, we show that a latent jointly trained to reconstruct both the input observations and behavioral outcome variables produces higher-quality counterfactuals over latents trained solely to reconstruct the observation inputs.

Before delegating a task to an autonomous system, a human operator may want a guarantee about the behavior of the system. This paper extends previous work on conformal prediction for functional data and conformalized quantile regression to provide conformal prediction intervals over the future behavior of an autonomous system executing a fixed control policy on a Markov Decision Process (MDP). The prediction intervals are constructed by applying conformal corrections to prediction intervals computed by quantile regression. The resulting intervals guarantee that with probability $1-\delta$ the observed trajectory will lie inside the prediction interval, where the probability is computed with respect to the starting state distribution and the stochasticity of the MDP. The method is illustrated on MDPs for invasive species management and StarCraft2 battles., Comment: 25 pages, 15 figures, 2 tables. Fixed typos and an error in the appendix plots