Search

Reinforcement Learning (RL) is a branch of Artificial Intelligence (AI) which focuses on training agents to make decisions by interacting with their environment to maximize cumulative rewards. An overview of RL is provided in this paper, which discusses its core concepts, methodologies, recent trends, and resources for learning. We provide a detailed explanation of key components of RL such as states, actions, policies, and reward signals so that the reader can build a foundational understanding. The paper also provides examples of various RL algorithms, including model-free and model-based methods. In addition, RL algorithms are introduced and resources for learning and implementing them are provided, such as books, courses, and online communities. This paper demystifies a comprehensive yet simple introduction for beginners by offering a structured and clear pathway for acquiring and implementing real-time techniques.

We study the perfect transfer of information in 1+1D continuous quantum systems. This includes effective descriptions of inhomogeneous spin chains, for which the notion of perfect state transfer in quantum information was introduced, and here phrased in terms of waves. We show that reflection symmetry is necessary for perfect wave transfer (PWT) in any inhomogeneous conformal field theory, and even sufficient when restricted to one-particle excitations. To determine if or when it is sufficient more generally, we first break conformal invariance and study a broad class of 1+1D bosonic theories. We show that the question can then be posed as an inverse Sturm-Liouville problem that determines when the bosonic theory exhibits PWT. We demonstrate how to uniquely solve this problem, which also shows that reflection symmetry is sufficient for the special case with conformal invariance. Using bosonization, our continuum results extend these notions to interacting quantum field theories., Comment: 7 pages + SM, RevTeX, 2 figures

Traditional sperm morphology analysis is based on tedious manual annotation. Automated morphology analysis of a high number of sperm requires accurate segmentation of each sperm part and quantitative morphology evaluation. State-of-the-art instance-aware part segmentation networks follow a "detect-then-segment" paradigm. However, due to sperm's slim shape, their segmentation suffers from large context loss and feature distortion due to bounding box cropping and resizing during ROI Align. Moreover, morphology measurement of sperm tail is demanding because of the long and curved shape and its uneven width. This paper presents automated techniques to measure sperm morphology parameters automatically and quantitatively. A novel attention-based instance-aware part segmentation network is designed to reconstruct lost contexts outside bounding boxes and to fix distorted features, by refining preliminary segmented masks through merging features extracted by feature pyramid network. An automated centerline-based tail morphology measurement method is also proposed, in which an outlier filtering method and endpoint detection algorithm are designed to accurately reconstruct tail endpoints. Experimental results demonstrate that the proposed network outperformed the state-of-the-art top-down RP-R-CNN by 9.2% [AP]_vol^p, and the proposed automated tail morphology measurement method achieved high measurement accuracies of 95.34%,96.39%,91.2% for length, width and curvature, respectively., Comment: Accepted to ICRA 2024

In this paper, we investigate the entanglement dynamics of a two qutrits system interacting with a spin environment. Using negativity as the entanglement measure, we study the entanglement dynamics of the system. The calculations show that in cases where the entanglement decays quickly, the environment will have a quantum phase transition., Comment: 10 pages, 9 figures

We demonstrate that the plasma analogy is generally unreliable at predicting the edge properties of quantum Hall (QH) states, as it fails to account for the local edge velocity. This discrepancy arises from a fundamental difference between QH droplets and Coulomb gases (CGs): the former are incompressible liquids subject to area-preserving deformations, while the latter are governed by electrostatics and thus involve conformal transformations. We illustrate this QH/CG mismatch across various examples and show its impact on physical quantities, measurable in both solid-state samples and quantum simulators. Specifically, we discuss fluctuations of local observables and their connection to state-of-the-art microwave absorption experiments., Comment: 9 pages + SM, RevTeX, 6 figures

This investigation delves into the intricate entanglement dynamics exhibited by qubit-qutrit systems when subjected to the intricate dynamics of both the XXX isotropic Heisenberg model and the anisotropic XYZ model. These models are enriched by the presence of Dzyaloshinskii-Moriya (DM) interactions and external magnetic fields, creating a complex interplay of quantum phenomena. The exploration commences by initializing the system in a superposition of spin-coherent states, setting the stage for a nuanced analysis of entanglement evolution over time. By employing the negativity metric as the pivotal criterion for assessing entanglement, we gain insights into the subtle relationships between DM interactions, external magnetic fields, and the entanglement dynamics within this composite system. Through this comprehensive investigation, we aim to unravel the intricate dynamics and behaviors that emerge from the interplay of these fundamental elements, shedding light on the underlying principles governing quantum entanglement in this context., Comment: 10 pages, 3 figures, The original file is in LaTeX format

In this paper, the quantum entanglement dynamics of a qubit qubit compound system in the isotropic XXX Heisenberg and anisotropic XYZ models with DM interaction under magnetic fields is investigated. The system's initial state is considered as a spin coherence state, and the entanglement dynamics of this compound system is analyzed using the negativity criterion as an entanglement measure to assess the impact of DM interaction and magnetic fields., Comment: 16 pages,5 figures, The original file is in LaTeX format

Apart from performing arithmetic operations, understanding numbers themselves is still a challenge for existing language models. Simple generalisations, such as solving 100+200 instead of 1+2, can substantially affect model performance (Sivakumar and Moosavi, 2023). Among various techniques, character-level embeddings of numbers have emerged as a promising approach to improve number representation. However, this method has limitations as it leaves the task of aggregating digit representations to the model, which lacks direct supervision for this process. In this paper, we explore the use of mathematical priors to compute aggregated digit embeddings and explicitly incorporate these aggregates into transformer models. This can be achieved either by adding a special token to the input embeddings or by introducing an additional loss function to enhance correct predictions. We evaluate the effectiveness of incorporating this explicit aggregation, analysing its strengths and shortcomings, and discuss future directions to better benefit from this approach. Our methods, while simple, are compatible with any pretrained model and require only a few lines of code, which we have made publicly available.

Traffic accidents pose a severe global public health issue, leading to 1.19 million fatalities annually, with the greatest impact on individuals aged 5 to 29 years old. This paper addresses the critical need for advanced predictive methods in road safety by conducting a comprehensive review of recent advancements in applying machine learning (ML) techniques to traffic accident analysis and prediction. It examines 191 studies from the last five years, focusing on predicting accident risk, frequency, severity, duration, as well as general statistical analysis of accident data. To our knowledge, this study is the first to provide such a comprehensive review, covering the state-of-the-art across a wide range of domains related to accident analysis and prediction. The review highlights the effectiveness of integrating diverse data sources and advanced ML techniques to improve prediction accuracy and handle the complexities of traffic data. By mapping the current landscape and identifying gaps in the literature, this study aims to guide future research towards significantly reducing traffic-related deaths and injuries by 2030, aligning with the World Health Organization (WHO) targets., Comment: A review paper, 26 pages

In federated learning, a server must periodically broadcast a model to the agents. We propose to use multi-resolution coding and modulation (also known as non-uniform modulation) for this purpose. In the simplest instance, broadcast transmission is used, whereby all agents are targeted with one and the same transmission (typically without any particular favored beam direction), which is coded using multi-resolution coding/modulation. This enables high-SNR agents, with high path gains to the server, to receive a more accurate model than the low-SNR agents do, without consuming more downlink resources. As one implementation, we use transmission with a non-uniform 8-PSK constellation, where a high-SNR receiver (agent) can separate all 8 constellation points (hence receive 3 bits) whereas a low-SNR receiver can only separate 4 points (hence receive 2 bits). By encoding the least significant information in the third bit, the high-SNR receivers can obtain the model with higher accuracy, while the low-SNR receiver can still obtain the model although with reduced accuracy, thereby facilitating at least some basic participation of the low-SNR receiver. We show the effectiveness of our proposed scheme via experimentation using federated learning with the MNIST data-set.

In this study, we explore the dynamics of quantum entanglement using the negativity criterion for the W_zeta quantum state. We investigate changes in negativity in terms of anisotropy parameters, gamma, the strength of the external magnetic field applied to the spin chain, eta, the triple interaction strength, alpha. We examine how these parameters affect the entanglement properties of the system and discuss the implications for quantum information processing and quantum communication protocols. By analyzing the negativity of the W_zeta state under different conditions, we gain insights into the behaviour of entanglement in complex quantum systems. Our results shed light on the intricate interplay between various factors that influence quantum entanglement and provide a foundation for further investigations in this field of research., Comment: 14 pages, 9 figures

In this article, we investigate the entanglement evolution of three-qubit states in the presence of a spin chain environment. Utilizing negativity as a metric for entanglement assessment, we focus on the GHZ, W, and W_zeta quantum states as the initial system states. We explore and analyze the entanglement dynamics of these states based on various parameters., Comment: 18 pages, 10 figures

Trajectory prediction plays an essential role in autonomous vehicles. While numerous strategies have been developed to enhance the robustness of trajectory prediction models, these methods are predominantly heuristic and do not offer guaranteed robustness against adversarial attacks and noisy observations. In this work, we propose a certification approach tailored for the task of trajectory prediction. To this end, we address the inherent challenges associated with trajectory prediction, including unbounded outputs, and mutli-modality, resulting in a model that provides guaranteed robustness. Furthermore, we integrate a denoiser into our method to further improve the performance. Through comprehensive evaluations, we demonstrate the effectiveness of the proposed technique across various baselines and using standard trajectory prediction datasets. The code will be made available online: https://s-attack.github.io/

One of the commonly used seismic force-resisting systems in structures is Reinforced Concrete (RC) Intermediate Moment Frames (IMF). Although using the IMF is not allowed in high seismic hazard zones according to ASCE 7-10, it is permitted in both Iran's 2800 Seismic Standard and New Zealand's Seismic Code. This study investigates the seismic behavior of a reinforced concrete IMF subjected to earthquake excitations using shaking table tests on a 2D RC structural model which is designed under the regulations of ACI318-19. The scale factor of 1/2.78 is selected for the frame fabrication due to the size limit of the shaking table. The constructed model has three stories with a height as 115 cm for each story, the clear length of beams as 151 cm, and cross-sectional dimensions of columns and beams as 11 $\times$ 11 cm and 12 $\times$ 11 cm, respectively. The whole structure is supported by a foundation that is 173 cm long, 52 cm wide, and 22 cm deep. Columns and beams are reinforced with 8 mm diameter longitudinal ribbed bars and stirrups with 6 mm diameter. The tests are conducted in stages with increasing peak ground acceleration (PGA) till the failure of the frame. Sarpol-E-Zahab earthquake seismic record is adopted for the experiment. The structural responses (e.g., displacements, longitudinal bars' strain, crack propagation, accelerations) are monitored during the test using both conventional sensors and vison-based sensors. As a comparative study, both conventional sensors and computer vision techniques are used to monitor the health state and to analyze the structural dynamics of the scaled RC frame structure., Comment: 8th World Conference on Structural Control and Monitoring (8WCSCM), Orlando, FL. (2022)

Dehumanization, characterized as a subtle yet harmful manifestation of hate speech, involves denying individuals of their human qualities and often results in violence against marginalized groups. Despite significant progress in Natural Language Processing across various domains, its application in detecting dehumanizing language is limited, largely due to the scarcity of publicly available annotated data for this domain. This paper evaluates the performance of cutting-edge NLP models, including GPT-4, GPT-3.5, and LLAMA-2, in identifying dehumanizing language. Our findings reveal that while these models demonstrate potential, achieving a 70\% accuracy rate in distinguishing dehumanizing language from broader hate speech, they also display biases. They are over-sensitive in classifying other forms of hate speech as dehumanization for a specific subset of target groups, while more frequently failing to identify clear cases of dehumanization for other target groups. Moreover, leveraging one of the best-performing models, we automatically annotated a larger dataset for training more accessible models. However, our findings indicate that these models currently do not meet the high-quality data generation threshold necessary for this task.

Previous studies on framing have relied on manual analysis or fine-tuning models with limited annotated datasets. However, pre-trained models, with their diverse training backgrounds, offer a promising alternative. This paper presents a comprehensive analysis of GPT-4, GPT-3.5 Turbo, and FLAN-T5 models in detecting framing in news headlines. We evaluated these models in various scenarios: zero-shot, few-shot with in-domain examples, cross-domain examples, and settings where models explain their predictions. Our results show that explainable predictions lead to more reliable outcomes. GPT-4 performed exceptionally well in few-shot settings but often misinterpreted emotional language as framing, highlighting a significant challenge. Additionally, the results suggest that consistent predictions across multiple models could help identify potential annotation inaccuracies in datasets. Finally, we propose a new small dataset for real-world evaluation on headlines from a diverse set of topics.

Reducing traffic accidents is a crucial global public safety concern. Accident prediction is key to improving traffic safety, enabling proactive measures to be taken before a crash occurs, and informing safety policies, regulations, and targeted interventions. Despite numerous studies on accident prediction over the past decades, many have limitations in terms of generalizability, reproducibility, or feasibility for practical use due to input data or problem formulation. To address existing shortcomings, we propose CrashFormer, a multi-modal architecture that utilizes comprehensive (but relatively easy to obtain) inputs such as the history of accidents, weather information, map images, and demographic information. The model predicts the future risk of accidents on a reasonably acceptable cadence (i.e., every six hours) for a geographical location of 5.161 square kilometers. CrashFormer is composed of five components: a sequential encoder to utilize historical accidents and weather data, an image encoder to use map imagery data, a raw data encoder to utilize demographic information, a feature fusion module for aggregating the encoded features, and a classifier that accepts the aggregated data and makes predictions accordingly. Results from extensive real-world experiments in 10 major US cities show that CrashFormer outperforms state-of-the-art sequential and non-sequential models by 1.8% in F1-score on average when using ``sparse'' input data., Comment: The paper is accepted In 1st ACM SIGSPATIAL International Workshop on Advances in Urban-AI (UrbanAI 23), November 13, 2023, Hamburg, Germany

Various methods have recently been proposed to estimate causal effects with confidence intervals that are uniformly valid over a set of data generating processes when high-dimensional nuisance models are estimated by post-model-selection or machine learning estimators. These methods typically require that all the confounders are observed to ensure identification of the effects. We contribute by showing how valid semiparametric inference can be obtained in the presence of unobserved confounders and high-dimensional nuisance models. We propose uncertainty intervals which allow for unobserved confounding, and show that the resulting inference is valid when the amount of unobserved confounding is small relative to the sample size; the latter is formalized in terms of convergence rates. Simulation experiments illustrate the finite sample properties of the proposed intervals and investigate an alternative procedure that improves the empirical coverage of the intervals when the amount of unobserved confounding is large. Finally, a case study on the effect of smoking during pregnancy on birth weight is used to illustrate the use of the methods introduced to perform a sensitivity analysis to unobserved confounding., Comment: 20 pages, 2 figures

Advancements in machine learning and artificial intelligence are transforming materials discovery. Yet, the availability of structured experimental data remains a bottleneck. The vast corpus of scientific literature presents a valuable and rich resource of such data. However, manual dataset creation from these resources is challenging due to issues in maintaining quality and consistency, scalability limitations, and the risk of human error and bias. Therefore, in this work, we develop a chemist AI agent, powered by large language models (LLMs), to overcome these challenges by autonomously creating structured datasets from natural language text, ranging from sentences and paragraphs to extensive scientific research articles. Our chemist AI agent, Eunomia, can plan and execute actions by leveraging the existing knowledge from decades of scientific research articles, scientists, the Internet and other tools altogether. We benchmark the performance of our approach in three different information extraction tasks with various levels of complexity, including solid-state impurity doping, metal-organic framework (MOF) chemical formula, and property relations. Our results demonstrate that our zero-shot agent, with the appropriate tools, is capable of attaining performance that is either superior or comparable to the state-of-the-art fine-tuned materials information extraction methods. This approach simplifies compilation of machine learning-ready datasets for various materials discovery applications, and significantly ease the accessibility of advanced natural language processing tools for novice users in natural language. The methodology in this work is developed as an open-source software on https://github.com/AI4ChemS/Eunomia.

Automatic evaluation of generated textual content presents an ongoing challenge within the field of NLP. Given the impressive capabilities of modern language models (LMs) across diverse NLP tasks, there is a growing trend to employ these models in creating innovative evaluation metrics for automated assessment of generation tasks. This paper investigates a pivotal question: Do language model-driven evaluation metrics inherently exhibit bias favoring texts generated by the same underlying language model? Specifically, we assess whether prominent LM-based evaluation metrics (e.g. BARTScore, T5Score, and GPTScore) demonstrate a favorable bias toward their respective underlying LMs in the context of summarization tasks. Our findings unveil a latent bias, particularly pronounced when such evaluation metrics are used in a reference-free manner without leveraging gold summaries. These results underscore that assessments provided by generative evaluation models can be influenced by factors beyond the inherent text quality, highlighting the necessity of developing more reliable evaluation protocols in the future.

Learning from free-text human feedback is essential for dialog systems, but annotated data is scarce and usually covers only a small fraction of error types known in conversational AI. Instead of collecting and annotating new datasets from scratch, recent advances in synthetic dialog generation could be used to augment existing dialog datasets with the necessary annotations. However, to assess the feasibility of such an effort, it is important to know the types and frequency of free-text human feedback included in these datasets. In this work, we investigate this question for a variety of commonly used dialog datasets, including MultiWoZ, SGD, BABI, PersonaChat, Wizards-of-Wikipedia, and the human-bot split of the Self-Feeding Chatbot. Using our observations, we derive new taxonomies for the annotation of free-text human feedback in dialogs and investigate the impact of including such data in response generation for three SOTA language generation models, including GPT-2, LLAMA, and Flan-T5. Our findings provide new insights into the composition of the datasets examined, including error types, user response types, and the relations between them., Comment: Accepted to be presented at EMNLP 2023