Search

Variability is a fundamental signature for active galactic nuclei (AGN) activity, and serve as an unbiased indicator for rapid instability happened near the center supermassive black hole (BH). Previous studies showed that AGN variability does not have strong redshift evolution, and scales with their bolometric luminosity and BH mass, making it a powerful probe to identify low-mass, low-luminosity AGNs at high redshift. JWST has discovered a new population of high-redshift galaxies likely hosting moderate accreting BHs ($10^6\,M_\odot$) -- the little red dots (LRDs, $z=4-10$). In this paper, we study the variability of a sample of 21 LRDs with V-shaped SEDs in three JWST deep fields that also have reliable HST observations in closely paired filters at 1-2 um (rest-frame UV), with the time difference between 6 and 11 years. This LRD sample covers a redshift range of $3

Cultural biases in multilingual datasets pose significant challenges for their effectiveness as global benchmarks. These biases stem not only from language but also from the cultural knowledge required to interpret questions, reducing the practical utility of translated datasets like MMLU. Furthermore, translation often introduces artifacts that can distort the meaning or clarity of questions in the target language. A common practice in multilingual evaluation is to rely on machine-translated evaluation sets, but simply translating a dataset is insufficient to address these challenges. In this work, we trace the impact of both of these issues on multilingual evaluations and ensuing model performances. Our large-scale evaluation of state-of-the-art open and proprietary models illustrates that progress on MMLU depends heavily on learning Western-centric concepts, with 28% of all questions requiring culturally sensitive knowledge. Moreover, for questions requiring geographic knowledge, an astounding 84.9% focus on either North American or European regions. Rankings of model evaluations change depending on whether they are evaluated on the full portion or the subset of questions annotated as culturally sensitive, showing the distortion to model rankings when blindly relying on translated MMLU. We release Global-MMLU, an improved MMLU with evaluation coverage across 42 languages -- with improved overall quality by engaging with compensated professional and community annotators to verify translation quality while also rigorously evaluating cultural biases present in the original dataset. This comprehensive Global-MMLU set also includes designated subsets labeled as culturally sensitive and culturally agnostic to allow for more holistic, complete evaluation.

Quantum computing holds promise across various fields, particularly with the advent of Noisy Intermediate-Scale Quantum (NISQ) devices, which can outperform classical supercomputers in specific tasks. However, challenges such as noise and limited qubit capabilities hinder its practical applications. Variational Quantum Algorithms (VQAs) offer a viable strategy to achieve quantum advantage by combining quantum and classical computing. Leveraging on VQAs, the performance of Variational Quantum Classifiers (VQCs) is competitive with many classical classifiers. This work implements a VQC using a silicon-based quantum photonic microprocessor and a classical computer, demonstrating its effectiveness in nonlinear binary and multi-classification tasks. An efficient gradient free genetic algorithm is employed for training. The VQC's performance was evaluated on three synthetic binary classification tasks with square-, circular-, and sine-shape decision boundaries and a real-world multiclass Iris dataset. The accuracies on the three binary classification tasks were 87.5%, 92.5%, and 85.0%, respectively, and 98.8% on the real world Iris dataset, highlighting the platform's potential to handle complex data patterns.

Traditional electrostatic simulation are meshed-based methods which convert partial differential equations into an algebraic system of equations and their solutions are approximated through numerical methods. These methods are time consuming and any changes in their initial or boundary conditions will require solving the numerical problem again. Newer computational methods such as the physics informed neural net (PINN) similarly require re-training when boundary conditions changes. In this work, we propose an end-to-end deep learning approach to model parameter changes to the boundary conditions. The proposed method is demonstrated on the test problem of a long air-filled capacitor structure. The proposed approach is compared to plain vanilla deep learning (NN) and PINN. It is shown that our method can significantly outperform both NN and PINN under dynamic boundary condition as well as retaining its full capability as a forward model.

The goal of music style transfer is to convert a music performance by one instrument into another while keeping the musical contents unchanged. In this paper, we investigate another style transfer scenario called ``failed-music style transfer''. Unlike the usual music style transfer where the content remains the same and only the instrumental characteristics are changed, this scenario seeks to transfer the music from the source instrument to the target instrument which is deliberately performed off-pitch. Our work attempts to transfer normally played music into off-pitch recorder music, which we call ``failed-style recorder'', and study the results of the conversion. To carry out this work, we have also proposed a dataset of failed-style recorders for this task, called ``FR109 Dataset''. Such an experiment explores the music style transfer task in a more expressive setting, as the generated audio should sound like an ``off-pitch recorder'' while maintaining a certain degree of naturalness., Comment: Accepted by APSIPA 2024

Generalizable Neural Radiance Field (GNeRF) across scenes has been proven to be an effective way to avoid per-scene optimization by representing a scene with deep image features of source images. However, despite its potential for real-world applications, there has been limited research on the robustness of GNeRFs to different types of degradation present in the source images. The lack of such research is primarily attributed to the absence of a large-scale dataset fit for training a degradation-robust generalizable NeRF model. To address this gap and facilitate investigations into the degradation robustness of 3D reconstruction tasks, we construct the Objaverse Blur Dataset, comprising 50,000 images from over 1000 settings featuring multiple levels of blur degradation. In addition, we design a simple and model-agnostic module for enhancing the degradation robustness of GNeRFs. Specifically, by extracting 3D-aware features through a lightweight depth estimator and denoiser, the proposed module shows improvement on different popular methods in GNeRFs in terms of both quantitative and visual quality over varying degradation types and levels. Our dataset and code will be made publicly available.

Quantum evolution is crucial for the understanding of complex quantum systems. However, current implementations of time evolution on quantum photonic platforms face challenges of limited light source efficiency due to propagation loss and merely single-layer complexity. In this work, we present a loop quantum photonic chip (Loop-QPC) designed to efficiently simulate quantum dynamics over multiple time steps in a single chip. Our approach employs a recirculating loop structure to reuse computational resources and eliminate the need for multiple quantum tomography steps or chip reconfigurations. We experimentally demonstrate the dynamics of the spin-boson model on a low-loss Silicon Nitride (SiN) integrated photonic chip. The Loop-QPC achieves a three-step unitary evolution closely matching the theoretical predictions. These results establish the Loop-QPC as a promising method for efficient and scalable quantum simulation, advancing the development of quantum simulation on programmable photonic circuits.

This paper explores the transformative role of artificial intelligence (AI) in enhancing scientific research, particularly in the fields of brain science and social sciences. We analyze the fundamental aspects of human research and argue that it is high time for researchers to transition to human-AI joint research. Building upon this foundation, we propose two innovative research paradigms of human-AI joint research: "AI-Brain Science Research Paradigm" and "AI-Social Sciences Research Paradigm". In these paradigms, we introduce three human-AI collaboration models: AI as a research tool (ART), AI as a research assistant (ARA), and AI as a research participant (ARP). Furthermore, we outline the methods for conducting human-AI joint research. This paper seeks to redefine the collaborative interactions between human researchers and AI system, setting the stage for future research directions and sparking innovation in this interdisciplinary field., Comment: Accepted to IEIR 2024, 10 pages, 4 figures

Image editing involves a variety of complex tasks and requires efficient and precise manipulation techniques. In this paper, we present MagicQuill, an integrated image editing system that enables swift actualization of creative ideas. Our system features a streamlined yet functionally robust interface, allowing for the articulation of editing operations (e.g., inserting elements, erasing objects, altering color) with minimal input. These interactions are monitored by a multimodal large language model (MLLM) to anticipate editing intentions in real time, bypassing the need for explicit prompt entry. Finally, we apply a powerful diffusion prior, enhanced by a carefully learned two-branch plug-in module, to process editing requests with precise control. Experimental results demonstrate the effectiveness of MagicQuill in achieving high-quality image edits. Please visit https://magic-quill.github.io to try out our system., Comment: Code and demo available at https://magic-quill.github.io

The alignment of large language models (LLMs) with human preferences remains a key challenge. While post-training techniques like Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO) have achieved notable success, they often introduce computational inefficiencies and training instability. In this paper, we propose Feature-level constrained Preference Optimization (FPO), a novel method designed to simplify the alignment process while ensuring stability. FPO leverages pre-trained Sparse Autoencoders (SAEs) and introduces feature-level constraints, allowing for efficient, sparsity-enforced alignment. Our approach enjoys efficiency by using sparse features activated in a well-trained sparse autoencoder and the quality of sequential KL divergence by using the feature-level offline reference. Experimental results on benchmark datasets demonstrate that FPO achieves a 5.08% absolute improvement in win rate with much lower computational cost compared to state-of-the-art baselines, making it a promising solution for efficient and controllable LLM alignments.

We report the discovery of a unique quasar-dusty star-forming galaxy (DSFG) system at $z = 5.63$, consisting of the bright quasar J1133+1603 ($M_{\rm UV} = -27.42$) and its compact, dust-obscured companion, J1133c. ALMA observations reveal a prominent [C II] bridge connecting the quasar and DSFG, indicating ongoing interaction at a projected separation of 1.8 arcsec ($\sim$10 proper kpc). J1133c exhibits unusually bright and broad [C II] emission ($L_{\rm [CII]} > 10^{43}$ erg s$^{-1}$, FWHM $> 500$ km s$^{-1}$), with a [C II] luminosity five times that of the quasar, suggesting intense star formation or potential AGN activity. The inferred star formation rate from [C II] is approximately $10^3$ M$_\odot$ yr$^{-1}$. The remarkable properties of this pair strongly suggest that galaxy interactions may simultaneously trigger both starburst and quasar activity, driving rapid evolution in the early universe., Comment: RNAAS; 4 pages, 1 figure; comments welcome

This paper presents an innovative pseudo-haptic model for weight simulation in virtual reality (VR) environments. By integrating visual feedback with voluntary exerted force through a passive haptic glove, the model creates haptic illusions of weight perception. Two VR cube games were developed to evaluate the model's effectiveness. The first game assesses participants' ability to discriminate relative weights, while the second evaluates their capability to estimate absolute weights. Twelve participants, aged 18 to 59, tested the games. Results suggest that the pseudo-haptic model is effective for relative weight discrimination tasks and holds potential for various VR applications. Further research with a larger participant group and more complex scenarios is recommended to refine and validate the model., Comment: Part of proceedings of 6th International Conference AsiaHaptics 2024

Electronic maps consist of diverse entities, such as points of interest (POIs), road networks, and land parcels, playing a vital role in applications like ITS and LBS. Map entity representation learning (MapRL) generates versatile and reusable data representations, providing essential tools for efficiently managing and utilizing map entity data. Despite the progress in MapRL, two key challenges constrain further development. First, existing research is fragmented, with models classified by the type of map entity, limiting the reusability of techniques across different tasks. Second, the lack of unified benchmarks makes systematic evaluation and comparison of models difficult. To address these challenges, we propose a novel taxonomy for MapRL that organizes models based on functional module-such as encoders, pre-training tasks, and downstream tasks-rather than by entity type. Building on this taxonomy, we present a taxonomy-driven library, VecCity, which offers easy-to-use interfaces for encoding, pre-training, fine-tuning, and evaluation. The library integrates datasets from nine cities and reproduces 21 mainstream MapRL models, establishing the first standardized benchmarks for the field. VecCity also allows users to modify and extend models through modular components, facilitating seamless experimentation. Our comprehensive experiments cover multiple types of map entities and evaluate 21 VecCity pre-built models across various downstream tasks. Experimental results demonstrate the effectiveness of VecCity in streamlining model development and provide insights into the impact of various components on performance. By promoting modular design and reusability, VecCity offers a unified framework to advance research and innovation in MapRL. The code is available at https://github.com/Bigscity-VecCity/VecCity., Comment: under review

Deep Reinforcement Learning (DRL) has achieved remarkable success in solving complex decision-making problems by combining the representation capabilities of deep learning with the decision-making power of reinforcement learning. However, learning in sparse reward environments remains challenging due to insufficient feedback to guide the optimization of agents, especially in real-life environments with high-dimensional states. To tackle this issue, experience replay is commonly introduced to enhance learning efficiency through past experiences. Nonetheless, current methods of experience replay, whether based on uniform or prioritized sampling, frequently struggle with suboptimal learning efficiency and insufficient utilization of samples. This paper proposes a novel approach, diversity-based experience replay (DBER), which leverages the deterministic point process to prioritize diverse samples in state realizations. We conducted extensive experiments on Robotic Manipulation tasks in MuJoCo, Atari games, and realistic in-door environments in Habitat. The results show that our method not only significantly improves learning efficiency but also demonstrates superior performance in sparse reward environments with high-dimensional states, providing a simple yet effective solution for this field.

Large Language Models (LLMs) have exhibited strong mathematical reasoning and computational prowess, tackling tasks ranging from basic arithmetic to advanced competition-level problems. However, frequently occurring subtle errors, such as miscalculations or incorrect substitutions, limit the models' full mathematical potential. Existing studies to improve mathematical ability typically involve distilling reasoning skills from stronger LLMs or applying preference learning to step-wise response pairs. Although these methods leverage samples of varying granularity to mitigate reasoning errors, they overlook the frequently occurring subtle errors. A major reason is that sampled preference pairs involve differences unrelated to the errors, which may distract the model from focusing on subtle errors. In this work, we propose a novel preference learning framework called eRror-Injected Self-Editing (RISE), which injects predefined subtle errors into partial tokens of correct solutions to construct hard pairs for error mitigation. In detail, RISE uses the model itself to edit a small number of tokens in the solution, injecting designed subtle errors. Then, pairs composed of self-edited solutions and their corresponding correct ones, along with pairs of correct and incorrect solutions obtained through sampling, are used together for subtle error-aware DPO training. Compared with other preference learning methods, RISE further refines the training objective to focus on predefined errors and their tokens, without requiring fine-grained sampling or preference annotation. Extensive experiments validate the effectiveness of RISE, with preference learning on Qwen2-7B-Instruct yielding notable improvements of 3.0% on GSM8K and 7.9% on MATH.

Fine-tuning and in-context learning (ICL) are two prevalent methods in imbuing large language models with task-specific knowledge. It is commonly believed that fine-tuning can surpass ICL given sufficient training samples as it allows the model to adjust its internal parameters based on the data. However, this paper presents a counterintuitive finding: For tasks with implicit patterns, ICL captures these patterns significantly better than fine-tuning. We developed several datasets featuring implicit patterns, such as sequences determining answers through parity or identifying reducible terms in calculations. We then evaluated the models' understanding of these patterns under both fine-tuning and ICL across models ranging from 0.5B to 7B parameters. The results indicate that models employing ICL can quickly grasp deep patterns and significantly improve accuracy. In contrast, fine-tuning, despite utilizing thousands of times more training samples than ICL, achieved only limited improvements. We also proposed circuit shift theory from a mechanistic interpretability's view to explain why ICL wins., Comment: EMNLP'24 Findings

We present paper II comprising a 35 arcmin$^2$ JWST/NIRCam imaging and wide-field slitless spectroscopy mosaic centered on J0305$-$3150, a luminous quasar at $z=6.61$. The F356W grism data reveals 124 [OIII]+H$\beta$ emitters at $5.3

ASPIRE (A SPectroscopic survey of bIased halos in the Reionization Era) is a quasar legacy survey primarily using JWST to target a sample of 25 $z>6$ quasars with NIRCam slitless spectroscopy and imaging. The first study in this series found evidence of a strong overdensity of galaxies around J0305-3150, a luminous quasar at $z=6.61$, within a single NIRCam pointing obtained in JWST Cycle 1. Here, we present the first results of a JWST Cycle 2 mosaic that covers 35 arcmin$^2$ with NIRCam imaging/WFSS of the same field to investigate the spatial extent of the putative protocluster. The F356W grism data targets [OIII]+H$\beta$ at $5.310$ cMpc., Comment: submitted to ApJ june 2024. 18 pages, 7 figures. companion paper II by Champagne+24b is also available on arXiv today

Understanding when and how reionization happened is crucial for studying the early structure formation and the properties of first galaxies in the Universe. At $z>5.5$, the observed IGM optical depth shows a significant scatter, indicating an inhomogeneous reionization process. However, the nature of the inhomogeneous reionization remains debated. ASPIRE is a JWST Cycle 1 program that has spectroscopically identified $>400$ [OIII] emitters in 25 quasar fields at $z>6.5$. Combined with deep ground-based optical spectroscopy of ASPIRE quasars, ASPIRE program provides the current largest sample for IGM-galaxy connection studies during cosmic reionization. We present the first results of IGM effective optical depth measurements around [OIII] emitters using 14 ASPIRE quasar fields. We find the IGM transmission is tightly related with reionization-era galaxies to the extent that significant excess of Ly$\alpha$ transmission exists around [OIII] emitters. We measure the stacked IGM effective optical depth of IGM patches associated with [OIII] emitters and find they reach the same IGM effective optical depth at least dz~0.1 ahead of those IGM patches where no [OIII] emitters are detected, supporting earlier reionization around [OIII] emitters. Our results indicate an enhancement in IGM Ly$\alpha$ transmission around [OIII] emitters at scales beyond 25 $h^{-1}$ cMpc, consistent with the predicted topology of reionization from fluctuating UV background (UVB) models., Comment: 27 pages, 19 figures, 1 table, accepted for publication in ApJ

This brief highlights the urgent need to address chronic absenteeism among American Indian and Alaska Native (AI/AN) students in the United States. It explores the challenges that AI/AN students and their families face in terms of regular school attendance. Systemic barriers, unwelcoming?school environments, biased disciplinary practices, and a lack of culturally relevant curricula are all referenced as?major contributors to chronic absenteeism. The brief emphasizes the importance of working with parents, tribal leaders, community members, educators, policymakers, and other allies to improve AI/AN student attendance rates. Recommendations include investing in positive learning environments tailored specifically to these students' needs, implementing tiered approaches that include prevention strategies, and enacting targeted interventions aimed at removing barriers to attendance. This brief provides practical solutions,?focusing?on successful initiatives such as attendance recognition programs, inclusive graduation events that foster a sense of belonging, and collaboration with tribal governments to provide transportation or medical services.

The increasing administrative burden of medical documentation, particularly through Electronic Health Records (EHR), significantly reduces the time available for direct patient care and contributes to physician burnout. To address this issue, we propose MediNotes, an advanced generative AI framework designed to automate the creation of SOAP (Subjective, Objective, Assessment, Plan) notes from medical conversations. MediNotes integrates Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and Automatic Speech Recognition (ASR) to capture and process both text and voice inputs in real time or from recorded audio, generating structured and contextually accurate medical notes. The framework also incorporates advanced techniques like Quantized Low-Rank Adaptation (QLoRA) and Parameter-Efficient Fine-Tuning (PEFT) for efficient model fine-tuning in resource-constrained environments. Additionally, MediNotes offers a query-based retrieval system, allowing healthcare providers and patients to access relevant medical information quickly and accurately. Evaluations using the ACI-BENCH dataset demonstrate that MediNotes significantly improves the accuracy, efficiency, and usability of automated medical documentation, offering a robust solution to reduce the administrative burden on healthcare professionals while improving the quality of clinical workflows., Comment: 8 Figures, 7 page, IEEE standard research paper

Multilingual large language models (LLMs) today may not necessarily provide culturally appropriate and relevant responses to its Filipino users. We introduce Kalahi, a cultural LLM evaluation suite collaboratively created by native Filipino speakers. It is composed of 150 high-quality, handcrafted and nuanced prompts that test LLMs for generations that are relevant to shared Filipino cultural knowledge and values. Strong LLM performance in Kalahi indicates a model's ability to generate responses similar to what an average Filipino would say or do in a given situation. We conducted experiments on LLMs with multilingual and Filipino language support. Results show that Kalahi, while trivial for Filipinos, is challenging for LLMs, with the best model answering only 46.0% of the questions correctly compared to native Filipino performance of 89.10%. Thus, Kalahi can be used to accurately and reliably evaluate Filipino cultural representation in LLMs., Comment: Accepted for presentation at Paclic 38, 2024

James Webb Space Telescope opens a new window to directly probe luminous quasars powered by billion solar mass black holes in the epoch of reionization and their co-evolution with massive galaxies with unprecedented details. In this paper, we report the first results from the deep NIRSpec integral field spectroscopy study of a quasar at $z = 7.5$. We obtain a bolometric luminosity of $\sim$$1.8\times10^{47}$ erg s$^{-1}$ and a black hole mass of $\sim$0.7--2.5$\times10^{9}$ M$_{\odot}$ based on H$\beta$ emission line from the quasar spectrum. We discover $\sim$2 kpc scale, highly blueshifted ($\sim$$-$870 km/s) and broad ($\sim$1400 km/s) [O III] line emission after the quasar PSF has been subtracted. Such line emission most likely originates from a fast, quasar-driven outflow, the earliest one on galactic-scale known so far. The dynamical properties of this outflow fall within the typical ranges of quasar-driven outflows at lower redshift, and the outflow may be fast enough to reach the circumgalactic medium. Combining both the extended and nuclear outflow together, the mass outflow rate, $\sim$300 M$_{\odot}$yr, is $\sim$60%--380% of the star formation rate of the quasar host galaxy, suggesting that the outflow may expel a significant amount of gas from the inner region of the galaxy. The kinetic energy outflow rate, $\sim$3.6$\times10^{44}$ erg s$^{-1}$, is $\sim$0.2% of the quasar bolometric luminosity, which is comparable to the minimum value required for negative feedback based on simulation predictions. The dynamical timescale of the extended outflow is $\sim$1.7 Myr, consistent with the typical quasar lifetime in this era.

Emotions, shaped by past experiences, significantly influence decision-making and goal pursuit. Traditional cognitive-behavioral techniques for personal development rely on mental imagery to envision ideal selves, but may be less effective for individuals who struggle with visualization. This paper introduces Emotional Self-Voice (ESV), a novel system combining emotionally expressive language models and voice cloning technologies to render customized responses in the user's own voice. We investigate the potential of ESV to nudge individuals towards their ideal selves in a study with 60 participants. Across all three conditions (ESV, text-only, and mental imagination), we observed an increase in resilience, confidence, motivation, and goal commitment, but the ESV condition was perceived as uniquely engaging and personalized. We discuss the implications of designing generated self-voice systems as a personalized behavioral intervention for different scenarios.

LLMs are an integral component of retrieval-augmented generation (RAG) systems. While many studies focus on evaluating the overall quality of end-to-end RAG systems, there is a gap in understanding the appropriateness of LLMs for the RAG task. To address this, we introduce Trust-Score, a holistic metric that evaluates the trustworthiness of LLMs within the RAG framework. Our results show that various prompting methods, such as in-context learning, fail to effectively adapt LLMs to the RAG task as measured by Trust-Score. Consequently, we propose Trust-Align, a method to align LLMs for improved Trust-Score performance. The LLaMA-3 family, aligned using our method, significantly outperforms open-source LLMs of similar sizes on ASQA (up 14.0), QAMPARI (up 28.9), and ELI5 (up 13.7). We also demonstrate the effectiveness of Trust-Align across different open-weight models, including the LLaMA series (1b to 8b), Qwen-2.5 series (0.5b to 7b), and Phi3.5 (3.8b). We release our code at \url{https://anonymous.4open.science/r/trust-align}

Machine learning has been pervasively touching many fields of science. Chemistry and materials science are no exception. While machine learning has been making a great impact, it is still not reaching its full potential or maturity. In this perspective, we first outline current applications across a diversity of problems in chemistry. Then, we discuss how machine learning researchers view and approach problems in the field. Finally, we provide our considerations for maximizing impact when researching machine learning for chemistry.

Scientific research indicates that for every hour spent in direct patient care, physicians spend nearly two additional hours on administrative tasks, particularly on electronic health records (EHRs) and desk work. This excessive administrative burden not only reduces the time available for patient care but also contributes to physician burnout and inefficiencies in healthcare delivery. To address these challenges, this study introduces MediGen, a fine-tuned large language model (LLM) designed to automate the generation of medical reports from medical dialogues. By leveraging state-of-the-art methodologies for fine-tuning open-source pretrained models, including LLaMA3-8B, MediGen achieves high accuracy in transcribing and summarizing clinical interactions. The fine-tuned LLaMA3-8B model demonstrated promising results, achieving a ROUGE score of 58% and a BERTScore-F1 of 72%, indicating its effectiveness in generating accurate and clinically relevant medical reports. These findings suggest that MediGen has the potential to significantly reduce the administrative workload on physicians, improving both healthcare efficiency and physician well-being., Comment: 4 pages, 3 Figures, 3 Tables. The final version will be published in the proceedings of the IEEE conference

Real-world problems often involve complex objective structures that resist distillation into reinforcement learning environments with a single objective. Operation costs must be balanced with multi-dimensional task performance and end-states' effects on future availability, all while ensuring safety for other agents in the environment and the reinforcement learning agent itself. System redundancy through secondary backup controllers has proven to be an effective method to ensure safety in real-world applications where the risk of violating constraints is extremely high. In this work, we investigate the utility of a stacked, continuous-control variation of universal successor feature approximation (USFA) adapted for soft actor-critic (SAC) and coupled with a suite of secondary safety controllers, which we call stacked USFA for safety (SUSFAS). Our method improves performance on secondary objectives compared to SAC baselines using an intervening secondary controller such as a runtime assurance (RTA) controller., Comment: 13 pages

Language models are exhibiting increasing capability in knowledge utilization and reasoning. However, when applied as agents in embodied environments, they often suffer from misalignment between their intrinsic knowledge and environmental knowledge, leading to infeasible actions. Traditional environment alignment methods, such as supervised learning on expert trajectories and reinforcement learning, encounter limitations in covering environmental knowledge and achieving efficient convergence, respectively. Inspired by human learning, we propose Exploration-based Error Correction Learning (E2CL), a novel framework that leverages exploration-induced errors and environmental feedback to enhance environment alignment for embodied agents. E2CL incorporates teacher-guided and teacher-free explorations to gather environmental feedback and correct erroneous actions. The agent learns to provide feedback and self-correct, thereby enhancing its adaptability to target environments. Extensive experiments in the VirtualHome environment demonstrate that E2CL-trained agents outperform those trained by baseline methods and exhibit superior self-correction capabilities., Comment: Accepted by EMNLP 2024

Variational regularization is a classical technique to solve statistical inference tasks and inverse problems, with modern data-driven approaches parameterizing regularizers via deep neural networks showcasing impressive empirical performance. Recent works along these lines learn task-dependent regularizers. This is done by integrating information about the measurements and ground-truth data in an unsupervised, critic-based loss function, where the regularizer attributes low values to likely data and high values to unlikely data. However, there is little theory about the structure of regularizers learned via this process and how it relates to the two data distributions. To make progress on this challenge, we initiate a study of optimizing critic-based loss functions to learn regularizers over a particular family of regularizers: gauges (or Minkowski functionals) of star-shaped bodies. This family contains regularizers that are commonly employed in practice and shares properties with regularizers parameterized by deep neural networks. We specifically investigate critic-based losses derived from variational representations of statistical distances between probability measures. By leveraging tools from star geometry and dual Brunn-Minkowski theory, we illustrate how these losses can be interpreted as dual mixed volumes that depend on the data distribution. This allows us to derive exact expressions for the optimal regularizer in certain cases. Finally, we identify which neural network architectures give rise to such star body gauges and when do such regularizers have favorable properties for optimization. More broadly, this work highlights how the tools of star geometry can aid in understanding the geometry of unsupervised regularizer learning., Comment: Accepted to NeurIPS 2024

The dense and dynamic environments within active galactic nuclei (AGN) accretion disks may serve as prolific birthplaces for binary black holes (BBHs) and one possible origin for some of the BBHs detected by gravitational-wave (GW) observatories. We show that a considerable fraction of the BBH in AGN disks will be strongly lensed by the central supermassive black hole (SMBH). Thus, the non-observation of lensed GW signals can be used to constrain the fraction of BBH binaries residing in AGN disks. The non-detection of lensing with current ${\cal O}(100)$ detections will be sufficient to start placing constraints on the fraction of BBHs living within accretion disks near the SMBH. In the next-generation detectors era, with ${\cal O}(10^5)$ BBH observations and no lensed events, we will be able to rule out most migration traps as dominant birthplaces of BBH mergers; moreover, we will be able to constrain the minimum size of the accretion disk. On the other hand, should AGNs constitute a major formation channel, lensed events from AGNs will become prominent in the future., Comment: 8 pages, 4 figures

Reward shaping is effective in addressing the sparse-reward challenge in reinforcement learning by providing immediate feedback through auxiliary informative rewards. Based on the reward shaping strategy, we propose a novel multi-task reinforcement learning framework, that integrates a centralized reward agent (CRA) and multiple distributed policy agents. The CRA functions as a knowledge pool, which aims to distill knowledge from various tasks and distribute it to individual policy agents to improve learning efficiency. Specifically, the shaped rewards serve as a straightforward metric to encode knowledge. This framework not only enhances knowledge sharing across established tasks but also adapts to new tasks by transferring valuable reward signals. We validate the proposed method on both discrete and continuous domains, demonstrating its robustness in multi-task sparse-reward settings and its effective transferability to unseen tasks.

This paper introduces a reinforcement learning (RL) platform that enhances end-to-end user journeys in healthcare digital tools through personalization. We explore a case study with SwipeRx, the most popular all-in-one app for pharmacists in Southeast Asia, demonstrating how the platform can be used to personalize and adapt user experiences. Our RL framework is tested through a series of experiments with product recommendations tailored to each pharmacy based on real-time information on their purchasing history and in-app engagement, showing a significant increase in basket size. By integrating adaptive interventions into existing mobile health solutions and enriching user journeys, our platform offers a scalable solution to improve pharmaceutical supply chain management, health worker capacity building, and clinical decision and patient care, ultimately contributing to better healthcare outcomes., Comment: Presented at the Third Workshop on End-to-End Customer Journey Optimization at KDD 2024 (KDD CJ Workshop '24), August 26, Barcelona, Spain

Pharmacies are critical in healthcare systems, particularly in low- and middle-income countries. Procuring pharmacists with the right behavioral interventions or nudges can enhance their skills, public health awareness, and pharmacy inventory management, ensuring access to essential medicines that ultimately benefit their patients. We introduce a reinforcement learning operational system to deliver personalized behavioral interventions through mobile health applications. We illustrate its potential by discussing a series of initial experiments run with SwipeRx, an all-in-one app for pharmacists, including B2B e-commerce, in Indonesia. The proposed method has broader applications extending beyond pharmacy operations to optimize healthcare delivery., Comment: Presented at The First Workshop on AI Behavioral Science (AIBS'24) at KDD 2024, August 25, Barcelona, Spain

Electromagnetic follow-up observations of gravitational wave events offer critical insights and provide significant scientific gain from this new class of astrophysical transients. Accurate identification of gravitational wave candidates and rapid release of sky localization information are crucial for the success of these electromagnetic follow-up observations. However, searches for gravitational wave candidates in real time suffer a non-negligible false alarm rate. By leveraging the sky localization information and other metadata associated with gravitational wave candidates, GWSkyNet, a machine learning classifier developed by Cabero et al. (2020), demonstrated promising accuracy for the identification of the origin of event candidates. We improve the performance of the classifier for LIGO-Virgo-KAGRA's fourth observing run by reviewing and updating the architecture and features used as inputs by the algorithm. We also retrain and fine-tune the classifier with data from the third observing run. To improve the prospect of electromagnetic follow-up observations, we incorporate GWSkyNet into LIGO-Virgo-KAGRA's low-latency infrastructure as an automatic pipeline for the evaluation of gravitational wave alerts in real time. We test the readiness of the algorithm on a LIGO-Virgo-KAGRA mock data challenge campaign. The results show that by thresholding on the GWSkyNet score, noise masquerading as astrophysical sources can be rejected efficiently and the majority of true astrophysical signals correctly identified.

In this article, we study the summatory function \begin{equation*} W(x)=\sum_{n\leq x}(-2)^{\Omega(n)}, \end{equation*} where $\Omega(n)$ counts the number of prime factors of $n$, with multiplicity. We prove $W(x)=O(x)$, and in particular, that $|W(x)|<2260x$ for all $x\geq 1$. This result provides new progress towards a conjecture of Sun, which asks whether $|W(x)|0$., Comment: 35 pages

Reward shaping is a technique in reinforcement learning that addresses the sparse-reward problem by providing more frequent and informative rewards. We introduce a self-adaptive and highly efficient reward shaping mechanism that incorporates success rates derived from historical experiences as shaped rewards. The success rates are sampled from Beta distributions, which dynamically evolve from uncertain to reliable values as data accumulates. Initially, the shaped rewards exhibit more randomness to encourage exploration, while over time, the increasing certainty enhances exploitation, naturally balancing exploration and exploitation. Our approach employs Kernel Density Estimation (KDE) combined with Random Fourier Features (RFF) to derive the Beta distributions, providing a computationally efficient, non-parametric, and learning-free solution for high-dimensional continuous state spaces. Our method is validated on various tasks with extremely sparse rewards, demonstrating notable improvements in sample efficiency and convergence stability over relevant baselines.

Low-luminosity AGNs with low-mass black holes (BHs) in the early universe are fundamental to understanding the BH growth and their co-evolution with the host galaxies. Utilizing JWST NIRCam Wide Field Slitless Spectroscopy (WFSS), we perform a systematic search for broad-line ${\rm H\alpha}$ emitters (BHAEs) at $z\approx 4-5$ in 25 fields of the ASPIRE (A SPectroscopic survey of biased halos In the Reionization Era) project, covering a total area of 275 arcmin$^2$. We identify 16 BHAEs with FWHM of the broad components spanning from $\sim$ 1000 km s$^{-1}$ to 3000 km s$^{-1}$. Assuming the broad linewidths arise due to Doppler broadening around BHs, the implied BH masses range from $10^7$ to $10^{8}~M_\odot$, with broad ${\rm H\alpha}$-converted bolometric luminosity of $10^{44.5}-10^{45.5}$ erg s$^{-1}$ and Eddington ratios of $0.07-0.47$. The spatially extended structure of the F200W stacked image may trace the stellar light from the host galaxies. The ${\rm H\alpha}$ luminosity function indicates an increasing AGN fraction towards the higher ${\rm H\alpha}$ luminosities. We find possible evidence for clustering of BHAEs: two sources are at the same redshift with a projected separation of 519 kpc; one BHAE appears as a composite system residing in an overdense region with three close companion ${\rm H\alpha}$ emitters. Three BHAEs exhibit blueshifted absorption troughs indicative of the presence of high-column-density gas. We find the broad-line and photometrically selected BHAE samples exhibit different distributions in the optical continuum slopes, which can be attributed to their different selection methods. The ASPIRE broad-line ${\rm H\alpha}$ sample provides a good database for future studies of faint AGN populations at high redshift., Comment: 19 pages, 13 figures, 4 tables. Accepted by the ApJ

Object detection can localize and identify objects in images, and it is extensively employed in critical multimedia applications such as security surveillance and autonomous driving. Despite the success of existing object detection models, they are often evaluated in ideal scenarios where captured images guarantee the accurate and complete representation of the detecting scenes. However, images captured by image sensors may be affected by different factors in real applications, including cyber-physical attacks. In particular, attackers can exploit hardware properties within the systems to inject electromagnetic interference so as to manipulate the images. Such attacks can cause noisy or incomplete information about the captured scene, leading to incorrect detection results, potentially granting attackers malicious control over critical functions of the systems. This paper presents a research work that comprehensively quantifies and analyzes the impacts of such attacks on state-of-the-art object detection models in practice. It also sheds light on the underlying reasons for the incorrect detection outcomes., Comment: 2024 IEEE International Conference on Multimedia and Expo (ICME), July 15 - July 19, 2024, Niagra Falls, Ontario, Canada

Action recognition technology plays a vital role in enhancing security through surveillance systems, enabling better patient monitoring in healthcare, providing in-depth performance analysis in sports, and facilitating seamless human-AI collaboration in domains such as manufacturing and assistive technologies. The dynamic nature of data in these areas underscores the need for models that can continuously adapt to new video data without losing previously acquired knowledge, highlighting the critical role of advanced continual action recognition. To address these challenges, we propose Decoupled Prompt-Adapter Tuning (DPAT), a novel framework that integrates adapters for capturing spatial-temporal information and learnable prompts for mitigating catastrophic forgetting through a decoupled training strategy. DPAT uniquely balances the generalization benefits of prompt tuning with the plasticity provided by adapters in pretrained vision models, effectively addressing the challenge of maintaining model performance amidst continuous data evolution without necessitating extensive finetuning. DPAT consistently achieves state-of-the-art performance across several challenging action recognition benchmarks, thus demonstrating the effectiveness of our model in the domain of continual action recognition.

Although passivization is productive in English, it is not completely general -- some exceptions exist (e.g. *One hour was lasted by the meeting). How do English speakers learn these exceptions to an otherwise general pattern? Using neural network language models as theories of acquisition, we explore the sources of indirect evidence that a learner can leverage to learn whether a verb can passivize. We first characterize English speakers' judgments of exceptions to the passive, confirming that speakers find some verbs more passivizable than others. We then show that a neural network language model can learn restrictions to the passive that are similar to those displayed by humans, suggesting that evidence for these exceptions is available in the linguistic input. We test the causal role of two hypotheses for how the language model learns these restrictions by training models on modified training corpora, which we create by altering the existing training corpora to remove features of the input implicated by each hypothesis. We find that while the frequency with which a verb appears in the passive significantly affects its passivizability, the semantics of the verb does not. This study highlight the utility of altering a language model's training data for answering questions where complete control over a learner's input is vital., Comment: Submitted to Journal of Memory and Language

Solid-state quantum emitters are essential sources of single photons, and enhancing their emission rates is of paramount importance for applications in quantum communications, computing, and metrology. One approach is to couple quantum emitters with resonant photonic nanostructures, where the emission rate is enhanced due to the Purcell effect. Dielectric nanoantennas are promising as they provide strong emission enhancement compared to plasmonic ones, which suffer from high Ohmic loss. Here, we designed and fabricated a dielectric Fano resonator based on a pair of silicon (Si) ellipses and a disk, which supports the mode hybridization between quasi-bound-states-in-the-continuum (quasi-BIC) and Mie resonance. We demonstrated the performance of the developed resonant system by interfacing it with single photon emitters (SPEs) based on nitrogen-vacancy (NV-) centers in nanodiamonds (NDs). We observed that the interfaced emitters have a Purcell enhancement factor of ~10, with sub-ns emission lifetime and a polarization contrast of 9. Our results indicate a promising method for developing efficient and compact single-photon sources for integrated quantum photonics applications., Comment: 20 pages, 4 figures