Search

The schooling behavior of fish is hypothesized to confer many survival benefits, including foraging success, safety from predators, and energy savings through hydrodynamic interactions when swimming in formation. Underwater robot collectives may be able to achieve similar benefits in future applications, e.g. using formation control to achieve efficient spatial sampling for environmental monitoring. Although many theoretical algorithms exist for multi-robot formation control, they have not been tested in the underwater domain due to the fundamental challenges in underwater communication. Here we introduce a leader-follower strategy for underwater formation control that allows us to realize complex 3D formations, using purely vision-based perception and a reactive control algorithm that is low computation. We use a physical platform, BlueSwarm, to demonstrate for the first time an experimental realization of inline, side-by-side, and staggered swimming 3D formations. More complex formations are studied in a physics-based simulator, providing new insights into the convergence and stability of formations given underwater inertial/drag conditions. Our findings lay the groundwork for future applications of underwater robot swarms in aquatic environments with minimal communication., Comment: Accepted at DARS 2024 (The 17th International Symposium on Distributed Autonomous Robotic Systems)

This paper presents a theoretical framework for an AI-driven data quality monitoring system designed to address the challenges of maintaining data quality in high-volume environments. We examine the limitations of traditional methods in managing the scale, velocity, and variety of big data and propose a conceptual approach leveraging advanced machine learning techniques. Our framework outlines a system architecture that incorporates anomaly detection, classification, and predictive analytics for real-time, scalable data quality management. Key components include an intelligent data ingestion layer, adaptive preprocessing mechanisms, context-aware feature extraction, and AI-based quality assessment modules. A continuous learning paradigm is central to our framework, ensuring adaptability to evolving data patterns and quality requirements. We also address implications for scalability, privacy, and integration within existing data ecosystems. While practical results are not provided, it lays a robust theoretical foundation for future research and implementations, advancing data quality management and encouraging the exploration of AI-driven solutions in dynamic environments.

A promising approach for improving reasoning in large language models is to use process reward models (PRMs). PRMs provide feedback at each step of a multi-step reasoning trace, potentially improving credit assignment over outcome reward models (ORMs) that only provide feedback at the final step. However, collecting dense, per-step human labels is not scalable, and training PRMs from automatically-labeled data has thus far led to limited gains. To improve a base policy by running search against a PRM or using it as dense rewards for reinforcement learning (RL), we ask: "How should we design process rewards?". Our key insight is that, to be effective, the process reward for a step should measure progress: a change in the likelihood of producing a correct response in the future, before and after taking the step, corresponding to the notion of step-level advantages in RL. Crucially, this progress should be measured under a prover policy distinct from the base policy. We theoretically characterize the set of good provers and our results show that optimizing process rewards from such provers improves exploration during test-time search and online RL. In fact, our characterization shows that weak prover policies can substantially improve a stronger base policy, which we also observe empirically. We validate our claims by training process advantage verifiers (PAVs) to predict progress under such provers, and show that compared to ORMs, test-time search against PAVs is $>8\%$ more accurate, and $1.5-5\times$ more compute-efficient. Online RL with dense rewards from PAVs enables one of the first results with $5-6\times$ gain in sample efficiency, and $>6\%$ gain in accuracy, over ORMs.

orbitize! is a package for Bayesian modeling of the orbital parameters of resolved binary objects from time series measurements. It was developed with the needs of the high-contrast imaging community in mind, and has since also become widely used in the binary star community. A generic orbitize! use case involves translating relative astrometric time series, optionally combined with radial velocity or astrometric time series, into a set of derived orbital posteriors. This paper is published alongside the release of orbitize! version 3.0, which has seen significant enhancements in functionality and accessibility since the release of version 1.0 (Blunt et al., 2020)., Comment: Published in JOSS

Value at Risk (VaR) and stress testing are two of the most widely used approaches in portfolio risk management to estimate potential market value losses under adverse market moves. VaR quantifies potential loss in value over a specified horizon (such as one day or ten days) at a desired confidence level (such as 95'th percentile). In scenario design and stress testing, the goal is to construct extreme market scenarios such as those involving severe recession or a specific event of concern (such as a rapid increase in rates or a geopolitical event), and quantify potential impact of such scenarios on the portfolio. The goal of this paper is to propose an approach for incorporating prevailing market conditions in stress scenario design and estimation of VaR so that they provide more accurate and realistic insights about portfolio risk over the near term. The proposed approach is based on historical data where historical observations of market changes are given more weight if a certain period in history is "more similar" to the prevailing market conditions. Clusters of market conditions are identified using a Machine Learning approach called Variational Inference (VI) where for each cluster future changes in portfolio value are similar. VI based algorithm uses optimization techniques to obtain analytical approximations of the posterior probability density of cluster assignments (market regimes) and probabilities of different outcomes for changes in portfolio value. Covid related volatile period around the year 2020 is used to illustrate the performance of the proposed approach and in particular show how VaR and stress scenarios adapt quickly to changing market conditions. Another advantage of the proposed approach is that classification of market conditions into clusters can provide useful insights about portfolio performance under different market conditions., Comment: arXiv admin note: text overlap with arXiv:2205.00605

In this paper, we develop a method for learning a control policy guaranteed to satisfy an affine state constraint of high relative degree in closed loop with a black-box system. Previous reinforcement learning (RL) approaches to satisfy safety constraints either require access to the system model, or assume control affine dynamics, or only discourage violations with reward shaping. Only recently have these issues been addressed with POLICEd RL, which guarantees constraint satisfaction for black-box systems. However, this previous work can only enforce constraints of relative degree 1. To address this gap, we build a novel RL algorithm explicitly designed to enforce an affine state constraint of high relative degree in closed loop with a black-box control system. Our key insight is to make the learned policy be affine around the unsafe set and to use this affine region to dissipate the inertia of the high relative degree constraint. We prove that such policies guarantee constraint satisfaction for deterministic systems while being agnostic to the choice of the RL training algorithm. Our results demonstrate the capacity of our approach to enforce hard constraints in the Gym inverted pendulum and on a space shuttle landing simulation., Comment: CDC 2024

The presence of dark gaps, a preferential light deficit along the bar minor axis, is observationally well known. The properties of dark gaps are thought to be associated with the properties of bars, and their spatial locations are often associated with bar resonances. However, a systematic study, testing the robustness and universality of these assumptions, is still largely missing. Here, we investigate the formation and evolution of bar-induced dark gaps using a suite of N-body models of (kinematically cold) thin and (kinematically hot) thick discs with varying thick disc mass fraction, and different thin-to-thick disc geometry. We find that dark gaps are a natural consequence of the trapping of disc stars by the bar. The properties of dark gaps (such as strength and extent) are well correlated with the properties of bars. For stronger dark gaps, the fractional mass loss along the bar minor axis can reach up to ~60-80 percent of the initial mass contained, which is redistributed within the bar. These trends hold true irrespective of the mass fraction in the thick disc and the assumed disc geometry. In all our models harbouring slow bars, none of the resonances (corotation, Inner Lindblad resonance, and 4:1 ultra-harmonic resonance) associated with the bar correspond to the location of dark gaps, thereby suggesting that the location of dark gaps is not a universal proxy for these bar resonances, in contrast with earlier studies., Comment: 13 pages, 12 figures, 1 table (including appendix), accepted for publication in MNRAS

Molecular testing of tumor samples for targetable biomarkers is restricted by a lack of standardization, turnaround-time, cost, and tissue availability across cancer types. Additionally, targetable alterations of low prevalence may not be tested in routine workflows. Algorithms that predict DNA alterations from routinely generated hematoxylin and eosin (H&E)-stained images could prioritize samples for confirmatory molecular testing. Costs and the necessity of a large number of samples containing mutations limit approaches that train individual algorithms for each alteration. In this work, models were trained for simultaneous prediction of multiple DNA alterations from H&E images using a multi-task approach. Compared to biomarker-specific models, this approach performed better on average, with pronounced gains for rare mutations. The models reasonably generalized to independent temporal-holdout, externally-stained, and multi-site TCGA test sets. Additionally, whole slide image embeddings derived using multi-task models demonstrated strong performance in downstream tasks that were not a part of training. Overall, this is a promising approach to develop clinically useful algorithms that provide multiple actionable predictions from a single slide.

A common, yet regular, decision made by people, whether healthy or with any health condition, is to decide what to have in meals like breakfast, lunch, and dinner, consisting of a combination of foods for appetizer, main course, side dishes, desserts, and beverages. However, often this decision is seen as a trade-off between nutritious choices (e.g., low salt and sugar) or convenience (e.g., inexpensive, fast to prepare/obtain, taste better). In this preliminary work, we present a data-driven approach for the novel meal recommendation problem that can explore and balance choices for both considerations while also reasoning about a food's constituents and cooking process. Beyond the problem formulation, our contributions also include a goodness measure, a recipe conversion method from text to the recently introduced multimodal rich recipe representation (R3) format, and learning methods using contextual bandits that show promising results., Comment: 6 pages (including references), 1 figure, 2 tables

Recent advancements in nanotechnology have led to the development of surface-enhanced Raman spectroscopy (SERS) based rapid and low-cost technologies for ultra-sensitive label-free detection and identification of molecular analytes. Herein, we utilized the synergistic plasmonic and chemical enhancement effects of Au-WS2 nanohybrids to attain the high-intensity Raman signals of targeted analytes. To develop these nanohybrids, a series of monodispersed Au nanoparticles (NPs) of varying diameters from 20 to 80 nm was chemically synthesized and successively blended with liquid-phase exfoliated WS2 nano-flakes of average lateral size 90 nm. They provided a maximum enhancement factor (EF) of ~1.80 109 corresponding to the characteristic peaks at 1364 cm-1 and 1512 cm-1 for R6G analyte molecules. Theoretical studies based on the finite-difference time-domain simulations on Au-WS2 nanohybrid systems revealed a huge field-intensity enhancement with an EF of more than 1000 at the plasmonic hotspots, which was induced by the strong coupling of individual plasmon oscillations of the adjacent Au NPs upon light interactions. These electromagnetic effects along with the chemical enhancement effects of WS2 nanoflakes were found to be mainly responsible for such huge enhancement in Raman signals. Furthermore, these hybrids were successfully employed for achieving highly sensitive detection of the E. coli ATCC 35218 bacterial strain with a concentration of 104 CFU/mL in phosphate-buffered saline media, indicating their real capabilities for practical scenarios. The findings of the present study will indeed provide vital information in the development of innovative nanomaterial-based biosensors, that will offer new possibilities for addressing critical public health concerns.

Language model (LM) post-training (or alignment) involves maximizing a reward function that is derived from preference annotations. Direct Preference Optimization (DPO) is a popular offline alignment method that trains a policy directly on preference data without the need to train a reward model or apply reinforcement learning. However, typical preference datasets have only a single, or at most a few, annotation per preference pair, which causes DPO to overconfidently assign rewards that trend towards infinite magnitude. This frequently leads to degenerate policies, sometimes causing even the probabilities of the preferred generations to go to zero. In this work, we analyze this phenomenon and propose distillation to get a better proxy for the true preference distribution over generation pairs: we train the LM to produce probabilities that match the distribution induced by a reward model trained on the preference data. Moreover, to account for uncertainty in the reward model we are distilling from, we optimize against a family of reward models that, as a whole, is likely to include at least one reasonable proxy for the preference distribution. Our results show that distilling from such a family of reward models leads to improved robustness to distribution shift in preference annotations, while preserving the simple supervised nature of DPO.

Forecast verification is critical for continuous improvement in meteorological organizations. The Jive verification system was originally developed to assess the accuracy of public weather forecasts issued by the Australian Bureau of Meteorology. It started as a research project in 2015 and gradually evolved to be a Bureau operational verification system in 2022. The system includes daily verification dashboards for forecasters to visualize recent forecast performance and "Evidence Targeted Automation" dashboards for exploring the performance of competing forecast systems. Additionally, Jive includes a Jupyter Notebook server with the Jive Python library which supports research experiments, case studies, and the development of new verification metrics and tools. This paper describes the Jive verification system and how it helped bring verification to the forefront at the Bureau of Meteorology, leading to more accurate, streamlined forecasts. Jive has provided evidence to support forecast automation decisions and has helped to understand the evolving role of meteorologists in the forecast process. It has given operational meteorologists tools for evaluating forecast processes, including identifying when and how manual interventions lead to superior predictions. Work on Jive led to new verification science, including novel metrics that are decision-focused, including diagnostics for extreme conditions. Jive also provided the Bureau with an enterprise-wide data analysis environment and has prompted a clarification of forecast definitions. These collective impacts have resulted in more accurate forecasts, ultimately benefiting society, and building trust with forecast users. These positive outcomes highlight the importance of meteorological organizations investing in verification science and technology.

Robot swarms can effectively serve a variety of sensing and inspection applications. Certain inspection tasks require a binary classification decision. This work presents an experimental setup for a surface inspection task based on vibration sensing and studies a Bayesian two-outcome decision-making algorithm in a swarm of miniaturized wheeled robots. The robots are tasked with individually inspecting and collectively classifying a 1mx1m tiled surface consisting of vibrating and non-vibrating tiles based on the majority type of tiles. The robots sense vibrations using onboard IMUs and perform collision avoidance using a set of IR sensors. We develop a simulation and optimization framework leveraging the Webots robotic simulator and a Particle Swarm Optimization (PSO) method. We consider two existing information sharing strategies and propose a new one that allows the swarm to rapidly reach accurate classification decisions. We first find optimal parameters that allow efficient sampling in simulation and then evaluate our proposed strategy against the two existing ones using 100 randomized simulation and 10 real experiments. We find that our proposed method compels the swarm to make decisions at an accelerated rate, with an improvement of up to 20.52% in mean decision time at only 0.78% loss in accuracy.

Retrieval-Augmented Generation (RAG) is essential for integrating external knowledge into Large Language Model (LLM) outputs. While the literature on RAG is growing, it primarily focuses on systematic reviews and comparisons of new state-of-the-art (SoTA) techniques against their predecessors, with a gap in extensive experimental comparisons. This study begins to address this gap by assessing various RAG methods' impacts on retrieval precision and answer similarity. We found that Hypothetical Document Embedding (HyDE) and LLM reranking significantly enhance retrieval precision. However, Maximal Marginal Relevance (MMR) and Cohere rerank did not exhibit notable advantages over a baseline Naive RAG system, and Multi-query approaches underperformed. Sentence Window Retrieval emerged as the most effective for retrieval precision, despite its variable performance on answer similarity. The study confirms the potential of the Document Summary Index as a competent retrieval approach. All resources related to this research are publicly accessible for further investigation through our GitHub repository ARAGOG (https://github.com/predlico/ARAGOG). We welcome the community to further this exploratory study in RAG systems., Comment: 14 pages, 8 figures, associated Github repo: https://github.com/predlico/ARAGOG

Understanding how robots plan and execute tasks is crucial in today's world, where they are becoming more prevalent in our daily lives. However, teaching non-experts the complexities of robot planning can be challenging. This work presents an open-source platform that simplifies the process using a visual interface that completely abstracts the complex internals of hierarchical planning that robots use for performing task and motion planning. Using the principles developed in the field of explainable AI, this intuitive platform enables users to create plans for robots to complete tasks, and provides helpful hints and natural language explanations for errors. The platform also has a built-in simulator to demonstrate how robots execute submitted plans. This platform's efficacy was tested in a user study on university students with little to no computer science background. Our results show that this platform is highly effective in teaching novice users the intuitions of robot task planning.

Networks of coupled nonlinear oscillators have been used to model circadian rhythms, flashing fireflies, Josephson junction arrays, high-voltage electric grids, and many other kinds of self-organizing systems. Recently, several authors have sought to understand how coupled oscillators behave when they interact according to a random graph. Here we consider interaction networks generated by a graphon model known as a $W$-random network, and examine the dynamics of an infinite number of identical phase oscillators, following an approach pioneered by Medvedev. We show that with sufficient regularity on $W$, the solution to the dynamical system over a $W$-random network of size $n$ converges in the $L^{\infty}$ norm to the solution of the continuous graphon system, with high probability as $n\rightarrow\infty$. This result suggests a framework for studying synchronization properties in large but finite random networks. In this paper, we leverage our convergence result in the $L^{\infty}$ norm to prove synchronization results for two classes of identical phase oscillators on Erd\H{o}s-R\'enyi random graphs. First, we show that the Kuramoto model on the Erd\H{o}s-R\'enyi graph $G(n, \alpha_n)$ achieves phase synchronization with high probability as $n$ goes to infinity, if the edge probability $\alpha_n$ exceeds $(\log n)/n$, the connectivity threshold of an Erd\H{o}s-R\'enyi random graph. Then we show that the Sakaguchi-Kuramoto model on the Erd\H{o}s-R\'enyi graph $G(n, p)$ achieves frequency synchronization with high probability as $n$ goes to infinity, assuming a fixed edge probability $p\in(0,1]$ and a certain regime for the model's phase shift parameter. A notable feature of the latter result is that it holds for an oscillator model whose dynamics are not simply given by a gradient flow., Comment: 29 pages, 3 figures

In this paper, we seek to learn a robot policy guaranteed to satisfy state constraints. To encourage constraint satisfaction, existing RL algorithms typically rely on Constrained Markov Decision Processes and discourage constraint violations through reward shaping. However, such soft constraints cannot offer verifiable safety guarantees. To address this gap, we propose POLICEd RL, a novel RL algorithm explicitly designed to enforce affine hard constraints in closed-loop with a black-box environment. Our key insight is to force the learned policy to be affine around the unsafe set and use this affine region as a repulsive buffer to prevent trajectories from violating the constraint. We prove that such policies exist and guarantee constraint satisfaction. Our proposed framework is applicable to both systems with continuous and discrete state and action spaces and is agnostic to the choice of the RL training algorithm. Our results demonstrate the capacity of POLICEd RL to enforce hard constraints in robotic tasks while significantly outperforming existing methods., Comment: Robotics: Science and Systems (RSS) 2024, https://www.roboticsproceedings.org/rss20/p104.html

Large language models (LLMs) hold promise to serve complex health information needs but also have the potential to introduce harm and exacerbate health disparities. Reliably evaluating equity-related model failures is a critical step toward developing systems that promote health equity. We present resources and methodologies for surfacing biases with potential to precipitate equity-related harms in long-form, LLM-generated answers to medical questions and conduct a large-scale empirical case study with the Med-PaLM 2 LLM. Our contributions include a multifactorial framework for human assessment of LLM-generated answers for biases, and EquityMedQA, a collection of seven datasets enriched for adversarial queries. Both our human assessment framework and dataset design process are grounded in an iterative participatory approach and review of Med-PaLM 2 answers. Through our empirical study, we find that our approach surfaces biases that may be missed via narrower evaluation approaches. Our experience underscores the importance of using diverse assessment methodologies and involving raters of varying backgrounds and expertise. While our approach is not sufficient to holistically assess whether the deployment of an AI system promotes equitable health outcomes, we hope that it can be leveraged and built upon towards a shared goal of LLMs that promote accessible and equitable healthcare.

With growing application of machine learning (ML) technologies in healthcare, there have been calls for developing techniques to understand and mitigate biases these systems may exhibit. Fair-ness considerations in the development of ML-based solutions for health have particular implications for Africa, which already faces inequitable power imbalances between the Global North and South.This paper seeks to explore fairness for global health, with Africa as a case study. We conduct a scoping review to propose axes of disparities for fairness consideration in the African context and delineate where they may come into play in different ML-enabled medical modalities. We then conduct qualitative research studies with 672 general population study participants and 28 experts inML, health, and policy focused on Africa to obtain corroborative evidence on the proposed axes of disparities. Our analysis focuses on colonialism as the attribute of interest and examines the interplay between artificial intelligence (AI), health, and colonialism. Among the pre-identified attributes, we found that colonial history, country of origin, and national income level were specific axes of disparities that participants believed would cause an AI system to be biased.However, there was also divergence of opinion between experts and general population participants. Whereas experts generally expressed a shared view about the relevance of colonial history for the development and implementation of AI technologies in Africa, the majority of the general population participants surveyed did not think there was a direct link between AI and colonialism. Based on these findings, we provide practical recommendations for developing fairness-aware ML solutions for health in Africa., Comment: 11 pages, 4 figures. arXiv admin note: text overlap with arXiv:2304.02190

Orbital evolution is a critical process that sculpts planetary systems, particularly during their early stages where planet-disk interactions are expected to lead to the formation of resonant chains. Despite the theoretically expected prominence of such configurations, they are scarcely observed among long-period giant exoplanets. This disparity suggests an evolutionary sequence wherein giant planet systems originate in compact multiresonant configurations, but subsequently become unstable, eventually relaxing to wider orbits--a phenomenon mirrored in our own solar system's early history. In this work, we present a suite of N-body simulations that model the instability-driven evolution of giant planet systems, originating from resonant initial conditions, through phases of disk dispersal and beyond. By comparing the period ratio and normalized angular momentum distributions of our synthetic aggregate of systems with the observational census of long-period Jovian planets, we derive constraints on the expected rate of orbital migration, the efficiency of gas-driven eccentricity damping, and typical initial multiplicity. Our findings reveal a distinct inclination toward densely packed initial conditions, weak damping, and high giant planet multiplicities. Furthermore, our models indicate that resonant chain origins do not facilitate the formation of Hot Jupiters via the coplanar high-eccentricity pathway at rates high enough to explain their observed prevalence., Comment: ApJ published version. Changes to Hot Jupiter production calculation in Section 5; main conclusions remain unchanged

Hand-crafted, logic-based state and action representations have been widely used to overcome the intractable computational complexity of long-horizon robot planning problems, including task and motion planning problems. However, creating such representations requires experts with strong intuitions and detailed knowledge about the robot and the tasks it may need to accomplish in a given setting. Removing this dependency on human intuition is a highly active research area. This paper presents the first approach for autonomously learning generalizable, logic-based relational representations for abstract states and actions starting from unannotated high-dimensional, real-valued robot trajectories. The learned representations constitute auto-invented PDDL-like domain models. Empirical results in deterministic settings show that powerful abstract representations can be learned from just a handful of robot trajectories; the learned relational representations include but go beyond classical, intuitive notions of high-level actions; and that the learned models allow planning algorithms to scale to tasks that were previously beyond the scope of planning without hand-crafted abstractions.

Bias benchmarks are a popular method for studying the negative impacts of bias in LLMs, yet there has been little empirical investigation of whether these benchmarks are actually indicative of how real world harm may manifest in the real world. In this work, we study the correspondence between such decontextualized "trick tests" and evaluations that are more grounded in Realistic Use and Tangible {Effects (i.e. RUTEd evaluations). We explore this correlation in the context of gender-occupation bias--a popular genre of bias evaluation. We compare three de-contextualized evaluations adapted from the current literature to three analogous RUTEd evaluations applied to long-form content generation. We conduct each evaluation for seven instruction-tuned LLMs. For the RUTEd evaluations, we conduct repeated trials of three text generation tasks: children's bedtime stories, user personas, and English language learning exercises. We found no correspondence between trick tests and RUTEd evaluations. Specifically, selecting the least biased model based on the de-contextualized results coincides with selecting the model with the best performance on RUTEd evaluations only as often as random chance. We conclude that evaluations that are not based in realistic use are likely insufficient to mitigate and assess bias and real-world harms.

A common approach for aligning language models to human preferences is to first learn a reward model from preference data, and then use this reward model to update the language model. We study two closely related problems that arise in this approach. First, any monotone transformation of the reward model preserves preference ranking; is there a choice that is ``better'' than others? Second, we often wish to align language models to multiple properties: how should we combine multiple reward models? Using a probabilistic interpretation of the alignment procedure, we identify a natural choice for transformation for (the common case of) rewards learned from Bradley-Terry preference models. The derived transformation is straightforward: we apply a log-sigmoid function to the centered rewards, a method we term ``LSC-transformation'' (log-sigmoid-centered transformation). This transformation has two important properties. First, it emphasizes improving poorly-performing outputs, rather than outputs that already score well. This mitigates both underfitting (where some prompts are not improved) and reward hacking (where the model learns to exploit misspecification of the reward model). Second, it enables principled aggregation of rewards by linking summation to logical conjunction: the sum of transformed rewards corresponds to the probability that the output is ``good'' in all measured properties, in a sense we make precise. Experiments aligning language models to be both helpful and harmless using RLHF show substantial improvements over the baseline (non-transformed) approach.

The number of Hindi speakers on social media has increased dramatically in recent years. Regret is a common emotional experience in our everyday life. Many speakers on social media, share their regretful experiences and opinions regularly. It might cause a re-evaluation of one's choices and a desire to make a different option if given the chance. As a result, knowing the source of regret is critical for investigating its impact on behavior and decision-making. This study focuses on regret and how it is expressed, specifically in Hindi, on various social media platforms. In our study, we present a novel dataset from three different sources, where each sentence has been manually classified into one of three classes "Regret by action", "Regret by inaction", and "No regret". Next, we use this dataset to investigate the linguistic expressions of regret in Hindi text and also identify the textual domains that are most frequently associated with regret. Our findings indicate that individuals on social media platforms frequently express regret for both past inactions and actions, particularly within the domain of interpersonal relationships. We use a pre-trained BERT model to generate word embeddings for the Hindi dataset and also compare deep learning models with conventional machine learning models in order to demonstrate accuracy. Our results show that BERT embedding with CNN consistently surpassed other models. This described the effectiveness of BERT for conveying the context and meaning of words in the regret domain.

A simple and effective method for the alignment of generative models is the best-of-$n$ policy, where $n$ samples are drawn from a base policy, and ranked based on a reward function, and the highest ranking one is selected. A commonly used analytical expression in the literature claims that the KL divergence between the best-of-$n$ policy and the base policy is equal to $\log (n) - (n-1)/n.$ We disprove the validity of this claim, and show that it is an upper bound on the actual KL divergence. We also explore the tightness of this upper bound in different regimes. Finally, we propose a new estimator for the KL divergence and empirically show that it provides a tight approximation through a few examples.

Reward models play a key role in aligning language model applications towards human preferences. However, this setup creates an incentive for the language model to exploit errors in the reward model to achieve high estimated reward, a phenomenon often termed \emph{reward hacking}. A natural mitigation is to train an ensemble of reward models, aggregating over model outputs to obtain a more robust reward estimate. We explore the application of reward ensembles to alignment at both training time (through reinforcement learning) and inference time (through reranking). First, we show that reward models are \emph{underspecified}: reward models that perform similarly in-distribution can yield very different rewards when used in alignment, due to distribution shift. Second, underspecification results in overoptimization, where alignment to one reward model does not improve reward as measured by another reward model trained on the same data. Third, overoptimization is mitigated by the use of reward ensembles, and ensembles that vary by their \emph{pretraining} seeds lead to better generalization than ensembles that differ only by their \emph{fine-tuning} seeds, with both outperforming individual reward models. However, even pretrain reward ensembles do not eliminate reward hacking: we show several qualitative reward hacking phenomena that are not mitigated by ensembling because all reward models in the ensemble exhibit similar error patterns., Comment: Published at the 2024 Conference on Language Modeling (CoLM)

In this study, we will look at an interacting dark energy model. In the framework of Friedmann-Robertson-Walker (FRW) space-time, we have made the hypothesis of an interacting scheme between two fields (dark matter (DM) and dark energy (DE)). The evolution of the dark energy model in the above-mentioned spatially and homogeneous space-time has been studied from the viewpoint of interaction between DM and DE. This interacting scenario has been established by choosing an appropriate ansatz of the scale factor $a(t)$. By computing cosmological parameters (geometrical and physical) such as deceleration parameter $q$, energy density $\rho$, pressure $p$, equation of state parameter $\omega$, and density parameter $\Omega$, we have performed energy conditions, scalar field description, and refined swampland conjecture to support the interaction model that we have developed., Comment: 19 pages, 17 figures

Transiting giant planets provide a natural opportunity to examine stellar obliquities, which offer clues about the origin and dynamical histories of close-in planets. Hot Jupiters orbiting Sun-like stars show a tendency for obliquity alignment, which suggests that obliquities are rarely excited or that tidal realignment is common. However, the stellar obliquity distribution is less clear for giant planets at wider separations where realignment mechanisms are not expected to operate. In this work, we uniformly derive line-of-sight inclinations for 47 cool stars ($T_\mathrm{eff}$ $<$ 6200 K) harboring transiting hot and warm giant planets by combining rotation periods, stellar radii, and $v \sin i$ measurements. Among the systems that show signs of spin-orbit misalignment in our sample, three are identified as being misaligned here for the first time. Of particular interest are Kepler-1654, one of the longest-period (1047 d; 2.0 AU) giant planets in a misaligned system, and Kepler-30, a multi-planet misaligned system. By comparing the reconstructed underlying inclination distributions, we find that the inferred minimum misalignment distributions of hot Jupiters spanning $a/R_{*}$ = 3-20 ($\approx$ 0.01-0.1 AU) and warm Jupiters spanning $a/R_{*}$ = 20-400 ($\approx$ 0.1-1.9 AU) are in good agreement. With 90$\%$ confidence, at least 24$^{+9}_{-7}\%$ of warm Jupiters and 14$^{+7}_{-5}\%$ of hot Jupiters around cool stars are misaligned by at least 10$^\circ$. Most stars harboring warm Jupiters are therefore consistent with spin-orbit alignment. The similarity of hot and warm Jupiter misalignment rates suggests that either the occasional misalignments are primordial and originate in misaligned disks, or the same underlying processes that create misaligned hot Jupiters also lead to misaligned warm Jupiters., Comment: AJ, accepted

The optimal robot assembly planning problem is challenging due to the necessity of finding the optimal solution amongst an exponentially vast number of possible plans, all while satisfying a selection of constraints. Traditionally, robotic assembly planning problems have been solved using heuristics, but these methods are specific to a given objective structure or set of problem parameters. In this paper, we propose a novel approach to robotic assembly planning that poses assembly sequencing as a sequential decision making problem, enabling us to harness methods that far outperform the state-of-the-art. We formulate the problem as a Markov Decision Process (MDP) and utilize Dynamic Programming (DP) to find optimal assembly policies for moderately sized strictures. We further expand our framework to exploit the deterministic nature of assembly planning and introduce a class of optimal Graph Exploration Assembly Planners (GEAPs). For larger structures, we show how Reinforcement Learning (RL) enables us to learn policies that generate high reward assembly sequences. We evaluate our approach on a variety of robotic assembly problems, such as the assembly of the Hubble Space Telescope, the International Space Station, and the James Webb Space Telescope. We further showcase how our DP, GEAP, and RL implementations are capable of finding optimal solutions under a variety of different objective functions and how our formulation allows us to translate precedence constraints to branch pruning and thus further improve performance. We have published our code at https://github.com/labicon/ORASP-Code., Comment: 6 conference page paper, 3 page appendix, 23 figures