Search

AI systems crucially rely on human ratings, but these ratings are often aggregated, obscuring the inherent diversity of perspectives in real-world phenomenon. This is particularly concerning when evaluating the safety of generative AI, where perceptions and associated harms can vary significantly across socio-cultural contexts. While recent research has studied the impact of demographic differences on annotating text, there is limited understanding of how these subjective variations affect multimodal safety in generative AI. To address this, we conduct a large-scale study employing highly-parallel safety ratings of about 1000 text-to-image (T2I) generations from a demographically diverse rater pool of 630 raters balanced across 30 intersectional groups across age, gender, and ethnicity. Our study shows that (1) there are significant differences across demographic groups (including intersectional groups) on how severe they assess the harm to be, and that these differences vary across different types of safety violations, (2) the diverse rater pool captures annotation patterns that are substantially different from expert raters trained on specific set of safety policies, and (3) the differences we observe in T2I safety are distinct from previously documented group level differences in text-based safety tasks. To further understand these varying perspectives, we conduct a qualitative analysis of the open-ended explanations provided by raters. This analysis reveals core differences into the reasons why different groups perceive harms in T2I generations. Our findings underscore the critical need for incorporating diverse perspectives into safety evaluation of generative AI ensuring these systems are truly inclusive and reflect the values of all users., Comment: 20 pages, 7 figures

It is imperative that robots can understand natural language commands issued by humans. Such commands typically contain verbs that signify what action should be performed on a given object and that are applicable to many objects. We propose a method for generalizing manipulation skills to novel objects using verbs. Our method learns a probabilistic classifier that determines whether a given object trajectory can be described by a specific verb. We show that this classifier accurately generalizes to novel object categories with an average accuracy of 76.69% across 13 object categories and 14 verbs. We then perform policy search over the object kinematics to find an object trajectory that maximizes classifier prediction for a given verb. Our method allows a robot to generate a trajectory for a novel object based on a verb, which can then be used as input to a motion planner. We show that our model can generate trajectories that are usable for executing five verb commands applied to novel instances of two different object categories on a real robot., Comment: 7 pages + 2 pages (references), 6 figures. Accepted at IROS 2023. Code, dataset info and demo videos can be found at: https://rachelma80000.github.io/SkillGenVerbs/

Continuing advances in Large Language Models (LLMs) in artificial intelligence offer important capacities in intuitively accessing and using medical knowledge in many contexts, including education and training as well as assessment and treatment. Most of the initial literature on LLMs in medicine has emphasized that LLMs are unsuitable for medical use because they are non-deterministic, may provide incorrect or harmful responses, and cannot be regulated to assure quality control. If these issues could be corrected, optimizing LLM technology could benefit patients and physicians by providing affordable, point-of-care medical knowledge. Our proposed framework refines LLM responses by restricting their primary knowledge base to domain-specific datasets containing validated medical information. Additionally, we introduce an actor-critic LLM prompting protocol based on active inference principles of human cognition, where a Therapist agent initially responds to patient queries, and a Supervisor agent evaluates and adjusts responses to ensure accuracy and reliability. We conducted a validation study where expert cognitive behaviour therapy for insomnia (CBT-I) therapists evaluated responses from the LLM in a blind format. Experienced human CBT-I therapists assessed responses to 100 patient queries, comparing LLM-generated responses with appropriate and inappropriate responses crafted by experienced CBT-I therapists. Results showed that LLM responses received high ratings from the CBT-I therapists, often exceeding those of therapist-generated appropriate responses. This structured approach aims to integrate advanced LLM technology into medical applications, meeting regulatory requirements for establishing the safe and effective use of special purpose validated LLMs in medicine., Comment: 25 pages, 4 figures

There exists an invisible barrier between healthcare professionals' perception of a patient's clinical experience and the reality. This barrier may be induced by the environment that hinders patients from sharing their experiences openly with healthcare professionals. As patients are observed to discuss and exchange knowledge more candidly on social media, valuable insights can be leveraged from these platforms. However, the abundance of non-patient posts on social media necessitates filtering out such irrelevant content to distinguish the genuine voices of patients, a task we refer to as patient voice classification. In this study, we analyse the importance of linguistic characteristics in accurately classifying patient voices. Our findings underscore the essential role of linguistic and statistical text similarity analysis in identifying common patterns among patient groups. These results allude to even starker differences in the way patients express themselves at a disease level and across various therapeutic domains. Additionally, we fine-tuned a pre-trained Language Model on the combined datasets with similar linguistic patterns, resulting in a highly accurate automatic patient voice classification. Being the pioneering study on the topic, our focus on extracting authentic patient experiences from social media stands as a crucial step towards advancing healthcare standards and fostering a patient-centric approach., Comment: 14 pages, 4 figures, 5 tables, funded by Talking Medicines Limited

AI Infrastructure plays a key role in the speed and cost-competitiveness of developing and deploying advanced AI models. The current demand for powerful AI infrastructure for model training is driven by the emergence of generative AI and foundational models, where on occasion thousands of GPUs must cooperate on a single training job for the model to be trained in a reasonable time. Delivering efficient and high-performing AI training requires an end-to-end solution that combines hardware, software and holistic telemetry to cater for multiple types of AI workloads. In this report, we describe IBM's hybrid cloud infrastructure that powers our generative AI model development. This infrastructure includes (1) Vela: an AI-optimized supercomputing capability directly integrated into the IBM Cloud, delivering scalable, dynamic, multi-tenant and geographically distributed infrastructure for large-scale model training and other AI workflow steps and (2) Blue Vela: a large-scale, purpose-built, on-premises hosting environment that is optimized to support our largest and most ambitious AI model training tasks. Vela provides IBM with the dual benefit of high performance for internal use along with the flexibility to adapt to an evolving commercial landscape. Blue Vela provides us with the benefits of rapid development of our largest and most ambitious models, as well as future-proofing against the evolving model landscape in the industry. Taken together, they provide IBM with the ability to rapidly innovate in the development of both AI models and commercial offerings., Comment: Corresponding Authors: Talia Gershon, Seetharami Seelam,Brian Belgodere, Milton Bonilla

Fast radio bursts (FRBs) are energetic, millisecond-duration radio pulses observed at extragalactic distances and whose origins are still a subject of heated debate. A fraction of the FRB population have shown repeating bursts, however it's still unclear whether these represent a distinct class of sources. We investigated the bursting behaviour of FRB 20220912A, one of the most active repeating FRBs known thus far. In particular, we focused on its burst energy distribution, linked to the source energetics, and its emission spectrum, with the latter directly related to the underlying emission mechanism. We monitored FRB 20220912A at $408$ MHz with the Northern Cross radio telescope and at $1.4$ GHz using the $32$-m Medicina Grueff radio telescope. Additionally, we conducted $1.2$ GHz observations taken with the upgraded Giant Meter Wave Radio Telescope (uGMRT) searching for a persistent radio source coincident with FRB 20220912A, which included high energy observations in the $0.3-10$ keV, $0.4-100$ MeV and $0.03-30$ GeV energy range. We report $16$ new bursts from FRB 20220912A at $408$ MHz during the period between October 16$^{\rm th}$ 2022 and December 31$^{\rm st}$ 2023. Their cumulative spectral energy distribution follows a power law with slope $\alpha_E = -1.3 \pm 0.2$ and we measured a repetition rate of $0.19 \pm 0.03$ hr$^{-1}$ for bursts having a fluence of $\mathcal{F} \geq 17$ Jy ms. Furthermore, we report no detections at 1.4 GHz for $\mathcal{F} \geq 20$ Jy ms. These non-detections imply an upper limit of $\beta < -2.3$, with $\beta$ being the $408$ MHz $-$ $1.4$ GHz spectral index of FRB 20220912A. This is inconsistent with positive $\beta$ values found for the only two known cases in which an FRB has been detected in separate spectral bands. We find that FRB 20220912A shows a decline of four orders of magnitude in its bursting activity at $1.4$ GHz over a .. (abridged), Comment: accepted for publication, 13 pages, 4 figures

Acceptance and Reappraisal are considered adaptive emotion regulation strategies. While previous studies have explored the neural underpinnings of these strategies using task based fMRI and sMRI, a gap exists in the literature concerning resting-state functional brain networks contributions to these abilities, especially for what concerns Acceptance. Another intriguing question is whether these strategies rely on similar or different neural mechanisms. Building on the well-known improved emotion regulation and increased cognitive flexibility of individuals who rely on acceptance, we expected to find decreased activity inside the Affective network and increased activity inside the Executive and Sensorimotor networks to be predicted of acceptance. We also expect that these networks may be associated at least in part with Reappraisal, indicating a common mechanism behind different strategies. To test these hypotheses, we conducted a functional connectivity analysis of resting-state data from 134 individuals (95 females). To assess acceptance and reappraisal abilities, we used the Cognitive Emotion Regulation Questionnaire (CERQ) and a group-ICA unsupervised machine learning approach to identify resting state networks. Subsequently, we conducted backward regression to predict acceptance and reappraisal abilities. As expected, results indicated that acceptance was predicted by decreased Affective, and increased Executive, and Sensorimotor networks, while reappraisal was predicted by an increase in the Sensorimotor network. Notably, these findings suggest both distinct and overlapping brain contributions to acceptance and reappraisal, with the Sensorimotor network potentially serving as a core common mechanism. These results not only align with previous findings but also expand upon them, demonstrating the complex interplay of cognitive, affective, and sensory abilities in emotion regulation., Comment: 33 pages, 6 figures, 3 tables

Game theory is the study of mathematical models of strategic interactions among rational agents. Language is a key medium of interaction for humans, though it has historically proven difficult to model dialogue and its strategic motivations mathematically. A suitable model of the players, strategies, and payoffs associated with linguistic interactions (i.e., a binding to the conventional symbolic logic of game theory) would enable existing game-theoretic algorithms to provide strategic solutions in the space of language. In other words, a binding could provide a route to computing stable, rational conversational strategies in dialogue. Large language models (LLMs) have arguably reached a point where their generative capabilities can enable realistic, human-like simulations of natural dialogue. By prompting them in various ways, we can steer their responses towards different output utterances. Leveraging the expressivity of natural language, LLMs can also help us quickly generate new dialogue scenarios, which are grounded in real world applications. In this work, we present one possible binding from dialogue to game theory as well as generalizations of existing equilibrium finding algorithms to this setting. In addition, by exploiting LLMs generation capabilities along with our proposed binding, we can synthesize a large repository of formally-defined games in which one can study and test game-theoretic solution concepts. We also demonstrate how one can combine LLM-driven game generation, game-theoretic solvers, and imitation learning to construct a process for improving the strategic capabilities of LLMs., Comment: 32 pages, 8 figures, code available @ https://github.com/google-deepmind/open_spiel/blob/master/open_spiel/python/games/chat_game.py

We improve a recent universality theorem for the Riemann zeta-function in short intervals due to Antanas Laurin\v{c}ikas with respect to the length of these intervals. Moreover, we prove that the shifts can even have exponential growth. This research was initiated by two questions proposed by Laurin\v cikas in a problem session of a recent workshop on universality., Comment: 14 pages, 1 figure

CTA (Cherenkov Telescope Array) is the next generation ground-based observatory for gamma-ray astronomy at very-high energies. The Large-Sized Telescope prototype (LST-1) is located at the Northern site of CTA, on the Canary Island of La Palma. LSTs are designed to provide optimal performance in the lowest part of the energy range covered by CTA, down to $\simeq 20$ GeV. LST-1 started performing astronomical observations in November 2019, during its commissioning phase, and it has been taking data since then. We present the first LST-1 observations of the Crab Nebula, the standard candle of very-high energy gamma-ray astronomy, and use them, together with simulations, to assess the basic performance parameters of the telescope. The data sample consists of around 36 hours of observations at low zenith angles collected between November 2020 and March 2022. LST-1 has reached the expected performance during its commissioning period - only a minor adjustment of the preexisting simulations was needed to match the telescope behavior. The energy threshold at trigger level is estimated to be around 20 GeV, rising to $\simeq 30$ GeV after data analysis. Performance parameters depend strongly on energy, and on the strength of the gamma-ray selection cuts in the analysis: angular resolution ranges from 0.12 to 0.40 degrees, and energy resolution from 15 to 50%. Flux sensitivity is around 1.1% of the Crab Nebula flux above 250 GeV for a 50-h observation (12% for 30 minutes). The spectral energy distribution (in the 0.03 - 30 TeV range) and the light curve obtained for the Crab Nebula agree with previous measurements, considering statistical and systematic uncertainties. A clear periodic signal is also detected from the pulsar at the center of the Nebula., Comment: Accepted in ApJ. v3: updated author list and acknowledgements, fixed typos and other minor issues

It is essential that healthcare professionals and members of the healthcare community can access and easily understand patient experiences in the real world, so that care standards can be improved and driven towards personalised drug treatment. Social media platforms and message boards are deemed suitable sources of patient experience information, as patients have been observed to discuss and exchange knowledge, look for and provide support online. This paper tests the hypothesis that not all online patient experience information can be treated and collected in the same way, as a result of the inherent differences in the way individuals talk about their journeys, in different therapeutic domains and or data sources. We used linguistic analysis to understand and identify similarities between datasets, across patient language, between data sources (Reddit, SocialGist) and therapeutic domains (cardiovascular, oncology, immunology, neurology). We detected common vocabulary used by patients in the same therapeutic domain across data sources, except for immunology patients, who use unique vocabulary between the two data sources, and compared to all other datasets. We combined linguistically similar datasets to train classifiers (CNN, transformer) to accurately identify patient experience posts from social media, a task we refer to as patient voice classification. The cardiovascular and neurology transformer classifiers perform the best in their respective comparisons for the Reddit data source, achieving F1-scores of 0.865 and 1.0 respectively. The overall best performing classifier is the transformer classifier trained on all data collected for this experiment, achieving F1-scores ranging between 0.863 and 0.995 across all therapeutic domain and data source specific test datasets., Comment: 14 pages, 1 figure, 7 tables

Ambulance Diversion (AD) is one of the possible strategies for relieving the worldwide phenomenon of Emergency Department (ED) overcrowding. It can be carried out when an ED is overloaded and consists of redirecting incoming by ambulance patients to neighboring EDs. Properly implemented, AD should result in reducing delays of patient treatment, ensuring safety and rescue of life-threatening patients. From an operational point of view, AD corresponds to a resource pooling policy among EDs in a network. In this paper we propose a novel model for studying the effectiveness of AD strategies, based on the Simulation-Based Optimization (SBO) approach. In particular, we developed a discrete event simulation model for reproducing the ED network operation. Then, for each AD policy considered, we formulate and solve an optimal resources allocation problem consisting of a bi-objective SBO problem where the target is the minimization of the non-value added time spent by patients and the overall cost incurred by the ED network. A set of optimal points belonging to the Pareto frontier is obtained for each policy. To show the reliability of the proposed approach, a real case study consisting of six large EDs in the Lazio region of Italy is considered, analyzing the effects of adopting different AD policies., Comment: 22 pages

This study presents a comparative analysis of three Generative Pre-trained Transformer (GPT) solutions in a question and answer (Q&A) setting: Drug-GPT 3, Drug-GPT 4, and ChatGPT, in the context of healthcare applications. The objective is to determine which model delivers the most accurate and relevant information in response to prompts related to patient experiences with atopic dermatitis (AD) and healthcare professional (HCP) discussions about diabetes. The results demonstrate that while all three models are capable of generating relevant and accurate responses, Drug-GPT 3 and Drug-GPT 4, which are supported by curated datasets of patient and HCP social media and message board posts, provide more targeted and in-depth insights. ChatGPT, a more general-purpose model, generates broader and more general responses, which may be valuable for readers seeking a high-level understanding of the topics but may lack the depth and personal insights found in the answers generated by the specialized Drug-GPT models. This comparative analysis highlights the importance of considering the language model's perspective, depth of knowledge, and currency when evaluating the usefulness of generated information in healthcare applications., Comment: 10 pages, funded by Talking Medicines Limited