Search

GPT-4o is an autoregressive omni model that accepts as input any combination of text, audio, image, and video, and generates any combination of text, audio, and image outputs. It's trained end-to-end across text, vision, and audio, meaning all inputs and outputs are processed by the same neural network. GPT-4o can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time in conversation. It matches GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages, while also being much faster and 50\% cheaper in the API. GPT-4o is especially better at vision and audio understanding compared to existing models. In line with our commitment to building AI safely and consistent with our voluntary commitments to the White House, we are sharing the GPT-4o System Card, which includes our Preparedness Framework evaluations. In this System Card, we provide a detailed look at GPT-4o's capabilities, limitations, and safety evaluations across multiple categories, focusing on speech-to-speech while also evaluating text and image capabilities, and measures we've implemented to ensure the model is safe and aligned. We also include third-party assessments on dangerous capabilities, as well as discussion of potential societal impacts of GPT-4o's text and vision capabilities.

The UltraViolet imaging of the Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey Fields (UVCANDELS) program provides deep HST F275W and F435W imaging over four CANDELS fields (GOODS-N, GOODS-S, COSMOS, and EGS). We combine this newly acquired UV imaging with existing HST imaging from CANDELS as well as existing ancillary data to obtain robust photometric redshifts and reliable estimates for galaxy physical properties for over 150,000 galaxies in the $\sim$430 arcmin$^2$ UVCANDELS area. Here, we leverage the power of the new UV photometry to not only improve the photometric redshift measurements in these fields, but also constrain the full redshift probability distribution combining multiple redshift fitting tools. Furthermore, using the full UV-to-IR photometric dataset, we measure the galaxy physical properties by fitting templates from population synthesis models with two different parameterizations (flexible and fixed-form) of the star-formation histories (SFHs). Compared to the flexible SFH parametrization, we find that the fixed-form SFHs systematically underestimate the galaxy stellar masses, both at the low- ($\lesssim10^9 M_\odot$) and high- ($\gtrsim10^{10} M_\odot$) mass end, by as much as $\sim0.5$ dex. This underestimation is primarily due the limited ability of fixed-form SFH parameterization to simultaneously capture the chaotic nature of star-formation in these galaxies., Comment: 22 pages, 6 figures; accepted to ApJS; catalogs available via MAST

Accurate recurrence risk stratification is crucial for optimizing treatment plans for breast cancer patients. Current prognostic tools like Oncotype DX (ODX) offer valuable genomic insights for HR+/HER2- patients but are limited by cost and accessibility, particularly in underserved populations. In this study, we present Deep-BCR-Auto, a deep learning-based computational pathology approach that predicts breast cancer recurrence risk from routine H&E-stained whole slide images (WSIs). Our methodology was validated on two independent cohorts: the TCGA-BRCA dataset and an in-house dataset from The Ohio State University (OSU). Deep-BCR-Auto demonstrated robust performance in stratifying patients into low- and high-recurrence risk categories. On the TCGA-BRCA dataset, the model achieved an area under the receiver operating characteristic curve (AUROC) of 0.827, significantly outperforming existing weakly supervised models (p=0.041). In the independent OSU dataset, Deep-BCR-Auto maintained strong generalizability, achieving an AUROC of 0.832, along with 82.0% accuracy, 85.0% specificity, and 67.7% sensitivity. These findings highlight the potential of computational pathology as a cost-effective alternative for recurrence risk assessment, broadening access to personalized treatment strategies. This study underscores the clinical utility of integrating deep learning-based computational pathology into routine pathological assessment for breast cancer prognosis across diverse clinical settings.

This work presents a method of incorporating low-cost capacitive tactile sensors on a soft socially assistive robot platform. By embedding conductive thread into the robot's crocheted exterior, we formed a set of low-cost, flexible capacitive tactile sensors that do not disrupt the robot's soft, zoomorphic embodiment. We evaluated the sensors' performance through a user study (N=20) and found that the sensors reliably detected user touch events and localized touch inputs to one of three regions on the robot's exterior., Comment: Accepted as a Work-In-Progress submission at the 2024 IEEE Haptics Symposium

We introduce a benchmark to directly evaluate the alignment between human observers and vision models on a 3D shape inference task. We leverage an experimental design from the cognitive sciences which requires zero-shot visual inferences about object shape: given a set of images, participants identify which contain the same/different objects, despite considerable viewpoint variation. We draw from a diverse range of images that include common objects (e.g., chairs) as well as abstract shapes (i.e., procedurally generated `nonsense' objects). After constructing over 2000 unique image sets, we administer these tasks to human participants, collecting 35K trials of behavioral data from over 500 participants. This includes explicit choice behaviors as well as intermediate measures, such as reaction time and gaze data. We then evaluate the performance of common vision models (e.g., DINOv2, MAE, CLIP). We find that humans outperform all models by a wide margin. Using a multi-scale evaluation approach, we identify underlying similarities and differences between models and humans: while human-model performance is correlated, humans allocate more time/processing on challenging trials. All images, data, and code can be accessed via our project page., Comment: Project page: https://tzler.github.io/MOCHI/ Code: https://github.com/tzler/mochi_code Huggingface dataset: https://huggingface.co/datasets/tzler/MOCHI

Hybrid double-network hydrogels are a class of material that comprise transiently and permanently crosslinked polymer networks and exhibit an enhanced toughness that is believed to be governed by the yielding of the transient polymer network. The precise role of the two polymer networks in this yielding transition and their interplay remains an open question that we address here through constructing a series of hydrogel designs in which the interaction within and between the two polymer networks are systematically inhibited or enhanced. We characterise each of the hydrogel designs using large amplitude oscillatory shear rheology (LAOS). Inspecting yielding through elastic stress across hydrogel designs, we elucidate that the hybrid double-network hydrogel exhibits a two-step yielding behaviour that originates from to the presence of transient crosslinks. Examining the rheological response within each oscillatory cycle and across the hydrogel designs, we show that the micro-structural changes in the transient network are crucial in the second stage of this yielding. We surmise that the first step of yielding is determined by the intermolecular interactions between the two polymer networks by systematically altering the strength of the interactions. These interactions also influence the second step of yielding, which we show is governed by the transient intermolecular interactions within the polymer networks. Our study therefore reveals that the interactions between the polymer networks are as crucial as within the polymer networks and therefore provides insights into how the yielding mechanisms in soft composite materials can be identified, adjusted, and controlled.

On-shell methods have found a new application to local observables such as asymptotic radiation fields and gravitational waveforms. While these observables are invariant under small gauge transformations, they are known to depend on a choice of asymptotic gauge; in gravity on asymptotically Minkowski spacetimes, this is a choice of BMS frame. In this letter, we provide a method for capturing these supertranslations, to all orders in perturbation theory, using the on-shell framework of scattering amplitudes., Comment: 9 pages

In the field of operational modal analysis (OMA), obtained modal information is frequently used to assess the current state of aerospace, mechanical, offshore and civil structures. However, the stochasticity of operational systems and the lack of forcing information can lead to inconsistent results. Quantifying the uncertainty of the recovered modal parameters through OMA is therefore of significant value. In this article, a new perspective on Bayesian OMA is proposed: a Bayesian stochastic subspace identification (SSI) algorithm. Distinct from existing approaches to Bayesian OMA, a hierarchical probabilistic model is embedded at the core of covariance-driven SSI. Through substitution of canonical correlation analysis with a Bayesian equivalent, posterior distributions over the modal properties are obtained. Two inference schemes are presented for the proposed Bayesian formulation: Markov Chain Monte Carlo and variational Bayes. Two case studies are then explored. The first is benchmark study using data from a simulated, multi degree-of-freedom, linear system. Following application of Bayesian SSI, it is shown that the same posterior is targeted and recovered by both inference schemes, with good agreement between the posterior mean and the conventional SSI result. The second study applies the variational form to data obtained from an in-service structure: The Z24 bridge. The results of this study are presented at single model orders, and then using a stabilisation diagram. The recovered posterior uncertainty is presented and compared to the classic SSI result. It is observed that the posterior distributions with mean values coinciding with the natural frequencies exhibit lower variance than values situated away from the natural frequencies.

Understanding and respecting personal space preferences is essential for socially assistive robots designed for older adult users. This work introduces and evaluates a novel personalized context-aware method for modeling users' proxemics preferences during human-robot interactions. Using an interactive augmented reality interface, we collected a set of user-preferred distances from the robot and employed an active transfer learning approach to fine-tune a specialized deep learning model. We evaluated this approach through two user studies: 1) a convenience population study (N = 24) to validate the efficacy of the active transfer learning approach; and 2) a user study involving older adults (N = 15) to assess the system's usability. We compared the data collected with the augmented reality interface and with the physical robot to examine the relationship between proxemics preferences for a virtual robot versus a physically embodied robot. We found that fine-tuning significantly improved model performance: on average, the error in testing decreased by 26.97% after fine-tuning. The system was well-received by older adult participants, who provided valuable feedback and suggestions for future work., Comment: M. Nigro, A. O'Connell, T. Groechel, A.M. Velentza and M. Matari\'c, "An Interactive Augmented Reality Interface for Personalized Proxemics Modeling: Comfort and Human-Robot Interactions," in IEEE Robotics & Automation Magazine, doi: 10.1109/MRA.2024.3415108

Fluorescence-guided surgery has emerged as a vital tool for tumour resection procedures. As well as intraoperative tumour visualisation, 5-ALA-induced PpIX provides an avenue for quantitative tumour identification based on ratiometric fluorescence measurement. To this end, fluorescence imaging and fibre-based probes have enabled more precise demarcation between the cancerous and healthy tissues. These sensing approaches, which rely on collecting the fluorescence light from the tumour resection site and its remote spectral sensing, introduce challenges associated with optical losses. In this work, we demonstrate the viability of tumour detection at the resection site using a miniature fluorescence measurement system. Unlike the current bulky systems, which necessitate remote measurement, we have adopted a millimetre-sized spectral sensor chip for quantitative fluorescence measurements. A reliable measurement at the resection site requires a stable optical window between the tissue and the optoelectronic system. This is achieved using an antifouling diamond window, which provides stable optical transparency. The system achieved a sensitivity of 92.3% and specificity of 98.3% in detecting a surrogate tumour at a resolution of 1 x 1 mm2. As well as addressing losses associated with collecting and coupling fluorescence light in the current remote sensing approaches, the small size of the system introduced in this work paves the way for its direct integration with the tumour resection tools with the aim of more accurate interoperative tumour identification.

We use the Ultraviolet Imaging of the Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey fields (UVCANDELS) to measure half-light radii in the rest-frame far-UV for $\sim$16,000 disk-like galaxies over $0.5\leq z \leq 3$. We compare these results to rest-frame optical sizes that we measure in a self-consistent way and find that the stellar mass-size relation of disk galaxies is steeper in the rest-frame UV than in the optical across our entire redshift range. We show that this is mainly driven by massive galaxies ($\gtrsim10^{10}$M$_\odot$), which we find to also be among the most dusty. Our results are consistent with the literature and have commonly been interpreted as evidence of inside-out growth wherein galaxies form their central structures first. However, they could also suggest that the centers of massive galaxies are more heavily attenuated than their outskirts. We distinguish between these scenarios by modeling and selecting galaxies at $z=2$ from the VELA simulation suite in a way that is consistent with UVCANDELS. We show that the effects of dust alone can account for the size differences we measure at $z=2$. This indicates that, at different wavelengths, size differences and the different slopes of the stellar mass-size relation do not constitute evidence for inside-out growth., Comment: Accepted for publication in ApJ. 22 pages, 12 figures, and 4 tables

In this paper we study topology-changing spacetimes occurring from pointlike sources. Following an old idea of Penrose, we will opt for a non-Hausdorff model of topology change in which an initial pointlike source is ``doubled" and allowed to propagate along null rays into an eventual cobordism. By appealing to recent developments in non-Hausdorff differential geometry, we will describe and evaluate gravitational actions on these topology-changing spacetimes. Motivated by analogous results for the Trousers space, we describe a sign convention for Lorentzian angles that will ensure the dampening of our non-Hausdorff topology-changing spacetimes within a two-dimensional path integral for gravity., Comment: 43 pages, 9 figures. Ver 2: fixed typos and elaborated on certain points

Math Corps is an evidence-based tutoring program serving students in grades 4 through 8 that increases opportunities to learn and practice fundamental math skills related to understanding whole and rational numbers. Its intent is that all students, in particular students who are Black, Latino, and/or experiencing poverty, will reach grade-level proficiency in math. AmeriCorps tutors are matched with a school and provide support to approximately 24 students, in pairs or groups of three, for 90 minutes each week in 30- to 45-minute, in-person sessions. The success of the program is built on the collective value of three core features: (1) tutors who share backgrounds or lived experiences with the students they serve; (2) personalized tutoring using evidence-based curriculum; and (3) tutor training and coaching. This study aims to provide evidence of the impact of the Math Corps tutoring program on student math achievement through a regression-adjusted matched comparison design, as well as descriptive evidence about program attendance and participants' math confidence, sense of belonging in Math Corps, and student-tutor relationship quality and focus group data. This report is one in a series of six reports on math tutoring programs. The goal of this report series is to inform the tutoring field more broadly and support the provision of high-quality tutoring to as many students in the priority communities as possible. [This report was prepared with Math Corps.]

Predicting batting averages for specific batters against specific pitchers is a challenging problem in baseball. Previous methods for estimating batting averages in these matchups have used regression models that can incorporate the pitcher's and batter's individual batting averages. However, these methods are limited in their flexibility to include many additional parameters because of the challenges of high-dimensional data in regression. Dimension reduction methods can be used to incorporate many predictors into the model by finding a lower rank set of patterns among them, providing added flexibility. This paper illustrates that dimension reduction methods can be useful for predicting batting averages. To incorporate binomial data (batting averages) as well as additional data about each batter and pitcher, this paper proposes a novel dimension reduction method that uses alternating generalized linear models to estimate shared patterns across three data sources related to batting averages. While the novel method slightly outperforms existing methods for imputing batting averages based on simulations and a cross-validation study, the biggest advantage is that it can easily incorporate other sources of data. As data-collection technology continues to improve, more variables will be available, and this method will be more accurate with more informative data in the future., Comment: 19 pages, 4 figures

Large language models (LLMs) have immense potential to make information more accessible, particularly in medicine, where complex medical jargon can hinder patient comprehension of clinical notes. We developed a patient-facing tool using LLMs to make clinical notes more readable by simplifying, extracting information from, and adding context to the notes. We piloted the tool with clinical notes donated by patients with a history of breast cancer and synthetic notes from a clinician. Participants (N=200, healthy, female-identifying patients) were randomly assigned three clinical notes in our tool with varying levels of augmentations and answered quantitative and qualitative questions evaluating their understanding of follow-up actions. Augmentations significantly increased their quantitative understanding scores. In-depth interviews were conducted with participants (N=7, patients with a history of breast cancer), revealing both positive sentiments about the augmentations and concerns about AI. We also performed a qualitative clinician-driven analysis of the model's error modes.

College students with ADHD respond positively to simple socially assistive robots (SARs) that monitor attention and provide non-verbal feedback, but studies have been done only in brief in-lab sessions. We present an initial design and evaluation of an in-dorm SAR study companion for college students with ADHD. This work represents the introductory stages of an ongoing user-centered, participatory design process. In a three-week within-subjects user study, university students (N=11) with self-reported symptoms of adult ADHD had a SAR study companion in their dorm room for two weeks and a computer-based system for one week. Toward developing SARs for long-term, in-dorm use, we focus on 1) evaluating the usability and desire for SAR study companions by college students with ADHD and 2) collecting participant feedback about the SAR design and functionality. Participants responded positively to the robot; after one week of regular use, 91% (10 of 11) chose to continue using the robot voluntarily in the second week.

Mindfulness-based therapies have been shown to be effective in improving mental health, and technology-based methods have the potential to expand the accessibility of these therapies. To enable real-time personalized content generation for mindfulness practice in these methods, high-quality computer-synthesized text-to-speech (TTS) voices are needed to provide verbal guidance and respond to user performance and preferences. However, the user-perceived quality of state-of-the-art TTS voices has not yet been evaluated for administering mindfulness meditation, which requires emotional expressiveness. In addition, work has not yet been done to study the effect of physical embodiment and personalization on the user-perceived quality of TTS voices for mindfulness. To that end, we designed a two-phase human subject study. In Phase 1, an online Mechanical Turk between-subject study (N=471) evaluated 3 (feminine, masculine, child-like) state-of-the-art TTS voices with 2 (feminine, masculine) human therapists' voices in 3 different physical embodiment settings (no agent, conversational agent, socially assistive robot) with remote participants. Building on findings from Phase 1, in Phase 2, an in-person within-subject study (N=94), we used a novel framework we developed for personalizing TTS voices based on user preferences, and evaluated user-perceived quality compared to best-rated non-personalized voices from Phase 1. We found that the best-rated human voice was perceived better than all TTS voices; the emotional expressiveness and naturalness of TTS voices were poorly rated, while users were satisfied with the clarity of TTS voices. Surprisingly, by allowing users to fine-tune TTS voice features, the user-personalized TTS voices could perform almost as well as human voices, suggesting user personalization could be a simple and very effective tool to improve user-perceived quality of TTS voice., Comment: Published in Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction, pp. 516-524. 2023

As artificial intelligence (AI) is playing an increasingly important role in our society and global economy, AI education and literacy have become necessary components in college and K-12 education to prepare students for an AI-powered society. However, current AI curricula have not yet been made accessible and engaging enough for students and schools from all socio-economic backgrounds with different educational goals. In this work, we developed an open-source learning module for college and high school students, which allows students to build their own robot companion from the ground up. This open platform can be used to provide hands-on experience and introductory knowledge about various aspects of AI, including robotics, machine learning (ML), software engineering, and mechanical engineering. Because of the social and personal nature of a socially assistive robot companion, this module also puts a special emphasis on human-centered AI, enabling students to develop a better understanding of human-AI interaction and AI ethics through hands-on learning activities. With open-source documentation, assembling manuals and affordable materials, students from different socio-economic backgrounds can personalize their learning experience based on their individual educational goals. To evaluate the student-perceived quality of our module, we conducted a usability testing workshop with 15 college students recruited from a minority-serving institution. Our results indicate that our AI module is effective, easy-to-follow, and engaging, and it increases student interest in studying AI/ML and robotics in the future. We hope that this work will contribute toward accessible and engaging AI education in human-AI interaction for college and high school students., Comment: Accepted to the Proceedings of the AAAI Conference on Artificial Intelligence (2024)

We study the classical dynamics of spinning particles using scattering amplitudes and eikonal exponentiation. We show that observables are determined by a simple algorithm. A wealth of complexity arises in perturbation theory as positions, momenta and spins must be iteratively corrected at each order. Even though we restrict ourselves to one-loop computations at quadratic order in spin, nevertheless we encounter and resolve a number of subtle effects. Finally, we clarify the links between our work and various other eikonal approaches to spinning observables., Comment: 27 pages plus references, v2 matches published version