Search

Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achieve over 90\% accuracy on popular benchmarks like MMLU, limiting informed measurement of state-of-the-art LLM capabilities. In response, we introduce Humanity's Last Exam (HLE), a multi-modal benchmark at the frontier of human knowledge, designed to be the final closed-ended academic benchmark of its kind with broad subject coverage. HLE consists of 3,000 questions across dozens of subjects, including mathematics, humanities, and the natural sciences. HLE is developed globally by subject-matter experts and consists of multiple-choice and short-answer questions suitable for automated grading. Each question has a known solution that is unambiguous and easily verifiable, but cannot be quickly answered via internet retrieval. State-of-the-art LLMs demonstrate low accuracy and calibration on HLE, highlighting a significant gap between current LLM capabilities and the expert human frontier on closed-ended academic questions. To inform research and policymaking upon a clear understanding of model capabilities, we publicly release HLE at https://lastexam.ai., Comment: 25 pages, 6 figures

Context. In dffuse interstellar clouds the excitation temperature derived from the lowest levels of H$^+_3$ is systematically lower than that derived from H2. The differences may be attributed to the lack of state-specific formation and destruction rates of H$^+_3$ needed to thermalize the two species. Aims. In this work, we want to check the role of rotational excitation collisions of H$^+_3$ with atomic hydrogen on its excitation temperature. Methods. A time independent close-coupling method is used to calculate the state-to-state rate coefficients, using a very accurate and full dimensional potential energy surface recently developed for H$^+_4$. A symmetric top approach is used to describe a frozen H$^+_3$ as equilateral triangle. Results. Rotational excitation collision rate coefficients of H$^+_3$ with atomic Hydrogen have been derived in a temperature range appropriate to diffuse interstellar conditions up to $(J; K; \pm) = (7; 6; +)$ and $(J; K; \pm) = (6; 4; +)$ for its ortho and para forms. This allows to have a consistent set of collisional excitation rate coefficients and to improve the previous study where these contributions were speculated. Conclusions. The new state-specific inelastic H$^+_3$ + H rate coeffcients yield differences up to 20% in the excitation temperature, and their impact increases with decreasing molecular fraction. We also confirm the impact of chemical state-to-state destruction reactions in the excitation balance of H$^+_3$ , and that reactive H + H$^+_3$ collisions are also needed to account for possible further ortho to para transitions, Comment: 9 pages,9 figures

Background: The rate of energy production in the hot-CNO cycle and breakout to the rapid-proton capture process in Type I X-ray bursts is strongly related to the $^{14}$O($\alpha,p$)$^{17}$F reaction rate. The properties of states in $^{18}$Ne near $E_x=6.1-6.3$ MeV are important for understanding this reaction rate. Experiment: The RESOLUT radioactive-ion beam facility at Florida State University was used to study $^{18}$Ne resonances around this energy region using $^{17}$F(p,p)$^{17}$F elastic scattering on a polypropylene target under inverse kinematics. Scattered protons were detected in a silicon-strip detector array while recoiling $^{17}$F ions were detected in coincidence in a gas ionization detector. Analysis: An $R$-matrix analysis of measured cross sections was conducted along with a reanalysis of data from previous measurements. Results: All the data analyzed are well described by a consistent set of parameters with with a $1^-$ assignment for a state at 6.14(1) MeV. A second comparable solution is also found with a $3^-$ assignment for the 6.14(1) MeV state. The rate of the $^{14}$O($\alpha$,p)$^{17}$F reaction that is determined from the two solutions differs by up to an order of magnitude., Comment: Update name of one of the authors

We present the results from spectroscopic and photometric analysis of 17 globular cluster (GC) candidates in the Irr II galaxy NGC 3077. The GC candidates were selected on the Hubble Space Telescope (HST) images and were cleaned of foreground Galactic stars using the GAIA parameters. We carried out aperture photometry using the multi-band archival images from SDSS, and 2MASS of all candidates, and low resolution (R= 1000) spectroscopic observations of 12 GC candidates and three suspected foreground stars using the OSIRIS/MOS mode at the Gran Telescopio Canarias (GTC). Age, metallicity and extinction values were determined both using spectroscopic and photometric data, independently. We find three of the 17 candidates are old (age >10 Gyr), metal-poor ([Fe/H]<-1.0 dex), massive (mass>10^5 Msun) GCs with characteristics similar to the classical GCs in the Milky Way. The rest are intermediate-age clusters (IACs) with typical ages of 3 to 4 Gyr, and in general metal-rich clusters. The radial velocities of both populations are within 100 km/s of the recessional velocity of the host galaxy. A relatively large population of IACs and low value of GC specific frequency (Sn=0.7) suggest that the pre-interaction galaxy was actively forming stars and star clusters, and is unlikely to be a dE as suggested in some previous works., Comment: Accepted in Astronomical Journal

We use the H$41\alpha$ recombination line to create templates of the millimeter free-free emission in the ALMA-IMF continuum maps, which allows to separate it from dust emission. This method complements spectral-index information and extrapolation from centimeter wavelength maps. We use the derived maps to estimate the properties of up to 34 HII regions across the ALMA-IMF protoclusters. The hydrogen ionizing-photon rate $Q_0$ and spectral types follow the evolutionary trend proposed by Motte et al. The youngest protoclusters lack detectable ionized gas, followed by protoclusters with increasing numbers of OB stars. The total $Q_0$ increases from $\sim 10^{45}$ s$^{-1}$ to $> 10^{49}$ s$^{-1}$. We used the adjacent He$41\alpha$ line to measure the relative number abundances of helium, finding values consistent with the Galactic interstellar medium, although a few outliers are discussed. A search for sites of maser amplification of the H$41\alpha$ line returned negative results. We looked for possible correlations between the electron densities ($n_e$), emission measures (EM), and $Q_0$ with HII region size $D$. The latter are the better correlated, with $Q_0 \propto D^{2.49\pm0.18}$. This favors interpretations where smaller ultracompact HII regions are not necessarily the less dynamically evolved versions of larger ones, but rather are ionized by less massive stars. Moderate correlations were found between dynamical width $\Delta V_\mathrm{dyn}$ with $D$ and $Q_0$. $\Delta V_\mathrm{dyn}$ increases from about one to two times the ionized-gas sound speed. Finally, an outlier HII region south of W43-MM2 is discussed. We suggest that this source could harbor an embedded stellar or disk wind., Comment: Accepted to AAS Journals (ApJS). Data available at https://dataverse.harvard.edu/dataverse/h41a-freefree

In this note we show that the usual notion of fuzzy norm defined on a linear space is equivalent to that of quasiconcave function, in the sense that every fuzzy norm $N:X\times\mathbb{R}[0,1]$ defined on a (real or complex) linear space X is uniquely determined by a quasiconcave function $f:X\to[0, 1]$. We explore the minimum requirements that we need to impose to some quasiconcave function $f:X\to[0, 1]$ in order to define a fuzzy norm $N:X\times\mathbb{R}[0,1]$. Later we use this equivalence to prove some properties of fuzzy norms, like a generalisation of the celebrated Decomposition Theorem.

External control arms (ECA) can inform the early clinical development of experimental drugs and provide efficacy evidence for regulatory approval. However, the main challenge in implementing ECA lies in accessing real-world or historical clinical trials data. Indeed, regulations protecting patients' rights by strictly controlling data processing make pooling data from multiple sources in a central server often difficult. To address these limitations, we develop a new method, 'FedECA' that leverages federated learning (FL) to enable inverse probability of treatment weighting (IPTW) for time-to-event outcomes on separate cohorts without needing to pool data. To showcase the potential of FedECA, we apply it in different settings of increasing complexity culminating with a real-world use-case in which FedECA provides evidence for a differential effect between two drugs that would have otherwise gone unnoticed. By sharing our code, we hope FedECA will foster the creation of federated research networks and thus accelerate drug development., Comment: code available at: https://github.com/owkin/fedeca, bug in SMD computation present in v1 and v2 has been fixed, many experiments on real data have been added + fix in YODA experiments using imputed data instead of raw data (v3->v4) as well as affiliations fix + more precise wording for acknowledgments

We present the combination of ALMA-IMF and single-dish continuum images from the Mustang-2 Galactic Plane Survey (MGPS90) at 3 millimeters and the Bolocam Galactic Plane Survey (BGPS) at 1 millimeter. Six and ten out of the fiffteen ALMA-IMF fields are combined with MGPS90 and BGPS, respectively. The combination is made via the feathering technique. We used the dendrogram algorithm throughout the combined images, and performed further analysis in the six fields with combination in both bands (G012.80, W43-MM1, W43-MM2, W43-MM3, W51-E, W51-IRS2). In these fields, we calculated spectral index maps and used them to separate regions dominated by dust or free-free emission, and then performed further structural analysis. We report the basic physical parameters of the dust-dominated (column densities, masses) and ionized (emission measures, hydrogen ionization photon rates) structures. We also searched for multi-scale relations in the dust-dominated structures across the analyzed fields, finding that the fraction of mass in dendrogram leaves (which we label as "Leaf Mass Eficiency", LME) as a function of molecular gas column density follows a similar trend: a rapid, exponential-like growth, with maximum values approaching 100% in most cases. The observed behaviour of the LME with gas column is tentatively interpreted as an indicator of large star formation activity within the ALMA-IMF protoclusters. W51-E and G012.80 stand out as cases with comparatively large and reduced potential for further star formation, respectively., Comment: Accepted to The Astrophysical Journal Supplement

We present an update of the survey of Active Galaxies with the High Altitude Water Cherenkov (HAWC) gamma-ray observatory. This work adds 567 days of HAWC data to the previously published survey, providing a refined analysis of an updated total exposure of 2090 days. The sample includes 138 nearby AGNs from the 3FHL catalog. We fit a modified power-law to their very high energy spectra, including the exponential attenuation caused by the Extragalactic Background Light. We found four sources with significant detections (above 5$\sigma$): the radio galaxy M87 and the BL Lac objects Mkn 421, Mkn 501 and 1ES 1215+303. We also report eight sources with a marginal detection (between 3$\sigma$ and 5$\sigma$) of which seven are classified as BL Lac objects and one as a radio galaxy., Comment: Contribution to the International Cosmic Ray Conference ICRC 2023. 7 pages, 3 figures

Blazars are the most abundant type of extragalactic gamma-ray source, usually presenting high variability across the electromagnetic spectrum. Their Very High Energy (VHE, above 0.1 TeV) emission has been studied in detail using Air Cherenkov Telescopes, with observations biased to flaring periods while their average activity has not been properly characterized. In this work, we report the results of 2090 days of quasi-continuous observations of the blazars 1ES 1215+303 and VER J0521+211, carried out with the High Altitude Water Cherenkov (HAWC) gamma-ray observatory. Fitting a power-law attenuated by photon-photon interaction with the extragalactic background light, we obtained a 6.2 $\sigma$ level detection for 1ES 1215+303 and a 4.3 $\sigma$ marginal detection for VER J0521+211. With the inclusion of the HAWC TeV spectrum, we built quasi-simultaneous multiwavelength spectral distributions and fitted a leptonic emission model to the observed data., Comment: Contribution to the International Cosmic Ray Conference ICRC 2023. 7 pages, 2 figures

Structured sentiment analysis (SSA) aims to automatically extract people's opinions from a text in natural language and adequately represent that information in a graph structure. One of the most accurate methods for performing SSA was recently proposed and consists of approaching it as a dependency parsing task. Although we can find in the literature how transition-based algorithms excel in dependency parsing in terms of accuracy and efficiency, all proposed attempts to tackle SSA following that approach were based on graph-based models. In this article, we present the first transition-based method to address SSA as dependency parsing. Specifically, we design a transition system that processes the input text in a left-to-right pass, incrementally generating the graph structure containing all identified opinions. To effectively implement our final transition-based model, we resort to a Pointer Network architecture as a backbone. From an extensive evaluation, we demonstrate that our model offers the best performance to date in practically all cases among prior dependency-based methods, and surpass recent task-specific techniques on the most challenging datasets. We additionally include an in-depth analysis and empirically prove that the overall time-complexity cost of our approach is quadratic in the sentence length, being more efficient than top-performing graph-based parsers.

We present an investigation on escape fractions of UV photons from a unique sample of lensed low-mass emission line selected galaxies at z < 3.5 found in the SHARDS Hubble Frontier Fields medium-band survey. We have used this deep imaging survey to locate 42 relatively low-mass galaxies, down to $log(M_{*}/M_{\odot}) = 7$, between redshifts 2.4 < z < 3.5 which are candidate line emitters. Using deep multi-band Hubble UVIS imaging we investigate the flux of escaping ionizing photons from these systems, obtaining 1$\sigma$ upper limits of $f^{rel}_{esc}$ ~7% for individual galaxies, and < 2% for stacked data. We measure potential escaping Lyman-continuum flux for two low-mass line emitters with values at $f^{\rm rel}_{\rm esc} = 0.032^{+0.081}_{-0.009}$ and $f^{\rm rel}_{\rm esc} = 0.021^{+0.101}_{-0.006}$, both detected at the ~3.2$\sigma$ level. A detailed analysis of possible contamination reveals a < 0.1% probability that these detections result from line-of-sight contamination. The relatively low Lyman-continuum escape fraction limit, and the low fraction of systems detected, is an indication that low-mass line emitting galaxies may not be as important a source of reionization as hoped if these are analogs of reionization sources. We also investigate the structures of our galaxy sample, finding no evidence for a correlation of escape fraction with asymmetric structure., Comment: Accept to ApJ, AAS Journals, 23 pages, 12 figures

The cross section and rate constants for the title reaction are calculated for all the spin-orbit states of N$^+$($^3P_{J_A}$) using two statistical approaches, one purely adiabatic and the other one mixing quantum capture for the entrance channel and adiabatic treatment for the products channel. This is made by using a symmetry adapted basis set combining electronic (spin and orbital) and nuclear angular momenta in the reactants channel. To this aim, accurate {\it ab initio} calculations are performed separately for reactants and products. In the reactants channel, the three lowest electronic states (without spin-orbit couplings) have been diabatized, and the spin-orbit couplings have been introduced through a model localizing the spin-orbit interactions in the N$^+$ atom, which yields accurate results as compared to {\it ab initio} calculations including spin-orbit couplings. For the products, eleven purely adiabatic spin-orbit states have been determined with {\it ab initio} calculations. The reactive rate constants thus obtained are in very good agreement with the available experimental data for several ortho-H$_2$ fractions, assuming a thermal initial distribution of spin-orbit states. The rate constants for selected spin-orbit $J_A$ states are obtained, to provide a proper validation of the spin-orbit effects to obtain the experimental rate constants., Comment: 14 pages, 10 figures, submitted to J. Chem. Phys

Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabilities are as yet poorly characterized. In order to inform future research, prepare for disruptive new model capabilities, and ameliorate socially harmful effects, it is vital that we understand the present and near-future capabilities and limitations of language models. To address this challenge, we introduce the Beyond the Imitation Game benchmark (BIG-bench). BIG-bench currently consists of 204 tasks, contributed by 450 authors across 132 institutions. Task topics are diverse, drawing problems from linguistics, childhood development, math, common-sense reasoning, biology, physics, social bias, software development, and beyond. BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models. We evaluate the behavior of OpenAI's GPT models, Google-internal dense transformer architectures, and Switch-style sparse transformers on BIG-bench, across model sizes spanning millions to hundreds of billions of parameters. In addition, a team of human expert raters performed all tasks in order to provide a strong baseline. Findings include: model performance and calibration both improve with scale, but are poor in absolute terms (and when compared with rater performance); performance is remarkably similar across model classes, though with benefits from sparsity; tasks that improve gradually and predictably commonly involve a large knowledge or memorization component, whereas tasks that exhibit "breakthrough" behavior at a critical scale often involve multiple steps or components, or brittle metrics; social bias typically increases with scale in settings with ambiguous context, but this can be improved with prompting., Comment: 27 pages, 17 figures + references and appendices, repo: https://github.com/google/BIG-bench

Semantic role labeling (SRL) focuses on recognizing the predicate-argument structure of a sentence and plays a critical role in many natural language processing tasks such as machine translation and question answering. Practically all available methods do not perform full SRL, since they rely on pre-identified predicates, and most of them follow a pipeline strategy, using specific models for undertaking one or several SRL subtasks. In addition, previous approaches have a strong dependence on syntactic information to achieve state-of-the-art performance, despite being syntactic trees equally hard to produce. These simplifications and requirements make the majority of SRL systems impractical for real-world applications. In this article, we propose the first transition-based SRL approach that is capable of completely processing an input sentence in a single left-to-right pass, with neither leveraging syntactic information nor resorting to additional modules. Thanks to our implementation based on Pointer Networks, full SRL can be accurately and efficiently done in $O(n^2)$, achieving the best performance to date on the majority of languages from the CoNLL-2009 shared task., Comment: Final peer-reviewed manuscript accepted for publication in Knowledge-Based Systems

Observations of 12CO J=1-0 and HCN J=1-0 emission from NGC 5194 (M51) made with the 50~meter Large Millimeter Telescope and the SEQUOIA focal plane array are presented. Using the HCN to CO ratio, we examine the dense gas mass fraction over a range of environmental conditions within the galaxy. Within the disk, the dense gas mass fraction varies along spiral arms but the average value over all spiral arms is comparable to the mean value of interarm regions. We suggest that the near constant dense gas mass fraction throughout the disk arises from a population of density stratified, self gravitating molecular clouds and the required density threshold to detect each spectral line. The measured dense gas fraction significantly increases in the central bulge in response to the effective pressure, P_e, from the weight from the stellar and gas components. This pressure modifies the dynamical state of the molecular cloud population and possibly, the HCN emitting regions, in the central bulge from self-gravitating to diffuse configurations in which P_e is greater than the gravitational energy density of individual clouds. Diffuse molecular clouds comprise a significant fraction of the molecular gas mass in the central bulge, which may account for the measured sublinear relationships between the surface densities of the star formation rate and molecular and dense gas., Comment: 18 pages, 8 figures, 5 tables. Accepted for publication in ApJ

The Multi-Sampling Ionization Chamber (MUSIC) detector is typically used to measure nuclear reaction cross sections relevant for nuclear astrophysics, fusion studies, and other applications. From the MUSIC data produced in one experiment scientists carefully extract an order of $10^3$ events of interest from about $10^{9}$ total events, where each event can be represented by an 18-dimensional vector. However, the standard data classification process is based on expert-driven, manually intensive data analysis techniques that require several months to identify patterns and classify the relevant events from the collected data. To address this issue, we present a method for the classification of events originating from specific $\alpha$-induced reactions by combining statistical and machine learning methods that require significantly less input from the domain scientist, relative to the standard technique. We applied the new method to two experimental data sets and compared our results with those obtained using traditional methods. With few exceptions, the number of events classified by our method agrees within $\pm20\%$ with the results obtained using traditional methods. With the present method, which is the first of its kind for the MUSIC data, we have established the foundation for the automated extraction of physical events of interest from experiments using the MUSIC detector.

We present unprecedented high fidelity radio images of the M87 jet. We analyzed Jansky Very Large Array (VLA) broadband, full polarization, radio data from 4 to 18 GHz. The observations were taken with the most extended configuration (A configuration), which allow the study of the emission of the jet up to kpc scales with a linear resolution $\sim$10 pc. The high sensitivity and resolution of our data allow to resolve the jet width. We confirm a double-helix morphology of the jet material between $\sim$300 pc and $\sim$1 kpc. We found a gradient of the polarization degree with a minimum at the projected axis and maxima at the jet edges, and a gradient in the Faraday depth with opposite signs at the jet edges. We also found that the behavior of the polarization properties along the wide range of frequencies is consistent with internal Faraday depolarization. All these characteristics strongly support the presence of a helical magnetic field in the M87 jet up to 1 kpc from the central black hole although the jet is most likely particle dominated at these large scales. Therefore, we propose a plausible scenario in which the helical configuration of the magnetic field has been maintained to large scales thanks to the presence of Kelvin-Helmholtz instabilities., Comment: Published on ApJL

In order to achieve deep natural language understanding, syntactic constituent parsing is a vital step, highly demanded by many artificial intelligence systems to process both text and speech. One of the most recent proposals is the use of standard sequence-to-sequence models to perform constituent parsing as a machine translation task, instead of applying task-specific parsers. While they show a competitive performance, these text-to-parse transducers are still lagging behind classic techniques in terms of accuracy, coverage and speed. To close the gap, we here extend the framework of sequence-to-sequence models for constituent parsing, not only by providing a more powerful neural architecture for improving their performance, but also by enlarging their coverage to handle the most complex syntactic phenomena: discontinuous structures. To that end, we design several novel linearizations that can fully produce discontinuities and, for the first time, we test a sequence-to-sequence model on the main discontinuous benchmarks, obtaining competitive results on par with task-specific discontinuous constituent parsers and achieving state-of-the-art scores on the (discontinuous) English Penn Treebank., Comment: Final peer-reviewed manuscript accepted for publication in Neurocomputing