Search

GPT-4o is an autoregressive omni model that accepts as input any combination of text, audio, image, and video, and generates any combination of text, audio, and image outputs. It's trained end-to-end across text, vision, and audio, meaning all inputs and outputs are processed by the same neural network. GPT-4o can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time in conversation. It matches GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages, while also being much faster and 50\% cheaper in the API. GPT-4o is especially better at vision and audio understanding compared to existing models. In line with our commitment to building AI safely and consistent with our voluntary commitments to the White House, we are sharing the GPT-4o System Card, which includes our Preparedness Framework evaluations. In this System Card, we provide a detailed look at GPT-4o's capabilities, limitations, and safety evaluations across multiple categories, focusing on speech-to-speech while also evaluating text and image capabilities, and measures we've implemented to ensure the model is safe and aligned. We also include third-party assessments on dangerous capabilities, as well as discussion of potential societal impacts of GPT-4o's text and vision capabilities.

Spherical proportional counters (SPCs) are gaseous particle detectors sensitive to single ionization electrons in their target media, with large detector volumes and low background rates. The $\mbox{NEWS-G}$ collaboration employs this technology to search for low-mass dark matter, having previously performed searches with detectors at the Laboratoire Souterrain de Modane (LSM), including a recent campaign with a 135 cm diameter SPC filled with methane. While in situ calibrations of the detector response were carried out at the LSM, measurements of the mean ionization yield and fluctuations of methane gas in SPCs were performed using a 30 cm diameter detector. The results of multiple measurements taken at different operating voltages are presented. A UV laser system was used to measure the mean gas gain of the SPC, along with $\mathrm{^{37}Ar}$ and aluminum-fluorescence calibration sources. These measurements will inform the energy response model of future operating detectors., Comment: 11 pages, 8 figures

Neural ODEs are a recently developed model class that combine the strong model priors of differential equations with the high-capacity function approximation of neural networks. One advantage of Neural ODEs is the potential for memory-efficient training via the continuous adjoint method. However, memory-efficient training comes at the cost of approximate gradients. Therefore, in practice, gradients are often obtained by simply backpropagating through the internal operations of the forward ODE solve - incurring high memory cost. Interestingly, it is possible to construct algebraically reversible ODE solvers that allow for both exact gradients and the memory-efficiency of the continuous adjoint method. Unfortunately, current reversible solvers are low-order and suffer from poor numerical stability. The use of these methods in practice is therefore limited. In this work, we present a class of algebraically reversible solvers that are both high-order and numerically stable. Moreover, any explicit numerical scheme can be made reversible by our method. This construction naturally extends to numerical schemes for Neural CDEs and SDEs., Comment: Preprint

In dense retrieval, deep encoders provide embeddings for both inputs and targets, and the softmax function is used to parameterize a distribution over a large number of candidate targets (e.g., textual passages for information retrieval). Significant challenges arise in training such encoders in the increasingly prevalent scenario of (1) a large number of targets, (2) a computationally expensive target encoder model, (3) cached target embeddings that are out-of-date due to ongoing training of target encoder parameters. This paper presents a simple and highly scalable response to these challenges by training a small parametric corrector network that adjusts stale cached target embeddings, enabling an accurate softmax approximation and thereby sampling of up-to-date high scoring "hard negatives." We theoretically investigate the generalization properties of our proposed target corrector, relating the complexity of the network, staleness of cached representations, and the amount of training data. We present experimental results on large benchmark dense retrieval datasets as well as on QA with retrieval augmented language models. Our approach matches state-of-the-art results even when no target embedding updates are made during training beyond an initial cache from the unsupervised pre-trained model, providing a 4-80x reduction in re-embedding computational cost., Comment: ICML 2024

In text generation, hallucinations refer to the generation of seemingly coherent text that contradicts established knowledge. One compelling hypothesis is that hallucinations occur when a language model is given a generation task outside its parametric knowledge (due to rarity, recency, domain, etc.). A common strategy to address this limitation is to infuse the language models with retrieval mechanisms, providing the model with relevant knowledge for the task. In this paper, we leverage the planning capabilities of instruction-tuned LLMs and analyze how planning can be used to guide retrieval to further reduce the frequency of hallucinations. We empirically evaluate several variations of our proposed approach on long-form text generation tasks. By improving the coverage of relevant facts, plan-guided retrieval and generation can produce more informative responses while providing a higher rate of attribution to source documents.

We present new JWST observations of the nearby, prototypical edge-on, spiral galaxy NGC 891. The northern half of the disk was observed with NIRCam in its F150W and F277W filters. Absorption is clearly visible in the mid-plane of the F150W image, along with vertical dusty plumes that closely resemble the ones seen in the optical. A $\sim 10 \times 3~{\rm kpc}^2$ area of the lower circumgalactic medium (CGM) was mapped with MIRI F770W at 12 pc scales. Thanks to the sensitivity and resolution of JWST, we detect dust emission out to $\sim 4$ kpc from the disk, in the form of filaments, arcs, and super-bubbles. Some of these filaments can be traced back to regions with recent star formation activity, suggesting that feedback-driven galactic winds play an important role in regulating baryonic cycling. The presence of dust at these altitudes raises questions about the transport mechanisms at play and suggests that small dust grains are able to survive for several tens of million years after having been ejected by galactic winds in the disk-halo interface. We lay out several scenarios that could explain this emission: dust grains may be shielded in the outer layers of cool dense clouds expelled from the galaxy disk, and/or the emission comes from the mixing layers around these cool clumps where material from the hot gas is able to cool down and mix with these cool cloudlets. This first set of data and upcoming spectroscopy will be very helpful to understand the survival of dust grains in energetic environments, and their contribution to recycling baryonic material in the mid-plane of galaxies., Comment: Accepted for publication in Astronomy & Astrophysics; 16 pages, 8 figures

The diagnosis of bronchiectasis requires measuring abnormal bronchial dilation. It is confirmed using a chest CT scan, where the key feature is an increased broncho-arterial ratio (BAR) (>0.8 in children), often with bronchial wall thickening. Image processing methods facilitate quicker interpretation and detailed evaluations by lobes and segments. Challenges like inclined nature, oblique orientation, and partial volume effect make it difficult to obtain accurate measurements in the upper and middle lobes using the same algorithms. Therefore, accurate detection and measurement of airway and artery regions for BAR and wall thickness in each lobe require different image processing/machine learning methods. We propose methods for: 1. Separating the right lower lobe (RLL) region from full-length CT scans using the tracheal bifurcation (Carina) point as a central marker; 2. Locating the inner diameter of airways and outer diameter of arteries for BAR measurement; and 3. Measuring airway wall thickness (WT) by identifying the outer and inner diameters of airway boundaries. Analysis of 13 HRCT scans with varying thicknesses (0.67mm, 1mm, 2mm) shows the tracheal bifurcation frame can be detected accurately, with a deviation of +/- 2 frames in some cases. A Windows app was developed for measuring inner airway diameter, artery diameter, BAR, and wall thickness, allowing users to draw boundaries around visible BA pairs in the RLL region. Measurements of 10 BA pairs revealed accurate results comparable to those of a human reader, with deviations of +/- 0.10-0.15mm. Additional studies and validation are needed to consolidate inter- and intra-rater variability and enhance the methods., Comment: 14 pages

The NEWS-G direct detection experiment uses spherical proportional counters to search for light dark matter candidates. New results from a 10 day physics run with a $135\,\mathrm{cm}$ in diameter spherical proportional counter at the Laboratoire Souterrain de Modane are reported. The target consists of $114\,\mathrm{g}$ of methane, providing sensitivity to dark matter spin-dependent coupling to protons. New constraints are presented in the mass range $0.17$ to $1.2\,\mathrm{GeV/c^2}$, with a 90% confidence level cross-section upper limit of $30.9\,\mathrm{pb}$ for a mass of $0.76\,\mathrm{GeV/c^2}$., Comment: 7 pages, 3 figures

Topic models are widely used to analyze document collections. While they are valuable for discovering latent topics in a corpus when analysts are unfamiliar with the corpus, analysts also commonly start with an understanding of the content present in a corpus. This may be through categories obtained from an initial pass over the corpus or a desire to analyze the corpus through a predefined set of categories derived from a high level theoretical framework (e.g. political ideology). In these scenarios analysts desire a topic modeling approach which incorporates their understanding of the corpus while supporting various forms of interaction with the model. In this work, we present EdTM, as an approach for label name supervised topic modeling. EdTM models topic modeling as an assignment problem while leveraging LM/LLM based document-topic affinities and using optimal transport for making globally coherent topic-assignments. In experiments, we show the efficacy of our framework compared to few-shot LLM classifiers, and topic models based on clustering and LDA. Further, we show EdTM's ability to incorporate various forms of analyst feedback and while remaining robust to noisy analyst inputs., Comment: Pre-print; Work in progress

Large language models have demonstrated impressive performance on commonsense tasks; however, these tasks are often posed as multiple-choice questions, allowing models to exploit systematic biases. Commonsense is also inherently probabilistic with multiple correct answers. The purpose of "boiling water" could be making tea and cooking, but it also could be killing germs. Existing tasks do not capture the probabilistic nature of common sense. To this end, we present commonsense frame completion (CFC), a new generative task that evaluates common sense via multiple open-ended generations. We also propose a method of probabilistic evaluation that strongly correlates with human judgments. Humans drastically outperform strong language model baselines on our dataset, indicating this approach is both a challenging and useful evaluation of machine common sense., Comment: ACL 2024 Camera Ready

A common retrieve-and-rerank paradigm involves retrieving relevant candidates from a broad set using a fast bi-encoder (BE), followed by applying expensive but accurate cross-encoders (CE) to a limited candidate set. However, relying on this small subset is often susceptible to error propagation from the bi-encoders, which limits the overall performance. To address these issues, we propose the Comparing Multiple Candidates (CMC) framework. CMC compares a query and multiple embeddings of similar candidates (i.e., neighbors) through shallow self-attention layers, delivering rich representations contextualized to each other. Furthermore, CMC is scalable enough to handle multiple comparisons simultaneously. For example, comparing ~10K candidates with CMC takes a similar amount of time as comparing 16 candidates with CE. Experimental results on the ZeSHEL dataset demonstrate that CMC, when plugged in between bi-encoders and cross-encoders as a seamless intermediate reranker (BE-CMC-CE), can effectively improve recall@k (+4.8%-p, +3.5%-p for R@16, R@64) compared to using only bi-encoders (BE-CE), with negligible slowdown (<7%). Additionally, to verify CMC's effectiveness as the final-stage reranker in improving top-1 accuracy, we conduct experiments on downstream tasks such as entity, passage, and dialogue ranking. The results indicate that CMC is not only faster (11x) but also often more effective than CE, with improved prediction accuracy in Wikipedia entity linking (+0.7%-p) and DSTC7 dialogue ranking (+3.3%-p)., Comment: accepted to EMNLP 2024 main track

Cross-encoder (CE) models which compute similarity by jointly encoding a query-item pair perform better than embedding-based models (dual-encoders) at estimating query-item relevance. Existing approaches perform k-NN search with CE by approximating the CE similarity with a vector embedding space fit either with dual-encoders (DE) or CUR matrix factorization. DE-based retrieve-and-rerank approaches suffer from poor recall on new domains and the retrieval with DE is decoupled from the CE. While CUR-based approaches can be more accurate than the DE-based approach, they require a prohibitively large number of CE calls to compute item embeddings, thus making it impractical for deployment at scale. In this paper, we address these shortcomings with our proposed sparse-matrix factorization based method that efficiently computes latent query and item embeddings to approximate CE scores and performs k-NN search with the approximate CE similarity. We compute item embeddings offline by factorizing a sparse matrix containing query-item CE scores for a set of train queries. Our method produces a high-quality approximation while requiring only a fraction of CE calls as compared to CUR-based methods, and allows for leveraging DE to initialize the embedding space while avoiding compute- and resource-intensive finetuning of DE via distillation. At test time, the item embeddings remain fixed and retrieval occurs over rounds, alternating between a) estimating the test query embedding by minimizing error in approximating CE scores of items retrieved thus far, and b) using the updated test query embedding for retrieving more items. Our k-NN search method improves recall by up to 5% (k=1) and 54% (k=100) over DE-based approaches. Additionally, our indexing approach achieves a speedup of up to 100x over CUR-based and 5x over DE distillation methods, while matching or improving k-NN search recall over baselines., Comment: ICLR 2024

Widespread, high altitude, diffuse ionized gas with scale heights of around a kiloparsec is observed in the Milky Way and other star forming galaxies. Numerical radiation-magnetohydrodynamic simulations of a supernova-driven turbulent interstellar medium show that gas can be driven to high altitudes above the galactic midplane, but the degree of ionization is often less than inferred from observations. For computational expediency, ionizing radiation from massive stars is often included as a post-processing step assuming ionization equilibrium. We extend our simulations of a Milky Way-like interstellar medium to include the combined effect of supernovae and photoionization feedback from midplane OB stars and a population of hot evolved low mass stars. The diffuse ionized gas has densities below 0.1 ${\rm cm^{-3}}$, so recombination timescales can exceed millions of years. Our simulations now follow the time-dependent ionization and recombination of low density gas. The long recombination timescales result in diffuse ionized gas that persists at large altitudes long after the deaths of massive stars that produce the vast majority of the ionized gas. The diffuse ionized gas does not exhibit the large variability inherent in simulations that adopt ionization equilibrium. The vertical distribution of neutral and ionized gas is close to what is observed in the Milky Way. The volume filling factor of ionized gas increases with altitude resulting in the scale height of free electrons being larger than that inferred from H$\alpha$ emission, thus reconciling the observations of ionized gas made in H$\alpha$ and from pulsar dispersion measurements., Comment: 17 pages, 16 figures, accepted for publication in MNRAS

We develop an efficient algorithm to find optimal observation times by maximizing the Fisher information for the birth rate of a partially observable pure birth process involving $n$ observations. Partially observable implies that at each of the $n$ observation time points for counting the number of individuals present in the pure birth process, each individual is observed independently with a fixed probability $p$, modeling detection difficulties or constraints on resources. We apply concepts and techniques from generating functions, using a combination of symbolic and numeric computation, to establish a recursion for evaluating and optimizing the Fisher information. Our numerical results reveal the efficacy of this new method. An implementation of the algorithm is available publicly.

Accurate detection and analysis of traces of persistent organic pollutants in water is important in many areas, including environmental monitoring and food quality control, due to their long environmental stability and potential bioaccumulation. While conventional analysis of organic pollutants requires expensive equipment, surface enhanced Raman spectroscopy (SERS) has demonstrated great potential for accurate detection of these contaminants. However, SERS analytical difficulties, such as spectral preprocessing, denoising, and substrate-based spectral variation, have hindered widespread use of the technique. Here, we demonstrate an approach for predicting the concentration of sample pollutants from messy, unprocessed Raman data using machine learning. Frequency domain transform methods, including the Fourier and Walsh Hadamard transforms, are applied to sets of Raman spectra of three model micropollutants in water (rhodamine 6G, chlorpyrifos, and triclosan), which are then used to train machine learning algorithms. Using standard machine learning models, the concentration of sample pollutants are predicted with more than 80 percent cross-validation accuracy from raw Raman data. cross-validation accuracy of 85 percent was achieved using deep learning for a moderately sized dataset (100 spectra), and 70 to 80 percent cross-validation accuracy was achieved even for very small datasets (50 spectra). Additionally, standard models were shown to accurately identify characteristic peaks via analysis of their importance scores. The approach shown here has the potential to be applied to facilitate accurate detection and analysis of persistent organic pollutants by surface-enhanced Raman spectroscopy.

We have conducted an extensive towed-magnetic-sled survey during the period of June 14-28, 2023, over the seafloor about 85 km north of Manus Island, Papua New Guinea, centered around the calculated path of the bolide CNEOS 2014-01-08 (IM1). We found about 850 spherules of diameter 0.1-1.3 millimeters in our samples. The samples were analyzed by micro-XRF, Electron Probe Microanalyzer and ICP Mass spectrometry. Here we report major and trace element compositions of the samples and classify spherules based on that analysis. Spherules comprising 22% of the collection, appear to all reflect planetary igneous differentiation and are all different from previously described spherules. A subset of the differentiated-spherules show an excess of Be, La and U, by up to three orders of magnitude relative to the solar system standard of CI chondrites. Detailed mass spectrometry of 12 of these "BeLaU"-type spherules, the population of which may constitute up to ~10% of our entire collected sample, suggests that they are derived from material formed by planetary igneous fractionation. Their chemical composition is unlike any known solar system material. The "BeLaU"-type spherules reflect a highly differentiated, extremely evolved composition of an unknown source., Comment: Accepted for publication in Chemical Geology

Extractive opinion summarization involves automatically producing a summary of text about an entity (e.g., a product's reviews) by extracting representative sentences that capture prevalent opinions in the review set. Typically, in online marketplaces user reviews accumulate over time, and opinion summaries need to be updated periodically to provide customers with up-to-date information. In this work, we study the task of extractive opinion summarization in an incremental setting, where the underlying review set evolves over time. Many of the state-of-the-art extractive opinion summarization approaches are centrality-based, such as CentroidRank (Radev et al., 2004; Chowdhury et al., 2022). CentroidRank performs extractive summarization by selecting a subset of review sentences closest to the centroid in the representation space as the summary. However, these methods are not capable of operating efficiently in an incremental setting, where reviews arrive one at a time. In this paper, we present an efficient algorithm for accurately computing the CentroidRank summaries in an incremental setting. Our approach, CoverSumm, relies on indexing review representations in a cover tree and maintaining a reservoir of candidate summary review sentences. CoverSumm's efficacy is supported by a theoretical and empirical analysis of running time. Empirically, on a diverse collection of data (both real and synthetically created to illustrate scaling considerations), we demonstrate that CoverSumm is up to 36x faster than baseline methods, and capable of adapting to nuanced changes in data distribution. We also conduct human evaluations of the generated summaries and find that CoverSumm is capable of producing informative summaries consistent with the underlying review set., Comment: Accepted at TMLR

This paper builds and extends on the authors previous work related to the algorithmic tool, Cylindrical Algebraic Decomposition (CAD), and one of its core applications, Real Quantifier Elimination (QE). These topics are at the heart of symbolic computation and were first implemented in computer algebra systems decades ago, but have recently received renewed interest as part of the ongoing development of SMT solvers for non-linear real arithmetic. First, we consider the use of iterated univariate resultants in traditional CAD, and how this leads to inefficiencies, especially in the case of an input with multiple equational constraints. We reproduce the workshop paper [Davenport \& England, 2023], adding important clarifications to our suggestions first made there to make use of multivariate resultants in the projection phase of CAD. We then consider an alternative approach to this problem first documented in [McCallum \& Brown, 2009] which redefines the actual object under construction, albeit only in the case of two equational constraints. We correct an important typo and provide a missing proof in that paper. We finish by revising the topic of how to deal with SMT or Real QE problems expressed using rational functions (as opposed to the usual polynomial ones) noting that these are often found in industrial applications. We revisit a proposal made in [Uncu, Davenport and England, 2023] for doing this in the case of satisfiability, explaining why such an approach does not trivially extend to more complicated quantification structure and giving a suitable alternative., Comment: To be submitted to Mathematics in Computer Science

While semidefinite programming (SDP) has traditionally been limited to moderate-sized problems, recent algorithms augmented with matrix sketching techniques have enabled solving larger SDPs. However, these methods achieve scalability at the cost of an increase in the number of necessary iterations, resulting in slower convergence as the problem size grows. Furthermore, they require iteration-dependent parameter schedules that prohibit effective utilization of warm-start initializations important in practical applications with incrementally-arriving data or mixed-integer programming. We present Unified Spectral Bundling with Sketching (USBS), a provably correct, fast and scalable algorithm for solving massive SDPs that can leverage a warm-start initialization to further accelerate convergence. Our proposed algorithm is a spectral bundle method for solving general SDPs containing both equality and inequality constraints. Moveover, when augmented with an optional matrix sketching technique, our algorithm achieves the dramatically improved scalability of previous work while sustaining convergence speed. We empirically demonstrate the effectiveness of our method across multiple applications, with and without warm-starting. For example, USBS provides a 500x speed-up over the state-of-the-art scalable SDP solver on an instance with over 2 billion decision variables.

Despite numerous successes, the field of reinforcement learning (RL) remains far from matching the impressive generalisation power of human behaviour learning. One possible way to help bridge this gap be to provide RL agents with richer, more human-like feedback expressed in natural language. To investigate this idea, we first extend BabyAI to automatically generate language feedback from the environment dynamics and goal condition success. Then, we modify the Decision Transformer architecture to take advantage of this additional signal. We find that training with language feedback either in place of or in addition to the return-to-go or goal descriptions improves agents' generalisation performance, and that agents can benefit from feedback even when this is only available during training, but not at inference., Comment: Accepted at Workshop on Goal-conditioned Reinforcement Learning, NeurIPS 2023

We study semi-supervised sequence generation tasks, where the few labeled examples are too scarce to finetune a model, and meanwhile, few-shot prompted large language models (LLMs) exhibit room for improvement. In this paper, we present the discovery that a student model distilled from a few-shot prompted LLM can commonly generalize better than its teacher to unseen examples on such tasks. We find that the student is able to learn a general pattern from the high-quality pseudolabels produced by the teacher during knowledge distillation (KD), and favorably not a general pattern from the low-quality pseudolables. Leveraging this discovery, we propose a new method, Multistage Collaborative Knowledge Distillation from an LLM (MCKD), for these tasks. MCKD first few-shot prompts an LLM to produce pseudolabels for unlabeled data. Then at each stage of an iterative KD process, a new pair of students is trained on disjoint partitions of the pseudolabeled data, and produces new and improved pseudolabels for their unseen partitions. We conduct extensive experiments on four syntactic and semantic parsing datasets and show the effectiveness of MCKD for low-resource semi-supervised sequence generation. On CRAFT biomedical parsing, for example, 3-stage MCKD with 50 labeled examples outperforms an LLM teacher and vanilla KD by 7.5% and 3.7% parsing F1, respectively, and matches the performance of supervised finetuning with 500 labeled examples., Comment: ACL 2024

When strongly pumped at twice their resonant frequency, non-linear resonators develop a high-amplitude intracavity field, a phenomenon known as parametric self-oscillations. The boundary over which this instability occurs can be extremely sharp and thereby presents an opportunity for realizing a detector. Here we operate such a device based on a superconducting microwave resonator whose non-linearity is engineered from kinetic inductance. The device indicates the absorption of low-power microwave wavepackets by transitioning to a self-oscillating state. Using calibrated wavepackets we measure the detection efficiency with zeptojoule energy wavepackets. We then apply it to measurements of electron spin resonance, using an ensemble of $^{209}$Bi donors in silicon that are inductively coupled to the resonator. We achieve a latched-readout of the spin signal with an amplitude that is five hundred times greater than the underlying spin echoes.

A Gallium interstitial defect (Ga$_{\textrm{i}}$) is thought to be responsible for the spectacular spin-dependent recombination (SDR) in GaAs$_{1-x}$N$_x$ dilute nitride semiconductors. Current understanding associates this defect with two in-gap levels corresponding to the (+/0) and (++/+) charge-state transitions. Using a spin-sensitive photo-induced current transient spectroscopy, the in-gap electronic structure of a $x$ = 0.021 alloy is revealed. The (+/0) state lies $\approx$ 0.27 eV below the conduction band edge, and an anomalous, negative activation energy reveals the presence of not one but \textit{two} other states in the gap. The observations are consistent with a (++/+) state $\approx$ 0.19 eV above the valence band edge, and a hitherto ignored, (+++/++) state $\approx$ 25 meV above the valence band edge. These observations can inform efforts to better model the SDR and the Ga$_{\textrm{i}}$ defect's local chemical environment., Comment: 6 pages, 5 figures plus Supplementary Material

Recent studies show that large language models (LLMs) can be instructed to effectively perform zero-shot passage re-ranking, in which the results of a first stage retrieval method, such as BM25, are rated and reordered to improve relevance. In this work, we improve LLM-based re-ranking by algorithmically selecting few-shot demonstrations to include in the prompt. Our analysis investigates the conditions where demonstrations are most helpful, and shows that adding even one demonstration is significantly beneficial. We propose a novel demonstration selection strategy based on difficulty rather than the commonly used semantic similarity. Furthermore, we find that demonstrations helpful for ranking are also effective at question generation. We hope our work will spur more principled research into question generation and passage ranking., Comment: Findings of EMNLP 2023

Recent studies suggest that the existing neural models have difficulty handling repeated items in sequential recommendation tasks. However, our understanding of this difficulty is still limited. In this study, we substantially advance this field by identifying a major source of the problem: the single hidden state embedding and static item embeddings in the output softmax layer. Specifically, the similarity structure of the global item embeddings in the softmax layer sometimes forces the single hidden state embedding to be close to new items when copying is a better choice, while sometimes forcing the hidden state to be close to the items from the input inappropriately. To alleviate the problem, we adapt the recently-proposed softmax alternatives such as softmax-CPR to sequential recommendation tasks and demonstrate that the new softmax architectures unleash the capability of the neural encoder on learning when to copy and when to exclude the items from the input sequence. By only making some simple modifications on the output softmax layer for SASRec and GRU4Rec, softmax-CPR achieves consistent improvement in 12 datasets. With almost the same model size, our best method not only improves the average NDCG@10 of GRU4Rec in 5 datasets with duplicated items by 10% (4%-17% individually) but also improves 7 datasets without duplicated items by 24% (8%-39%)!, Comment: WSDM 2024

To overcome the challenges associated with poor temporal stability of perovskite solar cells, methods are required that allow for fast iteration of fabrication and characterisation, such that optimal device performance and stability may be actively pursued. Currently, establishing the causes of underperformance is both complex and time-consuming, and optimisation of device fabrication thus inherently slow. Here, we present a means of computational device characterisation of mobile halide ion parameters from room temperature current-voltage (J-V) measurements only, requiring $\sim 2$ hours of computation on basic computing resources. With our approach, the physical parameters of the device may be reverse modelled from experimental J-V measurements. In a drift-diffusion model, the set of coupled drift-diffusion partial differential equations cannot be inverted explicitly, so a method for inverting the drift-diffusion simulation is required. We show how Bayesian Parameter Estimation (BPE) coupled with a drift-diffusion perovskite solar cell model can determine the extent to which device parameters affect performance measured by J-V characteristics. Our method is demonstrated by investigating the extent to which device performance is influenced by mobile halide ions for a specific fabricated device. The ion vacancy density $N_0$ and diffusion coefficient $D_I$ were found to be precisely characterised for both simulated and fabricated devices. This result opens up the possibility of pinpointing origins of degradation by finding which parameters most influence device J-V curves as the cell degrades.

An understanding of how input parameter uncertainty in the numerical simulation of physical models leads to simulation output uncertainty is a challenging task. Common methods for quantifying output uncertainty, such as performing a grid or random search over the model input space, are computationally intractable for a large number of input parameters, represented by a high-dimensional input space. It is therefore generally unclear as to whether a numerical simulation can reproduce a particular outcome (e.g. a set of experimental results) with a plausible set of model input parameters. Here, we present a method for efficiently searching the input space using Bayesian Optimisation to minimise the difference between the simulation output and a set of experimental results. Our method allows explicit evaluation of the probability that the simulation can reproduce the measured experimental results in the region of input space defined by the uncertainty in each input parameter. We apply this method to the simulation of charge-carrier dynamics in the perovskite semiconductor methyl-ammonium lead iodide MAPbI$_3$ that has attracted attention as a light harvesting material in solar cells. From our analysis we conclude that the formation of large polarons, quasiparticles created by the coupling of excess electrons or holes with ionic vibrations, cannot explain the experimentally observed temperature dependence of electron mobility.

Many useful tasks on scientific documents, such as topic classification and citation prediction, involve corpora that span multiple scientific domains. Typically, such tasks are accomplished by representing the text with a vector embedding obtained from a Transformer's single CLS token. In this paper, we argue that using multiple CLS tokens could make a Transformer better specialize to multiple scientific domains. We present Multi2SPE: it encourages each of multiple CLS tokens to learn diverse ways of aggregating token embeddings, then sums them up together to create a single vector representation. We also propose our new multi-domain benchmark, Multi-SciDocs, to test scientific paper vector encoders under multi-domain settings. We show that Multi2SPE reduces error by up to 25 percent in multi-domain citation prediction, while requiring only a negligible amount of computation in addition to one BERT forward pass.

We have conducted an extensive towed-magnetic-sled survey during the period 14-28 June, 2023, over the seafloor centered around the calculated path of the bolide CNEOS 2014-01-08 (IM1) about 85 km north of Manus Island, Papua New Guinea. We found about 700 spherules of diameter 0.05-1.3 millimeters in our samples, of which 57 were analyzed so far. The spherules were significantly concentrated along the expected meteor path. Mass spectrometry of 47 spherules near the high-yield regions along IM1's path reveals a distinct extra-solar abundance pattern for 5 of them, while background spherules have abundances consistent with a solar system origin. The unique spherules show an excess of Be, La and U, by up to three orders of magnitude relative to the solar system standard of CI chondrites. These "BeLaU"-type spherules, never seen before, also have very low refractory siderophile elements such as Re. Volatile elements, such as Mn, Zn, Pb, are depleted as expected from evaporation losses during a meteor's airburst. In addition, the mass-dependent variations in $^{57}$Fe/$^{54}$Fe and $^{56}$Fe/$^{54}$Fe are also consistent with evaporative loss of the light isotopes during the spherules' travel in the atmosphere. The "BeLaU" abundance pattern is not found in control regions outside of IM1's path and does not match commonly manufactured alloys or natural meteorites in the solar system. This evidence points towards an association of "BeLaU"-type spherules with IM1, supporting its interstellar origin independently of the high velocity and unusual material strength implied from the CNEOS data. We suggest that the "BeLaU" abundance pattern could have originated from a highly differentiated magma ocean of a planet with an iron core outside the solar system or from more exotic sources., Comment: Submitted for publication in a peer-reviewed journal

Spins in silicon that are accessible via a telecom-compatible optical transition are a versatile platform for quantum information processing that can leverage the well-established silicon nanofabrication industry. Key to these applications are long coherence times on the optical and spin transitions to provide a robust system for interfacing photonic and spin qubits. Here, we report telecom-compatible Er3+ sites with long optical and electron spin coherence times, measured within a nuclear spin-free silicon crystal (<0.01% 29Si) using optical detection. We investigate two sites and find 0.1 GHz optical inhomogeneous linewidths and homogeneous linewidths below 70 kHz for both sites. We measure the electron spin coherence time of both sites using optically detected magnetic resonance and observe Hahn echo decay constants of 0.8 ms and 1.2 ms at around 11 mT. These optical and spin properties of Er3+:Si are an important milestone towards using optically accessible spins in silicon for a broad range of quantum information processing applications., Comment: 14 pages, 6 figures