Search

Material discovery is a critical area of research with the potential to revolutionize various fields, including carbon capture, renewable energy, and electronics. However, the immense scale of the chemical space makes it challenging to explore all possible materials experimentally. In this paper, we introduce FlowLLM, a novel generative model that combines large language models (LLMs) and Riemannian flow matching (RFM) to design novel crystalline materials. FlowLLM first fine-tunes an LLM to learn an effective base distribution of meta-stable crystals in a text representation. After converting to a graph representation, the RFM model takes samples from the LLM and iteratively refines the coordinates and lattice parameters. Our approach significantly outperforms state-of-the-art methods, increasing the generation rate of stable materials by over three times and increasing the rate for stable, unique, and novel crystals by $\sim50\%$ - a huge improvement on a difficult problem. Additionally, the crystals generated by FlowLLM are much closer to their relaxed state when compared with another leading model, significantly reducing post-hoc computational cost.

The ability to discover new materials with desirable properties is critical for numerous applications from helping mitigate climate change to advances in next generation computing hardware. AI has the potential to accelerate materials discovery and design by more effectively exploring the chemical space compared to other computational methods or by trial-and-error. While substantial progress has been made on AI for materials data, benchmarks, and models, a barrier that has emerged is the lack of publicly available training data and open pre-trained models. To address this, we present a Meta FAIR release of the Open Materials 2024 (OMat24) large-scale open dataset and an accompanying set of pre-trained models. OMat24 contains over 110 million density functional theory (DFT) calculations focused on structural and compositional diversity. Our EquiformerV2 models achieve state-of-the-art performance on the Matbench Discovery leaderboard and are capable of predicting ground-state stability and formation energies to an F1 score above 0.9 and an accuracy of 20 meV/atom, respectively. We explore the impact of model size, auxiliary denoising objectives, and fine-tuning on performance across a range of datasets including OMat24, MPtraj, and Alexandria. The open release of the OMat24 dataset and models enables the research community to build upon our efforts and drive further advancements in AI-assisted materials science., Comment: 19 pages

Crystalline materials are a fundamental component in next-generation technologies, yet modeling their distribution presents unique computational challenges. Of the plausible arrangements of atoms in a periodic lattice only a vanishingly small percentage are thermodynamically stable, which is a key indicator of the materials that can be experimentally realized. Two fundamental tasks in this area are to (a) predict the stable crystal structure of a known composition of elements and (b) propose novel compositions along with their stable structures. We present FlowMM, a pair of generative models that achieve state-of-the-art performance on both tasks while being more efficient and more flexible than competing methods. We generalize Riemannian Flow Matching to suit the symmetries inherent to crystals: translation, rotation, permutation, and periodic boundary conditions. Our framework enables the freedom to choose the flow base distributions, drastically simplifying the problem of learning crystal structures compared with diffusion models. In addition to standard benchmarks, we validate FlowMM's generated structures with quantum chemistry calculations, demonstrating that it is about 3x more efficient, in terms of integration steps, at finding stable materials compared to previous open methods., Comment: https://github.com/facebookresearch/flowmm

Multi-wavelength polarimetry and radio observations of Swift J1727.8-1613 at the beginning of its recent 2023 outburst suggested the presence of a bright compact jet aligned in the north-south direction, which could not be confirmed without high angular resolution images. Using the Very Long Baseline Array and the Long Baseline Array, we imaged Swift J1727.8-1613, during the hard/hard-intermediate state, revealing a bright core and a large, two-sided, asymmetrical, resolved jet. The jet extends in the north-south direction, at a position angle of $-0.60\pm0.07\deg$ East of North. At 8.4 GHz, the entire resolved jet structure is $\sim110 (d/2.7\,\text{kpc})/\sin i$ AU long, with the southern approaching jet extending $\sim80 (d/2.7\,\text{kpc})/\sin i$ AU from the core, where $d$ is the distance to the source and $i$ is the inclination of the jet axis to the line of sight. These images reveal the most resolved continuous X-ray binary jet, and possibly the most physically extended continuous X-ray binary jet ever observed. Based on the brightness ratio of the approaching and receding jets, we put a lower limit on the intrinsic jet speed of $\beta\geq0.27$ and an upper limit on the jet inclination of $i\leq74\deg$. In our first observation we also detected a rapidly fading discrete jet knot $66.89\pm0.04$ mas south of the core, with a proper motion of $0.66\pm0.05$ mas hour$^{-1}$, which we interpret as the result of a downstream internal shock or a jet-ISM interaction, as opposed to a transient relativistic jet launched at the beginning of the outburst., Comment: Accepted in ApJL

Newborn animals have advanced perceptual skills at birth, but the nature of this initial knowledge is unknown. Is initial knowledge flexible, continuously adapting to the statistics of experience? Or can initial knowledge be rigid and robust to change, even in the face of opposing evidence? We address this question through controlled-rearing experiments on newborn chicks. First, we reared chicks in an impoverished virtual world, where objects never occluded one another, and found that chicks still succeed on object permanence tasks. Second, we reared chicks in a virtual world in which objects teleported from one location to another while out of view: an unnatural event that violates the continuity of object motion. Despite seeing thousands of these violations of object permanence, and not a single non-violation, the chicks behaved as if object permanence were true, exhibiting the same behavior as chicks reared with natural object permanence events. We conclude that object permanence develops prenatally and is robust to change from opposing evidence., Comment: 10 pages, 4 figures

Vision transformers (ViTs) are top performing models on many computer vision benchmarks and can accurately predict human behavior on object recognition tasks. However, researchers question the value of using ViTs as models of biological learning because ViTs are thought to be more data hungry than brains, with ViTs requiring more training data to reach similar levels of performance. To test this assumption, we directly compared the learning abilities of ViTs and animals, by performing parallel controlled rearing experiments on ViTs and newborn chicks. We first raised chicks in impoverished visual environments containing a single object, then simulated the training data available in those environments by building virtual animal chambers in a video game engine. We recorded the first-person images acquired by agents moving through the virtual chambers and used those images to train self supervised ViTs that leverage time as a teaching signal, akin to biological visual systems. When ViTs were trained through the eyes of newborn chicks, the ViTs solved the same view invariant object recognition tasks as the chicks. Thus, ViTs were not more data hungry than newborn visual systems: both learned view invariant object representations in impoverished visual environments. The flexible and generic attention based learning mechanism in ViTs combined with the embodied data streams available to newborn animals appears sufficient to drive the development of animal-like object recognition., Comment: Accepted in Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023)

Foundation models have been transformational in machine learning fields such as natural language processing and computer vision. Similar success in atomic property prediction has been limited due to the challenges of training effective models across multiple chemical domains. To address this, we introduce Joint Multi-domain Pre-training (JMP), a supervised pre-training strategy that simultaneously trains on multiple datasets from different chemical domains, treating each dataset as a unique pre-training task within a multi-task framework. Our combined training dataset consists of $\sim$120M systems from OC20, OC22, ANI-1x, and Transition-1x. We evaluate performance and generalization by fine-tuning over a diverse set of downstream tasks and datasets including: QM9, rMD17, MatBench, QMOF, SPICE, and MD22. JMP demonstrates an average improvement of 59% over training from scratch, and matches or sets state-of-the-art on 34 out of 40 tasks. Our work highlights the potential of pre-training strategies that utilize diverse data to advance property prediction across chemical domains, especially for low-data tasks. Please visit https://nima.sh/jmp for further information.

Using multi-frequency Very Long Baseline Interferometer (VLBI) observations, we probe the jet size in the optically thick hard state jets of two black hole X-ray binary (BHXRB) systems, MAXI J1820+070 and V404 Cygni. Due to optical depth effects, the phase referenced VLBI core positions move along the jet axis of the BHXRB in a frequency dependent manner. We use this "core shift" to constrain the physical size of the hard state jet. We place an upper limit of $0.3$\,au on the jet size measured between the 15 and 5 GHz emission regions of the jet in MAXI J1820+070, and an upper limit of $1.0$\,au between the $8.4$ and $4.8$\,GHz emission regions of V404 Cygni. Our limit on the jet size in MAXI J1820+070 observed in the low-hard state is a factor of $5$ smaller than the values previously observed in the high-luminosity hard state (using time lags between multi-frequency light curves), thus showing evidence of the BHXRB jet scaling in size with jet luminosity. We also investigate whether motion of the radio-emitting region along the jet axis could affect the measured VLBI parallaxes for the two systems, leading to a mild tension with the parallax measurements of Gaia. Having mitigated the impact of any motion along the jet axis in the measured astrometry, we find the previous VLBI parallax measurements of MAXI J1820+070 and V404 Cygni to be unaffected by jet motion. With a total time baseline of $8$ years, due to having incorporated fourteen new epochs in addition to the previously published ones, our updated parallax measurement of V404 Cygni is $0.450 \pm 0.018$\,mas ($2.226 \pm 0.091$\,kpc)., Comment: Accepted in MNRAS. 7 Figures and 3 Tables

Recent progress in artificial intelligence has renewed interest in building machines that learn like animals. Almost all of the work comparing learning across biological and artificial systems comes from studies where animals and machines received different training data, obscuring whether differences between animals and machines emerged from differences in learning mechanisms versus training data. We present an experimental approach-a "newborn embodied Turing Test"-that allows newborn animals and machines to be raised in the same environments and tested with the same tasks, permitting direct comparison of their learning abilities. To make this platform, we first collected controlled-rearing data from newborn chicks, then performed "digital twin" experiments in which machines were raised in virtual environments that mimicked the rearing conditions of the chicks. We found that (1) machines (deep reinforcement learning agents with intrinsic motivation) can spontaneously develop visually guided preference behavior, akin to imprinting in newborn chicks, and (2) machines are still far from newborn-level performance on object recognition tasks. Almost all of the chicks developed view-invariant object recognition, whereas the machines tended to develop view-dependent recognition. The learning outcomes were also far more constrained in the chicks versus machines. Ultimately, we anticipate that this approach will help researchers develop embodied AI systems that learn like newborn animals., Comment: 7 Pages. 4 figures, 1 table. This paper was accepted to the CogSci 2023 Conference. (https://cognitivesciencesociety.org/)

Flare frequency distributions represent a key approach to addressing one of the largest problems in solar and stellar physics: determining the mechanism that counter-intuitively heats coronae to temperatures that are orders of magnitude hotter than the corresponding photospheres. It is widely accepted that the magnetic field is responsible for the heating, but there are two competing mechanisms that could explain it: nanoflares or Alfv\'en waves. To date, neither can be directly observed. Nanoflares are, by definition, extremely small, but their aggregate energy release could represent a substantial heating mechanism, presuming they are sufficiently abundant. One way to test this presumption is via the flare frequency distribution, which describes how often flares of various energies occur. If the slope of the power law fitting the flare frequency distribution is above a critical threshold, $\alpha=2$ as established in prior literature, then there should be a sufficient abundance of nanoflares to explain coronal heating. We performed $>$600 case studies of solar flares, made possible by an unprecedented number of data analysts via three semesters of an undergraduate physics laboratory course. This allowed us to include two crucial, but nontrivial, analysis methods: pre-flare baseline subtraction and computation of the flare energy, which requires determining flare start and stop times. We aggregated the results of these analyses into a statistical study to determine that $\alpha = 1.63 \pm 0.03$. This is below the critical threshold, suggesting that Alfv\'en waves are an important driver of coronal heating., Comment: 1,002 authors, 14 pages, 4 figures, 3 tables, published by The Astrophysical Journal on 2023-05-09, volume 948, page 71

Tracking the motions of transient jets launched by low-mass X-ray binaries (LMXBs) is critical for determining the moment of jet ejection, and identifying any corresponding signatures in the accretion flow. However, these jets are often highly variable and can travel across the resolution element of an image within a single observation, violating a fundamental assumption of aperture synthesis. We present a novel approach in which we directly fit a single time-dependent model to the full set of interferometer visibilities, where we explicitly parameterise the motion and flux density variability of the emission components, to minimise the number of free parameters in the fit, while leveraging information from the full observation. This technique allows us to detect and characterize faint, fast-moving sources, for which the standard time binning technique is inadequate. We validate our technique with synthetic observations, before applying it to three Very Long Baseline Array (VLBA) observations of the black hole candidate LMXB MAXI J1803-298 during its 2021 outburst. We measured the proper motion of a discrete jet component to be $1.37\pm0.14$ mas/hr, and thus we infer an ejection date of MJD $59348.08_{-0.06}^{+0.05}$, which occurs just after the peak of a radio flare observed by the Australia Telescope Compact Array (ATCA) and the Atacama Large Millimeter/Sub-Millimeter Array (ALMA), while MAXI J1803-298 was in the intermediate state. Further development of these new VLBI analysis techniques will lead to more precise measurements of jet ejection dates, which, combined with dense, simultaneous multi-wavelength monitoring, will allow for clearer identification of jet ejection signatures in the accretion flow., Comment: 15 pages, 9 figures, 4 tables; Accepted for publication in MNRAS

Precision medicine is an emerging field that takes into account individual heterogeneity to inform better clinical practice. In clinical trials, the evaluation of treatment effect heterogeneity is an important component, and recently, many statistical methods have been proposed for stratifying patients into different subgroups based on such heterogeneity. However, the majority of existing methods developed for this purpose focus on the case with a dichotomous treatment and are not directly applicable to multi-arm trials. In this paper, we consider the problem of patient stratification in multi-arm trial settings and propose a two-stage procedure within the Bayesian nonparametric framework. Specifically, we first use Bayesian additive regression trees (BART) to predict potential outcomes (treatment responses) under different treatment options for each patient, and then we leverage Bayesian profile regression to cluster patients into subgroups according to their baseline characteristics and predicted potential outcomes. We further embed a variable selection procedure into our proposed framework to identify the patient characteristics that actively "drive" the clustering structure. We conduct simulation studies to examine the performance of our proposed method and demonstrate the method by applying it to a UK-based multi-arm blood donation trial, wherein our method uncovers five clinically meaningful donor subgroups.

Personalized medicine has gained much popularity recently as a way of providing better healthcare by tailoring treatments to suit individuals. Our research, motivated by the UK INTERVAL blood donation trial, focuses on estimating the optimal individualized treatment rule (ITR) in the ordinal treatment-arms setting. Restrictions on minimum lengths between whole blood donations exist to safeguard donor health and quality of blood received. However, the evidence-base for these limits is lacking. Moreover, in England, the blood service is interested in making blood donation both safe and sustainable by integrating multi-marker data from INTERVAL and developing personalized donation strategies. As the three inter-donation interval options in INTERVAL have clear orderings, we propose a sequential re-estimation learning method that effectively incorporates "treatment" orderings when identifying optimal ITRs. Furthermore, we incorporate variable selection into our method for both linear and nonlinear decision rules to handle situations with (noise) covariates irrelevant for decision-making. Simulations demonstrate its superior performance over existing methods that assume multiple nominal treatments by achieving smaller misclassification rates and larger value functions. Application to a much-in-demand donor subgroup shows that the estimated optimal ITR achieves both the highest utilities and largest proportions of donors assigned to the safest inter-donation interval option in INTERVAL.

In this work, we introduce a personalised and age-specific Net Benefit function, composed of benefits and costs, to recommend optimal timing of risk assessments for cardiovascular disease prevention. We extend the 2-stage landmarking model to estimate patient-specific CVD risk profiles, adjusting for time-varying covariates. We apply our model to data from the Clinical Practice Research Datalink, comprising primary care electronic health records from the UK. We find that people at lower risk could be recommended an optimal risk-assessment interval of 5 years or more. Time-varying risk-factors are required to discriminate between more frequent schedules for higher-risk people.

Computational catalysis is playing an increasingly significant role in the design of catalysts across a wide range of applications. A common task for many computational methods is the need to accurately compute the adsorption energy for an adsorbate and a catalyst surface of interest. Traditionally, the identification of low energy adsorbate-surface configurations relies on heuristic methods and researcher intuition. As the desire to perform high-throughput screening increases, it becomes challenging to use heuristics and intuition alone. In this paper, we demonstrate machine learning potentials can be leveraged to identify low energy adsorbate-surface configurations more accurately and efficiently. Our algorithm provides a spectrum of trade-offs between accuracy and efficiency, with one balanced option finding the lowest energy configuration 87.36% of the time, while achieving a 2000x speedup in computation. To standardize benchmarking, we introduce the Open Catalyst Dense dataset containing nearly 1,000 diverse surfaces and 100,000 unique configurations., Comment: 26 pages, 7 figures. Submitted to npj Computational Materials

We present and validate 20 deg$^2$ of overlapping synthetic imaging surveys representing the full depth of the Nancy Grace Roman Space Telescope High-Latitude Imaging Survey (HLIS) and five years of observations of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST). The two synthetic surveys are summarized, with reference to the existing 300 deg$^2$ of LSST simulated imaging produced as part of Dark Energy Science Collaboration (DESC) Data Challenge 2 (DC2). Both synthetic surveys observe the same simulated DESC DC2 universe. For the synthetic Roman survey, we simulate for the first time fully chromatic images along with the detailed physics of the Sensor Chip Assemblies derived from lab measurements using the flight detectors. The simulated imaging and resulting pixel-level measurements of photometric properties of objects span a wavelength range of $\sim$0.3 to 2.0 $\mu$m. We also describe updates to the Roman simulation pipeline, changes in how astrophysical objects are simulated relative to the original DC2 simulations, and the resulting simulated Roman data products. We use these simulations to explore the relative fraction of unrecognized blends in LSST images, finding that 20-30% of objects identified in LSST images with $i$-band magnitudes brighter than 25 can be identified as multiple objects in Roman images. These simulations provide a unique testing ground for the development and validation of joint pixel-level analysis techniques of ground- and space-based imaging data sets in the second half of the 2020s -- in particular the case of joint Roman--LSST analyses.