Search

High-quality machine learning (ML)-ready datasets play a foundational role in developing new artificial intelligence (AI) models or fine-tuning existing models for scientific applications such as weather and climate analysis. Unfortunately, despite the growing development of new deep learning models for weather and climate, there is a scarcity of curated, pre-processed machine learning (ML)-ready datasets. Curating such high-quality datasets for developing new models is challenging particularly because the modality of the input data varies significantly for different downstream tasks addressing different atmospheric scales (spatial and temporal). Here we introduce WxC-Bench (Weather and Climate Bench), a multi-modal dataset designed to support the development of generalizable AI models for downstream use-cases in weather and climate research. WxC-Bench is designed as a dataset of datasets for developing ML-models for a complex weather and climate system, addressing selected downstream tasks as machine learning phenomenon. WxC-Bench encompasses several atmospheric processes from meso-$\beta$ (20 - 200 km) scale to synoptic scales (2500 km), such as aviation turbulence, hurricane intensity and track monitoring, weather analog search, gravity wave parameterization, and natural language report generation. We provide a comprehensive description of the dataset and also present a technical validation for baseline analysis. The dataset and code to prepare the ML-ready data have been made publicly available on Hugging Face -- https://huggingface.co/datasets/nasa-impact/WxC-Bench

This technical report presents Prithvi-EO-2.0, a new geospatial foundation model that offers significant improvements over its predecessor, Prithvi-EO-1.0. Trained on 4.2M global time series samples from NASA's Harmonized Landsat and Sentinel-2 data archive at 30m resolution, the new 300M and 600M parameter models incorporate temporal and location embeddings for enhanced performance across various geospatial tasks. Through extensive benchmarking with GEO-Bench, the 600M version outperforms the previous Prithvi-EO model by 8\% across a range of tasks. It also outperforms six other geospatial foundation models when benchmarked on remote sensing tasks from different domains and resolutions (i.e. from 0.1m to 15m). The results demonstrate the versatility of the model in both classical earth observation and high-resolution applications. Early involvement of end-users and subject matter experts (SMEs) are among the key factors that contributed to the project's success. In particular, SME involvement allowed for constant feedback on model and dataset design, as well as successful customization for diverse SME-led applications in disaster response, land use and crop mapping, and ecosystem dynamics monitoring. Prithvi-EO-2.0 is available on Hugging Face and IBM terratorch, with additional resources on GitHub. The project exemplifies the Trusted Open Science approach embraced by all involved organizations.

Modern distributed applications in healthcare, supply chain, and the Internet of Things handle a large amount of data in a diverse application setting with multiple stakeholders. Such applications leverage advanced artificial intelligence (AI) and machine learning algorithms to automate business processes. The proliferation of modern AI technologies increases the data demand. However, real-world networks often include private and sensitive information of businesses, users, and other organizations. Emerging data-protection regulations such as the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) introduce policies around collecting, storing, and managing digital data. While Blockchain technology offers transparency, auditability, and immutability for multi-stakeholder applications, it lacks inherent support for privacy. Typically, privacy support is added to a blockchain-based application by incorporating cryptographic schemes, consent mechanisms, and self-sovereign identity. This article surveys the literature on blockchain-based privacy-preserving systems and identifies the tools for protecting privacy. Besides, consent mechanisms and identity management in the context of blockchain-based systems are also analyzed. The article concludes by highlighting the list of open challenges and further research opportunities.

Calibration is crucial as large language models (LLMs) are increasingly deployed to convert natural language queries into SQL for commercial databases. In this work, we investigate calibration techniques for assigning confidence to generated SQL queries. We show that a straightforward baseline -- deriving confidence from the model's full-sequence probability -- outperforms recent methods that rely on follow-up prompts for self-checking and confidence verbalization. Our comprehensive evaluation, conducted across two widely-used Text-to-SQL benchmarks and multiple LLM architectures, provides valuable insights into the effectiveness of various calibration strategies.

Massive stars lead the ionization and mechanical feedback within young star-forming regions. The Large Magellanic Cloud (LMC) is an ideal galaxy for studying individual massive stars and quantifying their feedback contribution to the environment. We analyze eight exemplary targets in LMC N11 B from the Hubble UV Legacy Library of Young Stars as Essential Standards (ULLYSES) program, using novel spectra from HST (COS and STIS) in the UV, and from VLT (X-shooter) in the optical. We model the spectra of early to late O-type stars by using state-of-the-art PoWR atmosphere models. We determine the stellar and wind parameters (e.g., $T_\star$, $\log g$, $L_{\star}$, $\dot{M}$, $v_\infty$) of the analyzed objects, chemical abundances (C, N, O), ionizing and mechanical feedback ($Q_\mathrm{H}$, $Q_\mathrm{He{\small{I}}}$, $Q_\mathrm{He{\small{II}}}$, $L_\mathrm{mec}$) and X-rays. We report ages of $2-4.5$ Myr and masses of $30-60$ $M_\odot$ for the analyzed stars in N11 B, consistent with a scenario of sequential star formation. We note that the observed wind-momentum luminosity relation is consistent with theoretical predictions. We detect nitrogen enrichment in most of the stars, up to a factor of seven. However, we do not find a correlation between nitrogen enrichment and projected rotational velocity. Finally, based on their spectral type, we estimate the total ionizing photons injected from the O-type stars in N11 B into its environment. We report $\log$ ($\sum$ $Q_\mathrm{H}$)$=50.5$ ph s$^{-1}$, $\log$ ($\sum$ $Q_\mathrm{He{\small{I}}}$)$=49.6$ ph s$^{-1}$ and $\log$ ($\sum$ $Q_\mathrm{He{\small{II}}}$)$=44.4$ ph s$^{-1}$, consistent with the total ionizing budget in N11., Comment: Accepted in Astronomy and Astrophysics

A fundamental problem in decentralized networked systems is to coordinate actions of different agents so that they reach a state of agreement. In such applications, it is additionally desirable that the actions at various nodes may not be anticipated by malicious eavesdroppers. Motivated by this, we investigate the problem of secure multi-terminal strong coordination aided by a multiple-access wiretap channel. In this setup, independent and identically distributed copies of correlated sources are observed by two transmitters who encode the channel inputs to the MAC-WT. The legitimate receiver observing the channel output and side information correlated with the sources must produce approximately i.i.d. copies of an output variable jointly distributed with the sources. Furthermore, we demand that an external eavesdropper learns essentially nothin g about the sources and the simulated output sequence by observing its own MAC-WT output. This setting is aided by the presence of independent pairwise shared randomness between each encoder and the legitimate decoder, that is unavailable to the eavesdropper. We derive an achievable rate region based on a combination of coordination coding and wiretap coding, along with an outer bound. The inner bound is shown to be tight and a complete characterization is derived for the special case when the sources are conditionally independent given the decoder side information and the legitimate channel is composed of deterministic links. Further, we also analyze a more general scenario with possible encoder cooperation, where one of the encoders can non-causally crib from the other encoders input, for which an achievable rate region is proposed. We then explicitly compute the rate regions for an example both with and without cribbing between the encoders, and demonstrate that cribbing strictly improves upon the achievable rate region., Comment: Extended version of ISIT 2024 paper

We present results for the distance sensitivity oracle (DSO) problem, where one needs to preprocess a given directed weighted graph $G=(V,E)$ in order to answer queries about the shortest path distance from $s$ to $t$ in $G$ that avoids edge $e$, for any $s,t \in V, e \in E$. No non-trivial results are known for DSO in the distributed CONGEST model even though it is of importance to maintain efficient communication under an edge failure. Let $n=|V|$, and let $D$ be the undirected diameter of $G$. Our first DSO algorithm optimizes query response rounds and can answer a batch of any $k\geq 1$ queries in $O(k+D)$ rounds after taking $\tilde{O}(n^{3/2})$ rounds to preprocess $G$. Our second algorithm takes $\tilde{O}(n)$ rounds for preprocessing, and then it can answer any batch of $k\geq 1$ queries in $\tilde{O}(k\sqrt{n}+D)$ rounds. We complement these algorithms with some unconditional CONGEST lower bounds that give trade-offs between preprocessing rounds and rounds needed to answer queries. Additionally, we present almost-optimal upper and lower bounds for the related all pairs second simple shortest path (2-APSiSP) problem, where for all pairs of vertices $x,y \in V$, we need to compute the minimum weight of a simple $x$-$y$ path that differs from the precomputed $x$-$y$ shortest path by at least one edge.

The rapid development in large language models (LLMs) has transformed the landscape of natural language processing and understanding (NLP/NLU), offering significant benefits across various domains. However, when applied to scientific research, these powerful models exhibit critical failure modes related to scientific integrity and trustworthiness. Existing general-purpose LLM guardrails are insufficient to address these unique challenges in the scientific domain. We provide comprehensive guidelines for deploying LLM guardrails in the scientific domain. We identify specific challenges -- including time sensitivity, knowledge contextualization, conflict resolution, and intellectual property concerns -- and propose a guideline framework for the guardrails that can align with scientific needs. These guardrail dimensions include trustworthiness, ethics & bias, safety, and legal aspects. We also outline in detail the implementation strategies that employ white-box, black-box, and gray-box methodologies that can be enforced within scientific contexts.

Mass loss through a stellar wind is an important physical process that steers the evolution of massive stars and controls the properties of their end-of-life products, such as the supernova type and the mass of compact remnants. For an accurate mass loss determination, the inhomogeneities in the wind, known as clumping, needs to be taking into account. We aim to improve empirical estimates of mass loss and wind clumping for hot main-sequence massive stars, study the dependence of both properties on the metallicity, and compare the theoretical predictions to our findings. We analyzed the optical and UV spectra of 13 O-type stars in the Small Magellanic Cloud galaxy, which has a metallicity of $\sim 0.2\,Z_\odot$. We quantified the stellar atmosphere, outflow, and wind-clumping properties. To probe the role of metallicity, we compared our findings to studies of Galactic and Large Magellanic Cloud samples that were analyzed with similar methods. We find significant variations in the wind-clumping properties of the target stars, with clumping starting at flow velocities $0.01 - 0.23$ of the terminal wind velocity and reaching clumping factors $f_{\rm cl} = 2 - 30$. In the luminosity ($\log L / L_{\odot} = 5.0 - 6.0$) and metallicity ($Z/Z_{\odot} = 0.2 - 1$) range we considered, we find that the scaling of the mass loss $\dot{M}$ with metallicity $Z$ varies with luminosity. At $\log L/L_{\odot} = 5.75$, we find $\dot{M} \propto Z^m$ with $m = 1.02 \pm 0.30$, in agreement with pioneering work in the field. For lower luminosities, however, we obtain a significantly steeper scaling of $m > 2$. The monotonically decreasing $m(L)$ behavior adds a complexity to the functional description of the mass-loss rate of hot massive stars. Although the trend is present in the predictions, it is much weaker than we found here., Comment: 37 pages, 29 figures. Accepted for publication in Astronomy and Astrophysics

Quantization of foundational models (FMs) is significantly more challenging than traditional DNNs due to the emergence of large magnitude features called outliers. Existing outlier-aware algorithm/architecture co-design techniques either use mixed-precision, retaining outliers at high precision but compromise hardware efficiency, or quantize inliers and outliers at the same precision, improving hardware efficiency at the cost of accuracy. To address this mutual exclusivity, in this paper, we propose MicroScopiQ, a novel co-design technique that leverages pruning to complement outlier-aware quantization. MicroScopiQ retains outliers at higher precision while pruning a certain fraction of least important weights to distribute the additional outlier bits; ensuring high accuracy, aligned memory and hardware efficiency. We design a high-throughput, low overhead accelerator architecture composed of simple multi-precision INT processing elements and a novel network-on-chip called ReCoN that efficiently abstracts the complexity of supporting high-precision outliers. Additionally, unlike existing alternatives, MicroScopiQ does not assume any locality of outlier weights, enabling applicability to a broad range of FMs. Extensive experiments across various quantization settings show that MicroScopiQ achieves SoTA quantization performance while simultaneously improving inference performance by 3x and reducing energy by 2x over existing alternatives., Comment: Under review

Large language models (LLMs) such as ChatGPT are fine-tuned on large and diverse instruction-following corpora, and can generalize to new tasks. However, those instruction-tuned LLMs often perform poorly in specialized medical natural language understanding (NLU) tasks that require domain knowledge, granular text comprehension, and structured data extraction. To bridge the gap, we: (1) propose a unified prompting format for 7 important NLU tasks, % through span extraction and multi-choice question-answering (QA), (2) curate an instruction-tuning dataset, MNLU-Instruct, utilizing diverse existing open-source medical NLU corpora, and (3) develop BioMistral-NLU, a generalizable medical NLU model, through fine-tuning BioMistral on MNLU-Instruct. We evaluate BioMistral-NLU in a zero-shot setting, across 6 important NLU tasks, from two widely adopted medical NLU benchmarks: Biomedical Language Understanding Evaluation (BLUE) and Biomedical Language Understanding and Reasoning Benchmark (BLURB). Our experiments show that our BioMistral-NLU outperforms the original BioMistral, as well as the proprietary LLMs - ChatGPT and GPT-4. Our dataset-agnostic prompting strategy and instruction tuning step over diverse NLU tasks enhance LLMs' generalizability across diverse medical NLU tasks. Our ablation experiments show that instruction-tuning on a wider variety of tasks, even when the total number of training instances remains constant, enhances downstream zero-shot generalization., Comment: 3 figures an 5 tables

Each request in LLM inference goes through two phases: compute-bound prefill and memory-bandwidth-bound decode. To improve GPU utilization, recent systems use hybrid batching that combines the prefill and decode phases of different requests into the same batch. Hybrid batching works well for linear operations as it amortizes the cost of loading model weights from HBM. However, attention computation in hybrid batches remains inefficient because existing attention kernels are optimized for either prefill or decode. In this paper, we present POD-Attention -- the first GPU kernel that efficiently computes attention for hybrid batches. POD-Attention aims to maximize the utilization of both compute and memory bandwidth by carefully allocating the GPU's resources such that prefill and decode operations happen concurrently on the same multiprocessor. We integrate POD-Attention in a state-of-the-art LLM inference scheduler Sarathi-Serve. POD-Attention speeds up attention computation by up to 75% (mean 28%) and increases LLM serving throughput by up to 22% in offline inference. In online inference, POD-Attention enables lower time-to-first-token (TTFT), time-between-tokens (TBT), and request execution latency versus Sarathi-Serve.

Field-induced assembly of colloidal particles into structures of desired configurations is extremely relevant from the viewpoint of producing field-assembled micro-swimmers and reconfigurable smart materials. However, the behaviour of colloidal particles under the influence of alternating current (AC) electric fields remains a topic of ongoing investigation due to the complex effects of various control parameters. Here we examine the role of several factors including particle size, zeta potential, voltage and frequency of the applied field in the formation of different structural configurations ranging from crystals to glasses, and observe interesting and unexpected behaviours in the structure formation. Additionally, we investigate the dynamics of structure formation; the nature of diffusion and active motion in these out-of-equilibrium systems, and show how that leads to the formation of close-packed or open structures. Lastly, we investigate the frequency-driven disorder-order-disorder phase transition in colloidal crystals, which is a starting point for building reconfigurable systems. Our findings contribute to a deeper understanding of the significant roles of various factors in electric field-induced assembly of colloidal particles, as well as pave the way for potential applications in micro-robotics and next-generation materials., Comment: 10 pages, 7 figures

In the current digital landscape, supply chains have transformed into complex networks driven by the Internet of Things (IoT), necessitating enhanced data sharing and processing capabilities to ensure traceability and transparency. Leveraging Blockchain technology in IoT applications advances reliability and transparency in near-real-time insight extraction processes. However, it raises significant concerns regarding data privacy. Existing privacy-preserving approaches often rely on Smart Contracts for automation and Zero Knowledge Proofs (ZKP) for privacy. However, apart from being inflexible in adopting system changes while effectively protecting data confidentiality, these approaches introduce significant computational expenses and overheads that make them impractical for dynamic supply chain environments. To address these challenges, we propose ZK-DPPS, a framework that ensures zero-knowledge communications without the need for traditional ZKPs. In ZK-DPPS, privacy is preserved through a combination of Fully Homomorphic Encryption (FHE) for computations and Secure Multi-Party Computations (SMPC) for key reconstruction. To ensure that the raw data remains private throughout the entire process, we use FHE to execute computations directly on encrypted data. The "zero-knowledge" aspect of ZK-DPPS refers to the system's ability to process and share data insights without exposing sensitive information, thus offering a practical and efficient alternative to ZKP-based methods. We demonstrate the efficacy of ZK-DPPS through a simulated supply chain scenario, showcasing its ability to tackle the dual challenges of privacy preservation and computational trust in decentralised environments.

Let $X$ be a smooth projective variety defined over a number field $k$ that has Brauer-Manin obstruction to the existence of rational points. We study the orders of the Brauer classes which capture the obstruction. We prove that one may require elements of arbitrarily large order to capture the obstruction. In other words, the exponent of the group which captures the obstruction has no upper bound., Comment: 8 pages. Comments are welcome

The precision measurement of the tritium $\beta$-decay spectrum performed by the KATRIN experiment provides a unique way to search for general neutrino interactions (GNI). All theoretical allowed GNI terms involving neutrinos are incorporated into a low-energy effective field theory, and can be identified by specific signatures in the measured tritium $\beta$-spectrum. In this paper an effective description of the impact of GNI on the $\beta$-spectrum is formulated and the first constraints on the effective GNI parameters are derived based on the 4 million electrons collected in the second measurement campaign of KATRIN in 2019. In addition, constraints on selected types of interactions are investigated, thereby exploring the potential of KATRIN to search for more specific new physics cases, including a right-handed W boson, a charged Higgs or leptoquarks.

A wide family of two dimensional (2D) systems, including stripe-phase superconductors, sliding Luttinger liquids, and anisotropic 2D materials, can be modeled by an array of coupled one-dimensional (1D) electron channels or nanowire arrays. Here we report experiments in arrays of conducting nanowires with gate and field tunable interwire coupling, that are programmed at the LaAlO$_3$/SrTiO$_3$ interface. We find a magnetically-tuned metal-to-insulator transition in which the transverse resistance of the nanowire array increases by up to four orders of magnitude. To explain this behavior, we develop a minimal model of a coupled two-wire system which agrees well with observed phenomena. These nanowire arrays can serve as a model systems to understand the origin of exotic behavior in correlated materials via analog quantum simulation.

Coupled quasi-one-dimensional (quasi-1D) electron systems host rich emergent physics that cannot be accounted for by understanding isolated 1D electron systems alone. Open questions remain about how transport in these arrays can be manipulated by the application of external electric and magnetic fields. In this theoretical study, we consider a pair of coupled nanowires with non-interacting electrons. We find that a metal-insulator transition is induced by an out-of-plane magnetic field and a transverse potential bias on an array of such coupled wires. We demonstrate the existence of distinct conductance features and highlight the crucial role played by the field dependence of the interwire potential barrier on transport properties. These predictions agree well with transport experiments performed on coupled nanowires sketched on an LaAlO3/SrTiO3 interface. Since our model makes minimal assumptions, we expect our predictions to hold for a wide class of coupled 1D systems.

The Fermi polaron problem, which describes a mobile impurity that interacts with a spin-polarized Fermi sea, is a paradigmatic system in quantum many-body physics and has been challenging to address quantitatively in its strong coupling regime. We present the first controlled thermodynamic calculations for the Fermi polaron at strong coupling using finite-temperature auxiliary-field quantum Monte Carlo (AFMC) methods in the framework of the canonical ensemble. Modeled as a spin-imbalanced system, the Fermi polaron has a Monte Carlo sign problem, but we show that it is moderate over a wide range of temperatures and coupling strengths beyond the unitary limit of the BCS-BEC crossover. We calculate the contact, a quantity which measures the strength of the short-range correlations, as a function of temperature at unitarity and as a function of the coupling strength at fixed temperature and find good agreement with a variational approach based on one particle-hole excitation of the Fermi sea. We compare our results for the contact with recent experiments and find good agreement at unitarity (within error bars) but discrepancies away from unitarity on the BEC side of the crossover. We also calculate the thermal energy gap at unitarity as a function of temperature., Comment: 8 pages, 7 figures

Deep learning-based methods have been widely researched in the areas of language and vision, demonstrating their capacity to understand long sequences of data and their usefulness in numerous helio-physics applications. Foundation models (FMs), which are pre-trained on a large-scale datasets, form the basis for a variety of downstream tasks. These models, especially those based on transformers in vision and language, show exceptional potential for adapting to a wide range of downstream applications. In this paper, we provide our perspective on the criteria for designing an FM for heliophysics and associated challenges and applications using the Solar Dynamics Observatory (SDO) dataset. We believe that this is the first study to design an FM in the domain of heliophysics., Comment: 31 Pages, 12 figures

`Extreme Classification'' (or XC) is the task of annotating data points (queries) with relevant labels (documents), from an extremely large set of $L$ possible labels, arising in search and recommendations. The most successful deep learning paradigm that has emerged over the last decade or so for XC is to embed the queries (and labels) using a deep encoder (e.g. DistilBERT), and use linear classifiers on top of the query embeddings. This architecture is of appeal because it enables millisecond-time inference using approximate nearest neighbor search (ANNS). The key question is how do we design training algorithms that are accurate as well as scale to $O(100M)$ labels on a limited number of GPUs. State-of-the-art XC techniques that demonstrate high accuracies (e.g., DEXML, Ren\'ee, DEXA) on standard datasets have per-epoch training time that scales as $O(L)$ or employ expensive negative sampling strategies, which are prohibitive in XC scenarios. In this work, we develop an accurate and scalable XC algorithm ASTRA with two key observations: (a) building ANNS index on the classifier vectors and retrieving hard negatives using the classifiers aligns the negative sampling strategy to the loss function optimized; (b) keeping the ANNS indices current as the classifiers change through the epochs is prohibitively expensive while using stale negatives (refreshed periodically) results in poor accuracy; to remedy this, we propose a negative sampling strategy that uses a mixture of importance sampling and uniform sampling. By extensive evaluation on standard XC as well as proprietary datasets with 120M labels, we demonstrate that ASTRA achieves SOTA precision, while reducing training time by 4x-15x relative to the second best.

As large language models (LLMs) evolve to handle increasingly longer contexts, serving inference requests for context lengths in the range of millions of tokens presents unique challenges. While existing techniques are effective for training, they fail to address the unique challenges of inference, such as varying prefill and decode phases and their associated latency constraints - like Time to First Token (TTFT) and Time Between Tokens (TBT). Furthermore, there are no long context inference solutions that allow batching requests to increase the hardware utilization today. In this paper, we propose three key innovations for efficient interactive long context LLM inference, without resorting to any approximation: adaptive chunking to reduce prefill overheads in mixed batching, Sequence Pipeline Parallelism (SPP) to lower TTFT, and KV Cache Parallelism (KVP) to minimize TBT. These contributions are combined into a 3D parallelism strategy, enabling Mnemosyne to scale interactive inference to context lengths at least up to 10 million tokens with high throughput enabled with batching. To our knowledge, Mnemosyne is the first to be able to achieve support for 10 million long context inference efficiently, while satisfying production-grade SLOs on TBT (30ms) on contexts up to and including 10 million.

Lattice gauge theories (LGTs) can be employed to understand a wide range of phenomena, from elementary particle scattering in high-energy physics to effective descriptions of many-body interactions in materials. Studying dynamical properties of emergent phases can be challenging as it requires solving many-body problems that are generally beyond perturbative limits. We investigate the dynamics of local excitations in a $\mathbb{Z}_2$ LGT using a two-dimensional lattice of superconducting qubits. We first construct a simple variational circuit which prepares low-energy states that have a large overlap with the ground state; then we create particles with local gates and simulate their quantum dynamics via a discretized time evolution. As the effective magnetic field is increased, our measurements show signatures of transitioning from deconfined to confined dynamics. For confined excitations, the magnetic field induces a tension in the string connecting them. Our method allows us to experimentally image string dynamics in a (2+1)D LGT from which we uncover two distinct regimes inside the confining phase: for weak confinement the string fluctuates strongly in the transverse direction, while for strong confinement transverse fluctuations are effectively frozen. In addition, we demonstrate a resonance condition at which dynamical string breaking is facilitated. Our LGT implementation on a quantum processor presents a novel set of techniques for investigating emergent particle and string dynamics.

Artificial Intelligence (AI) has demonstrated significant potential in automating various medical imaging tasks, which could soon become routine in clinical practice for disease diagnosis, prognosis, treatment planning, and post-treatment surveillance. However, the privacy concerns surrounding patient data present a major barrier to the widespread adoption of AI in medical imaging, as large, diverse training datasets are essential for developing accurate, generalizable, and robust Artificial intelligence models. Federated Learning (FL) offers a solution that enables organizations to train AI models collaboratively without sharing sensitive data. federated learning exchanges model training information, such as gradients, between the participating sites. Despite its promise, federated learning is still in its developmental stages and faces several challenges. Notably, sensitive information can still be inferred from the gradients shared during model training. Quantifying AI models' uncertainty is vital due to potential data distribution shifts post-deployment, which can affect model performance. Uncertainty quantification (UQ) in FL is particularly challenging due to data heterogeneity across participating sites. This review provides a comprehensive examination of FL, privacy-preserving FL (PPFL), and UQ in FL. We identify key gaps in current FL methodologies and propose future research directions to enhance data privacy and trustworthiness in medical imaging applications., Comment: 21 pages, 5 figures, 4 tables, Review paper, preprint to Radiology AI. arXiv admin note: text overlap with arXiv:2406.12815

Triggered by the realization that AI emulators can rival the performance of traditional numerical weather prediction models running on HPC systems, there is now an increasing number of large AI models that address use cases such as forecasting, downscaling, or nowcasting. While the parallel developments in the AI literature focus on foundation models -- models that can be effectively tuned to address multiple, different use cases -- the developments on the weather and climate side largely focus on single-use cases with particular emphasis on mid-range forecasting. We close this gap by introducing Prithvi WxC, a 2.3 billion parameter foundation model developed using 160 variables from the Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2). Prithvi WxC employs an encoder-decoder-based architecture, incorporating concepts from various recent transformer models to effectively capture both regional and global dependencies in the input data. The model has been designed to accommodate large token counts to model weather phenomena in different topologies at fine resolutions. Furthermore, it is trained with a mixed objective that combines the paradigms of masked reconstruction with forecasting. We test the model on a set of challenging downstream tasks namely: Autoregressive rollout forecasting, Downscaling, Gravity wave flux parameterization, and Extreme events estimation. The pretrained model with 2.3 billion parameters, along with the associated fine-tuning workflows, has been publicly released as an open-source contribution via Hugging Face.

Ultra-luminous X-ray sources (ULXs) are high-mass X-ray binaries with an X-ray luminosity above $10^{39}$ erg s$^{-1}$. These ULXs can be powered by black holes that are more massive than $20M_\odot$, accreting in a standard regime, or lighter compact objects accreting supercritically. There are only a few ULXs with known optical or UV counterparts, and their nature is debated. Determining whether optical/UV radiation is produced by the donor star or by the accretion disc is crucial for understanding ULX physics and testing massive binary evolution. We conduct, for the first time, a fully consistent multi-wavelength spectral analysis of a ULX and its circumstellar nebula. We aim to establish the donor star type and test the presence of strong disc winds in the prototypical ULX Holmberg II X-1 (Ho II X-1). We intent to obtain a realistic spectral energy distribution of the ionising source, which is needed for robust nebula analysis. We acquired new UV spectra of Ho II X-1 with the HST and complemented them with archival optical and X-ray data. We explored the spectral energy distribution of the source and analysed the spectra using the stellar atmosphere code PoWR and the photoionisation code Cloudy. Our analysis of the X-ray, UV, and optical spectra of Ho II X-1 and its nebula consistently explains the observations. We do not find traces of disc wind signatures in the UV and the optical, rejecting previous claims of the ULX being a supercritical accretor. The optical/UV counterpart of HoII X-1 is explained by a B-type supergiant donor star. Thus, the observations are fully compatible with Ho II X-1 being a close binary consisting of an $\gtrsim 66\,M_\odot$ black hole accreting matter from an $\simeq 22 M_\odot$ B-supergiant companion. Also, we propose a possible evolution scenario for the system, suggesting that Ho II X-1 is a potential gravitational wave source progenitor., Comment: Accepted in A&A. 14 pages (12 main body + 2 appendix), 6 figures, 6 tables

Evaluating models and datasets in computer vision remains a challenging task, with most leaderboards relying solely on accuracy. While accuracy is a popular metric for model evaluation, it provides only a coarse assessment by considering a single model's score on all dataset items. This paper explores Item Response Theory (IRT), a framework that infers interpretable latent parameters for an ensemble of models and each dataset item, enabling richer evaluation and analysis beyond the single accuracy number. Leveraging IRT, we assess model calibration, select informative data subsets, and demonstrate the usefulness of its latent parameters for analyzing and comparing models and datasets in computer vision.

Clinical notes contain unstructured representations of patient histories, including the relationships between medical problems and prescription drugs. To investigate the relationship between cancer drugs and their associated symptom burden, we extract structured, semantic representations of medical problem and drug information from the clinical narratives of oncology notes. We present Clinical Concept Annotations for Cancer Events and Relations (CACER), a novel corpus with fine-grained annotations for over 48,000 medical problems and drug events and 10,000 drug-problem and problem-problem relations. Leveraging CACER, we develop and evaluate transformer-based information extraction (IE) models such as BERT, Flan-T5, Llama3, and GPT-4 using fine-tuning and in-context learning (ICL). In event extraction, the fine-tuned BERT and Llama3 models achieved the highest performance at 88.2-88.0 F1, which is comparable to the inter-annotator agreement (IAA) of 88.4 F1. In relation extraction, the fine-tuned BERT, Flan-T5, and Llama3 achieved the highest performance at 61.8-65.3 F1. GPT-4 with ICL achieved the worst performance across both tasks. The fine-tuned models significantly outperformed GPT-4 in ICL, highlighting the importance of annotated training data and model optimization. Furthermore, the BERT models performed similarly to Llama3. For our task, LLMs offer no performance advantage over the smaller BERT models. The results emphasize the need for annotated training data to optimize models. Multiple fine-tuned transformer models achieved performance comparable to IAA for several extraction tasks., Comment: This is a pre-copy-editing, author-produced PDF of an article accepted for publication in JAMIA following peer review. The definitive publisher-authenticated version is available online at https://academic.oup.com/jamia/advance-article/doi/10.1093/jamia/ocae231/7748302

Ethanolamine (NH2CH2CH2OH) has recently been identified in the molecular cloud G+0.693-0.027, situated in the SgrB2 complex in the Galactic center. However, its presence in other regions, and in particular in star-forming sites, is still elusive. Given its likely role as a precursor to simple amino acids, understanding its presence in the star-forming region is required. Here, we present the experimentally obtained temperature-dependent spectral features and morphological behavior of pure ethanolamine ices under astrochemical conditions in the 2 - 12 micro meter (MIR) and 120 - 230 nm (VUV) regions for the first time. These features would help in understanding its photochemical behavior. In addition, we present the first chemical models specifically dedicated to ethanolamine. These models include all the discussed chemical routes from the literature, along with the estimated binding energies and activation energies from quantum chemical calculations reported in this work. We have found that surface reactions: CH2OH + NH2CH2 --> NH2CH2CH2OH and NH2 + C2H4OH --> NH2CH2CH2OH in warmer regions (60-90 K) could play a significant role in the formation of ethanolamine. Our modeled abundance of ethanolamine complements the upper limit of ethanolamine column density estimated in earlier observations in hot core/corino regions. Furthermore, we provide a theoretical estimation of the rotational and distortional constants for various species (such as HNCCO, NH2CHCO, and NH2CH2CO) related to ethanolamine that have not been studied in existing literature. This study could be valuable for identifying these species in the future., Comment: 18 pages, 6 figures. Accepted for the publication in The Astrophysical Journal

Quantum error correction provides a path to reach practical quantum computing by combining multiple physical qubits into a logical qubit, where the logical error rate is suppressed exponentially as more qubits are added. However, this exponential suppression only occurs if the physical error rate is below a critical threshold. In this work, we present two surface code memories operating below this threshold: a distance-7 code and a distance-5 code integrated with a real-time decoder. The logical error rate of our larger quantum memory is suppressed by a factor of $\Lambda$ = 2.14 $\pm$ 0.02 when increasing the code distance by two, culminating in a 101-qubit distance-7 code with 0.143% $\pm$ 0.003% error per cycle of error correction. This logical memory is also beyond break-even, exceeding its best physical qubit's lifetime by a factor of 2.4 $\pm$ 0.3. We maintain below-threshold performance when decoding in real time, achieving an average decoder latency of 63 $\mu$s at distance-5 up to a million cycles, with a cycle time of 1.1 $\mu$s. To probe the limits of our error-correction performance, we run repetition codes up to distance-29 and find that logical performance is limited by rare correlated error events occurring approximately once every hour, or 3 $\times$ 10$^9$ cycles. Our results present device performance that, if scaled, could realize the operational requirements of large scale fault-tolerant quantum algorithms., Comment: 10 pages, 4 figures, Supplementary Information

Current approaches to automated code generation often rely on monolithic models that lack real-time adaptability and scalability. This limitation is particularly evident in complex programming tasks that require dynamic adjustment and efficiency. The integration of neuroscience principles into Natural Language Processing (NLP) has the potential to revolutionize automated code generation. This paper presents CortexCompile, a novel modular system inspired by the specialized functions of the human brain's cortical regions. By emulating the distinct roles of the Prefrontal Cortex, Parietal Cortex, Temporal Lobe, and Motor Cortex, CortexCompile achieves significant advancements in scalability, efficiency, and adaptability compared to traditional monolithic models like GPT-4o. The system's architecture features a Task Orchestration Agent that manages dynamic task delegation and parallel processing, facilitating the generation of highly accurate and optimized code across increasingly complex programming tasks. Experimental evaluations demonstrate that CortexCompile consistently outperforms GPT-4o in development time, accuracy, and user satisfaction, particularly in tasks involving real-time strategy games and first-person shooters. These findings underscore the viability of neuroscience-inspired architectures in addressing the limitations of current NLP models, paving the way for more efficient and human-like AI systems., Comment: 17 pages, 6 figures

We introduce Imagen 3, a latent diffusion model that generates high quality images from text prompts. We describe our quality and responsibility evaluations. Imagen 3 is preferred over other state-of-the-art (SOTA) models at the time of evaluation. In addition, we discuss issues around safety and representation, as well as methods we used to minimize the potential harm of our models.

Behavior cloning is a common imitation learning paradigm. Under behavior cloning the robot collects expert demonstrations, and then trains a policy to match the actions taken by the expert. This works well when the robot learner visits states where the expert has already demonstrated the correct action; but inevitably the robot will also encounter new states outside of its training dataset. If the robot learner takes the wrong action at these new states it could move farther from the training data, which in turn leads to increasingly incorrect actions and compounding errors. Existing works try to address this fundamental challenge by augmenting or enhancing the training data. By contrast, in our paper we develop the control theoretic properties of behavior cloned policies. Specifically, we consider the error dynamics between the system's current state and the states in the expert dataset. From the error dynamics we derive model-based and model-free conditions for stability: under these conditions the robot shapes its policy so that its current behavior converges towards example behaviors in the expert dataset. In practice, this results in Stable-BC, an easy to implement extension of standard behavior cloning that is provably robust to covariate shift. We demonstrate the effectiveness of our algorithm in simulations with interactive, nonlinear, and visual environments. We also conduct experiments where a robot arm uses Stable-BC to play air hockey. See our website here: https://collab.me.vt.edu/Stable-BC/

The projected sensitivity of the effective electron neutrino-mass measurement with the KATRIN experiment is below 0.3 eV (90 % CL) after five years of data acquisition. The sensitivity is affected by the increased rate of the background electrons from KATRIN's main spectrometer. A special shifted-analysing-plane (SAP) configuration was developed to reduce this background by a factor of two. The complex layout of electromagnetic fields in the SAP configuration requires a robust method of estimating these fields. We present in this paper a dedicated calibration measurement of the fields using conversion electrons of gaseous $^\mathrm{83m}$Kr, which enables the neutrino-mass measurements in the SAP configuration., Comment: 19 pages, 11 figures

Metal-poor massive stars drive the evolution of low-mass galaxies, both locally and at high redshift. However, quantifying the feedback they impart to their local surroundings remains uncertain because models of stellar evolution, mass loss, and ionizing spectra are unconstrained by observations below 20% solar metallicity ($Z_\odot$). We present new Keck Cosmic Web Imager optical spectroscopy of three O stars in the nearby dwarf galaxies Leo P, Sextans A, and WLM, which have gas-phase oxygen abundances of 3-14% $Z_\odot$. To characterize their fundamental stellar properties and radiation-driven winds, we fit PoWR atmosphere models to the optical spectra simultaneously with Hubble Space Telescope far-ultraviolet (FUV) spectra and multi-wavelength photometry. We find that all three stars have effective temperatures consistent with their spectral types and surface gravities typical of main-sequence dwarf stars. Yet, the combination of those inferred parameters and luminosity for the two lower-$Z$ stars is not reproduced by stellar evolution models, even those that include rotation or binary interactions. The scenario of multiple-star systems is difficult to reconcile with all available data, suggesting that these observations pose a challenge to current evolution models. We highlight the importance of validating the relationship between stellar mass, temperature, and luminosity at very low $Z$ for accurate estimates of ionizing photon production and spectral hardness. Finally, all three stars' FUV wind profiles reveal low mass-loss rates and terminal wind velocities in tension with expectations from widely adopted radiation-driven wind models. These results provide empirical benchmarks for future development of mass-loss and evolution models for metal-poor stellar populations., Comment: Accepted at ApJ. 10 figures, 25 pages

The smoothed particle hydrodynamics (SPH) method has been widely used to simulate incompressible and slightly compressible fluid flows. Adaptive refinement strategies to dynamically increase the resolution of the particles to capture sharp gradients in the flow have also been developed. However, most of the SPH schemes in the literature are not second-order convergent (SOC). Both second-order convergence and adaptive resolution are considered grand challenge problems in the SPH community. In this paper, we propose, for the first time, a second-order convergent (SOC) adaptive refinement strategy along with a SOC weakly-compressible SPH scheme. We employ the method of manufactured solutions to systematically develop the scheme and validate the solver. We demonstrate the order of convergence of the entire scheme using the Taylor-Green vortex problem and then go on to demonstrate the applicability of the method to simulate flow past a circular cylinder.

Surveys in the Milky Way and Large Magellanic Cloud revealed that the majority of massive stars will interact with companions during their lives. However, knowledge of the binary properties of massive stars at low metallicity, which approaches the conditions of the Early Universe, remains sparse. We present the Binarity at LOw Metallicity (BLOeM) campaign - an ESO large programme designed to obtain 25 epochs of spectroscopy for 929 massive stars in the SMC - the lowest metallicity conditions in which multiplicity is probed to date (Z = 0.2 Zsun). BLOeM will provide (i) the binary fraction, (ii) the orbital configurations of systems with periods P < 3 yr, (iii) dormant OB+BH binaries, and (iv) a legacy database of physical parameters of massive stars at low metallicity. The stars are observed with the LR02 setup of the giraffe instrument of the Very Large Telescope (3960-4570A, resolving power R=6200; typical signal-to-noise ratio S/N=70-100). This paper utilises the first 9 epochs obtained over a three-month time. We describe the survey and data reduction, perform a spectral classification of the stacked spectra, and construct a Hertzsprung-Russell diagram of the sample via spectral-type and photometric calibrations. The sample covers spectral types from O4 to F5, spanning the effective temperature and luminosity ranges 6.5

Context. B supergiants (BSGs) represent an important connection between the main sequence and more extreme evolutionary stages of massive stars. Additionally, lying toward the cool end of the hot star regime, determining their wind properties is crucial to constrain the evolution and feedback of massive stars as, for instance, they might manifest the bi-stability jump phenomenon. Aims. We undertake a detailed analysis of a representative sample of 18 Small Magellanic Cloud (SMC) BSGs within the ULLYSES and XShootU datasets. Our UV and optical analysis spans BSGs from B0 to B8 - covering the bi-stability jump region. We aim to evaluate their evolutionary status and verify what their wind properties say about the bi-stability jump in a low-metallicity environment. Methods. We used the CMFGEN to model the spectra and photometry (from UV to infrared) of our sample. We compare our results with different evolutionary models, with previous determinations in the literature of OB stars, and with diverging mass-loss recipes at the bi-stability jump. Additionally, we provide the first BSG models in the SMC including X-rays. Results. (i) Within a single-stellar evolution framework, the evolutionary status of early BSGs seem less clear than that of late BSGs, which agree with H-shell burning models. (ii) UV analysis shows evidence that BSGs contain X-rays in their atmospheres, for which we provide constraints. In general, we find higher X-ray luminosity (close to the standard log(L_X/L) ~ -7) for early BSGs. For cooler BSGs, lower values are preferred, log(L_X/L) ~ -8.5. (iii) The obtained mass-loss rates suggest neither a jump nor a monotonic decrease with temperature. Instead, a rather constant trend is observed, which is at odds with the increase found for Galactic BSGs. (iv) The wind velocity behavior with temperature shows a sharp drop at ~19 kK, similar to what is observed for Galactic BSGs., Comment: 33 pages (23+10)+(22 at Zenodo), 34 figures (21+13)+(21 at Zenodo), 7 tables (3+4)+(1 at Zenodo), accepted for publication

The ability to create superlattices in van der Waals (vdW) heterostructures via moir\'e interference heralded a new era in the science and technology of two-dimensional materials. Through precise control of the twist angle, flat bands and strongly correlated phases have been engineered. The precise twisting of vdW layers is in some sense a bottom-up approach--a single parameter can dial in a wide range of periodic structures. Here, we describe a top-down approach to engineering nanoscale potentials in vdW layers using a buried programmable ferroelectric layer. Ultra-low-voltage electron beam lithography (ULV-EBL) is used to program ferroelectric domains in a ferroelectric Al_{1-x}B_{x}N thin film through a graphene/hexagonal boron nitride (hBN) heterostructure that is transferred on top. We demonstrate ferroelectric field effects by creating a lateral p-n junction, and demonstrate spatial resolution down to 35 nm, limited by the resolution of our scanned probe characterization methods. This innovative, resist-free patterning method is predicted to achieve 10 nm resolution and enable arbitrary programming of vdW layers, opening a pathway to create new phases that are inaccessible by moir\'e techniques. The ability to "paint" different phases of matter on a single vdW "canvas" provides a wealth of new electronic and photonic functionalities., Comment: 9 pages, 4 figures and supplemental material

Published research highlights the presence of demographic bias in automated facial attribute classification. The proposed bias mitigation techniques are mostly based on supervised learning, which requires a large amount of labeled training data for generalizability and scalability. However, labeled data is limited, requires laborious annotation, poses privacy risks, and can perpetuate human bias. In contrast, self-supervised learning (SSL) capitalizes on freely available unlabeled data, rendering trained models more scalable and generalizable. However, these label-free SSL models may also introduce biases by sampling false negative pairs, especially at low-data regimes 200K images) under low compute settings. Further, SSL-based models may suffer from performance degradation due to a lack of quality assurance of the unlabeled data sourced from the web. This paper proposes a fully self-supervised pipeline for demographically fair facial attribute classifiers. Leveraging completely unlabeled data pseudolabeled via pre-trained encoders, diverse data curation techniques, and meta-learning-based weighted contrastive learning, our method significantly outperforms existing SSL approaches proposed for downstream image classification tasks. Extensive evaluations on the FairFace and CelebA datasets demonstrate the efficacy of our pipeline in obtaining fair performance over existing baselines. Thus, setting a new benchmark for SSL in the fairness of facial attribute classification., Comment: 13 pages, IJCB 2024

When Large Language Models (LLMs) are compressed using techniques such as quantization, the predominant way to demonstrate the validity of such techniques is by measuring the model's accuracy on various benchmarks.If the accuracies of the baseline model and the compressed model are close, it is assumed that there was negligible degradation in quality.However, even when the accuracy of baseline and compressed model are similar, we observe the phenomenon of flips, wherein answers change from correct to incorrect and vice versa in proportion.We conduct a detailed study of metrics across multiple compression techniques, models and datasets, demonstrating that the behavior of compressed models as visible to end-users is often significantly different from the baseline model, even when accuracy is similar.We further evaluate compressed models qualitatively and quantitatively using MT-Bench and show that compressed models are significantly worse than baseline models in this free-form generative task.Thus, we argue that compression techniques should also be evaluated using distance metrics.We propose two such metrics, KL-Divergence and flips, and show that they are well correlated.

Buckling instabilities driven by tissue growth underpin key developmental events such as the folding of the brain. Tissue growth is disordered due to cell-to-cell variability, but the effects of this variability on buckling are unknown. Here, we analyse what is perhaps the simplest setup of this problem: the buckling of an elastic rod with fixed ends driven by spatially varying growth. Combining analytical calculations for simple growth fields and numerical sampling of random growth fields, we show that variability can increase as well as decrease the growth threshold for buckling, even when growth variability does not cause any residual stresses. For random growth, we find that the shift of the buckling threshold correlates with spatial moments of the growth field. Our results imply that biological systems can either trigger or avoid buckling by exploiting the spatial arrangement of growth variability., Comment: 6 pages, 3 figures; Supplemental Material: 7 pages, 1 figure